It's not that time is fixed, it's that the always-moving-forward, can't-go-back and can't-stand-still ASPECT of time is a "given" in our environment.
But this is one of the cases when over-analyzing isn't really helping, I think. We all understand the meaning of the sentence, even if we aren't sure what sense of the word he meant.