Interesting and thorny topic, that! "usually" is the keyword.
It appears to me that "for" is the only really common alternative to "of" in these constructions, with an occasional "to" or "with" or some other preposition. And it always depends on the verb, of course. Some take only "of"; some...