the key of or to the door?

to is preferable, in the sense of which goes with, which belongs with (as a pair, usually).

the key to the door
the door to the garage
the entrance to the building
the combination to the safe
the attachments to this vacuum cleaner
the answer to the question
my response to her letter

Note that these expressions with to are often in the context of finding a smaller thing that goes with a bigger thing.  The smaller thing takes on the role of something missing (in a vague and broad sense) that must be supplied to complete the pair.

But be careful - these can be highly idiosyncratic, and for is often used the same way, and substituting one for the other will sound peculiar to a native speaker.

the remote for this TV
the cover for the pot
a belt for these pants

Also, many of these are alternately expressed as compound nouns.

the garage door, the door key, the pot cover

All in all, this topic can become quite a muddle.
There's no end to the treats! 

CJ

There are subtle differences.  First of all, I personally would never say the cover of the pot.  I would say the pot cover.

I might look for a cover for the pot.  I merely want something that will serve as a cover for that pot.  It doesn't have to be that very cover that the manufacturer provided with the pot when I bought the pot (and its cover).

Or I might look for the cover to the pot.  I want the specific cover that goes with (was manufactured for) that pot.

Not everybody will interpret the difference as I have.  You may wish to wait for other opinions.

CJ

Onh1986 wrote:

So "keys to the car" is correct NOT "car keys", right?

I use "car keys" all the time.  Actually, I use both versions regularly.

car's key

OR

car key
OR

key of the car

OR

key to the car?