Well, if you are talking about the differences and similarities of people or things, both
with and
to are used. However, if you want to say that something ar a person is like another thing or person, the correct usage is
to:
I can only compare the experience to falling in love. (Cobuild). And there are a couple of contexts in which only
with is used (extracted from the cobuild again):
3 If one thing compares favourably with another, it is better than the other thing. If it compares unfavourably, it is worse than the other thing.
Our road safety record compares favourably with that of other European countries...
How do the two techniques compare in terms of application?
V-RECIP: V adv with n, pl-n V adv
4 If you say that something does not compare with something else, you mean that it is much worse.
The flowers here do not compare with those at home...
But back to your sentence, I think if the speaker means that the listener doen't have to compare his/her talents, skills, characteristics,... to/with anybody else to see how different or how similar they are, both are correct. But if it is used to say that the listener is worse than anybody else, with should be used (the context and the tone would determine if this is the case.)
Hope that it helps.