It's hard to think of contexts where you could say 'I agree to you'.
I agree with you carries both of the meaning Incho mentions.
'I agree to you' could only be used in circumstances similar to 'I agree to Fred becoming chairman' or 'I agree to Mary buying more biscuits'. In this context though, to be absolutely correct you should actually say 'I agree to your becoming Chairman' or 'I agree to your buying more biscuits' not 'you'. A lot of people will say 'you' here (incorrectly but common in speech).