Hi,
Switch sounds wrong to me. If Tom and Mary switch hats, it sounds like they each go home with a different hat, and that it's a permanent exchange. Same idea with phone numbers, which thus sounds odd.
If you want a word that starts with 's', I would say swap phone numbers.
Best wishes, Clive