Violence, is well, violent. Abusive sexual contact may not necessarily be violent. Rape may or may not be violent. Another name for rape is non-consensual sex. The key is whether the party involved consent to the act. Raping someone using so called date rape drug often does not involve violence.
EDIT:
Abusive does not necessarily mean physical violence. Leaving a little child at home alone for a long time with no food is an abusive act. Using different means of coersion to force someone to have sex is abusive even though no violence is involved.