American:
I'm in the hospital: I am a patient in the hospital or you are physcially inside the hospital.
I'm at the hospital: I am somewhere on the hospital grounds, either inside or outside, and you are NOT a patient.
The British will have a different answer for you.