Technically, at doesn't refer to anything. at is not a referring expression. Generally, only nouns and pronouns can refer.
Nevertheless, I understand what you're saying.
Here at means 'upon hearing' or 'when I heard' or 'as I heard' or 'in reaction to' or 'as a reaction to'.
I didn't laugh upon hearing many of his jokes.
CJ