B is a bit weird - I saw him looking angry, maybe.
Even A: I saw him acting in an angry manner, I saw him acting angrily.
Maybe because "anger" is an emotion, not a behavior, while "silly" is a way to behave. It's easy to see someone being silly, but you can't necessarily tell that someone is angry.