Most people do ask, sooner or later, but I hadn’t had Sam down as the inquisitive type, and I wondered why, exactly, he wanted to know.

to have (someone or something) down as

~ to estimate or consider (someone or something) in a certain way

I hadn't had him down as inquisitive
~ I hadn't thought he was inquisitive

It's related to putting something down in your notebook, writing it down.

You check your notebook, and there is no note there saying that he is inquisitive, so you say you didn't have him down as inquisitive.