A: How do you like your coffee?
B: I'm not a coffee drinker.

Does it mean B doesn't drink coffee or rarely?
He doesn't drink coffee.

I'm not much of a coffee drinker - only rarely
I'm not a big coffee drinker. - I don't drink much coffee - so the implication is rarely.

I'm not a big cigar smoker. - I don't smoke many cigars - again, implication is rarely.
Students: We have free audio pronunciation exercises.
I believe the same logic applies to "I'm not a cigar smoker". But I remember a Hollywood star said that when he meant he rarely smoked cigars. I could have misunderstood him. Please confirm. Thank you.
New2grammarhe rarely smoked cigars
It means that he seldom smoked cigars.
Students: Are you brave enough to let our tutors analyse your pronunciation?
 Optilang's reply was promoted to an answer.