less of the hats, yes, in that context, but less hats, no. fewer hats is the correct form if you don't have the of.
In fact, in the context of partially obscuring the view, I see fewer hats and I see less of the hats mean two different things, which you seem to be quite aware of. ...