Hello MrPedantic, yep, you are quite right!
Then {with X} and {without X} (should?) have the same status, in principle, as expressions which set some 'condition.'
Then we have to seek purely 'pragmatic' explanations to account for the oddity, as you and Clive did. (I won't disagree!)
I'm...