the phrase "Fortunately we had a map, without which we would have been lost" sounds actually a little harsh, but it should be ok, shouldn't it?
Probably that's due to the presence of "Fortunately"; it would sound better
"we had a map without which we would have been lost",
or, otherwise, if you want to point out that you were lucky, something like
"Fortunately we had a map: without it we would have been lost".
And I have another question: what about
"we had a map that we would have been lost without"
wrong? to avoid?