It is, however, commonly used in the way you suggest. If it's a really big window, it could mean standing in the window, ready to jump into the firemen's safety net.
You may ignore these: Google -
"standing in the window" 449,000
"standing by the window" 306,000
"standing at the window" 70,200
- A.