Your revision is good, but your absolute statement about not using past perfect for a time in the past is wrong.
Yesterday, we had just finished our lunch when the police stormed the house. They had mistaken our house for our neigbhors' and apologized, but not before they had scared the...