"... brings back memories of ..." is meant to be general. The memories of a particular person are not at issue.
If something brings back your own personal memories of something, you say, "... reminds me of ..." or "reminds me that ..."
Seeing that man with the dirty coat reminds me of the laundry I have to do.
... reminds me that I have to do some laundry.
CJ