No, it doesn't work that way. 'Took me for all I had' is an accepted phrase.
I think that 'took me' is being used in the sense of 'tricked me/exploited me', not taking something.
In any case, to take something away from someone is different to taking someone away from something.