B: Oh no. I didn't buy the lottery.
A: I told you so
B: If I had listened to you, I would be rich now./If I listened to you, I would be rich now.
There are some problems:
B: Oh no. I didn't buy the lottery. You cannot buy the lottery, you can buy a lottery ticket.
B: If I had listened to you, I would be rich now. If I had listened to you before the lottery draw, and if I had bought a ticket, I would be rich now (supposing that the numbers had been selected before the draw).
If I listened to you, I would be rich now. For me, this is more general - If I listened to you (over a period of time) I would be rich now.