No, you don't put a comma on the end of a paragraph. You attach the direct quote directly after the comma, without starting a new paragraph for that quote. The next person to talk gets a new paragraph.
Jenny was confused, and asked, "What do you mean?"
"I mean, there's no need for the colon," I...