The first one is the correct one. "My brother did not want to play cricket, but his teachers were able to persuade him."
Use "was able" or "were able" instead of "could" when you say that someone succeeded at some task. Here the teachers
did persuade him to play cricket as they intended to do.
A common formula is "... tried to X" amd "was/were able to X".
-- Did they try to persuade him to play cricket?
-- Yes, they did, and they were able to persuade him to play cricket. (They succeeded.)
-- Did you try to catch the bus?
-- Yes, I tried, and I was able to catch the bus. (I succeeded. I did catch the bus.)
CJ