Oh, geez... let's face it: every learner hates the present perfect continuous!

I would say "It's been bothering me" is ok if you are referring to NOW in some way, so it's ok if you are solving the problem NOW or you solved it just NOW ("just now" is basically still "now", if you want to see it that way)
And "It had been bothering me" is ok if you are referring to the past. One week ago, yesterday, or even "just now", if you want to consider what happened just now as something in the past.
As you see, it seems complicated to me, LOL

EDIT: seeing your original question, if the problem is solved and the trouble is over, maybe you'd better say "It had been bothering...", to stress it's part of the past, it's over, and you
solved it (<- past).