Hi
Though I am not good at speaking english at all,but my confidence is strong,or it is more proper to say I do not care my speaking is right or not,I just know one point:open your mouth at everychance!
yes,that's it.You will never 'have' the confidence about something,but 'build' it in practice.It is a verity to anything including 'speaking english'.
All above is my personal experience and I hope it will be helpful to you as I am not a native speaker either.
Good luck to you!