"Truth be told, I never graduated from college and this is the closest I've ever gotten to a college graduation."

Here the meaning of this structure is very clear. (= to tell you the truth) 
But I can't get why this structure is used in this way. A sentence adverbial? How does it function in the sentence?

Not only is it an idiom, it's a stilted and old-fashioned idiom. I love it. (Maybe subjunctive.)
Definitely old. Definitely subjunctive.
It's from the days when if clauses took the subjunctive:
(If the) truth be told
Modern: If the truth is told
Teachers: We supply a list of EFL job vacancies
Or: Should truth be told
 Thank you, guys.
Actually, it's from the commencement speech delivered by Steve Jobs at Stanford in 2005.  People like him always introducing new fantastic stuff to us tend to use very old expressions eh?!
Emotion: stick out tongue
One of the advantages of not going to college. Education stifles creativity.
Students: We have free audio pronunciation exercises.
Yep, but most of us have to do that.