"With regards" is fairly formal, viz "Give My Regards To Broadway." No, I would not be offended if someone ended their letter with "Regards." Regard, has several meanings. All serious and respectful.  "Fondly" on the other hand has a sexual connotation, especially in England.

 Fondle: Caress, to touch or stroke lovingly. (According to the OED).

 Fondly:  1) Affectionate, loving. 2) Over affectionate, doting.

Affectionate:  Loving, showing affection. (denoting emotion)

None of these should be used in a business letter.

The correct usage of 'faithfully' and 'sincerely' is taught in junior school English. If addressed Dear Sir or Madam then it is 'faithfully' or to a named person 'sincerely'.   

However, today I have been rebuked by my new boss for using 'sincerely' instead of 'faithfully' when the salutation is Dear Sir. Why did he do this?

Well, in my two previous companies I was told it was 'old fashioned' and 'sincerely' would suffice especially 'Sincerely' if the salutation is Dear Sir in a new modern world where formality is now more relaxed, 'faithfully' is to be used only if the letter is a legal  document or appertaining to strictest business formalities. 

So, in a marketing letter to an unknown recipient it acceptable to sign off Yours Sincerely with a capital.  My last company (British owned marketing company) paid for me to attend a course to be fed this information and my previous company was an American owned British based company, both of these companies abided by this ruling.

Today I have had my wrists slapped and been fed a great deal of sarcastic feedback, as though I am some sort of moron, as I abided by a newly introduced relaxed discipline. My current company is a marketing/sales company.

I was also taught that in an effective marketing letter it is a marketing ploy to always add on a  P.S. even if it is a business letter. The rules are now more flexible but it is not always easy to confirm what is correct or otherwise.  Feeling very miffed..........yours truly

MrPedantic

“

Hello Anon

"Yours truly" is mostly used in American English, so I'm not too sure. Maybe a passing AmE will let us know.

MrP

”

Passing American here.

Having spent most of my day today drafting business letters (dunning letters, actually), I can honestly say I have never closed a business letter with anything other than "Sincerely,"  In no way is this considered casual or informal in business use in the U.S.

I don't know that I have ever used "Yours truly," but I suppose I would use it in personal correspondence to someone I don't know well. For example, if a friend of a friend had hosted me on a visit, my "bread and butter note" might be closed that way. Since we are a shockingly casual race, we tend to end letters with things like "Thanks again," or "Looking forward to seeing you soon," and then signing our names.

Sometimes as I find myself typing "Sincerely" I have this passing fanciful notion that somewhere in a vault are all the letter that were written insincerely, but were not allowed to be sent out.

If you begin a letter with Dear Sir/Madam, then it should be signed off with Yours faithfully.  Everything else ie Dear Alan, or Dear Mr Jones, should be signed off with Yours sincerely.

If you are writing an informal letter to a friend then you can pretty much sign it off with whatever you like ie yours truly, kind regards or whatever.