Is there any distinctions between 'I thought he may have visited there' and 'I thought he might have visited there'? If there is, could you give examples elucidating in tense and other perspectives?

Thank you.
I would not use the first. "thought" is past; "may" is present. The most normal and neutral choice after "I thought (that)" and similar phrases in the past tense is "might" (or "could" or "would", etc.), not "may" (or "can" or "will", etc.).

I knew that he would agree. Emotion: smile
I knew that he will agree.

I thought he might go. I thought he might have gone. Emotion: smile
I thought he may go. I thought he may have gone.

I said I could have waited another 15 minutes. Emotion: smile
I said I can have waited another 15 minutes. Emotion: sad

I think "may" would be OK in BrE.....but less common, and probably sounding rather formal.
Teachers: We supply a list of EFL job vacancies
I think I may be more used to using 'may' after 'think'.

I think thought may be the past of think!
I may have thought think might have had to have may; but now I think I might have thought think might as well have had might (as well).
Site Hint: Check out our list of pronunciation videos.
But I think the point might have been that we may often be able to use may, not that we might often use might.