If English-speakers learnt how to pronounce the Latin expression
per se, they wouldn't want to transcribe it as
per say. The
e of
se is not a diphthong. In no English word is a solitary
e pronounced as a diphthong, so why imagine it is so in other languages? Try removing the
t from
set.