I agree with whoever it was who said it was any type of sex outside the sanctioned state of matrimonial bliss.
If you are 16 and not yet married, you can have "sex outside of marriage" by having pre-marital sex.
If you are 41 and divored and enjoying your freedom, you can have "sex outside of marriage" by having sex with anyone you can get your hands one.
I don't think it incudes the idea of "extra-marital sex" because that's obviously not sanctioned by those serving as the moral police.
You will not hear this term "sex outside of marriage" except in a discussion of morals, I should think.