Hi Somebody!
It depends on what you are trying to say.
"The process of modelling ionisation" Correct
"The process of modelling of ionisation" Incorrect
If you add 'of' before 'ionisation', the fact that it's a process is understood:
"The modelling of ionisation" Correct
Regarding your second question, you can write it like that, just make sure it's hyphenated:
"The process of computer-modelling ionisation"
This could, however, be a little bit confusing, especially if you wrote "The computer-modelling of ionisation". It'd be better if you wrote it like this:
"The process of modelling ionisation by computer"
"The modelling of ionisation by computer"
-Nyarlathotep