HI..
Here's some grammer rules that I know.
simple past for some time that you know when in the past. (yesterday, this morning, etc)
past perfect is some times that you don't know. (before, long time ago, etc)
so in your example..
I had a conversation with Jim this morning. (correct)...