Well, first: we don't
study a course, we
take a course; we
study Italian.
I took a course in Italian.-- this is the one that seems natural to me: Italian is the subject title.
I took a course on the evolution of the endoproct tentacular system. -- this explains the topic of the course rather than naming it.
Others may have different opinions.