Hello, DMC, welcome to the English Forums!
Let me try: In "contact manager", "contact" is treated as an adjective, and doesn't take an -s. But if you use it with an -s, I think it should be "contacts' manager", possessive form, manager of the contacts.
Other members may correct me, though...