Yes, your ideas are somewhat jumbled.
Yesterday I went to math class. -- Much like 'I went to school/church'. It is idiomatic usage for certain familiar (to schoolchildren) facilities. This is certainly the most common way to express your idea.
Yesterday I went to the math class. -- The context must somehow mention this class before or otherwise indicate that it is a unique or specific one, e.g.: I went to the math class [that I just enrolled in]. I signed up for a math class and a science class, but I only went to the math class.
Yesterday I went to a math class.-- As I exemplied above, use 'a' when it is one of many and previously unmentioned.
Yesterday I went to Math class. -- This is possible if the class has such a very simple name, but capitalization (and loss of article) normally occurs with more specific or definitive names: Yesterday I went to Differential Calculus 201.
In general use are subject classes i.e math, science, history - capitalised or not? -- No; such names usually just serve to indicate the general subject. Yesterday I went to a history class, Early Renaissance.