The 'short' answers are as follows.
'd taken to mean
had is an older interpretation. Over time, people have come to think of it as
would. The modern interpretation of
'd in that expression is now
would.
As for the tenses.
Case 1: The clause after
rather explicitly contains a subject different from the subject of the sentence.
I'd rather you [go / went]. Subject of sentence: I -- Subject of the preference: you.
Sue would rather Tom [do / did] the job. Subject of sentence: Sue -- Subject of the preference: Tom.
My son would rather I [buy / bought] the shirt. Subject of sentence: My son -- Subject of the preference: I.
In such cases either the bare infinitive or the simple past may be
used. (I''m not sure if either of these is actually preferred.)
Case 2: The clause after
rather implicitly contains the same subject as the subject of the sentence.
I'd rather {I} go. Subject of sentence: I -- Subject of the preference: I (unstated).
You'd rather {you}stay. Subject of sentence you -- Subject of the preference: you (unstated).
Mary would rather {Mary} do it herself. Subject of sentence: Mary -- Subject of the preference: Mary (unstated).
In such cases only the bare infinitive is used.
CJ