Thank you.
I am not so sure that this is what CalifJim was trying to get at but with all due respect, let me gingerly ask you this question?
The sentence of yours,
The book [which you told me to read, by the way] was very good.
The part in brackets (???) are not JUST additional information but essential information that gives restrictive meaning to the sentence. Which book? the book you told me to read and thus, should not set off aside as an element (piece) of additional information.
And I am so sure the usage of "that" or "which" is what set the distinction of being restrictive and non-restrictive apart. I think the nature of the phrase/clause that follows is what trully determines the choice.