These sentences are correct, it seems
To get to my house from yours is a three hour walk.
My house is three hours' walk from your house.
but my general question is when should I use the pattern " three hour's walk or five days' trip" and when to use the pattern " a three hour walk/ a five day trip". Thanks.
To get to my house from yours is a three-hour walk.
My house is three hours' walk from your house.
They are both rather formal, and not really conversational. The first is probably used more.
In a conversation, you night hear this:
It will take me three hours to walk to your house and another three hours to get back.