Can we use both gerund and infinitive with Know.

#1 He knows swimming.

#2 He knows to swim.

#3 He knows how to swim.

#4 He doesn't know to swim.

#3 is correct but why Sentence #4 is incorrect? I have doubt about #2 if it's correct or not.

#5 He knew to stay out of sight. (similar to #2)

#6 He knew how to stay out of sight.

What is the difference b/w #5 and #6 in context of meaning?
#1, #2 and #4 are possible, but they mean something different from #3.

"He knows swimming" means that he possesses wide general knowledge about the sport or recreation of swimming. It does not simply mean that he can swim himself.

"He knows / doesn't know to swim" means that he knows / doesn't know that he ought to swim (in some situation explained by the wider context). It would not be a very common thing to say. #5 is analagous: He knew that he ought to stay out of sight — that staying out of sight was the correct thing to do.
1. to know how to ... = to be able to ...
2. to know (enough) to ... = to know that one should ...
3. to know -ing = to be an expert at ...

1. Larry knows how to speak French. / Jane knows how to knit. / Sam knows how to drive a tractor.
2. Sue knows (enough) to remain silent about Mary's divorce. / Frank knows (enough) to run from an angry bear.
3. Jack knows gardening backwards and forwards. / Lilian knows publishing better than Betty.

1. is used most. Use this one as much as you like.
2. is not used so much. Rephrasing often gives a better sentence:

Sue knows that she should remain silent about Mary's divorce.
Frank is smart enough to know that he should run from an angry bear.

3. is comparatively rare. Not many verbs can be used in this construction. I recommend that you avoid it. Substitute a common noun for the gerund if you can, or rephrase.

From these notes you should be able to figure out the meaning of each of the sentences you posted.