+0
Are these sentences correct:

1-He wrote his first experimental novel in 1963.
2-He wrote his first, experimental novel in 1963.
3-He wrote his first, experimental, novel in 1963.

4-He wrote his first, his experimental novel in 1963.
5-He wrote his first, his experimental, novel in 1963.

In which case
a-His first novel was an experimental novel
b-He had written non-experimental novels before 1963
c-His first novel was his only experimental novel
+0
navitasanAre these sentences correct:

1-He wrote his first experimental novel in 1963.
2-He wrote his first, experimental novel in 1963.
3-He wrote his first, experimental, novel in 1963.

4-He wrote his first, his experimental novel in 1963.
5-He wrote his first, his experimental, novel in 1963.

In which case
a-His first novel was an experimental novel
b-He had written non-experimental novels before 1963
c-His first novel was his only experimental novel

I believe I understand what you're driving at, but in this particular case it doesn't work. In my opinion, only your first sentence is acceptable, and it is not of type (a), (b), or (c).

The only thing we may conclude, beyond what is stated, is that he had not previously written an experimental novel.Emotion: sad

Number four comes close, as an appositive, but it needs another comma after "novel."
Comments  
Edit. I meant to explain that with the second comma, "first" would serve as object of the verb, and be used as a noun. In your version five, both "first" and "experimental" are adjectives. But it's just too awkward. With version four, we'd assume that prior context makes it clear that "his first" was a novel.