You say it is "Not exactly a direct object." Are you able to say what kind of verb complement it    is,exactly?   Traditionally, verb complements are either nouns (and these include verbal phrases that stand for nouns) or adjectives.  It seems that you are saying that "to get hungry" is neither of these, but is, instead, adverbial and, thus, a modifying phrase rather than a true complement.

If you read my letter carefully, you will see that I was trying to fathom what Mr. Pedantic was saying.  That is all.  In view of that, I found the rhetorical thrust of your response a little sharp!  In one respect, I think your analysis errs. You said: "An adverbial is a constituent that modifies a verb or a sentence. If a constituent is an adverbial, the sentence can make any sense even if that constituent is elided."  What about a sentence like, "No one is here"?  It does not make sense without the adverb.  Tha same applies with the use of prepositional phrases that are used adverbially after linking verbs: "The boys are in the room."  Notwithstanding this, in traditional grammer -- as I have already stated and as you clearly already know -- verb complements are only nouns and adjectives.

What is interesting is that the original question remains unanswered!  Don't you think?  Infinitives may act adverbially, but that does not apply in this case.  I am inclined to say that Anon's hunch was correct: the infinitive phrase is a direct object verb complement.   I hope I am humble enough to take it if I am wrong!

This message is my monologue. I would not speak to anyone. It's just my monologue.

I think it's not wise to relate "complement" to certain parts of speech, i.e., nouns and adjectives. Take a construction of "He seems". It takes various forms of speech.
            Paco seems [foolish] …. adjective.
            Paco seems [a fool] ….. noun.
            Paco seems [to be a Japanese person]  … infinitive clause.
            Paco seems [as if he were mad]. …. as-if clause.
As seen above, a variety of forms can be as the complement of "He seems", but indeed all of them can function as the complement of "He seems".  

paco

I think I'll dig my heels in on this one.  Your exceptional cases may be understood within a conventional subject complement framework.