You don't have to use "ever" to be grammatical. Without "ever"
there's no question that the structure is grammatical. However,
to be fully idiomatic, you have to realize that, to the English
speaker, the structure "... superlative ... (that) ... (have/has/had) Past Participle ..." fairly...