The problem with increasingly complex HMMs is that their parameter space tends to explode with the nth-order of the HMMs. Higher number of parameters is often not great because it reduces the possible number of observations that go into training each parameter and can increase overfitting of the model.
From the information that you are providing it is possible that the 5th order model reaches the sweet spot of having great performance with a reasonably contained parameter space.
It is not clear how your model also works.
Is each state a single nucleotide or a single KMER?
Is it a generalized HMM with separate states for exons and introns with the KMERs being observations?
Work by the Bier lab has shown that 5-mers are very good at telling apart enhancers from background using a SVM model for classification. In your context it seems pretty reasonable to use 6-mers to find genes given this finding.
For more details please check "Biological Sequence Analysis" by Sean Eddy and the work of Dr. Michael Brent @ Washington University in Saint Louis (his lab has done a lot of research on HMMs for gene finding).
It'd be useful to have one or few paper references behind your question.