I linked to the paper above in a comment.
I'm not a behavioral scientist. So interpret with a grain of salt.
The study analyzes language used after tours of homes, to evaluate how stressful the homes appear to be. So it's an imperfect measurement probably, but in direct reference to their homes. It probably does reflect something about how stressful they feel their homes to be, but IDK.
Assuming that the slope of the diurnal cortisol curve is a meaningful measure (seems to be somewhat true based on my non-exhaustive search), and that the sample is adequately powered (n=30 couples, which is not too awful but not great either given the number of hypothesis tests). If you look at Tables 1 and 2, it does look like women are affected by a "stressful home" to a degree that men are not. Men also seem vaguely unhappy about stressful homes, though not as strongly as women. Granted, they only present regression tables, which is not always very helpful for understanding causal or statistical relationships.
Nonetheless, the paper seems well-cited in the literature (as might be expected if it made it into a book). A lot of the work citing it is saying things along the lines of "we need more work in this area to examine this effect". A small number of studies seem to confirm qualitatively that messy homes are stressful for women, see for example here or here. It is less clear that the same isn't true for men, but then the majority of housework and caregiving responsibility is concentrated on women.
Furthermore, it does seem like global cortisol differences in men and women are not surprising, and that cortisol-induced stress may affect women and men differently.
Generally, this study seems ok if not great. It fits into a broadly supportive narrative of (partially qualitative) research. The sum of the literature that I've skimmed very briefly cites this paper with approval, if perhaps making more of the results than is merited given the level of power for the study. No one has posted on pubpeer about it or found anything dodgy about the study, in spite of its relatively high profile, so it's not obviously awful. I don't love the linguistic analysis either, but clutter is itself subjective I guess.
There remains the issue that you raise that perhaps a person under stress will react more strongly than someone who is more relaxed. But it's a bit circular of course- in either case, it seems like women are more stressed, and they also get more stressed out about domestic clutter.
Interested to hear more from a real behavioral science professional. I hope that my answer doesn't discourage anyone with more credentials from posting.