I work with a lot of mice. I don't do any behavioral analysis, I just inject DNA or RNA and do imaging assays. However, I have noticed some effect of isolation on mouse behavior. Normally, the mice are housed in our university's animal facility with no more than 5 mice to a cage. Sometimes, 1 mouse will be put in a cage by itself. When I go to get these lone mice from their cages, they seem more skittish, harder to grab, and more likely to bite me than mice kept in groups.
Additionally, our Office of Animal Resources, which is responsible for getting in the mouse shipments and putting them into cages, has recently changed how our mice are caged, with a group of 4 mice being normal, any leftover mice are put into cages with other mice, making a 5 mouse cage. This means I haven't seen a single mouse in a cage for a while. So I might not be the only one to have noticed an effect of isolation on the animal's health.
I have heard of studies on monkeys by B.F. Skinner, where he induced very severe depression and other psychological issues by isolating them, so I suspect similar studies have been attempted on mice using far more quantitative methods than I've been able to do.