Actually, it's got nothing to do with evolution or even human biology.
What’s in a name? That which we call a rose,
By any other name would still smell as sweet. William Shakespeare.
First, try to define visible yourself. Next look at the definition of it in various respected dictionaries; in our context it means
that can be seen
Consider the case of Goffin's cuckatoo, they can see as far up to as ultra-violet spectrum. They don't speak English (or Portuguese for that matter) but for them visible would mean "what they can see".
So, coming back to your question, even if we had capability to see, say, infrared spectrum, visible would have meant: VIBGYOR + infra. And then we'd have complained about not being able to see bacteria with naked eye :P