I realised that I have to pee more often in winter or simply the temperature is low. Is there any scientific explanation, or just simply psychology?
This phenomenon has been demonstrated in studies  which show a significant increase in urine output in the cold. The latter study shows the biggest increase is when a person is initially exposed to the cold rather than when they continue to be exposed or rewarmed up. In the study, the average urine out was 157 mL/hour when a person was initially exposed to the cold, 103 mL/hour when they were being maintained at that cold temperature and 70 mL/hour when they were being rewarmed.
The mechanism however is unclear. This study showed that the mean arterial pressure increases. It is well known that the cold causes our blood vessels to constrict (in order to keep the blood and therefore heat inside our core body). The thought was therefore that our kidneys try to lower our blood pressure by taking more fluid out and emptying it into the bladder. This is called cold-induced diuresis.
Other thoughts include that we lose the water we ingest by sweating (perspiring), pooping (defaecation) and peeing (micturition/urination). In the winter, we don't typically sweat as much, which ISN'T counterbalanced with a lower water intake, so we need to pee more.
Another possibility is that aquaporins (the pores that water comes out from our kidneys) are inhibited by the cold. This means more fluid would be filtered out. However there isn't really much research in this area that I am able to find.
Cold-induced diuresis, where kidneys relieve blood pressure by increasing urine production.
The blood pressure is thought to be increased by vasoconstriction due to the cold. One source that supports that theory: Sun, Z. American Journal of Physiology -- Regulatory, Integrative, and Comparative Physiology, February 2005; vol 288: pp R433-R439. American Heart Association: "High Blood Pressure."