The distribution of temperate hardwood forests, as well as deserts, tundras, savannahs, and other ecosystems, is determined primarily by average climate conditions, specifically average annual temperature and precipitation. You can see how biomes are related to climate in this figure, taken from Whittaker (1975):
Temperate forests tend to be between 2.5-17°C, and between 800-2500 mm precipation. These are not exact boundaries but reflect long-term climate conditions. As climate changes, so will the distribution of the biomes.
Temperature is important in many ways. For example, most temperate trees go dormant during the winter. This requires metabolic preparation by the tree. If the weather turns cold too fast, then the trees may be be properly prepared. Alternatively, if the weather warms to quickly the trees may not be able to come out of dormancy properly. This technical report (Coder 2011) provides more detail.
The precise mechanism that restricts the distribution of any one species would require experimental determination. As it turns out, Populus trichocarpa (black cottonwood), plus other Populus and Salix (all in the family Salicaceae) are quickly becoming model organisms for study. This should shed more light on specific distribution limitations, such as changes in soil nutrients, light intensity, or other environmental parameters. There are now regular symposia held on these model organisms. You can read an introduction to one symposium here (Tognetti et al. 2011).
As a side note: One must be careful about the mapped distributions of popular trees like willows. Many species, including Salix viminalis, have been introduced far outside their native range, including through out the eastern United States, as shown by the map you linked to. The native distribution of the species is the Palearctic (Eurasia), shown here.
Coder, K.D. 2011. Trees & cold temperatures. Environmental Tolerance Series WSFNR11-12, Warnell School of Forestry and Natural Resources, University of Georgia, USA.
Tognetti, R. et al. 2011. Fifth international poplar symposium: Poplars and willows: From research to multipurpose trees for a bio-based society.
Whittaker, R.H. 1975. Communities and Ecosystems. MacMillan Publishing, New York, USA.