While the most popular image of a fish for many is as you say, a more "vertical" than horizontal body shape, however there are many different adaptations to body shape depending upon the lifestyle of the fish. An ultimate example would be the flat fishes, such as flounder and halibut, which lay on one side essentially flat on the ocean floor or other similar surface. These fish however have adapted to have both eyes on one lateral side. There do exist other fish, such as some angler fish, especially in the family Lophiidae, which are more wide than tall, and they sit on the ocean floor and use a lure like appendage to attract prey so that they can ambush it, using their fins more to walk along the bottom surface. The feeding/hunting style is also highly related to the body shape of a fish, some shapes are more suited for moderate constant speeds, while others are suited for short quick bursts. Generally though, flat fish are not well suited for swimming at the middle of the water column, giving them a smaller habitat in which to live, while the more commonly found fish body shapes are used by much more species throughout the water column.
More information from MEER:
More information regarding Fish locomotion on Wikipedia.