To add some detail to Christian H's answer, while fish tend to move from side to side (lateral undulation), the land ancestors of marine mammals had their limbs under them and so their spines were adapted to up and down movement (dorsoventral undulation). Hence, vertical tails in the former and horizontal in the latter (the wikipedia article on fins gives some more detail, and links to this webpage on Berkeley.edu). In fact, this paper suggests that for cetaceans, dorsoventral undulation as a swimming strategy came first, and the horizontal tail evolved later.
As always, there are complexities and exceptions. The icthyosaurs and other aquatic reptiles developed vertical tails, even though like marine mammals they evolved from four-footed land animals. This may be because the legs/spines/gaits of land reptiles differ from those of land mammals, so that even the earliest icthyosaurs swam with lateral undulation, as reflected in their spinal modifications.
This blog post gives a really superb run-down on the issue, with figures and citations. There's a particularly nice bit I'll quote, which deals both the the dorsoventral undulation theory, but also with some of the exceptions (emphasis mine):
...mammals and their relatives were carrying their legs underneath their bodies and not out to the sides since the late Permian, and so the motion of their spine adapted to move in an up-and-down motion rather than side-to-side like many living reptiles and amphibians. Thus Pakicetus[the pre-cetacean land mammal] would not have evolved a tail for side-to-side motion like icthyosaurs or sharks because they would have had to entirely change the way their spinal column was set up first. At this point some of you might raise the point that living pinnipeds like seals and sea lions move in a side-to-side motion underwater. That may be true on a superficial level, but pinnipeds primarily use their modified limbs (hindlimbs in seals and forelimbs in sea lions) to move through the water; they aren’t relying on propulsion from a large fluke or caudal fin providing most of the propulsion with the front fins/limbs providing lift and allowing for change in direction. This diversity of strategies in living marine mammals suggests differing situations encountered by differing ancestors with their own suites of characteristics, but in the case of whales it seems that their ancestors were best fitted to move by undulating their spinal column and using their limbs to provide some extra propulsion/direction.