Why is ATP the most prevalent form of chemical energy storage and utilization in most cells?
I really like this question as it is such a fundamental underpinning of all life on the planet, yet there is such sparsity of actual information on its origins and why selection rewarded ATP use over anything else. Here I am talking generally since no specific studies exist in ATP vs other candidates.
A lot of the below information is taken from a relatively old article mentioned in the comments by TomD that discusses: "Why nature chose phosphates." by Westheimer, 1987. The article is very influential and has been cited over a thousand times since publication. Another article that came out in the same year that this question was asked "Why nature really chose phosphate." by Kamerlin et al., 2013
Some of the below arguments are more convincing than others, but all of them should be thought of when attempting to answer this question.
ATP has ancestral dominance. Most other reasons derive from this.
Alternative phosphate groups or other molecules may not provide enough energy.
Alternatives may be toxic.
Other molecules, particularly phosphates, are used for inefficient high energy bursts.
Pi is a "good" leaving group.
Phosphates are fundamentally able to be regulated through electrostatic manipulation.
ATP synthase can efficiently reattach the Pi to ADP.
Lots of Pi available to organisms because of it's ancestral dominance ("if it ain't broken, why fix it?" is at play).
ATP can provide more energy if needed; it's scalable to the situation. (ADP becomes AMP + Pi)
Easily usable by a variety of proteins.
ATP is an efficient and relatively easily biosynthesised molecule that can fulfil multiple biochemical roles. Cells do have alternative energy carriers, some with more specialised roles, however, ATP is ubiquitous throughout our cells and inter-cellular spaces. There aren't a wealth of resources explaining why ATP is any better than other compounds, however, there is plenty of reasons why the phosphates are required.
Why not the alternatives?
Citric acids and their derivatives are a good candidate, with deductible groups and high bioavailability but they simply don't give enough energy to stabilise genetic material.
Another tribasic candidate is arsenic acid. This is a fundamentally toxic compound, though, which isn't particularly great for living things.
There are other phosphates too, and they are used in many organisms. In biology, they have specific functions, and not used as the general energy carrier. For example, creatine triphosphate provides a high energy phospho- anhydride bond, that is often used to quickly and anaerobically regenerate ATP, useful during high rate muscle activity for contraction.
GTP is structurally very similar to ATP. GTPases are used more to initiate cellular signalling pathways. It is sometimes used as an energy source. This is a good example of an alternative energy carrier.
Over the years, many proteins have specialised with a specific shape, and this chance is the primary reason behind ATP over GTP. In other words, the choice of ATP over GTP is primarily down to cellular preference of molecular shape. One of them had to emerge as being more widely used, and it was ATP that 'won'.
Efficiency and simplicity.
The reaction was once thought to be a relatively simple nucleophilic displacement. From the 2013 paper:
...this simplicity is deceptive, as, even in aqueous solution, the low-lying d-orbitals on the phosphorus atom allow for eight distinct mechanistic possibilities, before even introducing the complexities of the enzyme catalysed reactions.
Traditionally one will be taught that ATP is such a chemically efficient way of storing and transporting energy. This is due to the ATP->ADP & Pi hydrolysis reaction. The phosphate groups in ATP are full of negative charges and these are repelling one another. This means that the third phosphate is a great leaving group and breaking the phospho- anhydride bond is a favourable reaction. ...
...But the story is a lot more complicated than that. The above explanation isn't really satisfying because those same negative charge forces are repulsive of the nucleophile that is attempting to complete ATP->ADP & Pi. A more comprehensive explanation would go along the lines of 'although a negative charge repulsion exists between the nucleophile of the protein and the phosphate, that high energy barrier can be overcome by electrostatic manipulation'. This allows an "on-off switch" for the hydrolytic reaction by tweaking the electrostatic environment. This is another great regulatory tool that the phosphates provide. This regulatory feature is important for signal and metabolic/catabolic cascades.
When it comes to 'rebinding' the Pi to ADP, it is fairly easy since ADP seldom covalently binds to anything, which would require a lot of energy to recover the ADP. This also helps the bioavailability of free ADP to ATP synthase, an incredibly efficient enzyme, that uses membrane proton gradient to drive the production of ATP. Talking about actual numbers is difficult here as there is only data available from Rat hepatocytes. Who is to say mammals are representative of all organisms? The estimates of energy of hydrolysis range from ΔG˚ = -48 kJ mol-1 to -30.5 kJ mol-1. Note that these are considerable, but not exceptional values, so it's easy for many different proteins, that need not be very specialized, to break the bond all over the body. I couldn't even find the numbers for the synthase reaction per ATP, but a single ATP synthase can produce up to 600 ATP per minute.
The final point of this efficiency is that the elements in ATP are very abundant and established in the biosphere making it readily available. This makes the phosphates a convenient biomolecule.
ATP is ubiquitous in the body, but in some cases more energy is needed than there are ATP available. In these times of need, ATP can be used to produce more energy, breaking another phosphoanhydride bond to become AMP+2Pi. AMP however is typically a signalling molecule.
With the low activation energy required to break the phosphoanhydride bond, a multitude of enzymes, far too many to list here, can make use of ATP in order to gain energy towards the activation energy for many other functions.
I don’t like this sort of question because I don’t think it can really be answered and I’m very suspicious of arguments that seem to claim ATP is the only or even the best solution to the problem. Nature generally demonstrates that there is more than one way to kill a cat, but if one way works adequately you don’t always need to look for another.
This is not necessarily the case, of course, if we consider an example from the postulated RNA World — which I shall presently — catalysis using RNA was supplanted in most cases by catalysis using proteins. So sometimes a better solution provides an evolutionary advantage, and sometimes if things work well enough they stick as the limiting factor is elsewhere.
So I tend to the view that ATP worked, so it stuck. It was probably chance it wasn’t GTP or CTP or UTP, as these work as energy sources in signal transduction, phospholipid synthesis, and glycogen synthesis, respectively.
But this raises the question of the function or necessity of the purine or pyrimidine ring in the nucleoside triphosphates. As far as I can see the answer is this serves no indispensible function. (Sure, it binds to enzymes — but all sorts of other structure are able to do this.) So I would like to fly the following kite (which must have been flown before, although I am not aware of a reference).
A nucleotide triphosphate (ATP) became the preferred energy source in metabolism after a mechanism of RNA synthesis evolved that used NTPs as substrates.
When the synthesis of RNA evolved to use the free energy of hyrolysis of a ‘diphosphorylated extension’ of its structural building block (NMP), a system of using a related hydrolysis was extended to metabolism. Note that I say ‘related’, as RNA synthesis (like other macromolecular syntheses) hydrolyses the alpha-beta phosphodiester bond (releasing pyrophosphate), whereas in metabolism it is generally the beta-gamma bond that is hydrolysed (releasing orthophosphate).
The hydrolysis of ATP would have displaced what it is thought to have been a pre-replication system(s) of energy generation because it would presumably have been better and allowed integrated energy metabolism. (The energy demands of replication would have been large.) But that doesn’t mean it’s the best conceivable method — working well and being convenient could have been enough.
Although not part of my argument, there is another key molecule in metabolism that has what may be considered a ‘useless’ adenosine component — NAD (and NADP). The redox guts of this is the nicotinamide ring. Did this evolve from a form that was initially part of a ribozyme — perhaps involved in the formation of deoxyribose when the RNA genome was being displaced by the DNA genome?
I have already provided an answer to this question, addressing one aspect of it: why a nucleotide triphosphate — rather than any other molecule — was the choice for an energy carrier. In that answer I suggest that the choice of ATP, rather than any GTP, CTP or UTP, was mere chance.
This second question has, in fact, been posed, but was regarded — incorrectly in my opinion — as a duplicate. I have recently become aware of research that suggests to me a possible reason for the preference of ATP over other NTPs, and, as it is independent of my previous answer, I would like to present it as a separate answer.
Why ATP rather than other NTPs?
I am starting from the assumption of an RNA world in which some sort of RNA genome had developed the ability to replicate and exhibit enzymic activity. The ribosome, and especially ribosomal RNA, can be regarded as a fossil of such a world. Harry Noller, in a review of this topic in Science in 2005, considers the bases that are involved in base-pairing in the many RNA double-helical loops, and those that are unpaired in such loops. Many of the latter are involved in ternary interactions in the rRNA, which has an overall protein-like structure. A significant fact that he mentions about the bases that are unpaired in the helical secondary structures is their skewed distribution:
“However, the unpaired bases are not distributed evenly among the four bases. In Escherichia coli 16S rRNA, for example, the proportions of unpaired bases for G, C, and U are 31%, 29%, and 33%, respectively, whereas 62% of As are unpaired, a tendency that extends to other functional RNAs.”
It transpires that many of these ‘unpaired As’ are involved in what are called Type II A-minor nucleoside interactions, which are illustrated in Fig 3. of that paper, below:
In the legend to this figure Noller points out that:
These precise lock-and-key minor-groove interactions between (usually) an adenosine and a Watson–Crick base pair are found extensively in 16S and 23S rRNA. They were first observed in crystal packing of the hammerhead ribozyme and in the P4-P6 domain of the group I ribozyme. A-minor interactions play an important functional role in monitoring codon-anticodon interaction by the ribosome via their unique stereo-chemical fit to Watson-Crick base pairs.
From this I conclude that adenine has unique structural properties that would allow it to form a more precise and stronger interaction to base-pairs in a ribozyme RNA than the other three bases. One of the features of any ribozyme involved in using an NTP to drive chemical reactions would have been to bind the NTP. (We see this in contemporary proteins with the Rossman Fold which binds adenine.)
The greater suitability of the adenine of ATP for this role may be why it, rather then GTP, CTP or UTP, became the major (I would assume initial) choice for energy carrier.