You are asking a very interesting question.
As you correctly mention, the substrates of mitochondrial metabolism (TCA or Krebs Cycle) are pyruvate and NADH, and, through oxidative reactions, ATP is produced. Indeed, it seems unrealistic that when the "mitochondria" were by themselves there was pyruvate and NADH in the environment. Let's take a step back.
In the 1970s, Dr. Lynn Margulis proposed the extracellular origin of mitochondria. What does this mean? Margulis suggested that, due to the bacteria-like characteristics of mitochondria (e.g. circular DNA), it is plausible that the first mitochondrion was in fact a bacterial cell that had been introduced into another cell by endocytosis. Therefore, we cannot really describe mitochondria as having been "by themselves", since originally they were (most likely) bacteria. (I am emphasizing this because in your statement it seems like perhaps there is some misunderstanding with this point.)
Where were we? Ah, yes, so we have a bacterium (pre-mitochondrion) inside another (host) bacterium. Consider that the combination of one bacterium inside another works and, voilà!, we have symbiosis and, more precisely, endosymbiosis. This means that the two bacteria collaborate to survive, they benefit from interacting with each other (symbiosis) and one of them is inside another (endo-).
But, of course, the combination is not optimal: the two bacteria have common steps in their metabolism and this is a waste of energy. And evolution does not like that. Evolution encourages organisms to minimize their energy expenditure through natural selection: those organisms that manage to make the most out of their energy supplies will survive and thrive. This means that the pairs of endosymbiont bacteria that manage to use less energy to carry out the same actions will be positively selected for.
Indeed, what this possibly lead to was the host bacterium keeping its basic glucose metabolism (glycolysis), and the "inside"-bacterium exclusively keeping its TCA metabolism, effectively becoming a mitochondrion. Hence, the current arrangement of glycolysis in the cytosol and respiration in the mitochondria.
In short, it is important to look at the bigger picture: were there ever really "free" mitochondria? No, according to Margulis' Endosymbiosis Theory there mitochondria-precursors, which were bacteria. Once these bacteria were introduced into the cytoplasm of another cell, their metabolism degenerated and was reduced to the minimum necessary. This is a success story: the mitochondria are evolved bacteria that have managed to survive and thrive, and have made themselves fundamental for survival of many lifeforms, including humans.
I hope this is a useful answer to your question. Feel free to ask for clarifications in case I did not describe something well enough :)