The answer lies in the protein composition of mitochondrial membranes, which undoubtedly prove that the outer membrane (OM) is of alphaproteobacterial origin, and the phagosomal membrane (if there was any) is lost. Though the endosymbiotic origin is without question, bear in mind that the phagocytotic origin is still debated (cf. López-García & Moreira 2015). And if the symbiont has entered the host some other ways than phagocytosis (cf. papers of Tom Cavalier-Smith) or syntrophic engulfment (cf. papers by Bill Martin; Searcy; López-García & Moreira) (e.g. puncturing the plasma membrane as a pathogen), then there never was a third membrane.
From Symbiogenesis Wikipedia:
Porins (transport proteins) are found in the OM of mitochondria and
chloroplasts, are also found in bacterial cell membrane (Fischer et
al. 1994Zeth & Thein 2010Fairman et al. 2011). The
membrane lipid cardiolipin is exclusively found in the inner membrane
of mitochondria and bacterial cell membrane Mleykovskaya & Dowhan
2009.
According to many (Cavalier-Smith 2002, Cavalier-Smith 2006, Zeth 2010), the presence of β-barrel transport proteins proves that the OM evolved from the bacterial OM, not from the host food vacuole membrane. Both mitochondria and plastids contain in their OM β-barrel proteins (e.g. Tom40, Sam50 and porins in mitochondria, Toc75 homologue of Sam50 and their joint negibacterial ancestor Omp85 in chloroplasts) and these are only present in the OMs of gram-negative bacteria (cf. Schleiff & Soll 2005). All Alphaproteobacteria are gram-negative, hence the ancestor of mitochondria originating from the Rickettsiales or very close was also.
Note, that many secondary and tertiary plastids still retain their third, outermoest phagosomal membrane originating from the host's plasma membrane (cf. Keeling 2010).
