As the embryo reabsorbs calcium from the shell, it also reabsorbs an equal quantity of carbonate. Calcium is in quite high concentrations in the body, the same molarity for carbonate causes a very alkaline environment. Carbonate has a pKa near 10, seen in that the egg white in "food eggs" (non-incubated and non-impregnated) becomes alkaline as the egg ages, approaching pH 9 to 10.
Throwing some numbers in there, assuming 2.2 gram of CaCO3, 8% reabsorbed to cause the average 8% thinning documented here (this is also just day 3-5 I think. ) 2.2 gram*8% = 0.176 gram carbonate released. The molecular weight of carbonate is 61 g/mol, 0.176 g / 61 g/mol = 0.003 mol = 3 mmol. If newborn chicken weighs 85 gram, and is 70% fluid, 85 gram * 70% = 60 gram. 60 gram is 0.06 L water. 3 mmol carbonate equivalent to 3 mmol/0.06 L = 50 mmol/L. The solution gets a pH of 11, and if you add 25 mmol/L of hydrochloric acid it is still at pH 9.2.
How is that carbonate neutralized?
At least 1 carbonate ion in 5 can be neutralized by bone formation, since hydroxyapatite has a ratio of 5:1 calcium to hydroxide ions, and, 99% of calcium is in bone (assuming somewhat similar ratio in infant as well. ) What about the other 4?
EDIT: A user David emphasizes that Wikipedia's entry on Calcium carbonate gives a pKa of 9. I based pKa closer to 10 on equilibrium between carbonate and bicarbonate. 9 or 10 does not really change the question that much since its about the huge quantity of base, quite strong in either case. I used http://www.aqion.onl/ for pH calculations giving pH 11. Feel free to use the correct value.
EDIT: Carbonic acid from CO2 from cellular respiration was mentioned. Carbonic acid from cellular respiration is at least part of it. But I don't know if it is all of it or close to all of it. 3 mmol of calcium absorbed (see Crooks, 1973) if average egg volume is 50 mL, molarity is 60 mmol/L, and this is as carbonate which saturates pCO2/H2CO3/HCO3-/CO3^2- as well.
EDIT: Also quite a lot of uric acid created, 2.3 gram per L by day 13, jbc.org/content/70/2/535.full.pdf, molarity 14 mmol/L.
SUMMARY: Bone formation likely takes care of 1 in 5 carbonates. Carbonic acid from cellular respiration, uric acid from nitrogen excretion, both contribute. Other factors?