This article states that hermit crabs in the wild are extremely difficult to track because they constantly change shells and even gender. Later in the article, they say that hermit crabs can live 4 decades in the wild. How can they know this if they can't follow a single crab for an extended period of time?
Hermit crabs increase in size over time and this is used to provide a minimum estimate for their age1.
You may also find this article interesting.
SantAnna, Bruno Sampaio, Christofoletti, Ronaldo Adriano, Zangrande, Cilene Mariane, & Reigada, Alvaro Luis Diogo. (2008). Growth of the hermit crab Clibanarius vittatus (Bosc, 1802) (Crustacea, Anomura, Diogenidae) at São Vicente, São Paulo, Brazil. Brazilian Archives of Biology and Technology, 51(3), 547-550. https://dx.doi.org/10.1590/S1516-89132008000300014