From Wikipedia:
The retinoblastoma protein ... is a tumor suppressor protein that is dysfunctional in several major cancers
So, although it's commonly associated with retinoblastoma, it's not limited to a particular type of cancer.
Retinal cells are not sloughed off or replaced, and are subjected to high levels of mutagenic UV radiation, and thus most pRb knock-outs occur in retinal tissue (but it has also been documented in certain skin cancers in patients from New Zealand where the amount of UV radiation is significantly higher)
Since the gene is commonly damaged from UV radiation, the retina is particularly vulnerable. UV radiation does not penetrate very deeply, but the retina is exposed to light per its function.
Two forms of retinoblastoma were noticed: a bilateral, familial form and a unilateral, sporadic form. Sufferers of the former were over six times more likely to develop other types of cancer later in life, compared to individuals with sporadic retinoblastoma.[10] This highlighted the fact that mutated pRb could be inherited and lent support for the two-hit hypothesis
The two-hit hypothesis is the idea that often one functional copy of a tumor suppressor gene is sufficient. For people with familiar cancers involving tumor suppressor genes, including the BRCA genes associated with breast cancer, usually what you have are people who are heterozygous for some non-functional version of the gene. That means that they only need one "hit" to the functional gene to facilitate development of cancer.
pRb restricts the cell's ability to replicate DNA by preventing its progression from the G1 (first gap phase) to S (synthesis phase) phase of the cell division cycle
Because pRb is associated with G1 to S phase transitions, you wouldn't expect it to be as important of a gate in differentiated cells in G0; that's why you're finding it in a childhood cancer: the affected cells need to be immature and dividing to be affected.
In summary, Rb loss-of-function mutations occur via UV damage. To be vulnerable to this damage, a cell needs to be actively dividing and exposed to UV. The developing retina fits these characteristics. People who inherit one non-functional Rb copy are much more likely to develop cancer, because they only require additional damage to one copy of the gene, not two.