Interesting question and actually I haven't found a real answer here yet. The phages seem to bind rather hard, since the need to be blended (subjected to relatively high forces) to shear off. This was used in the Hershey-Chase-Experiment to find out what the genetic material of the cell is. It doesn't matter though, since most of the cells which are infected get destroyed anyway. If not, they are most likely taken up and broken down inside the cell, as it happens with other membrane proteins.
This is only valid for viruses which inject their genetic material into the host cell, while the capsid stays outside. This happens mostly for Bacteriophages.
There are a few other possibilities for viruses to enter cells where this is no problem:
Viruses which have a membrane envelope can enter the cell via membrane fusion. To make the fusion happen, either the membranes have to get very close, or the virus first binds receptors on the cell surface and then fuses its membrane with the cell. This causes the interior of the virus to be released into the cell. The picture below shows the fusion (from the Wikipedia article on membrane fusion). An interesting article on this topic is: "Membrane fusion."
Another possibility for a virus to enter a host cell is via endocytosis. Here the virus uses the uptake of material from the surrounding of the cell to get inside. This can either happen together with resources inside a vacuole.
Or it happens via the binding of specific receptors, which are internalized into the cell upon binding of the specific ligand. Here the virus mimics the ligand of the receptor to get internalized into the cell.
In both cases the virus needs to escape from the vacuole before it get broken down because the vacuole is turned into a lysosome.