I understand that inbreeding, after a number of generations of crossing genetically related individuals eventually yields homozygotes, however I can't seem to understand how it can be used for selecting mutations.
Consider a sibling-sibling mating system, the simplest inbreeding situation, with a simple Mendelian trait.
Step 0: Let's start with heterozygotes, Aa x Aa.
Step 1: Their offspring will be a collection of AA, Aa, Aa, aa, in roughly those ratios assuming equal survival.
Step 2: Now we breed siblings. If we happen to breed Aa x Aa, we are back to the original starting point. Return to Step 1.
If we happen to breed AA x AA or aa x aa, and then continue to breed those offspring with their siblings, all of the following generations will be homozygous unless there is a mutation. Stop here, you have homozygotes.
If we happen to breed AA x Aa or Aa x aa, the next offspring will match those parents: AA, Aa or Aa, aa. If we breed Aa x Aa, return to Step 1. If we breed AA x AA or aa x aa, stop here, you have homozygotes.
Of course, theoretically you could end up with heterozygotes doing this forever, but the odds get lower and lower the more generations you breed.
For selecting a particular mutation, it is often simpler to genotype for presence of an allele rather than zygosity (i.e., just use simple RT-PCR). For example, if you want to eventually produce aa, you can test for the 'a' allele, and not breed any offspring that do not have any 'a', therefore they are AA. If you do this, you will always eventually end in a path with aa homozygotes.
If you have an easily produced model organism, you can also do this simply by phenotype (depending on the trait): once a strain settles into a consistent phenotype, you know you have a homozygote of one type or the other. If your model organism is easy to produce, it's okay if some don't end up carrying the mutation.
For selecting a random mutation, the same is true, except you have many genes instead of just gene "A". Any random mutation in just one individual early in the breeding could potentially be fixed in the inbred population somewhere down the line, following the steps above. If you breed BB x Bb, where Bb has a random mutation, you have a chance to breed a Bb x Bb pair in the next generation, which starts you at Step 0 for that particular random mutation.