Relatedness of ants within a colony vary from species to species (as different species may use different mating systems) and from colony to colony. I will consider a basic (simplified) model of a honeybee colony and describe it a little bit to try to address your question.
Simplified Model of an ant colony
There is quite a lot of diversity in mating systems among ant lineages. See for example the amazing case described in Fournier et al. 2005 (which is quickly summarized at the end of this answer). I will assume a general and simplified model below.
In general, a queen can have several mates. The queen is storing the sperm of these males in her spermatheca. As you noted, the queen is diploid while the males are haploids. The queen will use the stored sperm for all of her life. Each egg that is fertilized will give rise to a worker. Although, a worker does not reproduce (in our simplified model), a worker is still considered a female for morphological reasons. At some point in the life of the colony, some larvea that could have become workers are fed with royal jelly and they will then develop into queen. These queens will fly away to mate with multiple males and make up a new colony. The queen also produce males. The males are haploids. The queen only uses her own gametes to make up a male.
Relatedness
In our simplified colony, a male has no father and has the same mother than any other male and any other worker. Two workers may or may not have the same father. Let's imagine to simplify the story that the queen mated with a single male so that all the workers have the same parents (they are sisters).
- queen-worker: 1/2
- worker-worker: 1/2
- This relatedness decreases with the number of father. At the extreme when there are an infinite number of fathers, the relatedness between two workers drops down to 1/4.
- queen-male: 1/2
- male-male: 1
- male-worker: 1/4
The calculation of relatedness is a little bit problematic when comparing individuals that differ in ploidy (queen - male comparison for example).
Consequences of these relatedness
The fact that the workers are more closely related to their sisters than to their brothers have been suggested to cause a queen-worker conflict for the sex-ratio. Indeed if workers are more closely related to their sisters than to their brothers we might think that they would tend to bias the sex-ratio of the colony toward more sisters to a degree that would not be optimal for the queen. The degree of relatedness between workers depends on the number of fathers. There more fathers there are, the lower is the relatedness among workers and the lower is the incentive to bias the sex-ratio. As a consequence, while the queen would wish a sex-ratio of 0.5, the workers will deviate this sex-ratio toward more workers with a strength that is decreasing with the number of father.
Personally, I have some trouble with these predictions (that I won't express here) but empirical evidences (Chapuisat et al. 1997; Passera et al. 2001) support that the sex-ratio differ depending on the number of fathers (and therefore on the worker - worker relatedness).
