Animals like Sponges and Hydra and fungi like Yeast and Mushroom can reproduce both asexually and sexually. However, they only reproduce sexually if the 'circumstances' are 'bad'. Is there a specific reason for that?
Here is a quite intuitive and straight forward possible explanation:
The key is to rephrase "if the conditions are bad" into "if the focal individual don't fit its environment" or "if the genome of the focal individual is not fit" or "if the focal individual is not fit".
If an individual has a high fitness, it has better to create clones which will be fit as well.
If an individual is not fit, it has better to take the risk to reproduce sexually hoping that some of the offsprings will be fit thanks random recombinations and mutations (there are usually much more mutations during meiosis than during mitosis).
Sex allocation and evolution of sex is a very broad subject. Your question is linked with discussion on sex allocation. You might be interested by these books:
But you might simply have a first look through wikipedia sex allocation, Muller's ratchet or have a look to the diversity of reproduction. You could also have a look to specialized literature, the work of S. Otto for example. Usually introductory books on evolutionary biology contain a chapter on the evolution and diversity of sexual reproduction.