I tested this out with my friends, and I find that after they hold their breath and can't hold it anymore, they exhale air, instead of inhaling air.
Interestingly, they all try to inhale in as much air as possible before starting to hold their breath. When I told them to exhale as much air as possible before starting to hold their breath, they inhaled air after they can't hold it anymore.
It's understandable that when one exhales then holds his breath, he needs air and thus he inhales afterwards. But when one inhales then holds his breath, shouldn't he inhale again after "using up all that air he inhaled before holding hi breath"?