An important question in polariton chemistry is whether reacting molecules are in thermal equilibrium with their surroundings. If not, can experimental changes observed in reaction rates of molecules in a cavity (even without optical pumping) be attributed to a higher/lower temperature inside the cavity? In this work, we address this question by computing temperature differences between reacting molecules inside a cavity and the air outside. We find this temperature difference to be negligible for most reactions. On the other hand, for phase transitions inside cavities, as the temperature of the material is actively maintained by a heating/cooling source in experiments, we show cavities can modify observed transition temperatures when mirrors and cavity windows are ideal (non-absorbing); however, this modification vanishes when real mirrors and windows are used. Finally, we find substantial differences in blackbody spectral energy density between free space and infrared cavities, which reveal resonance effects and could potentially play a role in explaining changes in chemical reactivity in the dark.