The formation of cosmic structure (clusters of galaxies) locally leads to large deviations from the mean matter density of the Universe. Since the effect of these fluctuations does not average to zero, they can lead to a modification of the average expansion law. This phenomenon is known as "back-reaction". The importance of the effect depends on the amount of fluctuations in the matter distribution and is expected to be small in our Universe. It turns out that one of the reasons for this is the fact that fluctuations are damped in the early Universe when radiation was the dominant form of energy. Consequently, if the temperature of the Universe had been lower, we would expect to see stronger fluctuations in the matter distribution and hence more back-reaction. We conduct a numerical study to better understand this connection and find that, contrary to this naive expectation, back-reaction remains a small effect independent of the temperature. The reason is that the effect of fluctuations saturates soon after structures have reached the strongly non-linear regime.