Deliberately blocking out a small portion of the incoming solar radiation would cool the climate. One such approach would be injecting SO$_2$ into the stratosphere, which would produce sulfate aerosols that would remain in the atmosphere for 1–3 years, reflecting part of the incoming shortwave radiation. This would not affect the climate the same way as increased greenhouse gas (GHG) concentrations, leading to residual differences when a GHG increase is offset by stratospheric sulfate geoengineering. Many climate model simulations of geoengineering have used a uniform reduction of the incoming solar radiation as a proxy for stratospheric aerosols, both because many models are not designed to adequately capture relevant stratospheric aerosol processes, and because a solar reduction has often been assumed to capture the most important differences between how stratospheric aerosols and GHG would affect the climate. Here we show that dimming the sun does not produce the same surface climate effects as simulating aerosols in the stratosphere. By more closely matching the spatial pattern of solar reduction to that of the aerosols, some improvements in this idealized representation are possible, with further improvements if the stratospheric heating produced by the aerosols is included. This is relevant both for our understanding of the physical mechanisms driving the changes observed in stratospheric sulfate geoengineering simulations, and in terms of the relevance of impact assessments that use a uniform solar dimming.