Einstein's theory of general relativity, which contains a universal value of the Planck mass, has been so far successfully invoked to explain gravitational dynamics from sub-millimeter scales to the scale of the cosmological horizon. However, one may envisage that in alternative theories of gravity, the effective value of the Planck mass (or Newton's constant), which quantifies the coupling of matter to metric perturbations, can run on the cosmological horizon scale. In this letter, we study the consequences of a glitch in the Planck mass from sub-horizon to super-horizon scales. We first give three examples of models that naturally exhibit this feature, and then show that current cosmological observations severely constrain this glitch to less than 1.2%. This is the strongest constraint to date, on natural (i.e. non-fine-tuned) deviations from Einstein gravity on the cosmological horizon scale.