We reveal an intriguing manifestation of topology, which appears in the depletion rate of topological states of matter in response to an external drive. This phenomenon is presented by analyzing the response of a generic two-dimensional (2D) Chern insulator subjected to a circular time-periodic perturbation. Because of the system's chiral nature, the depletion rate is shown to depend on the orientation of the circular shake; taking the difference between the rates obtained from two opposite orientations of the drive, and integrating over a proper drive-frequency range, provides a direct measure of the topological Chern number (n) of the populated band: This "differential integrated rate" is directly related to the strength of the driving field through the quantized coefficient eta(0) = n/h(2), where h = 2 pi h is Planck's constant. Contrary to the integer quantum Hall effect, this quantized response is found to be nonlinear with respect to the strength of the driving field, and it explicitly involves interband transitions. We investigate the possibility of probing this phenomenon in ultracold gases and highlight the crucial role played by edge states in this effect. We extend our results to 3D lattices, establishing a link between depletion rates and the nonlinear photogalvanic effect predicted for Weyl semimetals. The quantized circular dichroism revealed in this work designates depletion rate measurements as a universal probe for topological order in quantum matter.