Along with the increased interest in and volume of social cognition research. there has been higher awareness of a lack of agreement on the concepts and taxonomy used to study social processes. Two central concepts in the field, empathy and Theory of Mind (ToM), have been identified as overlapping umbrella terms for different processes of limited convergence. Here, we review and integrate evidence of brain activation, brain organization, and behavior into a coherent model of social-cognitive processes. We start with a meta-analytic clustering of neuroimaging data across different social-cognitive tasks. Results show that understanding others' mental states can be described by a multilevel model of hierarchical structure, similar to models in intelligence and personality research. A higher level describes more broad and abstract classes of functioning, whereas a lower one explains how functions are applied to concrete contexts given by particular stimulus and task formats. Specifically, the higher level of our model suggests 3 groups of neurocognitive processes: (a) predominantly cognitive processes, which are engaged when mentalizing requires self-generated cognition decoupled from the physical world; (b) more affective processes, which are engaged when we witness emotions in others based on shared emotional, motor, and somatosensory representations; (c) combined processes, which engage cognitive and affective functions in parallel. We discuss how these processes are explained by an underlying principal gradient of structural brain organization. Finally, we validate the model by a review of empathy and ToM task interrelations found in behavioral studies.