A look into emotions with neuroimaging

Research Unit Director
Mind, Brain Imaging, and
Canada Research Chair
University of Ottawa
The Michael Smith Chair
ELJB-CIHR, Royal Ottawa
Health Care Group

A look into emotions with neuroimaging

by G. Northoff, Canada

Emotions are fundamental to our life and are largely altered in many psychiatric disorders, such as depression. Recent imaging studies investigated various types of emotions, such as anger, fear, sadness, disgust, and happiness, aiming to “localize” them in specific regions of the brain. These studies reveal that many regions—including the amygdala, insula, ventro- and dorsolateral prefrontal cortex, ventro- and dorsomedial prefrontal cortex, periaqueductal gray, and anterior cingulate cortex—are implicated in various types of emotions, suggesting that emotions are mediated by different neural networks rather than specific regions. Complicating matters further, the brain’s spontaneous or intrinsic activity, eg, resting-state activity, is also closely related to emotion processing. Recent studies demonstrate that the level of resting-state activity may be modulated by preceding emotions, suggesting that these emotions are somehow encoded in a yet unclear way into the neural patterns of the brain’s intrinsic activity. Thus, the neural activity we observe when experiencing emotions may be the result of the integration of extrinsic stimuli and intrinsic activity. This is highly relevant in depression, where the brain’s intrinsic activity is abnormally imbalanced, with resting-state hyperactivity in medial regions and resting-state hypoactivity in lateral regions.

Medicographia. 2013;35:281-286 (see French abstract on page 286)

Emotions and the regions of the brain

A number of imaging studies using functional magnetic resonance imaging have been conducted over the last 10 years both in healthy and depressed subjects. Focusing on the main findings in the main regions of the brain in healthy subjects and the implications for depression, this general overview of the numerous imaging studies on emotion will first take a brief look at the methodology involved.

_ Emotion paradigms
Imaging studies on emotion apply different kinds of paradigms. The bulk of the studies use visual stimulation. Subjects view emotional pictures—eg, faces that are sad, happy, angry, etc—or they watch videos of emotional scenes. Other studies apply auditory stimulation as well, using emotional tones, for example. In the study design, it is also important to consider the task associated with the respective emotional stimulus. Subjects may be asked to merely perceive the emotional stimuli without much self-involvement. Alternatively, they may be required to imagine themselves in that particular scene and to experience the respective emotion.

Besides mere perception and actual experience of emotional stimuli, other studies require a judgment to be made, where subjects have to judge the emotional stimulus as positive or negative, for example, either during or immediately following its presentation. Finally, subjects may be asked to recall and retrieve specific emotional experiences from their own life. This introduces a strong memory component into the design. These so-called task-related effects are important to consider since they may confound and mix with the neural effects related to the emotional stimulus effect, the stimulus-related effects. For instance, it has been shown that task-related effects like judgment are associated with the lateral prefrontal cortex, while mere perception and actual experience of the same emotional stimulus yield neural activity in the medial prefrontal cortex. Hence, we have to distinguish between task and stimulus-related effects.

_ Brain regions involved in processing emotion
Various regions of the brain are implicated in emotion processing (see meta-analyses1-3). One core region is the amygdala, a subcortical region that lies anterior to the hippocampus. The amygdala has been shown to be involved in emotion processing in both animal and human studies, and has therefore often been considered “the emotion region” of the brain. More specifically, emotions yielding activity changes in the amygdala include fear, disgust, and anger, ie, negative emotions. Another subcortical region especially implicated in the processing of fear is the periaqueductal gray (PAG). The PAG is a convergence, or node station, for the confluence of interoceptive stimuli from the body, exteroceptive sensory stimuli from the sensory modalities, and motor stimuli for generating movement and action. This makes it perfectly suitable to be involved in emotion. Fear and anger have been especially associated with neural activity in the PAG.

The hippocampus, lying posterior to the amygdala, has also been implicated in emotion processing. The entorhinal cortex, especially, as part of the hippocampal complex has been shown to be recruited during disgust, sadness, anger, and fear. This is important as the hippocampus is a central focus in depression, in the context of stress and cortisol-related changes as an important part of the pathophysiology. How the stressrelated hippocampal changes in depression are related to the abnormal emotion processing in this and other regions, however, remains unclear.

Another region centrally implicated in emotion processing is the insula. The insula lies on the outer surface of the brain and, like the amygdala, receives both interoceptive, eg, vegetative, input from the body and exteroceptive input from the various sensory modalities. Such convergence seems to make the insula an ideal candidate for emotion processing since emotions are supposed to result from the integration between interoceptive and exteroceptive stimuli. One emotion prominently shown to consistently activate the insula is disgust. Several studies on disgust have been conducted and demonstrated strong insula involvement. This has often led to the assumption that the insula may be specific to the processing of disgust. However, the insula has been shown to be implicated in other emotions as well, such as anger and fear, in the same way that disgust also recruits other regions like the occipital cortex, the amygdala, and the lateral prefrontal cortex.

Another region involved in emotion processing is the orbitofrontal cortex, to which the insula sends many projections. As are the insula and the amygdala, the orbitofrontal cortex too has been associated with different emotions. Most prominent among them is anger, but fear and disgust also seem to recruit this region. The orbitofrontal cortex is closely related to the medial prefrontal cortex, which includes the ventro and dorsomedial prefrontal cortex (vmPFC and dmPFC, respectively). The vmPFC and dmPFC have both been associated with sadness, fear, and happiness. The dmPFC may be particularly involved in reflection upon the emotional experience, as for instance during evaluation or judgment, while the vmPFC may be more involved in the experience or perception of the emotion itself.

Turning laterally, we encounter the dorso- and ventrolateral prefrontal cortex (dlPFC and vlPFC, respectively). Both regions have been associated with emotion processing in general. Negative emotions may be more associated with the left vlPFC and dlPFC, while positive emotions are supposed to involve the right side. Moreover, the dlPFC, especially, has been associated with more cognitive aspects of emotion processing, such as cognitive control, executive attention, and evaluation/judgment of emotions. The vlPFC is especially recruited when showing emotional faces, possibly related to the involvement of this region in face processing, particularly where one’s own face is concerned.

Closely related is the cingulate cortex. The cingulate cortex comprises the sub/pregenual and supragenual anterior cingulate cortex (PACC and SACC, respectively) and the posterior cingulate cortex (PCC). The PACC has been associated with sadness in particular, but also with other emotions like anger and disgust. The PACC receives direct interoceptive and exteroceptive input from the insula and amygdala, while the SACC interacts with the lateral prefrontal cortex. Correspondingly, the SACC is often associated with the cognitive control of and executive attention to emotions in general. Finally, the PCC, lying posterior, is closely connected to the hippocampal complex and is therefore involved in memory processing, especially the retrieval of episodic or autobiographical memories. The PCC has also been implicated in a range of emotions, including anger, fear, and sadness.

Other regions implicated in emotion processing include the visual, or occipital, cortex and the temporal cortex. While predominantly accounting for visual processing, the occipital cortex has often been demonstrated to show heightened activity during different emotions, especially negative ones such as fear, anxiety, sadness, and disgust. This cannot be due to the type of stimuli since the purely visual processing aspects are usually cancelled out by comparing visual emotional stimuli with visual nonemotional stimuli. Hence, it seems that an emotional component enhances neural activity in the occipital cortex during visual processing. The occipital cortex is strongly implicated, especially in studies that require subjects to imagine specific emotions, possibly related to the fact that imagery is often visual. Moreover, emotional pictures often also elicit strong activity changes in the occipital cortex, as distinguished from films or faces, for instance.

_ Emotions are mediated by neural networks rather than specific regions of the brain
What do these findings tell us about the relationship between emotion and the brain? Confusing as these findings are, they indicate that one particular emotion is not associated with one or two particular regions in the brain. For instance, almost all studies demonstrate that specific emotions, such as disgust, fear, and anger, do not recruit one particular region, but many as indicated above. As each emotion recruits multiple regions, emotions seem to be mediated by neural networks rather than specific regions. One may thus want to speak of a network-rather than a region-based approach to emotion.

_ Brain regions are associated with specific processes rather than specific emotions
Moreover, there is no region that is involved only in one particular emotion. Instead, each region seems to be implicated in several emotions. The same region may make different contributions to the different emotions whose processing it mediates. What exactly these contributions are, though, we currently do not know. Thus, it cannot be said that one region’s neural activity and processing is specified for a particular emotion, or more generally put, a specific emotional content. Instead, the regions seem to mediate specific processes, with these processes being implicated in different emotions and their respective contents. One may thus want to speak of a process-based rather than content-based approach to the regions.

_ What is the relevance to depression?
One may now raise the question as to why and how all that is relevant to depression. The various forms of depression, such as major depressive disorder or bipolar depression, show changes in almost all of these regions. For instance, studies in major depressive disorder demonstrated abnormal activity during emotional stimulation in the PACC, SACC, insula, dlPFC, and vlPFC. These regions are also implicated in bipolar depression, though possibly in different ways. What the findings suggest is that no one particular region has abnormal activity in depression; rather, there is an imbalance in activity between regions. Hence, the whole network is altered, apparently undergoing changes that imbalance and shift the neural activity distribution across different regions. How that works in detail, though, remains unclear.

This suggests that in depression, rather than alteration of one particular emotion, all emotions are altered in an abnormal way. The neural network seems to be imbalanced, which in turn may lead to abnormal processing of the various emotional contents. As mentioned above, however, we do not understand the exact neural processes mediated by the various regions and neural networks, making it difficult to determine the exact pathophysiological mechanisms in depression.

Emotions and the brain’s intrinsic activity

This overview has thus far focused mainly on neural activity related to particular stimuli, eg, emotional stimuli. This is described as stimulus-induced activity that describes how stimuli extrinsic to the brain, eg, from the environment (or the body), yield neural activity changes in the brain. Such extrinsic stimulus- induced activity originating from stimuli outside the brain must be distinguished from neural activity originating from the brain itself and thus intrinsic to it. Such intrinsic activity is often described as spontaneous activity or in an operationalized form as resting-state activity signaling the absence of specific extrinsic stimuli.

_ Historical “extrinsic view” and “intrinsic view” of the brain
While the notion of intrinsic activity in the brain has been around for almost 100 years, it has recently come to the foreground again, especially in brain imaging. What is the brain and how does it operate? This was already the subject of controversial discussion in the early days of neuroscience at the beginning of the 20th century. One view of the brain, favored by the British neurologist Sir Charles Sherrington (1857-1952), assumed the brain and the spinal cord to be primarily reflexive. Reflexive means that the brain reacts in predefined and automatic ways to stimuli. Thus, the stimuli from outside the brain, originating extrinsically in either the body or environment, are assumed to determine completely and exclusively the subsequent neural activity. The resulting stimulus-induced activity, more generally any neural activity in the brain, is consecutively traced back to the extrinsic stimuli. This may be considered an “extrinsic view” of the brain (Figure 1A, page 284). For every view there is an opposing view, however. An alternative view was already suggested by one of Sherrington’s students, Thomas Graham Brown. In contrast to his teacher, he suggested that the brain’s activity, ie, in the spinal cord and brain stem, is not primarily driven by extrinsic stimuli from outside the brain, ie, the body and environment. Instead, the spinal cord and brain stem show spontaneous activity originating intrinsically within themselves. Other subsequent neuroscientists such as Karl Lashley, Kurt Goldstein, and Wolfgang Koehler followed Brown’s line of thought and assumed the brain to show intrinsic activity. This may be considered an “intrinsic view” of the brain.

Figure 1
Figure 1. Two views of the brain: the brain’s
neural activity as purely determined by
extrinsic stimuli (A) and by both the brain’s
intrinsic activity and the extrinsic stimuli
from the environment (B).

The image on the left in both figures represents
stimuli from the environment, while the brain in blue in
the middle represents the brain. The purple line in the
brain in (B) symbolizes the brain’s intrinsic activity—
its resting-state activity—that as such remains
independent of any extrinsic stimuli from the environment.
The bar diagram on the far right on both
panels stands for the neural activity we observe once
the person and his/ her brain encounter the stimuli
from the environment.
(A) In the case of a purely extrinsic view of the brain,
the observed stimulus-induced activity is exclusively
and completely determined by the stimulus itself; the
brain is passive and functions more or less like an automatic
and reflex-like machine. Any neural activity in
the brain can be traced back to stimuli and their interactions
with each other, ie, stimulus-stimulus interaction.
The brain itself has thus no say in what happens
in the brain. (B) This is different once one assumes
intrinsic activity in the brain itself, ie, resting state.
In this case, the observed stimulus-induced activity
results from the interaction between brain and stimuli
amounting to rest-stimulus interaction. The brain itself
thus has a say in what happens in the brain during its
encounter with extrinsic stimuli from the environment.
Colosseum. © SuperStock/Corbis.
“Blue brain.” © Courtesy of Joseph McNally
Photography/National Geographic.

_ Stimulus-induced activity: interaction between intrinsic activity and extrinsic stimulus
The assumption of intrinsic activity generated inside the brain itself has major implications for how we conceive stimulus induced activity. What we as observers describe as stimulus- induced activity and usually associate with the stimulus itself must then be regarded as the hybrid result of a specific interaction between the brain’s intrinsic activity and the extrinsic stimulus.

Stimulus-induced activity and any neural activity in the brain must consecutively be traced back to a double input that originates in both the brain’s intrinsic activity and the body’s and the environment’s extrinsic stimuli (Figure 1B).

_ Present status: extrinsic vs intrinsic view
Following this rather abbreviated history of neuroscience, let’s look at the present. The dichotomy between intrinsic and extrinsic views of the brain is still just as controversial and has most recently resurfaced, especially in functional brain imaging (see examples4,5). Let’s start with the extrinsic view.

Many domains of neuroscience, ranging from cellular over regional to behavioral, rely on experimental application of specific stimuli and tasks to probe neural activity. By comparing different stimuli and tasks, the resulting differences in neural activity are associated with the respective stimuli or tasks. Consequently, the experimental requirements may prime and draw us toward the extrinsic view. The extrinsic view has been predominant in behaviorism which, according to authors like Jaak Panksepp,6 finds its continuation in the cognitive neuroscience of our days.

However, the extrinsic view of the brain has recently been challenged again on several grounds. Even in the resting state, ie, in the absence of any (specific) extrinsic stimuli, the brain shows a rather high degree of metabolism, consuming, for instance, about 20% of the body’s overall energy budget (and oxygen fraction).4,5,7-9

Using functional imaging, this high metabolism has been especially observed in a particular set of regions—the default mode network (DMN)—which includes various anterior and posterior cortical midline structures as well as the bilateral posterior parietal cortex.4,5,9-11 The high degree of metabolism is indicative of continuously ongoing high levels of neural activity even in the absence of (specific) extrinsic stimuli, ie, in the resting state of the DMN. However, other regions outside the DMN also show spontaneous neural activity, independent of any extrinsic stimuli. This has been demonstrated in the auditory and visual cortex, thalamus, hippocampus, olfactory cortex, cortical midline regions, prefrontal cortex, motor cortex, and other subcortical regions, such as the brain stem and midbrain.8,9 The metabolic and neuronal signs of intrinsic activity are further complemented by behavioral evidence; spontaneous behavior, such as seeking or behavioral activation, can be observed even in the absence of extrinsic stimuli.6 Which view holds—the intrinsic or the extrinsic one? Rather than choosing one view and dismissing the other, the brain itself may force us to reconcile both views. Any neural activity in the brain may be assumed to result from the interaction between the brain’s intrinsic activity and the extrinsic stimuli from the body and environment. In place of intrinsic and extrinsic views, we may need to investigate how intrinsic activity and extrinsic stimuli interact with each other in order to understand the brain’s neural activity.

_ Relevance to processing of emotions
Why is all that relevant for the neural processing of emotions? It’s relevant because emotions may result from the interplay between intrinsic activity and extrinsic stimuli. Most recently, single studies demonstrated that there is direct interaction between extrinsically induced emotion and the brain’s intrinsic activity.

Focusing on emotions, a recent study12 investigated the impact of fearful, joyful, and neutral movie clips (50-s presentation) on subsequent resting-state activity (90-s period with eyes closed). After the resting-state period, participants were asked about their thoughts, revealing that personal relevant issues in the subjects’ thoughts were increased after neutral movies, increased, but less so, after joyful movies, and significantly decreased after fearful movies. These results show a clear behavioral effect or better psychological effect of emotions on thought content in subsequent resting-state periods; fearful movies seem to leave the strongest traces on thought content of subsequent resting states.

Resting-state neuronal activity in subcortical regions (pallidum, anterior thalamus, and hypothalamus) was higher after viewing fearful movies than after viewing neutral movies (resting- state activity greater after fearful stimulation than after neutral stimulation). Most interestingly, the reverse comparison (resting-state activity greater after neutral stimulation than after fearful stimulation) revealed more pronounced signal changes in various regions of the DMN (vmPFC, PACC, dmPFC, superior temporal gyrus) (see analogous overlap between emotion processing and the DMN13,14).

This means that the inclusion of fearful emotions in the preceding movie had a clear effect on the level of subsequent resting- state activity. The stronger resting-state effects of the preceding emotional movies are further confirmed by the more delayed recovery from the signal changes during the resting state period (90 s) after emotional movies.

Taken together, it seems that emotions are closely related to the resting-state activity. Studies show that emotions affect the level of activity in the resting state, thus indicating what can be described as stimulus-rest interaction.15 Conversely, one may also expect the resting-state activity to have an impact on emotions during presentation of particular stimuli. While such rest-stimulus interaction has been described in perception and cognitive functions, it remains to be demonstrated for emotion. The degree and intensity of emotions in psychological regard and the recruitment of particular regions and networks may then depend not only on the extrinsic stimulus itself and its associated emotional content, but also on the characteristics of the brain’s intrinsic activity.

_ Relevance of resting-state activity to depression
Why is all that relevant to depression? To start with, human and animal studies in depression demonstrate abnormal resting- state activity in various regions. For instance, the resting state activity in the PACC seems to be abnormally high in major depressive disorder, while that in the dlPFC is abnormally low in these patients (see overview16). Given the above-described findings, it seems certain that such resting-state abnormalities must have an impact on subsequent emotion processing in the various regions described above.

One may consequently hypothesize that some of the psychological and neural abnormalities observed in emotion processing in depression may be related to yet-to-be-specified abnormalities in intrinsic activity. In addition to providing insight into the pathophysiology of depression, this may lead to opportunities for more specific and effective therapeutic intervention. For instance, if we understand the biochemical mechanisms underlying the resting-state abnormalities in depression, we may be able to design drugs that specifically target those mechanisms and may thereby normalize subsequent emotion processing. That, however, is a scenario of the future, hopefully the near future.

_ Acknowledgments: My work is financially supported by CIHR, EJLBCIHR, and ISAN/HDRF.

1. Fitzgerald PB, Laird AR,Maller J, Daskalakis ZJ. Ameta-analytic study of changes in brain activation in depression. Hum Brain Mapp. 2008;29(6):683-695.
2. Phan KL, Wager T, Taylor SF, Liberzon I. Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI. Neuroimage. 2002;16(2):331-348.
3. Lindquist KA, Wager TD, Kober H, Bliss-Moreau E, Barrett LF. The brain basis of emotion: a meta-analytic review. Behav Brain Sci. 2012;35(3):121-143.
4. Raichle ME. A brief history of human brain mapping. Trends Neurosci. 2009;32 (2):118-126.
5. Raichle ME. The brain’s dark energy. Sci Am. 2010;302(3):44-49.
6. Panksepp J. Affective neuroscience. Oxford, NY: Oxford University Press; 1998.
7. Shulman RG, Hyder F, Rothman DL. Baseline brain energy supports the state of consciousness. Proc Natl Acad Sci U S A. 2009;106(27):11096-11101.
8. Buzsaki G. Rhythms of the brain. Oxford, NY: Oxford University Press; 2006.
9. Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL. A default mode of brain function. Proc Natl Acad Sci U S A. 2001;98(2):676-682.
10. Raichle ME, Gusnard DA. Intrinsic brain activity sets the stage for expression of motivated behavior. J Comp Neurol. 2005;493(1):167-176.
11. Buckner RL, Andrews-Hanna JR, Schacter DL. The brain’s default network: anatomy, function, and relevance to disease. Ann N Y Acad Sci. 2008;1124:1-38.
12. Eryilmaz H, Van De Ville D, Schwartz S, Vuilleumier P. Impact of transient emotions on functional connectivity during subsequent resting state: a wavelet correlation approach. Neuroimage. 2011;54(3):2481-2491.
13. Wiebking C, de Greck M, Duncan NW, Heinzel A, Tempelmann C, Northoff G. Are emotions associated with activity during rest or interoception? An exploratory fMRI study in healthy subjects. Neurosci Lett. 2011;491(1):87-92.
14. Pitroda S, Angstadt M, McCloskey MS, Coccaro EF, Phan KL. Emotional experience modulates brain activity during fixation periods between tasks. Neurosci Lett. 2008;443(2):72-76.
15. Northoff G, Qin P, Nakao T. Rest-stimulus interaction in the brain: a review. Trends Neurosci. 2010;33(6):277-284.
16. Alcaro A, Panksepp J, Witczak J, Hayes DJ, Northoff G. Is subcortical-cortical midline activity in depressionmediated by glutamate and GABA? A cross-species translational approach. Neurosci Biobehav Rev. 2010;34(4):592-605.