The young adolescent lives for several years with a sexually mature body and a brain activated by emotions, but with a relatively immature behavioral and emotional regulatory system. This “disconnect” predicts the risk of a panoply of behavioral and emotional problems. (Holzer, Halfon and Thoua 2011, p. 7)
Research that aims to elucidate the neural bases of adolescence from the structural as well as the functional perspective emerged during the 1990s and has become authoritative over the past decade [1]. In the field of developmental neurosciences, it is now accepted that after the tenth year of life, the brain continues to mature in a heterogeneous temporality, the prefrontal cortex reaching an adult-like stage of development only toward the age of 25 (Giedd et al. 1999, Sowell et al. 1999). The use of fMRI techniques aims to link the structural characteristics of the adolescent brain to behaviors that evolutionary psychology has designated as being constitutive of this stage of life. In other words, the specific configuration of the adolescent brain leads to a lack of emotional control that permits so-called risk behaviors which allow young people to gain the independence necessary to live as an adult and at the same time place them and their companions in dangerous situations (Spear 2000). This is the context in which research using fMRI to measure adolescent brain response to emotional-type visual stimulation is carried out.
To my knowledge, the report on the first study using fMRI to test adolescent responses to photographs of facial expressions of emotion was published in February 1999 (Baird et al. 1999) by Deborah Yurgelun-Todd’s research team at the cognitive and clinical neuroimaging center of Harvard University’s McLean Hospital. Yurgelun-Todd is a psychologist who specializes in research on the neuronal correlatives of the risks of developing mental illness and drug addiction.
Inspired primarily by Joseph LeDoux’s work on the neurobiology of the emotions using animal subjects (1992, 1994) and by the comparative studies of healthy and injured adult brains coauthored by Ralph Adolphs, Daniel Tranel, Hanna Damasio and Antonio Damasio (1994, 1995, Damasio 1994), Yurgelun-Todd’s team measured the activation of the amygdala in young subjects tasked to recognize the facial expression of fear. According to the authors, the purpose of this preliminary research was above all to validate the use of this experimental procedure with adolescent subjects, based on a synthesis of the above-mentioned research studies. The use of the Pictures of Facial Affect (POFA) developed by Paul Ekman and Wallace Friesen (1976) as the sole source of stimuli represented a stable element whose scientific legitimacy needed no further problematization. Indeed, the neurosciences of the emotions adopted this type of experimental tool during the 1990s as a static reference for emotion used to produce brain activation that could be visualized by fMRI (McClain 2004, Leys 2010) [2]. In the opinion of the neuroscientists, the use of such procedures in the study of adolescence allowed them to demonstrate a significant difference between adolescents and adults in brain activation in response to emotional stimulation. Thus the characteristic brain response of young people to emotional information could lead to risk behaviors (Dahl 2008, Steinberg 2008).
Some authors — sociologists, historians, but also neuroscientists — have produced critical reflections on the adolescent brain. While most of these studies highlight the fact that theories of cerebral adolescence tend to naturalize social and cultural issues related to youth (eg. Bessant 2008, Males 2009, Choudhury 2010, Payne 2011), they hardly problematize the question of gender which, in my view, is central to all levels of the construction of the teenage brain [3]. Moreover, the problematization of modern technological experimentation — the relationship between theory, visualization, observation and transcription in the construction of scientific facts — is a classic theme in studies on scientific research (Latour and Woolgar [1979] 1988, Hacking [1983] 1989, Daston and Galison 2007), including studies on functional imaging research (eg. Beaulieu 2002, Dumit 2004). The latter, however, have focused mainly on the scan-image and the interrogation of various experimental tools that aim to produce a cerebral activation of which the scan is the visible trace.
This paper will first address the use of photography as a tool to capture and represent facial expressions of the emotions, and then it will address the scan-image of the emotional adolescent brain. More specifically, following a cultural history and social science approach and using gender studies as a privileged analytical tool [4], I intend to investigate the process whereby the cognitive neurosciences, using photographic portraits of actors miming emotions, have produced valid stimuli that allow the fMRI visualization of a cerebral differentiation that distinguishes between adolescents and adults on the one hand, and on the other, between young girls and young boys, in the management of emotional information. In effect, the emotional adolescent brain is constructed according to two interlocking hierarchies : if age difference is taken to be the foundational hierarchy, sex difference is both a driving and a moderating factor within that hierarchy. I begin with the history of Ekman and Friesen’s set of Pictures of Facial Affect, setting aside the object of the proof, i.e. the universality of the basic expressions, and looking at the support and tool of the proof, i.e. photography. The purpose of the exercise is to review experiments in the neurosciences of the emotions concerning adolescents. By concentrating on one dimension of the research that was hardly problematized by the researchers, I hope to demonstrate and interrogate the trajectory of the image, whether photographic or scanned, and its incorporation/production of meaning, particularly in terms of gender, in the construction of an adolescent brain characterized by poor emotional control.
The POFA Set : Capturing, Fixing and Representing Emotions with Photography.
The set of Pictures of Facial Affect that Ekman and Friesen developed in the 1970s consists of 110 black-and-white photographic portraits of Caucasian-type men and women performing the facial expression of the six emotions which the authors consider to be universally recognized and expressed : joy, sadness, fear, anger, disgust and surprise. These portraits, whose function is “simply” to emit or to solicit emotions which are useful in the context of experimental protocols in the neurosciences of the emotions, are nevertheless the product of a history in which the polysemy of images in an iterative relationship between photographs and experts and laymen has produced a univocal cliché.
The historian Ruth Leys points out that when Ekman began studying nonverbal behaviors including facial affect in the 1960s, there was a resurgence of interest in the emotions in the field of psychology following years of neglect, as a result of the work of Ekman’s mentor, the psychologist Silvan Tomkins. Leys explains that Tomkins, inspired by the Darwinian theory of evolution and cybernetic developments, proposed a new approach to the study of emotions. He put forward the idea that there is a limited number of basic emotions which evolution has defined as the organism’s universal response to the environment : fear, anger, distress, disgust, interest, shame, joy, surprise and contentment. Tomkins described these basic emotions as “discrete” and “connected” and as emotional programs located in the subcortical region of the brain, whose behavioral response and physiological expression, especially facial affects, are unique to each emotion (Leys 2010, p. 67-68).
Research on the expression and the interpretation of emotions was nevertheless ongoing all through the first half of the twentieth century, notably using sets of photographs of the facial affect of emotions as material support for experiments (McClain 2004). If the use of photographs as a tool to represent the emotions in the laboratory was not new when Ekman and Friesen published their Pictures of Facial Affect [5], the specificity of their photographic set resides in the fact that its images participate not only in a scientific rhetoric of proof or experimental methodology, but they also and above all constitute a black box (Latour 1995) that stabilizes and preserves the principle of the universality of the emotions. Ekman and Friesen’s photos of the facial expression of emotions have incorporated all the theoretical productions connoting them (Barthes 1961), that is to say, they include all the meanings resulting from the work on emotions conducted by these researchers. Through a process of translation and transcription (Latour and Woolgar [1979] 1988), the creative and interpretive elements of these images seem to have left room only for the reproduction of the “real”, thus allowing the use of a photograph of a facial expression of emotion to signify the emotion itself.
Now I turn to the dynamic process among researchers, discourses, images, actors and observers that participated in the production of these photographs which offer a relationship of identity with what they signify that goes beyond mere analogy.
The brochure that accompanies the set of Pictures of Facial Affect explains that,
With the aid of the best current technology in lighting and photography, more than a dozen persons were photographed repeatedly while attempting to express one of six emotions. Hundreds of photographs were studied over a period of several years to obtain a series which yielded consistent agreement among viewers about the emotion being expressed. […] Posers were trained to contract or relax different facial muscles associated with various facial expressions. Generally, posers were instructed to activate certain muscles rather than to pose a particular emotion. (Ekman and Friesen 1976, p. 1)
This quote shows that the photographs are the result of the coordination of numerous elements whose direct relationship with the emotions is not stated upfront. It also underlines the fact that the univocal recognition of the emotions represented by these photographs depends upon preliminary theoretical and technical achievements that are embodied both in models with the ability to control their facial muscles and in observers who are expected to associate those muscular configurations with emotions. In other words, the validity of these images lies in the established proof (in the authors’ opinion) of the universality of six so-called basic emotions and of the equivalence between the facial actions and the emotions themselves ; the subsequent development of a standardized technique for describing facial affect, i.e. the Facial Affect Scoring Technique (FAST) (Ekman, Friesen and Tomkins 1971) ; the quality of the photographs ; the ability of the actors to activate their face in precise ways ; and the statistical consensus obtained from the observers. The POFA set is the answer to Ekman and Friesen’s intention to reproduce in the laboratory typical expressions of emotion whose scientific reliability can be validated, since they judge expressions of emotion captured in reality to be experimentally problematic due to their contextual contingence (Ekman, Friesen and Ellsworth 1972).
Nevertheless, it seems paradoxical that the primary tool that allowed these researchers to prove the universality of the emotions and develop the FAST consists precisely of photographs of the facial expression of emotion which they judge to be insufficiently reliable. In fact, between 1966 and 1970, Ekman and his colleagues show groups of people in more than twenty countries a series of photos provided by their mentor, Tomkins, according to the following protocol : the subjects are shown photographs of facial affect, one by one, for about ten seconds. These photos represent portraits of Caucasian-type men and women of various ages “spontaneously” playing emotions without reference to any kind of standardization of their expressions. The subjects must then choose from a list of six predefined basic emotions the one emotion that they believe corresponds to the image. The only exception is the study carried out with illiterate populations in New Guinea. Being unable to use a system of written lists, the researchers take inspiration from a protocol used with pre-school-age children and tell a story that evokes an emotional situation, with the subject having to choose from three photographs the one most compatible with the situation. Next, the researchers ask certain Papuans to mimic the expressions they would express with their own face if they found themselves in the situations described. The photographs of facial affect thus obtained are submitted to the judgment of American students. The high level of accord in the identification of emotions among the different cultures leads Ekman and his colleagues to the conclusion of the universality of the six basic emotions, even though photos of the facial affect of fear were rarely interpreted as expected by the subjects from New Guinea who failed to discriminate between expressions of fear and expressions of surprise. By contrast, those two emotions were reliably distinguished from expressions of anger, sadness, joy and disgust (Ekman 1998). Apparently, this element was not enough to rattle the thesis of the universality and the pan-culturality of the six emotions which the researchers had stipulated as basic. This point is striking when we know that the photos of the facial affect of fear from Ekman and Friesen’s set are widely used in research on the emotions in cognitive psychology and the neurosciences (Leys 2010), particularly in research on the neural bases of adolescent behaviors, as mentioned in the Introduction. The philosopher Ian Hacking (1998) has pointed out that LeDoux’s reference work on the neurosciences of the emotions, titled The Emotional Brain, should rather be titled The Fearful Brain, given the centrality of fear and its correlative states — anxiety, neuroses, phobias, panic or post-traumatic stress — in the author’s conception of the emotions. According to LeDoux, the emotions are complex neuronal systems that have evolved in order to permit survival. It is not a question of conscious feelings but of a cerebral production that comes from a “deeper” level (LeDoux 1998).
With respect to the results of the research that Ekman and his collaborators carried out in New Guinea, Hacking suggests that “one may well wonder whether the three groups, i.e. the American students, the primitive men of New Guinea and the researchers, did not collaborate to produce these phenomena” (1998, p. 11). In other words, he proposes that the protocol used in the experiments that strove to demonstrate the universality of the emotions through their pan-cultural expressions produced a circulation of knowledge and expectations among the researchers, the American subjects and the subjects from New Guinea which contributed to the existence of an understanding in the association of photographic portraits and of terms associated with emotional states. To this three-way production, I would add the role of the photographs which, in my view, are an integral part of the process of constructing the universality of the six basic emotions. In fact, they represent a common but nevertheless interpretable reference that allows the construction of a shared history around the emotions. In this context, the photos become entities that act upon and mediatize relations between the various actors as much by their capacity to refer to elements in space and time as by their power to incorporate the discourses produced upon contact with them (Edwards 2005).
In order to standardize the description of the expression of the emotions which have henceforth been validated as universal, Ekman, Friesen and Tomkins establish the Facial Affect Scoring Technique (FAST) in 1971. This is a method of describing the facial affect of the six emotions in terms of wrinkles, tension or relaxation in specific features of the face and the positions of those features. This analysis is done by dividing the face into three areas : the forehead and brow, the eyes and the bridge of the nose, and the lower face. As the authors explain,
Specifying the appearance of some of the components within a facial area required such lengthy, awkward descriptions, which sometimes even then were insufficient, that it became obvious that the only way to make clear to scorers just what was being described was to imitate the appearance ourselves or show a photographic sample. (Ekman, Friesen and Tomkins 1971, p. 41)
Getting around the difficulty of verbally describing the facial characteristics of each emotion, the FAST materializes in three sets of photographs of men and women performing the features of the basic emotions, each set portraying one area of the face, along with a few verbal instructions regarding the position of the head and the direction of the gaze, the purpose being to reach, through rationalization and abstraction, the elements essential to the representation and the recognition of the basic emotions. As the authors emphasize,
The rationale for using as many different stimulus persons as possible was to test FAST’s ability to ignore differences in facial appearance associated with age, sex, and physiognomy, and differences in lighting and contrast that would occur across pictures of different people. (Ekman, Friesen and Tomkins 1971, p. 43)
In other words, it is a matter of constructing images of the facial expression of the universal emotions consistent with the universality of the expression and the recognition of the basic emotions. As for fear and disgust, Ekman, Friesen and Tomkins (1971) concede that difficulties in the univocal identification of those emotions prevented them from obtaining the hoped-for number of photographs, without, however, explaining or drawing any conclusion from this observation.
To return to the POFA set obtained from actors moving their face according to the indications produced by the FAST, one observes that they proceed more from a mosaic than from a transfer of the “real” onto paper or screen. They are the final result of a particular arrangement of a number of images that have circulated in the hands of researchers, actors and observers according to a tautological relationship between the proof and the tool of the proof, being transformed into a consensual object whose faculty is to provoke a unique meaning — for example, the exclusive function of a photograph of a facial expression of fear is to evoke fear — independent of the age or sex of the subjects represented or of the observers. By considering their utilization in the context of experiments in the cognitive neurosciences, I will now interrogate their part in the construction of a difference between adolescents and adults, but also between young girls and young boys, in the management of emotional information. In other words, I hope to understand how these purportedly universal emotional stimuli participate, through the visual translation of fMRI, in producing specific characteristics, in this case of age and sex.
From Photography to Scan Image of Adolescent Emotions : A Visual Translation that Produces Age and Sex.
Since 1999, the date of the first experiment to use Ekman and Friesen’s set of photographs of facial expressions, complementary studies have been carried out using such stimuli [6] in the field of research devoted to the adolescent brain (for example, Killgore, Oki and Yurgelun-Todd 2001, Monk et al. 2003, Yang et al. 2003, McClure et al. 2004, Killgore and Yurgelun-Todd 2005, 2006, Yurgelun-Todd and Killgore 2006). According to researchers in the cognitive neurosciences, this work has taken part in the establishment of a neurobiological model of adolescence, postulating that young individuals experience the perceptible effects of a gap in development between the limbic and the prefrontal regions of the brain. The former, implicated in emotional reactions, is already developing by the beginning of the second decade under the effect of pubertal hormones, while the latter, devoted to control of the emotions and long-term planning, does not reach full maturity until the age of 25 (Casey et al. 2008, Ernst and Mueller 2008). This cerebral configuration favors impulsive behaviors regulated by emotional reactions rather than by rational reasoning, which suggests a neurodevelopmental explanation for statistics on so-called risk behaviors during adolescence, for example, drunk driving, unprotected sex, fighting, consumption of psychoactive substances, etc. These behaviors, which have become the focus of major social and political concern, were seen in the United States to be responsible for a 200 % increase in morbidity and mortality during this phase of life (Dahl 2004). Statistics also show that such behaviors concern boys more than girls and that, taking all types of risk behavior into account, boys are three times more likely than girls to die as a result of this kind of situation (Kelley et al. 2004, Eaton et al. 2006). According to a neurodevelopmental logic, it is therefore important in fMRI verification to evaluate the treatment of emotional information not only from the perspective of age but also from that of sex.
My purpose here is not to present an exhaustive review of the various neuroscientific studies of the emotions whose experimental protocol is based on photographs of facial affect, but by developing an example that is pertinent from the perspective of its importance in the field, to question the process of translation between the stimulus photographs and the images produced by fMRI, as well as its productivity in constructing knowledge about adolescents’ management of emotional information. That is why I shall focus on the work of Yurgelun-Todd’s research team which has become a touchstone for this topic in science as well as in the media (cf. PBS Frontline 2002, Wallis, Dell and Park 2004, Sender 2008).
The team’s first results concerning the management of emotional information with respect to age and sex were published in the scientific journal NeuroReport in February 2001 (Killgore, Oki and Yurgelun-Todd 2001). It was a matter of using fMRI to measure the activation of the amygdala versus that of the prefrontal cortex in young girls and young boys aged 9-17 upon viewing photographs of the facial expression of fear from Ekman and Friesen’s POFA set. The researchers believe this experimental protocol allowed them to make visible a difference in the activation of the amygdala versus that of the prefrontal cortex, partly in relation to the age and partly in relation to the sex of the participants (Killgore, Oki and Yurgelun-Todd 2001). Referring to these results and to the results of an unpublished study that evaluated the capacity of adolescents, as compared with adults, to recognize the facial affect of fear, Yurgelun-Todd said in an interview,
In general, the males in our studies showed more reaction from that gut region [the amygdala] of the brain, and less frontal or executive reaction. The relationship between the gut response and that executive region was very striking for the males, and somewhat striking for the females, but was not as extreme for the teenage females compared to the teenage males. (PBS Frontline 2002, p. 4)
Given that the adolescent state is defined by how the subjects activate the prefrontal cortex versus the amygdala, this statement suggests that adolescence is inflected in two similar versions but with different degrees of intensity depending on whether the subjects are girls or boys. By contrast, Yurgelun-Todd offers a dichotomous reading when she compares the responses of young subjects and adults, stating that,
The brain is responding differently to the outside world in teenagers compared to adults. And in particular, with emotional information, the teenager’s brain may be responding with more of a gut reaction than an executive or more thinking kind of response. And if that’s the case, then one of the things that you expect is that you’ll have more of an impulsive behavioral response, instead of a necessarily thoughtful or measured kind of response. (PBS Frontline 2002, p. 3)
Here the fMRI technique that is materialized in the scanned image, starting with photos of facial affect and using the bodies of volunteer subjects, has allowed the introduction of a dual hierarchy in the cerebral management of emotional information, first between adults and adolescents, and then between young girls and young boys.
In order to see this process of translation more clearly, let us review the description of this experiment and try to bring out all the elements involved in the trajectory from the photograph of the facial affect of fear to the production of the scanned image, of which the researchers’ reading and interpretation produces a classification as a function of the age and sex of the subjects in terms of their management of emotional information : nine boys and ten girls who have been predefined as adolescent and whose eyesight and mental health have been tested lie in a scanner and look at black-and-white photographs from the 1970s of Caucasian-type men and women performing the facial affect of fear according to the tension and muscle positioning indicators developed by Ekman and Friesen. By measuring the signal that reflects variations in the quantity of oxygen in the blood as a function of brain activity (the blood-oxygen-level-dependent or BOLD signal) during periods of visualizing photos of the facial affect of fear versus periods of visualizing a white circle, fMRI produces a statistical image by means of a system of digital computation and analysis which represents in colors a difference in the activation of the cerebral regions under study as a function of the age and sex of the individual in the scanner, cerebral regions which previous experiments on animals and humans suffering from brain lesions have established as being pertinent to the management of the emotional information of fear (Killgore, Oki and Yurgelun-Todd 2001) [7].
Now it is easier to see that a scanned image, similarly to a photographic image, maintains an indirect and synthetic relationship with the “real” that necessitates the incorporation and the preliminary solidification of numerous theoretical and technical elements. As regards the production/visualization of differences as a function of age and sex, one can observe that binary categorization by age and by sex is present at all stages of the experiment : adult men and women are represented in the stimulus photographs and young girls and young boys are tested. However, it is only through the translation of the results of the experiment into scanned images that the differences linked to age and sex become scientifically relevant in connection with the management of emotional information. It is the chain of recordings and transcriptions (Latour 2006) of the notions of fear, sex and age that allows their reorganization and reinterpretation in the course of the experimental process, retaining all the while traces of preceding stages in the process of constructing knowledge about cerebral adolescence. For example, the use of male and female models in the production of the photographs of facial affect makes no attempt to account for any difference between men and women in connection with the emotions, but on the contrary, aims to guarantee the universality of those images in the reflection of a reality thought to be universal, i.e. the division of the human genus between males and females. Whereas the distribution between as many girls as boys in the sample of adolescents tested is constitutive of the experimental protocol whose purpose is to detect a differentiation in cerebral activation between teenage girls and boys in an emotional situation. The young guinea pigs are expected to be representative of the behavioral differences demonstrated by epidemiological studies based on classifications of age and sex.
In other words, the scanned image contains and communicates not only visual evidence of a cerebral activation triggered by a stimulus which is considered a valid way of measuring the management of emotional information, but also the entire concept of adolescence that has emerged from developmental psychology and epidemiology and to which the neuroscience researchers themselves refer. In defining adolescence primarily by so-called risk behaviors, for which the difference between boys and girls is constitutive of public health policy interests concerning adolescence, the emotional adolescent brain is in fact gendered. Thus one observes that the sexing of the emotional adolescent brain proceeds less from the binary categorization by sex of the actors represented in the stimulus photographs and of the test subjects themselves than from the incorporation by the scanned image of discourses related to gender differences — here, the differential involvement in risk behaviors — which motivate the research.
Developing a set of photographs of facial affect, like mapping the treatment of emotional information among adolescents with fMRI, could be seen as the result of a practical and heuristic attempt to capture in sequential images the shifting and intangible states that are the emotions. In fact, the study of emotions in the laboratory requires that they be made controllable, both because of the practicalities of the experimental protocol — e.g. the limited space in a scanner — and in order to be able to make them compatible with the logic of recording and measuring inherent in scientific activity. In this context, photographs representing emotions, like fMRI-generated images of the brain reacting to those photographs, appear to be the preferred mechanisms for registering and reproducing emotions because of the undisputed relationship that would seem to exist between the photographic or the scanned image and the truth (Beaulieu 2002, Dumit 2004). That is to say, these images are considered to bear objective information that is independent of the context of their production and of the conditions that allow their reception. As I have shown in this account, however, the photo and the scan are dynamic elements that participate in the construction of scientific facts about the emotions. They participate in solidifying theories through their power to materialize scientific activity while also incorporating and circulating researchers’ discourses which are always contextually situated. For this reason, Ekman and Friesen’s photographs of facial affect are not only the reflection of actors miming an emotional state with their face, but they also signify the emotion itself, according to the principle of the universality of the emotions developed by these researchers, which allows the photos to be used as a stable stimulus of emotion. As for the scans of emotional adolescent brains produced from such stimuli, they offer, through the intermediary of the neuronal activation of young girls and young boys placed in the scanner, a visual translation of the emotions materialized in the photos in Ekman and Friesen’s POFA set. But through the succession of photographic images and then scanned images and their circulation among researchers, human guinea pigs and disciplines (psychology, epidemiology and the neurosciences), a space of transformation and added meaning has been created in which a difference can be seen between adults and adolescents as well as between young girls and young boys in the management of emotional information. It is principally through the integration of gendered research questions specific to adolescence, notably linked to the notion of risk behavior, in fMRI experiments that the image of a sexed emotional adolescent brain has taken shape. Paradoxically, this image contains traces of the photographs of facial affect whose scientific quality relies on their universality, independently of the specific age, sex or race of the individuals who express or perceive them.
