RLCS, Revista Latina de Comunicacion Social
Revista Latina

DOI, Digital Objetc Identifier 10.4185/RLCS-2015-1052en | ISSN 1138 - 5820 | RLCS # 70 | 2015 | Audio-visual explanation of the author |

Index h of the journal, according to Google Scholar Metrics, lgs

 

How to cite this article in bibliograhies / References

J Rúas-Araújo, M I Punín Larrea, H Gómez Alvarado,  P Cuesta-Morales,  S Ratté  (2015): “Neuroscience applied to perception analysis: Heart and emotion when listening to Ecuador’s national anthem”. Revista Latina de Comunicación Social, 70, pp. 401 to 422.
http://www.revistalatinacs.org/070/paper/1052/22en.html
DOI: 10.4185/RLCS-2015-1052en

Neuroscience applied to perception analysis: Heart and emotion when listening to Ecuador’s national anthem

J Rúas-Araújo [CV] Universidad de Vigo (España) – joseruas@uvigo.es 

M I Punín Larrea [CV] Universidad Técnica Particular de Loja (UTPL, Ecuador) – mipunin@utpl.edu.ec

H Gómez Alvarado [CV] Universidad Técnica Particular de Loja (UTPL, Ecuador) hfgomez@utpl.edu.ec

P Cuesta-Morales [CV]  Universidad de Vigo (España) – pcuesta@uvigo.es

S Ratté [CV]  Escuela Tecnológica Superior (ETS) de Montreal (Canadá) sylvie.ratte@etsmtl.ca

Abstract 
Introduction: The use of neuroscience methods, tools and techniques has become widespread in recent decades, and the prefix “neuro-” is now applied to disciplines such as the social sciences and communication as a consequence of the understanding that health and medicine are socially constructed. Method: This article analyses the impact of listening to Ecuador’s national anthem sung in different languages on a group of Ecuadorian university students based on the analysis of the physical and emotional changes (heart rate and facial expression, respectively) these students experience while listening to the different versions of the anthem. Results and conclusions: The heart rate variability results are not statistically significant and there are only minor differences in the perception of the anthem across gender groups. The analysis of facial expressions revealed an evolutionary sequential emotional pattern that involves “surprise and joy”, that is to say, a feeling of satisfaction.

Keywords
Neuroscience; neuro-communication; persuasion; emotion. 

Contents
1. Introduction. 1.2. Research purpose, objectives and hypotheses. 1.2. Contribution of neuroscience to the social and communication sciences: evolution and key findings. 1.2.1. Music, brain and heart. 1.2.2. Heart rate variability measurement and facial emotion recognition. 2. Method. 2.1. Population and sample. 2.2. Data collection instruments. 2.3. Procedure. 3. Results. 3.1. Qualitative HRV analysis by University. 3.2. Qualitative HRV analysis by ideology. 3.3. Qualitative HRV analysis by sex. 3.4. Significant differences in HRV. 3.5. Qualitative analysis with HER. 3.6. Qualitative analysis of the emotional load. 4. Discussion and Conclusions. 5. Notes. 6. List of references.

Abstract translation by Noel Byde. R.C.A./UCLES CERTIFICATE 1994. International House, Barcelona
Article translation by CA Martínez-Arcos, Ph.D. (Universidad Autónoma de Tamaulipas)

 [ Research ] [ Funded ] 
| word | Metadata | File PDF to print | Dynamic presentation - ISSUU | Paper with license Creative Commons | References |
| Series of files for e-books| mobi | htmlz + lit + lrf + pdb + pmlz + rb + snb + tcr + txtz |

1. Introduction
1.1. Research purpose, objectives and hypotheses

This research study has two purposes. The first one, is descriptive in nature, and consists in describing the progressive evolution of neuroscience towards the social sciences and the incorporation of its methods, techniques and tools to the analysis and measurement of people’s perception, providing new perspectives and approaches to qualitative and quantitative research, in all aspects related to persuasive communication.

The second purpose is experimental and consists in designing and implementing such tools to register the possible physical and emotional changes (through heart rate variability measurement devices and a facial emotion recognition program, respectively) produced by the exposure to different versions (in terms of language) of the Republic of Ecuador’s national anthem, in order to highlight the possible similarities or differences between people’s statements and actual measurements.

Based on the previous objectives, the following hypotheses were formulated:

H1: The application of neuroscience techniques and tools provides objective evidence on the measurement of the effectiveness of messages and, in particular, on the perception of different versions of Ecuador’s national anthem.

H2: The analysis of the collected data allows the observation of the possible similarities or differences between heart rate variability measurements, facial emotion recognition and participants’ responses to a questionnaire.
Based on these hypotheses, the following objectives were established in relation to the object of study:

  1. To identify differences or similarities in the perception of Ecuador’s national anthem, depending on the geographic residence of the selected samples of students (from the University of Los Andes, located at the central-northern region of the country; and from the Technological University of Loja, located at the south of the country).

  2. To identify potential differences or similarities in perception between men and women.

  3. To contrast the results obtained in the measurement of perception with the nationalistic and linguistic self-identification of participating students. 

1.2. Contribution of neuroscience to the social and communication sciences: evolution and key findings

The use of the term “neuro”, traditionally associated with the health sciences, became widespread as a noun in multiple scientific activities, from neurology, psychiatry and physiology, to the human and social sciences, from the perspective of functionalism and social constructionism, which understands that health and medicine, as well as science and technology, are socially constructed and thus can be addressed by medical sociology (Scambler, 2004), cognitive psychology (after the behaviourist movement was superseded), and the social psychology of health (Rodríguez Marín, 1995).

Neurosciences, in plural, emerged since “social neuroscience”, as such, held its first Congress in 2001 (Ochsner, Kevin & Matthew, 2001: 717), when neuroscientists observed that in order to understand the physical brain it could not be separated from “the social brain” –paraphrasing the title of Gazzániga’s 1993 book- in the search or the explanatory keys to human behaviour through the study of the brain’s emotional part -not just its rational part- (Damasio, 1999), after the overcoming of the Cartesian paradigm, which considered the individual only as mind and reason, and assessing the role of the unconscious (Mlodinow, 2012) in persuasive communication, both in its advertising and political modalities.

The functionalist perspective itself argues that brains are just the collection of water, blood vessels, cables, cells and processing modules that enable thought, and that the objective of the cognitive neuroscientist is to understand mental processes, thoughts, memories, emotions and experiences, beyond the purely chemical and biological aspects and the limited value of any brain mapping.

Like many cognitive psychologists, those who defend these ideas consider that a valid description for this new field is “empirical philosophy”, based on the ideas of Rosch (1977), who proposes experimental approaches to address subject matters that traditionally belonged to the field of philosophy. 

The idea is to open new avenues in the search for causal explanations about our thoughts, explaining not only what happens in our minds, but also why. This is to be achieved by going beyond traditional market research, which has been criticised by experts, despite its use of functional instruments, based on its limitations and, above all, based on the experience of observation and contrast between what people say they do and what they eventually do. Therefore, knowing what consumers really think and what motivates them to choose what they choose is the objective of this relatively young science, which seeks answers, for example, to the studies that indicate that more than 90% of cognition occurs under the level of consciousness, which causes a clear distortion between the stated behaviours and the reality of customers and voters (Lindstrom 2010; Braidot, 2011; Alvarez del Blanco, 2011). 

1.2.1. Music, brain and heart

Music and the human brain have been studied, jointly, in relation to anatomical, physiological and neuronal issues (Despins, 2010), while the relationship between music and the emotions it arouses in humans is a subject matter that has been investigated by composers for centuries and, at a later stage that might be called “scientific”, by researchers from the fields of psychology, neurology other fields of education and health sciences.

Díaz (2010) points out that it was during the baroque period when composers and scholars who studied music in its artistic dimension formulated some of the first findings about the effects of certain characteristics of musical pieces in people, like the finding that musical pieces written in major keys and with fast tempos cause joy, that pieces written in minor keys and slow tempos produce sorrow and that dissonance produces anxiety and fear.

On the influence of music at the physiological and psychological levels, Lacárcel (2003) also highlights the relaxing effect that music can have, reflected in a decline in the respiratory rhythm and the heart rate, while other scholars, such as Soria-Urios, Duque and García-Moreno (2011:49), highlight the ability of music to provoke emotional responses in two dimensions according to its valence (positive or negative) and intensity (low or high), as well as its capacity to provoke certain physiological alterations.

Another finding is that music, through the songs’ rhythm, tone and lyrics, stimulates the two cerebral hemispheres (the left one, responsible for processing the linguistic, semantic, symbolic and tempo-sequential aspects of music, and the right one, focused on emotional, plastic and aesthetic elements).

Musical activity involves almost all the regions of the brain, and almost all of the neurological subsystem. Different neurological regions process different aspects of music. Listening to or remembering lyrics invokes language-processing areas, including Broca’s area and Wernicke’s area, as well as other language-processing areas of the temporal and frontal lobes while the emotions that we experienced as a reaction to music affect deep structures of the primitive (reptilian) regions and the amygdala, the centre of emotional processing in the cortex (Levitin, 2008:95), which influences motivation and reward.

The positive effects of music on education have also been shown, especially its involvement in the activation of the cerebellum, a crucial structure in school learning (Brown, Martínez and Parsons, 2006), in addition to its contribution to other cognitive non-music related processes associated to reading and different linguistic activities (Moreno et al., 2008), the modulation of neuronal activity (Altenmüller, 2008) and, even, the regeneration of neurons (Fukui and Toyoshima, 2008).

Many of these authors, however, also recognise that, despite the obvious physical responses that our body experiences when listening to music, it is still very difficult to establish direct relationships between these variations, the music that produces them and the emotions that they represent.

But despite these difficulties, the importance music has had throughout history as political tool, to influence and generate behaviours and feelings of belonging, and as an identity-shaping tool and element of motivation and indoctrination (military marches, anthems), leads us to reflect on the evocative and inspirational ability that it may currently have.

Even with its supporters and detractors, Darwin’s theory of evolution (which gives music a role in the natural selection and mating rituals of humans, which are designed to attract the opposite sex) points out that music has accompanied our species for thousands of years. Musical instruments are among the oldest man-made artefacts ever found, like the Slovenian bone flute, created 50,000 years ago (Levitin, 2008: 273), while rhythmic music, which was very present among our ancestors, makes people move their bodies. Together, rhythm and melody connect the primitive brain, by activating the motor functions, with the central cortex, the most advanced part of the brain. Moreover, the combination of language (lyrics) and music activates thoughts, feelings and emotions.

The theory of evolution also conceives music as a means of social bonding and cohesion, and as a tool to shape our personal and group identity. This theory also highlights the possible influence of sociocultural, economic and even sexual factors, as a consequence of certain customs and traditions and unhealthy cultural and educational stereotypes (Despins, 2010: 69).

Precisely, one of the issues that this analysis of the perception of Ecuador’s national anthem aims to address is the possible influence of such factors as people’s sex and geographical, cultural and linguistic origin when evaluating the different versions of this anthem (in Spanish, Quechua and other indigenous languages).

1.2.2. Heart rate variability measurement and facial emotion recognition

The use of neuroscience methods and techniques in the social sciences, in general, and in communication sciences, in particular, is effective in the search for objective physical measurements capable of providing new evidence to classic audience studies and message evaluation.

To this end, one of the paths selected is physiology and, specifically, psychophysiology, which examines the dialogue and relations between the heart and the brain, through the observation and measurement of cardiovascular activity in the perception of the world and response to stimuli (Lacey, J. and Lacey, B., 2007).

The measurement of hearth rate variability (HRV) is one of the basic parameters in the analysis and assessment of cardiac activity, and can be defined as the number of heartbeats per unit of time, usually minutes. Heart rate is not constant, and changes to adapt itself to internal and external stress factors. HRV refers to alterations, heartbeat to heartbeat, in the cardiac rhythm. HRV is the result of adjusting the influence between the parasympathetic nervous system (heart rate acceleration) and the sympathetic nervous system (heart rate deceleration) (Malik et al., 1995).

Alterations in HRV can be studied in terms of duration and frequency through the non-linear analysis of the electrocardiogram (ECG).

One of the most frequently used techniques is the spectral analysis, which provides a quantitative analysis and an assessment of the state of the autonomic nervous system. The low-frequency (LF) components, with frequencies ranging from 0.04 to 0.15 Hz, are influenced by both the sympathetic and parasympathetic nervous systems, while the high-frequency (HF) components, which involve frequencies higher than 0.15 Hz, represent the parasympathetic tone related to the respiratory rhythm (Akselrod et al., 1981:220). In addition to the LF and HF parameters, another HRV indicator is the LF/HF ratio, which can be a good method to measure the balance between the sympathetic and parasympathetic nervous systems.

It is a proven fact that the peripheral nervous system prepares the body for various actions (Cacioppo and Bernston, 1994), which can occur as a response to certain external stimuli and emotional reactions to, for example, the viewing of political TV ads (Wang et al., 2014).

On the other hand, facial emotion recognition has been a field of research that started with Darwin (1873) and continues to the present day, given that the human face reflects our emotions and, to some extent, our behaviour. Hence the interest of different fields, like the health sciences, to assess the emotional state and empathy of patients, and the governmental forces and security bodies, which seek to increase their effectiveness in the detection of possible offences (Gordillo et al., 2013).

Similarly, facial emotion recognition is also important in the field of trade and business relations, where the recognition of emotions can contribute to the establishment of beneficial relations and to the improvement of communication with different audiences (customers, voters), in such areas as marketing and advertising [1].

Since the 1970s, some scholars attempted to develop an encoding, cataloguing and classification protocol or system for facial emotions (Ekman and Friesen, 1976; Ekman, 2003), in order to determine which muscle movements are related to the different types of emotions, starting with the six basic emotional expressions (joy, sadness, fear, disgust, contempt, surprise and anger), defined in the early 1970s by Ekman and Friesen (1971) and Izard (1971), and supported with strong empirical evidence since then to the present day (Matsumoto and Ekman, 2010).

Based on the conclusions drawn by Ekman, different research studies have focused on the analysis of such aspects as auditory stimuli (Müller et al., 2011; Sherman et al., 2012), scenes (Lee, Choi and Cho, 2012; Righart and de Gelder, 2008) and words (Lindquist et al., 2006).

Moreover, several studies have supported the fact that the facial expression of emotions has a biological and genetic origin (Matsumoto and Willingham, 2009), to such an extent that the same facial muscles that humans use for the facial expression of emotions are the same in chimpanzees (Burrows et al., 2006). These studies also defend the universality of the facial expression of emotion, even across cultures and countries.

However, some scholars have started to consider the influence of the personal context, understood as the set of expressions that summarise the knowledge held by the observer about the person producing the facial expression (Barret and Kensinger, 2010), as well as the emotional and affective state of the person (Gordillo et al., 2011, 2013:132), through, for example, the use of positive and negative valences on words with emotional content.

Scholars have also criticised the use in the analysis of expressions and prototypical scenarios drawn mostly from pre-existing standardised tests, such as the “Pictures of Facial Affect” created by Ekman and Friesen (1976), by applying the classical test theory (CTT), calculating the sum of participants’ scores, without taking into account the existence of expressions with very varying levels of difficulty when identifying the multiplicity of expressions and micro expressions manifested by participants (Márquez and Delgado, 2012).

To be precise, micro-expressions appear and disappear from the face in a fraction of a second (sometimes as fast as 1/30 of a second) and they can be understood as either signs of emotions that individuals try to hide, or, on the contrary, as signs of quickly-processed emotional states in which there has been no intention of hiding (Matsumoto et al., 2013:124). In fact, the idea of the existence of micro-expressions is rooted in Darwin’s hypothesis of the inhibition, which suggests that facial movements can occur unintentionally, even if the person tries to control their expression.

2. Method
2.1. Population and sample

The sample consisted of 20 Ecuadorian university students: 5 men and 5 women from the University of Los Andes (Universidad de Los Andes/UDLA), located in the capital of Ecuador, Quito, in the Pichincha province, in the central-northern region of the country; and 5 men and 5 women from the Technological University of Loja (Universidad Técnica Particular de Loja/UTPL), located in the Loja province, in the south of the country, near the border with Peru. All students were mestizos, and therefore were likely to be influenced by the exposure to the national anthem in Ecuadorian indigenous languages.

The 20 students were asked to listen to three different randomly-played versions of Ecuador’s national anthem: the official version sung in Spanish (02:40 minute), the Kichwa version (2:54 minutes) and the version sung in several indigenous languages (04:13 minutes). Each person listened to a total of 9 minutes of recorded audio, which is more than three hours of total material for exposure and analysis in the population under study. In addition, each student took some time to answer a brief questionnaire.

The version sung in Kichwa starts (minute 00:08 to 00:17) and ends (minute 02:41 to 02:50) with the following phrase spoken in Spanish by Jaime Roldós Aguilera: Este Ecuador amazónico, desde siempre y hasta siempre. ¡Viva la Patria! (“This Amazonian Ecuador, forever and always. Long live the homeland!”). Jaime Roldós Aguilera was the 33rd constitutional President of the Republic of Ecuador, from 10 August 1979 to 24 May 1981, when he died tragically. He is a very well-known and admired person in Ecuador.

With regards to the version in several indigenous languages, it begins with a short spoken story and chant delivered by a man in Amazonian Kichwa (minute 00:02 to 00:12) and ends with another chant, also in Amazonian Kichwa, delivered by a woman (minute 03:36 to 04:09). The middle part of this version combines the anthem sung in Spanish and Kichwa with other three languages: Chapala (minute 02:30 to 02:47), spoken by the “Chachi” indigenous people of the coastal province known as Esmeraldas; Awa pit or Awa (minute 02:57 to 03:05), the indigenous language of southern Colombia and northern Ecuador, spoken in the provinces known as Sucumbíos; and Shuar-Chicham, typical of the southeast of Ecuador and spoken in the provinces known as Pastaza, Morona Santiago and Zamora Chinchipe. It is also important to point out that, in addition to the aforementioned languages, the version sung in several languages includes the use of marimba (minute 02:10 to 03:00), a percussion instrument whose precursor is the Balophon built by Africans in America, and which in Ecuador has a clear identification with the peoples of African origin.

Finally, each of the students answered a questionnaire that investigated their and their parents’ geographical origin, gender, ethnic group; their nationalistic identification (centralist, pluri-nationalist, ethnic nationalist) and language preferences (Spanish as the country’s dominant language, Spanish as a language at the same level as other languages spoken in the country), and whether they defended the dominance of other languages of Ecuador over Spanish).

Participants were also asked to name the language or languages they recognised in each of the three versions of the anthem. This information was requested in order to be able to compare the answers given by participants in the questionnaire with their physical and emotional measurements (HRV and facial emotion recognition, respectively).

Here it is important to note that participation in the study was completely voluntary and that participants’ identity data were treated confidentially. 

2.2. Data collection instruments

Two types of tools were used for data collection, one type to measure HRV and another for facial emotion recognition.

For the HRV analysis we used the R Package for Heart Rate Variability (RHRV), a free software designed by the MILE (bioMedical Informatics and signaL procEssing) Group (www.milegroup.net) [2] of the University of Vigo (which makes this software available on its website), and the VARVI (Heart Rate Variability Analysis in Response to Visual Stmuli), which allows studying HRV in response to visual and auditory stimuli (Rodríguez-Liñares et al. 2013).

The commercial sensor Polar WearLink®+ transmitter with Bluetooth® wireless technology (http://www.polar.es) was used for heart rate measurements. This non-invasive sensor is a chest strap with two areas equipped with plastic electrodes that pick up users’ heart signals. It has a wireless transmitter that transfers the heart rate data into a compatible mobile device, which in this case was a laptop. The sensor is powered by a CR 2025 battery that lasts 150 hours, provided it is put to work in temperatures ranging from 0° C to 50° C.

The other tool is the HER software, designed by the team of artificial intelligence specialists of the Technological University of Loja (UTPL), to identify people’s emotions through their facial expressions. It is based on Parrot’s matrix of emotions (Frijda and Parrot, 2011), which is subsequently based on the recognition of five emotions: passion, surprise, joy, disgust and fear. When the HER program does not recognise any of Parrot’s emotions in a participant’s face it concludes that the person is expressing a state of normality.

The Binary and Canny filters [3] were used for the recognition of facial emotions in order to detect image patterns and the Bézier curve (Lee, Han and Kim, 2013) in order to identify parts of the face (eyes, eyebrows, mouth). Bezier curves are a practical solution to the problem of finding a polynomial curve for a function or an array of points. A Bézier curve is a line that can become a curve from one end to the other, passing through a point or the control points of the array.

HER identifies the emotion in the person’s face, but it is the person who decides whether the emotion detected is correct or not. This is what gives a dynamic character to HER. The faces processed in each experiment refine HER’s face database and improves its recognition capacity so that this tool is constantly improved and updated. This allows HER to be able to identify more complex faces and emotions through the use of ontologies and the processing of new faces after their emotions have been identified. The entry data in HER are videos or images of peoples’ faces and the output is the percentage of emotions recognised in the face of participants.

2.3. Procedure

The programs developed to reproduce the three versions of Ecuador’s national anthem were written in Python (http://python.org), an object-oriented programming language that allowed the random selection of the reproduction order of the musical pieces. The program was connected to the heart rate sensor and, as it collected the previous data, it stored them in a file, which in turn labelled the values depending on the version of the anthem that was being listened to, with the duration of each one of the versions and the associated start-up time.

The next step was to measure HRV while the content of the anthems was being accessed, and to distinguish the data collected while each of the three versions of the anthem was being listened to. To do this, each participant adjusted the chest strap heart rate sensor by themselves, stood in front of the laptop computer, and the process of data collection started once the sensor’s frequency was detected by the computer program.

Different versions of the anthem were played back in a random order to each participant, guaranteeing that all the sample of participants listened to each of the three versions, only once each. After the data collection was completed, the program stored two different files. The first of them contained the HRV data, and the second associated these data to each of the three versions listened to by participants.

In order to perform the HRV analysis that would allow us to detect and draw conclusions about the differences between the sample of participants when listening to the different versions of the anthem, the HRV signals stored in the database were processed with the R-HRV software package (Rodríguez-Liñares et al., 2011), which was developed with the R language (http://www.r-project.org).

g1en

 

Figure 1: HRV calculated for each participant and each version of the anthem

 The process started with the initial analysis of the heart rate signals stored for each participant, with an algorithm that allows the elimination of artefacts and noise that may be present in the data and may cause distortions in the results. This, along with other algorithms, allowed us to obtain a corrected heart rate signal, which was subjected to spectral analysis, by extracting the parameters of low-frequency (LF) components, high frequency (HF) components, ratio (LF/HF), and the resulting total frequency (HRV).

The previous values were obtained for each participant in two different ways: 1) globally, for all participants of the two universities; and 2) individually, in order to explore whether there are differences in the population when listening to each of the three versions of Ecuador’s national anthem, and whether the reception is conditioned by its geographical origin (north or south), gender (male or female), ideology and nationalistic identification (centralist, pluri-nationalist, or ethnic nationalist).

With regards to the facial emotion recognition program, we applied HER to the collection of videos of the experiment. The procedure was as follows:

  1. Selection of the videos of the experiment. These videos captured participants while they were listening to the national anthem in its different versions.

  2. Each video was scanned and analysed with HER to obtain the percentage of recognised emotions.

  3. Selection of the time intervals in which the heart rate was measured so that they coincided with the analysis of emotions in the videos. After establishing the time intervals, emotions were filtered to identify only those that were expressed in that period.

  4. Selection of the emotion(s) with the highest percentage in the time interval. To this end, we selected a threshold of difference (being able to select different thresholds in future experiments). If the difference between the emotion with the highest percentage and the other emotions did not exceed the threshold of 5%, then the first emotion was considered as present in the time interval.

  5. Location of the emotions that appeared the most in the time interval as data columns, next to the HRV analysis.

g2en

Figure 2: Example of the application of HER

The previous figure shows an example of the execution of HER, before calculating the emotional percentage for each face. Pressing the “Emotion Detection” button triggers the procedures to calculate the Bézier curve and initiates the filters, in order to identify the emotion according to the facial expression. Afterwards, the percentage values are exported in order to calculate the most common emotion in each time interval. In this case the time interval is 10 seconds, i.e. a face is captured every ten seconds for analysis. 

Although the size of the time interval can be considered too large, we wanted to make a first experimentation capable of creating a “general” pattern, and not necessarily to search for micro-expressions, because we know that, in that case, the time interval is reduced to a quarter of a second. As a result, we are not looking for a pattern, but the immediate emotional difference, and the former applies more when the objective is to identify a detailed feature as, for example, identify when someone lies [4].

In addition to identifying significant differences in HRV, another objective of this experiment was to analyse its relationship with the emotional response.

In this sense, we analysed the videos recordings of the participants while they were listening to the three anthems, and the emotions expressed were obtained automatically. The five patterns of temporal sequences of emotions that were identified among the participants of the experiment when listening to the versions of the anthem were: “surprise-joy”, “surprise-joy-disgust”, “normal-surprise”, “surprise-anger” and “normal”. A spectral analysis was performed for each of these categories, extracting the different frequency components (LF, HF, LF/HF and HRV). 

3. Results
3.1. Qualitative HRV analysis by University

The results of the qualitative analysis, depending on the geographical location of participants’ university, indicate that participants from the South of Ecuador (from the Technological University of Loja) registered higher high frequency (HF) values than participants from the central-northern region of the country (from the University of Los Andes). A possible interpretation for this result is that listening to all three versions of the anthem causes the former group greater HRV.

Moreover, since the low-frequency (LF) values are similar for both universities, the ratio between low and high frequencies (LF/HF) is therefore lower for participants from the Technological University of Loja than for those from the University of Los Andes. 

3.2. Qualitative HRV analysis by ideology

Since only one participant indicated that the option that best characterised its nationalistic identity was the ethnic nationalist option, the spectral analysis by ideology only distinguished between those participants who identified themselves with the centralist or the pluri-nationalist options.

The results of the study show some differences in the analysis of low-frequencies (LF). Participants who identified with the centralist option presented more low-frequency (LF) values than those who identified with the pluri-nationalist option. This occurred when listening to any of the three version of the anthems and seems a bit more pronounced when listening to the official anthem. 

3.3. Qualitative HRV analysis by sex

Based on an initial analysis, the most important differences in the spectral analysis of HRV occurred between gender groups.

The total HRV of women is greater than that of men when listening to each of the three anthems (official, Kichwa and several languages). In the same way, women’s high and low frequency values when listening to any of the anthems are greater than those of men when listening to the same anthem. In terms of the LF/HF values, they are lower in women than in men when listening to any of the anthems. 

3.4. Significant differences in HRV

Since the initial qualitative analysis showed that the main differences occurred between gender groups, it was decided to carry out a more detailed study focused on the gender differences. The results corroborated this finding, as some of the spectral values showed significant differences between men and women. 

 

Average in women

Average in men

95% CI

p-value

HRV

996.556

559.504

(-27.670, 901.803)

0.063

LF

282.950

226.312

(-79.566, 192.842)

0.392

HF

398.804

100.539

(4.329,592.202)

0.047

LF/HF

1.641

4.516

(-5.354, -0.394)

0.027

Table 1. Comparison of spectral values for HRV, LF, HF and LF/HF, between men and women. The table also presents the confidence intervals at 95% (95% CI) and the p-values.

 Considering a statistical significance level of 0.05, there are significant differences in the spectral HF and LF/HF values. In these cases, in addition, the CI of 95% does not contain the value 0, which, together with the “p-value”, allows us to reject the null hypothesis.

Taking into account that the most significant differences occur across gender groups, it was decided to focus this study also on the ideology variable. In this case, we found out significant differences in the study of the total HRV. While HRV is higher among centralist men (633.812 against 500.054) the situation is the opposite among women, where this variable is higher among those self-identified as pluri-nationalist (926.923 against 847.648). 

3.5. Qualitative analysis with HER

This section will explain the qualitative analysis performed with HER. The “normal” emotion category was detected only in one individual. This emotion occurs when the person is in an expectant state because he/she does not know what he/she is going to experience, or does not show enough interest in the experiment, and as a consequence, the result does not provide any statistical significance. This individual pattern will be named “1-sequence” since it comprises a single emotion. The following table shows the results of the study in relation to the remaining emotional patterns: 


Sequential pattern:
N-sequence

Pattern

University

Registered in

2-sequence

Normal-surprise

Andes

Few participants. More women

2-sequence

Surprise-joy

Andes-Loja

Most participants.

3-sequence

Surprise- joy-disgust

Loja

Mostly men.

Table 2: Outstanding sequential patterns

 The 2-sequence corresponding to the “normal-surprise” pattern, is a sequential change and suggests that from the second stanza of the anthem the person is surprised by what he/she is listening to in the anthem.

The distinction by universities presents some notable differences. The most repeated “2-sequence” pattern is “surprise-joy”. This pattern is typical of participants from both universities (Andes and Loja) and corresponds to a complex emotion known as “Complacency” (Pitterman, Pitterman and Minker, 2010), i.e. the participants in the study displayed satisfaction when listening to the national anthem. This could be related to what some authors, like Rivera (2003), have pointed out, in the sense that ‘all Ecuadorian people feel emotion when singing the National Anthem, a melody that highlights the history, the civic values, which permanently lift the spirit of the Ecuadorian people, recognises their character and their daily effort that nourishes freedom, justice, and the law. 

In addition, it could be the case that some of the phrases reminiscent of war contained in Ecuador’s national anthem or the invocation and use of the anthem in incidents and conflicts with other countries (some relatively recent events, like the incursion of the Colombian army into Ecuadorian territory with the pretext of pursuing members of the FARC, and events that happened long time ago, like the Peruvian-Ecuadorian war) influence in some way the expressed emotions, although in this particular case the experiment involves young students.

The Technological University of Loja is the only one where the “3-sequence” corresponding to the “surprise-joy-disgust” pattern appeared. The explanation to the contrast in the sequential emotions could be the fact that “patriotic anthems and tribal verses constitute rhetorical components that build differentiated feelings of love” but “despite their differences, anthems and poems promote love towards these imagined communities as a tool to hide or mask the emotional ambiguity of the contemporary individual before the collective subjects” (Gutiérrez-Estévez, 2004).

Gender differences emerged once again. The “3-sequence” corresponding to the “surprise-joy-disgust” pattern and the “2-sequence” corresponding to the “surprise-anger” patter emerged more among men, while the “2-sequence” corresponding to the “normal-surprise” pattern emerged more among women. The contrasting sequence of emotions (joy-disgust) responds to the possible ambiguity or mixed feelings that arise when listening to the national anthem, and while the sequence “normal-surprise”, as mentioned previously, reflects the start of the complex emotion known as “pleasure”. 

3.6. Qualitative analysis of the emotional load

Based on the analysis performed with HER, we carried out a qualitative analysis of the spectral parameters for the different emotional categories detected. Of the five categories, the one corresponding to “normal” was detected in only one participant while listening to only one of the anthems. For this reason, we decided to discard it from the spectral analysis.

The results show that the “normal-surprise” category presents the highest frequency rates (HF as well as LF and HRV), in comparison to the other three categories (“surprise-joy”, “surprise-joy-disgust” and “surprise-anger”), which present similar average values (with no noticeable differences). It is also important to note that with regards to the LF/HF ratio, the category with the highest frequency values is “surprise-anger”.

g3en

Figure 3: Box plot showing the relationship between HRV and emotions

In any case, largely due to the small number of case studies that made up the sample (only 20 people), we did not find significant differences in HRV, in terms of the detected emotional categories.

Despite this, we did carried out a qualitative analysis of the different emotional categories according to the participants’ university and gender, and the versions of the anthems. In the first case, the distinction by universities presents some notable differences. The most repeated category is “surprise-joy”, which occurs in both universities. However, the Technological University of Loja is the only university where the “surprise-joy-disgust” category appears, while the University of Los Andes in Quito is the only one in which the “normal-surprise”, “surprise-anger” and “normal” categories appear.

In the case of the differences depending on the version of the anthem they were not noticeable, since all the emotional categories were present in all the anthems, with the exception of the ‘normal’ category which was expressed only once, by one person while listening to the Kichwa version of the anthem.

Finally, the differences across gender groups once again are the most significant. The “surprise-joy-disgust” and “surprise-anger” patterns are more common among men, while the “normal-surprise” emotional category is more common among women.

CATEGORY

  Loja

Andes

Official

Kichwa

Several languages

Men

Women

Surprise-
joy

22

16

12

14

12

15

23

Surprise-
joy-disgust

8

0

3

2

3

6

2

Normal-
Surprise

0

5

1

1

3

1

4

Surprise-
Anger

0

5

3

1

1

4

1

Normal

0

1

0

1

0

1

0

Table 3: Distribution of emotional categories by participants’ geographical origin, (central-northern/Andes and south/Loja), version of the national anthem and participants’ gender.

4. Discussion and conclusions 

The main conclusion that can be drawn from the study is that there are no significant differences among participants, in terms of HRV, while listening to each of the three versions of Ecuador’s national anthem.
The only significant differences detected in terms of HRV were observed across gender groups. Some of spectral values (HF and LF/HF) show statistical significance between men and women.

The most important conclusions regarding the HER program are that the emotions that most commonly form a pattern are: “surprise-joy-disgust”. This is an apparently contradictory evolution which, apart from indicating the experiencing of possible mixed feelings while listening to different versions of the anthem, as already mentioned, could be due to certain resistance from participants to the take part in the experiment (students did not receive any compensation or reward for their “voluntary participation”), environmental resistances (the environment) or, even, as a result of long duration of the material which they asked to listen to (a total of nine minutes between the three versions, of which more than two minutes correspond to the versions in Spanish and Kichwa and more than four to the version in several languages), which provoked certain degree of boredom in some participants, mostly during the second part of each version.

In any case, the combination of “surprise and joy” reflects the more complex emotion of “complacency”, i.e. the majority of people enjoyed listening to the national anthem, according to the analysis of their faces.

Having a sample composed of more than 20 students could possibly result in the detection of more significant differences, while a new experiment could be carried out to detect micro-expressions by reducing the time interval to a quarter of a second. This would substantially increase the number of images available for analysis and would result in a much more detailed study than the search for a pattern, which can be the product of a new research study, as it would allow researchers to note the exact part of the song that “distracts” participants’ face.

However, regardless of the discussion undoubtedly provoked by the conclusions of this work, the initial hypotheses confirm that the use of neuroscience methods, techniques and tools provides objective evidence on the measurement of the effectiveness of messages, despite the HRV analysis and the facial recognition of emotions do not show statistically significant differences between the different samples and participants nor in terms of the information obtained from the questionnaires, except on some specific issues and cases.

Apart from the results finally obtained, it is important to highlight the richness of interdisciplinary collaboration, which opens new avenues for research in neuroscience applied to the social sciences and allows for the contrast and combination of very useful tools for the future.

 

  • This work is part of the results of the research project titled “Neuroscience applied to social sciences and political-institutional communication”, directed by the main author of this article, José Rúas Araújo, beneficiary of the “Prometeo” programme of the Ministry of Higher Education, Science, Technology and Innovation (Senescyt), of the National Government of the Republic of Ecuador (a public government agency).

  • This work is part of the results of the research project titled “Neuroscience applied to social sciences and political-institutional communication”, directed by the main author of this article, José Rúas Araújo, beneficiary of the “Prometeo” programme of the Ministry of Higher Education, Science, Technology and Innovation (Senescyt), of the National Government of the Republic of Ecuador (a public government agency).

5. Notes

[1] Even so, the systematic review carried out by Marquez and Delgado (2012) of scientific articles devoted to the recognition and expression of emotions did not confirm the existence of works nor the application of test directly related to the fields of marketing and communication, although there are studies that value their importance for the efficient negotiation of emotional recognition (Elfenbein et al., 2007).

[2] The MILE group has several research lines, including HRV analysis, which has been applied in different contexts and studies, such as the diagnosis of sleep apnoea (Lado et al. 2011), and the analysis of people’s reactions to electoral campaign ads, in collaboration with the “Necom: neuro-communication, advertising and politics” research group of the University of Vigo (necom.uvigo.es).

[3] More information about the filters used in face identification available at:http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html 

[4] More details about micro-expressions available at: http://www.dailymotion.com/video/xf6fu8_microexpresiones-faciales-paul-ekma_school

6. List of references

Akselrod, S., Gordon, D., Ubel, F., Shannon, D., Berger, A., Cohen, R. (1981): “Power spectrum analysis of heart rate fluctuation: a quantitative probe of beat-to-beat cardiovascular control”, in Science, 213(4504), pp. 220-222.

Altenmüller, E. (2008): «Neurology of musical performance», in Clin Med, 8(4), pp. 410-413.

Álvarez del Blanco, R. (2011): Neuromarketing. Madrid: Prentice Hall/Financial Times.

Barrett, L.F. & Kensinger, A.K. (2010): «Context is routinely encoded during emotion perception», in Psychological Science, 21, pp. 595-599.

Braidot, N. (2011): Neuromarketing. ¿Por qué tus clientes se acuestan con otro si dicen que les gustas tú? Barcelona: Gestión 2000.

Brown, S., Martínez, M.J. & Parsons, L.M. (2006):“The Neural Basis of Human Dance”, in Cerebral Cortex, 16(8), pp. 1157-1167.

Burrows, A.M., Waller, B.M., Parr, L.A. & Bonar, C.J. (2006): «Muscles of facial expression in the chimpanzee: Descriptive, comparative and phylogenetic contexts», in Journal of Anatomy, 208, p. 153-167.

Cacioppo, J.T. & Bernston, G. (1994): “Relationships between attitudes and evaluative space: A critical review with emphasis on the separability of positive and negative substrates”, in Psychological Bulletin, 115, pp. 401-423.

Damasio, A. (1999): El error de Descartes: la razón de las emociones. Madrid: Andrés Bello.

Darwin, C. (1873): The expression of emotions in animals and man. New York: Appleton.

Despins, J.P. (2010): La música y el cerebro. Barcelona: Gedisa.

Díaz, J.L. (2010): «Música, lenguaje y emoción: una aproximación cerebral», in Salud Mental, 6, p. 553; retrieved on 12/10/2014, from http://www.scielo.org.mx/pdf/sm/v33n6/v33n6a9.pdf.

Ekman, P. (2003): El rostro de las emociones. Barcelona: RBA.

Ekman, P. & Friesen, W.V. (1976): «Measuring facial movement», in Journal of Environmental Psychology, 1, pp. 56-75.

Ekman, P. & Friesen, W.V. (1971): «Constants across culture in the face and emotion», in Journal of Personality and Social Psychology, 17, pp. 124-129.

Elfenbein, H.A., Foo, M.D., White, J., Tan, H.H. & Aik, V.C. (2007): «Reading your conunterpart: The benefit of emotion recognition accuracy for effectiveness in negotiation», in Journal of Nonverbal Behaviour, 31, pp. 205-223.

Frijda, N., & Parrot, W. (2011): «Basic emotions or ur-emotions?», in Emotion Review, 10,  pp. 406-415.

Fukui, H., Toyoshima, K. (2008): «Music facilitate the neurogenesis, regeneration and repair of neurons», in Med Hypotheses, 7.

Gazzániga, M. (1993): El cerebro social. Madrid: Alianza Editorial.

Gordillo, F., Arana, J.M., Mestas, L. & Salvador, J. (2011): «Modulación emocional en la valoración de la expresión facial. ¿Artefacto o realidad?», in EduPsykhé, 10, pp. 277-290.

Gordillo, F., Lozano, J.H., López, R.M., Pérez, M.A., Arana, J.M. & Mestas, L. (2013): «Emoción, contexto verbal y percepción de la expresión facial: dime quién eres y te diré cómo te percibo», in Ansiedad y Estrés, 19, pp. 131-147.

Gutierrez-Estevez, M. (2004): “El amor a la patria y a la tribu. Las retóricas de la memoria incómoda”, in Revista de Antropologia, 2, pp. 345-377.

Izard, C.E. (1971): The face of emotion. East Norwalk, CT: Appleton-Century-Crofts.

Lacárcel, J. (2003): «Psicología de la música y emoción musical», in Education Siglo XXI, 20-21, pp. 213-226 ; retrieved on 16/10/2014, from http://revistas.um.es/educatio/article/view/138/122

Lacey, J. & Lacey, B. (2007): «Heart Rate in Sensorimotor Behaviour». In: P.A.Obrist, A.H.Black, J.Brener & L.Dicara (Eds.): Cardivascular Psycophysiology: Current Issues in Response Mechanisms, Biofeedback and Methodology. Proceedings of a Conference held on 1-3 May, 1972, at the University of North Carolina at Chapel Hill. Chicago (USA): Aldine Pub.Co.

Lee, T.H., Choi, J.S. & Cho, Y.S. (2012): «Context modulation of facial emotion perception differed by individual difference”, in PloS ONE, 7.

Lee, Y.-H., Han, W. & Kim, Y. (2013): «Emotional Recognition from Facial Expression Analysis using Bezier Curve Fitting». 16th International Conference on Network-Based Information Systems (pp. 250-254). IEEEXplore.
Levitin, D. (2008): Tu cerebro y la música. El estudio científico de una obsesión humana. Barcelona: RBA.

Lindquist, K., Barret, L.F., BlissMoreau, E. & Russell, J.A. (2006): “Language and the perception of emotion”, in Emotion, 6, pp. 125-138.

Lindstrom, M. (2010): Buyology. Verdades y mentiras de por qué compramos. Barcelona: Gestión 2000.

Malik, M., Camm, A.J.  (1995): Heart rate variability. New York: Futura Publishing Company.

Márquez, M. & Delgado, A. (2012): «Revisión de las medidas de reconocimiento y expresión de emociones», in Anales de Psicología, vol.28, 3, pp. 978-985.

Matsumoto, D., Sung, H., López, R. & Pérez-Nieto, M.A. (2013): «Lectura de la expresión facial de las emociones: investigación básica en la mejora del reconocimiento de emociones», in Ansiedad y Estrés, 19 (2-3), pp. de 121-129.

Matsumoto, D. & Ekman, P. (2010): «Subjective Experience and the Expression of Emotion in Man». In Koob, G.F., Le Moal, M. & Thompson R.F. (Eds.): Encyclopedia of Behavioural Neuroscience (Vol.3). Oxford: Academic Press.

Matsumoto, D. & Willingham, B. (2009): «Spontaneous facial expressions of emotion of congenitally and noncongenitally blind individuals», in Journal of Personality and Social Psochology, 96, pp. 1-10.

Mlodinow, L. (2013): Subliminal: Cómo tu inconsciente gobierna tu comportamiento. Barcelona  Crítica.

Müller, V.I., Habel, U., Derntl, B., Schneider, F., Zilles, K., Turetsky B.I. & Eickhoff (2011): «Incongruence effects in crossmodal emotional integration, in Neuroimage», 54, pp. 2257-2266.

Ochsner, Kevin, Matthew & Lieberman (2001): “The Emergence of Social Cognitive Neuroscience”, in American Psychologist, 56, pp. 717-728.

Pitterman, J., Pitterman, A., & Minker, W. (2010): Handling Emotions in Human-Computer Dialogues. New York: Springer.

Righart, R. & Gelder, B. (2008): “Recognition of facial expressions is influenced by emotion scene gist”, in Cognitive, Affective and Behavioural Neuroscience, 8, pp. 264-278.

Rivera, O. (2003): “Himno Nacional: reverencia y civismo”, in La Hora-Loja, 26 November, p. 2.

Rodríguez-Liñares et al. (2013): “VARVI: a software tool for hearth rate Variability Analysis in Response to Visual Stimul”, in Computers in Cardiology. Zaragoza, 22-25 September.

Rodríguez-Liñares, L., Méndez, A.J., Lado, M.J., Olivieri, D.N., Vila, X.A., Gómez-Conde, I. (2011): “An Open Source Tool for Heart Rate Variability Spectral Analysis”, in Comput Methods Programs Biomed, 103, pp. 39-50.

Rodríguez Marín, J. (1995): Psicología Social de la Salud. Madrid: Síntesis.

Rosch, E. (1999): “Human categorization”, in Advances in Crosscultural Psychology. London: Academic Press.

Soria Urios, G., Duque, P. & García-Moreno J.M. (2011): «Música y cerebro: fundamentos neurocientíficos y trastornos musicales», in Revista de Neurología, 52, p. 49.

Scambler, G. (2004): Medical Sociology: Major Themes in Health and Social Welfare (Major Themes in Health and Social Welfare). N. York: Routledge.

Wang, Z., Morey, A.C. & Srivastava, J. (2014): “Motivated selective attention during political ad processing: the dynamic interplay between emotional ad content and candidate evaluation”, in Communication Research, 41(1).

Related articles:
Gordillo Fernando;  Lozano Héctor,  López  Rafael  et al. (2013). Emoción, contexto verbal y percepción de la expresión facial. Dime quién eres y te diré cómo te percibo. Ansiedad y Estrés Vol. 19 (2-3). Available at:  www.ansiedadyestres.org

Vitale, Luis (2012). Música popular e identidad latinoamericana, Archivo Chile. Historia Política-Movimiento Popular. Centro de Estudios Miguel Enrique. Available at: www.archivochile.com/Ideas_Autores/vitalel/9lvc/09otros0009.pdf  

___________________________

How to cite this article in bibliographies / References

J Rúas-Araújo, M I Punín Larrea, H Gómez Alvarado,  P Cuesta-Morales,  S Ratté  (2015): “Neuroscience applied to perception analysis: Heart and emotion when listening to Ecuador’s national anthem”. Revista Latina de Comunicación Social, 70, pp. 401to 422.
http://www.revistalatinacs.org/070/paper/1052/22en.html
DOI: 10.4185/RLCS-2015-1052en

Article received on 1 March 2015. Accepted on 26 May. Published on 17 June 2015.

___________________________________________________