10.4185/RLCS-2017-1189en | ISSN 1138 - 5820 | RLCS, 72-2017 | |
Augmented Reality for Learning in People with Down Syndrome: an exploratory study
Rosa-María Martín-Sabarís [CV] ORCID orcid.org/0000-0002-0793-3749
Translation by Yuhanny Henares
Information and Communication technologies have demonstrated to be a powerful tool for learning in individuals with Special Educational Needs. Currently, the proliferation of mobile devices is enabling that sectors such as advertisement could increase their efficacy and business possibilities through geolocalization, interactivity and targetization. However, segmentation of publics has not yet reached individuals with Down Syndrome and other intellectual disabilities, that are not being considered in mobile applications that facilitate learning, communication, leisure or everyday life.
The recent phenomenon Pokemon Go allowed massive publics to approach Augmented Reality. This technology allows combining audiovisual, textual and positioning elements, as well as to overlap virtual information in real time, images or markers over real fields, both indoors and outdoors. Through glasses such as the ones developed by Google Glass or the smartwatch of the Korean company Samsung, it is possible to have a juxtaposition between the virtual and real world. However, despite the generalization of this technology, the digital gap continues and it does not occur only between rich and poor countries, between the north or the south, or between urban and rural zones, but also between the healthy population and the population with mental diseases. In general, the research has related the intellectual disability and the Down Syndrome focusing in the educational context and formal learning. However, the collective of individuals with intellectual disabilities is still not considered a market niche for the industry and the market of Augmented Reality and Virtual Reality, nor a population object of analysis in the field of Social Communication.
2. Objectives and hypothesis
The lack of studies about the use of this technology among individuals with intellectual disability obliges to think about an exploratory research which final goal is to analyse the utility of Augmented Reality (AR) in the communication and learning in individuals with Down Syndrome (DS). Regarding this main objective, the following specific objectives are set forth:
The hypothesis has been formulated from the reading and analysis of bibliography about the use of Augmented Reality and its application in different areas, but also through the own personal and professional experience in the field of technologies development, production of audiovisual contents and intellectual disability.
3. Theoretical background and state of the art
The concept of Augmented Reality (AR) appears for the first time in 1990, as a variation of a revolutionary technology like Virtual Reality. Authors such as Kato-Billinghurst (1999), Milgram et alt. (1994, 1995) and others coincide in the fact that the definition described in the work of Roland Azuma in 1997 called “A survey of augmented reality” brought the first theoretical postulates in the matter:
To avoid limiting the development AR to specific technologies, Azuma defines as Augmented Reality, any system that can develop these characteristics: a) combining real and virtual elements, b) being interactive in real time, and c) be recorded in 3D.
By stablishing these postulates for defining Augmented Reality, Azuma overlapped the conceptual field to the technologies of the moment, but social consumptions and trends led Oliver Bimber and Ramesh Raskar (Raskar et al. 1998; Bimber et al., 2005) to suggest the aggregate of spatial Augmented Reality or its term in English spatial augmented, which consists in 3D objects, which are real and virtual but not interactive.
Since in 1962 Morton Heiling designed a motorcycle simulator that recreated a route in the streets of New York called Sensorama, the progresses in the development of virtual reality or augmented reality have been numerous.
The concept of virtual reality appears in 1965 when Ivan Sutherland publishes his scientific article entitled “A Head-mounted Three-Dimensional Display” where he says that “the screen is a window through which one can see a virtual world. The challenge is making this world look real, act real, sound real, to feel real” (Sutherland,1968: 7).
Almost 30 years later, in 1992, David Mizell and Tom Caudell suggest the definition of Augmented Reality. The research was called “Augmentative reality: an application of heads up display technology to manual manufacturing processes”. The term was used to refer to the enriching of the real world and it was applied to a case of the manufacturing industry: a series of virtual graphs that overlapped over an assembly board of the vision of a HDM called private eye. (Caudell et al., 1992).
Karma (1994), ARToolKit (1999), ARQuake (2000), AR-PDA (2001), Wikitude (2008), Layar (2009), Arhrrrr (2012), Google Glass (2012), Ingress (2013), Illutio (2014) or Pokemon Go (2016) do not stop being milestones in a process of consolidation of Augmented Reality in the market of a technology that will keep exploring new uses and applications.
Different authors have stablished different levels of AR depending on the complexity of systems (Billinghurst et al, 2015). Reinoso (2016) mentions these four levels: level 0) hyperlinks in the physical world, level 1) AR based on markers, level 2) markerless AR, level 3) augmented vision. On the other hand, Portales Ricart (2008) groups the different systems of the AR within four dichotomies: indoors/outdoors, local/ubiquitous, mobile/spatial and individual/ collaborative experience. All these have had different applications in diverse fields such as medicine (Juan et al, 2005), architecture (Chi et al, 2013), emergency and military services (Zhu et al, 2015), industry, tourism (Kounavis et al. 2012), patrimony preservation (Martínez, 2010), museums (Torres, 2011), education (Kaufmann-Schmalstieg, 2003; McMahon, 2016) or communication (Galán Cubillo, 2008) just to mention some of the most important. Massification has come hand by hand with the entertainment industry, which has offered its large publics to marketing and advertising.
One of the most innovative projects applied to education is one called Magic Book developed by the department of Human Interface Technology Laboratory of Washington University in USA (Billinghurst et al, 2001). It consists in a real book format where pages can be turned, texts read and pictures looked at, but if the reader prefers using an Augmented Reality device, an additional augmented information that the book has in virtual 3D models can be seen as well.
In the European sector, there are different projects that develop innovative applications integrating Augmented Reality, CONNECT, CREATE  and ARiSE  projects use tools based on 3D presentations and with a great interaction make understanding of contents easier (Basogain, 2007).
3.2. Down Syndrome
La American Association on Mental Deficiency (AAMD) defines intellectual disability as “a disability characterized by significative limitations both in the intellectual functioning and adaptive behaviour that manifests in conceptual, social and practical adaptive abilities. This disability starts before eighteen years old.” (AMMD, 2004: 18).
The Down Syndrome is defined as a congenital anomaly due to the presence of an additional chromosome in the 21 pair. The different chromosomic abnormalities are: trisomy 21, translocation and mosaicism (Pueschel, 1991).
Individuals with Down Syndrome use the same mechanisms or learning storages than individuals who do not have cognitive immaturity: sensorial registration, short-term memory and long-term memory. Information is obtained through stimulation of senses, it is processed and stored to be used afterwards whenever needed. Learning requires four essential mechanisms: attention, memory, motivation and communication (Flórez and Troncoso, 2001; Valverde-Montesinos, 2006).
Individuals with Down Syndrome show attention deficiencies at an early age. The following are the main attention difficulties:
Latency period for responding that is longer than other individuals of his or her same mental age (Flórez and Troncoso, 2001; Valverde-Montesinos, 2006).
Besides, often there is impossibility to clearly perceive a sound sequentiality or extensions of words and resonant complex phrases. To a lesser extent, they also show difficulty in decodification and interpreting visual stimuli (Valverde-Montesinos, 2006).
In individuals with Down Syndrome, short-term memory (about 30 seconds) is reduced to half, which is reflected in the difficulty of performing a mental or motor operation in an immediate manner, especially when information presents itself verbally or auditorily (Flórez and Troncoso, 2001).
Regarding long-term memory, these authors summarize limitations of DS individuals as follows:
Motivation in individuals with Down Syndrome entails a very complex challenge: they have the capacity to be motivated and show signs and behaviours that demonstrate their motivation, but have a low level of wakefulness and their reaction capacity to satisfy a need might be less (Flórez and Troncoso, 2001; Valverde-Montesinos, 2006). Motivations depend on the spaces they develop in and the level of early stimulation. Besides, the fact they show difficulty towards effort generates an overprotection in their context that delays developing motivation behaviours even more.
The linguistic level of this collective is inferior than their social and intellectual competence. The social age of individuals with Down Syndrome is higher than their mental age. They use body language considering the difficulties to communicate verbally in a fluid manner. Valderde-Montesinos (2006) describes the language problems of individuals with Down Syndrome regarding:
3.3. Augmented Reality, Education and Down Syndrome
The digital gap between healthy individuals and individuals with intellectual disability have been object of attention in different public institutions and private entities working in projects which purpose has been approaching the advantages and possibilities of the Society of Information to these collectives (Ballestero, 2002).
A project of great relevance is the Poseidon Project, promoted by a consortium comprised of small and middle-sized companies, university and research institutes and Down Syndrome associations of Germany, United Kingdom, Norway and Sweden. Its objective is the development of smart and customized environments so to achieve a greater level of autonomy and independence in daily life (home, work, education, leisure…), as well as to facilitate their relationship with others.
The III International conference on adaptation and accessible virtual learning environment recently held in Cartagena de Indias (Colombia) has evidenced the huge potential of Augmented Reality in inclusive education (VVAA, 2016). The promoter of Aumenta.me Project, Raúl Reinoso, mentions different potentialities of Augmented Reality in Education: visualization and interaction with 3D models, creations of books, cards and multimedia resources, learning based on game, learning based on discovery, development of abilities and skills, distance education and e-learning (Reinoso, 2016).
On the other hand, Pajares (2015) after an exhaustive analysis of almost half a hundred didactic activities present in the Auméntame 2015 Meeting, concludes that there are significative relationships between the way this technology is used and other variables such as Bloom taxonomy, applicated methodology, students and professors’ role, curricular approach, learning theories and augmented reality tools used. Consequently, any evaluation of the AR experience as a learning and communication strategy should consider these aspects.
The exploratory nature of the study recommends a methodological design based, firstly, in qualitative techniques. Initially, in-depth interviews were conducted with different experts and professionals of “Foundation for Down Syndrome and other intellectual disabilities of the Basque Country”, which collaboration has been strategical in our research. The information obtained has been determinant for the design of the subsequent experiential phase. On the other hand, direct observation was used both for approaching and familiarizing with the subjects in different occupational workshops organized by the Foundation, and to gather information during the experience and the later evaluation.
The core element of our study is the design, execution and evaluation of an Augmented Reality experience of adult individuals with Down Syndrome. It consisted in the visit to two museums in Bilbao, where participants must use Augmented Reality for moving from one place to another and to learn contents relative to museums.
The selection of participants was done through the mentioned Foundation, which works with a total of 354 individuals with different intellectual disabilities, out of which 244 suffer Down Syndrome, among them 113 adults (59 men and 54 women). They were provided a recruitment questionnaire to select subjects depending on:
Considering the type of pathology of this collective, the filling of the questionnaire required assistance and support in understanding questions or in the expression of answers. This data gathering was conducted in interviews developed for several days through the attendance to three workshops organized depending on the cognitive maturity (Day Attention Service, Labour Education and Self-managers Association). These contacts enabled the creation of a closer bond, start explaining what was the experience about and facilitate subsequent phases. All individuals selected responded the questionnaire and showed their interest and will for collaboration. Finally, in the experience participated 15 adult individuals with Down Syndrome, divided into three homogeneous groups depending on the level of cognitive maturity and familiarization in the use of CIT tools.
Participants were summoned together with the support staff of the Foundation in the Guggenheim Museum of Bilbao, where they should walk on foot to the Museum of Fine Arts, both in the centre of the city and separated by 400 meters. In each museum, participants received written and oral information by a tour guide.
The AR application had the objective of orientating them in going from one place to another, besides providing extra audiovisual contents about both museums. This prototype of AR for mobile devices was developed together with the laboratory of technological innovation of Universidad de La Matanza (Argentina) and the company Ialcubo.
The application had two functionalities: geopositioning, developed on Layar , to help with the spatial orientation, and patterns recognition to visualize a video. The user interface was designed with a traditional development logic allowing a positive user experience, therefore the access icon to the application was the Foundation logo, an image that can be quickly identified by users. The player icon was enlarged x2 of its size by default.
The evaluation of the experience has been an “evaluation by and from users” (Cabero-Almenara et al, 2017) and, like in other researches analysing the user experience, the design of applications or utility, was done through two techniques: direct observation from researcher, in-depth interviews and discussion groups with participant . These last two conversational techniques obviously had limitations derived from the pathologies of the study participants.
Regarding the ethical dimension of the study, we are aware about being analysing a particularly vulnerable collective. We have complied with broad consensus guidelines such as the informed consent, confidentiality and privacy guarantee or the risk-benefit balance, always under the supervision of “Foundation of Down Syndrome of the Basque Country”. However, as Santi (2015) says “it is about looking for a balance between respect and the protection of these people and groups, and the commitment and need to perform research to know more about them and to collaborate in their empowerment.” (Santi, 2015:71)
5. Analysis of results
During the experience, participants showed a superior capacity to recall images when they could associate them with daily life scenes, therefore this was the resource used to fix concepts. A frequent attitude in these individuals is the fast dispersion or loss of attention whenever there is saturation of information, or an external agent like the environment surrounding us in the public space.
Likewise, social over-exposure generated a noticeable emotional contraction in several participants: from the start of the exposure, there was an atmosphere of having to approve the experience, a feeling of pressure that was a negative factor of the exposure. The lack of personal confidence reflected in the fear of making mistakes was a constant in the group. The three participant groups, with different levels of cognitive maturity, tried to achieve approval, which led to a certain blockage at some point.
Another constant in the experience was the confidence with which participants confirmed to understand concepts related to the use of AR; However, they had difficulties explaining said concepts, therefore silence was often the response.
The aspects that made the experience more difficult was the discomfort of needing to move around with a non-quotidian element, the excess of uncontrolled stimuli outdoors and the lack of familiarity and practice with the Layar application.
The visualization of audiovisual contents in public spaces and in front of real objects generated a positive impact that worked as stimulus for concentration, which favoured long-term recall. It happened mainly when participants were in front the dog Puppy of the Guggenheim Museum or in front the Fine Arts Museum.
Participants of the second and third group -with a greater technological imprinting and greater use of CIT tools– understood the iconic languages of the applications and procedures in a more intuitive and natural manner, although in some occasion they did it with help. The logic of the architecture was comprehended by all participants, who naturally understood and accepted that the itinerary was marked and visualized through the application browser. The interpretation of icons and symbols was one of the most difficult points of interpretation in this collective.
The information provided through audiovisual elements was recalled longer than the one provided through textual/ verbal communication. During the group evaluation, for example, it was recalled that “the dog Puppi had the flowers changed” or “we saw naked women.”
One of the participants stood out, in an outstanding manner, by suggesting improvements in the application, such as enlarging maps and texts, as well as to reinforce audiovisual contents with texts or increase the number of explanatory images. On her own, she quickly solved the browsing of applications thanks to her previous knowledge in browsing systems. This user of the Foundation has studied her pathology and she assumes herself as disabled. She shows open to continue with further Augmented Reality experiences and ensures us that audiovisual contents of the application will help her solve tasks in her job.
6. Discussion and conclusions
The conclusions of this exploratory study intent to be the baseline for reflection and improvement in the development of Augmented Reality applied to individuals with Down Syndrome, as well as an unavoidable starting point for the formulation of research questions and hypothesis of a wider and more structured research. The dimensions of the experience carried out, make advisable its replication in larger size samples and in different contexts to be able to confirm hypothesis, but allow us to advance conclusions about the objectives proposed at the start of the research.
Audiovisual tools provided by Augmented Reality generate a high emotional impact and are determinant to recall ideas and concepts in individuals with Down Syndrome. The maximum attention and concentration is produced during the experience of patterns recognition, a fact that was not only observed during the experience itself, but also confirmed by participants in the discussion.
A lesser level of cognitive maturity or a lesser use of CIT tools does not hinder assuming Augmented Reality. All experience participants completed the experience just as it was designed. There are no difficulties to interpret the juxtaposition between reality and the images generated by a computer in real time or to target the mobile camera to images so to reproduce the associated contents. Nevertheless, it is advisable to have an assistant or guide just as it is done in other activities they develop.
All participants intuitively understood what was the button to start the audiovisual contents, just as it was also intuitive, recognizing an arrow to understand what was the icon representing them while moving around (GPS).
Long-term memory is clearly related with the visual elements of AR. However, contents provided through written texts or sounds were scarcely recalled. Often, talking texts produced tiredness and overload in participants, who focus their attention on illustrative images instead and, to a lesser extent, in superimposed texts. The confirmation of hypothesis H5 regarding long-term memory would need additional evaluation methods.
The level of cognitive maturity, autonomy and insertion in social life determines a more or less exploitation of the different uses of AR. Participants with less cognitive maturity can develop the activity, but do not seem to be able to relate it with other activities of everyday life. The dimension of our study didn’t allow us to confirm whether a training with AR devices would enable these individuals to incorporate it to their everyday life, for example, as a support tool in orientation and moving outdoors.
Participants with more maturity are able to interpret the functioning of the application, move about in an autonomous manner, delve in an intuitive and ludic manner in the possibilities of the application without fear of mistaking, and even, develop a satisfactory user experience transferable to other fields such as labour.
The previous use of CIT was determinant for a faster and more comfortable incorporation of AR. Assimilation of concepts was more efficacious in individuals used to touchscreens, connected devices or participation in social networks. Nevertheless, the rest of subjects were able to interpret how to execute Augmented Reality commands. Consequently, Augmented Reality presents relevant possibilities of improvement in learning and communication in individuals with Down Syndrome. A greater digital alphabetization as well as the acquisition of usability protocols specific for individuals with Down Syndrome are strategical and indispensable to overcome the digital gap existing between healthy individuals and individuals suffering this intellectual disability.
 This foundation develops different programs of early attention, physiotherapy, game groups, support to reading-writing, labour insertion, school follow-up, psychotherapy, speech therapy, etc. Programs such as Hazi for enhancing communicative strategies, Program for Personal Autonomy or Program for Leisure are directly related to the objectives of our study.
 Software of Dutch origin that provides architecture for the development of AR applications. It uses camera, GPS, compass and mobile acceleration sensor. Using GPS or mobile network data, the app localizes the exact position and overlaps to the vision of the camera in the path.
 These focus groups were done immediately after the tour. Initially they should have been done in Centro Cultural de la Alhóndiga. But due to the high temperatures recorded that day, it was decided to go back to the facilities of the Foundation where participants could express themselves more calmly.
8. List of references
RT Azuma (1997): “A survey of augmented reality”. Presence, 6(4), pp. 355-385.
F Ballestero (2002): La brecha digital. El riesgo de exclusión Información. Madrid: Fundación Retevisión-Auna.
X Basogain, M Olabe, K Espinosa, C Rouèche & JC Olabe (2007): “Realidad Aumentada en la Educación: una tecnología emergente”. Comunicación presentada a la 7ª Conferencia Internacional de la Educación y la Formación basada en las Tecnologías. Madrid. Recuperado de
O Bimber & R Raskar (2005): Spatial augmented reality: merging real and virtual worlds. CRC press. Wellesley, Mass: A K Peters.
M Billinghurst, A Clark & G Lee (2015). “A survey of augmented reality”. Foundations and Trends® Human–Computer Interaction, 8(2-3), 73-272.
M Billinghurst, H Kato & I Poupyrev (2001): “The MagicBook - Moving Seamlessly between Reality and Virtuality”. IEEE Computer Graphics and Applications, 21(3), 6-8.
J Cabero-Almenara et al (2017): “Evaluación por y desde los usuarios: objetos de aprendizaje con Realidad aumentada”. RED. Revista de Educación a Distancia, 53. Recuperado de http://www.um.es/ead/red/53/cabero_et_al.pdf
TP Caudell & DW Mizell (1992): “Augmented reality: An application of heads-up display technology to manual manufacturing processes”. System Sciences, 1992. Proceedings of the Twenty-Fifth Hawaii International Conference on (Vol. 2, pp. 659-669). IEEE.
HL Chi, SC Kang & X Wang (2013): “Research trends and opportunities of augmented reality applications in architecture, engineering, and construction”. Automation in construction, 33, pp. 116-122.
J Flórez & MV Troncoso (dir.) (2001): Síndrome de Down y educación. Barcelona: Masson S.A., Santander: Fundación Síndrome de Down de Cantabria.
E Galán Cubillo (2008): "Escenografía virtual en TV. Análisis del uso de escenografía virtual en la realización de un programa de televisión", en Revista Latina de Comunicación Social, 63, pp. 31-42. La Laguna (Tenerife): Universidad de La Laguna. Recuperado de http://www.ull.es/publicaciones/latina/_2008/04/Galan_Cubillo.html
BPM Haro, PCS Mancilla & MAG Ruiz (2012): “Uso de interfaces tangibles en la enseñanza de lectura a niños con síndrome de Down”. El Hombre y la Máquina, 39, pp.19-25. Recuperado de http://www.redalyc.org/articulo.oa?id=47824590004
MC Juan et al. (2005): “Using augmented reality to treat phobias”. IEEE computer graphics and applications, 25(6), pp. 31-37.
H Kato & M Billinghurst (1999): “Marker tracking and hmd calibration for a video-based augmented reality conferencing system”. Augmented Reality, 1999.(IWAR'99) Proceedings. 2nd IEEE and ACM International Workshop on. IEEE.
H Kaufmann & D Schmalstieg (2003): “Mathematics and geometry education with collaborative augmented reality”. Computers and Graphics (Pergamon), 27 (3), pp.339-345.
CD Kounavis, AE Kasimati & ED Zamani (2012): ”Enhancing the tourism experience through mobile augmented reality: Challenges and prospects. International Journal of Engineering Business Management, 4, 10.
I Martínez (2010): Realidad Aumentada Móvil para la Conservación del Patrimonio (tesis doctoral). Leioa: Universidad del País Vasco/Euskal Herriko Unibertsitatea.
DD McMahon, DF Cihak, RE Wright & SM Bell (2016): “Augmented reality for teaching science vocabulary to postsecondary education students with intellectual disabilities and autism”. Journal of Research on Technology in Education, 48(1), pp.38-56. Recuperado de http://www.tandfonline.com/doi/full/10.1080/15391523.2015.1103149?scroll=top&needAccess=true
P Milgram, H Takemura, A Utsumi & F Kishino (1995): “Augmented reality: A class of displays on the reality-virtuality continuum”. Photonics for Industrial Applications, pp. 282-292. International Society for Optics and Photonics.
P Milgram & F Kishino (1994): “A taxonomy of mixed reality visual displays”. IEICE TRANSACTIONS on Information and Systems, 77(12), pp. 1321-1329.
C Portalés Ricart (2008): Entornos multimedia de Realidad Aumentada en el campo del arte (tesis doctoral). Valencia: Universitat Politècnica de València.
R Raskar, G Welch y H Fuchs (1998): “Spatially augmented reality”. First IEEE Workshop on Augmented Reality (IWAR’98), pp. 11-20.
R Reinoso (2016): “Realidad aumentada posibilidades y usos educativos” en Baldiris, S. y otros (eds). Recursos Educativos Aumentados Una oportunidad para la inclusión, pp. 8-29. Colombia: Sello Editorial Tecnológico Comfenalco.
MF Santi (2015): “Vulnerabilidad y ética de la investigación social: perspectivas actuales”. Revista Latinoamericana de Bioética, 15 (2), pp. 52-73. Recuperado de http://www.redalyc.org/pdf/1270/127040727005.pdf
IE Sutherland (1968): “A head-mounted three dimensional display”. Proceedings of the December 9-11, 1968, fall joint computer conference, part I, pp. 757-764.
DR Torres (2011): “Realidad Aumentada, educación y museos”. Revista Icono14. Revista científica de Comunicación y Tecnologías emergentes, 9(2), pp. 212-226. Recuperado de http://www.icono14.net/ojs/index.php/icono14/article/view/24/42
S Valverde Montesinos (2006): El aprendizaje de las Tecnologías de la Información y la Comunicación en personas con síndrome de Down (tesis doctoral). Madrid: Universidad Complutense de Madrid: Servicio de Publicaciones. Recuperado de http://biblioteca.ucm.es/tesis/edu/ucm-t28915.pdf
VVAA (2016): Recursos Educativos Aumentados. Una oportunidad para la Inclusión. Cartagena de Indias: Sello Editorial Tecnológico Comfenalco. Recuperado de http://cava2016.com/wp-content/uploads/2016/10/REAumentados.pdfZ Zhu et al. (2015): “AR-Weapon: live augmented reality based first-person shooting system”. Applications of Computer Vision (WACV). IEEE Winter Conference on, pp. 618-625. IEEE.
How to cite this article in bibliographies / References
RM Martín-Sabarís, G Brossy-Scaringi (2017): “Augmented Reality for Learning in People with Down Syndrome: an exploratory study”. Revista Latina de Comunicación Social, 72, pp. 737 to 750.
Article received on 20 December 2016. Accepted on 17 June.