ADL Newsletter for Educators and Educational Researchers

Games Break Free: Augmented and Alternate Reality Gaming

Daniel Livingstone
School of Computing
University of the West of Scotland
UK

Jonathon Richter
Center for Learning in Virtual Environments
University of Oregon
USA

Introduction

Over the past decade, digital games for training and education have become increasingly embedded in formal, informal, corporate and military settings. Games cover a wide range of disciplines and come in many forms. Computer Games and Instruction (Tobias & Fletcher, 2011) describes the research dealing with a wide range of the games currently available. Most work to date is based on the traditional desktop gaming metaphor – learners use workstations or consoles to play through a variety of games and scenarios, either individually or (less commonly) as members of a team in a networked game. But other forms of digital – or digitally supported – gaming exist which break free from the desktop constraints. In our chapter on Multi-User Games (Richter & Livingstone, 2011), we provided a very brief introduction to two areas of interest that have been barely explored: Augmented Reality (AR) and Alternate Reality Games (ARG). We find this to be a worthwhile overview of these types of games because of their potential and their emerging uses for developing learning games and the challenges already documented in implementing them as successful learning activities.

Augmented and Alternate Realities

As the names imply, both AR and ARG use the real world as their setting – though in quite different, but possibly overlapping, ways. Where AR adds a layer of virtual information – potentially including audio and video data – to the real world as seen through a digital camera, ARG use a range of media to create an imaginary world that intersects our own, and which may reach into the players’ world in surprising ways. A number of commercial and critical success stories illustrate the power of both – but only limited examples have so far emerged with educational applications of these technologies.

It is worth noting that the commonplace use of AR tends to refer strictly to visually augmented reality – where virtual content is typically overlaid on images from a live camera input, and this is the sense with which the term AR is used on Wikipedia and by many phone apps such as Layar or Wikitude. A somewhat broader definition of AR is used by Squire and Jan (2007) who use the term to refer to games played in real world contexts with an added fictional narrative layer. This definition somewhat blurs the distinction between AR and ARG, and while it has been adopted for other notable projects, such as HARP (O'Shea et al. 2009), we will limit our definition of AR to applications which augment live video with virtual data.

Current Technologies and Solutions for AR

AR combines video input and computer generated overlays to present an augmented view of the real world. Milgram's Reality-Virtuality continuum (Milgram et al., 1994) places augmented reality on a continuum between real environments and purely virtual environments, such as those typically found in computer games. These might simply use a camera and screen to place some descriptive or informative text next to a view of a museum artifact (Wojciechowski et al., 2004). Alternatively, virtual creatures might be placed in secret locations around a house –only becoming visible when the view is mediated using a handheld computer as a camera, as in the Sony PSP game Invisimals. These examples use visual markers which allow software to position the camera in a 3D scene, and to appropriately render the virtual content over the captured images.

GPS positioning data can be combined with orientation data from internal sensors to locate a user and determine where the camera is pointing – and this can be used to determine what virtual content should be added to the current view. A user can wander around an area viewing the world through the screen of their smartphone or PDA as data about points of interest, or property for sale are layered over the camera input. This was a fantastic concept just a few years ago; now there are multiple toolkits and software libraries and solutions available to allow rapid development of such applications – ARToolKit, Layar, Wikitude, Junaio and many more. ARDrone is a camera equipped radio controlled helicopter which streams video to, and is controlled from, a nearby iPhone. This modestly priced toy for ordinary consumers can also augment the video feed with virtual targets and obstacles to create a range of games, and a developer app is available to enable the creation of third party applications.

All of these AR applications use the camera to add virtual information over live images, but there are other ways to augment reality, such as by adding virtual soundscapes or information spaces, although these do not fall within the strict definition of AR used here. Use of QR codes and other markers on physical objects, data fed to users via augmented map displays, or other non-video based applications stretch the definition of AR. There is an admitted need for scholars and practitioners to better frame and share common language in this rapidly emerging area.

AR Games in Education

A range of work has applied AR to educational or learning contexts – though often as a means to display additional information or 3D views of objects rather than through some form of augmented reality gaming. For example, augmented reality books may include printed markers which allow readers to view and manipulate 3D objects related to the text using their handheld computers or smartphones. One example of this is Second Sight, a PlayStation Portable application with a PC based authoring tool for schools. This allows teachers to link video and 3D content to visual markers which can then be placed in text books and used as triggers to launch the enhanced content where appropriate. Other educational and commercial uses have varied from applications allowing medical students to view skeletons inside moving bodies to applications that show live technical guidance to automotive engineers (Knopfle, et al. 2005).

Early examples tended to be restricted to enclosed settings - in large part due to the need to have access to computers powerful enough to process the video data (for example, see Billinghurst, 2002). Examples of mobile AR games for education are reported by Klopfer et al. (2005), who outlines three different collaborative AR games for learning – Environmental Detective, Charles River City and Mad City Murder. These team based AR games differentiated player roles and abilities and used carefully set problems to promote and require cooperation between players to solve puzzles. But while these games engendered some amount of cooperation (forced by the differentiated roles), the actual effects on learning outcomes are not reported. Museums and other educational sites such as science centers or zoos have proven popular locations for creating AR enhanced experiences (for example: Perry et al., 2008; Woods et al., 2004; Schmalsteig & Wagner, 2005).

Two case-studies showing how AR games have been used to enrich school field trips to zoos are reported by Perry et al. (2008). While the findings here are quite exploratory, some interesting issues are identified. For example, when using handheld devices to implement the AR games, some explanation is required so that the devices themselves do not break the game narrative. So, in a zoo based game, a PDA might be an advanced translator that allows students to understand what animals might be saying to them. The game and user interface design also needs to ensure that the learning activity takes precedence over the use of technology – an observation that applies equally to a wide range of technology enhanced learning activities and media (c.f. Laurillard et al. 2000).

The Handheld Augmented Reality Project, or HARP, at Harvard University developed a science inquiry AR game with an Alternate Reality narrative for middle school students called Alien Contact! (O'Shea, et al. (2009). This game emphasised collaboration, math, and literacy skills while engaging students in solving the riddle as to why aliens are “here” on earth. The Alien Contact! case study generated a number of design heuristics of possible interest for future developers of educational AR Games, including the highlighted ease of motivating students within such real world, playful narratives, the noted difficulty in managing the hardware and software needed to deliver such an experience, and issues of cognitive overload that mounted for some participants – resulting in their giving up in frustration, despite their being motivated by the narrative and the task.

Alternate Reality Gaming

Almost any successful computer game generates some form of online community and support or social interactions around the game. Gee (2011) distinguishes games themselves from the social interaction systems that emerge around games, which he calls the "big 'G' Game". ARG, more than any other form of gaming, use such meta-games as a core gameplay feature.

Apparently unconnected web-pages, instant-messages, phone calls, poster adverts, voice calls and interactions in physical spaces may all be used to establish an alternative reality that intersects the digital and physical spaces that we live and play in. The I Love Bees ARG (Kim, Allen and Lee, 2008) was used to raise public awareness of the game Halo 2 in 2004, and to create interest and excitement prior to the launch of the game.

I Love Bees began with cinema trailers for the game briefly showing the www.ilovebees.com URL. Anyone intrigued enough to then visit the web-site found what appeared to be the strangely corrupted homepage of an amateur bee keeper. The supposed site-administrator posted a plea for help on the page, and answered a few email questions before seemingly going into hiding, leaving only the corrupted data on the site as clues. A large text file of numbers was found to be a large set of map coordinates of public telephones, while other clues hinted at dates and times of calls. Crowds turned up to answer phones and receive messages across the USA. Recordings were then released online, building up the story of a rogue AI from the Halo universe that had crash landed on Earth. Making sense of the narrative involved collaboration between thousands of game players. Players created web-sites and forums and used internet chat-rooms to try to crack the game's puzzles.

With a story that relies on users collaborating online to solve puzzles and on some players being in a particular physical location at a particular time, ARGs need to be able to react if players solve problems too rapidly or appear to get collectively stuck. Developers respond by amending, adapting or creating new challenges as needed:

"You're coming up with about 5,000 little games, but then you're always leaving about 20 per cent of your budget to create content on the fly in response to the audience... it's a performance. It's not a canned piece of software." (Weiseman 2009, p.83)

Despite the extensive use of collaborative problem solving typical of ARG, there have been few attempts to integrate ARG into formal education. One notable example, ARGOSI - Alternate Reality Games for Orientation, Socialization and Induction, not only used ARG in a higher education setting, it also produced a toolkit to help others develop similar ARG for their own institutions (Whitton, et al. 2008). ARGOSI uses a range of games and challenges requiring student collaboration to help introduce new students to university. Through the use of collaborative challenges, students were not only inducted into university life through the game, they were also able to meet, and form bonds with, their new colleagues.

A more formal application of ARG was taken in a pan-European language learning project. Connolly et al. (2009, 2011) developed and evaluated the Tower of Babel ARG to support high-school language learners. This large pilot involved over 300 children and almost 100 teachers across 25 schools in 17 different countries - and had promising results. While subsequent studies are planned to extend this work to other subject areas, it is clear that there is some distance to go in exploiting ARGs for teaching and learning. Issues such as developing systems that can respond to variations in how different groups of users respond, with limited resources in comparison to those available to commercial projects like I love bees will be challenging.

Unlike virtual worlds which collapse distance, mobile AR gaming and ARG typically layer a game-space over a distinct physical location – which may create further challenges in developing cost effective, reusable games for learning that fully exploit these technologies, games that can be mapped from one site to another without significant cost and redevelopment. While our understanding of how mobile multi-user games can work and how they should be designed for maximum engagement and effectiveness is in its infancy, these may prove to be highly disruptive innovations that profoundly change expectations of formal education in years to come.

Conclusions

Augmented Reality and Alternate Reality Gaming applications both have obvious allure to engage students in compelling ways directly within a variety of environments where learning is sought. By situating digital content within the world around us we can amplify, nudge, or add to our physical surroundings and thus, extend information, engagement, and narrative in ways that provide potentially highly effective educational and training opportunities. These innovations allow us to participate directly within the physical environment while simultaneously engaging with other media or collaborating with others. Such promise, like many other emerging technologies, is rife with challenges in designing a usable interface, managing the technical infrastructure, and in managing cognitive overload and adapting to unanticipated consequences (O'Shea, et al., 2009). Despite these challenges, the promise to education as well as enterprise are thought to be substantial enough to merit a positive forecast from the 2011 New Media Consortium’s widely read Horizon Report (Johnson et al., 2011) that Augmented Reality and Game-based Learning are within the two to three-year forecast for adoption in formal education and training. We’re thus well advised to keep one eye on the screen and another on the changing world around us.

References

Billinghurst, M. (2002). Augmented reality in education. New Horizons for Learning.

Connolly, T., Stansfield, M., Hainey, T., Cousins, I., Josephson, J., O'Donovan, A., Rodriguez Ortiz, C., et al. (2009). ARGuing for multilingual motivation in Web 2.0: an evaluation of a large-scale European pilot. In Proceedings of the 3rd European Conference on Games-based Learning. Graz, Austria.

Connolly, T. M., Stansfield, M., & Hainey, T. (2011). An alternate reality game for language learning: ARGuing for multilingual motivation. Computers & Education, 57(1), 1389-1415. doi:10.1016/j.compedu.2011.01.009

Gee, J. P. (2011). Multi-User Games and Learning: A Review of the Research. In S. Tobias & J. D. Fletcher (Eds.), Computer Games and Instruction (pp. 223-232). Information Age Pub Inc.

Johnson, L., Smith, R., Willis, H., Levine, A., and Haywood, K., (2011). The 2011 Horizon Report. Austin, Texas: The New Media Consortium. Downloaded May 10th, 2011 from http://wp.nmc.org/horizon2011/

Kim, J. Y., Allen, J. P., & Lee, E. (2008). Alternate reality gaming. Commun. ACM, 51(2), 36-42. doi:10.1145/1314215

Klopfer, E., Perry, J., Squire, K., & Jan, M. (2005). Collaborative learning through augmented reality role playing. In Proceedings of the 2005 conference on Computer support for collaborative learning: learning 2005: the next 10 years! (pp. 311-315). Taipei, Taiwan: International Society of the Learning Sciences.

Knopfle, C., Weidenhausen, J., Chauvigne, L., & Stock, I. (2005). Template based authoring for AR based service scenarios. Virtual Reality, 2005. Proceedings. VR 2005. IEEE (p. 237–240).

Laurillard, D., Stratfold, M., Luckin, R., Plowman, L., & Taylor, J. (2000). Affordances for Learning in a Non-Linear Narrative Medium. Journal of Interactive Media in Education, (2), 1-19.

Milgram, P., Takemura, H., Utsumi, A., & Kishino, F. (1994). Augmented reality: A class of displays on the reality-virtuality continuum. Proceedings of Telemanipulator and Telepresence Technologies (p. 282–292).

O'Shea, P., Mitchell, R., Johnston, C., & Dede, C. (2009). Lessons learned about designing augmented realities. International Journal of Gaming and Computer-Mediated Simulations, 1(1), 1-15, January-March.

Perry, J., Klopfer, E., Norton, M., Sutch, D., Sandford, R., & Facer, K. (2008). AR gone wild: two approaches to using augmented reality learning games in Zoos. In Proceedings of the 8th international conference on International conference for the learning sciences - Volume 3 (pp. 322-329). Utrecht, The Netherlands: International Society of the Learning Sciences.

Schmalstieg, D., & Wagner, D. (2005). A handheld augmented reality museum guide. Proceedings of IADIS International Conference on Mobile Learning (Vol. 49).

Richter, J. & Livingstone, D. (2011). Multi-user games and learning: A review of the research. In Tobias, S., Fletcher, J. D. (Eds.), Computer games and instruction. Information Age Publishing, Charlotte, NC (pp. 101 - 126).

Squire, K. D., & Jan, M. (2007). Mad City Mystery: Developing Scientific Argumentation Skills with a Place-based Augmented Reality Game on Handheld Computers. Journal of Science Education and Technology, 16(1), 5-29.

Tobias, S., & Fletcher, J. D. (2011)(Eds.), Computer games and instruction. Information Age Publishing, Charlotte, NC

Weisman, J. (2009, December). Traveller: Interview with Jordan Weisman. Edge, (208), 78-83.

Whitton, N., Wilson, S., Jones, R., & Whitton, P. (2008). Innovative induction with Alternate Reality Games. In Proceedings of the 2nd European Conference on Game-Based Learning. Barcelona.

Wojciechowski, R., Walczak, K., White, M., & Cellary, W. (2004). Building Virtual and Augmented Reality museum exhibitions. Proceedings of the ninth international conference on 3D Web technology, Web3D ’04 (p. 135–144). New York, NY, USA: ACM.

Woods, E., Billinghurst, M., Looser, J., Aldridge, G., Brown, D., Garrie, B., & Nelles, C. (2004). Augmenting the science centre and museum experience. Proceedings of the 2nd international conference on Computer graphics and interactive techniques in Australasia and South East Asia, GRAPHITE ’04 (p. 230–236). ACM. doi:10.1145/988834.988873