Getting Started with Virtual Reality as Assistive Technology

Getting Started with Virtual Reality as Assistive Technology By Don McMahon

emerging technology

Getting started with Virtual Reality as Assistive Technology

New technologies such as Immersive learning tools like aug- mented reality and virtual reality have many potential applica- tions as assistive technologies to support the needs of individ- uals with disabilities. Like many new technologies though, early adoption and use is tending towards video gaming and enter- tainment applications with some educational uses beginning to get established. Unfortunately, the accessibility features and potential assistive technology applications of these tools are still in a very early phase of development. At the Assistive Technology Research and Development Lab at Washington State University our primary goal is exploring the capabilities of these new technologies and matching them to the needs of individuals with disabilities. We believe that the best way to support early adoption of these innovative technol- ogies is to get them into schools, hospitals, communities, and homes while working to educate our community of stakehold- ers about what is available on these platforms to support the needs of people with disabilities. Brief background on AR and VR There are a multitude of terms in the immersive learning field but the primary ones are augmented reality (AR) and virtual reality (VR). In the fields of computer science and engineering

these terms are well established and defined in the mixed reality taxonomy (Milgram & Kishino, 1994) which is shown in Figure 1 below. Starting at the left side is actual reality which hopefully all of us can recognize most of the time. AR is a live view of the physical world with some digital information overlaid on top of it. Augmented virtuality is a term rarely used now and is gener- ally considered high quality AR with 3D objects. Finally, VR is on the right side of the taxonomy and is defined as a fully artificial digital environment in which a user navigates in order to com- plete tasks or gain experiences.

Mixed Reality Continuum

These terms can get confusing and are often misused or conflated with other technologies. For example some previous

Dr. Don McMahon is an Assistant Professor of Special Education Technology at Washington State Univer- sity in Pullman, WA. His research interests include, Augmented Reality, Virtual Reality, Wearable devices, Universal Design for Learning, Assistive Technology, and Instructional technology. At WSU Don McMahon coordinates the Assistive Technology Research and Development Lab. He recently was one of the authors of a handbook on using mobile devices in secondary classroom settings. In addition he is one of the Cofounders of the WSU ROAR postsec- ondary education program for students with intellectual and developmental disabilities at Washington State University. Don was a special education teacher for 7 years and then became an instructional Co-teaching coach for general education and special education teachers. Don graduated from the Special Education Ph.D. program at the Univer- sity of Tennessee, Knoxville. Don is also a national presenter providing professional development sessions for both general educators and special educators on using technology to support diverse groups of learners.

BACK TO CONTENTS

assistive technology and special education technology work has used the terms like virtual worlds (like the computer games played on computer screen or tv) interchangeably with virtual reality. One of the primary things that separates VR with these virtual world games is that the user views these experiences with a head mounted display which provides a 360 degree ex- perience in the virtual environment. Understanding what is and is not VR is an important first step in advancing more ways to use VR as an assistive technology. In comparison, AR as an assistive technology is probably in a more mature state of development than VR. The 2016 cultural phenomenon of Pokemon Go at least helped to grow the popular understanding of what AR is. VR has not hit that type of widespread adoption yet. AR research in spe- cial education and assistive technology is also more common currently than VR examples. Luckily as a field of AT profession- als there are many ways we can advance VR adoption outside of waiting for the VR tools designed with accessibility at the fore- front and waiting for a larger body of research to provide lessons on the best practices. Capabilities and Limitations of VR So as we explore what VR can bring as an AT resource, un- derstanding how it is used is an important consideration of its capabilities and limitations. VR like any potential tool used as an assistive technology is not universally going to be an ideal tool for all individuals. This will improve as more developers design for accessibility from the start but many current VR offerings are limited in their accessibility. There are some important efforts to improve accessibility in VR. For example, the University of Mel- bourne provides a great resource very detailed examination of VR accessibility challenges and opportunities across many dif- ferent types of disability needs (https://unimelb.edu.au/accessi- bility/guides/vr-old). The best way to learn about VR capabilities and limitations is to start trying it out. Once you try it you immediately appreciate that VR is a very visually focused medium. While head mounted displays and VR experiences also include sounds and sometimes haptic feed- back from the controllers, being able to see is still a very central assumption being made by most developers. So VR in its cur- rent form is not likely to be an ideal resource for individuals with vision impairments. Even users with glasses may find using VR head mounted displays a challenge to use comfortably. Another limitation is that VR experiences such as games, videos, and educational simulations each vary by their design features for using and navigating the experiences. Just as there are millions of mobile apps there are currently thousands of VR games and experiences which can all vary in terms of accessibil- ity. Some VR experiences are very text heavy in their instructions and interfaces which could be a barrier for users who need assis- tance in reading. Common screen reading tools and accessibility features generally will not work in these 360 degree VR environ- ments. Some provide on screen text instructions but no options

for audio prompts. VR by its nature is generally a stationary activity. When we teach people about AR and VR one of our favorite examples is that you can’t cross the street in VR. While using AR apps (i.e. Pokemon Go, AR Navigation apps) most of us are able to safe- ly move around our communities and find what we are looking for in the physical environment around us. VR’s head mounted displays showing completely virtual environments make physi- cally moving any distance potentially dangerous. That does not mean users can’t stand or even in some cases move around a small area in what is generally called room scale VR. But even in most room scale VR experience a participant uses a fairly small area of no more than 10 by 10 feet. A great example of this is Google Tilt Brush which is a creative drawing and painting app that uses one controller as the “brush” and one controller as an interactive palette of colors and tools. Because VR is primarily a stationary activity that also means that for individuals with mobility challenges, for example using a wheelchair, it can still be very accessible. Some VR systems games require one hand held controller while more advanced ones use two controllers. A user with more limited mobility, for example a patient in a bed, would be better serviced by a system with one controller to se- lect and interact in VR. For example, the Oculus Go uses a single controller and the games and experiences in this system tend to require less movement. 360 Degree Video One of the easiest ways to get started using a VR headset is to use it to watch existing 360 degree videos. These videos are tak- en with a special camera that captures video in a 360 circle that can be watched in the VR headset. Unlike a traditional video the user can choose what to pay attention to in these immersive vid- eos. For example in a 360 video swimming under the ocean, one person might focus on a sea turtle swimming by and another might choose to focus on a nearby clown fish. Some popular 360 degree video apps in VR are VEER (Free), National Geographic VR (Free), and Discovery VR (Free). But the largest collection of 360 video is also the world’s largest collection of video Youtube on the Youtube VR app on which you can also upload your own immersive 360 videos. Creating your own 360 degree videos to make immersive video models is one of the most promising assistive technology applications of VR. Video modeling is an evidence based prac- tice with decades of research that we can easily extend to this new platform of VR. The 360 cameras are available for as little as $200 to $500 and very easy to operate. Some use a single lens to create a flat standard video while other cameras use two lens to create 3D video. In our experiments we prefer systems that cre- ate 3D video for the VR video models. We also suggest putting the 360 camera on a stand or selfie stick to help keep the image steady because your video will be what the user experiences and too much turning and shaking can be very disorienting. We

www.closingthegap.com/membership | June / July, 2019

BACK TO CONTENTS

Closing The Gap

VR as Assistive Technology: WSU Assistive Technology Research & Development Lab https://www.youtube.com/watch?v=HvamGtYg_mw

VR Platforms After we understand the capabilities and limitations of VR broadly we need to choose some VR platforms to get started discovering the assistive technology of VR. The easiest way to get started using VR involves your smartphone’s screen which is placed in cardboard or plastic headsets ($5-10) to create smart- phone based VR experiences. Students and educators can get started using these VR experiences by searching their mobile devices app stores such as Google Play or Apple’s App Store for the search term “virtual reality”. Some of the currently popular VR applications using smartphones and headsets are education focused experiences such as Google Expeditions and Nearpod which take users through interactive presentations and allow teachers to create their own interactive immersive presenta- tions as well. In addition to smartphone based VR there are several ded- icated VR platforms which have their own unique features, controllers, and collections of VR applications. The primary VR hardware platforms are made by HTC (HTC Vive, Vive Focus Plus),

learned this lesson by partnering with the WSU ARMY ROTC who mounted the camera on a cadet’s helmet during a field exercise that involved many obstacles, running, and jumping. We now use this video as an example of how to make people motion sick. Beyond that limitation of slowing down and keeping the camera steady, 360 video is easy to learn to produce. Just like traditional video modeling has been applied to a wide range of skills, VR video modeling also has an incredible potential to address skills. Imagine a student who struggles with changes to routine and new transitions, for example learning to ride a school bus home. We can record the new experience, in this case the new bus ride experience start to finish with all of its sights and sounds so our learner can practice that experience in advance. The student’s teachers can see if there are major distractors or other problems that could be reduced or eliminated before the student has to attempt the transition in real life. The 360 video models could be applied to individual needs for AT to address functional skills, academic needs, or employment related activities using similar best practices to traditional video modeling.

BACK TO CONTENTS

sensors that communicate to the VR games how fast the user is peddling the bike. From the users experience in the VR game they are sitting in race car, riding a horse, or kayaking, and the faster they peddle the faster they go in the VR game. The par- ticipants in the VR experience greatly increased their duration and intensity of physical activity while playing the exergaming system. A picture of the system is shown in Figure 2 below. There are also many VR exergaming apps that don’t require an exercise bike or additional equipment. Holoball is a VR rack- etball like game played in standing room VR using the control- lers to serve, redirect, and whack as hard as you can a ball on court. It makes a great single player or competitive exergame to get the participants moving and exercising. The most popu- lar VR exergame is probably Beatsaber in which participants use the controllers to hit the beat with lightsaber drumsticks while listening to a range of popular songs. Playing one song seems easy but after 20 minutes most people are starting to feel the Beatsaber burn.

Facebook’s Oculus (Oculus Rift, Oculus Go), Samsung (Gear VR), and Sony (Playstation VR). Some of these require a dedicated computer in addition to the VR head mounted display (Oculus Rift, HTC VIVE) while others are stand alone VR headsets (Oculus Go). Each system has its own collection of VR games, experienc- es, and applications. All of these systems provide free apps to play and control 360 video. VR exercise gaming One of our first projects in the WSU ATR&D Lab was to ex- plore the use of VR exercise gaming (exergaming) to increase the physical activity of individuals with intellectual and devel- opmental disabilities (IDD) (McMahon, Barrio, McMahon, Tutt, & Firestone, 2019). We chose to examine physical activity because we felt that exergaming in VR could help address concerns about reduced physical activity for students as they transition from their K-12 schools. The study examined the exercise dura- tion and intensity for a group of students with IDD when using a VR exercising system called Virzoom (Virzoom.com). The Vir- zoom exercise games are controlled using an exercise bike with

Figure 2: VR Exergaming session in the WSU ATR&D Lab.

www.closingthegap.com/membership | June / July, 2019

BACK TO CONTENTS

Closing The Gap

Mindfulness VR Apps There are several VR apps focused on relaxation and mindful- ness which can be used to support behavior regulation. Guided Meditation VR (Free) takes the user to beautiful simulated VR en- vironments such as a beach, forest under the aurora borealis, or outer space and guides the user through a calming deep breath- ing mindfulness script. FlowVR ($3.99) uses high resolution 360 degree video of nature scenes (waterfalls, oceans, mountains), gentle music, and guided meditation prompts to support relax- ation and mindfulness. For individuals who may need a break to regroup emotionally these VR mindfulness apps can provide an easy to implement self regulation tool using VR. VR Education The capability of VR experiences to put the user into a com- pletely artificial environment means that there is incredible po- tential to use VR for education. Currently, VR education focused apps are not quite as common as VR games but there are still many high quality VR educational apps. Google Earth VR (Free) brings the power of the Google Earth desktop app to VR so the user can virtually visit and explore anywhere on Earth. Google Earth in VR is easy to navigate and learn and includes detailed 3D models of most buildings in big cities and all the available street level imagery. Using the street level imagery a learner can virtually visit all around the world. Lifeliqe VR Museum (Free) al- lows users to interact with a range of topics from how volcanoes work, parts of a plant cell, and exploring outer space. Medical Holodeck (Free) is an interactive human anatomy viewer with tools for teaching and assessing user knowledge. These are just a few of the available educational VR experiences that could pro- vide immersive learning opportunities that are hands on (virtual hands on anyways), interactive, and learner directed. For stu- dents that are struggling to master content from a textbook or lecture these VR experiences are new resources to help educa- tors meet their students needs. VR Social It may seem counterintuitive to think of VR as a social activ- ity but there are many social applications and games that can provide structured practice for improving social skills. We are not recommending this as the only social skills instruction of course, but the capabilities of VR may provide some novel tools to your social skills instruction. For example the VR app Altspace- VR (Free) allows the user to meet up with friends in a room as a cartoonish avatar but the movements and voices are all real people. A student working on employment skills could practice answering interview questions from their teacher or job coach in this VR space. A practice interview in a simulated environment may be less socially stressful than initially practicing face to face in real life where issues like making eye contact could be barri- er to success. There are many multiplayer games that are avail- able in VR that can be played with friends from anywhere in the

world with internet access. A user with limited transportation could meet up with friend in Oculus Rooms (free) for a chat and some board games like chess, checkers, and many others or get together a group of friends for a fun game of Settlers of Catan in VR ($9.99). Based on individual needs some users may prefer playing complex games with many small pieces in VR which re- duces the need for fine motor activities such as reaching across the board and placing small objects. VR Entertainment Access to entertainment is another interesting capability on VR headsets. In addition to games like the ones mentioned above popular streaming video services like Netflix and Hulu are available on many VR platforms. Some like the Oculus Go even let you and a friend watch the same movie or show together from anywhere with internet access. Using these VR video apps a user can make the screen as large or small as they want based on their needs. The nature of wearing a head mounted VR dis- play also means that the user does not see the world around them while watching their favorite show unlike when they watch Netflix or Hulu on a mobile device. Representatives from our lab recently met with the administration of a local hospi- tal to examine the potential benefits of offering VR headset as another patient care option to relax and entertain during long treatments that can be distressing such as infusion based treat- ments. Rather than seeing needles, tubes, and monitors and a small mobile tablet screen patients could explore simulated en- vironments, play games in virtual environments, or just watch their favorite episode of Star Trek on a virtual screen as large as a movie theater screen. Future directions of VR as AT Hopefully this brief overview of the current capabilities of VR demonstrates some of its potential as an assistive technology. The next generations of these platforms will provide additional applications and design features that will enhance accessibility in VR for a larger group of users. Several platforms are work- ing including eye tracking to allow users to select and navigate experiences on the VR head mounted display. Major platforms in VR are also working to incorporate brain computer interfac- es that could use biofeedback to allow the user to select and control their VR experiences with less dependence on control- lers. These additions could dramatically increase future acces- sibility by incorporating eye tracking and other technologies to increase the options for access. Eye tracking tools could allow a user with mobility challenges that limit their use of hand held controllers to still enjoy and experience interactive VR games and simulations. Advancing VR as AT Advancing VR as a resource for individuals with disabilities is an exciting opportunity that will benefit from more people

exploring and informally trying to see what works for them. That doesn’t mean there isn’t outstanding research work being done currently to advance VR as assistive technology. Dr. Sar- ah Howorth at the University of Maine is working on using 360 degree video modeling to support social skills development for students with autism. According to Dr. Howorth: Students with disabilities such as autism and intellectual dis- ability benefit from practicing new skills in real world situations. Often it is difficult to replicate real world situations in a safe envi- ronment, or the environment may be too risky from a social or a safety perspective. 360 degree video modeling allows these in- dividuals to immerse themselves in an environment to practice a skill. Social skills can be rehearsed, and vocational skills prac- ticed in a way that allows for skill rehearsal in an environment that replicates the real world environment without real world risks. Once the individual becomes fluent in the skill, real world rehearsal may begin. Dr. David Cihak at the University of Tennessee is working on advancing VR interactive simulations for virtual job training for students and adults with Intellectual and developmental disabil- ities. According to Dr. Cihak “The advantage of VR simulations for job training is that we can provide multiple job environments and situations in a structured and safe setting. We can see what the learner is paying attention to in the situation and provide prompts and feedback based on the learners needs.” The goal of these simulations would be to provide a robust job training and point of view experiential practice to precoach critical job skills. In additional VR research projects the ATR&D Lab at WSU works to advance VR in education by partnering with classes in the teacher education programs to provide hands on experienc- es using VR and other immersive tools that preservice teachers then implement into their assignments. These introductory VR experiences have become student favorites as shown in student evaluations, lesson plans that include VR tools to support Uni- versal Design for Learning, and student research projects using VR. To reach current educators we also work to provide ongoing professional development. The ATR&D LAB sponsored its third full day preconference workshop at the Division of Autism and Developmental Disabilities conference on using Current and Emerging technology to support students with disabilities. These sessions start the day covering many traditional AT topics such as text to speech, accessibility tools, accessible instruction- al materials and build up to wearable and immersive learning tools such as VR. Our favorite and the audience’s favorite part of the day is the hands on afternoon session where we set up multiple stations of virtual reality experiences for the audience to try. In each station we provide many examples of how to use VR to support students with disabilities. In addition to our exam- ples, every time our audience comes up with new ways to use VR to address needs that they are trying to meet in their class- rooms. We need more of those great ideas and we need to share

them with each other. The best way to advance the adoption of new assistive tech- nology platforms such as VR is to get them to individuals with disabilities, special educators, related service providers, parents, families, and other stakeholders. AT is a diverse field touching on many different needs and contexts some of which VR can support. Our users’ strengths and needs are unique to each in- dividual and VR tools are one additional means of supporting the people we serve. A process of discovery and innovation is needed to advance VR as an assistive technology. The more VR devices and experiences we all try the faster we will build the knowledge base around this topic. Working with all of our com- munity of stakeholders, we can explore how to apply these new VR tools to empower and include individuals with disabilities. References McMahon, D.D, Barrio, B. L., McMahon, A. K., Tutt, K., & Fires- tone, J., (2019) Virtual reality exercise bike intervention for high school students with intellectual/developmental disability. Journal of Special Education Technology. 34, 2, Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems, E77- -D(12), 1321–1329. Retrieved from http://web.cs.wpi.edu/~go- go/hive/papers/Milgram_IEICE_1994.pdf

www.closingthegap.com/membership | June / July, 2019

BACK TO CONTENTS

Closing The Gap

Page 1 Page 2 Page 3 Page 4 Page 5 Page 6

www.closingthegap.com

Made with FlippingBook Ebook Creator