New study establishes VR’s potential for helping autistic people navigate the world
Recent technological developments have made virtual reality (VR) a hot topic in the video gaming industry. However, these new developments are also lending other areas, such as research and design, the technological ability to improve the lives of autistic people.
A new study out of the University of Georgia provides a foundation for using virtual reality and artificial intelligence (AI) to help autistic people better navigate the world.
For the study, associate professor Matthew Schmidt and his research team created a VR intervention for autistic adults called Virtuoso, which—given the dangers associated with the use of public transit—was designed specifically to support their transportation needs.
“We were interested in leveraging this technology to create immersive learning experiences for people on the autism spectrum like we’ve been doing in the past, but to use 360-degree videos to speed up the development timelines and reduce the amount of costs associated with it,” said Schmidt, who was hired under the Presidential Interdisciplinary Faculty Hiring Initiative in Data Science and Artificial Intelligence and holds a joint appointment in the UGA Mary Frances Early College of Education and the College of Pharmacy.
Autism is a complex neurodevelopmental condition that impacts individuals’ social communication and is often accompanied by restricted or repetitive behaviors that may result in difficulties impacting their quality of life, educational performance, vocational success, and more.
To assess autistic users’ ability to manage complex, real-world situations—like navigating transportation systems—the study presented both autistic and neurotypical participants with an Oculus Rift headset.
When worn, the headset placed them in a simulated, 360-degree video where participants were tasked with checking the daily bus schedule, walking to the bus, waiting for the bus, and then lastly, getting on the bus.
“When you’re working with people who have communication and cognitive differences, it can be difficult to convey ideas and messages,” said Schmidt. “But with VR, instead of trying to communicate by explaining verbally or with a video, we can present a simulated, vicarious experience which can be overlaid with educational content.”
Another aspect of the study focuses on using AI to automate the video data analysis process, which could impact the way research is conducted in the future by streamlining a currently resource-intensive and time-consuming video analysis process.
AI enables the automation process by interpreting visual information, like what objects are appearing in the viewer’s field of vision as they navigate the bus system.
As a result, researchers would no longer need to manually identify patterns in oftentimes large datasets. This more cost-effective and efficient process may also yield more accurate outcomes for immersive learning interventions for autistic individuals.
“We’ve got the computer vision part on one end, which makes this nice dataset of what it is that our participants are seeing and when, and then we have machine learning algorithms and AI on the back end that allow us to look at that data, find patterns, and extract those patterns to understand how participants are experiencing our interventions,” said Schmidt. “The different AI approaches we used gave us a range of perspectives on at the data, for example, predicting what might come next in a series of events.”
Schmidt and his colleagues performed a study that utilized AI to compare how neurodiverse and neurotypical utilized Virtuoso. After using machine learning approaches and AI to analyze the dataset, Schmidt and his team found that the neurotypical group showed greater homogeneity and predictability in their behaviors when using the bus system, while the autistic group displayed greater heterogeneity and variability in their behaviors accomplishing the same tasks.
“Compared to neurotypical people, autistic people can have different ways of looking at things, which is something that came to life in our data set,” said Schmidt. “This was the first time that I could see this so clearly in one of my studies. To see such results emerge from a dataset is validating and shows the approach has promise.”
Schmidt’s study showcases the viability of a new method for gaining insight into how autistic users interact with and view VR videos.
Furthermore, the study confirms previous findings on autistic individuals’ usage behaviors, as well as the differences between how autistic and neurotypical individuals navigate complex processes.
“When we’re creating interventions for people who have some level of independence, why wouldn’t we go to that population and ask them what their needs and preferences are and what kinds of interventions they think would be useful for them,” said Schmidt. “And that’s kind of where I’m coming from. I’m researching usage behaviors because I want to design things that will be effective for that group, which is only possible by understanding how they actually use the technology.”