VirtuaSpace – Mobile VR Music Visualization (HTC)
Role: UX/UI DESIGNER, DEVELOPER, EVALUATOR
Project Type: Capstone project for the Human-Cenetered Design and Engineering program.
Client: HTC Corporation
Team: Alan Qiu, Derek Lee, Elika Berlin, John Allen, Nathaniel Tabit
Tools: Adobe Photoshop, Adobe Illustrator, Unity3D, Slack, Google Cardboard
Two interaction methods that the team researched and implemented in Unity were “Glance” and “Click”.
This undergraduate capstone project formed around the opportunity presented by HTC in the form of their mobile virtual reality (VR). Team VirtuaPlush came together in February 2015. Our team decided to focus on creating a music visualizer with the addition of novel interaction methods on Google Cardboard, a mobile VR platform.
After performing research in the field, including discussions with different luminaries in the VR field, we focused on the limited interaction modes that mobile VR provides. Mobile VR has the ability to reach billions of people, thanks to its low-hardware requirements, but most of the attention has been on heavier systems like the Oculus Rift–we wanted to highlight the capabilities of mobile VR and put users at the forefront by demonstrating the most promising methods of interaction, mobile phones can’t offer the same level of input methods as dedicated VR systems.
Research involved reading literature on the VR space, specific research into mobile VR, conversations with prominent VR experts, analysis of the Google Cardboard, and competitive analysis of competing platforms and apps. Our research brought us to the conclusion that indirect manipulation was the best path for our app.
Another key step that we took during this research phase was to talk to Professor Thomas A. Furness III, of the University of Washington. Professor Furness is the ‘Grandfather of VR’, having pioneered the field in the 1970s and with five decades of experience spanning from the military, to the private sector, and academia. Important to Professor Furness is the ability for VR to excite and engage people in a manner not possible in the real world. In particular, he opened us up to the world of music in VR, by demonstrating a research project. This project used augmented reality to control music in a captivating and intuitive way.
We brainstormed space, underwater, and poolside locations as possible, but space won out, due to its uniqueness (unlike poolside, which we had seen some examples of). Our developers were able to begin work on the VR space environment, as the designers and testers were able to begin work on the interaction methods. Our major focus during this phase was our efforts to design a new and usable form of indirect manipulation in VR. As the form-factor of mobile VR and Google Cardboard was a major factor to the success of our project, we spent a lot of time on getting the indirect manipulation aspect right.
We attempted to create an expandable methodology to interact with objects in a mobile VR space. The goal would be to create an interaction method that could be scaled up to multiple interaction uses, including (but not limited to) state changes, movement, selection, and manipulation. We began brainstorming interaction methods and exploring different environments and methods for testing and showcasing our proposed solutions. We explored a multitude of interaction methods, but settled on the two we thought could best solve the issues. This is based on the data that has come through our research regarding indirect manipulation in a mobile VR space.
Development - "Glance" Interaction
Glance works with a UI reticle that appears on-screen during the Explore mode. When the user’s view direction move and the reticle moves over an object that can be interacted with, the reticle begins to countdown. Once the countdown is complete, the action takes place (in VirtuaSpace, this is movement to that object). The washer interaction on the Google Cardboard switches between Explore and Observe mode. Explore mode allows for this interaction. Observe turns off interaction and the UI reticle, which allows users to look around in their environment without inadvertently moving to a new location.
In addition to handling various design tasks (i.e. the PowerPoint and video), I was also one of the two developers on the team. Nathaniel and I wrote and implemented all of the code for the final product. We learned Unity individually throughout the course of the project outside of normal meeting times. Some of the challenges that I tackled were the implementation of both “Click” and “Glance” interaction methods in the VirtuaSpace app. I was also a major contributor to the design and implementation of the space environment and the music visualizer itself.
The capstone as a whole improved my software development skills. Not only was I able to learn Unity within the course of two months, I connected the new-found knowledge to my previous skills in 3D modeling and animation. Learning how to develop games or VR applications has been one of my long-term goals, and being able to meet that goal within this undergraduate capstone made it very valuable. Researching the various interaction methods within VR has also been very interesting.
- Process Book: book that summarizes and synthesizes the work done over the quarter
- VirtuaSpace Poster: poster presented at HCDE Open House
- VirtuaSpace: final deliverable from VirtuaPlush, APK file for Google Cardboard
- Technical Whitepaper: Milestone #1 detailing our research and focusing decisions
- VirtuaSpace Trailer: project video, shown at the Open House and presentation on June 8th