XR prototyping tool
Challenge
There is an increasing need for developing tools to design AR/VR scenes. Currently most of the tools are single-user based or only limited to either AR/VR. There are too many interaction limitation on these tools. Therefore, our goal was to create a tool that enables designers to freely express interactions using AR and VR devices.
Solution
I conducted secondary research to explore different roles in filmmaking, also to find out what makes AR/VR games engaging. After that, I applied similar techniques in AR/VR prototyping. I also recreated one VR game (Beats Saber) and one AR game to assess how difficult/easy it is to prototype interactions.
Role & Team
Interaction Designer on this part. Other Interaction Designers work on other parts of the research led by Dr. Michael Nebling at Human-Computer-Interaction Lab
Outcome
A research paper was published in CHI, a top academic journal in the field of human-computer interaction
Source: XR Director
Beats Saber (VR)
I chose this VR game mostly because of its popularity, as well as the sound and light effects.
​
Physical prototype: I was able to produce all the scene elements and movement but it was difficult to produce flashing flight in the background. Cutting the blocks was also hard because it requires coordination between players and two pup-peers.
​
​​​VR prototype: I used A-frame (html, css, js) to create the scene and interaction. Code can be found on my CodePen. Making the beat of the music match the light was difficult because it requires a lot of adjustment in the timing for animation in code.
​​
ARcade plane (AR)
I chose ARcade plane mobile AR game because the plane moves and tilts as the plane moves ups and down. These are common set of features of mobile AR games.
​
Physical prototype: It was easy to create the effect in which the shadow of the plane moves with the plane, because each person in the scene played an role and the coordination was easy because of real-time communication.
​
AR prototype: I used a-frame to create the animation of the plane and the shadows. Code can be found on my CodePen. Even with a-frame, it was difficult to collaborate online because a lot of adjustments (adding/removing things) were hard to be communicated
Final output
Start viewing from 2:28 to see prototyping experiments in action