Date of Graduation

Spring 2018

Document Type

Honors Research Project

Degree Name

Bachelor of Science in Computer Science


Computer Science - Systems

Research Sponsor

Yingcai Xiao

First Reader

En Cheng

Second Reader

Willis Crissey, Jr


We present an implementation of a Natural User Interface (NUI) virtual piano keyboard. Using the Unity game engine and a Leap Motion hand controller, users are able to use their hands to interact with and play music in a virtual scene. Various approaches were attempted in refining the user experience of the virtual piano, and the successes and shortcomings of each implementation method are outlined and elaborated upon. Additionally, the most successful method, which was achieved by using Unity’s physics engine to detect rigid body collisions between virtual fingers and keys, was used to complete a usability study involving both experienced and amateur pianists. Subjects were presented with a series of playability tasks designed to gauge their ability to accurately use the virtual keyboard. Both the usability test data and the subject’s personal feedback were collected and analyzed.