FingerIO is a novel fine-grained finger tracking solution that transforms any space around off-the-shelf smartphones or smartwatches into an interactive surfaces. FingerIO does not require instrumenting the finger with sensors and works even in the presence of occlusions between the finger and the device. We achieve this by transforming the device into an active sonar system that transmits inaudible sound signals and tracks the echoes of the finger at its microphones. To achieve subcentimeter level tracking accuracies, we present an innovative approach that uses a modulation technique common in wireless communication called Orthogonal Frequency Division Multiplexing (OFDM). Our evaluation shows that FingerIO can achieve 2-D finger tracking with an average accuracy of 8 mm using the built-in microphones and speaker of an Android smartphone. It also tracks subtle finger motion around the device, when the phone is inside a pocket. Finally, we prototype a smart watch form-factor FingerIO device and show that it can extend the interaction space to a 0.5 X 0.25 m2 region on either side of the device and work even when it is fully occluded from the finger.
Our current research prototype is designed to work on Android smarphones with at least 2 microphones. It also supports simple 1D gesture interaction for Android smartwatches with a single microphone.
FingerIO is unaffected by audible noise. Because we use high frequency, inaudible sound signals, we filter audible noise from the environment while processing.
This code is currently the intellectual property of the University of Washington. We are in the process of exploring options for commercializing the technology and releasing it to developers.