FingerIO

FingerIO is a novel fine-grained finger tracking solution that transforms any space around off-the-shelf smartphones or smartwatches into an interactive surfaces. FingerIO does not require instrumenting the finger with sensors and works even in the presence of occlusions between the finger and the device. We achieve this by transforming the device into an active sonar system that transmits inaudible sound signals and tracks the echoes of the finger at its microphones. To achieve subcentimeter level tracking accuracies, we present an innovative approach that uses a modulation technique common in wireless communication called Orthogonal Frequency Division Multiplexing (OFDM). Our evaluation shows that FingerIO can achieve 2-D finger tracking with an average accuracy of 8 mm using the built-in microphones and speaker of an Android smartphone. It also tracks subtle finger motion around the device, when the phone is inside a pocket. Finally, we prototype a smart watch form-factor FingerIO device and show that it can extend the interaction space to a 0.5 X 0.25 m² region on either side of the device and work even when it is fully occluded from the finger.