Scratch Input: Creating Large, Inexpensive, Unpowered and Mobile Finger Input Surfaces

Authors

Chris Harrison    Scott E. Hudson

Human-Computer Interaction Institute

Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213

{chris.harrison, scott.hudson}@cs.cmu.edu

Summary

Scratch Input

Scratch as a computer input has never really been exploited on a mass scale before.  Scratching a surface such as wood, paint, or fabric produces a high frequency sound that is actually unique to the gesture that it corresponds with.  

These sounds are better propagated through the material itself rather than the air.  For instance, if you scratch the surface of your desk you will be able to hear it audibly with some clarity just as you would hear someone next to you talking.  However, if you put your ear to the surface of the desk the sound becomes amplified and the effects of outside noise is minimal in the sound that you hear.  Because sound is preserved so well in dense materials, gestures from several meters away can be accurately recognized.

Using a microphone attached to a stethoscope and a high pass filter to remove noise, the frequency graph of the sound input can be constructed and categorized using any decision making tree.  These gestures can then correspond to specific actions in a computing environment such launching and exit programs, silencing music players, increasing/decreasing volume, etc.  Nearly every dense surface could be turned into an input device including cell phones, tables, and walls.

Participants were shown how to use the scratch input system and then participants used a program which told them which gesture to produce.  Each participant repeated each gesture 5 times for a total of 30 trials per participants(15 participants).  


The results are shown in the above box.  The results were sufficient for a proof of concept for the use of scratch as a source of input for computers.

Discussion

At first I was incredibly skeptical about this research. I thought about how ridiculous it would be for people to scratch things in order to manipulate their computers.  However, the video definitely sold me on the concept.  Installing just a few of these cheap microphones around your house could transform nearly every surface into a computer input console.  Every wall, desk, or even your car's dashboard or steering wheel could be used as an input device.  I will say though that the success rates will probably need to be much higher for introduction into the mass market.  Having a gesture wrongly categorized 1 out of every 5 or 10 times is not satisfactory for any mass distribution.