Gesture Recognition
June 17, 2013
At the start of my
undergraduate education, I briefly attended a
college whose emphasis was on
engineering. This was a mistake for a
physics major, so I left a short time later. I did, however, gain some valuable
extracurricular knowledge about
radio broadcasting. Since it was a select school, I met quite a few creative and interesting people. The school was so select, in fact, that fully half the students were above
average.[1]
The conversations at group lunches were highly technical. We would often
invent useful devices, at least on
paper -
napkin paper, that is. One of these was a
gesture control system in which waving an unaided
hand would control a device such as a
television. Nowadays, there are quite a few
technologies to tackle this problem, the most effective being
machine vision systems, as used in the
Microsoft Kinect device.
In the late 1960s, however, our technology choices were limited, but there were two that would have enabled such a device. The first is
radar, since
Doppler signals from a waving hand would be easy to detect. The radar generator would have been something like a
klystron, although
solid state microwave generators, such as the
IMPATT diodes,
tunnel diodes were just becoming available.
Cutaway of a Varian V-260 reflex klystron. Such devices attached to rectangular waveguides, usually at the X band.
(Cutaway model provided by Walter Z. Seldon and photographed by Erbade, via Wikimedia Commons.)
Another, more easily built device would be based on the
Theremin. I mentioned
Léon Theremin, the inventor of the Theremin, in a
previous article (Cyber Warfare, November 10, 2011). The Theremin uses the
electrically conducting human body as a
plate in the variable
capacitance of a
radio frequency oscillator; so, proximity of a body part, such as a hand, will cause a detectable frequency change.
We sit in our offices and living rooms immersed in
electromagnetic fields, some of which are intentional, and some are not. The most useful is the
light that comes from our lamps and
display screens. The most annoying is the
power line emission that gives us the buzz on our
AM radios and
stereo music systems.
Environmental fields can serve as the basis for object detection. One interesting example of this was the use of
cosmic rays by
Nobel Physics Laureate,
Luis Alvarez, in a search for hidden chambers in an
Egyptian pyramid in the 1960s.
Scientists in
Mexico have applied this same technique to the pyramid at
Teotihuacan.[2] Recently, more primitive methods have been used at the
Nohmul complex in
Belize.[3]
The Pyramids at Gizah in Egypt. During my college years in the 1960s, many New Agers believed that the pyramid shape focused "cosmic energy."
(Photograph by Ricardo Liberato, via Wikimedia Commons.)
One type of
electromagnetic radiation pervading our home and office environments is
Wi-Fi, which is a
physical layer in
local area networking.
Tablet computers,
laptop computers,
desktop computers,
game consoles,
computer peripherals and
smartphones all have Wi-Fi capability. It sure beats snaking
Ethernet cable throughout a house, as I did many years ago.
University of Washington computer scientists have decided to harness this sea of Wi-Fi radio waves as a means for gesture recognition. The Washington research team includes
Shwetak Patel and
Shyam Gollakota, both
assistant professors of
Computer Science and Engineering, and
doctoral student,
Sidhant Gupta. Their technology, which they call "WiSee," will be presented at
The 19th Annual International Conference on Mobile Computing and Networking later this year, but details are available online.[4-7]
A display showing the change in wireless signal strength across frequency channels in real time as a user moves his hand. (University of Washington image.)
One advantage of this method is that Wi-Fi signals travel through walls, so gesture detection is not limited by the
line-of-sight restriction of
camera systems. The method is based on the Doppler effect, which in the gesture case is just a few
hertz. The entire Wi-Fi
spectrum is scanned, looking for subtle changes in signal strength as a function of
frequency.[4]
At this point, the WiSee system can identify nine different whole-body gestures, such as pushing, pulling and punching. In a test environment of five users in a two-bedroom
apartment and an office environment, 94% of 900 gestures were accurately detected.[4] As for
security, a specific sequence of gestures can be used as a
password.[4] My college classmates and I were just a few decades too far ahead of the
technology curve on this one.
References:
- I'm sure you caught that joke. If you haven't, read the statistics page on Wikipedia.
- Claire Marshall, "Physicists probe ancient pyramid," May 13, 2004.
- Elizabeth Snodgrass, "Ancient Maya Pyramid Destroyed in Belize," National Geographic, May 15, 2013
- Michelle Ma, "Wi-Fi signals enable gesture recognition throughout entire home," University of Washington Press Release, June 4, 2013.
- Web Site of The 19th Annual International Conference on Mobile Computing and Networking, Miami, Florida, Sept 30 - Oct 4 2013.
- Whole-Home Gesture Recognition Using Wireless Web Site at the University of Washington.
- YouTube video about University of Washington gesture capture research.
Permanent Link to this article
Linked Keywords: Undergraduate education; college; engineering; physics; extracurricular activity; radio broadcasting; mean; average; invention; paper; napkin; gesture recognition; gesture control system; hand; television; technology; machine vision system; Microsoft; Kinect; radar; Doppler effect; Doppler signal; reflex klystron; klystron; solid state; microwave; IMPATT diode; tunnel diode; Varian Associates; rectangular waveguide; X band; Wikimedia Commons; Theremin; Léon Theremin; electrical resistivity and conductivity; human body; parallel-plate capacitor; capacitance; radio frequency; oscillator; electromagnetic field; light; computer monitor; display screen; power line; electromagnetic interference; EMI; AM radio; stereophonic sound; stereo music system; environment; cosmic rays; Nobel Physics Laureate; Luis Alvarez; Egyptian pyramid; scientist; Mexico; Teotihuacan; Nohmul complex; Belize; The Pyramids at Gizah; Egypt; college; New Age; pyramid shape; Ricardo Liberato; Wikimedia Commons; electromagnetic radiation; Wi-Fi; physical layer; local area network; tablet computer; laptop computer; desktop computer; game console; computer peripheral; smartphone; ethernet cable; University of Washington; computer scientist; Shwetak Patel; Shyam Gollakota; assistant professor; Computer Science and Engineering; doctoral student; Sidhant Gupta; The 19th Annual International Conference on Mobile Computing and Networking; line-of-sight; camera; hertz; spectrum; frequency; apartment; computer security; password; technology life cycle; technology curve.