FlightGear Demo

Ah, the miracle of simulated flight. We take a stab at creating 3D gesture controls for piloting an aircraft in FlightGear, a free, open source flight simulator.

Here's a video detailing the different gestures and the operations that they map to. Disclaimer: I'm not a pilot, or even a seasoned simulated pilot, so pardon my reckless maneuvers.

Let me give a more detailed explanation of the gestures.

Startup controls
We mapped the operations to each hand, based on how one might operate the controls in a real cockpit. The throttle and the parking brake can only be accessed by pinching with the right hand. The physical interaction space in front of the monitor was divided such that pinching in the rightmost area of the interaction space could trigger only the throttle. Moving the hand forward and back mimics the physical act of pushing a throttle forward and back. Pinching in the center of the interaction space accesses the parking brake. We use a rotation of the hand to reflect the motion of the parking brake being released or set. Finally, the ignition is located on the left side of the flight controls, so we use a left hand pinch to access the ignition. Rotating clockwise closely mimics the act of turning a key.

Flight controls
Flight controls only become engaged when both hands perform a pointing gestures. I admit that a real pilot probably doesn't point his or her fingers while piloting a plane, but this is a demonstration of the different hand poses we are working on exposing to the developer. Robust recognition of different hand poses would allow the developer to use hand pose to set the mode of the application. The two hands control three things: ailerons, elevators, and rudders, which control the roll, pitch, and yaw of the aircraft respectively.

Ailerons:

Elevators:

Rudders:

In summary the gestures are as follows:

• Throttle - A pinch with the right hand, pushing forward and back, in the right side of the interaction space
• Parking brake - A pinch with the right hand, rotating clockwise to releast the break and counter-clockwise to set the break, in the center of the interaction space
• Ignition - A pinch with the left hand, rotating clockwise
• Ailerons - Pointing with both hands, moving the hands like turning a vertical steering wheel
• Elevators - Pointing with both hands, rotating both hands upwards or downwards
• Rudders - Pointing with both hands, moving the hands like turning a horizontal steering wheel

Kenrick