About this blog...

The people of Esri UK sharing what we do, what inspires us and how we get the most out of Esri technology.

Entries on this blog are the views of the individual authors and do not necessarily represent the views of Esri UK.

Follow us

LVF Blog Page
Schools Blog Page
Technical Support Blog Journal
Think Location
« The next geographers | Main | Maplex - easier to use than you might think! »

Giving eyes and ears to GIS

Anyone who's seen the sci-fi film Minority Report will remember the scene where Tom Cruise's character, a police detective who prevents murders before they can take place, stands in front of a futuristic computer and sifts through a stream of pictures and videos just by making gestures with his hands in mid-air (although he does have to wear special gloves to do it).

More recently, Iron Man 2 shows how every GIS interface should work! If you have a minute to spare, it's definitely worth watching the beginning of this video:

How close are we to this reality? We're certainly not there yet, although some very innovative stuff is being done in the area of "natural user interfaces" (or NUI, pronounced "NOO-ee"). NUIs are user interfaces we control using natural human actions such as talking and hand gestures, as opposed to more artificial actions such as typing on a keyboard or clicking a button on a mouse. Perhaps the most influential NUI product released recently for the consumer market is the Kinect sensor, originally designed by Microsoft for the Xbox 360 games console, but which was quickly repurposed by developers for other platforms such as the PC.

Kinect provides a depth-sensing camera and microphone array that can be used with the right software to respond to body movements and listen for voice commands. Affordable speech recognition technologies have been around for a long time, but this is the first time that such a high-quality body-tracking camera has been widely available for around £100. This opens up some exciting possibilities for GIS user interfaces.

At Esri UK, we've been building some simple prototypes with Kinect, using hand gestures to pan and zoom around a custom WPF mapping application, and voice commands to geocode locations (postcodes only at the moment) and provide additional map navigation (zooming, panning). It's a cool gimmick, but we are hoping to find a practical use for it soon! A Kinect add-in for ArcGlobe has also been released by Esri Inc, which may also be of interest.

So far, we have worked with two software toolkits to develop Kinect applications for the PC. An official SDK for Windows 7 was released by Microsoft Research in June 2011, which has full support for speech and gesture recognition, but does not work on other platforms and is limited to non-commercial applications only. It is in beta at the moment. We have also worked with the open-source OpenNI framework, which does not support speech recognition using the Kinect, but does allow the development of gesture-controlled apps. The Microsoft Research SDK and the various unofficial SDKs cannot be installed on the same machine at the same time, so you will have to choose between them. Windows users who doesn't want to write code will also be interested in FAAST, a desktop application that works with OpenNI and allows you to associate body gestures (such as a jump or a hand being raised) with key presses and mouse movements by editing a simple configuration file. This then allows you to use gestures to control any Windows application.

If anyone has been doing the same or is interested in finding out more, please get in touch by leaving a comment and your email address. Your email address will not be made public.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
All HTML will be escaped. Hyperlinks will be created for URLs automatically.