Today, there are already few systems that are using Adobe AIR to build a multi-touch GUI. There is an open project called Touchlib that allows listening to user gestures and building your own touch-screen. Intuilab has presented in Adobe MAX in Milan an application that takes full advantage of surface computers. The system integrates many contents such as: image, video, web content, Adobe pdf, Flash, Illustrator, Photoshop, Microsoft Office etc.

The idea behind Touch and Multi-touch computer screens is to follow the instructions of finger/s or hand/s and be able to track gestures. The idea is simple:

1. Register to receive gestures events.
2. Handle gesture events.
3. Interpret the gesture events.

Creating a touch screen application on UMPC can be done today, and we can rely on the UMPC device to handle the touch screen; however you may find that relying entirely on the device may be buggy.

For instance, the device may register a user gesture when your fingertip is on the device and didn’t even move your finger since a slight move also get registered, however you may not need your application to be that sensitive.

I recommend implementing your own instructions to register user gestures. Microsoft released the beta version of Windows 7 which track user gestures via WM_GESTURECOMMAND, WM_GESTURE and WM_TOUCH so I think it’s possible to communicate with these method using a proxy such as Merapi or you can create your Java proxy.

I have created a simple POC that I havn’t test much but it gives you an idea. The API is for touch screen application for mobile devices such as UMPC that register user gestures and can be expended to register multi-touch screen user gestures. The way it works is that the API translate the mouse events into user gestures based on time and movement.

Here’s a screen shot of the application on a UMPC:

And you can download it from here:
http://elromdesign.com/blog/Flex/TouchScreen/TouchScreen/Touchscreen.zip

Take a look at the class below that gives its own instructions on when to register user gestures.

package com.elad.framework.touchscreen
{
	import com.elad.framework.touchscreen.events.TouchEvent;
	import com.elad.framework.touchscreen.vo.TouchVO;

	import flash.events.EventDispatcher;
	import flash.events.MouseEvent;
	import flash.utils.Timer;

	import mx.core.UIComponent;

	public class TouchManager extends EventDispatcher
	{
		private var moveTimer:Timer;
		private var previousX:int = 0;
		private var previousY:int = 0;
		private var component:UIComponent;

		public function TouchManager(component:UIComponent)
		{
			this.component = component;
		}

		public function start():void
		{
			initialize();
			this
		}

		public function stop():void
		{
			component.removeEventListener(MouseEvent.MOUSE_DOWN, onMouseDownHandler);
			component.removeEventListener(MouseEvent.MOUSE_UP, onMouseUpHandler);

			moveTimer.stop();
			moveTimer = null;
		}		

		protected function initialize():void
		{
			component.addEventListener(MouseEvent.MOUSE_DOWN, onMouseDownHandler);
			component.addEventListener(MouseEvent.MOUSE_UP, onMouseUpHandler);
		}

		private function startTimer():void
		{
			moveTimer = new Timer(100,1000);
			moveTimer.start();
		}

		private function onMouseDownHandler(event:MouseEvent):void
		{
			startTimer();
			component.addEventListener(MouseEvent.MOUSE_MOVE, onMouseMoveHandler);

			var touch:TouchVO = new TouchVO(this.previousX, this.previousY, event.localX, event.localY, this.moveTimer.currentCount);
			this.dispatchEvent( new TouchEvent(	TouchEvent.TOUCH_DOWN, touch ) );
		}

		private function onMouseUpHandler(event:MouseEvent):void
		{
			moveTimer.stop();
			component.removeEventListener(MouseEvent.MOUSE_MOVE, onMouseMoveHandler);

			var touch:TouchVO = new TouchVO(this.previousX, this.previousY, event.localX, event.localY, this.moveTimer.currentCount);
			this.dispatchEvent( new TouchEvent(	TouchEvent.TOUCH_UP, touch ) );
		}

		private function onMouseMoveHandler(event:MouseEvent):void
		{
			var isMove:Boolean = isTouchMove(event.localX, event.localY);
			var touch:TouchVO = new TouchVO(this.previousX, this.previousY, event.localX, event.localY, this.moveTimer.currentCount);

			if (isMove)
			{
				this.dispatchEvent( new TouchEvent(TouchEvent.TOUCH_DRAG, touch) );
			}
		}

		private function isTouchMove(x:int, y:int):Boolean
		{
			var retVal:Boolean = false;
			var ignore:int = 3;
			var isXmoved:Boolean;
			var isYmoved:Boolean;

			if (previousX != 0 && previousY != 0)
			{
				isXmoved = (x > previousX+ignore || x < previousX-ignore) ? true : false;
				isYmoved = (y > previousY+ignore || y < previousY-ignore) ? true : false;

				if ( isXmoved || isYmoved )
				{
					retVal=true;
				}
			}

			previousX = x;
			previousY = y;

			return retVal;
		}		

	}
}

And than we can implement the class with a simple interface:

<WindowedApplication xmlns="http://ns.adobe.com/mxml/2009" layout="absolute"
	width="800" height="600"
	initialize="initializeHandler()">

	<Script>
		<![CDATA[
			import com.elad.framework.touchscreen.vo.TouchVO;
			import com.elad.framework.touchscreen.events.TouchEvent;
			import com.elad.framework.touchscreen.TouchManager;
			import mx.collections.ArrayCollection;

			[Bindable]
			private var arrayCollection:ArrayCollection = new ArrayCollection;

			private var touch:TouchManager;

			protected function initializeHandler():void
			{
				touch = new TouchManager(this);
				touch.addEventListener(TouchEvent.TOUCH_DOWN, onMouseDownHandler);
				touch.addEventListener(TouchEvent.TOUCH_UP, onMouseUpHandler);
				touch.addEventListener(TouchEvent.TOUCH_DRAG, onMouseDragHandler);

				touch.start();
			}						

			private function onMouseDownHandler(event:TouchEvent):void
			{
				ellipse.visible = true;
				moveEllipse(event.touchVO.currentX, event.touchVO.currentY);
				registerLocation(ellipse.x, ellipse.y, "MouseDown", 0);
			}	

			private function onMouseUpHandler(event:TouchEvent):void
			{
				ellipse.visible = false;
			}					

			private function onMouseDragHandler(event:TouchEvent):void
			{
					moveEllipse(event.touchVO.currentX, event.touchVO.currentY);
					registerLocation(ellipse.x, ellipse.y, "MouseMove", event.touchVO.moveTimer);
			}

			private function registerLocation(x:int, y:int, type:String, time:int):void
			{
				arrayCollection.addItem({locationX: ellipse.x, locationY: ellipse.y, type: type, time: time});
				dg.dataProvider = arrayCollection;
			}

			private function moveEllipse(x:int, y:int):void
			{
				ellipse.x = x - ellipse.width/2;
				ellipse.y = y - ellipse.height/2;
			}			

		]]>
	</Script>

	<Group>
		<Ellipse height="80" width="80" id="ellipse" visible="false">
			<stroke>
				<SolidColorStroke color="0x000000" weight="2"/>
			</stroke>
		</Ellipse>
	</Group>

	<DataGrid x="341" y="4" id="dg" dataProvider="{arrayCollection}" height="410">
		<columns>
			<DataGridColumn headerText="X" dataField="locationX"/>
			<DataGridColumn headerText="Y" dataField="locationY"/>
			<DataGridColumn headerText="Type" dataField="type"/>
			<DataGridColumn headerText="Time" dataField="time"/>
		</columns>
	</DataGrid>

I am looking for a .NET developer to spare time to create a simple class that gets Windows 7 events and compile them into a DLL, so we can pick them up in a Java proxy, which will make a request every few millisecond and look for user gesture changes, so if you are interested, please reply in this blog post.