Andy McDonald

Experimenting with DIY multi-touch

Last week, I finally got a chance to build my first multi-touch surface. I’m using a DSI (Diffused Surface Illumination) method that differs from the FTIR (Frustrated Total Internal Reflection) in that it uses a special type of acrylic called Plexiglas Endlighten that actually diffuses the IR light outwards rather than trapping it inside. The basic principle is fairly similar – when you place your finger on the surface, it directs IR light towards a webcam that has been modified to detect infra-red. Software such as CCV (Community Core Vision) analyses this video feed and outputs the corresponding blob data in the TUIO (Tangible User Interface Object) format which is based on the OSC (Open Sound Control) protocol.

Next, you’ll need to setup a way for your application to receive and parse the incoming data – I’d recommend using the tuio-as3 library. Until recently, the only way to get this data into a Flash / AIR application was to either i) establish a TCP connection over a standard socket or ii) convert the TUIO data to XML and send it through an XMLSocket. Since AIR 2.0 introduced the new DatagramSocket class, I decided to add UDP support to the tuio-as3 library resulting in lower latency (you can download my updated version of the latest release here). As well as low-level TuioEvents based on the raw data, the library also includes TouchEvents that similar to using MouseEvents – a few of us at the NUI Group are working on better integration with the native Multitouch API. Having identified when / where touches occur, it’s just a matter of responding to the events.

I’ll be writing a lot more about multi-touch in the coming weeks but for now here is a video of my first experiment…

Multi-touch test from TechStyleLAB on Vimeo.