In Finland, the homeland of Nokia, mobile phones are referred to as Kannys, which literally means “an extension of the hand”. I would like to make this extension of the hand very visceral by combining Nokia’s N900 with retro virtual-reality peripheral, the Nintendo Power Glove. The resulting hardware hack would enable hand gesture based interaction with the N900, controlled through the glove interface.

In order to answer a telephone call, the user makes the ‘telephone gesture’ and their hand becomes the handset, bringing a whole new meaning to “talk to the hand”!

Also as part of ISEA 2009 I’m running a workshop on ‘Hacking Toys Into Tangible Interfaces‘ at Dundalk IT. We’ll be building unique game control gizmos by adding usb connections to cheap electronic toys. The workshop is an introduction to physical computing, covering some basic interaction design theory along with practical hardware hacking techniques. There’s more on the workshop and everything else that’s happening as part of the conference at isea2009.org.

First speaker was Gavin Burke from Galway based Future Audio Workshop who spoke about the physics of sound, how synth technology can reproduce these and his companies virtual synth product ‘Circle‘. Two years in development, Circle functions as a standalone instrument, a vst or a pro tools plugin and combines synth modeling with an interface that doesn’t try to reproduce the look of a hardware synths control panel but instead uses the full visual and control affordances the computer offers.

Next up Peter Kirn from CreateDigitalMusic examined the common metaphors that are used in music software, like the piano roll, the linear timeline and tape splicing and the possibilities for breaking out beyond these conventions. He cited current examples like the Ableton Live interface, the Tenori-on and many of the new touchscreen music apps for the iPhone like Brian Eno’s ‘Bloom’ and RJDJ. The possibilities of using Processing to make experimental music interfaces was also explored, one example shown was a circular sequencer that took inspiration from Balinese Gong circles.

We then viewed Niamh Ahern’s ‘Totally Wired‘, a documentary centering around Analog Synth store Schneiders Büro in Berlin, the musicians who buy their equipment there, the hardware manufacturers (many of which are based around Germany and are small family run businesses) and provides a great look into what motivates people to pursue Analog Synth hardware in this age of digital everything. Afterwards there was a Q+A with Niamh and Andreas Schneider. Andreas also presented an improv session on a patch cable controlled synth, and was so in the zone that afterwards he started talking to the audience in German :)

The final session was delivered by Mark Jenkins and Dave Vorhaus of White Noise and the BBC Radiophonic Workshop. Mark has wrote a book detailing the evolution of synth technology from analog through to virtual analog and took us on an audio journey through this development and his use of all forms of synths in his compositions.

Dave spoke of his experience recording the first ever electronic album ‘Electric Storm’ in 1968 while working at the BBC. The process was painstakingly laborious, the synth was tuned to play a note, this was recorded to tape. Then the oscillator was retuned, and the next note recorded… and so on. Interestingly he said this pioneering ‘cut and paste’ production method was inspired by the tape editing techniques of his father who was a blacklisted film director. Dave also said how he really doesn’t like keyboards because that their scale imposes limitations on how the instrument can be played, limitations he has sought to overcome with his double bass like Kaleidophon controller.

There were a couple of common themes through all these presentations. First was that all agreed (apart from one dissenting voice in the ‘Totally Wired’ documentary) that synth software along with regular computer hardware have finally reached the level of exactly modeling the sound qualities of analog hardware.

The other theme was the importance of the musician to instrument connection through the interface development, whether its through the cutting edge control and visual feedback of Circle or the visceral musician to hardware relationship that ensures the continued sales of analog gear at Schneiders Büro.

Either way the new language of electronic music control and notation is still under development and is going to keep evolving in new directions, not just due to commercial concerns but also due to the new found ease of access to software and hardware creation tools for the traditional end-user group, now able to undertake authorship of their own instruments and interfaces to create a new generation of music performance technologies.

‘Tiny Tag’ by team ‘Play It!’ is a graffiti simulator inspired by Tag Tool. ‘Play It!’ converted the steering wheel toy into their tag controller, allowing you to select a background, then pick your tags and move, rotate and scale them as you please.

Here’s a look inside the controller:

‘Music Maker’ by ‘Spanner in the works’ hacked this sound board toy into a controller for an ambient music generator.

A closeup of the keyboard hack, safely encased in an Ikea lunchbox!

Here’s a screenshot of Music Maker in action, when you hit a key on the board the corresponding shape lights up on screen and you hear a note, it plays the theme from ‘Close Encounters’ quite nicely!

Pixcube by the ‘Banjoes’ is an image browser, the ‘My First Keyboard’ was heavily modified to become the pixcube controller, with a new paintjob and several new buttons.

Here is the Pixcube itself, each side’s colour corresponds to a button on the controller.

When you press a button on the controller, the cube unfolds and you can navigate through the pictures, this is the portfolio side.

Magical Star Guitar by the ‘Three Amigos’ uses a converted Barney Guitar to control an game for young children that teaches counting and shape recognition. The battery case was carved away to make room for the keyboard module.

Finally, ‘Pass the Bomb’ by ‘Kawaii Productions’ combines this kids globe and a wireless keyboard to create an elimination game for up to six players.

Evil frogs have taken over the world and it’s only with the help of creatures from around the world that you can defeat them! After selecting the number of players the timer starts and you have to press the button corresponding to the continent shown onscreen, then pass it to the next person. If the timer goes to zero before you hit the button your eliminated!

Circuit Bending (2008) from Kieran Nolan on Vimeo.

Next up was a workshop by Brown Bag where they described their experiences using the animation toolset ‘Toon Boom Studio‘ to create a new children’s tv series ‘Wobbly Land‘. It seems to have a steep learning curve, particularly if you are used to Flash (no tweening!) but they had great technical support from Toon Boom’s creators, including 2 weeks of training. TB also created custom effects plugins for Brown Bag, that helped give the show it’s unique (and very non-generic vector animation) look.

The last workshop I attended was ‘Machinima FilmMaking’ (the art of making movies inside video game engines) by Friedrich Kirschner, the developer of Movie Sandbox, a heavy modification of the game Unreal Tournament 2004 that turns it into a Machinima application. The game world becomes your stage and the characters your actors, who you can direct as you please, either ‘live’ or through pre-defined scripts.

Some cool input methods were shown, one involved controlling your ‘actor’ like a virtual puppet using the strings on a usb golf controller. Another made use of the Xbox 360′s Guitar Hero controller so your machinima actor can jam on some chords.

Also demoed was an innovative supporting technology called Milkscanner, a low budget but highly effective method of scanning 3d objects using a webcam, lego, a bowl and some milk(!). A (small plastic) elephant was gradually submerged in milk, each stage captured by the webcam and then the software assembled each ‘slice’ into a 3d wireframe suitable for import into Movie Sandbox. Great stuff, innovative and nutricious!