I was very excited when I heard about Wekinator project. It looked like a very developed  implementation of the functionality I wanted.

I tried to hook it up to SuperCollider and Kinect.

Short note on Kinect: I’m using OSCeleton, which depends on OpenNI / NITE (info on installation is on the OSCeleton website). It tracks the sceleton and outputs the data through OSC. I’m getting that data in the SuperCollider with SC’s OscResponderNode and running the osceleton by ‘./osceleton -p 57120′ command (without quotes) from the terminal from the folder I built it in.

I decided to use Wekinator to map one OSC messages to another – map Kinect input to sound manipulation is SC. The first problem I encountered was sending the data from Wekinator to SC: Wekinator has a function to send messages to a generic OSC-based synth, however it does it only on port 12000. On the other hand – as great as an environment SuperCollider is – it cannot receive OSC messages on port other than 57120, at least as far as I know.
To make this communication possible I downloaded Wekinator source package and changed the hardcoded port to 57120. The two files that need to be adjusted are: ChuckConfiguration.java (line 41) and ChuckConfigurationForm.java (line 593). After that I built the program according to the enclosed instructions.

This way I managed to get the data flow: Kinect -> SC -> Wekinator -> SC (sound generation). See attached scKinectAndWekinator file.

However the issue with Wekinator is that it doesn’t allow – as far as I know – inputting OSC messages during the learning process. You need to specify the OSC values – as “snapshots” of settings – in Wekinator manually for the learning part. Once the system is “trained”, you can use continuous OSC input to get modified OSC output. But I wanted to have a continuous input for learning ability.

I found out about neural network implementation in SC. Next tests will be performed using it.