I was very excited when I heard about Wekinator project. It looked like a very developed implementation of the functionality I wanted.
I tried to hook it up to SuperCollider and Kinect.
Short note on Kinect: I’m using OSCeleton, which depends on OpenNI / NITE (info on installation is on the OSCeleton website). It tracks the sceleton and outputs the data through OSC. I’m getting that data in the SuperCollider with SC’s OscResponderNode and running the osceleton by ‘./osceleton -p 57120′ command (without quotes) from the terminal from the folder I built it in.
I decided to use Wekinator to map one OSC messages to another – map Kinect input to sound manipulation is SC. The first problem I encountered was sending the data from Wekinator to SC: Wekinator has a function to send messages to a generic OSC-based synth, however it does it only on port 12000. On the other hand – as great as an environment SuperCollider is – it cannot receive OSC messages on port other than 57120, at least as far as I know.
To make this communication possible I downloaded Wekinator source package and changed the hardcoded port to 57120. The two files that need to be adjusted are: ChuckConfiguration.java (line 41) and ChuckConfigurationForm.java (line 593). After that I built the program according to the enclosed instructions.
This way I managed to get the data flow: Kinect -> SC -> Wekinator -> SC (sound generation). See attached scKinectAndWekinator file.
However the issue with Wekinator is that it doesn’t allow – as far as I know – inputting OSC messages during the learning process. You need to specify the OSC values – as “snapshots” of settings – in Wekinator manually for the learning part. Once the system is “trained”, you can use continuous OSC input to get modified OSC output. But I wanted to have a continuous input for learning ability.
I found out about neural network implementation in SC. Next tests will be performed using it.
English
polski
Hey there,
Thanks for giving Wekinator a try. A few comments:
- If you grab the latest version of the software, you can set the OSC port to 57120 (or anything else you want) from within the OSC synth configuration GUI (same place you specify the # of parameters). No need to change the code.
- You can train from continuously changing examples in 2 different ways. First, you can use “playalong learning” in the Wekinator to specify a “score” of different parameter sets, specify the score to smoothly interpolate from each parameter set to the next, then creating training examples while gesturing along to the score. Or, you can use the fact that Wekinator is constantly listening for OSC messages indicating a parameter change initiated in the synth itself– within your supercollider code, a GUI for your SC code, etc., you can control your parameter changes and send them to Wekinator continuously. Again, you can create training examples by gesturing along in realtime.
Hope this helps. Send me an email or email the wekinator users mailing list if you have any questions.
Rebecca
Oh, I think I forgot to mention that you can look at OtherExamples/blotar_synth.maxpat for an example of an OSC synth sending continuous parameter updates back to Wekinator. Your OSC message just needs to be the message “/realValue” followed by a string of floats, one per parameter, sent to 6448 (documented here http://wiki.cs.princeton.edu/index.php/ChucK/Wekinator/Instructions#Editing_your_synth)