Category: C++

Netduino Robot Drummer

The Netduino is Microsoft’s answer to the Arduino.  It runs using the .NET Micro Framework (.NETMF) which means you can program it using C# within Visual Studio and with whole bunch of useful .NET standard libraries at your disposal.  It is a breeze to setup and get started.

The goal of this project is to construct a robotic drummer using a microcontroller as the robot brain and a few solenoids as actuators that make drum sounds.  I had a Netduino lying around and it was the perfect choice for this project since multi-threading is supported natively. This helps to simplify the code base, since when I want to activate a solenoid, I can simply spawn a thread which turns on the solenoid, waits a moment and then turns off the solenoid, and I don’t have to worry about this delay effecting the timing of subsequent drum hits.  I can even trigger a drum roll or complicated riff if I want. Accomplishing the same task on an Arduino without the help of multi-threading is not too difficult either, but would take some extra effort, so this was a good excuse to try out the Netduino and it’s capabilities.

I eventually was able to leverage the .NETMF Threading and Timer classes for even more.  Here is a great reference on the basics of multithreading on .NETMF.  I have each Beat in a BeatPattern running on a separate “Timer” thread.  The KnockDetector also runs on a separate thread so the (relatively) heavy number crunching doesn’t interfere with the timing of the drums.

Playing a Beat Pattern

I wanted to use solenoids to produce drum sounds by banging on things.  One of the most frustrating things about the 12V push solenoids i had was that they had no frame and return spring.  To make them usable, I had to construct custom frames out of metal and find some really weak springs to spring them back to their starting position after they’ve struck a beat.  I purchased a variety pack of 100’s of springs but none of them were weak enough!  I finally found some that were sold individually at a local hardware store, but i had to “weaken” them further.  The strength of a spring depends on the thickness of the wound metal and also the radius of the winding, so stretching them out to increase that radius works.  Lacking a frame also made mounting your solenoids a challenge.  It is much easier to just purchase solenoids that include a frame and return spring.  You also end up with something less rusted and ugly (good thing they’ll be out of sight eventually)

The circuit for each solenoid resembled the following diagram, which comes courtesy of this Bildr page.

There are a few differences in my circuit.  My power source comes out of the netduino 5V pin, so its weaker than the 12V used in this diagram.  I also used a schottky diode instead of the zener diode pictured and a 1kΩ  resistor instead of a 2.2kΩ resistor.  If anyone has any insight on whether I’d be better off modifying my circuit, I would love to hear your suggestions, please do post a comment.

On the software side, I created a BeatPattern class which, among other things, consists of an array of Beats.  Each beat has its own thread which I set up using the .NETMF Timer library.  At the beginning of a measure, the main program calls BeatPattern.update() which tells each Beat in the pattern to get ready to play it’s corresponding solenoid at the right moment, depending on the tempo.

Here is an early version of the beat making portion of the project:

Knock Detection

Hardware-wise, knock detection was quite simple.  A piezoelectric buzzer can easily be used as a sort of knock or vibration or movement sensor if it’s correctly hooked up to an input pin on the microcontroller.  Below is a diagram for the arduino.cc tutorial on this topic.

The software side of knock detection was more tricky.  I set up a new thread to just listen for values coming from the knock sensor every 10 ms.  Since this is an analog sensor, it has to be hooked up to an analog pin on the Netduino and will give you a continuous value (as opposed to a discrete 1 or 0).  This means you end up getting a string of numbers for each hit.  For example a single knock may give you the following values:

0 - 0 - 0 - 55 - 127 - 358 - 82 - 0 - 0 - 0

It looks like you can just take the highest numbers between 0’s to get your knock’s timing.  This is true in some cases, however, a couple of knocks in quick succession may give you something like:

0 - 0 - 55 - 127 - 358 - 82 - 20 - 101 - 530 - 222 - 71 - 0 - 0

So to get both distinct knocks, I keep track of whether the current reading is on a “rising edge” or a “falling edge”.  If rising and current value is starting to fall, the last reading was the peak.  As always, see the code which is linked below for more details.

Finally I made a custom class called Knock with members “interval” and “volume”.  Volume is simply the peak number from above, and to calculate the interval I used a Stopwatch class (copy it into a code file and include it in your project).  Now I can store all my peaks in an array of Knocks for interpretation later.

Here are all the parts of this project prototype.  The 3 solenoids, the breadboard circuit, the microcontroller, and of course the box which will house everything and has the piezoelectric buzzer installed on top.  I also added a button because I’m considering using it as a very simple user interface to tell the drums when to start and stop.

Tempo Extraction

Unless your knocks are in a steady “4 on the floor” pattern (think techno), determining the user’s tempo is a difficult task.  Sometimes called “Beat Detection”, Tempo Extraction is a complicated process which is the subject of many scientific papers.  It involves a lot of number crunching and can bring your microcontroller, which lacks a powerful processor, to a grinding halt, so to speak.

Drawing clues from research papers I found online, I wrote a very simplified version of a Tempo Extraction algorithm which performs well and yields a fairly reliable beat interval reading.  It only works for the 4/4 meter which simplifies the task quite a bit.  Keep in mind, this means there are 4 beats in a measure.  A beat interval is the time between 2 consecutive beats.  The beat interval value is computed in milliseconds and can easily be converted to a BPM (beats per minute) by dividing 60K by it.  To convert back, again, divide 60K by your BPM value.

BPM = 60000 / BeatInterval

The rational behind the Tempo Extraction algorithm is that beats in a 4/4 pattern usually fall on a whole, half, quarter, or eighth note.  So consecutive beats should generally have one of these intervals between them.  I compare each beat to it’s previous beat, two beats ago, and three beats ago, then “normalize” the intervals by either dividing by 2 or multiplying by 2 since all the intervals should be multiples by this theory.

Then I perform the following steps on the array:

1.  Sort the array of intervals
2.  Find the median
3.  Throw out the outliers
4.  Take the average of what’s left

This gives me a pretty decent beat interval value.  See the code linked below for the nitty gritty, though keep in mind its a work in progress.   I would love to hear from anyone who has experience with any of the techniques described above and can suggest an improvements or alternative methods!

https://github.com/asaeed/RobotDrummer

On a side note, I noticed one shortcoming of .NETMF is that it lacks support for generics, which means a lot of nice built in functions – like Sort() on an ArrayList – are just missing.  I used extension methods to make my own algorithms on an ArrayList of integers.  But extension methods don’t work out of the box, there’s a little trick I learned thanks to this guy to get them working.

Arduino + WiFly sends OSC

 
OSC is a widely used protocol for wireless communication between devices.  Arduino is a tremendously popular platform to create physical computing devices.  So naturally, one would think that engaging in OSC communication using Arduino’s would be straight-forward and well documented.  Unfortunately, I found very little help on the internet on this topic.  Most searches led to one of two existing methods to get OSC out of an Arduino:
 
1.  Using Processing as a middleman – this requires sending data from the Arduino over serial to a computer running Processing, which then generates and transmits OSC.  So to talk between two Arduino’s, an entire computer is required?  No good.
 
2.  Using the Ethernet Sheild and a library called ArdOSC – excellent stuff, but this library is written to work with the ethernet library only, and I’m using an RN-XV WiFly module.  No wires.
 
Neither of these solutions worked for me so I figured this is a topic worth exploring further.  Previously, I had been able to use the RN-XV to send raw data over UDP.  OSC data is also transmitted over UDP, so it should just be a matter of correctly generating OSC on the Arduino.
 
Start Simple
 
To test this hypothesis, I took some simple OSC from OpenSoundControl.org
 2f (/)  6f (o)  73 (s)  63 (c)
69 (i) 6c (l) 6c (l) 61 (a)
74 (t) 6f (o) 72 (r) 2f (/)
34 (4) 2f (/) 66 (f) 72 (r)
65 (e) 71 (q) 75 (u) 65 (e)
6e (n) 63 (c) 79 (y) 0 ()
2c (,) 66 (f) 0 () 0 ()
43 (C) dc (Ü) 0 () 0 ()

The byte array equivalent in C for Arduino looks like this:

char data[] = {0x2f, 0x6f, 0x73, 0x63, 0x69, 0x6c, 0x6c, 0x61, 0x74, 0x6f, 0x72, 0x2f, 0x34, 0x2f, 0x66, 0x72, 0x65, 0x71, 0x75, 0x65, 0x6e, 0x63, 0x79, 0x00, 0x2c, 0x66, 0x00, 0x00, 0x43, 0xdc, 0x00, 0x00};

Since this is kind of difficult to look at and work with, we can instead use char arrays.  Char arrays can be more readable, but how do you convert a float to a char array?  There’s a neat “trick” to do this in C using the union keyword.  Here’s some working Arduino code for sending OSC messages with a single float argument.  The first message sends 0.1, the next 0.2, and so on until 1.0 is reached and then it’s reset back to 0.1.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#include "RNXV.h"
 
RNXV rnxv = RNXV();
float f;
union u_tag {
  byte b[4];
  float fval;
} u;
void setup() {
  f = 0.1;
 
  Serial.begin(9600);
  Serial1.begin(9600);
 
  rnxv.setUart(&Serial1);
  rnxv.join("SSID_HERE","PASSPHRASE_HERE");
  rnxv.enableUdp("192.168.1.5", 3000, 4000);
  rnxv.sendCommand("join", "AOK");
  Serial.println(rnxv.ip());
}
void loop()
{
  delay(1000);
  if (f > 1) f = 0;
  f = f + 0.1;
 
  char data[32] = "/oscillator/4/frequency\0,f\0\0";
  u.fval = f;
  data[28] = u.b[3];
  data[29] = u.b[2];
  data[30] = u.b[1];
  data[31] = u.b[0];
 
  Serial.print("print use loop: ");
  for (int i = 0; i <= 31; i++){
    Serial.print(data[i]);
  }
  Serial.println();
 
  rnxv.sendOsc(data, 32);
}

The code references an RN-XV library I’ve been working on to set up UDP and send the byte array as follows:

1
2
3
4
5
6
void RNXV::sendOsc(char data[], int size)
{
  for(int i = 0; i < size; i++)
    _uart->print(data[i]);
  delay(75);
}

I tried various delays and a delay of 75 seemed just enough that there wouldn’t be any corrupted OSC messages even when running the code for hours. This delay and the minimal code above resulted in about 13 OSC messages transmitting per second.  It is also necessary to loop and send the exact number of chars as a message contains, or else the receiving end doesn’t recognize the end of the message and throws errors.

 
On the receiving end, I once again used the sample app that comes with Cinder and it’s OSC Cinder block.  See the “Listening with Cinder” section in this post to see how to run the sample app, no code is required.  (Note that you may need to restart the app once because at first it inadvertently receives a message “exit” which is the RN-XV going into data mode).  If everything is working you should see a window like this one where the white section slowly expands:

 

 OscPkt and AVR-STL

At this point, having proved the concept, I was about to dive into writing my own library to generate the OSC, but thought I’d try one last time to find something off the shelf to do this.

OscPkt, the minimalittlest OSC library for C++ seemed perfect for the job, being super concise and all contained in a single header file. It also was not tightly coupled with any UDP library so I could use my sendOsc function (above). However it depended heavily on C++ Standard Library classes (like vector) which don’t exist for Arduino.

Except they do – a brilliant piece of work by Andy Brown is AVR-STL. It brings STL to Arduino. He has all the instructions on his blog post.  Simply download and put all header files in the include folder into …arduino-1.0.1\hardware\tools\avr\avr\include.  

Once AVR-STL was integrated, I took oscpkt.hh, renamed it to oscpkt.h and added the following two #includes at the top (as per the instructions found on the AVR-STL page):

1
2
#include 
#include

Then placed it in a folder “oscpkt” which I threw into my Arduino “libraries” folder. Now you can add the following to any Arduino sketch:

1
2
3
4
5
6
7
8
9
#include "oscpkt.h"
setup() {
  oscpkt::Message msg;
  oscpkt::PacketWriter wr;
  msg.init("/foo");
  wr.init().addMessage(msg);
  Serial.println(wr.packetSize());
  Serial.println(wr.packetData());
}
 
Previous posts about this project include the following topics:
 
1.  A low-cost and easy Robot Chassis to serve as the body of the robot.
2.  Adding a RN-XV WiFly Module to enable wi-fi connectivity.
3.  A Python Server which communicates with the robot.
4.  Adding a Sensor and Actuator and transmitting data as JSON.
 
In #3 and #4, I utilized a Python Server with JSON as the data format.  Here we try out a C++/Cinder Server with OSC as the communication protocol.
 

 

Cinder + Box2D

Box2D is a popular 2D physics engine for C++.  It’s free and open source and pretty easy to integrate with Cinder.  I recently tried out another engine called MSAPhysics and wrote a tutorial on setting it up with Cinder.  It’s a great little library, super easy to get up and running, and can even do a good job of 3D physics, however it lacks some advanced features and there’s very little community contributed code behind it.  So I thought I’d try out an alternative and see how they compare.  My friends at OneDayItWillMake and Wu-Media had spoken highly of Box2D and have used it in iOS games and with Cinder in Eclipse on OSX.  In this post, I attempt to integrate it with Cinder in VisualStudio 2010 on Windows 7.

Using b2cinder

First, I tried out a sorta Cinder block in progress called b2cinder by David Wicks (aka sansumbrella).  His sample project already takes care of some of the code needed to convert Box2D objects to Cinder objects and vice versa.  No coding needed for this part…

 
2.  Place sc-Box2D in cinder/blocks folder
 
3.  Open BasicBox2D sample app in vc10 folder
 
4.  Make sure these properties have correct paths:
    C/C++ –> General –> Additional Include Directories
    Linker –> General –> Additional Library Directories
 
5.  Had to add “using namespace ci;” to the following code files:
    Sandbox.cpp
    BoxElement.cpp
    BoundaryElement.cpp
    
6.  In Solution right-click Header Files and Add Existing…
    Add all files and sub-directories within sc-Box2D/include/b2Cinder and sc-Box2D/include/Box2D.
 
Now you should be able to run the app and this is what it looks like.
Integrating Box2D From Scratch
 
Next I wanted to try out the latest and greatest version of Box2D, since sansumbrella’s version was slightly older.  
 
1. Create BasicApp in TinderBox called Box2DTest
 
2. Download Box2D from  http://code.google.com/p/box2d/ 
 
3. Open vc10 .sln file and build Box2D project.
 
4. Copy Box2D.lib file created into new folder in Box2DTest called “lib”

5. In Linker –> General –> Additional Library Directories

    add path to “lib” folder you just created
 
    In C/C++ –> General –> Additional Include Directories 
    add path to Box2D_v2.2.1
 
    Note: be sure to add these to both Debug and Release run configs

6. Add this to top of code file:
    #include <Box2D/Box2d.h>
 
7. Under “Header Files”, add all .h and .cpp files to Solution.
 
8. Create a Box2D World with gravity.
    b2Vec2 gravity(0.0f, 10.0f);
    b2World world(gravity);
 
9. Add the file Conversions.h from b2cinder lib which does pretty good job of converting between box2d and cinder floats, vec2f, and more.
 
10. Code
 
Here is the source code on GitHub for a quick example I created using these steps:
 
Another screenshot: