# niltoid
Ahmad Saeed builds things that respond: robots and LED rigs, interactive installations, games for activism, generated visuals, and AI tools. This site is a scrapbook of the projects he’s explored over the years.
## Dead Zone RTS
https://niltoid.com/blog/dead-zone-rts/
Date: 2026-10-19
Themes: games, ai-agents
Tags: JavaScript, Canvas, RTS, browser game
Skills: game development, game design, isometric graphics, JavaScript
It was an incredibly fun experience vibe coding this game from scratch using Claude Code and Codex. It only took about four weeks in total. What a time to be a maker.
Zombie survival meets real-time strategy. Dead Zone is a browser-based game that runs entirely on the front end, rendered with HTML Canvas. To stay alive, you must fortify your defences against hordes of zombies, scavenge supplies, build your camp and keep it lit through the night.
Building up the camp while it's still light.
Supplies have to be found and hauled back to camp. Sending people farther out can bring back more, but it also means running into zombies that have been waiting out there. Back at the base, there are walls, guard towers and places to train and heal your survivors.
Night makes things a bit more interesting. Vision shrinks, and defenders can only shoot zombies they can see in the light. Light towers and flares help keep the perimeter visible, so lighting the camp becomes part of building its defenses.
Night time, with another wave on the way.
Sprite animation
With no tooling other than those models, the graphics took several tries to get right. I went through some pains teaching Codex a strategy to create high-quality sprite animations of rigged 3D figures. It paid off in the end, and it's now encoded into a skill you can find here.
The game uses directional walking and running animations for its survivors and zombies.
Soldier walkingBuilder walkingHeavy walking
WalkerRunning exploderBrute walking
There's also an eight-direction turnaround for the armored truck:
Survival is endless: the waves keep coming until the headquarters falls.
Play the game
## Video Sync
https://niltoid.com/blog/video-sync/
Date: 2026-09-16
Themes: sound-visuals, ai-agents
Tags: tutorial, React, Rekordbox, MIDI, Ableton Link, VJ
Skills: Web MIDI, real-time graphics, JavaScript, C++
I've been working on a way to put film visuals behind a DJ set. Video Sync takes information from Rekordbox and a MIDI controller, then uses it to cut and mix video in time with the music.
The Live Console, with the scene queue and two-deck controls beside the video.
Each deck has its own video playback. Channel levels and the crossfader control the visual mix, while Hot Cues and Beat FX can change the loops and effects. A queue of scenes keeps the visuals moving as tracks change.
The graphics run in a browser, with a separate window for the projector. A small native bridge handles MIDI and Ableton Link, while a local relay carries the performance data. Scenes are reusable arrangements of clips, so they don't have to be built again for every song.
It's still an experimental proof of concept. The current hardware-tested setup is on macOS with Rekordbox and a DDJ-GRV6. Other controllers need their own mappings, and Windows deck-state integration is still to do.
The DDJ-GRV6 is the controller I use for testing. Photo: AlphaTheta.
From footage to a set
1. Sources & Clips
I start in Scene Studio by opening a scene or importing a local video. In the Sources & Clips tab, I find the bits I want, mark their in and out points, then name and color the clips. These are reusable ranges in the source footage, rather than separate copies of the film.
Sources & Clips: finding and trimming the footage.
2. Loop Builder
Next I drag those clips onto a timeline measured in bars and beats. I can arrange cuts, layer clips and adjust their speed, framing and opacity, then play the loop at a chosen BPM. The first eight loops map to Hot Cues A–H in list order; the star picks the scene's default loop.
Loop Builder: arranging the clips into musical loops.
3. Try it in the simulator
Save keeps the editable project; Publish makes it available to the Live Console. After publishing, refresh the console, load a deck, then choose the scene from its Visual scene menu. Without any DJ hardware attached, the on-screen controls act as a simulator: set the BPM, press Play, try the Hot Cues and move the channel levels and crossfader to check the mix.
4. Put it on the projector
Connect the projector as an extended display. In the Live Console, click Open projector output, move the new window onto the projector and click Full screen. The controls stay on the laptop while the projector shows the visuals. Refresh an already-open output window after publishing a revised scene.
For a live set, connect the DDJ-GRV6, open Rekordbox and run the native bridge alongside the local server. On macOS, the Node process also needs Accessibility permission to read the visible decks. The Bridge drawer in the console shows whether live input and the local media are ready before playing.
## Concert Radar
https://niltoid.com/blog/concert-radar/
Date: 2026-04-21
Themes: ai-agents, sound-visuals
Tags: Spotify, Node, Python, CLI
Skills: web scraping, data pipelines, CLI design
I wanted a way to find upcoming shows by the artists I actually listen to, without checking a pile of different sites. Concert Radar matches concert listings with my Spotify artists and the ones I add manually.
The dashboard, shown here for Los Angeles, with upcoming shows and my Spotify rotation side by side.
Underneath are two small command-line tools. spotify-cli reads my listening data, and concerts-cli gathers listings from Songkick, Bandsintown, Ticketmaster and local venue pages. The results are combined so I can look through them in one place.
There is a daily roundup and a local dashboard for browsing the shows. I can filter by artist, venue or date, and mark shows I'm interested in.
The tools return JSON, so they can also be used directly from the terminal or by an AI coding assistant. The Spotify CLI is linked above; the concert dashboard runs locally.
## 90s Party Mix
https://niltoid.com/blog/90s-party-mix/
Date: 2025-04-08
Themes: sound-visuals
Tags: DJ mix, SoundCloud, house music, 90s
Skills: DJing, mixing
A house set I played at a 90s party. Lots of throwbacks and fuzzy memories.
The full mix is about 44 minutes. Hit play below.
## Indentured Servitude
https://niltoid.com/blog/ind-serv/
Date: 2023-04-10
Themes: games
Tags: browser game, JavaScript, Canvas, Konva
Skills: game development, game design, JavaScript
This game was one of the last things I coded by hand, in the before times. I established the game and graphics engine and got it about 70% of the way there the old-fashioned way. Eventually I recruited Claude to help me finish the job.
It's a small browser game about being trapped in debt slavery at a Pakistani brick kiln—a hopeless situation. You and your family work endlessly to pay off a debt that you will never overcome.
Making bricks, with the debt always in view.
You shovel mud, mold bricks, bake them and load them onto a truck. Each brick earns a little money towards what you owe. Your wife and son join you, but so do the bills. Lodging, fuel and other charges keep adding to the debt. There is no win state. Getting better at making bricks doesn't fix the arrangement.
It's built with JavaScript, Canvas and Konva. The story events and dialogue live in data files and are triggered by progress through the game. The little sound effects are synthesized in the browser.
The game ends with some background on debt bondage and links to learn more. You can play it in a desktop or mobile browser.
The end screen after clicking Accept your fate, shown here using the game's development preview.
## Page Divider Studies
https://niltoid.com/blog/page-divider-studies/
Date: 2022-04-18
Themes: graphics-3d
Tags: SVG, JavaScript, CSS, animation
Skills: SVG animation, CSS animation, web animation
Four page-divider studies exploring different ways to mark a transition between sections.
The sketches range from an SVG scroll cue and an animated line to triangular and wavy dividers.
## sparks & honey
https://niltoid.com/blog/sparks-and-honey/
Date: 2022-02-01
Themes:
Tags: Squarespace, web development, marketing site
Skills: template development, UI development
I built a marketing site for sparks & honey as a freelance project, using a custom Squarespace template.
The homepage from the earlier site.
The reports page from the earlier site.
The reports section collects their intelligence reports in a grid of covers and titles, with links through to the individual reports.
The introduction to the Intelligence Reports section.
The site has since rebranded. These screenshots show the earlier version I worked on.
## VR Homeroom
https://niltoid.com/blog/vr-homeroom/
Date: 2020-10-17
Themes: graphics-3d, games
Tags: VR, Unity, OpenVR, WebSockets
Skills: C#, real-time sync, product design
I worked on Homeroom with a couple of friends to make running a VR arcade a little easier. One of them ran VR Bar in Dumbo, Brooklyn, so we had somewhere to try it with actual customers.
Homeroom's coming-soon page from 2018, recovered from the original site.
My part was the Unity and OpenVR overlay inside the headset. A customer could pick another game, check and extend their session, order a drink or ask for help, without taking the headset off.
Each station stayed in sync with a venue server over WebSockets. The staff side handled station reservations, signing people in, waivers and a wait list. A bit like managing tables in a restaurant, except the tables have people waving controllers around.
Our team included a 3D animator working in Maya and another developer working on the WebSocket server, with me on the overlay. We had a working prototype and two beta customers when the pandemic closed VR Bar and most of the venues we were hoping to work with.
## Quick Comic
https://niltoid.com/blog/quick-comic/
Date: 2020-02-25
Themes: graphics-3d
Tags: Vue.js, Vuex, interact.js, Flash
Skills: JavaScript, drag and drop, UI development
Quick Comic was a little site for making political comic strips. I built the original with Eric Socolofsky, with illustrations by Alex Eben Meyer.
The original Flash editor.
You picked a character, dragged it into a panel and gave it something to say. The original ran on Flash with an ASP.NET API.
A panel from the original site.
In 2020 I started rebuilding it in Vue.js. So far the rewrite has a four-panel strip, drag-and-drop characters with interact.js, and an editable title held in Vuex. A small start at bringing the old editor back to the browser.
## Food Photo
https://niltoid.com/blog/food-photo/
Date: 2018-11-30
Themes:
Tags: React, AWS Lambda, S3, Serverless
Skills: JavaScript, camera capture, serverless development
I built Food Photo pro bono for an intermittent fasting study. The idea was simple: let participants take photos of their meals on their phones, and give the researchers a way to review them.
A test photo in the preview screen.
It's a small React app. Take a photo, add an optional caption, mark it as before or after a meal, and submit. The images go to S3. A scheduled job sends text reminders, and a separate report page lets researchers browse the photos.
The start screen, under the study's 2-A-Day name.
The backend uses AWS Lambda and the Serverless Framework. I also tried serving the report page directly from a Lambda function. It worked, though I wouldn't make it my first choice for serving a website.
## Alliance for Positive Change
https://niltoid.com/blog/alliance-for-positive-change/
Date: 2017-07-01
Themes:
Tags: Squarespace, nonprofit, web development
Skills: template development, UI development
I worked on a custom Squarespace template for Alliance for Positive Change while I was at frog. It's a small nonprofit based in New York City.
The homepage from the original Squarespace implementation.
The site brings together the organization's services, ways to get involved, and news and events. My part was the template implementation, bringing the design into Squarespace.
The Squarespace site linked above is a copy of the version we launched.
The Get Care overview page in the same template.
## Dots
https://niltoid.com/blog/dots/
Date: 2016-07-19
Themes: graphics-3d
Tags: three.js, particles, point clouds, WebSockets
Skills: WebGL, real-time graphics, JavaScript
Some more experiments with three.js. Dots is a way to draw thousands of particles in the browser and move them around without making a separate object for every dot.
Another view of the particle experiments.
One of the particle experiments running in the browser.
It uses a point cloud, BufferGeometry and custom shaders to control the positions, sizes and colors. Separate animators work on the same particles, with experiments for waves, spheres, physics and points arranged around a 3D model. The camera moves between views too.
There is also an input path for camera tracking data over WebSockets. It's a collection of visual experiments rather than a finished app. Click the demo to switch between them.
The particles arranged into a sphere.
## CubeGrid
https://niltoid.com/blog/cubegrid/
Date: 2016-04-21
Themes: graphics-3d
Tags: three.js, RealSense, openFrameworks, Electron, WebSockets
Skills: WebGL, depth cameras, C++, C#, real-time graphics
Another experiment with three.js, this time using a grid of cubes and an Intel RealSense depth camera. Movement in front of the camera flips the cubes on screen.
The browser version, before any camera input.
The camera talks to a small C# server, which sends tracking data to the browser over WebSockets. The browser handles the drawing and animation. This was also a workaround for the JavaScript support in the RealSense SDK at the time.
I also packaged it as an Electron app and made an openFrameworks version with support for multiple RealSense sensors. Same idea, a few different ways of connecting the camera to the graphics.
## Raspberry Pi Telepresence Robot
https://niltoid.com/blog/simple-telepresence-robo/
Date: 2016-03-03
Themes: robots-hardware
Tags: Raspberry Pi, Arduino, Node, robotics, video
Skills: video streaming, motor control, sensor integration
A telepresence robot allows users to interact with remote environments. You see what the robot see’s, hear what it hears, and you’re able to command it to move around and pick up objects. This project is a very simplified version of a telepresence robot. It includes the following features:
– live video streaming
– pan and tilt control
– motion sensor data display
## Unity RTS
https://niltoid.com/blog/unityrts/
Date: 2016-01-20
Themes: games
Tags: Unity, C#, RTS
Skills: game development, C#, prototyping
I started playing with a small real-time strategy game in Unity. You select a survivor, send them around the map, and try to deal with the zombies wandering in. Very much a prototype.
The unit logic is written in C#, with Unity's NavMesh agents handling movement. There are separate behaviours for survivors and zombies, and mouse and touch input for selecting things on the map.
Survivors can shoot at targets, while zombies lurk, wander and chase survivors. The bullets use an object pool so they can be reused instead of constantly creating new objects.
This is mostly an experiment in unit controls and enemy behaviour. There are notes in the code about gathering resources and building defenses, but those are ideas rather than a finished game.
## Isometric Cubes
https://niltoid.com/blog/isometric-cubes/
Date: 2015-09-03
Themes: graphics-3d
Tags: JavaScript, Canvas, IsomerJS
Skills: isometric graphics
An HTML canvas experiment using IsomerJS
## Space Shooter
https://niltoid.com/blog/space-shooter/
Date: 2014-05-13
Themes: games
Tags: JavaScript, Canvas, CodePen, game prototype
Skills: game development, animation, JavaScript
A couple of small space-shooter sketches I made on CodePen, using JavaScript and Canvas.
The first version is just a ship, a scrolling starfield and shooting. The second adds enemies that move towards the ship, with some basic collision detection for the bullets.
Both versions are embedded below, without enemies first. Click inside the first demo, then use the arrow keys to move and Space to shoot.
The enemy version currently needs a compatibility fix before it will run again.
## Vine Explorer
https://niltoid.com/blog/vine-explorer/
Date: 2013-11-15
Themes: sound-visuals
Tags: JavaScript, Python, HTML5 video, three.js, Box2D
Skills: video interfaces, physics simulation
Vine Explorer allows you to search and browse these 6-second videos in unusual ways. The Vine data is supplied using this Python API. The site is an exploration of HTML5 video elements, three.js, box2djs, and torture by being subjected to endless human absurdity.
## Portable Power
https://niltoid.com/blog/portable-power/
Date: 2013-07-27
Themes: robots-hardware
Tags: Raspberry Pi, batteries
Skills: hardware integration
My friend Jamie told me about this power supply available for $12 on one of those group deal sites. It’s 2600mAh which seems to power an idle Raspberry Pi for about 5 hours!
## HTML5 Canvas Animation
https://niltoid.com/blog/html5-canvas-animation/
Date: 2013-04-15
Themes: graphics-3d
Tags: JavaScript, Canvas, particles
Skills: animation, real-time graphics
A few simple experiments with HTML5 Canvas Animation
## Raspberry Pi + Arduino + Tornado
https://niltoid.com/blog/raspberry-pi-arduino-tornado/
Date: 2013-03-02
Themes: robots-hardware
Tags: tutorial, Raspberry Pi, Arduino, Python, Tornado, WebSockets
Skills: multiprocessing, serial communication, real-time sync
Tornado is a great open source, Python based web framework. It is designed to be light-weight, easily scalable, non-blocking and it supports websockets and some other nice features. What’s more, it runs smooth and quick on the Raspberry Pi.
Thus far, my experiments with the Raspberry Pi have involved running Node.js on it to serve web pages and to interact over serial with an Arduino. Here are some posts on how to do that:
1. Raspberry Pi Setup
2. Raspberry Pi + Arduino: Getting Started
3. Raspberry Pi + Arduino + Node = Home Automation
I wanted to try accomplishing something similar using Python. Specifically, a Python program which can simultaneously handle web requests and communicate with an Arduino. I had always heard great things about Tornado and noticed some of the Raspberry Pi community had success with it.
Tornado Setup
First things first, install Tornado on your Raspberry Pi:
1. Good thing to do before every install – update agt-get and installed packages
sudo apt-get update
sudo apt-get upgrade
2. Install easy_install and pip (which are Python package managers)
sudo apt-get install python-pip python2.7-dev
3. Update your easy_install (and pip) package definitions
sudo easy_install -U distribute
4. Now install Tornado using pip
sudo pip install tornado
5. Follow the steps in this previous post to install Arduino on your Raspberry Pi
sudo apt-get install arduino
sudo usermod -a -G tty pi
sudo usermod -a -G dialout pi
6. At this point might as well also install GPIO package as well, in case you need it later
sudo pip install RPi.GPIO
7. Install another Python package to serial communication
sudo apt-get install python-serial python3-serial
To make sure everything works, create a file named “server.py” containing the following code (taken from Tornado’s site), and run it from terminal using the command “python server.py”
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
application = tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
application.listen(8080)
tornado.ioloop.IOLoop.instance().start()
Now on a separate computer on the same network, navigate your browser to http://raspberrypi:8080 (or use the IP address for example 192.168.1.7:8080). You should see “Hello, world” if everything goes right.
Adding Websockets
Adding websockets to your Tornado app is easy. Below is the modified version of server.py. It also includes support for command line arguments on line 8 (so you can run the program using “python server.py –port=8080”).
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.websocket
from tornado.options import define, options
define("port", default=8080, help="run on the given port", type=int)
class IndexHandler(tornado.web.RequestHandler):
def get(self):
self.render('index.html')
class WebSocketHandler(tornado.websocket.WebSocketHandler):
def open(self):
print 'new connection'
self.write_message("connected")
def on_message(self, message):
print 'message received %s' % message
self.write_message('message received %s' % message)
def on_close(self):
print 'connection closed'
if __name__ == "__main__":
tornado.options.parse_command_line()
app = tornado.web.Application(
handlers=[
(r"/", IndexHandler),
(r"/ws", WebSocketHandler)
]
)
httpServer = tornado.httpserver.HTTPServer(app)
httpServer.listen(options.port)
print "Listening on port:", options.port
tornado.ioloop.IOLoop.instance().start()
And here is the corresponding index.html, which connects to the websocket server. I used KineticJS to quickly create a simple red circle on a canvas which listens from click events. Be sure that your “ws://” url is correct.
Now to test again start up your python script by executing “python server.py” in your terminal. Next, direct your browser to http://raspberrypi:8080. A couple of lines will show up in your terminal showing a connection is made. In the browser you should see a small red circle and when you click it look for the following output in both your browsers console and your Raspberry Pi terminal (see screenshots).
Adding Multiprocessing
One key difference between Node and Python, however is that Node is by nature highly asynchronous and event-driven, while Python is generally not. With Node, you can easily set up events to be fired for both incoming web requests from clients as well as incoming serial data from the Arduino. This is demonstrated in the previous post. In a Python + Tornado setup, your program is in a single continuous IO loop waiting for web requests, which is initiated by the last line of every Tornado app:
tornado.ioloop.IOLoop.instance().start()
Any code placed after it won’t execute. You’re allowed to interrupt this IO loop (see PeriodicCallback) to perform other tasks, for example talk to an Arduino over serial, but web requests will be blocked while other processing is going on. Another option is to use a component of Tornado called Gen which can help you set up the asynchronous execution of a separate loop. However, what I found to be the cleanest solution was to use Python’s own native “multiprocessing” module to spawn a separate process to perform the needed serial communication.
You’re probably thinking “That Node stuff was way easier. Isn’t multiprocessing a huge pain in the rear cause you have to deal with locks and concurrency and stuff like that?” Well yes, but for what we’re trying to do it doesn’t get that complex, as you’ll see. Also, Node’s asynchronicity is a double edged sword. As your Node (JavaScript) code base grows, its easy to fall into the fiery pit known as callback hell. Seasoned JavaScript developers have ways to get around this, but I found it to be a nuisance. Personally I find a larger Python program to be easier to trace through and maintain. Ultimately, which language/programming style you go with will come down to your personal preference. Try out both and pick your poison.
There’s an excellent tutorial about Python multiprocessing by Doug Hellmann. It’s all I needed to get up and running with the module and I would urge anyone interested in the topic to check it out.
Inter-process communication is accomplished using the Queue data structure which comes with the multiprocessing module. Two queue’s are created, one for outgoing messages (taskQ) and one for incoming messages (resultQ). The queue’s are then passed to the spawned SerialProcess which sends anything in the taskQ to the Arduino over serial and puts any incoming data from the Arduino onto the resultQ. Tornado monitors the resultQ using PeriodicCallback.
A complete round trip looks like this:
1. Client sends a message to Tornado
2. Tornado sends it to SerialProcess using taskQ
3. SerialProcess sends it to the Arduino over serial
4. Arduino sends something back to SerialProcess over serial
5. SerialProcess sends it to Tornado using resultQ
6. Tornado sends it to all Clients
Below are all the code files (except index.html which remains the same as above). Here’s the final version of server.py:
#!/usr/bin/python27
import tornado.httpserver
import tornado.ioloop
import tornado.web
import tornado.websocket
import tornado.gen
from tornado.options import define, options
import time
import multiprocessing
import serialProcess
define("port", default=8080, help="run on the given port", type=int)
clients = []
class IndexHandler(tornado.web.RequestHandler):
def get(self):
self.render('index.html')
class WebSocketHandler(tornado.websocket.WebSocketHandler):
def open(self):
print 'new connection'
clients.append(self)
self.write_message("connected")
def on_message(self, message):
print 'tornado received from client: %s' % message
self.write_message('got it!')
q = self.application.settings.get('queue')
q.put(message)
def on_close(self):
print 'connection closed'
clients.remove(self)
################################ MAIN ################################
def main():
taskQ = multiprocessing.Queue()
resultQ = multiprocessing.Queue()
sp = serialProcess.SerialProcess(taskQ, resultQ)
sp.daemon = True
sp.start()
# wait a second before sending first task
time.sleep(1)
taskQ.put("first task")
tornado.options.parse_command_line()
app = tornado.web.Application(
handlers=[
(r"/", IndexHandler),
(r"/ws", WebSocketHandler)
], queue=taskQ
)
httpServer = tornado.httpserver.HTTPServer(app)
httpServer.listen(options.port)
print "Listening on port:", options.port
#tornado.ioloop.IOLoop.instance().start()
def checkResults():
if not resultQ.empty():
result = resultQ.get()
print "tornado received from arduino: " + result
for c in clients:
c.write_message(result)
mainLoop = tornado.ioloop.IOLoop.instance()
scheduler = tornado.ioloop.PeriodicCallback(checkResults, 10, io_loop = mainLoop)
scheduler.start()
mainLoop.start()
if __name__ == "__main__":
main()
A new file called serialProcess.py is as follows:
import serial
import time
import multiprocessing
class SerialProcess(multiprocessing.Process):
def __init__(self, taskQ, resultQ):
multiprocessing.Process.__init__(self)
self.taskQ = taskQ
self.resultQ = resultQ
self.usbPort = '/dev/ttyACM0'
self.sp = serial.Serial(self.usbPort, 115200, timeout=1)
def close(self):
self.sp.close()
def sendData(self, data):
print "sendData start..."
self.sp.write(data)
time.sleep(3)
print "sendData done: " + data
def run(self):
self.sp.flushInput()
while True:
# look for incoming tornado request
if not self.taskQ.empty():
task = self.taskQ.get()
# send it to the arduino
self.sp.write(task + "\n");
print "arduino received from tornado: " + task
# look for incoming serial data
if (self.sp.inWaiting() > 0):
result = self.sp.readline().replace("\n", "")
# send it back to tornado
self.resultQ.put(result)
Finally, here’s the Arduino program which simply listens for incoming data and returns it:
#define LED_PIN 13
int msgByte= -1; // incoming byte
const int msgSize = 50; // max message size
char msgArray[msgSize]; // array for incoming data
int msgPos = 0; // current position
void setup() {
Serial.begin(115200);
pinMode(LED_PIN, OUTPUT);
}
void loop() {
handleSerial();
}
void handleSerial() {
if (Serial.available() > 0) {
digitalWrite(LED_PIN, HIGH);
msgByte = Serial.read();
if (msgByte != '\n') {
// add incoming byte to array
msgArray[msgPos] = msgByte;
msgPos++;
} else {
// reached end of line
msgArray[msgPos] = 0;
// here the message is processed
// for now just send it back over serial
Serial.println(msgArray);
// reset byte array
for (int c = 0; c < msgSize; c++)
msgArray[c] = ' ';
msgPos = 0;
digitalWrite(LED_PIN, LOW);
}
}
}
Test it out as shown above and you should now see the following output (click to enlarge):
## Raspberry Pi + Arduino + Node = Home Automation
https://niltoid.com/blog/web-controlled-rgb-led-pixels/
Date: 2013-02-09
Themes: robots-hardware
Tags: tutorial, Raspberry Pi, Arduino, Node, Socket.IO, KineticJS, LEDs
Skills: real-time sync, hardware integration, Canvas
After figuring out how to set up the Raspberry Pi and how to communicate between a Raspberry Pi and an Arduino, I wanted to try a simple home-automation project that incorporated several really awesome and useful technologies:
– Raspberry Pi
– Node
– Socket.IO
– Canvas with KineticJS
– Arduino
– RGB LED Pixels
The following diagram shows how all this tech is arranged.
This is by no means the only way to do home-automation, but its a fun project if you don’t mind programming, and there are a lot of advantages to using these particular tools. Having a browser based interface allows you to work the controls from virtually any computer or mobile device. Canvas gives you the ability to create a rich interactive experience for the end-user. KineticJS makes it incredibly easy to make canvas animations. The Raspberry Pi gives you the benefit of a fully functional web server and allows the project to be portable. Node makes developing that web server as well as serial communication with the Arduino quick and easy. Websockets and Socket.IO allow the server to easily push updates to all clients in real time, so if I tweak a slider, everyone viewing the control panel sees me move it. Finally the Arduino gives us an easy way to control the physical world and access to countless libraries including one that helps to work with the RGB LED Pixels .
Here is a video of the whole thing in action:
In my previous tests, Firmata was a great choice for integrating the Pi and Arduino since it simplified the code base – instead of writing a C program for the Arduino, the Arduino was controlled directly from my Node program. In this case however, I opted to leverage this Adafruit Arduino library to work with the RGB LED Pixels. So it made more sense to write a separate Arduino sketch which incorporates this lib and communicates with Node over serial. Couple of things to note:
1. Arduino libraries go in this location:
/usr/share/arduino/libraries/
2. Node serial communication can be done using this package
sudo npm install -g serialport
3. Optional: Since the Arduino IDE is quite slow on the Pi, Ino is a great option for building and uploading arduino code from the command line.
Here is our holiday tree outfitted with RGB LED Pixels from Adafruit.com
From another angle you can see the attached Raspberry Pi and Arduino (in the dark)
A clearer daytime pic of the setup. Breadboard with BBB Arduino Clone –> USB BUB –> male-to-male mini-A USB adapter –> Raspberry Pi
The Raspberry Pi is running a Node web server which communicates with clients using Socket.IO. Here it is in its entirety. For the canvas and arduino code, see the Github repo below.
When on the same network as the Pi, navigating a browser to http://raspberrypi:8080 brings up this canvas based control panel. I used KineticJS to make this happen faster
Next, I’d like to take this project further by having a line of communication coming back from the Arduino to the canvas front-end. When multiple clients are looking at the control panel and one user moves the sliders, everyone should see them move in close to real time.
In addition to my desktop browser, I tried this out from my Android device (LG Nexus 4 running Chromium). I was disappointed to find that the animation is clunky and jumpy, and touch control is inaccurate. I’ve heard that unless they’re very simple, canvas animations require extensive optimization to work in mobile browsers. I have friends that are struggling with similar issues, so hopefully there will be some advancement in this area on mobile browsers to make canvas truly universal.
The source code for this project is available here on Github.
## Raspberry Pi + Arduino: Getting Started
https://niltoid.com/blog/raspberry-pi-arduino/
Date: 2013-01-09
Themes: robots-hardware
Tags: tutorial, Raspberry Pi, Arduino, Node, Firmata
Skills: serial communication, hardware integration
The Raspberry Pi kind of looks like an Arduino replacement as far as it’s physical appearance. At first glance, it’s specs are much more impressive for the same price point as an Arduino. However, it is by no means an “Arduino killer”. You could certainly swap out one for the other in some simple use cases. However the two devices have very different capabilities, and I find that they actually complement each other quite well when combined.
Pictured here left to right: Raspberry Pi, Arduino Mega, Arduino NG, Arduino Pro Mini
Here is one important reason the Arduino is still essential, from the author of Getting Started with Raspberry Pi:
When you have a problem that requires exact control in real time, such as a controller for a 3D printer. Raspbian is not a Real Time operating system, and programs can’t necessarily depend on the same “instruction per clock cycles” rigor of a microcontroller.
Here are some other ways the Arduino has a leg up:
– Supports analog pins required for many types of sensors and actuators
– Supports a larger number of digital pins
– Includes PWM pins (pulse width modulation) used for some types of sensors and actuators
– Lower power consumption – Pi is about 7x the power consumption – ie, Arduino will last about 7x longer on battery
– Some Arduino clones are cheap as fudge
– Lots of parts made to integrate directly with Arduino, such as sheilds.
– Lots of code libraries exist for the Arduino
As for the Pi, the advantages are pretty obvious:
– Still the most processing power for your money
– Ability to easily program and drive graphics
– Integration with computer peripherals
– Access to tons of open source Linux and Python tools
– Easier to use for advanced network functionality (VNC on to it, run a websocket server, etc)
– Much more…
When connected to each other, the two devices can be use for the tasks they are best at. The Arduino for controlling hardware and the Pi for controlling software. There are a number of ways for them to communicate:
1. They can have their own separate programs running and communicate over Serial, I2C, or SPI (only tried Serial so far)
2. You can use something called “firmata” to control a connected Arduino directly from Node or Python on the Pi (eg: Node Firmata)
The advantage of #1 is that you can still use libs written for Arduino to interface with hardware more efficiently. The advantage of #2 is that you have just a single code base to maintain, without the need for context switching. A HUGE plus with any method is that you can program your Arduino by VNC’ing into the Pi. this is great if your setup is mounted somewhere inaccessible like on the ceiling or inside a casing.
It’s hard to see what’s going on in the above images. On the breadboard is an Arduino clone called the BBB from ModernDevice.com (white). To cut it’s cost, it doesn’t have an on-board USB port, so to give it USB I use an component called a USB BUB (red). Then there’s a male A to male mini-A USB adapter (black). Finally I can connect the Raspberry Pi (green).
Install Arduino
Anyways, enough talking and more doing. To get started, install the Arduino IDE on the Raspberry Pi:
sudo apt-get update
sudo apt-get install arduino
You’ll need to do this before running the Arduino IDE.
sudo usermod -a -G tty pi
sudo usermod -a -G dialout pi
Now you can launch Arduino IDE while running X11 by going to Start menu -> Programming -> Arduino IDE. If your Pi is on WiFi, even better – VNC into it for remote and wireless control. Keep in mind, you now have slightly more power consumption which may require you to use a power adapter with more amperage. I was able to power a Pi, hooked up to an Arduino, hooked up to 25 addressable RGB LED pixels using a 5V 1.2A phone charger with no problem.
Install Node
I decided to start out programming on the Pi a little differently. The more common way is to use Python which comes pre-installed. I’ve recently been exploring Node.js and decided to see how it plays with the Pi.
Install Node.js with the following commands:
wget http://nodejs.org/dist/v0.8.17/node-v0.8.17.tar.gz
tar -zxf node-v0.8.17.tar.gz
cd node-v0.8.17
./configure
make
sudo make install
Next create a file as follows to test (must use 0.0.0.0 instead of 127.0.0.1):
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(8080, "0.0.0.0");
console.log('Server running at http://0.0.0.0:8080/');
To test, open up a browser on a computer which is on the same network as the Pi, and navigate to http://raspberrypi:8080 (or the IP address for example http://192.168.1.7:8080). You should see “Hello World”.
Now that we have Arduino and Node working individually, lets get them working together.
Node + Arduino + Firmata
As mentioned above, Firmata is an interesting way to control the Arduino and get out of writing any C code. It wouldn’t be my preferred method since you lose out on some amazing open source Arduino libraries, but it’s a really easy way to control an Arduino on a very basic level.
1. Install Node and Arduino as explained above
2. Run Arduino IDE, open Examples->Firmata->StandardFirmata, and upload it to your Arduino board.
3. From terminal install the package firmata for Node:
sudo npm install -g firmata
4. Create a file called app.js containing the following code
/**
* Sample script to blink LED 13
*/
console.log('blink start ...');
var ledPin = 13;
var firmata = require('/usr/local/lib/node_modules/firmata');
var board = new firmata.Board('/dev/ttyUSB0', function(err) {
if (err) {
console.log(err);
return;
}
console.log('connected');
console.log('Firmware: ' + board.firmware.name + '-' + board.firmware.version.major + '.' + board.firmware.version.minor);
var ledOn = true;
board.pinMode(ledPin, board.MODES.OUTPUT);
setInterval(function(){
if (ledOn) {
console.log('+');
board.digitalWrite(ledPin, board.HIGH);
}
else {
console.log('-');
board.digitalWrite(ledPin, board.LOW);
}
ledOn = !ledOn;
},500)
});
5. Run the app as sudo:
sudo node app.js
Integrate Socket.io
So far, we’ve created a simple Node server script that controls the Arduino. How about giving it a front-end? That way you can interact with your Arduino using a web browser. Just for kicks, why not use websockets since that’s a nice and easy-to-work-with feature Node offers. Socket.io makes it even easier. Simply install Socket.io using the following command:
sudo npm install -g socket.io
This is what I used for index.html:
web led switch
This is app.js:
var app = require('http').createServer(handler),
io = require('/usr/local/lib/node_modules/socket.io').listen(app),
fs = require('fs'),
firmata = require('/usr/local/lib/node_modules/firmata'),
board = new firmata.Board('/dev/ttyUSB0', arduinoReady);
var ledPin = 13;
function arduinoReady(err) {
if (err) {
console.log(err);
return;
}
console.log('Firmware: ' + board.firmware.name
+ '-' + board.firmware.version.major
+ '.' + board.firmware.version.minor);
var ledOn = true;
board.pinMode(ledPin, board.MODES.OUTPUT);
}
app.listen(8080);
console.log("Listening on http://raspberrypi:8080...");
// directs page requests to html files
function handler (req, res) {
fs.readFile(__dirname + '/index.html',
function (err, data) {
if (err) {
res.writeHead(500);
return res.end('Error loading index.html');
}
res.writeHead(200);
res.end(data);
});
}
// this handles socket.io comm from html files
io.sockets.on('connection', function(socket) {
socket.send('connected...');
socket.on('message', function(data) {
if (data == 'turn on') {
console.log('+');
board.digitalWrite(ledPin, board.HIGH);
socket.broadcast.send("let there be light!");
}
if (data == 'turn off') {
console.log('-');
board.digitalWrite(ledPin, board.LOW);
socket.broadcast.send("who turned out the light?");
}
return;
});
socket.on('disconnect', function() {
socket.send('disconnected...');
});
});
## Raspberry Pi Setup
https://niltoid.com/blog/raspberry-pi-setup/
Date: 2013-01-01
Themes: robots-hardware
Tags: tutorial, Raspberry Pi, Raspbian, Linux
Skills: Linux administration, remote access
I ordered a Raspberry Pi (Model B Rev 2) back in March and it finally arrived in November. Since, I’ve been exploring the capabilities of this $35 computer-on-a-chip. For its price and size it packs a very formidable punch. From the raspberrypi.org FAQ:
… graphics capabilities are roughly equivalent to Xbox 1 level of performance. Overall real world performance is something like a 300MHz Pentium 2, only with much, much swankier graphics.
Turn it into a arcade game console, a media center, a portable music player, pair it with an Arduino, or simply use it as a computer for word processing and internet browsing, the possibilities are endless.
Up and Running
I found the Adafruit tutorials to be helpful for getting started. First off, setting up the SD card and the initial config…
1. Tutorial to setup SD card
2. Tutorial to do rasp-config
In addition, also change_locale as US – UTF8 – UTF8
and configure_keyboard to make it American layout
Now at this point the Pi started inexplicably crashing after 5-10 minutes of being on. Looking online, I was disappointed to learn that some others experienced this and the unit could be from a faulty batch. The next day I decided to give it another shot, this time removing the Pi from this case purchased from RS Online. That seemed to fix the issue and I have had no crashes since! It appears the case was causing the Pi to overheat and crash. Since I’ve discarded the case, I’ve been able to run the Pi without fail for days at a time.
“Headless” mode
There are a couple ways to control your Pi remotely and free up the monitor and keyboard it is connected to.
a) SSH access using Putty is possible by default. The only setting needed is the Host Name for which you can use the Raspberry Pi’s IP address or “raspberrypi”. Once prompted for login/password, use “pi”/”raspberry”.
b) VNC into an X11 session using these instructions.
Note that if you configured to launch X11 on startup, these instructions need a slight modification – use :1 instead of :0
Git installation and Dropbox integration
Dropbox Uploader is a great tool written in bash which makes it easy to automate reading from and writing to Dropbox.
a) Install Git using the command: sudo apt-get install git-core
b) To install Dropbox Uploader script
cd /usr/local/bin
sudo git clone https://github.com/andreafabrizi/Dropbox-Uploader.git dropbox
c) optional: delete everything except the file dropbox_uploader.sh
d) run dropbox_uploader.sh and follow instructions
Now you can call dropbox_uploader.sh from anywhere.
Setup WiFi
To set up a WiFi USB Adapter, modify file /etc/wpa_supplicant/wpa_supplicant.conf by appending this after update_config=1
network={
ssid="YOUR_SSID"
scan_ssid=1
psk="YOUR_PASSPHRASE"
proto=RSN
key_mgmt=WPA-PSK
pairwise=CCMP
auth_alg=OPEN
}
Key:
ssid: your wifi name
scan_ssid: value of 1 means broadcast and value of 2 means hidden (suggest value of 1)
psk: your wifi password
proto: your choice of RSN or WPA. RSN is WP2 and WPA is WPA1. (most config is RSN)
key_mgmt: your choice of WPA-PSK or WPA-EAP (pre-shared or enterprise respectively)
pairwise: your choice of CCMP or TKIP ( WPA2 or WPA1 respectively)
auth_alg: OPEN option is required for WPA and WPA2 (other option, SHARED & LEAP)
Keeping the Screen On
When in X Windows, the screen goes blank after 10 min and needs to be “woken”. There are many suggestions in the forums but most failed to keep my Raspi awake. This solution which is a sort of attack on all fronts finally worked out. Here’s an excerpt:
Approach 1
On Pi 1 I don’t want screensaver – but don’t want to autorun x. When I start up x I do it manually and it references /home/pi/.xinitrc so managed to stop the screensaver on there by using information from http://raspberrypi.stackexchange.com/qu … oing-blank
apt-get install x11-xserver-utils
Now open up your ~/.xinitrc file (if you don’t have one then create it) and enter this:
xset s off # don’t activate screensaver
xset -dpms # disable DPMS (Energy Star) features.
xset s noblank # don’t blank the video device
Approach 2
On Pi 2 I have it auto-logging in and auto-starting to x using the raspi-config script use of lightdm. This seems to ignore /home/pi/.xinitrc
For this I used belt and braces.
The Belt:
Edit /etc/lightdm/lightdm.conf
[SeatDefaults]
xserver-command=X -s 0 -dpms
The Braces:
Step 1. Do as per Approach 1
Step 2. Run the .xinitrc as an autostart script
cd ~/.config
mkdir autostart
cd autostart
vi screensaver.desktop
press “i” to insert, paste the following lines, then press the esc button and then “:wq” to save:
[Desktop Entry]
Type=Application
Exec=/home/pi/.xinitrc
## Ruby on Rails on Ubuntu on VirtualBox
https://niltoid.com/blog/ruby-on-rails-on-ubuntu-on-virtualbox/
Date: 2012-12-21
Themes:
Tags: tutorial, Ruby on Rails, Ruby, Ubuntu, VirtualBox
Skills: development environments
I started exploring Ruby on Rails for a potential client project which never kicked off. These are the steps I took to set up RoR on Ubuntu running on VirtualBox.
1. Install VirtualBox 4.1.20
guide: https://www.virtualbox.org/wiki/Downloads
2. Install Ubuntu 12.04
guide: http://www.psychocats.net/ubuntu/virtualbox
For resizing, copy/paste, share folders…
a) “Install Guest Additions”, sometimes it doesn’t work and you have to go to /media/VBOXADDITIONS_… and run
sh VBoxLinuxAdditions.run
b) then enable clipboard in settings as Bidirectional
c) then restart the instance.
For it’s own IP address
a) go to settings->network and select Bridged Adapter
b) if already running instance, reset network by running
sudo /etc/init.d/networking restart
3. Install Ruby on Rails using rvm
guide: https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm
4. Run this to make sure rvm runs every time you open a new terminal
echo ‘[[ -s “$HOME/.rvm/scripts/rvm” ]] && source “$HOME/.rvm/scripts/rvm”‘ >> ~/.bashrc
…or else this needs to be run when new terminal is opened
source ~/.rvm/scripts/rvm
5. as suggested on http://rubyonrails.org/download, create a skeleton site:
rails new ~/code/ror/HelloRoR
cd ~/code/ror/HelloRoR
rails server
…however this gives the following error:
Could not find a JavaScript runtime.
6. To resolve: sudo apt-get install nodejs
7. open a browser and go to http://localhost:3000
8. ctrl-c to stop server (don’t just close term window or else process must be killed manually before restarting server)
9. gedit-gmate as code editor
http://kibyegon.wordpress.com/2011/11/15/using-gedit-as-a-ruby-ide/
http://blog.sudobits.com/2012/03/11/gedit-plugins-for-ruby-on-rails/
10. mysql
sudo apt-get install mysql-server mysql-client
sudo apt-get install libmysql-ruby libmysqlclient-dev
gem install mysql2
11. Download mysql workbench for ubuntu 12.04
(later try to use workbench on host windows 7 machine and connect to mysql server on ubuntu, but it’s more involved)
From terminal, run
mysql-workbench
12. Create a RoR project which connects to MySQL
mkdir HelloRoRMySQL
cd HelloRoRMySQL
rails new HelloRoRMySQL -d mysql
13. useful generate scaffolding commands:
rails generate scaffold User userName:string fbId:bigint
– since bigint is not accepted, go into db/migrate open the file and change
t.bigint :fbId to t.column :fbId, :bigint
rake db:migrate
rails destroy scaffold User
rails g migration add_bio_to_user bio:text
– this doesn’t modify views
14. to test
http://localhost:3000/users
http://localhost:3000/users.json
## Netduino Robot Drummer
https://niltoid.com/blog/netduino-robot-drummer/
Date: 2012-08-25
Themes: robots-hardware, sound-visuals
Tags: tutorial, Netduino, C#, .NET Micro Framework, solenoids
Skills: motor control, multithreading, signal processing
The Netduino is Microsoft’s answer to the Arduino. It runs using the .NET Micro Framework (.NETMF) which means you can program it using C# within Visual Studio and with whole bunch of useful .NET standard libraries at your disposal. It is a breeze to setup and get started.
The goal of this project is to construct a robotic drummer using a microcontroller as the robot brain and a few solenoids as actuators that make drum sounds. I had a Netduino lying around and it was the perfect choice for this project since multi-threading is supported natively. This helps to simplify the code base, since when I want to activate a solenoid, I can simply spawn a thread which turns on the solenoid, waits a moment and then turns off the solenoid, and I don’t have to worry about this delay effecting the timing of subsequent drum hits. I can even trigger a drum roll or complicated riff if I want. Accomplishing the same task on an Arduino without the help of multi-threading is not too difficult either, but would take some extra effort, so this was a good excuse to try out the Netduino and it’s capabilities.
I eventually was able to leverage the .NETMF Threading and Timer classes for even more. Here is a great reference on the basics of multithreading on .NETMF. I have each Beat in a BeatPattern running on a separate “Timer” thread. The KnockDetector also runs on a separate thread so the (relatively) heavy number crunching doesn’t interfere with the timing of the drums.
Playing a Beat Pattern
I wanted to use solenoids to produce drum sounds by banging on things. One of the most frustrating things about the 12V push solenoids i had was that they had no frame and return spring. To make them usable, I had to construct custom frames out of metal and find some really weak springs to spring them back to their starting position after they’ve struck a beat. I purchased a variety pack of 100′s of springs but none of them were weak enough! I finally found some that were sold individually at a local hardware store, but i had to “weaken” them further. The strength of a spring depends on the thickness of the wound metal and also the radius of the winding, so stretching them out to increase that radius works. Lacking a frame also made mounting your solenoids a challenge. It is much easier to just purchase solenoids that include a frame and return spring. You also end up with something less rusted and ugly (good thing they’ll be out of sight eventually)
The circuit for each solenoid resembled the following diagram, which comes courtesy of this Bildr page.
There are a few differences in my circuit. My power source comes out of the netduino 5V pin, so its weaker than the 12V used in this diagram. I also used a schottky diode instead of the zener diode pictured and a 1kΩ resistor instead of a 2.2kΩ resistor. If anyone has any insight on whether I’d be better off modifying my circuit, I would love to hear your suggestions, please do post a comment.
On the software side, I created a BeatPattern class which, among other things, consists of an array of Beats. Each beat has its own thread which I set up using the .NETMF Timer library. At the beginning of a measure, the main program calls BeatPattern.update() which tells each Beat in the pattern to get ready to play it’s corresponding solenoid at the right moment, depending on the tempo.
Here is an early version of the beat making portion of the project:
Knock Detection
Hardware-wise, knock detection was quite simple. A piezoelectric buzzer can easily be used as a sort of knock or vibration or movement sensor if it’s correctly hooked up to an input pin on the microcontroller. Below is a diagram for the arduino.cc tutorial on this topic.
The software side of knock detection was more tricky. I set up a new thread to just listen for values coming from the knock sensor every 10 ms. Since this is an analog sensor, it has to be hooked up to an analog pin on the Netduino and will give you a continuous value (as opposed to a discrete 1 or 0). This means you end up getting a string of numbers for each hit. For example a single knock may give you the following values:
0 - 0 - 0 - 55 - 127 - 358 - 82 - 0 - 0 - 0
It looks like you can just take the highest numbers between 0′s to get your knock’s timing. This is true in some cases, however, a couple of knocks in quick succession may give you something like:
0 - 0 - 55 - 127 - 358 - 82 - 20 - 101 - 530 - 222 - 71 - 0 - 0
So to get both distinct knocks, I keep track of whether the current reading is on a “rising edge” or a “falling edge”. If rising and current value is starting to fall, the last reading was the peak. As always, see the code which is linked below for more details.
Finally I made a custom class called Knock with members “interval” and “volume”. Volume is simply the peak number from above, and to calculate the interval I used a Stopwatch class (copy it into a code file and include it in your project). Now I can store all my peaks in an array of Knocks for interpretation later.
Here are all the parts of this project prototype. The 3 solenoids, the breadboard circuit, the microcontroller, and of course the box which will house everything and has the piezoelectric buzzer installed on top. I also added a button because I’m considering using it as a very simple user interface to tell the drums when to start and stop.
Tempo Extraction
Unless your knocks are in a steady “4 on the floor” pattern (think techno), determining the user’s tempo is a difficult task. Sometimes called “Beat Detection”, Tempo Extraction is a complicated process which is the subject of many scientific papers. It involves a lot of number crunching and can bring your microcontroller, which lacks a powerful processor, to a grinding halt, so to speak.
Drawing clues from research papers I found online, I wrote a very simplified version of a Tempo Extraction algorithm which performs well and yields a fairly reliable beat interval reading. It only works for the 4/4 meter which simplifies the task quite a bit. Keep in mind, this means there are 4 beats in a measure. A beat interval is the time between 2 consecutive beats. The beat interval value is computed in milliseconds and can easily be converted to a BPM (beats per minute) by dividing 60K by it. To convert back, again, divide 60K by your BPM value.
BPM = 60000 / BeatInterval
The rational behind the Tempo Extraction algorithm is that beats in a 4/4 pattern usually fall on a whole, half, quarter, or eighth note. So consecutive beats should generally have one of these intervals between them. I compare each beat to it’s previous beat, two beats ago, and three beats ago, then “normalize” the intervals by either dividing by 2 or multiplying by 2 since all the intervals should be multiples by this theory.
Then I perform the following steps on the array:
1. Sort the array of intervals
2. Find the median
3. Throw out the outliers
4. Take the average of what’s left
This gives me a pretty decent beat interval value. See the code linked below for the nitty gritty, though keep in mind its a work in progress. I would love to hear from anyone who has experience with any of the techniques described above and can suggest an improvements or alternative methods!
https://github.com/asaeed/RobotDrummer
On a side note, I noticed one shortcoming of .NETMF is that it lacks support for generics, which means a lot of nice built in functions – like Sort() on an ArrayList – are just missing. I used extension methods to make my own algorithms on an ArrayList of integers. But extension methods don’t work out of the box, there’s a little trick I learned thanks to this guy to get them working.
## Arduino + WiFly sends OSC
https://niltoid.com/blog/arduino-wifly-sends-osc/
Date: 2012-07-10
Themes: robots-hardware
Tags: tutorial, Arduino, WiFly, OSC, UDP, C++
Skills: protocol implementation, wireless networking
OSC is a widely used protocol for wireless communication between devices. Arduino is a tremendously popular platform to create physical computing devices. So naturally, one would think that engaging in OSC communication using Arduino’s would be straight-forward and well documented. Unfortunately, I found very little help on the internet on this topic. Most searches led to one of two existing methods to get OSC out of an Arduino:
1. Using Processing as a middleman – this requires sending data from the Arduino over serial to a computer running Processing, which then generates and transmits OSC. So to talk between two Arduino’s, an entire computer is required? No good.
2. Using the Ethernet Sheild and a library called ArdOSC – excellent stuff, but this library is written to work with the ethernet library only, and I’m using an RN-XV WiFly module. No wires.
Neither of these solutions worked for me so I figured this is a topic worth exploring further. Previously, I had been able to use the RN-XV to send raw data over UDP. OSC data is also transmitted over UDP, so it should just be a matter of correctly generating OSC on the Arduino.
Start Simple
To test this hypothesis, I took some simple OSC from OpenSoundControl.org
2f (/) 6f (o) 73 (s) 63 (c)
69 (i) 6c (l) 6c (l) 61 (a)
74 (t) 6f (o) 72 (r) 2f (/)
34 (4) 2f (/) 66 (f) 72 (r)
65 (e) 71 (q) 75 (u) 65 (e)
6e (n) 63 (c) 79 (y) 0 ()
2c (,) 66 (f) 0 () 0 ()
43 (C) dc (Ü) 0 () 0 ()
The byte array equivalent in C for Arduino looks like this:
char data[] = {0x2f, 0x6f, 0x73, 0x63, 0x69, 0x6c, 0x6c, 0x61, 0x74, 0x6f, 0x72, 0x2f, 0x34, 0x2f, 0x66, 0x72, 0x65, 0x71, 0x75, 0x65, 0x6e, 0x63, 0x79, 0x00, 0x2c, 0x66, 0x00, 0x00, 0x43, 0xdc, 0x00, 0x00};
Since this is kind of difficult to look at and work with, we can instead use char arrays. Char arrays can be more readable, but how do you convert a float to a char array? There’s a neat “trick” to do this in C using the union keyword. Here’s some working Arduino code for sending OSC messages with a single float argument. The first message sends 0.1, the next 0.2, and so on until 1.0 is reached and then it’s reset back to 0.1.
#include "RNXV.h"
RNXV rnxv = RNXV();
float f;
union u_tag {
byte b[4];
float fval;
} u;
void setup() {
f = 0.1;
Serial.begin(9600);
Serial1.begin(9600);
rnxv.setUart(&Serial1);
rnxv.join("SSID_HERE","PASSPHRASE_HERE");
rnxv.enableUdp("192.168.1.5", 3000, 4000);
rnxv.sendCommand("join", "AOK");
Serial.println(rnxv.ip());
}
void loop()
{
delay(1000);
if (f > 1) f = 0;
f = f + 0.1;
char data[32] = "/oscillator/4/frequency\0,f\0\0";
u.fval = f;
data[28] = u.b[3];
data[29] = u.b[2];
data[30] = u.b[1];
data[31] = u.b[0];
Serial.print("print use loop: ");
for (int i = 0; i <= 31; i++){
Serial.print(data[i]);
}
Serial.println();
rnxv.sendOsc(data, 32);
}
The code references an RN-XV library I’ve been working on to set up UDP and send the byte array as follows:
void RNXV::sendOsc(char data[], int size)
{
for(int i = 0; i < size; i++)
_uart->print(data[i]);
delay(75);
}
I tried various delays and a delay of 75 seemed just enough that there wouldn’t be any corrupted OSC messages even when running the code for hours. This delay and the minimal code above resulted in about 13 OSC messages transmitting per second. It is also necessary to loop and send the exact number of chars as a message contains, or else the receiving end doesn’t recognize the end of the message and throws errors.
On the receiving end, I once again used the sample app that comes with Cinder and it’s OSC Cinder block. See the “Listening with Cinder” section in this post to see how to run the sample app, no code is required. (Note that you may need to restart the app once because at first it inadvertently receives a message “exit” which is the RN-XV going into data mode). If everything is working you should see a window like this one where the white section slowly expands:
OscPkt and AVR-STL
At this point, having proved the concept, I was about to dive into writing my own library to generate the OSC, but thought I’d try one last time to find something off the shelf to do this.
OscPkt, the minimalittlest OSC library for C++ seemed perfect for the job, being super concise and all contained in a single header file. It also was not tightly coupled with any UDP library so I could use my sendOsc function (above). However it depended heavily on C++ Standard Library classes (like vector) which don’t exist for Arduino.
Except they do – a brilliant piece of work by Andy Brown is AVR-STL. It brings STL to Arduino. He has all the instructions on his blog post. Simply download and put all header files in the include folder into …arduino-1.0.1\hardware\tools\avr\avr\include.
Once AVR-STL was integrated, I took oscpkt.hh, renamed it to oscpkt.h and added the following two #includes at the top (as per the instructions found on the AVR-STL page):
#include
#include
Then placed it in a folder “oscpkt” which I threw into my Arduino “libraries” folder. Now you can add the following to any Arduino sketch:
#include "oscpkt.h"
setup() {
oscpkt::Message msg;
oscpkt::PacketWriter wr;
msg.init("/foo");
wr.init().addMessage(msg);
Serial.println(wr.packetSize());
Serial.println(wr.packetData());
}
Previous posts about this project include the following topics:
1. A low-cost and easy Robot Chassis to serve as the body of the robot.
2. Adding a RN-XV WiFly Module to enable wi-fi connectivity.
3. A Python Server which communicates with the robot.
4. Adding a Sensor and Actuator and transmitting data as JSON.
In #3 and #4, I utilized a Python Server with JSON as the data format. Here we try out a C++/Cinder Server with OSC as the communication protocol.
## Cinder + Box2D
https://niltoid.com/blog/cinder-box2d/
Date: 2012-06-24
Themes: graphics-3d
Tags: tutorial, Cinder, Box2D, C++
Skills: physics simulation, real-time graphics
Box2D is a popular 2D physics engine for C++. It’s free and open source and pretty easy to integrate with Cinder. I recently tried out another engine called MSAPhysics and wrote a tutorial on setting it up with Cinder. It’s a great little library, super easy to get up and running, and can even do a good job of 3D physics, however it lacks some advanced features and there’s very little community contributed code behind it. So I thought I’d try out an alternative and see how they compare. My friends at OneDayItWillMake and Wu-Media had spoken highly of Box2D and have used it in iOS games and with Cinder in Eclipse on OSX. In this post, I attempt to integrate it with Cinder in VisualStudio 2010 on Windows 7.
Using b2cinder
First, I tried out a sorta Cinder block in progress called b2cinder by David Wicks (aka sansumbrella). His sample project already takes care of some of the code needed to convert Box2D objects to Cinder objects and vice versa. No coding needed for this part…
1. Download this repo https://github.com/sansumbrella/sc-Box2D
2. Place sc-Box2D in cinder/blocks folder
3. Open BasicBox2D sample app in vc10 folder
4. Make sure these properties have correct paths:
C/C++ –> General –> Additional Include Directories
Linker –> General –> Additional Library Directories
5. Had to add “using namespace ci;” to the following code files:
Sandbox.cpp
BoxElement.cpp
BoundaryElement.cpp
6. In Solution right-click Header Files and Add Existing…
Add all files and sub-directories within sc-Box2D/include/b2Cinder and sc-Box2D/include/Box2D.
Now you should be able to run the app and this is what it looks like.
Integrating Box2D From Scratch
Next I wanted to try out the latest and greatest version of Box2D, since sansumbrella’s version was slightly older.
1. Create BasicApp in TinderBox called Box2DTest
2. Download Box2D from http://code.google.com/p/box2d/
3. Open vc10 .sln file and build Box2D project.
4. Copy Box2D.lib file created into new folder in Box2DTest called “lib”
5. In Linker –> General –> Additional Library Directories
add path to “lib” folder you just created
In C/C++ –> General –> Additional Include Directories
add path to Box2D_v2.2.1
Note: be sure to add these to both Debug and Release run configs
6. Add this to top of code file:
#include
7. Under “Header Files”, add all .h and .cpp files to Solution.
8. Create a Box2D World with gravity.
b2Vec2 gravity(0.0f, 10.0f);
b2World world(gravity);
9. Add the file Conversions.h from b2cinder lib which does pretty good job of converting between box2d and cinder floats, vec2f, and more.
10. Code
Here is the source code on GitHub for a quick example I created using these steps:
http://github.com/asaeed/Box2DTest
Another screenshot:
## Cinder + MSAPhysics
https://niltoid.com/blog/cinder-msaphysics/
Date: 2012-06-07
Themes: graphics-3d
Tags: tutorial, Cinder, MSAPhysics, C++
Skills: physics simulation, real-time graphics
MSAPhysics is a simple and light-weight physics engine for Cinder or OpenFrameworks created by Memo Aktens. I downloaded it from his git repository where he hosts a bunch of useful C++ libraries and decided to try it in conjunction with Cinder in VisualStudio 2010. Unfortunately the VS2010 sample apps that came with it, didn’t work right away. The MSAPhysics lib hasn’t been kept up to date in a couple of years. For this reason many prefer to go with Box2D for a better supported, fully featured, and robust physics engine. MSAPhysics is a great alternative if you want to do something really quick in a 2D or 3D world. Here are the simple steps for first getting MSAPhysics samples working and then to add the MSAPhysics library to a new project. This requires setting up Cinder as described here, as well as VisualStudio 2010.
Get msaPhysicsBasic Cinder Sample App Working
The sample app is very useful for learning how to use MSAPhysics and see it’s potential.
1. Open the vc10 solution msaPhysicsBasic
2. Fix the following properties. Adjust the paths to point to your cinder directories…
C/C++ –> General –> Additional Include Directories
Linker –> General –> Additional Library Directories (added extra ../ in my case)
3. Copy all the .h and .cpp files found in the following mslibs src folders
MSAPhysics
MSACore
MSAObjCPointer
Paste into the sample projects include folder
4. Now right-click Sln –> Header Files and add all the files you just copied. Looks like this now:
5. In MSACore.h, uncomment this line: #define MSA_HOST_CINDER
6. Make sure both of the following are included in your main app:
#include “MSAPhysics.h”
#include “MSAPhysics3D.h”
7. Make following replacements in the main app:
Physics::World3D physics;
Physics::Particle3D mouseNode;
//Physics::World physics;
//Physics::Particle mouseNode;
Also change all occurrences of Particle, Spring, and more to Particle3D, Spring3D, etc
9. Change anywhere you have something like
a->getX()
to
a->getPosition().x
10. Change loadResource() call to loadAsset() and put ball.png into a folder called “assets” in sample app folder
Get MSAPhysics into a new Cinder App
1. Create Basic App in TinderBox (which comes with the windows distribution of Cinder 0.84)
2. Copy all the .h and .cpp files found in the following mslibs src folders
MSAPhysics
MSACore
MSAObjCPointer
Paste into the sample projects include folder
3. Add the files you just copied under Sln -> Header Files (see screenshot above).
4. In MSACore.h, uncomment this line: #define MSA_HOST_CINDER
5. Add this line to the top of your main cpp file: #include “MSAPhysics2D.h”
Optionally add this line: using namespace MSA;
6. Code
A Quick Test App I Created
Here is some source code from my own experiments with MSAPhysics. I wanted to be able to abstract out a particle system that uses MSAPhysics. I define the Physics::World2D in the main app. Then, when initializing the ParticleController, I pass in a reference to World2D. Then when creating each Particle, the ParticleController also creates a Physics::Particle2D, and passes it’s reference to its corresponding Particle instance to store. This way a Particle can draw itself using data from the physics world, as well as it’s own member data such as a color.
Here is the end result
## Ableton Live Talks OSC to Cinder
https://niltoid.com/blog/ableton-live-talks-osc-to-cinder/
Date: 2012-05-15
Themes: sound-visuals
Tags: tutorial, Ableton Live, LiveOSC, OSC, Cinder, C++
Skills: real-time graphics, audio-reactive visuals
The goal here is to have Ableton Live transmit all track data and user actions to a C++ program which can use that data to generate unique visuals. If you want to create interactive visuals for a music performer, the obvious way is to use a live audio feed. However you can only do so much with such a feed and visuals end up only responding to the master level. With the help of a tool called LiveOSC, it’s possible to get OSC data out of Ableton Live. OSC or Open Sound Control is a data format which is usually transmitted over a wifi using a protocol called UDP, and this data can be picked up by a C++ application. In this case we’re using a C++ framework called Cinder and the OSC is interpreted into visualizations. This OSC data can tell us stuff like:
– when the set begins or ends
– when each beat occurs (as in once a second for 60 bpm)
– when any track level or the master level is changed
– when the user switches to the next track
– when the user tweaks any knob
I’m using the following tools to accomplish this task.
– Ableton Live 8.2.6 (OSX in my case)
– LiveOSC
– Cinder
– XCode or VisualStudio 2010 (or Eclipse which I’m using, but that’s a whole other topic)
Ableton Live gets LiveOSC
LiveOSC is a simple way to get OSC data out of Live. It’s hosted here by its creator:
There’s not much help on the site, but it really is that simple to get it working.
1. install python 2.5.1 (Do not skip!)
2. place LiveOSC in Live.App/Contents/App-Resources/MIDI Remote Scripts/
3. go to Preferences -> MIDI Sync and select LiveOSC as a Control Surface
Live should now be transmitting OSC data over UDP on port 9001, as well as listening on port 9000. However we don’t yet have a way of listening to the transmission. You could probably use some sort of network monitoring tool to check if it’s working. But it’s actually easier to proceed with setting up the next tool, Cinder, since it comes with a sample app called OSCListener which will pick up this OSC data and parse and display it.
[As an aside, by default LiveOSC seems to transmit to it’s own host machine’s IP, my next goal is to make it transmit to a different IP]
Listening with Cinder
Cinder is an excellent C++ framework that makes it a breeze to make complex interactive and visual apps. You can download it for free and it comes with many sample apps as well as a set of tutorials by a talented artist, Robert Hodgin. My preferred way of setting up Cinder is with Git, using these instructions.
Once you are setup correctly, you should be able to launch any of the sample apps that the Cinder library comes with. Go into cinder/blocks/osc/samples/OscListener/ and then launch either the Xcode or VS2010 app depending on your platform of choice.
Now in the source file OscListenerApp.cpp, make one simple change. Look for the following line and change the port number from 3000 to 9001:
listener.setup(9001);
Now run OscListenerApp and then go to Ableton Live and move the master level slider up and down. You should see some OSC data pop up in you Xcode console!
The master level slider will also effect the OscListener app screen. The white part of the window should extend to the right as you increase the volume. This slider will have this effect while others may not. In the code the simple visual effect is only performed when the first argument is a float.
Making sense of the OSC
I’m still figuring out the nitty gritty of interpreting the OSC messages from LiveOSC. Most of it is pretty self explanatory. Here’s an example. The output is reformatted so it looks more readable (to me)
–> /live/play #args:1 [0]int32: 2
–> /live/beat #args:1 [0]int32: 1
–> /live/beat #args:1 [0]int32: 2
–> /live/beat #args:1 [0]int32: 3
–> /live/beat #args:1 [0]int32: 4
–> /live/beat #args:1 [0]int32: 5
–> /live/beat #args:1 [0]int32: 6
–> /live/scene #args:1 [0]int32: 3
–> /live/clip/info #args:3 [0]int32: 0 [1]int32: 2 [2]int32: 3
–> /live/beat #args:1 [0]int32: 7
–> /live/clip/info #args:3 [0]int32: 0 [1]int32: 1 [2]int32: 1
–> /live/clip/info #args:3 [0]int32: 0 [1]int32: 2 [2]int32: 2
–> /live/beat #args:1 [0]int32: 8
–> /live/beat #args:1 [0]int32: 9
–> /live/beat #args:1 [0]int32: 10
–> /live/beat #args:1 [0]int32: 11
–> /live/play #args:1 [0]int32: 1
Things to note:
– we get a message on start and stop with the address /live/play
– we get a message on every beat with the address /live/beat
– we get a message when switching tracks with the address /live/clip/info
– the last argument in some cases is an int that represents the following values:
0 = empty
1 = stopped
2 = playing
3 = triggered
So the 3 /live/clip/info messages were sent when I triggered the new clip, the old clip stopped, and the new clip started.
TouchOSC talks to Cinder
This section is not really related to the above, but is another simple proof of concept. Another way to send OSC signals to a Cinder app is to use your phone. Just install an app called TouchOSC (free on Android, $5 and more full featured on iOS). Make sure your phone and computer are connected to the same wifi network, open up TouchOSC, and configure it with the following settings:
Host: 192.168.1.2 Port (outgoing): 9001
Port (incoming): 9000
Now just start up one of the touch layouts in TouchOSC, start up your OscListener app, and like magic, you should see messages coming in to your Cinder app.
## WiFi Bike Lock
https://niltoid.com/blog/wifi-bike-lock/
Date: 2012-05-07
Themes: robots-hardware
Tags: Arduino, WiFly, iOS, servo
Skills: prototyping, motor control
Took part in a Yahoo! and WPP sponsored hackathon this weekend and in 24 hours we whipped up this prototype for an electronic bike lock that can be controlled from an iOS app. The lock consists of an Arduino, an RN-XV WiFly, and a servo.
## Robot’s First Sensor and Actuator
https://niltoid.com/blog/robots-first-sensor-and-actuator/
Date: 2012-04-11
Themes: robots-hardware
Tags: tutorial, Arduino, robotics, servo, magnetometer
Skills: sensor integration, motor control
This is the fourth post in a pseudo-series about an autonomous wi-fi connected robot I’m working on. So far we’ve looked at:
1. A low-cost and easy Robot Chassis to serve as the body of the robot.
2. Adding a RN-XV WiFly Module to enable wi-fi connectivity.
3. A Python Server which communicates with the robot.
To reiterate, the idea is to have an Arduino robot which transmits sensor data over wi-fi to a Python server. The server then interprets the data and sends actuator commands back to the robot. We’ve achieved a round trip of data so far, next we try to refine what we have and add an actual sensor and actuator.
Robot Torso and Head
My impulse buy a few weeks ago has actually come in handy to build up the body of the robot… The Eitech Classic Construction Kit includes all the flat metal pieces with holes which you see in the images above and below.
As these pictures show, I used the Eitech parts to make a simple rectangular frame for the torso. The Eitech screws are unfortunately a tiny bit too wide to screw into the acrylic top of the Magician Chassis, but some slightly smaller 4-40 size screws did the job.
Neck Servo
Technically, the first actuators I added were the 2 wheel motors. However, I’m leaving them unplugged from their power source for now. Time to add a rotating head. On top of this frame, I mounted an upside-down servo to serve as a head. Luckily the face plate easily attached to the servo using a couple of Eitech screws and nuts.
Close-up of how servo is mounted
A Sense of Direction
Next I added an HMC5883L Magnetometer to serve as the robots first sensor. A magnetometer is just a simple digital compass. Its fairly straightforward to use and there’s a very good tutorial on Bildr which covers how to set it up with an Arduino. I plopped it onto the robot’s breadboard, hooked up the 4 wires required, tried the sample code from Bildr, and started getting data from it.
One thing to note however is that it’s not incredibly accurate. Especially if it’s placed physically next to a magnet (like a motor) or just a hunk of metal (like a breadboard)… Simply bringing my wire cutter near it would throw it off by 10-20 degrees or more. In addition, its lacking something called tilt compensation. Unfortunately compasses that include tilt compensation are about 10x the price. So you have to either (a) fork up the cash, (b) live with some error or (c) figure out ways to minimize the error in code or by re-positioning the module. I’ll try to achieve (c) and hopefully revisit this in a later post.
A Common Language
Now that I had at least 1 sensor (the compass) and 1 actuator (the neck servo), there was finally some real data to communicate between the Arduino Robot and the Python Robot Base. It was time to establish a common language for client and server.
In an earlier post, I set up communication between the 2 endpoints using UDP. By the end, I had a simple string of text being transmitted back and forth:
"here is a longer string of data being sent back and forth"
Not very useful until you actually include some data. In the past I’ve used simple comma delimited strings such as:
"12,34,56,78,90"
This works fine if both endpoints knows what all these numbers mean. For instance the arduino and the server both may know that the first number is the compass value (12), the second number is the proximity sensor value (34). Works but is not very readable for humans.
A better option is to use JSON, which is a concise, readable, standardized, and widely accepted data format. Now I’m transmitting the following JSON string from the Arduino:
{'ip':'192.168.1.6','heading':144}
First you may note that I’m sending the IP address of the robot. This is in case messages are being sent from multiple robots or devices, the Python server will know who to respond to with what commands. I created a funtion in my RNXV library called ip() to do this. You can take a closer look at that library to see how the IP is retrieved. Second, how do we actually create this JSON string? One option is to construct the string manually. A better way is to use a tremendously helpful library called aJSON. Check out the source code to see how it’s used.
On the Python side, once the string is received, we parse it and interpret the data. For the parsing, it just so happens that the JSON format very closely resembles the code to create a dictionary in Python. This is how you’d create a dictionary in Python:
data = { 'ip': '192.168.1.6', 'heading': 144 }
So we can simply take the JSON string and use the eval function to build a dictionary.
dataRaw, addr = sock.recvfrom( 1024 )
data = eval(dataRaw)
This is a temporary measure. It may break down when we have more complex JSON but is sufficient for now. Later I would probably try to use a JSON library for Python.
Now the Python server constructs a response JSON string to send back to the robot. For starters we aren’t going to try to make any meaningful interpretations. The input value (compass heading) ranges from 0-359 and the output value (neck servo position) ranges from 0-179. So a simple divide by 2 will give us some result to observe.
s = '{"neck":' + str(heading / 2) + "}"
Now as you turn the robot in place 360 degree’s, you can observe the head turn 180 degrees.
As an aside, I experienced something very strange when I unplugged the USB cable from the robot and attempted to power it from an external battery. The Arduino would crash almost instantly and stop sending messages to the server. After some research, I discovered that this could be because of message fragments left on the serial buffer which it was trying to process but failing at. I’m still not sure why it only occurs when using external power. I did manage to find a very simple fix. To make sure the JSON message was complete, I added a simple check to make sure the first character is a ‘{‘ and the last is a ‘}’.
Errors would occasionally also occur on the Python side because of badly formatted JSON. So I added the same check in the Python code and that reduced crashes substantially. Once in a while there is still a crash because somehow an odd byte gets lost in the transmission in either direction. Eventually, I will add some more JSON validation to avoid such crashes. At this point, however, I’m still able to keep the robot running for hours with no problems.
Here is the python RobotServer code:
import socket
import time
import math
UDP_IP="192.168.1.7"
UDP_PORT=3000
sock = socket.socket( socket.AF_INET, # Internet
socket.SOCK_DGRAM ) # UDP
sock.bind( (UDP_IP,UDP_PORT) )
while True:
dataRaw, addr = sock.recvfrom( 1024 )
#prints a string but crashes on OSC msg
#print("received: " + data.decode(“utf-8″))
if dataRaw[0] == 123 and dataRaw[-1] == 125: #check for '{' and '}'
data = eval(dataRaw)
ip = data["ip"]
heading = data["heading"]
s = '{"neck":' + str(heading / 2) + "}"
print(dataRaw + b" —-> ” + bytes(s,"utf-8"))
if len(dataRaw) > 0:
sock.sendto( bytes(s,"utf-8") + b"\n", 0, (ip, 4000) )
Here is the repo containing the WiFiRobot’s Arduino code.
## Arduino talks to Python
https://niltoid.com/blog/arduino-talks-to-python/
Date: 2012-03-11
Themes: robots-hardware
Tags: tutorial, Arduino, Python, WiFly, UDP
Skills: wireless networking, socket programming
I’m in the process of creating a “Robot Command Center” in my home. Basically my goal is to have Arduino devices sending sensor data to a central server and receiving commands from this server on what to do next.
My previous post described how to use a wi-fi chip called the RN-XV to get an arduino talk to a PHP web site. The problem with this HTTP over TCP based method was that all the connecting and disconnecting was slowing everything down. The best it could do was just one exchange of data per second. Luckily the RN-XV Wifly module has a UDP mode which seems better suited for fast communication. UDP (an alternative to TCP) forgoes the formalities of handshaking and just starts yelling data over a port.
The following 4 commands are needed in the Arduino sketch to configure UDP on your RN-XV. These commands enable UDP mode, set the ip of the host computer, and set the sending and receiving ports. You’ll find them used in my Arduino code later in this post.
set ip proto 1
set ip host 192.168.1.XXX
set ip remote 3000
set ip local 4000
Now the question of what to use for the central server. My first attempt was to use C++ and Cinder which I’ve been exploring lately. Cinder can easily communicate using OSC (Open Sound Control) data over a UDP connection. However I had some trouble generating the OSC messages on the Arduino. I’m shelving this Arduino + WiFly + OSC + Cinder project for another day.
My next attempt was to use Python, which has great built-in networking functionality. First, tried installing Python on Windows 7, but after some early hurdles and research it seemed clear that it would be easier to run Python on an Ubuntu instance using VirtualBox.
Psychocats has some great instructions for setting up VirtualBox + Ubuntu.
1. Download Ubuntu
2. Get VirtualBox + Ubuntu
3. Install Ubuntu
4. Select Bridged mode in VirtualBox settings
And there you have it, a clean new Linux build with it’s own IP Address.
Next I created a Shared Folder so I can edit the code on the host computer and have it copy right over to the Ubuntu instance.
1. Go to Devices->Install Guest Additions – this is required first
2. Go to Devices->Shared Folders – make a shared folder
3. Ctrl-Alt-T to open Terminal
4. Type sudo mount -t vboxsf SHARE_FOLDER ~/Share
(here substitute “SHARE_FOLDER” for what you created in step 2 and “~/Share” for whatever location you want that share to exist in Ubuntu)
Back on the host machine, I made a quick python script called test.py containing 1 line of code:
print "hi"
and placed it in the shared folder. Now you can run it by going back to Ubuntu Terminal, navigating to where test.py lives, and typing
python test.py
You should see “hi” show up as the output.
Now use the following code on the Arduino (once you have hooked up the RN-XV as I’ve described in this post)
#include "RNXV.h"
int loopCount;
RNXV rnxv = RNXV();
void setup() {
Serial.begin(9600);
Serial1.begin(9600);
Serial.println("setup begin");
rnxv.setUart(&Serial1);
Serial.println("uart set");
rnxv.connect("your_ssid","your_passphrase");
rnxv.sendCommand("set ip proto 1", "AOK");
rnxv.sendCommand("set ip host 192.168.1.7", "AOK");
rnxv.sendCommand("set ip remote 3000", "AOK");
rnxv.sendCommand("set ip local 4000", "AOK");
rnxv.sendCommand("save", "AOK");
rnxv.sendCommand("reboot", "READY");
rnxv.sendCommand("join", "AOK");
loopCount = 0;
Serial.println("setup end");
}
void loop()
{
sendData();
receiveData();
}
void sendData()
{
loopCount++;
Serial1.print(loopCount);
Serial1.print(“here is a string of data being sent back and forth”);
delay(200);
}
void receiveData()
{
while (Serial1.available() > 0) {
Serial.write(Serial1.read());
}
}
I whipped up a simple library to work with the RN-XV.
Here is the header file RNXV.h:
#ifndef RNXV_h
#define RNXV_h
#include "Arduino.h"
class RNXV
{
public:
RNXV();
void setUart(Stream* uart);
void print(char* str);
void println(char* str);
void enterCommandMode();
void exitCommandMode();
void sendCommand(char* cmd, char* response);
bool connect(char* ssid, char* phrase);
bool awaitResponse(const char* toMatch, unsigned int timeOut);
bool awaitResponse(const char* toMatch);
private:
Stream* _uart;
};
#endif
And here is the corresponding RNXV.cpp. Remember that both these files go in a folder called “RNXV” which is placed in your Arduino libraries directory.
#include "Arduino.h"
#include "RNXV.h"
RNXV::RNXV()
{
}
void RNXV::setUart(Stream* uart)
{
_uart = uart;
enterCommandMode();
println("set comm idle 0");
awaitResponse("AOK");
println("set ip proto 2");
awaitResponse("AOK");
}
void RNXV::print(char* str)
{
_uart->print(str);
delay(250);
}
void RNXV::println(char* str)
{
_uart->print(str);
_uart->print("\r");
delay(250);
}
void RNXV::enterCommandMode()
{
print("$$$");
awaitResponse("CMD");
}
void RNXV::exitCommandMode()
{
print("exit\r");
awaitResponse("EXIT");
}
void RNXV::sendCommand(char* cmd, char* response)
{
enterCommandMode();
print(cmd);
print("\r");
awaitResponse(response);
exitCommandMode();
}
bool RNXV::connect(char* ssid, char* phrase)
{
enterCommandMode();
print("set wlan phrase "); println(phrase);
awaitResponse("AOK");
print("set wlan ssid "); println(ssid);
awaitResponse("AOK");
println("save");
awaitResponse("AOK");
println("reboot");
awaitResponse("READY");
delay(2000);
println("join");
awaitResponse("Associated!");
exitCommandMode();
return true;
}
bool RNXV::awaitResponse(const char* toMatch){
awaitResponse(toMatch, 1000);
}
bool RNXV::awaitResponse(const char* toMatch, unsigned int timeOut = 1000) {
int byteRead;
unsigned long timeOutTarget;
for (unsigned int offset = 0; offset < strlen(toMatch); offset++) {
timeOutTarget = millis() + timeOut;
while (!_uart->available()) {
if (timeOut > 0) {
if (millis() > timeOutTarget) {
return false;
}
}
delay(1);
}
byteRead = _uart->read();
delay(10);
if (byteRead != toMatch[offset]) {
offset = 0;
if (byteRead != toMatch[offset]) {
offset = -1;
}
continue;
}
}
return true;
}
And use the following basic Python code on Ubuntu
import socket
import time
UDP_IP="192.168.1.7"
UDP_PORT=3000
sock = socket.socket( socket.AF_INET, # Internet
socket.SOCK_DGRAM ) # UDP
sock.bind( (UDP_IP,UDP_PORT) )
while True:
data, addr = sock.recvfrom( 1024 )
print "received message:", data
if len(data) > 0:
sock.sendto( data + "\n", 0, ('192.168.1.5', 4000) )
time.sleep( .2 )
As you may have guessed, in my case 192.168.1.7 is the IP of the Ubuntu instance (which listens on port 3000) and 192.168.1.5 is the IP of the Arduino’s WiFly module (which listens on port 4000). Replace these to suit your local network.
I’m sure my code could use numerous improvements but this is a good start to having your Arduino device talk to a Python server over your wi-fi network. Here is the output you should see on your Arduino serial terminal:
And here is the corresponding output that the Python program produces:
As an aside, try downloading an app called TouchOSC on your Android or iOS phone or tablet. Since OSC is transmitted over UDP as well, you can easily configure TouchOSC to point to your Ubuntu’s IP and listening port, and you get instant communication!
## RN-XV WiFly
https://niltoid.com/blog/rn-xv-wifly/
Date: 2012-02-23
Themes: robots-hardware
Tags: tutorial, Arduino, WiFly, Wi-Fi, robotics
Skills: wireless networking, serial communication
The RN-XV WiFly Module from Roving Networks is an affordable way to go wireless with your Arduino project. It’s $35 and you only need 1 to get started, assuming you have a wi-fi adapter in your computer as well. Previously, only the more expensive options such as the WiFly GSX ($85) were available. I’m not sure, but I assume the price difference is mainly due to signal strength differences. There are currently more resources and Arduino libraries available online for the WiFly GSX or WiFly Sheild, since they’ve been around longer. However, the RN-XV is gaining popularity fast and I hope to see future versions of these WiFly libraries work more reliably for the RN-XV.
If you don’t care about connecting to a wi-fi network, and instead want simple point to point wireless serial communication, another alternative is the hugely popular Xbee Module. However you do need a couple of them to set up 2 endpoints and get started, as well as a couple of breakout boards (an XBee Explorer Regulated and an XBee Explorer USB or Dongle). So the cost adds up quickly and you end up paying close to $80 total. XBee’s are totally worth the higher price point though. i found them to be much more easy to work with than WiFly Modules and there’s an endless amount of documentation and examples of their use on the web.
But back to the RN-XV. It was made to be easily swapped with an XBee so it has the exact same footprint. This means it’s pins have a 2mm pitch, so you need a breakout board to hook it up to a breadboard or regular PCB which have slightly wider pin holes. For this purpose, be sure to use a plain old XBee Breakout Board ($3). DO NOT use the XBee Explorer Regulated! I spent hours banging my head against a wall when I originally hooked it up to a Regulated I had lying around. The circuitry on the XBee Explorer Regulated prevents the RN-XV from properly working. They are NOT compatible.
I started out by attaching a bread board to my Robot Chassis. Since I didn’t want to permanently attach the breadboard using the adhesive tape, I instead stuck it to a piece of cardboard and then used little screws and nuts to attach the cardboard to the acrylic. Then plug the RN-XV into the XBee Breakout Board and the whole thing into the bread board. Then make the following 4 connections between the Arduino (Mega in my case) and the RN-XV.
RN-XV VCC (Pin 1) – Regulated 3.3V
RN-XV GND (Pin 10) – GND
RN-XV TX (Pin 2) – Arduino Serial1 RX
RN-XV RX (Pin 3) – Arduino Serial1 TX
Note: The RN-XV has 10 pins on each side but some Breakouts like the one I got on RobotShop, have 11. So be sure you figure which side has the 2 unused pins.
Here’s what it looks like so far. Ignore the little square board towards the bottom right, that’s a magnetometer which I plan to use later.
I found the right library for the job with a little help from Tedbot, who has a couple of blog posts on getting started with the RN-XV on his site. Here he recommends using this WiFly library by jcrouchley.
There is the start of a “polling client” sketch, where the arduino is repeatedly requesting new data from a server. The idea is to have the arduino send it’s sensor data to the server via HTTP requests and the response would contain commands that the robot will follow.
#include "WiFly.h"
#include "MotorDriver.h"
#define motorAPWM 11
#define motorAIN1 12
#define motorAIN2 13
#define motorBPWM 10
#define motorBIN1 9
#define motorBIN2 8
MotorDriver leftMotor = MotorDriver(motorAIN1, motorAIN2, motorAPWM);
MotorDriver rightMotor = MotorDriver(motorBIN1, motorBIN2, motorBPWM);
WiFlyClient client("192.168.1.101", 800);
void setup() {
Serial.begin(9600);
Serial1.begin(9600);
WiFly.setUart(&Serial1);
WiFly.begin();
Serial1.println("set comm idle 2"); // close an inactive conn in 2 seconds
WiFly.join("your_ssid", "your_passphrase");
}
void loop()
{
getData();
}
void getData()
{
if (client.available()) {
char c = client.read();
Serial.print(c);
}
if (!client.connected()) {
//delay(100);
sendRequest();
}
}
void sendRequest()
{
Serial.println();
Serial.println("connecting…");
if (client.connect()) {
Serial.println("connected");
delay(100); // seems to prevent an infrequent failed connection
client.println("GET /robotServer/server.php");
client.println();
}
}
The PHP code is dead simple at this point. I am using WAMP as a PHP server hosted on my laptop.
In your Arduino IDE terminal window you should now see the string “Robot Server” returned every 1 second or so. Once in a while a request fails. I’m not sure why it fails, but thanks to the line of code “set comm idle 2″, the failure doesn’t stay open indefinitely and instead closes in 2 seconds on inactivity. I also noticed putting a short delay here or there greatly reduces the number of failed requests.
Ad-hoc Mode
When I first tried to make an Arduino program to control the RN-XV, I tried various libraries I found online, none of which seemed to work properly. Obviously the next step was to RTFM, which in this case was pretty helpful and I was able to make an ad-hoc connection to the module. What this means is that the little RN-XV can turn itself into a wireless endpoint, which you can use your computer to connect to, just as you would to any wi-fi network. This part is completely optional (not that the rest is required), it just helps to better understand the capabilities of the module. Here’s how to use ad-hoc mode:
In addition to GND, 3.3V, TX, RX, also connect PIO9 (pin 8) to 3.3V
Recycle power, since PIO9 is high, it goes into ADHOC mode
Connect to the “WiFly” ssid from my laptop
Telnet to the IPAddress 169.254.1.1:2000 You should now see HELLO on your telnet screen
Type $$$ (with no return carriage). You shold now see CMD on your telnet screen
Perform the commands (with return carriage after each):
set wlan phrase YOUR_PASS_PHRASE
set wlan ssid YOUR_SSID
save
reboot
when i did this each letter showed up twice as i typed, but didn’t seem to matter
When it rebooted, i lost the connection, but since my CoolTerm window was open, i noticed the RN-XV automatically connected to specified SSID and got an ip. My laptop jumped back to my usual wifi network as well.
I telnet to that new IP
It should again say HELLO, Again type $$$ and again it says CMD.
Now type “ftp u” and the firmware update is performed using your wi-fi network’s internet access!
Now everytime it powers on, it jumps onto your wi-fi network, and it can turn your arduino into a webserver. You can try this by typing the IP and port number into a browser like so:
http://192.168.1.114:2000
## Processing + Eclipse + Android = Proclipsoid?
https://niltoid.com/blog/processing-eclipse-android-proclipsoid/
Date: 2012-02-06
Themes: graphics-3d
Tags: tutorial, Processing, Android, Java, Eclipse
Skills: Android development
Right off the bat I have doubts about using Processing for Android development. Processing only supports up to Android version 2.2 at this point. Android is up to 4+ now. This can’t be good. But I’m not sure what the repercussions are. Maybe simple apps will be forwards compatible? Features like NFC, which only the latest Android versions support, are going to be out of the question, right? I hope to delve into Android more to answer such questions, but for now I have a few simple steps to get a basic Android app, which leverages Processing, up and running in the Eclipse IDE.
First we need to perform steps as documented by Google to get Android up and running on its own.
1. Install Android SDK.
Run SDK Manager and select the components from the list as follows. My screenshot says “update” in some places because I last played with Android a year ago.
2. If you haven’t already, install Eclipse for Java.
3. Install the Eclipse ADT Plugin.
4. Create an AVD using Android version 2.2.
5. Create a blank Android Project again using Android version 2.2. On the 3rd screen for Package Name, be sure to use something like “com.example.helloandroid” and not “HelloAndroidProcessing” or you’ll get strange errors.
6. In Eclipse, create a new run configuration using the AVD you created earlier.
7. Now make sure the Android app runs on its own. Launch using your new run configuration and you should see the Android emulator pop up. If you’ve gotten this far, you are ready to develop for Android sans Processing.
8. If you haven’t already, install Processing (just need to download and unzip). I’m using version 2.04a. Stable version 1.5.1 won’t work according to the Processing Wiki.
9. Now import android-core from your Processing installation. I created a folder in my new Android project directory called “lib” and placed android-core.jar into it. Then in Eclipse go to Project -> Properties -> Java Build Path -> Add JARS… and find and select android-core.jar. (No Proclipsing happening here, though that would be my method of choice for developing Processing in Eclipse for non-Android purposes)
10. Now make the following code replacement in your main activity class. Notice I commented out most of what was there by default. I grabbed a simple Processing sketch from Dan Shiffman’s Learning Processing.
package com.example.helloandroid;
//import android.app.Activity;
//import android.os.Bundle;
import processing.core.*;
/*
public class HelloAndroidProcessing extends Activity {
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
}
}
*/
public class HelloAndroidProcessing extends PApplet {
public static void main(String args[]) {
PApplet.main(new String[] { “–present”, “something.whatever” });
}
//@Override
//public String sketchRenderer() {
// return P2D;
//}
PFont f;
String message = “this text is spinning”;
float theta;
@Override
public void setup() {
size(200,200);
f = createFont(“Arial”, 20, true);
}
@Override
public void draw() {
background(255);
fill(0);
textFont(f);
translate(width/2,height/2);
rotate(theta);
textAlign(CENTER) ;
text(message,0,0);
theta += 0.05;
}
}
Running your run configuration should start up the slow slow emulator. Once I unlock, I see the following:
Also you will now have a file called HelloAndroidProcessing.apk in your bin folder. Simply copy it over to your Android device and install it and you’ll see your app in action.
## Impulse Buy of the Week
https://niltoid.com/blog/impulse-buy-of-the-week/
Date: 2012-02-03
Themes:
Tags: Eitech, prototyping
Skills:
## SoundGround
https://niltoid.com/blog/soundground/
Date: 2012-01-29
Themes: sound-visuals
Tags: Processing, Java, Eclipse, audio, particles
Skills: audio-reactive visuals, physics simulation
Graphics programming is a new realm for me and I’ve been searching for the ideal tool sets to use for practice. Processing is a great way to get started. Created by Ben Fry and Casey Reas in 2001, it’s a work of art for creating works of art. It’s incredibly quick and easy to setup, and it’s very satisfying to see your creations moving around on the screen after a few lines of code. However, the Processing IDE leaves much to be desired, lacking any sort of debugging or intellisense. Luckily Eclipse is a great free IDE which makes using Processing an even more pleasant experience.
Processing + Eclipse = Proclipsing
Proclipsing makes using Eclipse for Processing very straightforward. Plus, it allows you to use some powerful Java constructs in your Processing app. It was recommended to me by my friend Mario (aka OneDayItWillMake) who has been my guide to proclipsing effectively.
SoundGround is a very basic app that responds to mic input. I used a simple particle engine and modeled some 2D physics. It’s easier than it sounds, I learned everything from Robert Hodgin’s cinder tutorials a couple years ago. They are an excellent resource for anyone doing any interactive programming in any language. Also made use of controlP5, a great library to create a parameter box in Processing.
Here’s the code.
## A Dev Blog
https://niltoid.com/blog/a-dev-blog/
Date: 2012-01-15
Themes:
Tags: Arduino, physical computing
Skills:
This blog is an attempt to try and document some of my learning experiences. I’m a web developer by trade and a physical computing hobbyist. I’ve been exploring the arduino platfrorm for a few years, and it’s incredibly fun to experiment with. Along the way, I’ve gotten help countless times from an amazing online community of DIY hackers. Hope this blog returns just a small part of the favor!
## Robot Chassis
https://niltoid.com/blog/robot-chassis/
Date: 2012-01-15
Themes: robots-hardware
Tags: tutorial, Arduino, robotics, DC motors
Skills: motor control, C++
The Magician Chassis from Sparkfun is a very good option for a beginner robot. It’s cheap but not cheapo. $15 is such a good price, i want to assemble an army of these (Not sure why it’s $28 everywhere else). Even includes a screwdriver.
It was dirt simple to assemble, took all of 15 minutes. The instructions are a little vague, but just remember that the wheel attaches to the side of the motor with the little protruding stub. You’ll see what I mean when you first try to orient the motors.
The next day I hooked up a couple of components to the top. A Sparkfun Arduino Pro 5V and a Polulu TB6612FNG Dual Motor Driver. After soldering some headers, the Pro easily attaches using some extra hardware that comes with the Magician. For the Motor Driver, I stuck a mini breadboard to a piece of cardboard and screwed that on with more Magician hardware.
When driving DC motors, the standard practice seems to be using separate power sources for the Motors and the Arduino (and everything else). On an earlier robot learned this the hard way when I started seeing very strange sensor readings when a motor kicked in. I used the 4 AA pack (which came with the Magician) for VMOT, and USB for the Arduino for now.
Hook up the Motor Driver as follows:
GND’s to a common ground (Arduino’s GND and GND from motor battery).
VCC to Arduino 5V
STBY to Arduino 5V (unless you want to control this in code)
VMOT to motor battery pack
AO1 and AO2 to one motor
BO1 and BO2 to other motor
the rest to your Arduino as the code suggests
Polulu TB6612FNG Dual Motor Driver
Here’s the code, which I found on the Arduino forum here (great stuff cause it’s concise), and then modified and made into a C++ class called MotorDriver. Check Github for the latest version of the code.
#include "MotorDriver.h"
#define out_A_PWM 11
#define out_A_IN1 12
#define out_A_IN2 13
#define out_B_PWM 10
#define out_B_IN1 9
#define out_B_IN2 8
MotorDriver leftMotor = MotorDriver(12, 13, 11);
MotorDriver rightMotor = MotorDriver(9, 8, 10);
void setup()
{
Serial.begin(9600);
Serial.println("begin");
}
void loop()
{
for (int i=-20; i<20; i++) {
leftMotor.setSpeed(i);
rightMotor.setSpeed(i);
delay(20);
}
leftMotor.setSpeed(0);
rightMotor.setSpeed(0);
delay(1000);
}
MotorDriver.h
#ifndef MotorDriver_h
#define MotorDriver_h
#include "Arduino.h"
class MotorDriver
{
public:
MotorDriver(int pinFwd,int pinRev,int pinPwm);
void setSpeed(int speed);
int getSpeed();
private:
int _pinFwd;
int _pinRev;
int _pinPwm;
int _speed;
int _pwm;
};
#endif
MotorControl.cpp
#include "Arduino.h"
#include "MotorDriver.h"
MotorDriver::MotorDriver(int pinFwd,int pinRev, int pinPwm)
{
pinMode(pinFwd, OUTPUT);
pinMode(pinRev, OUTPUT);
pinMode(pinPwm, OUTPUT);
_pinFwd = pinFwd;
_pinRev = pinRev;
_pinPwm = pinPwm;
}
void MotorDriver::setSpeed(int speed)
{
_speed = constrain(speed, -100, 100);
_pwm = map(abs(_speed), 0, 100, 50, 255);
if (_speed > 0)
{
digitalWrite(_pinFwd, HIGH );
digitalWrite(_pinRev, LOW);
}
else if (_speed == 0)
{
digitalWrite(_pinFwd, HIGH);
digitalWrite(_pinRev, HIGH);
}
else if (_speed < 0)
{
digitalWrite(_pinFwd, LOW);
digitalWrite(_pinRev, HIGH);
}
analogWrite (_pinPwm, _pwm);
}
int MotorDriver::getSpeed()
{
return _speed;
}
I’ve realized that the Arduino Pro is not going to have enough pins. Just this 1 component takes up almost half it’s digital pins. I could try a 8-bit shift register, but it will take up precious space and I’ll have to keep adding more. Instead, I decided to swap out the Pro for a Mega.
Another modification, I replaced the screws holding the battery pack with one of my favorite prototyping tools – velcro. This will make it easier to replace batteries.