Raspberry Pi + Arduino + Tornado

IMG_20130215_213350

Tornado is a great open source, Python based web framework.  It is designed to be light-weight, easily scalable, non-blocking and it supports websockets and some other nice features.  What’s more, it runs smooth and quick on the Raspberry Pi.

Thus far, my experiments with the Raspberry Pi have involved running Node.js on it to serve web pages and to interact over serial with an Arduino.  Here are some posts on how to do that:

1.  Raspberry Pi Setup
2.  Raspberry Pi + Arduino: Getting Started
3.  Raspberry Pi + Arduino + Node = Home Automation

I wanted to try accomplishing something similar using Python.  Specifically, a Python program which can simultaneously handle web requests and communicate with an Arduino.  I had always heard great things about Tornado and noticed some of the Raspberry Pi community had success with it.  

Tornado Setup

First things first, install Tornado on your Raspberry Pi:

1.  Good thing to do before every install – update agt-get and installed packages

sudo apt-get update
sudo apt-get upgrade

2.  Install easy_install and pip (which are Python package managers)

sudo apt-get install python-pip python2.7-dev

3.  Update your easy_install (and pip) package definitions

sudo easy_install -U distribute

4.  Now install Tornado using pip

sudo pip install tornado

5.  Follow the steps in this previous post to install Arduino on your Raspberry Pi

sudo apt-get install arduino
sudo usermod -a -G tty pi
sudo usermod -a -G dialout pi

6.  At this point might as well also install GPIO package as well, in case you need it later

sudo pip install RPi.GPIO

7.  Install another Python package to serial communication

sudo apt-get install python-serial python3-serial

To make sure everything works, create a file named “server.py” containing the following code (taken from Tornado’s site), and run it from terminal using the command “python server.py”

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import tornado.ioloop
import tornado.web
 
class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")
 
application = tornado.web.Application([
    (r"/", MainHandler),
])
 
if __name__ == "__main__":
    application.listen(8080)
    tornado.ioloop.IOLoop.instance().start()

Now on a separate computer on the same network, navigate your browser to http://raspberrypi:8080 (or use the IP address for example 192.168.1.7:8080).  You should see “Hello, world” if everything goes right.

Adding Websockets

Adding websockets to your Tornado app is easy. Below is the modified version of server.py. It also includes support for command line arguments on line 8 (so you can run the program using “python server.py –port=8080″).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.websocket
 
from tornado.options import define, options
define("port", default=8080, help="run on the given port", type=int)
 
class IndexHandler(tornado.web.RequestHandler):
    def get(self):
        self.render('index.html')
 
class WebSocketHandler(tornado.websocket.WebSocketHandler):
    def open(self):
        print 'new connection'
        self.write_message("connected")
 
    def on_message(self, message):
        print 'message received %s' % message
        self.write_message('message received %s' % message)
 
    def on_close(self):
        print 'connection closed'
 
if __name__ == "__main__":
    tornado.options.parse_command_line()
    app = tornado.web.Application(
        handlers=[
            (r"/", IndexHandler),
            (r"/ws", WebSocketHandler)
        ]
    )
    httpServer = tornado.httpserver.HTTPServer(app)
    httpServer.listen(options.port)
    print "Listening on port:", options.port
    tornado.ioloop.IOLoop.instance().start()

And here is the corresponding index.html, which connects to the websocket server. I used KineticJS to quickly create a simple red circle on a canvas which listens from click events.  Be sure that your “ws://” url is correct.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

<!DOCTYPE HTML>
<html>
  <head>
    <style>
      body { margin: 0px; padding: 0px; }
      canvas { border: 1px solid #9C9898; }
    </style>
    <script src="http://www.html5canvastutorials.com/libraries/kinetic-v4.2.0.js"></script>
    <script>
 
      var socket = new WebSocket("ws://raspberrypi:8080/ws");
 
      socket.onopen = function(){  
        console.log("connected"); 
      }; 
 
      socket.onmessage = function (message) {
        console.log("receiving: " + message.data);
      };
 
      socket.onclose = function(){
        console.log("disconnected"); 
      };
 
      sendMessage = function(message) {
        socket.send(message);
      };
 
      var value = 0;
 
      window.onload = function() {
        var stage = new Kinetic.Stage({
          container: "container",
          width: 610,
          height: 400
        });
 
        var layer  = new Kinetic.Layer();
 
        var circle = new Kinetic.Circle({
          x: 100,
          y: 100,
          radius: 10,
          fill: "red",
          stroke: "black",
          strokeWidth: 4,
          draggable: true
        });
 
        circle.on("mousedown", function() {
          value = value + 20;
          console.log('sending: ' + value);
          sendMessage("{value:" + value + "}");
        });
 
        layer.add(circle);
        stage.add(layer);
      };
 
    </script>
  </head>
  <body>
    <div id="container"></div>
  </body>
</html>

torduinoTermNow to test again start up your python script by executing “python server.py” in your terminal.  Next, direct your browser to http://raspberrypi:8080.  A couple of lines will show up in your terminal showing a connection is made.  In the browser you should see a small red circle and when you click it look for the following output in both your browsers console and your Raspberry Pi terminal (see screenshots).

Adding Multiprocessing

One key difference between Node and Python, however is that Node is by nature highly asynchronous and event-driven, while Python is generally not.  With Node, you can easily set up events to be fired for both incoming web requests from clients as well as incoming serial data from the Arduino.  This is demonstrated in the previous post.  In a Python + Tornado setup, your program is in a single continuous IO loop waiting for web requests, which is initiated by the last line of every Tornado app:

37

tornado.ioloop.IOLoop.instance().start()

Any code placed after it won’t execute.  You’re allowed to interrupt this IO loop (see PeriodicCallback) to perform other tasks, for example talk to an Arduino over serial, but web requests will be blocked while other processing is going on.  Another option is to use a component of Tornado called Gen which can help you set up the asynchronous execution of a separate loop.  However, what I found to be the cleanest solution was to use Python’s own native “multiprocessing” module to spawn a separate process to perform the needed serial communication.

tornado1-800x1200You’re probably thinking “That Node stuff was way easier. Isn’t multiprocessing a huge pain in the rear cause you have to deal with locks and concurrency and stuff like that?”  Well yes, but for what we’re trying to do it doesn’t get that complex, as you’ll see.  Also, Node’s asynchronicity is a double edged sword.  As your Node (JavaScript) code base grows, its easy to fall into the fiery pit known as callback hell. Seasoned JavaScript developers have ways to get around this, but I found it to be a nuisance.  Personally I find a larger Python program to be easier to trace through and maintain.  Ultimately, which language/programming style you go with will come down to your personal preference. Try out both and pick your poison.

There’s an excellent tutorial about Python multiprocessing by Doug Hellmann. It’s all I needed to get up and running with the module and I would urge anyone interested in the topic to check it out.

Inter-process communication is accomplished using the Queue data structure which comes with the multiprocessing module. Two queue’s are created, one for outgoing messages (taskQ) and one for incoming messages (resultQ). The queue’s are then passed to the spawned SerialProcess which sends anything in the taskQ to the Arduino over serial and puts any incoming data from the Arduino onto the resultQ. Tornado monitors the resultQ using PeriodicCallback.

A complete round trip looks like this:

1. Client sends a message to Tornado
2. Tornado sends it to SerialProcess using taskQ
3. SerialProcess sends it to the Arduino over serial
4. Arduino sends something back to SerialProcess over serial
5. SerialProcess sends it to Tornado using resultQ
6. Tornado sends it to all Clients

Below are all the code files (except index.html which remains the same as above). Here’s the final version of server.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78

#!/usr/bin/python27
 
import tornado.httpserver
import tornado.ioloop
import tornado.web
import tornado.websocket
import tornado.gen
from tornado.options import define, options
 
import time
import multiprocessing
import serialProcess
 
define("port", default=8080, help="run on the given port", type=int)
 
clients = []
 
class IndexHandler(tornado.web.RequestHandler):
    def get(self):
        self.render('index.html')
 
class WebSocketHandler(tornado.websocket.WebSocketHandler):
    def open(self):
        print 'new connection'
        clients.append(self)
        self.write_message("connected")
 
    def on_message(self, message):
        print 'tornado received from client: %s' % message
        self.write_message('got it!')
        q = self.application.settings.get('queue')
        q.put(message)
 
    def on_close(self):
        print 'connection closed'
        clients.remove(self)
 
################################ MAIN ################################
 
def main():
 
    taskQ = multiprocessing.Queue()
    resultQ = multiprocessing.Queue()
 
    sp = serialProcess.SerialProcess(taskQ, resultQ)
    sp.daemon = True
    sp.start()
 
    # wait a second before sending first task
    time.sleep(1)
    taskQ.put("first task")
 
    tornado.options.parse_command_line()
    app = tornado.web.Application(
        handlers=[
            (r"/", IndexHandler),
            (r"/ws", WebSocketHandler)
        ], queue=taskQ
    )
    httpServer = tornado.httpserver.HTTPServer(app)
    httpServer.listen(options.port)
    print "Listening on port:", options.port
    #tornado.ioloop.IOLoop.instance().start()
 
    def checkResults():
        if not resultQ.empty():
            result = resultQ.get()
            print "tornado received from arduino: " + result
            for c in clients:
                c.write_message(result)
 
    mainLoop = tornado.ioloop.IOLoop.instance()
    scheduler = tornado.ioloop.PeriodicCallback(checkResults, 10, io_loop = mainLoop)
    scheduler.start()
    mainLoop.start()
 
if __name__ == "__main__":
    main()

A new file called serialProcess.py is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

import serial
import time
import multiprocessing
 
class SerialProcess(multiprocessing.Process):
 
    def __init__(self, taskQ, resultQ):
        multiprocessing.Process.__init__(self)
        self.taskQ = taskQ
        self.resultQ = resultQ
        self.usbPort = '/dev/ttyACM0'
        self.sp = serial.Serial(self.usbPort, 115200, timeout=1)
 
    def close(self):
        self.sp.close()
 
    def sendData(self, data):
        print "sendData start..."
        self.sp.write(data)
        time.sleep(3)
        print "sendData done: " + data
 
    def run(self):
 
    	self.sp.flushInput()
 
        while True:
            # look for incoming tornado request
            if not self.taskQ.empty():
                task = self.taskQ.get()
 
                # send it to the arduino
                self.sp.write(task + "\n");
                print "arduino received from tornado: " + task
 
            # look for incoming serial data
            if (self.sp.inWaiting() > 0):
            	result = self.sp.readline().replace("\n", "")
 
                # send it back to tornado
            	self.resultQ.put(result)

Finally, here’s the Arduino program which simply listens for incoming data and returns it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

#define LED_PIN 13
 
int msgByte= -1;         // incoming byte
const int msgSize = 50;  // max message size
char msgArray[msgSize];  // array for incoming data
int msgPos = 0;          // current position
 
void setup() {
    Serial.begin(115200);
    pinMode(LED_PIN, OUTPUT);
}
 
void loop() {
    handleSerial();
}
 
void handleSerial() {  
  if (Serial.available() > 0) {
  	digitalWrite(LED_PIN, HIGH);
    msgByte = Serial.read();
 
    if (msgByte != '\n') {
      // add incoming byte to array
      msgArray[msgPos] = msgByte;
      msgPos++;
    } else {
      // reached end of line
      msgArray[msgPos] = 0;
 
      // here the message is processed
      // for now just send it back over serial
      Serial.println(msgArray);
 
      // reset byte array
      for (int c = 0; c < msgSize; c++) 
        msgArray[c] = ' ';
 
      msgPos = 0;
      digitalWrite(LED_PIN, LOW);
    }
  }
}

Test it out as shown above and you should now see the following output (click to enlarge):

torduinoTermFinal

One comment

  1. Richard Jeans

    Great post many thanks. Had to add an address argument to the listen statement to avoid a socket error:

    application.listen(8080,address=”0.0.0.0″)

    Reply

Post a comment

You may use the following HTML:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>