Playing with Asterisk PBX

I’ve been meaning to get back and have a proper play with Asterisk again for a while. Last week Amazon sent me one of those emails about things you’ve looked at but not bought and I spotted this:

It was down from £60 to £35 so I did exactly what they wanted and bought one.

Now normally I don’t use my land line at all, it’s just there to let the internets in, it doesn’t even have a handset plugged in. But there are a few little projects kicking around the back of my mind I’ve been thinking about for a while and the OBi110 should let me play with them.

The first is to see if the (unused, never given to anybody but my ISP to set up the conection) number for the land line has ended up on any lists for scamers/spammers and people generally trying to sell me stuff. My mobile gets at least 1 call a week about payment protection and the like and even my work office number has started getting recorded calls about getting my boiler replaced.

I could have probably just used the call log on the OBi110 but I wanted to be able to potentially record these calls and a few other things so I needed something a little smarter which is were Asterisk comes in. Asterisk is a opensource VoIP PBX this basically means it acts like a telephone exchange for calls made over the internet. I’ve seen people run Asterisk on the old Linksys Slugs so I was sure it should run fine on a Raspberry Pi as long as it wasn’t dealing with too many calls and not doing much codex transcoding. As I already had a Pi running my SMS/MMS rig it seamed like a good place to put all my telephone stuff.

Installing Asterisk on the Pi was just a case of running apt-get install asterisk. It comes with a bunch of default config files (in /etc/asterisk), but there are 2 main ones that I needed to change to make some simple things work.

sip.conf
This file is where you can configure what clients can connect to your asterisk instance via the SIP protocol. To start with I’m going to set up 2 different clients, one for a softphone running on my laptop and one for the OBi110. It sets up few things, but the import bit for later is the context which controls which bit of the extentions.conf file we jump to when receiving a call from each client.

[general]
context=local
srvlookup=yes

[softphone]
defaultuser=softphone
type=friend
secret=password123
qualify=no
nat=no
host=dynamic
canreinvite=no
context=local
disallow=all ; only the sensible codecs
allow=ulaw
allow=alaw
allow=gsm

[obihai]
defaultuser=obihai
type=friend
secret=password123
qualify=yes
dtmfmode=rfc2833
canreinvite=no
context=external
disallow=all
allow=ulaw

extensions.conf
This file defines how Asterisk should handle calls, it has two contexts called local and external. The local context defines 2 paths, the first for extension 100, when this number is called from the softphone Asterisk calls out to a small python program called agi-mqtt which publishes a JSON object to the calls/local MQTT topic which contains all the information Asterisk has about the call. It then answers the call then plays audio file containing HelloWorld and finally hangs the call up. I’m mainly using this local context to testing things out before copying them over to the external context.

The second path through the local context uses a special case extension number “_0Z.”, this matches any number that starts with 0[1-9] (so won’t match against 100). This path forwards the dialed number on to the OBi110 to place the call via the PSTN line.

The external context only contains 1 path which matches the phone number of the PSTN line and currently matches the 100 extension (play HelloWorld). At some point later I’ll setup this path to forward calls to a local softphone or forward to a voicemail account.

[local]
exten => _0Z.,1,AGI(/opt/asterisk/agi-mqtt/mqtt,/opt/asterisk/agi-mqtt/mqtt.cfg,calls/local)
exten => _0Z.,2,Dial(SIP/${EXTEN}@obihai);
exten => _0Z.,3,Congestion()
exten => _0Z.,103,Congestion()
exten => t,1,Hangup()

exten => 100,1,AGI(/opt/asterisk/agi-mqtt/mqtt,/opt/asterisk/agi-mqtt/mqtt.cfg,calls/local)
exten => 100,2,Answer()
exten => 100,3,Playback(en_US/hello-world)
exten => 100,4,Hangup()

[inbound]

exten => 0123456789,1,AGI(/opt/asterisk/agi-mqtt/mqtt,/opt/asterisk/agi-mqtt/mqtt.cfg,calls/pstn-in)
exten => 0123456789,2,Answer()
exten => 0123456789,3,Playback(en_US/hello-world)
exten => 0123456789,4,Hangup()

Now Asterisk is all working properly I setup the OBi110 using the instructions found here.

After a bit of playing I have inbound and outbound calls working and some MQTT enabled logging. Next up is looking at using the SIP Client built into Android to allow calls to be made and received from my mobile phone.

Unpacking binary data from MQTT in Javascript

While doing trawl of Stackoverflow for questions I might be able to help out with I came across this interesting looking question:

Receive binary with paho mqttws31.js

The question was how to unpack binary MQTT payloads into double precision floating point numbers in javascript when using the Paho MQTT over WebSockets client.

Normally I would just send floating point numbers as strings and parse them on the receiving end, but sending them as raw binary means much smaller messages, so I thought I’d see if I could help to find a solution.

A little bit of Googling turned up this link to the Javascript typed arrays which looked like it probably be in the right direction. At that point I got called away to look at something else so I stuck a quick answer in with a link and the following code snippet.

function onMessageArrived(message) {
  var payload = message.payloadByte()
  var doubleView = new Float64Array(payload);
  var number = doubleView[0];
  console.log(number);
}

Towards the end of the day I managed to have a look back and there was a comment from the original poster saying that the sample didn’t work. At that point I decided to write a simple little testcase.

First up quick little Java app to generate the messages.

import java.nio.ByteBuffer;
import org.eclipse.paho.client.mqttv3.MqttClient;
import org.eclipse.paho.client.mqttv3.MqttException;
import org.eclipse.paho.client.mqttv3.MqttMessage;

public class MessageSource {

  public static void main(String[] args) {
    try {
      MqttClient client = new MqttClient("tcp://localhost:1883", "doubleSource");
      client.connect();

      MqttMessage message = new MqttMessage();
      ByteBuffer buffer = ByteBuffer.allocate(8);
      buffer.putDouble(Math.PI);
      System.err.println(buffer.position() + "/" + buffer.limit());
      message.setPayload(buffer.array());
      client.publish("doubles", message);
      try {
        Thread.sleep(1000);
      } catch (InterruptedException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
      }
      client.disconnect();
    } catch (MqttException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
  }
}

It turns out that using typed arrays is a little more complicated and requires a bit of work to populate the data structures properly. First you need to create an ArrayBuffer of the right size, then wrap it in a Uint8Array in order to populate it, before changing to the Float64Array. After a little bit of playing around I got to this:

function onMessageArrived(message) {
  var payload = message.payloadBytes
  var length = payload.length;
  var buffer = new ArrayBuffer(length);
  uint = new Uint8Array(buffer);
  for (var i=0; i<length; i++) {
	  uint[i] = payload[i];
  }
  var doubleView = new Float64Array(uint.buffer);
  var number = doubleView[0];
  console.log("onMessageArrived:"+number);
};

But this was returning 3.207375630676366e-192 instead of Pi. A little more head scratching and the idea of checking the byte order kicked in:

function onMessageArrived(message) {
  var payload = message.payloadBytes
  var length = payload.length;
  var buffer = new ArrayBuffer(length);
  uint = new Uint8Array(buffer);
  for (var i=0; i<length; i++) {
	  uint[(length-1)-i] = payload[i];
  }
  var doubleView = new Float64Array(uint.buffer);
  var number = doubleView[0];
  console.log("onMessageArrived:"+number);
};

This now gave an answer of 3.141592653589793 which looked a lot better. I still think there may be a cleaner way to do with using a DataView object, but that’s enough for a Friday night.

EDIT:

Got up this morning having slept on it and came up with this:

function onMessageArrived(message) {
  var payload = message.payloadBytes
  var length = payload.length;
  var buffer = new ArrayBuffer(length);
  uint = new Uint8Array(buffer);
  for (var i=0; i<length; i++) {
	  uint[i] = payload[i];
  }
  var dataView = new DataView(uint.buffer);
  for (var i=0; i<length/8; i++) {
      console.log(dataView.getFloat64((i*8), false));
  }
};

This better fits the original question in that it will decode an arbitrary length array of doubles and since we know that Java is big endian, we can set the little endian flag to false to get the right conversion without having to re-order the array as we copy it into the buffer (which I’m pretty sure wouldn’t have worked for more than one value).

Network Attached Light Sensor

It was one of the projects that started out with an innocent enough question….

What do you know about light sensors?

The answer was not much, but I can find out, if you can give me some context (which is a pretty standard answer from ETS if it’s something new to us). The person asking was interested in measuring the natural light levels at a reasonable number of out door locations in order to help in making a call on if a given activity was safe to continue.

The client had already been looking at an existing solution which was using USB enabled light meters being routed over ethernet back to a central location. This felt like a really clunky solution so I was asked to have a look to see if I could come up with something a little different.

My first thought was to look at using something with a LDR but that would only really give relative light levels or require a lot of work to calibrate the system. It was time to have a bit of a poke around.

After a bit of searching I found reference to a TSL2561 ic which is a light level sensor that outputs via I2C. I found 2 boards with these sensors mounted, the first from Adafruit but I couldn’t find a UK supplier to get one to test with. A bit more digging and I found a very similar Sparkfun board that was available from Cool Components in the UK. The sensor has a range of 0.1 to 40k lux

The client has a PoE enabled ethernet network covering the site they wish to measure which means I should be able to thuse the a PoE enabled Ethernet Arduino to read from the sensor and report the values back to a central location. I used the sample library from Sparkfun as a starting point and then extended it with the MQTT client library from Nick O’leary.

I threw a prototype together with a breadboard and installed it on the window sill in my office to see how it performed. On the whole it seams pretty good, but I need to find a off the shelf meter that measures in lux to see how the values compare.

I now had a sensor that is publishing it’s light levels about once a second, this solved the initial problem but I needed to visualise the results. I fired up Node-Red to take the values and do a few things with them.

  • Stash the timestamp and light level in a mongo db store
  • Publish alerts when the value dropped below a given threshold
  • Use the light level to change the brightness of my Digispark RGB LED

I also ran up a visualisation using the Rickshaw Javascript library which allows you to draw time series charts using the D3 visualisation library.

The chart on the left show the sun coming over the building opposite my east facing office windows then cutting off sharply as it tracks round to the west.

The next challenge is to come up with an enclosure for the whole thing so it can survive outside in the British summer.

d3 MQTT Tree visualiser updated

I’ve been having a bit of a play updating my d3 based MQTT topic tree visualiser this weekend.

  1. I’ve been trying to tidy things up a bit and break the code out into a little standalone library and only leave the MQTT connection code in the index.html.
  2. I’ve been improving the handling of long payload. There is now a nicer popup tooltip with a longer version of the payload, it’s still cropped at 100 chars but I’m still working on wrapping it and showing a few lines to get more in.
  3. I’ve been moving the MQTT connection code over to use binary WebSocket connections* rather than the JSON based proxy I was using before. The version hosted here is still using the proxy for now, but the binary versions work and I’ll move it over once I’ve finished playing with my internal broker setup.

I still need to try and make the whole thing resize or at least not have hard coded dimensions, but that might have to wait for next time.

I’ve stuck the updated version up here, I’ll stick the code up on github once I get sign off from the boss.

https://github.com/hardillb/d3-MQTT-Topic-Tree

* There is a new IBM WebSphere MQ feature pack supporting this, details can be found here

developerWorks Days Zurich 2012

This week I had a day out of the office to go to Zurich to talk at this years IBM developerWorks Days. I had 2 sessions back to back in the mobile stream, the first an introduction to Android Development and the second on MQTT.

The slots were only 35mins long (well 45mins, but we had to leave 5 mins on each end to let people move round) so there was a limit to how much detail I could go into. With this in mind I decided the best way to give people a introduction to Android Development in that amount of time was to quickly walk through writing reasonably simple application. The application had to be at least somewhat practical, but also very simple so after a little bit of thinking about I settled on an app to download the latest image from the web comic XKCD. There are a number apps on Google Play that already do this (and a lot better) but it does show a little Activity GUI design. I got through about 95% of the app live on stage and only had to copy & paste the details for the onPostExecute method to clear the progress dialog and update the image in the last minute to get it to the point I could run it in the emulator.

Here are the slides for this session

And here is the Eclipse project for the Application I created live on stage:
http://www.hardill.me.uk/XKCD-demo-android-app.zip

The MQTT pitch was a little easier to set up, there is loads of great content on MQTT.org to use as a source and of course I remembered to include the section on the MQTT enabled mouse traps and twittering ferries from Andy Stanford-Clark.

Here are the slides for the MQTT session:

For the Demo I used the Javascript d3 topic tree viewer I blogged about last week and my Raspberry Pi running a Mosquitto broker and a little script to publish the core temperature, load and uptime values. The broker was also bridged to my home broker to show the feed from my weather centre and some other sensors.

d3 MQTT topic tree visualiser

As part of writing my talks for the developerWorks Days Zurich conference next week I’ve been working on some code for the demo in the MQTT session. I was looking for a nice way to display a MQTT topic structure and the most recent value posted to topic.

In the past I have used a plugin to the freemind mindmap tool that plots the topics as and their values. This worked well but it does require a machine with Java and a very old version of freemind to make it work so it was time to see if there was another approach.

Nick O’Leary had created a neat little python MQTT to Websockets* bridge that we have been using round the office for a while and there are some really great javascript visualisation libraries out there now, so I thought I would see how hard it would be to run up a browser based version.

Starting with the d3 Collapsible Tree Layout example seamed like a good idea and it didn’t take too long to tweak it a little to make it dynamically add new topic branches when a message was received.

function addNode(topic, body) {
	var parts = topic.split("/");
	if (root.children[0]===undefined){
		var newnode = {"name": parts.shift(), "children":[]};
		root.children = [newnode];
		walk(parts,newnode,body);
	} else {
		walk(parts,root,body);
	}
	update(root);
}

function walk(parts, node, body) {
	if (parts.length != 0) {
		var current = parts.shift()
		if (node.children && node.children.length != 0) {
			var z=0;
			for(z=0; z < node.children.length; z++) {
				if (node.children[z].name == current) {
					walk(parts,node.children[z], body);
					break;
				}
			}
			if (z == node.children.length) {
				var newnode = {"name": current, "children":[]};
				node.children.push(newnode);
				walk(parts,node.children[z],body);
			}
		} else {
			var newnode = {"name": current, "children":[]};
			node.children = [newnode];
			walk(parts,node.children[0],body);
		}
	}
}

I also swapped out the where the data was loaded to use a static root rather than loading it from a json file on the server.

var root = {"name": "/","children": []};
root.x0 = h / 2;
root.y0 = 0;
update(root);

This worked pretty well so next I set about looking to see if I could add the payload of the message. To do this I added a data field to the json object for each node when populating the tree.

function walk(parts, node, body) {
	if (parts.length != 0) {
		var current = parts.shift()
		if (node.children && node.children.length != 0) {
			var z=0;
			for(z=0; z < node.children.length; z++) {
				if (node.children[z].name == current) {
					walk(parts,node.children[z], body);
					break;
				}
			}
			if (z == node.children.length) {
				var newnode = {"name": current, "children":[]};
				node.children.push(newnode);
				walk(parts,node.children[z],body);
			}
		} else {
			var newnode = {"name": current, "children":[]};
			node.children = [newnode];
			walk(parts,node.children[0],body);
		}
	} else {
		console.log("body");
		node.data = body;
	}
}

Then updated where the SVG is inserted to add a text element in the right place.

...
  nodeEnter.append("svg:text")
      .attr("x", function(d) { return d.children || d._children ? -10 : 10; })
      .attr("dy", ".35em")
      .attr("text-anchor", function(d) { return d.children || d._children ? "end" : "start"; })
      .text(function(d) { return d.name; })
      .style("fill-opacity", 1e-6);
  
  // Add message payload
  nodeEnter.append("svg:text")
  	.attr("class","data")
	.attr("x", 10)
	.attr("y", 10)
	.attr("dy", ".35em")
	.text(function(d) { return d.data || ""; })
	.style("fill-opacity", 1);
  
  // Transition nodes to their new position.
  var nodeUpdate = node.transition()
      .duration(duration)
      .attr("transform", function(d) { return "translate(" + d.y + "," + d.x + ")"; });
...

Again this worked very well, except that it would not update the payload value after adding the first version. I tried a number of things to get this to work, but didn’t get very far, until James Thomas pointed me at this page about how some of the internals of d3 work. It talks about entry, update and exit methods to work on new, existing and removed elements in the data. There are examples of entry and exit functions in the example so I tried adding an update method but kept getting errors that it wasn’t found. I went back and had another look at the examples in the page James had linked to and noticed that update code doesn’t actually use a functions.

...
// Add message payload
  nodeEnter.append("svg:text")
  	.attr("class","data")
	.attr("x", 10)
	.attr("y", 10)
	.attr("dy", ".35em")
	.text(function(d) { return d.data || ""; })
	.style("fill-opacity", 1);

  // Update payload if changed  
  var nodeChange = node.selectAll("text.data")
      .text(function(d) {return d.data || "";});

  // Transition nodes to their new position.
  var nodeUpdate = node.transition()
      .duration(duration)
      .attr("transform", function(d) { return "translate(" + d.y + "," + d.x + ")"; });
...

An that was basically it. I’ve added a bit more code to update values when the tree branches are collapsed and to trim payloads to no more than 20 chars so the display is not too cluttered. I’m planning on improving the style sheet a little soon and possibly adding an effect so you see which payloads have been updated recently and maybe add hover text with the full payload for the shortened ones.

You can see the whole thing running here

* There is also some work being done to support Websockets with the Mosquitto MQTT broker, you can see the testing page here

Even More MQTT enabled TVs

Kevin Modelling the headset

A project at work recently came up to do with using one of the Emotiv headsets to help out a former Italian IBMer who was suffering from locked in syndrome. The project was being led by Kevin Brown who was looking for ways to use the headset to drive things sending email and browsing the web but he was also looking for a way to interact with some other tech around the house. The TV was the first on the list.

Continuing on from my previous work with controlling TVs and video walls with MQTT I said I would have a crack at this. My earlier solution was limited to LG TVs with serial ports and this needed to work with any make so a different approach was needed. It also needed to run on Windows so it was also a chance to play with C# and .Net.

To be TV agnostic it was decided to use a USB IR remote from a company called RedRat. They make a number of solutions, but their RedRat III was perfect for what was needed.

RedRat IR transmitter & receiver

The RedRat API comes with bindings for C++, .Net on Windows (and a LIRC plugin for Linux). The RedRat III is not just a IR transmitter, it is also a receiver which means it can “learn” from existing remote controls so it can be used with any TV.

Kevin is using Emotiv headset to drive Dasher as the input device. Dasher is a sort of keyboard replacement that allows the user to build up words using at a minimum a single input e.g. a single push button. As well as building words other actions can be added to the selector, Kevin added actions for browsing and also to control the TV. These actions publish MQTT messages to a topic with payloads like “volUp”, “chanDown” or “power”.

So now we had the inputs it was time to get round to writing the code to turn those messages into IR signals. There are 2 MQTT .NET libraries listed on the MQTT.org software page. I grabbed the first off the list MqttDotNet and got things working pretty quickly.

The following few lines sets up a connection and subscribes to a topic.

String connectionString = "tcp://" + 
  Properties.Settings.Default.host.Trim() + ":" +
  Properties.Settings.Default.port;
IMqtt client = MqttClientFactory.CreateClient(connectionString, "mqtt2ir");
try
{
	client.Connect(true);
	client.PublishArrived += new PublishArrivedDelegate(onMessage);
	client.ConnectionLost += new ConnectionDelegate(connectionLost);
	client.Subscribe(Properties.Settings.Default.topic.Trim(), 
	  QoS.BestEfforts);
}

Where the onMessage callback looks like this:

bool onMessage(object sender, PublishArrivedArgs msg)
{
	String command = msg.Payload.ToString().Trim();
	IRPacket packet = loadSignal(command);
	if (packet != null) 
	{
		redrat.OutputModulatedSignal(packet);
	}
	return true;
}

MQTT2IR Settings Window

And that is pretty much the meat of the whole application, the rest was just some code to initialise the RedRat and to turn it into a System Tray application with a window for entering the broker details and training the commands.

Unfortunately just a few days before we were due to have delivered this project we learned that the intended recipient had picked up a respiratory infection and had passed away. I would like to extend my thoughts to their family and I hope we can find somebody who may find this work useful in the future.

MQTT Inside Sticker Engine

For a bit of fun round a project last year I created some Moo “MQTT Inside” stickers to put on the cases.

First sticker deployed

These became very popular around the office, with everybody wanting to label up their MQTT enabled projects and the first batch of 90 quickly disappeared.

Given demand was so high Andy Piper suggested it might be worth making it possible for others to order their own sets and pointed me at the Moo API.

The Moo API Doc is pretty good, but there aren’t any examples so it took a bit of trial and error. The API works by building up an object called a ‘pack‘. The Moo website uses the same API for the wizard you use to manually create any of their products and there is a very nice little Greasemonkey script called ‘Show API JSON’ which shows you the state of the pack object at each step. The API also lets you skip any steps in the process by just uploading a pack object already filled in. This meant I could reproduce the stickers by choosing to edit my original set and then using the script to save the pack.json file.

{
    "productVersion" : "1",
    "cards" : [
    ],
    "numCards" : 90,
    "extras" : [
        {
            "value" : "blue",
            "key" : "pack_colour"
        }
    ],
    "imageBasket" : {
        "name" : null,
        "immutable" : false,
        "items" : [
            {
                "cacheId" : "partner_interface_uploader:e34bb9c2-637b-558545db-4e61502d-ff34.png",
                "imageBox" : null,
                "imageItems" : [
                    {
                        "rotation" : 0,
                        "height" : 297,
                        "resourceUri" : "http://www.moo.com/is/o/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                        "width" : 297,
                        "type" : "print"
                    },
                    {
                        "rotation" : 0,
                        "height" : 1024,
                        "resourceUri" : "http://www.moo.com/is/r/1024/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                        "width" : 1024,
                        "type" : "preview"
                    },
                    {
                        "rotation" : 0,
                        "height" : 75,
                        "resourceUri" : "http://www.moo.com/is/t/75/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                        "width" : 75,
                        "type" : "thumbnail"
                    }
                ],
                "copyrightOwner" : null,
                "resourceUri" : "filestore://image_original/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                "croppable" : true,
                "shouldEnhance" : false,
                "type" : "front",
                "removable" : true
            }
        ],
        "type" : null
    },
    "productCode" : "sticker",
    "sides" : [
        {
            "type" : "image",
            "data" : [
                {
                    "imageStoreFileId" : "e34bb9c2-637b-558545db-4e61502d-ff34",
                    "resourceUri" : "filestore://image_original/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                    "linkId" : "variable_image_front",
                    "enhance" : false,
                    "imageBox" : {
                        "center" : {
                            "y" : 12,
                            "x" : 12
                        },
                        "height" : 20.88,
                        "width" : 20.88,
                        "angle" : 0
                    },
                    "type" : "imageData"
                }
            ],
            "templateCode" : "sticker_image",
            "sideNum" : 1
        }
    ]
}

Resubmitting this with the ‘moo.pack.createPack‘ method and then using the ‘finish’ Drop in point to direct the user to Moo checkout page to pay for their new stickers.

I used a little library called scrib to do the oAuth and the whole thing can be done with just these few lines:

protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
	String authTokenString = request.getParameter("oauth_token");
	String authVerifyString = request.getParameter("oauth_verifier");
	
	Token authToken = MemoryStore.data.get(authTokenString);
	MemoryStore.data.remove(authTokenString);
	
	Verifier verifier = new Verifier(authVerifyString);
	Token accessToken = service.getAccessToken(authToken, verifier);
    
	String createPack = rootURL + "method=moo.pack.createPack&product=sticker&pack=" + pack;
	createPack = createPack + "&friendlyName=" + URLEncoder.encode("MQTT Inside", "UTF-8");

	OAuthRequest oAuthRequest = new OAuthRequest(Verb.POST, createPack);
    
	service.signRequest(accessToken, oAuthRequest);
	Response oAuthResponse = oAuthRequest.send();
	if (oAuthResponse.getCode() == 200) {
		System.err.println(oAuthResponse.getBody());
		JSONTokener tok = new JSONTokener(oAuthResponse.getBody());
		try {
			JSONObject json = new JSONObject(tok);
			JSONObject dropings = json.getJSONObject("dropIns");
			String finishURL = dropings.getString("finish");
			response.sendRedirect(finishURL);
		} catch (JSONException e) {
			e.printStackTrace();
		}
	} else {
		System.err.println(oAuthResponse.getBody());
	}
}

Andy is working to get a page set up on the mqtt.org site, but in the mean time you can click on the image bellow to order own set.

EDIT:

Andy has now added a Goodies page to the mqtt.org site that you can use to order stickers

First Android App

After mentioning my new toy a couple of posts ago I mentioned I was hoping to do a bit of Android development.

I’ve been doing a bit of Android work as part of my new job (I recently joined a very cool team at work) making some existing Java code run, but this is my first real application.

I had left the TV on and gone to the office one day last week and while it did not permanent damage, there was a slight bit of ghosting from the MythTV menu for a day or 2 afterwards. I already have my TV hooked up a couple of topic on my home MQTT broker, so I though that running up a simple Android Widget to show the state would be a good starting point.

MQTT Topic Tree
MQTT Topic Tree

The TV/Status topic updates every 30 seconds and currently shows either off or a string like on:RGB:6 which is made up of state, input and volume.

The first task is to create a Android Service that will subscribe to this topic and watch for changes. The standard IA92 MQTT drivers can be used with the Android SDK to get access to the broker, so the connect code is very similar to the code used in my Twitter2MQTT bridge except the id for the client is generated from the phones unique id.

private String android_id =Secure.getString(this.getContentResolver(),Secure.ANDROID_ID);

private boolean connect() {
  try {
    client = (MqttClient) MqttClient.createMqttClient("tcp://tiefighter.loc:1883", null);
    client.registerSimpleHandler(new MessageHandler());
    client.connect("droid-"+android_id, true, (short) 240);
    String topics[] = {"TV/Status"};
    int qos[] = {1};
    client.subscribe(topics, qos);
    return true;
  } catch (MqttException e) {
    e.printStackTrace();
    return false;
  }
}

The MessageHandler is an inner class and handles both incoming messages and reconnection using the following 2 methods.

public void publishArrived(String topic, byte[] payload, int qos, boolean retained) throws Exception {
	String message = new String(payload);
	// This is to allow access to the notification service from a none 	
	if (Looper.myLooper() == null) {
		Looper.prepare();
	}
			
	RemoteViews updateViews = new RemoteViews(TVLongService.this.getPackageName(), R.layout.main);
			
	String toastMsg = "";
	String toastHead = "";
	int icon = 0;
	boolean alert = false;

	if (message.equals("off")) {
		if (TVWidget.on) {
			toastMsg = "TV now OFF";
			toastHead = "TV OFF";
			icon = R.drawable.tv_off;
			alert = true;
			TVWidget.on = false;
		}
	} else {
		if (!TVWidget.on) {
			toastMsg = "TV now ON";
			toastHead = "TV ON";
			icon = R.drawable.tv_on;
			alert = true;
			TVWidget.on = true;
		}
	}

	if (alert) {
		Intent intent = new Intent();
		PendingIntent contentIntent = PendingIntent.getActivity(TVLongService.this, 0, intent, 0);
		NotificationManager notificationManager = (NotificationManager) TVLongService.this.getSystemService(Context.NOTIFICATION_SERVICE);
		Notification notification = new Notification(icon, toastHead, System.currentTimeMillis());
		notification.setLatestEventInfo(TVLongService.this, "TV Monitor", toastMsg, contentIntent);
		notificationManager.notify(1, notification);
		updateViews.setImageViewResource(R.id.img, icon);
	}

	ComponentName componentName = new ComponentName(TVLongService.this, TVWidget.class);
	AppWidgetManager appWidgetManager = AppWidgetManager.getInstance(TVLongService.this);
	appWidgetManager.updateAppWidget(componentName, updateViews);
}

TVWidget
The MessageHandler onMessage method fires when a message is posted to the TV/Status topic and then updates the widget Icon from tv_on to tv_off and then triggers a notification pop-up if the state has changed since the previous state.

The connectionLost method fires up a background thread to try and reconnect to the broker every 10 seconds.
TVWidget_on

TVWidget_toast

public void connectionLost() throws Exception {
	client = null;
	Log.v("TVMonitor","connection dropped");
	Thread t = new Thread(new Runnable() {
		
		@Override
		public void run() {
			do {//pause for 5 seconds and try again;
				Log.v("TVMonitor","sleeping for 10 seconds before trying to reconnect");
				try {
					Thread.sleep(10 * 1000);
				} catch (InterruptedException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}
			} while (!connect());
			System.err.println("reconnected");
		}
	});			
}

Next Steps

  • Survive network changes

    At the moment the code does not handle moving from WIFI to 3G networking all that well, there are collection of BroadCast future PendingIntents that should signal changes in network state.

  • Geo-Fencing

    I don’t really need to know that the TV is on while I’m at home, so I’m planning on adding a filter so notifications are only fired when I’m outside say 200m of home

  • MythTV

    May be add some indication as to what channel is currently being watched and what the upcoming recordings may be

Twitter2MQTT bridge

The recent switching off of Basic Auth by Twitter meant I had to rework some of the applications I have running at home. One of these application bridges Twitter to my local MQTT broker so that any DMs and mentions get published to two topics and also a topic that updates my status to what ever is posted.

The original version of the application just made basic http requests and parsed the XML responses, rather than just try and bolt on OAuth support I thought I would be better to start again and use one of the existing libraries that drive the Twitter APIs. There is a list of libraries for different languages http://dev.twitter.com/pages/libraries, I had a bit of a look at a couple of them and settled on Twitter4J.

In order to use OAuth you need to register your application with Twitter, you can do that here http://twitter.com/apps/new. Once registered you will get Consumer Key and a Consumer Secret. Because Twitter are using these keys to help to cut off spammers, keys need to be kept secret in order to prevent spammers knocking legitimate applications off-line, If you want to build the code attached here you will need to apply for your own key.

Before you can use a Twitter application you need to authorise it to act on your behalf, this is a 3 stage process.

  1. The application creates a URL based on it’s Consumer Key and Consumer Secret.
  2. The user follows the URL and signs into Twitter and is then asked if they want to allow the application to access on their behalf. If they allow the application then Twitter supplies a PIN.
  3. The user passes the PIN to the application which uses this to retrieve a Token, This Token is used to authenticate the application when ever it needs to act on the users behalf.
Twitter Application Authentication
Twitter Application Authentication

To do this with Twitter4J you need to do something like this:

    Twitter twitter = new TwitterFactory().getInstance();
    twitter.setOAuthConsumer(consumerKey, consumerSecret);
    try {
        RequestToken requestToken = twitter.getOAuthRequestToken();
        AccessToken accessToken = null;
        BufferedReader reader = new BufferedReader(
                                       new InputStreamReader(System.in));
        while (accessToken == null) {
            System.out.println(requestToken.getAuthorizationURL());
            System.out.println("Please follow the URL above," + 
                      " enter the PIN provided and then press Enter");
            String pin = "";
            try {
                pin = reader.readLine();
            } catch (IOException e) {
                e.printStackTrace();
            }
            accessToken = twitter.getOAuthAccessToken(requestToken, pin);
        }
        String token = accesToken.getToken();
        String secret = accesToken.getTokenSecret();
    } catch (TwitterException e) {
        e.printStackTrace();
    }

The token and the secret should be stored so the user doesn’t need to authorise the application each time it starts. Now the application is authorised you can post status updates and access the DM and Mention streams like this:

    List<DirectMessage> dms = twitter.getDirectMessages();
    List<Status> mentionsList = twitter.getMentions();
    twitter.updateStatus(newStatus);

Now that I can access the updates I need to publish them to topics and listen for when a new status is published. There is a Java library for accessing an MQTT broker provided by IBM known as the IA92 package. This package provides both J2SE and J2ME versions of the libraries. To create a connection to the broker

    IMqttClient client = null;
    try {
        client = MqttClient.createMqttClient("tcp://brocker.loc:1883"
                                                            , null);
        client.connect(clientID, true, (short) 100);
        client.subscribe({"Twitter/send"}, {1});
        client.registerSimpleHandler(new MqttSimpleCallback() {
            public void connectionLost() throws Exception {
            }

            public void publishArrived(String topic, byte[] payload, int qos,
                            boolean retained) throws Exception {
                twitter.updateStatus(new String(payload));
            }
        });

    } catch (MqttException e) {
        e.printStackTrace();
    }

Each client needs a unique id, or the older client will be kicked off when the new one collects, if you want to run multiple versions of the bridge then it may be a good idea to append the OAuth access token or the twtter screen name to the clientID.

Taking all that and knocking some of the rough edges off I have an app that will publish DMs on Twitter/<screenName>/dm & mentions on Twitter/<screenName>/mention and listens on Twitter/<screenName>/send for status updates to publish. There is a second app which subscribes to the mention and DM topics and forwards these messages on to the topic that sends SMS messages to my phone.

Next on the list of additions is a filter that will forward photos attached to tweets as MMS messages and probably some support for bridging search terms as well.

You can download my code from here, remember you will need your own Consumer Key and Consumer Secret to build it yourself, but there is a pre-built version with a valid key included.

Twitter2MQTT.tar.gz

Resources