WEMO Event notifications

As part of my on going playing with some Belkins WEMO devices I started to have a look at the UPNP Event support.

The UPNP standard as well as including discover and endpoints for control has an event system, this allows devices to publish their status changes to interested parties. The event system uses some extensions to HTTP.

SUBSCRIBE

To subscribe to events from a given device you send a request similar to the following to the eventSubURL given as defined in the setup.xml which is linked to in the UPNP discovery response.

SUBSCRIBE /upnp/event/bridge1 HTTP/1.1
CALLBACK: <http://192.168.1.1:3000/>
NT: upnp:event
TIMEOUT: Second-600
Host: 192.168.1.2:49154

CALLBACK -> is the URL to be notified
TIMEOUT -> how long to send notifications

Gets the following response:

HTTP/1.1 200 OK
DATE: Sun, 11 Jan 2015 18:27:05 GMT
SERVER: Unspecified, UPnP/1.0, Unspecified
CONTENT-LENGTH: 0
X-User-Agent: redsonic
SID: uuid:7206f5ac-1dd2-11b2-80f3-e76de858414e
TIMEOUT: Second-600

SID -> the Subscription ID

The SID is used to identify which notification come from this subscription, it can also be used to renew the subscription before the timeout expires by sending a SUBSCRIBE message like this:

SUBSCRIBE /upnp/event/bridge1 HTTP/1.1
SID: uuid:7206f5ac-1dd2-11b2-80f3-e76de858414e
TIMEOUT: Second-600
Host: 192.168.1.2:49154

NOTIFY

Incoming event notifications get delivered every time the state of the device changes and look like this for the WeMo Socket:

NOTIFY / HTTP/1.1
HOST: 192.168.1.1:3000
CONTENT-TYPE: text/xml; charset="utf-8"
CONTENT-LENGTH: 212
NT: upnp:event
NTS: upnp:propchange
SID: uuid:7206f5ac-1dd2-11b2-80f3-e76de858414e
SEQ: 0

<e:propertyset xmlns:e="urn:schemas-upnp-org:event-1-0">
  <e:property>
    <BinaryState>0</BinaryState>
  </e:property>
</e:propertyset>

The events from the light bulbs is a bit more complex, how parse them is demonstrated in the code.

UNSUBSCRIBE

And when your done you can unsubscribe with the following:

UNSUBSCRIBE / HTTP/1.1
Host: 192.168.1.2:49154
SID: uuid:7206f5ac-1dd2-11b2-80f3-e76de858414e

WeMo

Having worked all these bits out I put the following node js app together to test it all out with the WeMo devices I have:

Next step is to put all this together to build a new WeMo node module and then a improved Node-RED node.

Out Of Office Engine

Over Christmas (among other things) I’ve been messing around writing a little app to handle Out Of Office notifications for my Dad’s business.

Beaches and Blenders

Dad’s mail server runs Postfix on Linux and while there are a number of options to set up auto response messages, such as the vacation application. This seams to work well, but requires the user to edit files on the server to make it work. I wanted something a little bit more user friendly as none of the userbase for this tool have any idea what a Linux command line looks like.

I’ve finally started to move all my little side project from Java to NodeJS so I set out by crawling through npmjs to see which nodes I could use to help.

I started out with mailparser to be able to read and pull out the useful stuff from the original mail like the headers and any attachments.

Next up came email-templates to allow me to built a standard reply template then slot in a per user specific message. This was paired with nodemailer to actually send the response.

That was it for the bit that actually read and replied to the mail, but I needed user interface so I opted for a simple express web server using ejs to template the data.

Out Of Office Editor

All this is tied together with a small Mongodb to hold the details of each Out Of Office period and the list of email addresses that have been replied to. Mongodb is probably overkill for this but it’s simple to setup. It holds a record like this one for each user:

{
   "_id" : ObjectId("54a7d6f4176a69088a22b4b5"), 
  "name" : "b.hardill", 
  "startDate" : "2015-01-02T12:29:57.238Z", 
  "endDate" : "2015-01-30T21:59:17.971Z", 
  "subject" : "Away", 
  "message" : "I'll be out of the office for the next week, I'll get back to you when I get back", 
  "list" : [
    "bob@example.com", 
    "jim@example.com" 
  ]
}

I’ve stuck the project up on github here

It probably still needs some more testing but it should be good enough for what my Dad needs.

If I end up feeling really adventurous I may even try and remember hot to write a Thunderbird plugin to enable/disable things and set the dates.

Quick tip, don’t run a public facing SMTP server if you can help it

In between the Gingerbread engineering and Lego building and as part of my usual round of Christmas time IT support I ran into a new issue for me.

Towards the end of last year my dad acquired a small engineering business, as usual I got roped in to help sort out the IT side of things. I had a quick look round at what they had already over a weekend and worked out how to set up a quick off site backup system similar to the one I’ve mentioned before. The system was all Windows based so I stuck a Raspberry Pi in the wiring cupboard to give a remote access jumping off point to be able to poke at things from a far and to host a OpenVPN server to allow proper remote access.

The previous owners had set up a email system using a Windows based SMTP/POP/IMAP server app called MDaemon. It all seamed to work and I didn’t have time to swap it out for something I’m familiar with (Postfix/Dovecot on Linux) so I left it alone when I had my first look round. It turns out that the server was setup to not only accept incoming email on it’s internet facing side of things but also forward mail for signed in users. Normally I would set a mail server to only forward from users on the private internal LAN side of things, but if properly secured this arrangement should be OK. In this case things were not properly secured, lets just say that some people will be getting into the office in the new year to polite notes about selecting secure password and probably a link to this.

So one of the account had their password compromised and the server was used to send a bunch of SPAM. Once this was spotted the offending account was removed and the sever config tweaked to be a bit more sensible. This stopped the flow of SPAM and things seamed to be OK for a while until a couple of people that the business interacts with started to mention that they had not been receiving some of the email that had been sent. It turns out the IP address for the mail server had been added to a number of blacklists.

There is a site called mxtoolbox which hosts a tool for checking IP addresses against a number of the more popular blacklists. In our case we had ended up on 3 of them. The tool provides links to the sites which manage the lists and have forms to submit requests to be removed from the list once what ever has been sending the SPAM has been fixed. I have managed to successfully submit removal requests to 2 out of the 3 lists, but the Barracuda form is currently terminating the connection when ever I submit the form. As it’s the holiday period it looks like I’m going to wait until these guys are back in the office to get this fixed and hopefully get everybody accepting email.

I also found a very well hidden link to a form to get Google gmail to accept mail from us again as well, the pretty much say they you will never hear anything back from them and it could take weeks to get actioned or not…. I’ve sent this off but I’m hoping getting cleared from the other lists will help with this as well.

The other option I considered was to see if I could get the static IP assigned to the ASDL changed, but the ISP didn’t really want to entertain that so it looks like we’ll be changing ISP if I can’t get that list fixed.

So the moral of this story is find somebody to host your email for you.

Moo Sticker engines fixed

A few years ago I create a web app to allow people to order their own sets of Moo stickers with the MQTT, Node-RED and Owntracks(at the time called MQTTitude) logos.

These worked well with people ordering about 1 pack a month until Moo changed the way they did authentication. I didn’t have time to fix it until today.

I’ve moved the app from being a J2EE app over to one written in NodeJS using express, this along with the change in Moo’s authentication method has made the code a lot shorter and easier to read. I’ve published the new code on github here.

To order a set of stickers click on the appropriate image:

Playing with a set of WEMO light bulbs

I’ve been looking things like the Philips Hue since they shipped, but the price had been putting me off (and the fact that they initially only supported iOS devices and only come in screw fit).

When the Belkin WeMo Light Bulbs came round they were a cheaper option and they come in bayonet fit which meant I wouldn’t need to change any of the light fittings in the flat to play with them.

Belkin have created a mobile app to interact with the bulbs, but having to use your phone to turn the light on when you enter a room is bit of a bind. There is a replacement light switch but it’s only available in the US at the moment, so I was looking for other ways to control things. I was looking to hook the lights up to Node-RED so I could drive them based on a whole list of events.

There is a Node-RED node and a bunch of npm modules for controlling WEMO sockets but none of them currently seam to support the light bulbs. So I started to see if I could work out what was needed to get things up and running. As I was going I tweeted some of my progress which led to this exchange:

I decided to not hold my breath while waiting for a useful response and started out with a couple of the existing npm modules.

WEMO devices are all UPNP devices, which means discovering them is actually pretty easy. It also means the interface to interact with them should be pretty self descriptive. The response to the discovery request is a XML document that includes a number of useful things.

  • IP address
  • Port number
  • Unique Device Number
  • List of Services

The list of services has some interesting looking targets, but after some trial and error the bridgeservice turned out to look like the one I wanted. I ended up getting stuck and I was getting ready to set up a separate WiFi network so I could run Wireshark to see how the mobile app was actually driving the lights.

Some sterling work by Jesus Rafael Carrillo (@jescarri on github) on this issue got me the bit of info I was missing.

Taking this I managed to build a simple hard coded app that would turn the lights on and off or set the dim level and the fade times.

The next step is to pick one of the existing WEMO npm modules to add this new capability, then update the Node-RED node to go support the new features.

Node-RED Geofence node

We’ve been looking at some new location based uses for Node-RED recently. To this end we’ve been standardising on adding the information under the msg.location property.

msg: {
  topic: "foo",
  payload: "bar",
  location: {
    lat: 51.02477
    lon: -1.39724
  }
}

As part of this I’ve been writing a geofence node. This will allow messages to be filtered against specific regions.

The first pass worked well supporting both circular and rectangular regions, but configuring it by entering coordinates is not the most user friendly.

Screenshot from 2014-11-11 22:23:33

After a bit of playing with leafletjs and the leaflet.draw plugin I managed to come up with this which should be a lot more intuitive.

Geofence config dialog

At the moment it needs to load the leaflet js and css from a external server until we come up with a way to bundle static content inside nodes. This shouldn’t really be a problem as it’s having to load the map tiles from the internet as well. I’d like to find a way to pass in a base URL for the map server and to gracefully fall back to the text version if there is no access to a map server.

The map version also supports creating arbitrary polygon regions by joining up multiple points.

I’m still trying to get the config dialog to gracefully fall back to the basic version when there is no access to a mapserver but having problems with detecting the failure to download a map title. Any pointers would be gratefully accepted.

The code is on github and on npm. It can be installed with:

npm install node-red-node-geofence

LDAP and NFC Node-RED Nodes

About a week ago a colleague asked me to help resurrect some code I had written to use our work ID badges to look up information on the card owner in order to log into a system for a demonstration.

The ID badges are basically mifare cards so can be read by a NFC reader. The content of the cards is encrypted, but each card has a unique ID. Unfortunately the security team will not share the mapping of these IDs to actual people, but since this is for a demonstration that will only be given by a relatively small number of people it’s not a problem to set up a list of mappings our selves.

The original version of this code used nfc-eventd and some java code to the IDs then do a lookup in a little database to convert these to email addresses. It worked but was a bit of a pig to setup and move between machines as it required a number of different apps and config files so I decided to have a go at rewriting it all in Node-RED.

NFC ID Flow

To do this I was going to need 2 new nodes, one to read the NFC card and one to look up details in the LDAP. Both of these actually proved reasonable easy and quick to write as there are existing Node NPM modules that do most of the heavy lifting. The flow has a couple of extra bit, it uses a mongodb to store the id to email address mappings and if there is no match it uses websockets to populate a field in a separate web page to enter a email address to update the database.

NFC

I did a first pass using the nfc npm and it worked but there was no way to shut the connection to the NFC reader down in the code which meant I couldn’t clean up properly when Node-RED shut down or when the node needed to be restarted.

The nfc on npmjs.org is actually a bit out of date compared to the git repository it’s hosted in. So I moved over to using the upstream version of the code. This changed the API a little and still didn’t have a mechanism to allow the interface to be stopped. I forked the project and after a little bit of playing I ended up with some working shutdown code.

The only call back is for when at NFC tag is detected and it polls in tight loop so the stream of data from the node is way too high to feed into a Node-RED flow really. The Node-RED wrapper rate limits to only reporting the same tag once every 10 seconds. This is good enough for the original problem I was looking to solve but I still think it can be done better. I’m planning on adding call backs for tag seen and when it is removed, this is similar to how nfc-eventd works. I also want to look at doing NDEF decoding.

You can install the current version of the node with:

npm install https://github.com/hardillb/node-red-contrib-nfc/archive/master.tar.gz

It depends on libnfc which should work on the Linux and OSx and I’ve even seen instructions to build it for Windows.
Once I’ve got a bit further I’ll add it to npmjs.org.

LDAP

This one was even simpler. The LDAP npm modules links to the openldap libraries and does all the hard work.

It just needed a config dialog creating to take a base DN and a filter and a connection setup that takes a server, port and if needed a bind DN and password. The filter is a mustache template so values can be passed in.

This node is pretty much done, you can find the code on github here and the node can be installed with the following:

npm install node-red-node-ldap

Like the NFC node openldap should be available for Linux and OSx and there looks to be a Windows port

More playing with Asterisk

I have been playing some more with Asterisk and I’ve got 2 useful bits to share.

MP3 Voicemail

The simple one first, a recent update to Android (not sure what exactly) means it won’t playback wav files attached to emails. This is a problem as when Asterisk records a voicemail it can be configured to email the recording in wav to mailbox owner. A little bit of googling turned this up http://bernaerts.dyndns.org/linux/179-asterisk-voicemail-mp3. It needed updating a little to get it to work on my Raspberry Pi.

First I needed to change one of the sed line to match WAV not wav to fix the file name from this:

...| sed 's/.wav/.mp3/g' > stream.part3.mp3.head

to this:

...| sed 's/.WAV/.mp3/g' > stream.part3.mp3.head

Secondly lame doesn’t like the encoding for the wav file (I think it’s because the stored values are unsigned) so we need to run it through sox to fix it first.

sox stream.part3.wav -e signed stream.part3a.wav
lame -m m -b 24 stream.part3a.wav stream.part3.mp3

This not only makes it so I can listen to the messages on my phone and tablet it also makes the mails smaller so they take up less bandwidth.

Using a 3G Stick to make calls

Having used OBi110 to hook my Asterisk VoIP rig up to a standard phone line I was looking for a way to hook a mobile phone to the system. There are 2 options with Asterisk, chan_bluetooth and chan_dongle.

Chan_bluetooth uses a bluetooth connection to a mobile phone to make and receive calls. I had a look at this but it meant keeping phone plugged into a charger and having another bluetooth adapter plugged in.

Chan_dongle work with certain Huawei 3G USB modems. These 3G sticks are basically full phones with a USB interface. I’ve already been using one of listed modems for my SMS/MMS project and I had a spare one kicking around. It needed the firmware updating to make it work, which was a bit of a challenge as it required setting up a real Windows machine as I couldn’t get it to work in a VM.

Setting up the dongle was a little tricky at first as I couldn’t get a set of udev rules to match the stick properly to ensure it always ends up with the same device names. The code does let you specify the stick using it’s IMEI which helps if you have multiple sticks plugged into the same computer.

Once configured it was easy to set up the extensions.conf to allow making and receiving calls. The main reason for setting this up was to have a portable VoIP rig that I can take to different places and not have to worry about a fixed phone line. There is a upcoming hackday that I have a plan for.