Moo Sticker engines fixed

A few years ago I create a web app to allow people to order their own sets of Moo stickers with the MQTT, Node-RED and Owntracks(at the time called MQTTitude) logos.

These worked well with people ordering about 1 pack a month until Moo changed the way they did authentication. I didn’t have time to fix it until today.

I’ve moved the app from being a J2EE app over to one written in NodeJS using express, this along with the change in Moo’s authentication method has made the code a lot shorter and easier to read. I’ve published the new code on github here.

To order a set of stickers click on the appropriate image:

Node-RED Geofence node

We’ve been looking at some new location based uses for Node-RED recently. To this end we’ve been standardising on adding the information under the msg.location property.

msg: {
  topic: "foo",
  payload: "bar",
  location: {
    lat: 51.02477
    lon: -1.39724
  }
}

As part of this I’ve been writing a geofence node. This will allow messages to be filtered against specific regions.

The first pass worked well supporting both circular and rectangular regions, but configuring it by entering coordinates is not the most user friendly.

Screenshot from 2014-11-11 22:23:33

After a bit of playing with leafletjs and the leaflet.draw plugin I managed to come up with this which should be a lot more intuitive.

Geofence config dialog

At the moment it needs to load the leaflet js and css from a external server until we come up with a way to bundle static content inside nodes. This shouldn’t really be a problem as it’s having to load the map tiles from the internet as well. I’d like to find a way to pass in a base URL for the map server and to gracefully fall back to the text version if there is no access to a map server.

The map version also supports creating arbitrary polygon regions by joining up multiple points.

I’m still trying to get the config dialog to gracefully fall back to the basic version when there is no access to a mapserver but having problems with detecting the failure to download a map title. Any pointers would be gratefully accepted.

The code is on github and on npm. It can be installed with:

npm install node-red-node-geofence

LDAP and NFC Node-RED Nodes

About a week ago a colleague asked me to help resurrect some code I had written to use our work ID badges to look up information on the card owner in order to log into a system for a demonstration.

The ID badges are basically mifare cards so can be read by a NFC reader. The content of the cards is encrypted, but each card has a unique ID. Unfortunately the security team will not share the mapping of these IDs to actual people, but since this is for a demonstration that will only be given by a relatively small number of people it’s not a problem to set up a list of mappings our selves.

The original version of this code used nfc-eventd and some java code to the IDs then do a lookup in a little database to convert these to email addresses. It worked but was a bit of a pig to setup and move between machines as it required a number of different apps and config files so I decided to have a go at rewriting it all in Node-RED.

NFC ID Flow

To do this I was going to need 2 new nodes, one to read the NFC card and one to look up details in the LDAP. Both of these actually proved reasonable easy and quick to write as there are existing Node NPM modules that do most of the heavy lifting. The flow has a couple of extra bit, it uses a mongodb to store the id to email address mappings and if there is no match it uses websockets to populate a field in a separate web page to enter a email address to update the database.

NFC

I did a first pass using the nfc npm and it worked but there was no way to shut the connection to the NFC reader down in the code which meant I couldn’t clean up properly when Node-RED shut down or when the node needed to be restarted.

The nfc on npmjs.org is actually a bit out of date compared to the git repository it’s hosted in. So I moved over to using the upstream version of the code. This changed the API a little and still didn’t have a mechanism to allow the interface to be stopped. I forked the project and after a little bit of playing I ended up with some working shutdown code.

The only call back is for when at NFC tag is detected and it polls in tight loop so the stream of data from the node is way too high to feed into a Node-RED flow really. The Node-RED wrapper rate limits to only reporting the same tag once every 10 seconds. This is good enough for the original problem I was looking to solve but I still think it can be done better. I’m planning on adding call backs for tag seen and when it is removed, this is similar to how nfc-eventd works. I also want to look at doing NDEF decoding.

You can install the current version of the node with:

npm install https://github.com/hardillb/node-red-contrib-nfc/archive/master.tar.gz

It depends on libnfc which should work on the Linux and OSx and I’ve even seen instructions to build it for Windows.
Once I’ve got a bit further I’ll add it to npmjs.org.

LDAP

This one was even simpler. The LDAP npm modules links to the openldap libraries and does all the hard work.

It just needed a config dialog creating to take a base DN and a filter and a connection setup that takes a server, port and if needed a bind DN and password. The filter is a mustache template so values can be passed in.

This node is pretty much done, you can find the code on github here and the node can be installed with the following:

npm install node-red-node-ldap

Like with the NFC node, openldap should be available for Linux and OSx and there looks to be a Windows port.

Running Node-Red as a Windows or OSx Service

For a recent project I needed to run Node-RED on windows and it became apparent that being able to run it as a service would be very useful.

After a little poking around I found a npm module called node-windows.

You install node-windows with as follows:

npm install -g node-windows

followed by:

npm link node-windows

in the root directory of your project. This is a 2 stage process as node-windows works better when installed globally.

Now the npm module in installed you configure the Windows service by writing a short nodejs app. This windows-service.js should work for Node-Red

var Service = require('node-windows').Service;

var svc = new Service({
  name:'Node-Red',
  description: 'A visual tool for wiring the Internet of Things',
  script: require('path').join(__dirname,'red.js')
});

svc.on('install',function(){
  svc.start();
});

svc.on('uninstall',function(){
  console.log('Uninstall complete.');
  console.log('The service exists: ',svc.exists);
});

if (process.argv.length == 3) {
  if ( process.argv[2] == 'install') {
    svc.install();
  } else if ( process.argv[2] == 'uninstall' ) {
    svc.uninstall();
  }
}

Run the following to install the service:

node windows-service.js install

and to remove the service:

node windows-service.js uninstall

There is also a OSx version of node-windows called node-mac, the same script with a small change should work on both:

if (process.platform === 'win32') {
  var Service = require('node-windows').Service;
} else if (process.platform === 'darwin') {
  var Service = require('node-mac').Service;
} else {
  console.log('Not Windows or OSx');
  process.exit(1);
}

var svc = new Service({
  name:'Node-Red',
  description: 'A visual tool for wiring the Internet of Things',
  script: require('path').join(__dirname,'red.js')
});


svc.on('install',function(){
  svc.start();
});

svc.on('uninstall',function(){
  console.log('Uninstall complete.');
  console.log('The service exists: ',svc.exists);
});

if (process.argv.length == 3) {
  if ( process.argv[2] == 'install') {
    svc.install();
  } else if ( process.argv[2] == 'uninstall' ) {
    svc.uninstall();
  }
}

I have submitted a pull request to include this in the base Node-RED install.

EDIT:

I’ve added node-linux to the pull request as well to generate /etc/init.d SystemV start scripts.

Node-RED at Zurich developerWorks days 2013

Node-RED icon
This year I was lucky enough to be asked back to speak at developerWorks days 2013 in Zurich after giving 2 presentations last year.

This year I was presenting on a great piece of work done by Nick O’Leary and Dave Conway-Jones called Node-RED.

Node-RED is a light weight, edge of the network event processing engine. The main aim is to make it easy to bridge a wide variety of input and output sources and to allow logic to be applied to the events/messages that flow between them.

For my presentation I wanted to try and use Node-RED as much as possible so I set about seeing if I could use it to host and control my slide deck. I started out with a impress.js presentation the Nick had written. Impress is a Javascript framework to build HTML5 presentation similar to prezi, but it also exposes an API to drive the slide transitions from and external source. Combining this feature with the MQTT over WebSockets will allow me to drive things remotely.

I added the following bit of code to the end of the presentation.html

<script type="text/javascript" src="js/mqttws31.js"></script>
<script src="js/impress.js"></script>
<script src="js/impressConsole.js"></script>
<script>
    var imp = impress();
    imp.init();

    var client;
    var slide;

    function setupMQTT() {
    	client = new Messaging.Client(document.location.hostname,8181,"presentation");
		client.onConnectionLost = onConnectionLost;
		client.onMessageArrived = onMessageArrived;
		client.connect({onSuccess:onConnect});
    }

   function onConnectionLost(response) {
	setTimeout(setupMQTT, 500);
	document.removeEventListener('impress:stepenter', sendStepEnter);
   }

   function onMessageArrived(message) {
	if (message.payloadString === "next") {
		imp.next();
	} else if (message.payloadString === "prev") {
		imp.prev();
	} else {
		console.log(slide);
		if (slide === "demotimeagain") {
			//update with twitter details
			console.log(message.payloadString)
			obb = JSON.parse(message.payloadString);
			console.log(obb);
			document.getElementById('injected-twitter-screen').innerHTML = obb.sender.screen_name;
			document.getElementById('injected-twitter-id').innerHTML = obb.sender.name;
			document.getElementById('injected-twitter-tweet').innerHTML = obb.body;
		} else {
			imp.goto(message.payloadString);
		}
	}
   }

   function onConnect() {
	client.subscribe("pres");
	document.addEventListener('impress:stepenter', sendStepEnter);
   }

   function sendStepEnter(step) {
	console.log(step.target.id);
	slide = step.target.id;
	message = new Messaging.Message(step.target.id);
	message.destinationName = "slide";
	client.send(message);
    }

    setupMQTT();

</script>

This first sets up impress.js then starts to set up some basic boiler plate to create a MQTT client connection over Web Sockets. The onConnect function subscribes this clients to the ‘pres’ topic that will be used to receive ‘next’ & ‘prev’ messages to advance the slides. It also adds a event listener that receives events from impress.js each time a new slide is displayed.

The onMessage function handles the ‘next’& ‘prev’ and also a couple of special case to populate some data into a slide following a demonstration.

This a basic Node-RED flow to make this all work can be found here

In order to get Node-RED to serve the presentation html and required javascript libraries used to require embedding Node-RED into a custom application, but this requirement was removed with a new feature in Node-RED 0.4.0. 0.4.0 include a new configuration setting called httpStatic which allows you to specify a directory holding a collection of static content, when you use this setting you also need to specify httpRoot to move the Node-RED gui to a different root directory.

The basic version of the presentation is embedded here:

If you click on the slide you can then navigate back and forth using the arrow keys.

You can access it full screen here

Node-Red – Ti SensorTag Node

Last weekend I spent some time working on yet another Node-Red node. This one is an input node that reads the data published by a small sensor platorm from Ti.

The Ti SensorTag is a Bluetooth 4.0 LE platform designed to be a test source for building new BLE and is very accessible at only $25 dollars especially with the following list of sensors on board:

  • Ambient Temperature
  • IR remote Temperature
  • Air Pressure
  • Humidity
  • Accelerometer
  • Magnetometer
  • Gyroscope
  • 2 Push Buttons

Ti have also set up a open wiki to allow people to document they experiments with the device.

You can find the node in the new node-red-nodes repo on github here it relies on a slightly updated version of Sandeep Mistry‘s node-sensortag, the read me explains how to install my update (until I get round to submitting the pull request) but here is the command to run in the root of your Node-Red directory:

npm install sensortag@https://api.github.com/repos/hardillb/node-sensortag/tarball

(you may need to install the libbluetooth-dev package for Debian/Ubuntu based distros and bluez-libs-devel on Redhat/Fedora first)

UPDATE:
Sandeep has now merged my changes so the sensortag node can now be installed normally with npm with:

npm install sensortag

Once installed you need to run Node-RED as root as this is the only way to get access to the BLE functions, then you can add the node to the canvas and configure which sensors are pushed as events

Please feel free to have a play and let me know what you think

Node-Red – Delay Node (formally Pause Node)

I’ve just been updating my Pause node for Node-Red after a request to support message rate limiting as well as pausing individual messages.

To help make it a little clearer the node has also been renamed to Delay (thanks to deldrid1 for the suggestion)

Delay Mode

In this mode the node allows you to delay any message passing through it by a given number of milliseconds, seconds, hours or days.

Rate Limit Mode

This time the node ensures that no more than the given number of messages are delivered per millisecond, second, hour or day.

All the code is in my fork of the original project for now and soon to be rolled up in to the main stream.

Taking a look at the neighbourhood

A recent article about detecting offline social networks from information about preferred WIFI networks leaked by mobile devices led me to have another look at the work I’d done looking at WIFI location detection to see what other information I could derive.

I had been having problems with finding WIFI adapters with the right chipsets to allow me to use them in monitor mode in order to capture all the available packets from different networks, but recent updates to some of the WIFI drivers in the Linux kernel have enabled some more of the devices I have access to.

Previously I build a small application which works with the Kismet wireless scanner application to publish details of each device spotted to a MQTT topic tree. With a small modification it now also published the data about the WIFI networks that are being searched for.

Then using 2 simple Node-RED flows this data is stored into a MongoDB instance

You can have a look at the flow here.

From the device’s MAC address it is possible to determine the manufacture, so with another little node application to query the MongoDB store I can generate this d3js view of what type of devices are in use in the area round my flat.

The view dynamically updates every 5 seconds to pick up the latest information.

Now I know who owns what type of device, time to see who might know who. By plotting a force directed graph of all the clients detected and linking them based on the networks they have been searching for I can build up a view of which devices may belong to people who know each other.

Force directed network graph

There are a couple of clusters in the data so far, but most of them are from public WIFI networks like BTOpenzone and O2 Wifi. After filtering these services out there was still the 3 devices that look to be using Mike’s Lumina 800 for internet access and 4 devices connected to the same Sky Broadband router. I expect the data to be a lot more interesting when I get to run it somewhere with a few more people.

At the moment this is all running on my laptop, but it should run fine on my raspberry pi or my home server, as soon as I’ve transferred it over I’ll put a link up to live version of the charts.