LDAP and NFC Node-RED Nodes

About a week ago a colleague asked me to help resurrect some code I had written to use our work ID badges to look up information on the card owner in order to log into a system for a demonstration.

The ID badges are basically mifare cards so can be read by a NFC reader. The content of the cards is encrypted, but each card has a unique ID. Unfortunately the security team will not share the mapping of these IDs to actual people, but since this is for a demonstration that will only be given by a relatively small number of people it’s not a problem to set up a list of mappings our selves.

The original version of this code used nfc-eventd and some java code to the IDs then do a lookup in a little database to convert these to email addresses. It worked but was a bit of a pig to setup and move between machines as it required a number of different apps and config files so I decided to have a go at rewriting it all in Node-RED.

NFC ID Flow

To do this I was going to need 2 new nodes, one to read the NFC card and one to look up details in the LDAP. Both of these actually proved reasonable easy and quick to write as there are existing Node NPM modules that do most of the heavy lifting. The flow has a couple of extra bit, it uses a mongodb to store the id to email address mappings and if there is no match it uses websockets to populate a field in a separate web page to enter a email address to update the database.

NFC

I did a first pass using the nfc npm and it worked but there was no way to shut the connection to the NFC reader down in the code which meant I couldn’t clean up properly when Node-RED shut down or when the node needed to be restarted.

The nfc on npmjs.org is actually a bit out of date compared to the git repository it’s hosted in. So I moved over to using the upstream version of the code. This changed the API a little and still didn’t have a mechanism to allow the interface to be stopped. I forked the project and after a little bit of playing I ended up with some working shutdown code.

The only call back is for when at NFC tag is detected and it polls in tight loop so the stream of data from the node is way too high to feed into a Node-RED flow really. The Node-RED wrapper rate limits to only reporting the same tag once every 10 seconds. This is good enough for the original problem I was looking to solve but I still think it can be done better. I’m planning on adding call backs for tag seen and when it is removed, this is similar to how nfc-eventd works. I also want to look at doing NDEF decoding.

You can install the current version of the node with:

npm install https://github.com/hardillb/node-red-contrib-nfc/archive/master.tar.gz

It depends on libnfc which should work on the Linux and OSx and I’ve even seen instructions to build it for Windows.
Once I’ve got a bit further I’ll add it to npmjs.org.

LDAP

This one was even simpler. The LDAP npm modules links to the openldap libraries and does all the hard work.

It just needed a config dialog creating to take a base DN and a filter and a connection setup that takes a server, port and if needed a bind DN and password. The filter is a mustache template so values can be passed in.

This node is pretty much done, you can find the code on github here and the node can be installed with the following:

npm install node-red-node-ldap

Like with the NFC node, openldap should be available for Linux and OSx and there looks to be a Windows port.

Running Node-Red as a Windows or OSx Service

For a recent project I needed to run Node-RED on windows and it became apparent that being able to run it as a service would be very useful.

After a little poking around I found a npm module called node-windows.

You install node-windows with as follows:

npm install -g node-windows

followed by:

npm link node-windows

in the root directory of your project. This is a 2 stage process as node-windows works better when installed globally.

Now the npm module in installed you configure the Windows service by writing a short nodejs app. This windows-service.js should work for Node-Red

var Service = require('node-windows').Service;

var svc = new Service({
  name:'Node-Red',
  description: 'A visual tool for wiring the Internet of Things',
  script: require('path').join(__dirname,'red.js')
});

svc.on('install',function(){
  svc.start();
});

svc.on('uninstall',function(){
  console.log('Uninstall complete.');
  console.log('The service exists: ',svc.exists);
});

if (process.argv.length == 3) {
  if ( process.argv[2] == 'install') {
    svc.install();
  } else if ( process.argv[2] == 'uninstall' ) {
    svc.uninstall();
  }
}

Run the following to install the service:

node windows-service.js install

and to remove the service:

node windows-service.js uninstall

There is also a OSx version of node-windows called node-mac, the same script with a small change should work on both:

if (process.platform === 'win32') {
  var Service = require('node-windows').Service;
} else if (process.platform === 'darwin') {
  var Service = require('node-mac').Service;
} else {
  console.log('Not Windows or OSx');
  process.exit(1);
}

var svc = new Service({
  name:'Node-Red',
  description: 'A visual tool for wiring the Internet of Things',
  script: require('path').join(__dirname,'red.js')
});


svc.on('install',function(){
  svc.start();
});

svc.on('uninstall',function(){
  console.log('Uninstall complete.');
  console.log('The service exists: ',svc.exists);
});

if (process.argv.length == 3) {
  if ( process.argv[2] == 'install') {
    svc.install();
  } else if ( process.argv[2] == 'uninstall' ) {
    svc.uninstall();
  }
}

I have submitted a pull request to include this in the base Node-RED install.

EDIT:

I’ve added node-linux to the pull request as well to generate /etc/init.d SystemV start scripts.

Network Attached Light Sensor

It was one of the projects that started out with an innocent enough question….

What do you know about light sensors?

The answer was not much, but I can find out, if you can give me some context (which is a pretty standard answer from ETS if it’s something new to us). The person asking was interested in measuring the natural light levels at a reasonable number of out door locations in order to help in making a call on if a given activity was safe to continue.

The client had already been looking at an existing solution which was using USB enabled light meters being routed over ethernet back to a central location. This felt like a really clunky solution so I was asked to have a look to see if I could come up with something a little different.

My first thought was to look at using something with a LDR but that would only really give relative light levels or require a lot of work to calibrate the system. It was time to have a bit of a poke around.

After a bit of searching I found reference to a TSL2561 ic which is a light level sensor that outputs via I2C. I found 2 boards with these sensors mounted, the first from Adafruit but I couldn’t find a UK supplier to get one to test with. A bit more digging and I found a very similar Sparkfun board that was available from Cool Components in the UK. The sensor has a range of 0.1 to 40k lux

The client has a PoE enabled ethernet network covering the site they wish to measure which means I should be able to thuse the a PoE enabled Ethernet Arduino to read from the sensor and report the values back to a central location. I used the sample library from Sparkfun as a starting point and then extended it with the MQTT client library from Nick O’leary.

I threw a prototype together with a breadboard and installed it on the window sill in my office to see how it performed. On the whole it seams pretty good, but I need to find a off the shelf meter that measures in lux to see how the values compare.

I now had a sensor that is publishing it’s light levels about once a second, this solved the initial problem but I needed to visualise the results. I fired up Node-Red to take the values and do a few things with them.

  • Stash the timestamp and light level in a mongo db store
  • Publish alerts when the value dropped below a given threshold
  • Use the light level to change the brightness of my Digispark RGB LED

I also ran up a visualisation using the Rickshaw Javascript library which allows you to draw time series charts using the D3 visualisation library.

The chart on the left show the sun coming over the building opposite my east facing office windows then cutting off sharply as it tracks round to the west.

The next challenge is to come up with an enclosure for the whole thing so it can survive outside in the British summer.

Node-RED at Zurich developerWorks days 2013

Node-RED icon
This year I was lucky enough to be asked back to speak at developerWorks days 2013 in Zurich after giving 2 presentations last year.

This year I was presenting on a great piece of work done by Nick O’Leary and Dave Conway-Jones called Node-RED.

Node-RED is a light weight, edge of the network event processing engine. The main aim is to make it easy to bridge a wide variety of input and output sources and to allow logic to be applied to the events/messages that flow between them.

For my presentation I wanted to try and use Node-RED as much as possible so I set about seeing if I could use it to host and control my slide deck. I started out with a impress.js presentation the Nick had written. Impress is a Javascript framework to build HTML5 presentation similar to prezi, but it also exposes an API to drive the slide transitions from and external source. Combining this feature with the MQTT over WebSockets will allow me to drive things remotely.

I added the following bit of code to the end of the presentation.html

<script type="text/javascript" src="js/mqttws31.js"></script>
<script src="js/impress.js"></script>
<script src="js/impressConsole.js"></script>
<script>
    var imp = impress();
    imp.init();

    var client;
    var slide;

    function setupMQTT() {
    	client = new Messaging.Client(document.location.hostname,8181,"presentation");
		client.onConnectionLost = onConnectionLost;
		client.onMessageArrived = onMessageArrived;
		client.connect({onSuccess:onConnect});
    }

   function onConnectionLost(response) {
	setTimeout(setupMQTT, 500);
	document.removeEventListener('impress:stepenter', sendStepEnter);
   }

   function onMessageArrived(message) {
	if (message.payloadString === "next") {
		imp.next();
	} else if (message.payloadString === "prev") {
		imp.prev();
	} else {
		console.log(slide);
		if (slide === "demotimeagain") {
			//update with twitter details
			console.log(message.payloadString)
			obb = JSON.parse(message.payloadString);
			console.log(obb);
			document.getElementById('injected-twitter-screen').innerHTML = obb.sender.screen_name;
			document.getElementById('injected-twitter-id').innerHTML = obb.sender.name;
			document.getElementById('injected-twitter-tweet').innerHTML = obb.body;
		} else {
			imp.goto(message.payloadString);
		}
	}
   }

   function onConnect() {
	client.subscribe("pres");
	document.addEventListener('impress:stepenter', sendStepEnter);
   }

   function sendStepEnter(step) {
	console.log(step.target.id);
	slide = step.target.id;
	message = new Messaging.Message(step.target.id);
	message.destinationName = "slide";
	client.send(message);
    }

    setupMQTT();

</script>

This first sets up impress.js then starts to set up some basic boiler plate to create a MQTT client connection over Web Sockets. The onConnect function subscribes this clients to the ‘pres’ topic that will be used to receive ‘next’ & ‘prev’ messages to advance the slides. It also adds a event listener that receives events from impress.js each time a new slide is displayed.

The onMessage function handles the ‘next’& ‘prev’ and also a couple of special case to populate some data into a slide following a demonstration.

This a basic Node-RED flow to make this all work can be found here

In order to get Node-RED to serve the presentation html and required javascript libraries used to require embedding Node-RED into a custom application, but this requirement was removed with a new feature in Node-RED 0.4.0. 0.4.0 include a new configuration setting called httpStatic which allows you to specify a directory holding a collection of static content, when you use this setting you also need to specify httpRoot to move the Node-RED gui to a different root directory.

The basic version of the presentation is embedded here:

If you click on the slide you can then navigate back and forth using the arrow keys.

You can access it full screen here

Node-Red – Ti SensorTag Node

Last weekend I spent some time working on yet another Node-Red node. This one is an input node that reads the data published by a small sensor platorm from Ti.

The Ti SensorTag is a Bluetooth 4.0 LE platform designed to be a test source for building new BLE and is very accessible at only $25 dollars especially with the following list of sensors on board:

  • Ambient Temperature
  • IR remote Temperature
  • Air Pressure
  • Humidity
  • Accelerometer
  • Magnetometer
  • Gyroscope
  • 2 Push Buttons

Ti have also set up a open wiki to allow people to document they experiments with the device.

You can find the node in the new node-red-nodes repo on github here it relies on a slightly updated version of Sandeep Mistry‘s node-sensortag, the read me explains how to install my update (until I get round to submitting the pull request) but here is the command to run in the root of your Node-Red directory:

npm install sensortag@https://api.github.com/repos/hardillb/node-sensortag/tarball

(you may need to install the libbluetooth-dev package for Debian/Ubuntu based distros and bluez-libs-devel on Redhat/Fedora first)

UPDATE:
Sandeep has now merged my changes so the sensortag node can now be installed normally with npm with:

npm install sensortag

Once installed you need to run Node-RED as root as this is the only way to get access to the BLE functions, then you can add the node to the canvas and configure which sensors are pushed as events

Please feel free to have a play and let me know what you think

Node-Red – Delay Node (formally Pause Node)

I’ve just been updating my Pause node for Node-Red after a request to support message rate limiting as well as pausing individual messages.

To help make it a little clearer the node has also been renamed to Delay (thanks to deldrid1 for the suggestion)

Delay Mode

In this mode the node allows you to delay any message passing through it by a given number of milliseconds, seconds, hours or days.

Rate Limit Mode

This time the node ensures that no more than the given number of messages are delivered per millisecond, second, hour or day.

All the code is in my fork of the original project for now and soon to be rolled up in to the main stream.

Taking a look at the neighbourhood

A recent article about detecting offline social networks from information about preferred WIFI networks leaked by mobile devices led me to have another look at the work I’d done looking at WIFI location detection to see what other information I could derive.

I had been having problems with finding WIFI adapters with the right chipsets to allow me to use them in monitor mode in order to capture all the available packets from different networks, but recent updates to some of the WIFI drivers in the Linux kernel have enabled some more of the devices I have access to.

Previously I build a small application which works with the Kismet wireless scanner application to publish details of each device spotted to a MQTT topic tree. With a small modification it now also published the data about the WIFI networks that are being searched for.

Then using 2 simple Node-RED flows this data is stored into a MongoDB instance

You can have a look at the flow here.

From the device’s MAC address it is possible to determine the manufacture, so with another little node application to query the MongoDB store I can generate this d3js view of what type of devices are in use in the area round my flat.

The view dynamically updates every 5 seconds to pick up the latest information.

Now I know who owns what type of device, time to see who might know who. By plotting a force directed graph of all the clients detected and linking them based on the networks they have been searching for I can build up a view of which devices may belong to people who know each other.

Force directed network graph

There are a couple of clusters in the data so far, but most of them are from public WIFI networks like BTOpenzone and O2 Wifi. After filtering these services out there was still the 3 devices that look to be using Mike’s Lumina 800 for internet access and 4 devices connected to the same Sky Broadband router. I expect the data to be a lot more interesting when I get to run it somewhere with a few more people.

At the moment this is all running on my laptop, but it should run fine on my raspberry pi or my home server, as soon as I’ve transferred it over I’ll put a link up to live version of the charts.

Node-Red – Digispark RGB Node

Following on from last weekends post about adding a pause node to Node-Red this weekend I’ve been trying to create a node to support my Digispark RGB USB LED.

The RGB shield for the Digispark came with a sketch and a some python code to send colours to the device using the HID protocol. I had a bit of a poke round and I found node-hid and thought things where going to be easy.

Unfortunately just trying to duplicate the python code in nodejs didn’t seam to work no matter what I tried.

After a bit more digging I found this blog post by Dougal Campbell who looked to have been pursuing a similar idea a little earlier in the year. I couldn’t see any follow up post on his blog but a quick comment later and I had pointers to what he’d managed to get working. I was hoping to leave the default sketch on the digiSpark but it needed a little change to the USB library to make things work properly, you can find the changes here.

88-digiRGB.js

// Require main module
var RED = require("../../red/red");
var HID = require('node-hid');
var device;
var node;

// The main node definition - most things happen in here
function DigiRGBNode(n) {
    // Create a RED node
    RED.nodes.createNode(this,n);
    node=this;
    //look up the matching devices
    var devices = HID.devices(0x16c0,0x05df);
    for (var i=0; i< devices.length; i++) {
      if (devices[i].product == 'DigiUSB') {
        path = devices[i].path;
        node.log("found: " + path);
        try {
          device = new HID.HID(devices[i].path);
          //only work with the first one found
          break;
        } catch (e) {
          node.log(e)
        }
      }  
   }
   
   if (device) {   
      this.on("input", function(msg) {
        if (msg != null) {
          var args = msg.payload.split(',');
          if (args.length == 3) {
            device.sendFeatureReport([115,parseInt(args[0]),parseInt(args[1]),parseInt(args[2])]);
          }
        }
      });
   } else {
      node.warn("no digispark RGB found");
   }
   
}

// Register the node by name. This must be called before overriding any of the
// Node functions.
RED.nodes.registerType("digiRGB",DigiRGBNode);


DigiRGBNode.prototype.close = function() {
    // Called when the node is shutdown - eg on redeploy.
    // Allows ports to be closed, connections dropped etc.
    // eg: this.client.disconnect();
    device.close();
}

If you want to have a play unzip 78-digiRGB.zip in the node-red directory to unpack the whole node into the nodes/hardware directory. I have now saved this a Github GIST which you can check out directly into /node-red/nodes/hardware directory as follows:

[user@node node-red]$ git clone https://gist.github.com/6573158.git

You will also need to run npm import node-hid in the node-red directory to add the dependency.

(Remember you will need to modify the DigiUSB.cpp and the usbconfig.h in the DigisparkUSB example as mentioned in the link above).

This is unlikely to make it into core Node-RED as the hardware is a little specialist, but if Nick and Dave decide to host a collection of hardware nodes it may end up in there.

Node-Red – Pause Node

Two guys from my team (Nick O’Leary and Dave Conway-Jones) at IBM released a very cool tool for wiring up IoT last week called Node-Red.

Node-Red lets you wire together a selection of inputs, things like MQTT messages, Twitter feeds, serial port and a whole lot more, perform transforms on them with Javascript functions before outputting them to a similar collection of options.

I’ve been playing with it to replace a couple things that used to be standalone scripts, but one of the things that was missing was the way to delay an action happening until a certain time after a message has arrived. For example if I want to set my Blink(1) clone to green when somebody mentions me in the tweet I may want to reset it to off 20 seconds later

To help with this I have written a new node called pause that delays passing on a given message by a configurable amount of time. Nodes are made up of 2 files and are placed in the node-red/nodes directory. As well as all the built in nodes there is a example node (99-sample) which you can use to

Each node is made up of 2 parts, the first is javascript file which controls how the node should behave and a html file which contains the descriptive text,

99-pause.js

...
// Simple node to introduce a pause into a flow

//Require main module
var RED = require("../red/red");

function PauseNode(n) {
   RED.nodes.createNode(this,n);
   
   this.timeout = n.timeout * 1000;
   
   this.on("input", function(msg) {
       var node= this;
       setTimeout(function(){node.send(msg);}, node.timeout);
   });
}

RED.nodes.registerType("pause",PauseNode);

You can see in the code above I’ve attached a small function to the ‘input’ event for the pause node, all it does setup a callback after the timeout expires to send the passed in message object to the next node in the flow.

99-pause.html

...
<!-- First, the content of the edit dialog is defined.                       -->
<script type="text/x-red" data-template-name="pause">
  <div class="form-row">
     <label for="node-input-topic"><i class="icon-tasks"></i> Pause</label>
    <input type="text" id="node-input-timeout" placeholder="Time">
  </div>
    
  <div class="form-row">
    <label for="node-input-name"><i class="icon-tag"></i> Name</label>
    <input type="text" id="node-input-name" placeholder="Name">
  </div>
</script>

<!-- Next, some simple help text is provided for the node.                   -->
<script type="text/x-red" data-help-name="pause">
    <p>Introduces a pause into a flow</p>
    <p>Default pause is 5 seconds but can be configured</p>
</script>

<!-- Finally, the node type is registered along with all of its properties   -->
<script type="text/javascript">
    RED.nodes.registerType('pause',{
        category: 'function',      // the palette category
        color:"#E6E0F8",
        defaults: {             // defines the editable properties of the node
            name: {value:""},   //  along with default values.
            timeout: {value:"5", required:true, validate:RED.validators.number()}
        },
        inputs:1,                // set the number of inputs - only 0 or 1
        outputs:1,               // set the number of outputs - 0 to n
        icon: "arrow-in.png",    // set the icon (held in public/icons)
        label: function() {      // sets the default label contents
            return this.name||this.topic||"pause";
        },
        labelStyle: function() { // sets the class to apply to the label
            return this.name?"node_label_italic":"";
        }
    });
</script>

The HTML sets up configuration dialog for the node, the help text, the nodes look and behaviour and the default values for any properties the node has.

The configuration dialog for the pause node looks like this:

I’ve checked this new node into my fork of the Node-Red project here, but I’ll be submitting a pull request to get it included in the main project as soon as I get a chance to have a chat with Nick and Dave.

EDIT:
This is now a core node called delay and is also capable of rate limiting messages