Provisioning WiFi IoT devices

or

“How do you get the local WiFi details into a consumer device that doesn’t have a keyboard or screen?”

It’s been an on going problem since we started to put WiFi in everything. The current solution seams to be have the device behave like it’s own access point first and then connect a phone or laptop to this network in order to enter the details of the real WiFi network. While this works it’s a clunky solution that relies on a user being capable of disconnecting from their current network and connecting to the new device only network. It also requires the access point mode to be shutdown in order to try the supplied WiFi credentials which causes the phone/laptop to disconnect and normally reconnect to it’s normal network. This means if the user makes a typo then they have to wait for the device to realise the credentials are wrong and at best go back into access point mode or you have to manually reset the device to have it start over. This really isn’t a good solution.

I’ve been trying to come up with a better solution, specifically for my linear clock project. You may ask why a wall clock needs to be connected to the internet, but since it’s Raspberry Pi powered, which has no persistent real time clock it needs a way to set the time in the case of power outages. I also want a way to allow users to change the colours for the hours, mins and seconds and adjust brightness levels.

In order to add WiFi to the Raspberry Pi Zero I’m using the RedBear IoT pHat which as well as providing WiFi it includes Bluetooth 4.0 support. This opens up a new avenue to use for configuration. The idea is to have the device start as a Eddystone beacon broadcasting a URL. The URL will host a page that uses Web Bluetooth to connect to the device and allow the WiFi SSID and password to be entered. Assuming the credentials are correct the IP address can be pushed to the page allowing a link to be presented to a page served by the device to then configure the more day to day options.

RedBear IoT pHat

We had a hackday at the office recently and I had a go at implementing all of this.

Firstly the device comes up as an Eddystone Beacon that broadcasts the URL to the configuration page:

wpid-wp-1473775044897.png

When the page loads is shows a button to search for the devices with bluetooth. This is because the Web Bluetooth spec currently requires a user interaction to start a BLE session.

wpid-wp-1473775152699.png

Chrome then displays a list of matching devices to connect to.

wpid-wp-1473775152702.png

When the user selects a given device the form appears to allow the SSID and password to be entered and submitted.

wpid-wp-1473775152708.png

Once the details are submitted the page will wait until the device has bound to the WiFi network, it then displays a link directly to the device so any further configuration can be carried out.

The code for this is all up on GitHub here if anybody wants to play. The web front end needs a little work and I’m planning to add WiFi AP scanning and pre-population soon.

Now we just need Apple to get their finger out and implement Web Bluetooth

Physical Web FatBeacons

I’ve been playing with Eddystone beacons (A Open BLE beacon technology backed by Google) for a while. Combined with things like WebBluetooth they make for a really nice way to interact with physical object from mobile devices.

Physical Web Logo

Recently there has been a suggested extension to the Physical Web specification to include something called a FatBeacon. A FatBeacon is one that rather than advertising a URL to load a web page from it actually hosts the web page on the device and services it up from the BLE characteristic. This means that it becomes possible to interact with a device when there is no network back haul to load the page.

This may sounds a little strange in the western world where cellular coverage is ubiquitous, but there are places where coverage is still spotty and it also allows for totally self contained solutions. For simple things like my Physical Web Light Switch this would work well.

A request went up on the Physical Web GitHub page asking if anybody wanted to have a go at implementing a software stack to allow people to have a play with the concept so I threw my hat into the ring to have a go.

My first thought was to make use of Sandeep Mistry‘s bleno library and I ran up a simple characteristic to serve up the HTML.

var bleno = require('bleno');
var util = require('util');

var html = "<html>" + 
			"<head>" +
			 "<title>Fat Beacon Demo</title>" +
			 "<meta name='description' content='FatBeacon Demo'/>" +
			"</head>" + 
			"<body>" +
			 "<h1>HelloWorld</h1>" +
                         "This is a test" +
			"</body>" + 
		   "</html>";

function HTMLCharacteristic() {
	bleno.Characteristic.call(this, {
		uuid: 'd1a517f0249946ca9ccc809bc1c966fa',
		properties: ['read'],
		descriptors: [
			new bleno.Descriptor({
				uuid: '2901',
				value: 'HTML'
			})
		]
	});
}

util.inherits(HTMLCharacteristic,bleno.Characteristic);

HTMLCharacteristic.prototype.onReadRequest = function(offset, callback) {
	console.log("read request");
	var buf = new Buffer(html, 'utf8');
	if (offset < buf.length) {
		var slice = buf.slice(offset);
		callback(this.RESULT_SUCCESS, slice);
	} else {
		//problem
	}

}

module.exports = HTMLCharacteristic;

I wrapped this in a simple service but I had missed the need to put the right bits in the advertisement packet, luckily Jacob Rosenthal spotted what needed doing and ported my code over to Don Coleman‘s node-eddystone-beacon project.

After a couple of tweaks I now have it all working (on Linux, iirc there are some limitation on OSx that mean you can’t connect to services with bleno). My code is available on GitHub here and I will be submitting a pull request to Don to get it included in the node-eddystone-beacon project soon.

The beacon shows up like this in the Physical Web android app

FatBeacon in Physical Web App

Which when loaded looks like this:

FatBeacon page

There are some interesting issues around security to be looked at but it’s still an interesting area.

Adding Web Bluetooth to the Lightswitch

I’ve just got Web Bluetooth working properly this weekend to finish off my Physical Web lightswitch.

Web Bluetooth is a draft API to allow webpages to interact directly with Bluetooth LE devices. There is support in the latest Chrome builds on Android and can be turned on by a flag: enable-web-bluetooth (Also coming to Linux in version 50.x).

Some folks have already been playing with this stuff and done things like controlling a BB-8 droid from Chrome.

Chrome Physical Web Notification

I started off following the instructions here which got me started. One of the first things I ran into was that in order to use Web Bluetooth the page needs to be loaded from a trusted source, which basically means localhost or a HTTPS enabled site. I’d already run into this with the Physical Web stuff as Chrome won’t show details of a discovered URL unless it points to a HTTPS site and it even barfs on self signed/private CA certificates. I got round this by using a letsencrypt.org (Which reminds me I really need to change my domain registrar so I can get back to setting up DNSSEC).

Now that I was allowed to actually use the API I had a small problem discovering the BLE device. I had initially thought I would be able to filter local devices based on the Primary Services they possessed, something like this:

navigator.bluetooth.requestDevice({
    filters: [{
        services: ['ba42561b-b1d2-440a-8d04-0cefb43faece']
    }]
})

Web Bluetooth Device Discovery

But after not getting any devices returned I had to reach out on Stackoverflow with the this question. This turned out to be because the beacon was only advertising 1 of it’s 3 Primary Services along with the URL. The answer to my question posted by Jeffrey Yasskin pointed me at using a device name prefix and listing the alternative services the device should provide. I’m going to have a look at the code for the eddystone-beacon node to see if it can be altered to advertise more of the services as well as a URL.

navigator.bluetooth.requestDevice({
    filters: [{
        namePrefix: 'Light'
    }],
    optionalServices: ['ba42561b-b1d2-440a-8d04-0cefb43faece']
})

This now allows the user to select the correct device if there are more than one within range. Once selected the web app switches over from making posts to the REST control endpoints to talking directly to the device via BLE. The device is surfacing 2 characteristics at the moment, one for the toggling on and off and one to set the brightness levels.

All the code is up and Github here.

Next I need to see if there is a way to skip the device selection phase if the user has already paired the page and the device to save on the number of steps required before you can switch the lights on/off. I expect this may not be possible for privacy/security reasons at the moment. Even with the extra step it’s still quicker than waiting for the Offical Belkin WeMo app to load.

Physical Web Lightswitch

Physical Web Logo
As I’ve mentioned before I’ve been having a playing a bunch of WeMo kit and also looking at using Bluetooth LE and Physical Web beacons. I’ve been looking at putting the 2 together to solve a problem I’ve been having round the house. (Also one mentioned by somebody [@mattb I’m told by @knolleary], sorry can’t remember who, at this years ThingMonk)

Belkin ship a mobile phone app to control their products but it has a few drawbacks:
Belkin WeMo App

  1. Launching the app takes ages
  2. Vistors need to know what type of lights you own, then they have to install the right app
  3. You have to give visitors access to you Wifi
  4. Once you’ve granted access to the WiFi, Visitors are granted full control of your lights, including when no longer attached to the same network with no way to revoke access

(Having been reminded by @knolleary -> More of these types of problems discussed in @mattb’s Thingmonk talk at this year)

The Physical Web approach deals with the first 2 of these really nicely, a phone will detect the Eddystone beacon and offer a link to the control web page, so no app needed and no need to identify what type of devices you have, you just need to be close enough to it.

The second 2 problems are a little bit more tricky. Due to mitigation of some privacy problems at the moment to get the best out of a Physical Web URL it needs to be publicly accessible, this is because when a device detects a beacon it tries to access the URL in order to pull some summary/meta data to help with presenting it to the user. The problem with this is it exposes the devices IP address to the guys who deployed the beacon, which allows for the possibility of tracking that user. The workaround is that the Physical Web spec says that the URLs should be accessed by a proxy hence shielding the device from the URL, the problem is these proxies are all on the public internet and can only see public sites. But this does mean since it’s on the public internet you don’t need to give guest net access.

All this gets round problem number 3, but means that control for your living room lights needs to be publicly exposed to the internet. Which brings up nicely to problem 4, if it’s on the public internet how do you control who has access, once somebody has used the URL in the beacon it will be in their internet history and they can comeback and mess with your lights from home again.

You can add authentication to the URL, but a careful balance about how long any signed in session lasts will need to be struck as you don’t want to be flapping around in the dark trying to enter a password to turn the lights on. While this is big scary problem there is a potential solution to all that I’ll touch on at the end of this post.

There is one other problem, URLs broadcast via Eddystone beacons have to be less than 18 bytes long which is pretty short, while there are some encoding tricks for common start and end sections (e.g. ‘http://www.’ & ‘.com’) that reduce these sections to just 1 byte, that still doesn’t leave room for much more. You need to use a URL shortner to do anything major.

Trying things out

While thinking about all this I decided to spin up a little project (on github) to have a play. I took the core code from my Node-RED WeMo node and wrapped it up in a little web app along with the bleno and eddystone-beacon npm modules. I could have done this using Node-RED but I wanted to be able to support control via straight BLE as well.

The code uses discovery to find the configured light bulb or group of bulbs

wemo.start();

if (!wemo.get(deviceID)) {
  wemo.on('discovered', function(d){
    if (d === deviceID) {
      device = wemo.get(d);
      console.log("found light");
    }
  });
} else {
  device = wemo.get(deviceID);
  console.log("found light");
}

It then starts up an express js webserver and creates 2 routes, 1 to toggle on/off and one to set the brightness

app.post('/toggle/:on', function(req, res){
  console.log("toggle " + req.params.on);
  if (req.params.on === 'on') {
    wemo.setStatus(device,'10006',1);
  } else {
    wemo.setStatus(device,'10006',0);
  }
  res.send();
});

app.post('/dim/:range', function(req,res){
  console.log("dim " + req.params.range);
  wemo.setStatus(device,'10006,10008','1,' + req.params.range);
  res.send();
});

It also sets up a directory to load static content out of. Once that is all setup then it sets up the Eddystone beacon

eddystone.advertiseUrl(config.shortURL, {name: config.name});

Enabling Physical Web on You Phone

If you want to have a play yourselves there are a couple of approaches to enable Physical Web discovery on your phone. The first is a mobile app built by the physicalweb.org guys, it’s available for both Android and iOS. When you launch this app it will seek out any Eddystone beacons in range and display a list along with a summary

Recently Google announced that they were rolling Physical web capability into the their Chrome Web browser. At the moment it is only available in the beta release. You can down it on Android here and the this link has instructions for iOS. I have not tried iOS instructions as I don’t have a suitable device.

Once it’s been installed these instructions explain how to enable it.

Now we have a beacon and some way to detect it what does it look like

Discovered beacons

The Physical Web app has detected 2 beacons (actually they are both the same beacon, using both BLE and mDNS to broadcast the URL). Both beacon are on my private network at the moment so the proxy could not load any of the meta data to enrich the listing, it could also not find my private URL shortener http://s.loc. If I click on one of the beacons then it will take me to a page to control the light. At the moment the interface is purely functional.

Light interface

This is working nicely enough for now, but it needs to be made to look a bit nicer and could do with presenting what brightness level the bulb is currently set to.

Direct device communication

I mentioned earlier there was a possible solution for the public network access requirement and authentication. There is a working group developing a specification to allow web pages to interact with local BLE devices. The Web Bluetooth API Specification is not yet finished but an early version is baked into Chrome (you can enable it via these instructions). This is something I intend to play with because it solves the whole public facing site problem and how to stop guests keeping remote access to your lights. It doesn’t matter that you can download the control page if you still need to be physically close to the beacon to connect via BLE to control the lights.

I’ve added 2 BLE GATT characteristics to the beacon (1 for on/off and 1 for dimming) and when I get another couple of free hours I’m going to improve the webpage served up from the beacon to include this support. Once this works I can move the page to my public site and use a public URL shortener which should mean all the meta data will load properly.

All this also means that with the right cache headers the page only needs to be downloaded once and can then loaded directly from the on device cache in the future.

Flic.io Linux library

As I mentioned I recently got my hands on a set of 4 flic.io buttons. I pretty immediately paired one of them with my phone and started playing. It soon became obvious that while fun, the use cases for a button paired to a phone where limited to a single user environment and not what I had in mind.

What was needed was a way to hook the flic buttons up to something like a Raspberry Pi and Node-RED. While I was waiting for the buttons to arrive I was poking round the messages posted to the indiegogo one of the guys from Shortcut Labs mentioned that such a library was in the works. I reached out to their developer contact point asking about getting access to the library to build a Node-RED node round, I said I was happy to help test any code they had. Just before Christmas I managed got hold of a early beta release to have a play with.

From that I was able to spin up a npm module and a Node-RED node.

The Node-RED node will currently listen for any buttons that are paired with the computer and publish a message as to if it was a single, double or long click

Flic.io Node-RED node

I said I would sit on these nodes until the library shipped, but it appeared on github yesterday so hence this post. The build includes binaries for Raspberry Pi, i386 and x86_64 and needs the very latest Bluez packages (5.36+).

Both my nodes need a little bit of cleaning up and a decent README writing, once that is done I’ll push them to npm.

UPDATE:
Both nodes are now on npmjs.org:
https://www.npmjs.com/package/node-flic-buttons
https://www.npmjs.com/package/node-red-contrib-flic-buttons

Physical Web Node-RED nodes

Been playing playing with some NodeJS to build Eddystone beacons after watching this:

And having a poke round here

I had earlier been looking at BLE and NodeJS to build BLE devices that can have their state updated via MQTT

Physical web nodes

I’ve written 2 new nodes, the first will listen for any Eddystone beacons in the area and publish the details about the beacon.

The second creates a Eddystone beacon and broadcasts a given URL, it can also update the URL with the msg.payload value.

The code is here* and it can be installed from npmjs with:

npm install node-red-node-physical-web

* this will be moved to the node-red-nodes repo shortly

Flic.io button finally arrived

Along with a lot of other people I’ve been waiting for these to drop through my door for most of the year.

A flic button

I even started to look at building something similar (if a fair bit bigger) using a Raspberry Pi and the noble Node.JS module.

Flic.io buttons are small silicon rubber buttons that can be used to trigger up to three different actions based on a single click, double click and a push and hold. They connect to your phone via BLE. The app comes with support for a whole bunch of actions such as WeMO, Philips Hue, Android actions such as taking photos and IFTTT for a bunch of extra web actions.

I’m starting to look at what it will take to build a Node-RED node for these as I want to set them up to control my WeMO lights at home and a bunch of other stuff. There is talk of a C library for use on the Raspberry Pi as well as the iOS/Android SDKs which I should be able to wrap as a Node.js module if I can get hold of it. Otherwise I’ll have to get down and dirty with reverse engineering the GATT profile.

flic.io box

flic.io open box

4 flic.io in the box

Building Bluetooth LE devices

While waiting for my flic.io buttons to turn up I’ve been playing with building my own Bluetooth Low Energy devices.

Since I already had a couple of sensors hooked up to publish their values via MQTT I thought I would try and build a bridge between MQTT and BLE.

I’m using a Raspberry Pi, a Bluetooth 4.0 USB dongle and a NodeJS npm module called bleno.

It turned out to be petty easy, first a short file to set up the BLE service and connect to MQTT:

var util = require('util');
var bleno = require('bleno');
var mqtt = require('mqtt');

var BlenoPrimarySerivce = bleno.PrimaryService;

var TopicCharacteristic = require('./topicCharacteristic');

var config = require("./config");

var client = mqtt.connect(config.broker);

var topicCharacteristic = new TopicCharacteristic(config);

client.on('connect', function(){
  client.subscribe(config.topic);
});

client.on('message',function(topic, message){
  topicCharacteristic.update(message);
});

bleno.on('stateChange', function(state){
  if (state === 'poweredOn') {
    bleno.startAdvertising('mqtt', ['ba42561bb1d2440a8d040cefb43faece']);
  } else {
    bleno.stopAdvertising();
  }
});

bleno.on('advertisingStart', function(error){
  if(!error) {
  	bleno.setServices([
  	  new BlenoPrimarySerivce({
  	  	uuid: 'ba42561bb1d2440a8d040cefb43faece',
  	  	characteristics: [
  	  	  topicCharacteristic
  	  	]
  	  })
  	]);
  }
});

And then something to add the characteristic for the topic:

var util = require('util');
var bleno = require('bleno');

function TopicCharacteristic(config, client) {
	bleno.Characteristic.call(this, {
		uuid: '6bcb06e2747542a9a62a54a1f3ce11e6',
		properties: ['read', 'write', 'notify'],
		descriptors: [
			new bleno.Descriptor({
				uuid: '2901',
				value: 'Topic: ' + config.topic
			})
		]
	});

	this._value = new Buffer(0);
	this._updateValueCallback = null;
	this._client = client;
	this._topic = config.topic;
}

util.inherits(TopicCharacteristic, bleno.Characteristic);

TopicCharacteristic.prototype.onWriteRequest = function(data, offset, withoutResponse, callback) {
	this._value = data;
	client.publish(this._topic, data);
	callback(this.RESULT_SUCCESS);
}


TopicCharacteristic.prototype.onReadRequest = function(offset, callback) {
	callback(this.RESULT_SUCCESS, this._value);
}

TopicCharacteristic.prototype.onSubscribe = function(maxValueSize, updateValueCallback) {
	this._updateValueCallback = updateValueCallback;
}

TopicCharacteristic.prototype.onUnsubscribe = function () {
	this._updateValueCallback = null;
}

TopicCharacteristic.prototype.update = function(value) {
	this._value = value;
	if (this._updateValueCallback) {
		this._updateValueCallback(this._value);
	}
}

module.exports = TopicCharacteristic;

I’ve used 2 randomly generated UUIDs, one for the service and one for the characteristic.

The code should allow you to read the last value published, publish a new value and subscribe to notifications when new values arrive.

I pulled together a quick Android app to subscribe to the notifications and update when a new value is published and it seams to be working well.

The code is all up on Github and on npmjs so feel free to have a play.

You can test it with the Bluez gatttoool.

[hardillb@bagend ~]$ gatttool -b 00:1A:7D:DA:71:15 -I
[00:1A:7D:DA:71:15][LE]> connect
Attempting to connect to 00:1A:7D:DA:71:15
Connection successful
[00:1A:7D:DA:71:15][LE]> primary
attr handle: 0x0001, end grp handle: 0x0005 uuid: 00001800-0000-1000-8000-00805f9b34fb
attr handle: 0x0006, end grp handle: 0x0009 uuid: 00001801-0000-1000-8000-00805f9b34fb
attr handle: 0x000a, end grp handle: 0x000e uuid: ba42561b-b1d2-440a-8d04-0cefb43faece
[00:1A:7D:DA:71:15][LE]> characteristics 
handle: 0x0002, char properties: 0x02, char value handle: 0x0003, uuid: 00002a00-0000-1000-8000-00805f9b34fb
handle: 0x0004, char properties: 0x02, char value handle: 0x0005, uuid: 00002a01-0000-1000-8000-00805f9b34fb
handle: 0x0007, char properties: 0x20, char value handle: 0x0008, uuid: 00002a05-0000-1000-8000-00805f9b34fb
handle: 0x000b, char properties: 0x1a, char value handle: 0x000c, uuid: 6bcb06e2-7475-42a9-a62a-54a1f3ce11e6
[00:1A:7D:DA:71:15][LE]> char-write-req 0x000d 0300
Characteristic value was written successfully
Notification handle = 0x000c value: 42 61 72 
Notification handle = 0x000c value: 48 65 6c 6c 6f 57 6f 72 6c 64 
Notification handle = 0x000c value: 54 65 73 74 69 6e 67 20 31 32 33 
[00:1A:7D:DA:71:15][LE]> 
[00:1A:7D:DA:71:15][LE]> quit

The lines that start Notification handle contain the bytes published, in this case

  • Bar
  • HelloWorld
  • Testing 123

Node-Red – Ti SensorTag Node

Last weekend I spent some time working on yet another Node-Red node. This one is an input node that reads the data published by a small sensor platorm from Ti.

The Ti SensorTag is a Bluetooth 4.0 LE platform designed to be a test source for building new BLE and is very accessible at only $25 dollars especially with the following list of sensors on board:

  • Ambient Temperature
  • IR remote Temperature
  • Air Pressure
  • Humidity
  • Accelerometer
  • Magnetometer
  • Gyroscope
  • 2 Push Buttons

Ti have also set up a open wiki to allow people to document they experiments with the device.

You can find the node in the new node-red-nodes repo on github here it relies on a slightly updated version of Sandeep Mistry‘s node-sensortag, the read me explains how to install my update (until I get round to submitting the pull request) but here is the command to run in the root of your Node-Red directory:

npm install sensortag@https://api.github.com/repos/hardillb/node-sensortag/tarball

(you may need to install the libbluetooth-dev package for Debian/Ubuntu based distros and bluez-libs-devel on Redhat/Fedora first)

UPDATE:
Sandeep has now merged my changes so the sensortag node can now be installed normally with npm with:

npm install sensortag

Once installed you need to run Node-RED as root as this is the only way to get access to the BLE functions, then you can add the node to the canvas and configure which sensors are pushed as events

Please feel free to have a play and let me know what you think