Tag Archives: wemo

Making the WeMo Link work with 3rd party bulbs

innr bulb
I’ve been playing with my WeMo Link device again. A couple of weekends ago I made a mistake, I used the Belkin WeMo Android app to turn a light on (rather than ask Alexa to turn it on via Node-RED). While doing this the app suggested I update the firmware on the Link device. I decided to let it do the update, this led to whole host of issues with the device not wanting to connect to the WiFi.

In the end I had to reset the device, and re-pair the bulbs. The re-pairing worked, but the Osram bulbs didn’t show up in the WeMo android app any more. It turns out the app no longer show bulbs which haven’t been “WeMo Certified”. They still show up when you query the API directly and I can control them via my Node-RED nodes but I couldn’t add the bulbs to groups. The WeMo Link supports the ZigBee Light Link protocol just like Osram Lightify, Philips Hue and the Innr lights so there is no reason why all of these things shouldn’t be able to play nicely with each other.

This sent me back to reverse engineering the SOAP API to interact with the WeMo Link.

I’ve had a pretty good go at working out the protocol already, resulting in the wemo-control.js script and the Node-RED WeMo nodes, but this is just basic discovery and control not really “admin” tasks. I decided to break this work out into it’s own script.

There are 2 main tasks that this script will have to do, add new bulbs and create groups.

Adding bulbs

This was actually pretty easy, the API end points were pretty obvious in the list.

  • OpenNetwork – allows bulbs to join the mesh
  • GetEndDevices – when used with the UNPAIRED_LIST filter it shows just the new bulbs.
  • IdentifyDevice – makes a bulb flash so you can work out which bulb is which if you discover new bulbs
  • AddDevice – adds a new bulb to the mesh
  • CloseNetwork – stops bulbs joining the mesh

None of these calls take any complex arguments and all are available either via discovery responses or other simple calls. If you chain them together in the order above you end up with your new bulbs available (at least to my scripts even if not in the WeMo app).

Creating Groups

This one was a little harder, while the API list a CreateGroup endpoint, it says that it takes as a single argument if type ReqCreateGroup which is listed to be a string. Now from experience I can guess that this string is actually a URL encoded XML fragment. There are no hints as to what this XML fragment might look like. This led to a slight diversion down a rabbit hole to set up a raspberry pi as a WiFi AccessPoint bridged on to my local network so I could run tcpdump to make sure I captured all sides of the conversation between my tablet and the WeMo Link. A little bit of formatting and collating in Wireshark and we hit pay dirt:

<?xml version="1.0" encoding="UTF-8"?>
<CreateGroup>
    <GroupID>1489757700</GroupID>
    <GroupName>Lighting Group</GroupName>
    <DeviceIDList>94103EA2B278030F,94103EA2B27803ED</DeviceIDList>
    <GroupCapabilityIDs>10008,10006,30008,30009,3000A</GroupCapabilityIDs>
    <GroupCapabilityValues>255:0,0,,,</GroupCapabilityValues>
</CreateGroup>

The full capture can be found here.

It all looks pretty self explanatory,

  • A unique id for the group (looks like it’s epoch time in seconds)
  • Name of the group (probably should be XML escaped, but we’ll keep them simple)
  • The list of device IDs to include in the group
  • The subset of capabilities that all the devices in the group support
  • Some starting values for those capabilities

Now I had the format of the messages I need to send it’s time to actually write some code. The first pass is up here, it’s a little rough and ready but I’ll try and clean it up a bit later and add a command to rename devices, but it’s Friday afternoon and I’m typing this up in a bar…

To add a bulb you would follow this flow of commands:

node bridge.js open
...
node bridge.js unpaired
...
<?xml version="1.0" encoding="utf-8"?><DeviceLists><DeviceList><DeviceListType>Unpaired</DeviceListType><DeviceInfos><DeviceInfo><DeviceIndex>5</DeviceIndex><DeviceID>00158D0001696252</DeviceID><FriendlyName>OnOff Light</FriendlyName><IconVersion>1</IconVersion><FirmwareVersion>01</FirmwareVersion><CapabilityIDs>10006,10008,30008,30009,3000A</CapabilityIDs><CurrentState>,,,,</CurrentState><Manufacturer>innr</Manufacturer><ModelCode>BY 165</ModelCode><productName>lighting</productName><WeMoCertified>NO</WeMoCertified></DeviceInfo></DeviceInfos></DeviceList></DeviceLists>
node bridge.js indentify <device id>
...
node bridge.js add <device id>
...
node bridge.js close
...

To create a new group:

node bridge.js 1489757700 "Group Name" "00158D0001696252,00158D0001696253" "10006,10008,30008,30009,3000A", "0,255:0,,,"

Alexa Skills with Node-RED

I had a Amazon Echo Dot delivered on release day. I was looking forward to having a voice powered butler around the flat.

Amazon advertise WeMo support, but unfortunately they only support WeMo Sockets and I have a bunch of WeMo bulbs that I’d love to get to work

Alexa supports 2 sorts of skills:

  • Normal Skills
  • Home Skills

Normal Skills are triggered by a key word prefixed by either “Tell” or “Ask”. These are meant for information retrieval type services, e.g. you can say “Alexa, ask Network Rail what time the next train to London leaves Southampton Central”, which would retrieve train time table information. Implementing these sort of skills can be as easy as writing a simple REST HTTP endpoint that can handle the JSON format used by Alexa. You can also use AWS Lambda as an endpoint.

Home Skills are a little trickier, these only support Lambda as the endpoint. This is not so bad as you can write Lambda functions in a bunch of languages like Java, Python and JavaScript. As well as the Lambda function you also need to implement a website to link some sort of account for each user to their Echo device and as part of this you need to implement OAuth2.0 authentication and authorisation which can be a lot more challenging.

First Skill

The first step to create a skill is to define it in the Amazon Developer Console. Pick the Alexa tab and then the Alexa Skill Kit.

Defining a new Alexa Skill
Defining a new Alexa Skill

I called my skill Jarvis and set the keyword that will trigger this skill also to Jarvis.

On the next tab you define the interaction model for your skill. This is where the language processing gets defined. You outline the entities (Slots in Alexa parlance) that the skill will interact with and also the sentence structure to match for each Intent. The Intents are defined in a simple JSON format e.g.

{
  "intents": [
    {
      "intent": "TVInput",
      "slots": [
        {
          "name": "room",
          "type": "LIST_OF_ROOMS"
        },
        {
          "name": "input",
          "type": "LIST_OF_INPUTS"
        }
      ]
    },
    {
      "intent": "Lights",
      "slots": [
        {
          "name": "room",
          "type": "LIST_OF_ROOMS"
        },
        {
          "name": "command",
          "type": "LIST_OF_COMMANDS"
        },
        {
          "name": "level",
          "type": "AMAZON.NUMBER"
        }
      ]
    }
  ]
}

Slots can be custom types that you define on the same page or can make use of some built in types like AMAZON.NUMBER.

Finally on this page you supply the sample sentences prefixed with the Intent name and with the Slots marked.

TVInput set {room} TV to {input}
TVInput use {input} on the {room} TV
TVInput {input} in the {room}
Lights turn {room} lights {command}
Lights {command} {room} lights {level} percent

The next page is used to setup the endpoint that will actually implement this skill. As a first pass I decided to implement a HTTP endpoint as it should be relatively simple for Node-RED. There is a contrib package called node-red-contrib-alexa that supplies a collection of nodes to act as HTTP endpoints for Alexa skills. Unfortunately is has a really small bug that stops it working on Bluemix so I couldn’t use straight away. I’ve submitted a pull request that should fix things, so hopefully I’ll be able to give it a go soon.

The reason I want to run this on Bluemix is because endpoints need to have a SSL certificate and Bluemix has a wild card certificate for everything deployed on the .mybluemix.net domain which makes things a lot easier. (Update: When configuring the SSL section of the skill in the Amazon Developer Console, pick “My development endpoint is a sub-domain of a domain that has a wildcard certificate from a certificate authority” when using Bluemix)

Alexa Skill Flow
Alexa Skill Flow

The key parts of the flow are:

  • The HTTP-IN & HTTP-Response nodes to handle the HTTP Post from Alexa
  • The first switch node filters the type of request, sending IntentRequests for processing and rejecting other sorts
  • The second switch picks which of the 2 Intents defined earlier (TV or Lights)

    {
      "type": "IntentRequest",
      "requestId": "amzn1.echo-api.request.4ebea9aa-3730-4c28-9076-99c7c1555d26",
      "timestamp": "2016-10-24T19:25:06Z",
      "locale": "en-GB",
      "intent": { 
        "name": "TVInput", 
        "slots": { 
          "input": { 
            "name": "input",
            "value": "chromecast"
          }, 
          "room": {
            "name": "room",
            "value": "bedroom"
          }
        }
      }
    }
  • The Lights Function node parses out the relevant slots to get the command and device to control
  • The commands are output to a MQTT out node which published to a shared password secured broker where they are picked up by a second instance of Node-RED running on my home network and sent to the correct WeMo node to carry out the action
  • The final function node formats a response for Alexa to say (“OK”) when the Intent has been carried out

    rep = {};
    rep.version  = '1.0';
    rep.sessionAttributes = {};
    rep.response = {
        outputSpeech: {
            type: "PlainText",
            text: "OK"
        },
        shouldEndSession: true
    }
    
    msg.payload = rep;
    msg.headers = {
        "Content-Type" :"application/json;charset=UTF-8"
    }
    return msg;

It could do with some error handling and the TV Intent needs a similar MQTT output adding.

This works reasonably well, but having to say “Alexa, ask Jarvis to turn the bedroom light on” is a little long winded compared to just “Alexa turn on the bedroom light”, in order to use the shorter form you need to write a Home Skill.

I’ve started work on a Home Skill service and a matching Node-RED Node to go with it. When I’ve got it working I’ll post a follow up with details.

Adding Web Bluetooth to the Lightswitch

I’ve just got Web Bluetooth working properly this weekend to finish off my Physical Web lightswitch.

Web Bluetooth is a draft API to allow webpages to interact directly with Bluetooth LE devices. There is support in the latest Chrome builds on Android and can be turned on by a flag: enable-web-bluetooth (Also coming to Linux in version 50.x).

Some folks have already been playing with this stuff and done things like controlling a BB-8 droid from Chrome.

Chrome Physical Web Notification

I started off following the instructions here which got me started. One of the first things I ran into was that in order to use Web Bluetooth the page needs to be loaded from a trusted source, which basically means localhost or a HTTPS enabled site. I’d already run into this with the Physical Web stuff as Chrome won’t show details of a discovered URL unless it points to a HTTPS site and it even barfs on self signed/private CA certificates. I got round this by using a letsencrypt.org (Which reminds me I really need to change my domain registrar so I can get back to setting up DNSSEC).

Now that I was allowed to actually use the API I had a small problem discovering the BLE device. I had initially thought I would be able to filter local devices based on the Primary Services they possessed, something like this:

navigator.bluetooth.requestDevice({
    filters: [{
        services: ['ba42561b-b1d2-440a-8d04-0cefb43faece']
    }]
})

Web Bluetooth Device Discovery

But after not getting any devices returned I had to reach out on Stackoverflow with the this question. This turned out to be because the beacon was only advertising 1 of it’s 3 Primary Services along with the URL. The answer to my question posted by Jeffrey Yasskin pointed me at using a device name prefix and listing the alternative services the device should provide. I’m going to have a look at the code for the eddystone-beacon node to see if it can be altered to advertise more of the services as well as a URL.

navigator.bluetooth.requestDevice({
    filters: [{
        namePrefix: 'Light'
    }],
    optionalServices: ['ba42561b-b1d2-440a-8d04-0cefb43faece']
})

This now allows the user to select the correct device if there are more than one within range. Once selected the web app switches over from making posts to the REST control endpoints to talking directly to the device via BLE. The device is surfacing 2 characteristics at the moment, one for the toggling on and off and one to set the brightness levels.

All the code is up and Github here.

Next I need to see if there is a way to skip the device selection phase if the user has already paired the page and the device to save on the number of steps required before you can switch the lights on/off. I expect this may not be possible for privacy/security reasons at the moment. Even with the extra step it’s still quicker than waiting for the Offical Belkin WeMo app to load.

Physical Web Lightswitch

Physical Web Logo
As I’ve mentioned before I’ve been having a playing a bunch of WeMo kit and also looking at using Bluetooth LE and Physical Web beacons. I’ve been looking at putting the 2 together to solve a problem I’ve been having round the house. (Also one mentioned by somebody [@mattb I’m told by @knolleary], sorry can’t remember who, at this years ThingMonk)

Belkin ship a mobile phone app to control their products but it has a few drawbacks:
Belkin WeMo App

  1. Launching the app takes ages
  2. Vistors need to know what type of lights you own, then they have to install the right app
  3. You have to give visitors access to you Wifi
  4. Once you’ve granted access to the WiFi, Visitors are granted full control of your lights, including when no longer attached to the same network with no way to revoke access

(Having been reminded by @knolleary -> More of these types of problems discussed in @mattb’s Thingmonk talk at this year)

The Physical Web approach deals with the first 2 of these really nicely, a phone will detect the Eddystone beacon and offer a link to the control web page, so no app needed and no need to identify what type of devices you have, you just need to be close enough to it.

The second 2 problems are a little bit more tricky. Due to mitigation of some privacy problems at the moment to get the best out of a Physical Web URL it needs to be publicly accessible, this is because when a device detects a beacon it tries to access the URL in order to pull some summary/meta data to help with presenting it to the user. The problem with this is it exposes the devices IP address to the guys who deployed the beacon, which allows for the possibility of tracking that user. The workaround is that the Physical Web spec says that the URLs should be accessed by a proxy hence shielding the device from the URL, the problem is these proxies are all on the public internet and can only see public sites. But this does mean since it’s on the public internet you don’t need to give guest net access.

All this gets round problem number 3, but means that control for your living room lights needs to be publicly exposed to the internet. Which brings up nicely to problem 4, if it’s on the public internet how do you control who has access, once somebody has used the URL in the beacon it will be in their internet history and they can comeback and mess with your lights from home again.

You can add authentication to the URL, but a careful balance about how long any signed in session lasts will need to be struck as you don’t want to be flapping around in the dark trying to enter a password to turn the lights on. While this is big scary problem there is a potential solution to all that I’ll touch on at the end of this post.

There is one other problem, URLs broadcast via Eddystone beacons have to be less than 18 bytes long which is pretty short, while there are some encoding tricks for common start and end sections (e.g. ‘http://www.’ & ‘.com’) that reduce these sections to just 1 byte, that still doesn’t leave room for much more. You need to use a URL shortner to do anything major.

Trying things out

While thinking about all this I decided to spin up a little project (on github) to have a play. I took the core code from my Node-RED WeMo node and wrapped it up in a little web app along with the bleno and eddystone-beacon npm modules. I could have done this using Node-RED but I wanted to be able to support control via straight BLE as well.

The code uses discovery to find the configured light bulb or group of bulbs

wemo.start();

if (!wemo.get(deviceID)) {
  wemo.on('discovered', function(d){
    if (d === deviceID) {
      device = wemo.get(d);
      console.log("found light");
    }
  });
} else {
  device = wemo.get(deviceID);
  console.log("found light");
}

It then starts up an express js webserver and creates 2 routes, 1 to toggle on/off and one to set the brightness

app.post('/toggle/:on', function(req, res){
  console.log("toggle " + req.params.on);
  if (req.params.on === 'on') {
    wemo.setStatus(device,'10006',1);
  } else {
    wemo.setStatus(device,'10006',0);
  }
  res.send();
});

app.post('/dim/:range', function(req,res){
  console.log("dim " + req.params.range);
  wemo.setStatus(device,'10006,10008','1,' + req.params.range);
  res.send();
});

It also sets up a directory to load static content out of. Once that is all setup then it sets up the Eddystone beacon

eddystone.advertiseUrl(config.shortURL, {name: config.name});

Enabling Physical Web on You Phone

If you want to have a play yourselves there are a couple of approaches to enable Physical Web discovery on your phone. The first is a mobile app built by the physicalweb.org guys, it’s available for both Android and iOS. When you launch this app it will seek out any Eddystone beacons in range and display a list along with a summary

Recently Google announced that they were rolling Physical web capability into the their Chrome Web browser. At the moment it is only available in the beta release. You can down it on Android here and the this link has instructions for iOS. I have not tried iOS instructions as I don’t have a suitable device.

Once it’s been installed these instructions explain how to enable it.

Now we have a beacon and some way to detect it what does it look like

Discovered beacons

The Physical Web app has detected 2 beacons (actually they are both the same beacon, using both BLE and mDNS to broadcast the URL). Both beacon are on my private network at the moment so the proxy could not load any of the meta data to enrich the listing, it could also not find my private URL shortener http://s.loc. If I click on one of the beacons then it will take me to a page to control the light. At the moment the interface is purely functional.

Light interface

This is working nicely enough for now, but it needs to be made to look a bit nicer and could do with presenting what brightness level the bulb is currently set to.

Direct device communication

I mentioned earlier there was a possible solution for the public network access requirement and authentication. There is a working group developing a specification to allow web pages to interact with local BLE devices. The Web Bluetooth API Specification is not yet finished but an early version is baked into Chrome (you can enable it via these instructions). This is something I intend to play with because it solves the whole public facing site problem and how to stop guests keeping remote access to your lights. It doesn’t matter that you can download the control page if you still need to be physically close to the beacon to connect via BLE to control the lights.

I’ve added 2 BLE GATT characteristics to the beacon (1 for on/off and 1 for dimming) and when I get another couple of free hours I’m going to improve the webpage served up from the beacon to include this support. Once this works I can move the page to my public site and use a public URL shortener which should mean all the meta data will load properly.

All this also means that with the right cache headers the page only needs to be downloaded once and can then loaded directly from the on device cache in the future.

New WeMo Nodes for Node-RED

Based on my previous playing with a set of Belkin WeMo sockets and lightbulbs I decided to have a go at improving support in Node-RED.

I’ve built 2 new nodes, control and event nodes.

WeMo Control Node
WeMo Control Node

The control node accepts the following values in the msg.payload

  • on/off
  • 1/0
  • true/false
  • A JSON object like this
    {
      state: 'on'
      brightness: 255
      color: '255,0,0'
    }
WeMo Event node
WeMo Event node

The input (event) node now uses uPNP events rather than polling status from the devices every 2 seconds. This means you won’t get the “nc” no change messages but you will get events when lights change brightness or colour as well as on/off messages.

Both nodes use a shared config node that uses uPNP discovery to locate WeMo devices on your local network so you don’t have to work out what IP address and port number they are using.

WeMo Device discovery
WeMo Device discovery

Discovery runs once a minute to ensure all devices are found and any change in IP address or port number are quickly picked up. First discovery may take a little while so please allow a little time if you don’t see all the devices you expect listed when you look in the config node.

All the code is up on github here, I’ll push them to npmjs after people have given them a bit more of a test and I’ll have a chat with the Node-RED guys about maybe swapping out the original WeMo node. There is basic backwards compatibility with the original WeMo node, but the nodes work better if after upgrading you use the configuration dialog to pick a discovered device from the list.

Updated WEMO control script

A while ago I wrote a little nodejs script to control WeMo devices. At the time I only had two bulbs that I had in different rooms so it didn’t make sense to support light groups.

Towards the end of 2014 Belkin and Osram announced that Osram’s Lightify bulbs would support WeMo’s bridge controller. In this line was a GU10 bulb and I recently purchased two of these.

Both of these have been fitted into a single light fitting so it made sense to group them together, but this left me with a problem that my script wouldn’t work. After a little bit of playing (and another fruitless back and forth with @WEMOCares on twitter) I managed to get group support working properly. It could do with a bit of cleaning up and refactoring but it works:

WEMO Event notifications

As part of my on going playing with some Belkins WEMO devices I started to have a look at the UPNP Event support.

The UPNP standard as well as including discover and endpoints for control has an event system, this allows devices to publish their status changes to interested parties. The event system uses some extensions to HTTP.

SUBSCRIBE

To subscribe to events from a given device you send a request similar to the following to the eventSubURL given as defined in the setup.xml which is linked to in the UPNP discovery response.

SUBSCRIBE /upnp/event/bridge1 HTTP/1.1
CALLBACK: <http://192.168.1.1:3000/>
NT: upnp:event
TIMEOUT: Second-600
Host: 192.168.1.2:49154

CALLBACK -> is the URL to be notified
TIMEOUT -> how long to send notifications

Gets the following response:

HTTP/1.1 200 OK
DATE: Sun, 11 Jan 2015 18:27:05 GMT
SERVER: Unspecified, UPnP/1.0, Unspecified
CONTENT-LENGTH: 0
X-User-Agent: redsonic
SID: uuid:7206f5ac-1dd2-11b2-80f3-e76de858414e
TIMEOUT: Second-600

SID -> the Subscription ID

The SID is used to identify which notification come from this subscription, it can also be used to renew the subscription before the timeout expires by sending a SUBSCRIBE message like this:

SUBSCRIBE /upnp/event/bridge1 HTTP/1.1
SID: uuid:7206f5ac-1dd2-11b2-80f3-e76de858414e
TIMEOUT: Second-600
Host: 192.168.1.2:49154

NOTIFY

Incoming event notifications get delivered every time the state of the device changes and look like this for the WeMo Socket:

NOTIFY / HTTP/1.1
HOST: 192.168.1.1:3000
CONTENT-TYPE: text/xml; charset="utf-8"
CONTENT-LENGTH: 212
NT: upnp:event
NTS: upnp:propchange
SID: uuid:7206f5ac-1dd2-11b2-80f3-e76de858414e
SEQ: 0

<e:propertyset xmlns:e="urn:schemas-upnp-org:event-1-0">
  <e:property>
    <BinaryState>0</BinaryState>
  </e:property>
</e:propertyset>

The events from the light bulbs is a bit more complex, how parse them is demonstrated in the code.

UNSUBSCRIBE

And when your done you can unsubscribe with the following:

UNSUBSCRIBE / HTTP/1.1
Host: 192.168.1.2:49154
SID: uuid:7206f5ac-1dd2-11b2-80f3-e76de858414e

WeMo

Having worked all these bits out I put the following node js app together to test it all out with the WeMo devices I have:

Next step is to put all this together to build a new WeMo node module and then a improved Node-RED node.

Playing with a set of WEMO light bulbs

I’ve been looking things like the Philips Hue since they shipped, but the price had been putting me off (and the fact that they initially only supported iOS devices and only come in screw fit).

When the Belkin WeMo Light Bulbs came round they were a cheaper option and they come in bayonet fit which meant I wouldn’t need to change any of the light fittings in the flat to play with them.

Belkin have created a mobile app to interact with the bulbs, but having to use your phone to turn the light on when you enter a room is bit of a bind. There is a replacement light switch but it’s only available in the US at the moment, so I was looking for other ways to control things. I was looking to hook the lights up to Node-RED so I could drive them based on a whole list of events.

There is a Node-RED node and a bunch of npm modules for controlling WEMO sockets but none of them currently seam to support the light bulbs. So I started to see if I could work out what was needed to get things up and running. As I was going I tweeted some of my progress which led to this exchange:

I decided to not hold my breath while waiting for a useful response and started out with a couple of the existing npm modules.

WEMO devices are all UPNP devices, which means discovering them is actually pretty easy. It also means the interface to interact with them should be pretty self descriptive. The response to the discovery request is a XML document that includes a number of useful things.

  • IP address
  • Port number
  • Unique Device Number
  • List of Services

The list of services has some interesting looking targets, but after some trial and error the bridgeservice turned out to look like the one I wanted. I ended up getting stuck and I was getting ready to set up a separate WiFi network so I could run Wireshark to see how the mobile app was actually driving the lights.

Some sterling work by Jesus Rafael Carrillo (@jescarri on github) on this issue got me the bit of info I was missing.

Taking this I managed to build a simple hard coded app that would turn the lights on and off or set the dim level and the fade times.

EDIT:

Updated here

The next step is to pick one of the existing WEMO npm modules to add this new capability, then update the Node-RED node to go support the new features.