Category Archives: Projects

DIY IoT button

I’ve been looking for a project for a bunch of ESP-8266 ESP-01 boards I’ve had kicking around for a while.

The following describes a simple button that when pushed publishes a MQTT message that I can subscribe to with Node-RED to control different tasks.

It’s been done many times before, but I wanted to have a go at building my own IoT button.

Software

The code is pretty simple:

  • The MQTT PubSubClient (Thank’s Nick)
  • Some hard coded WiFi and MQTT Broker details
  • The setup function which connects to the network
  • The reconnect function that connects to the MQTT broker and publishes the message
  • The loop function which flashes the LED to show success then go into deep sleep

In order to get the best battery life you want the ESP8266 to be in deep sleep mode for as much as possible, so for this reason the loop function sends the message to signify the button has been pushed then indefinitely enters the deepest sleep state possible. This means the loop function will only run once on start up.

#include <ESP8266WiFi.h>
#include <PubSubClient.h>

#define ESP8266_LED 1

const char* ssid = "WifiName";
const char* passwd = "GoodPassword";
const char* broker = "192.168.1.114";

WiFiClient espClient;
PubSubClient client(espClient);

void setup() {

  Serial.begin(115200);
  delay(10);
  
  pinMode(ESP8266_LED, OUTPUT);

  WiFi.hostname("Button1");
  WiFi.begin(ssid, passwd);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");  
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  client.setServer(broker, 1883);
  reconnect();
}

void reconnect() {
  while(!client.connected()) {
    if (client.connect("button1")){
      client.publish("button1", "press");
    } else {
      delay(5000);
    }
  }
}

void loop() {

  if (client.connected()) {
    reconnect();
  }
  client.loop();
  
  // put your main code here, to run repeatedly:
  digitalWrite(ESP8266_LED, HIGH);
  delay(1000);
  digitalWrite(ESP8266_LED, LOW);
  delay(1000);
  ESP.deepSleep(0);
}

Hardware

By using a momentary push button wake the ESP-01 I bridged reset pin to ground the chip resets each time it’s pushed, this wakes it from the deep sleep state and runs the all the code, then drops back into the sleep state.

Button diagram

  • The green line is the Chip enable
  • The blue line is the the links reset to ground via the push button

Now I had the basic circuit and code working I needed to pick a power supply. The ESP-01 needs a 3.3v supply and most people seem to opt for using a small LiPo cell. A single cell has a nominal fully charged voltage of 3.7v which is probably close enough to use directly with the ESP-01, but the problem is you normally need to add a circuit to cut them out before the voltage gets too low as they discharge to prevent permanently damage them. You would normally add a charging circuit to allow recharging from a USB source.

This wasn’t what I was looking for, I wanted to use off the shelf batteries so I went looking for a solution using AAA batteries. The board will run directly from a fresh set of 2 AAA batteries, but the voltage can quickly drop too low. To help with this I found a part from Pololu that would take an input between 0.5v and 5v and generate a constant 3.3v. This meant that even as the batteries discharged I should be able to continue to run the device.

At first things didn’t look to work, because converter was not supplying enough current for the ESP-01 at start up, to get round this I added a 100uF capacitor across the outputs of the regulator. I don’t really know how properly to size this capacitor so I basically made a guess.

The final step was to use the soldering iron to remove the red power LED from the board as this was consuming way more power than the rest of the system.

Prototype

Next Steps

  • Make the MQTT topic based on the unique id of the ESP-01 so it isn’t hard coded
  • Look at adding Access Point mode to allow configuration of the WiFi details if the device can not connect to the existing configuration
  • Design a circuit board to hold the voltage converter, capacitor, button and the ESP-01
  • Create a case to hold it all
  • Work out just how long a set of batteries will last

SIP to SMS bridging

I’ve recently updated my local Asterisk PBX. The main reason for the update was that processing the logs in order to set up the firewall rules to block the folk that hammer on it all day long trying to make long distance calls or run up big bills on premium rate numbers was getting too much for the original Mk i Raspberry Pi B (it now runs on a Pi 3 b+ which more up to the task).

As part of the set up I have 3G USB dongle that also supports voice calls using the chan_dongle plugin. This plugin also supports sending and receiving SMS messages (this is different from the MMS/SMS gateway I also run on a separate pi) and they are handled by the dial plan.

My first pass at the dial plan just publishes the incoming message to a MQTT topic and it is then processed by Node-RED, which emails a copy to me as well as logging it to a file.

[dongle-incoming]
exten => sms,1,Verbose(1,Incoming SMS from ${CALLERID(num)} ${BASE64_DECODE(${SMS_BASE64})})
exten => sms,n,AGI(/opt/asterisk/agi-mqtt/mqtt,/opt/asterisk/agi-mqtt/mqtt.cfg,foo/sms,${BASE64_DECODE(${SMS_BASE64})})
exten => sms,n,Hangup()

This works OK for incoming messages but sending them is a bit harder, as the only way to send them from outside the dialplan (from within the dialpan you can use DongleSendSMS) is to use the asterisk CLI tool which is a bit clunky.

What I was really looking for was a way to send/receive SMS messages like you do with a mobile phone. To make calls I normally use Linphone on my tablet and this includes support for SIP messaging. This lets you send messages between SIP clients and you can get Asterisk to consume these.

You can send SIP messages with the MessageSend asterisk dailplan function

The following is a basic echo bot:

[messaging]
exten => _.,1,Answer()
exten => _.,n,Verbose(1,Text ${MESSAGE(body)})
exten => _.,n,Verbose(1,Text from ${MESSAGE(from)})
exten => _.,n,Verbose(1,Text to ${MESSAGE(to)})
exten => _.,n,Set(FROM=${MESSAGE(from)})
exten => _.,n,Set(TO=${REPLACE(FROM,<'>,)})
exten => _.,n,MessageSend(pj${TO},${CUT(MESSAGE(to),:,2)})
exten => _.,n,Hangup()

Because I’m using the PJSIP module rather than the legacy SIP module I need to prefix the outbound address with pjsip: rather than sip:. This also matches any target extension which will be useful a little later.

To enable a specific context for SIP messages you need to add message_context to the PJSIP endpoint for the SIP user:

[tablet]
type = endpoint
context = internal
message_context = messaging
...

Now if we put the 2 bits together we get a dialplan that looks like this:

[dongle-incomming]
exten => sms,1,Verbose(1,Incoming SMS from ${CALLERID(num)} ${BASE64_DECODE(${SMS_BASE64})})
exten => sms,n,AGI(/opt/asterisk/agi-mqtt/mqtt,/opt/asterisk/agi-mqtt/mqtt.cfg,foo/sms,${BASE64_DECODE(${SMS_BASE64})})
exten => sms,n,Set(MESSAGE(body))=${BASE64_DECODE(${SMS_BASE64})})
exten => sms,n,MessageSend(pjsip:nexus7,${CALLERID(num)})
exten => sms,n,Hangup()

[messaging]
exten => _.,1,Answer()
exten => _.,n,Verbose(1,Text ${MESSAGE(body)})
exten => _.,n,Verbose(1,Text from ${MESSAGE(from)})
exten => _.,n,Verbose(1,Text to ${MESSAGE(to)})
exten => _.,n,Set(FROM=${MESSAGE(from)})
exten => _.,n,Set(TO=${REPLACE(FROM,<'>,)})
exten => _.,n,DongleSendSMS(dongle0,${EXTEN},${MESSAGE(body)},1440,no)
exten => _.,n,Hangup()

The first part handles the incoming SMS messages delivered to the dongle and passed to the sms extension in the dongle-incomming context. This logs the message to the console and via MQTT then fires it off to my tablet as a SIP message. The second is the context for the incoming SIP messages from the tablet, this will accept messages to any extension, logs the message, who it’s to/from then sends it to the number in the extension via the dongle.

Using SIP Client to send SMS

DNS-Over-HTTPS

I saw the recent announcements from Mozilla, Cloudflare and Google about running a trial to try and make DNS name resolution more secure.

The basic problem is that most users get their DNS server set via DHCP which is controlled by who ever runs the network (at home this tends to be their ISP, but when using public wifi this could be anybody). The first approach to help with this was Google’s 8.8.8.8 public DNS service (followed by the IBM’s 9.9.9.9 and Cloudflares 1.1.1.1). This helps if people are technically literate enough know how to change their OS’s DNS settings and fix them to one of these providers. Also DNS is UDP based protocol which makes it particularly easy for a bad actor on the network to spoof responses.

The approach the 3 companies are taking is to run DNS over an existing secure protocol, in this case HTTPS. From Firefox version 60 (currently in beta) it is possible to set it up to do name host name resolution via DNS-Over-HTTPS.

There are currently 2 competing specifications for how to actually implement DNS-Over-HTTPS.

DNS Wireformat

This uses exactly the same data structure as existing DNS. Requests can be made via a HTTP GET or POST. For a POST the body is the binary request and the Content-Type is set to application/dns-udpwireformat.

For GET requests the payload is BASE64 encoded and passed as the dns query parameter.

In both cases the response is the same binary payload as would be made by a normal DNS server.

This approach is currently covered by this draft RFC

JSON

For this approach the request are made as a HTTP GET request with the hostname (or IP address) being passed as the name and the query type being passed as the type query parameters.

A response looks like this:

{
    "Status": 0,
    "RA": true,
    "RD": true,
    "TC": false,
    "AD": false,
    "CD": true,
    "Additional": [],
    "Answer": [
        {
            "TTL": 86400,
            "data": "93.184.216.34",
            "name": "example.com",
            "type": 1
        }
    ],
    "Question": [
        {
            "name": "example.com",
            "type": 1
        }
    ]
}

With a Content-Type of application/dns-json

You can find the spec for this scheme from Google here and Cloudflare here.

Both of these schemes have been implemented by both Google and Cloudflare and either can be used with Firefox 60+.

Privacy Fears

There has already been a bit of a backlash against this idea, mainly around privacy fears. The idea of Google/CloudFlare being able to collect information about all the hosts your browser resolves scared some people. Mozilla has an agreement in place with CloudFlare about data retention for the initial trial.

Given these fears I wondered if people might still want to play with DNS-Over-HTTPS but not want to share data with Google/Cloudflare. With this in mind I thought I’d try and see how easy it would be to implement a DNS-Over-HTTPS server. Also people may want to try this out on closed networks (for things like performance testing or security testing).

It turned out not to be too difficult, I started with a simple ExpressJS based HTTP server and then started to add DNS support. Initially I tried a couple of different DNS NodeJS nodes to get all the require details and in the end settled on dns-packet and actually sending my own UDP packets to the DNS server.

I’ve put my code up on github here if anybody wants a play. The README.md should include details about how to set up Firefox to use an instance.

Adding audio to the CCTV of your burgular

In a full on throw back to the 2nd ever post to this blog, back in February 2010, I’ve recently been updating the system that sends me a video when there is movement in my flat via MMS and email.

I thought I’d try and add audio to the video that gets sent. A quick Google turned up two options, one was to use the sox command and it’s silence option, the second uses the on_event_start triggers in motion as a way to record the audio at the same time as capturing the movement video. I went with the second option and tweaked it a bit to make it pick the right input for my system and to direct encode the audio to MP3 rather than WAV to save space.

on_event_start arecord --process-id-file /var/motion/arecord.pid -D sysdefault:CARD=AF -f S16_LE -r 22050 - | lame  -m m - /var/motion/%Y%m%d_%H%M%S.mp3

The other useful addition was the –process-id-file /var/motion/arecord.pid which writes the process id to a file so I can just use this to stop the recording rather than having to use grep and awk to find the process in the ps output.

on_event_end kill `cat /var/motion/arecord.pid`

Now it’s just a case of combining the video from motion with the audio. I can do this with ffmpeg when I re-encode the video into the 3gp container to make it suitable for sending via a MMS message.

ffmpeg -i movement.avi -i movement.mp3 -map 0:v -map 1:a -c:a aac -strict -2 -s qcif -c:v h263 -y /var/www/html/cam1/intruder.3gp

This seams to work but the output file is a little large. The default audio encoding seams to be at 160k bitrate, I wound it down to 32k and the file size got a lot better.

ffmpeg -i movement.avi -i movement.mp3 -map 0:v -map 1:a -c:a aac -b:a 32k -strict -2 -s qcif -c:v h263 -y /var/www/html/cam1/intruder.3gp

I’d like to try the AMR audio codec but I can’t get ffmpeg to encode with it at the moment, so I’m just going to email the mp3 of the audio along with the high res AVI version of the video and just send the low res 3GP version via MMS.

Multipart HTTP Post requests with Volley on Android

It’s been a little while since I’ve done any really serious Android development, but a couple of projects have brought me back to it.

Early on in one of those projects I had to make some HTTP requests, my first thought was to make use of the Apache HTTP Client classes as I had done many times before on Android. Which is why I was a little surprised when the usual ctrl-space didn’t auto complete any of the expected class names.

It turns out the classes were removed in Android 6.0 and the notice suggests using the HttpURLConnection class. A little bit more digging turned up a wrapper for this called Volley.

Volley is a wrapper round the HttpURLConnection class to provides a neat asynchronous interface that does IO in the background and then delivers results to the Main thread so UI updates can be done with out further faffing around switching threads. There is also a nice set of tutorials on the Android Developers pages.

The first few requests all worked fine, but there was one which was a little bit more tricky. The HTTP endpoint in question accepts a multipart-form payload. A bit of googling/searching on Stackoverflow turned up a number of approaches to this and best seamed to be documented in this gist

This was close to what I wanted but not quite what I needed so I have taken some of the core concepts and built my own MultipathRequest object.

...
MultipartRequest request = new MultipartRequest(url, headers, 
    new Response.Listener<NetworkResponse>() {
        @Override
        public void onResponse(NetworkResponse response) {
        ...
        }
    },
    new Response.ErrorListener() {
        @Override
        public void onErrorResponse(VolleyError error) {
        ...
        }
    });
    
request.addPart(new FormPart(fieldName,value));
request.addPart(new FilePart(fileFieldName, mimeType, fileName, data);

requestQueue.add(request);
...

I’ve stuck the code up on github here. You can include it in your Android Project by adding the following to the build.gradle in the root of the project:

allprojects {
  repositories {
    ...
    maven { url 'https://jitpack.io' }
  }
}

And then this to the dependencies section of the modules build.gradle:

dependencies {
  compile 'com.github.hardillb:MultiPartVolley:0.0.3'
}

Fist pass TRÅDFRI MQTT Bridge

I’ve been working on integrating the new IKEA TRÅDFRI Lights into my Home Automation system. I’d really like a native NodeJS system so I can plug it directly into Node-RED, but I’ve not found a working CoAP over DTLS setup just yet.

So in the mean time I’ve been working on a very basic MQTT to CoAP client bridge in Java using the Eclipse Californium library.

It still needs some work, but here is the first pass:

package uk.me.hardill.coap2mqtt;

import java.net.InetSocketAddress;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.HashMap;
import java.util.logging.Level;

import org.eclipse.californium.core.CaliforniumLogger;
import org.eclipse.californium.core.CoapClient;
import org.eclipse.californium.core.CoapHandler;
import org.eclipse.californium.core.CoapResponse;
import org.eclipse.californium.core.Utils;
import org.eclipse.californium.core.coap.MediaTypeRegistry;
import org.eclipse.californium.core.network.CoapEndpoint;
import org.eclipse.californium.core.network.config.NetworkConfig;
import org.eclipse.californium.scandium.DTLSConnector;
import org.eclipse.californium.scandium.ScandiumLogger;
import org.eclipse.californium.scandium.config.DtlsConnectorConfig;
import org.eclipse.californium.scandium.dtls.pskstore.StaticPskStore;
import org.eclipse.paho.client.mqttv3.IMqttDeliveryToken;
import org.eclipse.paho.client.mqttv3.MqttCallback;
import org.eclipse.paho.client.mqttv3.MqttClient;
import org.eclipse.paho.client.mqttv3.MqttException;
import org.eclipse.paho.client.mqttv3.MqttMessage;
import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence;
import org.json.JSONArray;
import org.json.JSONObject;

/**
 * @author hardillb
 *
 */
public class Main {
  
  static {
    CaliforniumLogger.disableLogging();
    ScandiumLogger.disable();
//    ScandiumLogger.initialize();
//    ScandiumLogger.setLevel(Level.FINE);
  }
  
  private DTLSConnector dtlsConnector;
  private MqttClient mqttClient;
  private CoapEndpoint endPoint;
  
  private String ip;
  
  private HashMap<String, Integer> name2id = new HashMap<>();
  
  Main(String psk, String ip) {
    this.ip = ip;
    DtlsConnectorConfig.Builder builder = new DtlsConnectorConfig.Builder(new InetSocketAddress(0));
    builder.setPskStore(new StaticPskStore("", psk.getBytes()));
    dtlsConnector = new DTLSConnector(builder.build());
    endPoint = new CoapEndpoint(dtlsConnector, NetworkConfig.getStandard());
    
    MemoryPersistence persistence = new MemoryPersistence();
    try {
      mqttClient = new MqttClient("tcp://localhost", MqttClient.generateClientId(), persistence);
      mqttClient.connect();
      mqttClient.setCallback(new MqttCallback() {
        
        @Override
        public void messageArrived(String topic, MqttMessage message) throws Exception {
          // TODO Auto-generated method stub
          System.out.println(topic + " " + message.toString());
          String parts[] = topic.split("/");
          int id = name2id.get(parts[1]);
          System.out.println(id);
          String command = parts[3];
          System.out.println(command);
          try{
            JSONObject json = new JSONObject("{\"9001\":\"Living Room Light\",\"9020\":1491515804,\"9002\":1491158817,\"9003\":65537,\"9054\":0,\"5750\":2,\"3\":{\"0\":\"IKEA of Sweden\",\"1\":\"TRADFRI bulb E27 opal 1000lm\",\"2\":\"\",\"3\":\"1.1.1.0-5.7.2.0\",\"6\":1},\"9019\":1,\"3311\":[{\"5850\":1,\"5851\":10,\"9003\":0}]}");
            JSONObject settings = json.getJSONArray("3311").getJSONObject(0);
            if (command.equals("dim")) {
              settings.put("5851", Integer.parseInt(message.toString()));
            } else if (command.equals("on")) {
              if (message.toString().equals("0")) {
                settings.put("5850", 0);
                settings.put("5851", 0);
              } else {
                settings.put("5850", 1);
                settings.put("5851", 128);
              }
            }
            String payload = json.toString();
            Main.this.set("coaps://" + ip + "//15001/" + id, payload);
          } catch (Exception e) {
            e.printStackTrace();
          }
        }
        
        @Override
        public void deliveryComplete(IMqttDeliveryToken token) {
          // TODO Auto-generated method stub
        }
        
        @Override
        public void connectionLost(Throwable cause) {
          // TODO Auto-generated method stub
        }
      });
      mqttClient.subscribe("TRÅDFRI/+/control/+");
    } catch (MqttException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
  }
  
  private void discover() {
    try {
      URI uri = new URI("coaps://" + ip + "//15001");
      CoapClient client = new CoapClient(uri);
      client.setEndpoint(endPoint);
      CoapResponse response = client.get();
      JSONArray array = new JSONArray(response.getResponseText());
      for (int i=0; i<array.length(); i++) {
        String devUri = "coaps://"+ ip + "//15001/" + array.getInt(i);
        this.watch(devUri);
      }
      client.shutdown();
    } catch (URISyntaxException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
  }
  
  private void set(String uriString, String payload) {
    System.out.println("payload\n" + payload);
    try {
      URI uri = new URI(uriString);
      CoapClient client = new CoapClient(uri);
      client.setEndpoint(endPoint);
      CoapResponse response = client.put(payload, MediaTypeRegistry.TEXT_PLAIN);
      if (response.isSuccess()) {
        System.out.println("Yay");
      } else {
        System.out.println("Boo");
      }
      
      client.shutdown();
      
    } catch (URISyntaxException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
  }
  
  private void watch(String uriString) {
    
    try {
      URI uri = new URI(uriString);
      
      CoapClient client = new CoapClient(uri);
      client.setEndpoint(endPoint);
      CoapHandler handler = new CoapHandler() {
        
        @Override
        public void onLoad(CoapResponse response) {
          System.out.println(response.getResponseText());
          JSONObject json = new JSONObject(response.getResponseText());
          if (json.has("3311")){
            MqttMessage message = new MqttMessage();
            int state = json.getJSONArray("3311").getJSONObject(0).getInt("5850");
            message.setPayload(Integer.toString(state).getBytes());
            message.setRetained(true);
            String topic = "TRÅDFRI/" + json.getString("9001") + "/state/on";
            String topic2 = "TRÅDFRI/" + json.getString("9001") + "/state/dim";
            name2id.put(json.getString("9001"), json.getInt("9003"));
            MqttMessage message2 = new MqttMessage();
            int dim = json.getJSONArray("3311").getJSONObject(0).getInt("5851");
            message2.setPayload(Integer.toString(dim).getBytes());
            message2.setRetained(true);
            try {
              mqttClient.publish(topic, message);
              mqttClient.publish(topic2, message2);
            } catch (MqttException e) {
              // TODO Auto-generated catch block
              e.printStackTrace();
            }
          } else {
            System.out.println("not bulb");
          }
        }
        
        @Override
        public void onError() {
          // TODO Auto-generated method stub
          
        }
      };
      client.observe(handler);
    } catch (URISyntaxException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
    
  }

  /**
   * @param args
   */
  public static void main(String[] args) throws InterruptedException {
    String psk = args[0];
    String ip = args[1];
    Main m = new Main(psk, ip);
    
    m.discover();
  }

}

I’ve tagged the code onto the gist as well for now, but I’ll check the whole thing in as a separate project soon.

EDIT: now with it’s own Github repo here

node-red-contrib-alexa-home-skill

It’s finally ready. I’ve been working on a Node-RED node to act on Amazon Alexa Home Skill directives since November last year. The skill was approved some time very early this morning and now should be available in the UK, US and Germany.

I’ll be mailing all the folks that have already signed up some time later today to let them know they can finally start using the skill, but for the rest of you here is a brief introduction (full details in earlier post).

Alexa Home Skill’s allow you to say the much more natural “Alexa, turn on the kitchen light” rather than “Alexa, ask Jeeves to turn on the kitchen light”, where “Jeeves” is the name of skill you have to remember. Some of the basic commands are:

  • Turn On/Off
  • Dim/Brighten
  • Set/Get Temperature
  • Lock/Unlock

With this node and service you can wire those commands to nearly anything you can control via Node-RED.

Node-RED - Alexa Smart Home Skill

You can install the node with the following commands:

cd ~/.node-red
npm install node-red-contrib-alexa-home-skill

Or via the Manage Palette option in the Node-RED editor.

If you have already installed this module please make sure you update to the latest version (0.1.13) to get the best support for all the voice commands.

There are detailed instructions on how to set everything up here.

Here is an example flow using the node. This turns a light on then automatically turns it off after 5mins. It uses the switch node to detect if it’s a request to turn the light on or off. When following the On branch it uses a trigger node to first send a payload of true then, 5 minutes later it sends false to the WeMo node.

On then Auto Off flow

This sort of flow would be great for a set of outside lights or maybe an electric heater. I also have some updates to the node-red-nodes-wemo package to make dimming/brightening by specific amounts easier, I’ll try and get them out by the weekend.

EDIT:
If you have problems with this node please do not post comments here, it really isn’t the best place to work issues. Open a issue on github here then it can be properly tracked.

Updated Pi Zero Gadgets

Following on from my last post I’ve continued to work on improving my instructions for a USB connectable gadget based on a Raspberry Pi Zero.

Firstly I’ve got a slight improvement to the dnsmasq config I mentioned last time. This removes the dnsmasq.leases file which can cause issues if you plug the Zero into multiple computers. This can be a problem because while I had managed to fix the mac address for Host computer end of the connection the OS still pushes down the host name and it’s own unique id when it requests a DHCP address from dnsmasq, this causes dnsmasq to cycle through it’s small pool of addresses. This combined with the fact the clock on Zero is not battery backed up so only gets set correctly when it can reach internet can cause strangeness with addresses getting expired in strange ways. Anyway there is a pretty simple fix.

Adding leasefile-ro to the dnsmasq config causes it to not write lease information to disk, but rely on the dhcp-script to keep track of things. To do this I needed to add handling for a new option to the script to deal with dnsmasq wanting to read the current lease state at startup.

#!/bin/bash
op="${1:-op}"
mac="${2:-mac}"
ip="${3:-ip}"
host="${4}"

if [[ $op == "init" ]]; then
  exit 0
fi

if [[ $op == "add" ]] || [[ $op == "old" ]]; then
  route add default gw $ip usb0
fi

Now on to getting things working better with Windows machines.

To do this properly we need to move from the g_ether to the g_multi kernel module, this lets the Zero be a USB Mass Storage device, a network device (and a serial device) at the same time. This is useful because it lets me bundle .inf files that tell Windows which drivers to use on the device it’s self so it they can be installed just by plugging it in.

The first order of business is to fix the cmdline.txt to load the right module, after making the changes in the last post it looks like this:

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait modules-load=dwc2,g_ether

The g_ether needs replacing with g_multi so it looks like this:

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait modules-load=dwc2,g_multi

next we need to fix the options passed to the module, these are in the /etc/modprobe.d directory probably in a file called g_ether.conf. We should don’t need to, but to make it obvious when we come back to this a lot later on we’ll rename it to g_multi.conf. Now we’ve renamed it we need to add a few more options.

It currently looks like this:

options g_ether host_addr=00:dc:c8:f7:75:15 dev_addr=00:dd:dc:eb:6d:a1

It needs the g_ether needs changing to e_multi and some options adding to point to a disk image.

options g_multi file=/opt/disk.iso cdrom=y ro=y host_addr=00:dc:c8:f7:75:15 dev_addr=00:dd:dc:eb:6d:a1

Now we have the config all sorted we just need to build the disk image. First we need to create a directory called disk-image to use as the root of the file system we will share from the Zero. Then we need to get hold of the 2 .inf files, they ship with the Linux Kernel doc, but can be found online (serial port, RNDIS Ethernet).

Once we have the files place them in a directory called disk-image/drivers. We should also create a README.txt to explain what’s going on, put that in the root of disk-image. Along side that we want to create a file called Autorun.inf, this tell Windows about what’s on the “cd” when it’s inserted and where it should search for the driver definitions.

[AutoRun]
open="documentation\index.html"
icon=clock.ico,0

[DeviceInstall]
DriverPath=drivers

[Content]
MusicFiles=no
PictureFiles=no
VideoFiles=no

Full details of what can go in the Autorun.inf file can be found here, but the interesting bits are the DriverPath=drivers which points to the directory that we put the .inf files in earlier. Also the open=”documentation\index.html” which should open documentation/index.html when the device is plugged in which explains how to install the drivers. I also added an icon file so the drive shows up looking like a clock in the file manager.

That should be the bare minimum that needs to be on the image, but I ran into an issue with the g_multi module complaining the disk image being too small, to get round this I just stuck a 4mb image in the directory as well. To build the iso image run the following command:

mkisofs -o disk.iso -r -J -V "Zero Clock" disk-image

This will output a file called disk.iso that you should copy to /opt/disk.iso on the Zero (I built the image on my laptop as it’s easier to edit files and mkisofs is not installed in the default raspbian image).

This is working well on Linux and Windows, but I’m still getting errors on OSx to do with the file system. It’s not helped by the fact I don’t have a Mac to test on so I end up having to find friends that will let me stick a random but of hardware in to the side of their MacBook.

Once I’ve got the OSx bits sorted out I’ll put together script to build images for anything you want.

So now we have a Pi Zero that should look like a CD-ROM drive and a Network adapter when plugged into pretty much any machine going, it brings the driver information with it for windows, sets up a network connection with a static IP address and a Avahi/Bonjour/mDNS address to access it. I’m planning on using this to set up my Linear Clock on the local WiFi but there are all manner of interesting things that could be done with a device like this. e.g. an offline Certificate Authority, a 2FA token, a Hardware VPN solution or a Web Controllable display device.

brr

Raspberry Pi Zero Gadgets

I’m still slowly plugging away at my linear clock project. Currently I’m working out how to make it easy to configure it to connect to WiFi network.

One approach is the Physical Web/Web Bluetooth approach I’ve talked about before. This is a really neat solution but it only works with Android phones at the moment as Apple don’t look to be planning any Web Bluetooth support at the moment.

While looking for a more general solution I decided to look at expanding the method I’ve been using to develop the clock. Raspberry Pi Zero’s USB port can act in both Host and Device mode. This means you can plug it into another computer and have it show up as a peripheral. There is support for several different modes, Ethernet adapter, Mass Storage, Serial port plus a few others. The most popular is the Ethernet adapter. You can find some really good instructions on setting all this up here.

This works pretty well for a developer looking to access the Pi Zero to poke around but it’s still a little bit brittle for a consumer device as it relies on Bonjour/avahi to locate the device as it will end up with a randomly assigned 169.254.x.x address. This can be solved by adding some more config options after the instructions in the blog I linked to earlier.

  • First we need to fix the mac address that the Ethernet device presents. This is so that when you plug the Zero into a computer it always recognises it as the same device. To do this we need to add some options to the g_ether module. Create a file in the /etc/modprobe.d directory called g_ether.conf with the following content:
    options g_ether host_addr=02:dd:c8:f7:75:15 dev_addr=02:dd:dc:eb:6d:a1

    This sets the mac address for Zero end of the connection to

  • Next we need to give the Zero a fixed IP address, to do this we add the following to the /etc/network/interfaces file:
    auto usb0
    iface usb0 inet static
      address 10.33.0.1
      netmask 255.255.255.0
    

    The 10.0.0.0/8 range is one of the RFC 1918 private address ranges, I picked 10.33.0.0/24 as it’s not likely to clash with the average home network address range.

  • We also need to stop the dhcp client from adding that 169.254.x.x address, this is the bit that took me ages to find, but in the end you just need to add noipv4ll to the end of /etc/dhcpcd.conf
  • That takes care of the network from the Zero’s point of view but we still need to find a way to assign a network address to the Host computer. This is done with a DHCP server and the simplest to set up for this is dnsmasq. This is where the first tricky bit happens as dnsmasq is not installed by default in the raspbian lite image*. Once installed add file called local.conf to /etc/dnsmasq.d/ with the following:
    interface=usb0
    dhcp-range=usb0,10.33.0.2,10.33.0.5,255.255.255.0,1h
    dhcp-option=3
    

    This tells the Zero to serve up an address between 10.33.0.2 an 10.33.0.5, but given we fixed the mac address of the host end of the network connection it will always end up just handing out 10.33.0.2.

After all that you should have a Pi Zero you can plug into any computer and it should always be available on 10.33.0.1 (as well as raspberrypi.local if the connected computer supports Bonjour/Avahi). This will make writing documentation a lot easier.

I have a couple of extra bits as well, such as a script that sets the default route to the IP address handed out by dnsmasq so if you have internet sharing/Masquerading enabled on the host then the Zero can access the internet. (There is also DHCP option 19 which should enable packet forwarding on the DHCP client, but I need to investigate how this actually works and what effects it has on different OS)

The script lives in /root/route.sh looks like this:

#!/bin/bash
op="${1:-op}"
mac="${2:-mac}"
ip="${3:-ip}"
host="${4}"

if [[ $op == "add" ]] || [[ $op == "old" ]]; then
  route add default gw $ip usb0
fi

And to enable it add the following line to the end of the /etc/dnsmasq.d/local.conf

dhcp-script=/root/route.sh

There is still on niggle, that while the driver (RNDIS Ethernet driver) for this is shipped with Windows you still need to manually install it before it will work. There are some inf files that ship with the Linux kernel docs that can make this a lot easier to do so my next task is to work out how to user the g_multi mode which allows the Zero to be both a Ethernet adapter and a Mass Storage device. This will mean that the Zero will show up as a thumb drive as well the network adapter. I can then include some Documentation and the inf files on that drive. I have most of it working, but it still needs a little polishing, I’ll post again when I’ve got it all working nicely.

*You need to find a way to get the Zero online to install it, I used the RedBear IoT pHAT which lets me get on to my WiFi while still powering/accessing the Zero via the USB socket, but you can also boot the Zero normally with a USB Ethernet or WiFi adapter. To install dnsmasq run the following:

apt-get install dnsmasq

Auto Launch Webpages full screen on Android

While waiting for Amazon to get round to reviewing my Node-RED Alexa Smart Home Skill I’ve needed something to hack on.

I’ve been keeping an eye on what folk have been doing with Node-RED Dashboard and a common use case keeps coming up. That is running the UI on a Android tablet mounted on a wall as a way to keep track of and control things around the house. This got me thinking about how to set something like this up.

Icon

For the best results you really want to run this totally full screen, there are ways to get this to happen with Chrome, but it’s a bit convoluted. Sure you can add it as a shortcut on the home screen but I thought there had to be a easier/better way.

So I started to have a bit of a play and came up with a new app. It’s basically just a full screen Activity with a WebView, with a few of extra features.

Settings
Settings
  • Set URL – Pick a URL to load when the phone boots, or the app is launched.
  • Launch on boot – The app can be configured to start up as soon as the phone finishes booting.
  • Take Screen Lock – This prevents the screen from powering off or locking so the page is always visible.

You can change the URL by swiping from the left hand edge of the screen, this will cause the action bar to appear, from this you can select “Settings”.

Set URL to Load
Set URL to Load

The app is in the Google Play store here, the code is on Github here