Raspberry Pi USB Gadget Creator Update

A few years ago I wrote a script that would take a standard Raspberry Pi OS image and boot it up in a VM to make all the required modifications so a Pi Zero or Pi 4 would work as a USB Ethernet device connected to a host computer.

One of my uses for these images is an offline Certificate Authority that runs on a Raspberry Pi Zero (note not a Zero W so it’s totally offline), so that it is only powered on and accessible when plugged into one of the USB ports on my laptop. This has worked well, but recently I upgraded my laptop to Ubuntu 22.04 and this upgraded OpenSSL to 3.0.x which also include a bunch of security related changes to do with default cypher sets enabled and minimum key sizes it considers secure.

When I came to issue some new certificates for my VPN setup this cause some problems as the PKCS12 files generated by my offline CA could not be opened on the new laptop without adding a bunch of extra options to the commands to re-enable the obsolete configuration. To remedy this it was time to upgrade the CA image to the latest version of Raspberry Pi OS.

The script worked pretty well until Raspberry Pi OS stopped enabling a default user called pi with the password raspberry. This was a sensible change as many Raspberry Pi machines had ended up connected directly to the Internet without the password being changed.

Since there is no default user anymore you have supply a userconf.txt file in the /boot partition on the SDCard that is used to create the user on first boot. The problem is that the /boot partition is bundled into the image so not easily accessible until the image is written to a card.

We can fix this using a loopback mount on Linux, we just need to know the offset into the image file that the partition starts at. We can find that by running fdisk -l against the image:

$ fdisk -l 2022-09-22-raspios-bullseye-armhf-lite.img 
Disk 2022-09-22-raspios-bullseye-armhf-lite.img: 1.75 GiB, 1874853888 bytes, 3661824 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb1214a26

Device                                      Boot  Start     End Sectors  Size Id Type
2022-09-22-raspios-bullseye-armhf-lite.img1        8192  532479  524288  256M  c W95 FAT32 (LBA)
2022-09-22-raspios-bullseye-armhf-lite.img2      532480 3661823 3129344  1.5G 83 Linux

We can see that the first partition is the one we want and the off set it at 8192 sectors (each sector is 512 bytes as shown in the header). Using awk we can then extract and calculate the right value to pass to mount

$ OFFSET=$(fdisk -l $1 | awk '/^[^ ]*1/{ print $2*512 }')
$ sudo mount -o loop,offset=$OFFSET 2022-09-22-raspios-bullseye-armhf-lite.img boot

Now we have the partition mounted we can go about generating the userconf.txt file. This is done using openssl as described in the Raspberry Pi Org docs

$ openssl passwd -6
Password:
Verifying - Password:
$6$k6CgL8YerZDjNAeD$N0HnHZGUPufxjewErapVEalmZll/1KgPlD0ybBXubaAvp7CZEOZBw8FDFIQ2jIyUevanKvGHmmc33YyXaZnf./

The output is appended to username: and placed in the /boot/userconf.txt file, then we can unmount the partition and boot the system to install the required parts for the USB gadget.

I’ve put the whole thing in a shell script which also exports the password as an environment variable so it can be picked up by the expect script that actually does the install.

#!/bin/bash

OFFSET=$(fdisk -l $1 | awk '/^[^ ]*1/{ print $2*512 }')
mkdir boot
sudo mount -o loop,offset=$OFFSET $1 boot

read -rsp "Please enter password for pi user: " PASSWORD
echo
PASS=$(echo $PASSWORD |  openssl passwd -6 -stdin)
echo "pi:$PASS" > userconf.txt

sudo cp userconf.txt boot/userconf.txt
sudo sync

sudo umount boot
rmdir boot
export PASSWORD
./create-image $1

I’ve updated the git repo with the new script.

Pimoroni Keybow Upgrade

I’ve had a little 3 key Pimoroni Keybow sat on my desk for a while. It was running the same basic config I had setup when I bought it, namely mapping the three buttons to volume down, mute and volume up respectively.

Pimoroni Keybow Mini

While this was useful, it felt like there where better uses for it.

With more and more time being spent in video meetings having quick shortcuts to mute the mic or toggle the camera on/off sounded like a good idea. But then I wondered if I could find a way to switch the key mapping on the fly.

The key mapping is done by editing a short Lua script. This is stored on the sdcard that the Pi Zero that holds the Keybow boots from. This means the layout is normally fixed. Except the latest version (0.0.4) of the sdcard image on the Pimoroni Github page added support for starting a USB serial link as well as the HID used to send the keyboard events. This is exposed in the Lua environment so I managed to build the following script.

require "keybow"

-- Keybow MINI volume/zoom controls --

function setup()
    keybow.use_mini()
    keybow.auto_lights(false)
    keybow.clear_lights()
    keybow.set_pixel(0, 0, 255, 255)
    keybow.set_pixel(1, 255, 0, 255)
    keybow.set_pixel(2, 0, 255, 255)
end

-- Key mappings --

state = 'zoom'

function handle_minikey_02(pressed)
    if state == 'zoom' then
	if pressed then
            keybow.set_modifier(keybow.LEFT_ALT, keybow.KEY_DOWN)
            keybow.tap_key("v")
            keybow.set_modifier(keybow.LEFT_ALT, keybow.KEY_UP)
        end
    elseif state == 'media' then
        keybow.set_media_key(keybow.MEDIA_VOL_UP, pressed)
    end
end

function handle_minikey_01(pressed)
    if state == 'zoom' then
	if pressed then
            keybow.set_modifier(keybow.LEFT_ALT, keybow.KEY_DOWN)
            keybow.tap_key("a")
            keybow.set_modifier(keybow.LEFT_ALT, keybow.KEY_UP)
        end
    elseif state == 'media' then
        keybow.set_media_key(keybow.MEDIA_MUTE, pressed)
    end
end

function handle_minikey_00(pressed)
    if state == 'zoom' then
	if pressed then
             keybow.set_modifier(keybow.LEFT_ALT, keybow.KEY_DOWN)
             keybow.tap_key("n")
             keybow.set_modifier(keybow.LEFT_ALT, keybow.KEY_UP)
        end
    elseif state == 'media' then
        keybow.set_media_key(keybow.MEDIA_VOL_DOWN, pressed)
    end
end

local function isempty(s)
  return s == nil or s == ''
end

function tick()
    local line
    line = keybow_serial_read()
    if not isempty(line) then 
        -- keybow_serial_write( line .. "\n" )
        if line == 'zoom' then
            keybow.clear_lights()
            keybow.set_pixel(0, 0, 255, 255)
            keybow.set_pixel(1, 255, 0, 255)
            keybow.set_pixel(2, 0, 255, 255)
            state = 'zoom'
        elseif line == 'media' then
            keybow.clear_lights()
            keybow.set_pixel(0, 255, 0, 255)
            keybow.set_pixel(1, 0, 255, 255)
            keybow.set_pixel(2, 255, 0, 255)
            state = 'media'
        end
    end

end

The serial port gets setup on /dev/ttyACM0 on my laptop so I’m toggling between the 2 modes with echo media > /dev/ttyACM0 and echo zoom > /dev/ttyACM0.

In media mode it works exactly the same as before, but in zoom mode it toggles the camera on/off, toggles mute on/off and cycles through the available cameras.

This worked but keybow_serial_read() call added 1 second of latency to each call to the tick function which really wasn’t great as it was possible to miss key presses.

A bit of digging in the git rep turned up the file that implemented the serial access and this bit of code:

int serial_open(){
    if(port_fd > -1) return 0;

    port_fd = open(KEYBOW_SERIAL, O_RDWR);

    if(port_fd > -1){
        printf("Open success\n");
        tcgetattr(port_fd, &termios);
        termios.c_lflag &= ~ICANON;
        termios.c_cc[VTIME] = 10;
        termios.c_cc[VMIN] = 0;
        tcsetattr(port_fd, TCSANOW, &termios);
    }
    return 0;
}

The termios.c_cc[VTIME] = 10; was what was causing the delay. I rebuilt the library changing the value to 1 and 0. The value is in deciseconds (1/10 seconds)

With 1 the delay was cut to a tenth of a second, which was OK, but meant you had to be very deliberate in pushing the button to make sure it didn’t get missed, which with a mute toggle is a little risky.

With 0 it worked perfectly.

The script also changes the backlight colour for the keys based on mode so I can see which is active. It should be possible to add more modes as needed.

Next up is to see if I can script the toggling the mode based on if Zoom is the currently active window. Looks like it should be possible with tools like xprop or xdotool.

64bit Raspberry Pi OS build with USB-C Ethernet enabled

As a follow up to the 32bit version I built, using the script I described in my post about building custom SD card images, I have created a 64bit version based on the Raspberry Pi OS beta.

You can download the 64bit version from here. This is a full Raspberry Pi OS image, not a “lite” version as only full versions are available in beta.

This is a manual build (following these instructions) because at the moment the script won’t work with the 64bit version because DockerPi doesn’t support USB devices when emulating 64bit CPUs. USB is needed to attach the emulated Network Adapter. There are a set of patches for QUEMU pending that should enable this.

As soon as the patches become generally available I’ll update the project on github.

Pi4 USB-C Gadget

Pi4 Gadget

I’ve previously blogged about using Pi Zero (and Zero W) devices as USB Gadgets. This allows them to be powered and accessed via one of the micro USB sockets and it shows up as both a CD-Drive and a ethernet device.

A recent update to the Raspberry Pi 4 bootloader not only enables the low power mode for the USB hardware, allows the enabling of Network boot and enables data over the USB-C port. The lower power means it should run (without any hats) with the power supplied from a laptop.

Details of how to check/update the bootloader can be found here.

Given that the Pi4 has a Gigabit Ethernet adapter, WiFi and 4 USB sockets (need to keep the power draw low to be safe) and up to 4Gb RAM to go with it’s 4 x 1.5Ghz core processor it makes for a very attractive plugin compute device.

With this enabled all the same script from the Pi Zero’s should just work but here is the updated version for Raspbian Buster.

  • Add dtoverlay=dwc2 to the /boot/config.txt
  • Add modules-load=dwc2 to the end of /boot/cmdline.txt
  • If you have not already enabled ssh then create a empty file called ssh in /boot
  • Add libcomposite to /etc/modules
  • Add denyinterfaces usb0 to /etc/dhcpcd.conf
  • Install dnsmasq with sudo apt-get install dnsmasq
  • Create /etc/dnsmasq.d/usb with following content
interface=usb0
dhcp-range=10.55.0.2,10.55.0.6,255.255.255.248,1h
dhcp-option=3
leasefile-ro
  • Create /etc/network/interfaces.d/usb0 with the following content
auto usb0
allow-hotplug usb0
iface usb0 inet static
  address 10.55.0.1
  netmask 255.255.255.248
  • Create /root/usb.sh
#!/bin/bash
cd /sys/kernel/config/usb_gadget/
mkdir -p pi4
cd pi4
echo 0x1d6b > idVendor # Linux Foundation
echo 0x0104 > idProduct # Multifunction Composite Gadget
echo 0x0100 > bcdDevice # v1.0.0
echo 0x0200 > bcdUSB # USB2
echo 0xEF > bDeviceClass
echo 0x02 > bDeviceSubClass
echo 0x01 > bDeviceProtocol
mkdir -p strings/0x409
echo "fedcba9876543211" > strings/0x409/serialnumber
echo "Ben Hardill" > strings/0x409/manufacturer
echo "PI4 USB Device" > strings/0x409/product
mkdir -p configs/c.1/strings/0x409
echo "Config 1: ECM network" > configs/c.1/strings/0x409/configuration
echo 250 > configs/c.1/MaxPower
# Add functions here
# see gadget configurations below
# End functions
mkdir -p functions/ecm.usb0
HOST="00:dc:c8:f7:75:14" # "HostPC"
SELF="00:dd:dc:eb:6d:a1" # "BadUSB"
echo $HOST > functions/ecm.usb0/host_addr
echo $SELF > functions/ecm.usb0/dev_addr
ln -s functions/ecm.usb0 configs/c.1/
udevadm settle -t 5 || :
ls /sys/class/udc > UDC
ifup usb0
service dnsmasq restart
  • Make /root/usb.sh executable with chmod +x /root/usb.sh
  • Add /root/usb.sh to /etc/rc.local before exit 0 (I really should add a systemd startup script here at some point)

With this setup the Pi4 will show up as a ethernet device with an IP address of 10.55.0.1 and will assign the device you plug it into an IP address via DHCP. This means you can just ssh to pi@10.55.0.1 to start using it.

Addendum

Quick note, not all USB-C cables are equal it seems. I’ve been using this one from Amazon and it works fine.

The latest revision (as of late Feb 2020) of the Pi 4 boards should work with any cable.

There is also now a script to create pre-modified Raspbian images here with a description here and a copy of the modified image here.

Updated Pi Zero Gadgets

Following on from my last post I’ve continued to work on improving my instructions for a USB connectable gadget based on a Raspberry Pi Zero.

Firstly I’ve got a slight improvement to the dnsmasq config I mentioned last time. This removes the dnsmasq.leases file which can cause issues if you plug the Zero into multiple computers. This can be a problem because while I had managed to fix the mac address for Host computer end of the connection the OS still pushes down the host name and it’s own unique id when it requests a DHCP address from dnsmasq, this causes dnsmasq to cycle through it’s small pool of addresses. This combined with the fact the clock on Zero is not battery backed up so only gets set correctly when it can reach internet can cause strangeness with addresses getting expired in strange ways. Anyway there is a pretty simple fix.

Adding leasefile-ro to the dnsmasq config causes it to not write lease information to disk, but rely on the dhcp-script to keep track of things. To do this I needed to add handling for a new option to the script to deal with dnsmasq wanting to read the current lease state at startup.

#!/bin/bash
op="${1:-op}"
mac="${2:-mac}"
ip="${3:-ip}"
host="${4}"

if [[ $op == "init" ]]; then
  exit 0
fi

if [[ $op == "add" ]] || [[ $op == "old" ]]; then
  route add default gw $ip usb0
fi

Now on to getting things working better with Windows machines.

To do this properly we need to move from the g_ether to the g_multi kernel module, this lets the Zero be a USB Mass Storage device, a network device (and a serial device) at the same time. This is useful because it lets me bundle .inf files that tell Windows which drivers to use on the device it’s self so it they can be installed just by plugging it in.

The first order of business is to fix the cmdline.txt to load the right module, after making the changes in the last post it looks like this:

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait modules-load=dwc2,g_ether

The g_ether needs replacing with g_multi so it looks like this:

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait modules-load=dwc2,g_multi

next we need to fix the options passed to the module, these are in the /etc/modprobe.d directory probably in a file called g_ether.conf. We should don’t need to, but to make it obvious when we come back to this a lot later on we’ll rename it to g_multi.conf. Now we’ve renamed it we need to add a few more options.

It currently looks like this:

options g_ether host_addr=00:dc:c8:f7:75:15 dev_addr=00:dd:dc:eb:6d:a1

It needs the g_ether needs changing to e_multi and some options adding to point to a disk image.

options g_multi file=/opt/disk.iso cdrom=y ro=y host_addr=00:dc:c8:f7:75:15 dev_addr=00:dd:dc:eb:6d:a1

Now we have the config all sorted we just need to build the disk image. First we need to create a directory called disk-image to use as the root of the file system we will share from the Zero. Then we need to get hold of the 2 .inf files, they ship with the Linux Kernel doc, but can be found online (serial port, RNDIS Ethernet).

Once we have the files place them in a directory called disk-image/drivers. We should also create a README.txt to explain what’s going on, put that in the root of disk-image. Along side that we want to create a file called Autorun.inf, this tell Windows about what’s on the “cd” when it’s inserted and where it should search for the driver definitions.

[AutoRun]
open="documentation\index.html"
icon=clock.ico,0

[DeviceInstall]
DriverPath=drivers

[Content]
MusicFiles=no
PictureFiles=no
VideoFiles=no

Full details of what can go in the Autorun.inf file can be found here, but the interesting bits are the DriverPath=drivers which points to the directory that we put the .inf files in earlier. Also the open=”documentation\index.html” which should open documentation/index.html when the device is plugged in which explains how to install the drivers. I also added an icon file so the drive shows up looking like a clock in the file manager.

That should be the bare minimum that needs to be on the image, but I ran into an issue with the g_multi module complaining the disk image being too small, to get round this I just stuck a 4mb image in the directory as well. To build the iso image run the following command:

mkisofs -o disk.iso -r -J -V "Zero Clock" disk-image

This will output a file called disk.iso that you should copy to /opt/disk.iso on the Zero (I built the image on my laptop as it’s easier to edit files and mkisofs is not installed in the default raspbian image).

This is working well on Linux and Windows, but I’m still getting errors on OSx to do with the file system. It’s not helped by the fact I don’t have a Mac to test on so I end up having to find friends that will let me stick a random but of hardware in to the side of their MacBook.

Once I’ve got the OSx bits sorted out I’ll put together script to build images for anything you want.

So now we have a Pi Zero that should look like a CD-ROM drive and a Network adapter when plugged into pretty much any machine going, it brings the driver information with it for windows, sets up a network connection with a static IP address and a Avahi/Bonjour/mDNS address to access it. I’m planning on using this to set up my Linear Clock on the local WiFi but there are all manner of interesting things that could be done with a device like this. e.g. an offline Certificate Authority, a 2FA token, a Hardware VPN solution or a Web Controllable display device.

brr

Raspberry Pi Zero Gadgets

I’m still slowly plugging away at my linear clock project. Currently I’m working out how to make it easy to configure it to connect to WiFi network.

One approach is the Physical Web/Web Bluetooth approach I’ve talked about before. This is a really neat solution but it only works with Android phones at the moment as Apple don’t look to be planning any Web Bluetooth support at the moment.

While looking for a more general solution I decided to look at expanding the method I’ve been using to develop the clock. Raspberry Pi Zero’s USB port can act in both Host and Device mode. This means you can plug it into another computer and have it show up as a peripheral. There is support for several different modes, Ethernet adapter, Mass Storage, Serial port plus a few others. The most popular is the Ethernet adapter. You can find some really good instructions on setting all this up here.

This works pretty well for a developer looking to access the Pi Zero to poke around but it’s still a little bit brittle for a consumer device as it relies on Bonjour/avahi to locate the device as it will end up with a randomly assigned 169.254.x.x address. This can be solved by adding some more config options after the instructions in the blog I linked to earlier.

  • First we need to fix the mac address that the Ethernet device presents. This is so that when you plug the Zero into a computer it always recognises it as the same device. To do this we need to add some options to the g_ether module. Create a file in the /etc/modprobe.d directory called g_ether.conf with the following content:
    options g_ether host_addr=02:dd:c8:f7:75:15 dev_addr=02:dd:dc:eb:6d:a1

    This sets the mac address for Zero end of the connection to

  • Next we need to give the Zero a fixed IP address, to do this we add the following to the /etc/network/interfaces file:
    auto usb0
    iface usb0 inet static
      address 10.33.0.1
      netmask 255.255.255.0
    

    The 10.0.0.0/8 range is one of the RFC 1918 private address ranges, I picked 10.33.0.0/24 as it’s not likely to clash with the average home network address range.

  • We also need to stop the dhcp client from adding that 169.254.x.x address, this is the bit that took me ages to find, but in the end you just need to add noipv4ll to the end of /etc/dhcpcd.conf
  • That takes care of the network from the Zero’s point of view but we still need to find a way to assign a network address to the Host computer. This is done with a DHCP server and the simplest to set up for this is dnsmasq. This is where the first tricky bit happens as dnsmasq is not installed by default in the raspbian lite image*. Once installed add file called local.conf to /etc/dnsmasq.d/ with the following:
    interface=usb0
    dhcp-range=usb0,10.33.0.2,10.33.0.5,255.255.255.0,1h
    dhcp-option=3
    

    This tells the Zero to serve up an address between 10.33.0.2 an 10.33.0.5, but given we fixed the mac address of the host end of the network connection it will always end up just handing out 10.33.0.2.

After all that you should have a Pi Zero you can plug into any computer and it should always be available on 10.33.0.1 (as well as raspberrypi.local if the connected computer supports Bonjour/Avahi). This will make writing documentation a lot easier.

I have a couple of extra bits as well, such as a script that sets the default route to the IP address handed out by dnsmasq so if you have internet sharing/Masquerading enabled on the host then the Zero can access the internet. (There is also DHCP option 19 which should enable packet forwarding on the DHCP client, but I need to investigate how this actually works and what effects it has on different OS)

The script lives in /root/route.sh looks like this:

#!/bin/bash
op="${1:-op}"
mac="${2:-mac}"
ip="${3:-ip}"
host="${4}"

if [[ $op == "add" ]] || [[ $op == "old" ]]; then
  route add default gw $ip usb0
fi

And to enable it add the following line to the end of the /etc/dnsmasq.d/local.conf

dhcp-script=/root/route.sh

There is still on niggle, that while the driver (RNDIS Ethernet driver) for this is shipped with Windows you still need to manually install it before it will work. There are some inf files that ship with the Linux kernel docs that can make this a lot easier to do so my next task is to work out how to user the g_multi mode which allows the Zero to be both a Ethernet adapter and a Mass Storage device. This will mean that the Zero will show up as a thumb drive as well the network adapter. I can then include some Documentation and the inf files on that drive. I have most of it working, but it still needs a little polishing, I’ll post again when I’ve got it all working nicely.

*You need to find a way to get the Zero online to install it, I used the RedBear IoT pHAT which lets me get on to my WiFi while still powering/accessing the Zero via the USB socket, but you can also boot the Zero normally with a USB Ethernet or WiFi adapter. To install dnsmasq run the following:

apt-get install dnsmasq