Raspberry Pi Streaming Camera

I talked about using a ONVIF camera to stream to a Chromecast earlier because they come with an open well documented interface for pulling video from them (as well as pan/tilt/zoom control if available).

If you don’t have a camera that supports ONVIF you can build something similar with a Raspberry Pi and the Camera module.

This should work with pretty much all of the currently available Raspberry Pi models (With the exception of the basic Pi Zero that doesn’t have Wifi)

  1. Flash a SD card with the Raspbian Lite image
  2. Insert the camera ribbon cable into both the camera module and the Pi
  3. Once the card has booted use the raspi-conf command to enable the Camera interface
  4. Install ffmpeg sudo apt-get install ffmpeg
  5. Create a script with the following content

v4l2-ctl --set-ctrl video_bitrate=300000

ffmpeg -f video4linux2 -input_format h264 -video_size 640x360 -framerate 30 -i /dev/video0  -vcodec copy -an -f flv rtmp://
  • This script sets the max video bitrate to 30kps
  • If you need to rotate the video you can insert v4l2-ctl --set-ctrl=rotate=180 before ffmpeg to rotate 180 degrees
  • ffmpeg uses the videolinux2` driver to read from the attached camera (/dev/video0)
  • Takes h264 encoded feed at 640x360 and 30 frames per second and outputs it to the same nginx instance that I mentioned in my previous post. The feed is called pi

ffmpeg uses the on board hardware support for the video encoding so even a Pi Zero W runs at about 5% CPU load. This means that if you only have 1 camera you could probably run nginx on the same device, else you can have a multiple cameras all feeding to a central video streaming server.

If you want a kit that comes with the Pi Zero W, Camera and a case to mount it to a window have a look at the Pimoroni OctoCam.

The instructions should also work for pretty much any USB (or built in) camera attached to a Linux machine.

Streaming Camera to Chromecast

I have a little cheap WiFi camera I’ve been meaning to do something with for a while. The on board web access doesn’t really work any more because it only supports Flash or Java Applet to view the stream.

But camera supports the ONVIF standard so it offers a rtsp:// feed so I can point Linux apps like mplayer at it and see the stream.

The camera is currently sat on my window sill looking out over the valley which is a pretty good view.

View from upstairs window

I thought it would be interesting to stream the view to the TV in my Living room while I’m working from home at the moment. It is also a way to check the weather without having to get up in the morning and open the blind.

I have a Chromecast in the back back of both TVs so using this seamed like it would be the easiest option.


Chromecasts support a number of different media types but for video we have 2 common codec that will work across all the currently available device types.

  • H264
  • VP8

And we have 3 options to deliver the video stream

  • HLS
  • DASH
  • SmoothStreaming

These are all basically the same, they chop the video up into short segments and generate a play list that points to the segments in order and the consumer downloads each segment. When it reaches the end of the list it downloads the list again which will now hold the next list of segments.

There is a plugin for Nginx that supports both HLS and DASH which looked like a good place to start.


I’m running this whole stack on a Raspberry Pi 4 running Raspbian Buster.

$ sudo apt-get install nginx libnginx-mod-rtmp ffmpeg

Once the packages are installed the following needs to be added to the end of the /etc/nginx/nginx.conf file. This sets up a rtmp listener that we can stream the video to which will then be turned into both a HLS and DASH stream to be consumed.

rtmp {
  server {
    listen 1935; # Listen on standard RTMP port
    chunk_size 4000;

    application show {
      live on;
      # Turn on HLS
      hls on;
      hls_type live;
      hls_path /var/www/html/hls/;
      hls_fragment 5s;
      hls_playlist_length 20s;
      # Turn on DASH      
      dash on;
      dash_path /var/www/html/dash/;
      dash_fragment 5s;
      dash_playlist_length 20s;

      # disable consuming the stream from nginx as rtmp
      deny play all;

The playlist and video segments get written to /var/www/html/hls and /var/www/html/dash respectively. Because they will be short lived and replaced very regularly it’s a bad idea to write these to an SD card as they will just cause excessive flash wear.

To get round this I’ve mounted tmpfs filesystems at those points with the following entries in /etc/fstab

tmpfs	/var/www/html/dash	tmpfs	defaults,noatime,size=50m
tmpfs	/var/www/html/hls	tmpfs	defaults,noatime,size=50m

Now we have the playlists and segments being generated in a sensible way we need to server them up. I added the following to the /etc/nginx/sites-enabled/default file

server {
  listen 8080;
  listen [::]:8080;

  sendfile off;
  tcp_nopush on;
  directio 512;
  default_type application/octet-stream;

  location / {
    add_header 'Cache-Control' 'no-cache';
    add_header 'Access-Control-Allow-Origin' '*' always;
    add_header 'Access-Control-Allow-Credentials' 'true';
    add_header 'Access-Control-Expose-Headers' 'Content-Length';

    if ($request_method = 'OPTIONS') {
      add_header 'Access-Control-Allow-Origin' '*';
      add_header 'Access-Control-Allow-Credentials' 'true';
      add_header 'Access-Control-Max-Age' 1728000;
      add_header 'Content-Type' 'text/plain charset=UTF-8';
      add_header 'Content-Length' 0;
      return 204;

    types {
      application/dash+xml mpd;
      application/vnd.apple.mpegurl m3u8;
      video/mp2t ts;

    root /var/www/html/;

Now we have the system to stream the content in an acceptable format we need to get the video from the camera into nginx. We can use ffmpeg to do this.

ffmpeg -re -rtsp_transport tcp -i rtsp:// -vcodec libx264 -vprofile baseline -acodec aac -strict -2 -f flv rtmp://localhost/show/stream

This reads from the RTSP stream rtsp:// and streams it into the rtmp://localhost/show/stream. The showpart is the name of the application declared in the rtmp section in the nginx.conf and the stream will be the name of the HLS or DASH stream. In this case the following:

  • HLS ->
  • DASH ->

If you change the end of the `rtmp://localhost/show/XXXX` URL you can create multiple streams from different sources with different names (just make sure the tmpfs mounts have enough space for all the streams).


Node-RED Chromecast flow

I’ve been using the Node-RED Chromecast node to test the streams. The DASH stream is working pretty well, but the HLS is a bit more fragile for some reason. Latency is currently about 20-30 seconds which is appears mainly to be determined by the size and number of chunks in the playlist used but if I wind the fragment size down any lower than 5s or the 20s for the playlist length.


Now it’s basically working the next steps are to add support for Camera devices to my Google Assistant Node-RED service so I can request the stream via voice and have it show on my Google Home Hub as well. I’m also building a standalone Google Smart Home Action just for Camera feeds using an all Google stack just as a learning exercise in Firebase.

At the moment the stream is only available from inside my network, I’ll probably proxy it to my external web server as well and add on authentication. The Google Assistant can be given a Bearer Auth Token along with the URL which means I’ll be able to view the stream on my phone while out. While not important for this stream it would be for other Security camera type applications.

Openstreetmap Video overlays

So as I mentioned in my last post I’ve been playing with generating map overlays for the cycling videos I’ve been making while out training. I’d run into a rate limiting problem when using Google Maps static map API.

To work round this I thought I’d see what I could do using Openstreetmap. Openstreetmap doesn’t have a static image API so I’m going to try and build something similar using LeafletJS and a way to grab images of the pages generated.

	<link type="text/css" href="leaflet.css" rel="stylesheet"/>
	<script type="application/javascript" src="leaflet.js"></script>
	<style >
	#map { 
		height: 250px; 
		width: 250px;
	<div id="map"/>
	<script type="application/javascript">
function getUrlVars()
    var vars = [], hash;
    var hashes = window.location.href.slice(window.location.href.indexOf('?') + 1).split('&');
    for(var i = 0; i < hashes.length; i++)
        hash = hashes[i].split('=');
        vars[hash[0]] = hash[1];
    return vars;

var args = getUrlVars();

var line = args["line"].split("|");

var last = line[(line.length - 1)];

var centre = [last.split(',')[0], last.split(',')[1]];

var map = L.map('map',{
	zoomControl: false,
	zoom: 15
map.setView(centre, 15);

		maxZoom: 20,

var latlngs = [];

for (var i=0; i<line.length; i++) {

var polyline = L.polyline(latlngs, {color: 'red'}).addTo(map);

This generates the map tiles and overlays the route, but it’s as a web page, now I needed a way to convert this into a PNG image. There are two options, html2canvas or PhantomJS. I decided to go with PhantomJS first. The following loads and renders the page and then generates a PNG image.

var page = require('webpage').create();
var system = require('system');

page.onConsoleMessage = function(msg, lineNum, sourceId) {
  //console.log('CONSOLE: ' + msg + ' (from line #' + lineNum + ' in "' + sourceId + '")');

page.viewportSize = {
  width: 265,
  height: 250

var url = "file:///opt/share/playing/map-overlay/index.html?" + system.args[1];


page.open(url, function(){
  setTimeout(function() {	

The coordinates for the line are passed in on the command line along with the file name to write the file to.


phantomjs --local-to-remote-url-access=true --max-disk-cache-size=1024 --disk-cache=true map.js [Path] [file name]

Running PhantomJS with the disk cache enabled should keep the load on the Openstreetmap servers to a minimum but I’m also looking at how easy it is to set up my own tile server.

I can now feed this in to the scripts I spun up last time.

DIY Video overlay

I got myself a Garmin Virb Elite in the post Christmas sales, the plan was to use it while riding my bike and when snowboarding.

The camera will shoot in full 1080p HD and has a super wide angle lens to grab loads of what’s going on.

I finally managed to get out on my bike at the weekend and remembered to hook the camera up so it was hung under the handle bars. The raw video looks pretty good.

Having got some video I wanted to overlay some of the stats from my Garmin 810 cycling computer like location, speed, elevation, gradient and heart rate. Garmin provide some software called Virb Edit which will do all this, unfortunately it only runs on Windows or OSx. No problem I thought, I’ll just throw it in the same Windows 7 VM I use for Garmin Express, the app I use to upload my workouts. This was going well until I tried to view one of the videos I’d just imported and it complained about not having DirectX 10 support. This is the bit of Windows that handles all the multimedia stuff and for video tends to need access to a GPU to accelerate things. While it is possible to get things like this to work with a VM it is a lot of work and requires the machine your using to have 2 graphics cards1.

Since the standard software wasn’t going to work I thought I’d have a go at trying to build some scripts to do this with Linux. I decided to start with a simple map overlay. I’ve played with Google Map’s static maps API before so I ran up some simple NodeJS code to read a TCX file I generated from the FIT file created by the camera.

var tcx = require('tcx-js');
var http = require('https');
var fs = require('fs');

var parser = new tcx.Parser();

var tail = [];

parser.activity.trackpoints.every(function(element, index, array){
	var filename = index + '.png';
	var url = "https://maps.googleapis.com/maps/api/staticmap?center=" + element.lat + "," + element.lng + "&zoom=16&size=300x300";
	if (index != 0) {
		url = url + '&path=color:0x0000ff80|weight:2|' + tail.join('|');

	 	http.get(uri, function(res){
	 		//console.log("statusCode: ", res.statusCode);
 //  			//console.log("headers: ", res.headers);
	 		var data = '';
	 		res.on('data', function (chunk){
	 			data += chunk;

	 		res.on('end', function(){
	 			fs.writeFile(fn, data, 'binary', function(err) {
	 				if (err) {
	 })(url, filename);

	tail.push(element.lat + "," + element.lng);
	if (tail.length >= 25) {

	if (index != (array.length-1)) {
		var now = new Date(element.time);
		var then = new Date(array[index +1].time);

		var diff = Math.abs((then.getTime() - now.getTime() ) / 1000);
		//console.log('%d', diff);
		for (var i=0; i<diff; i++) {
			fs.appendFile('list.txt',filename + '\n',function(err){});

	return true;

This worked pretty well until I hit Google’s rate limit so only got the first 100 map titles.

I used the ImageMagick scripts to build a composite image of the map tile on transparent background.

Create an empty 1920×1800 image with a transparent background

$ convert -size 1920x1080 xc:transparent background.png

Overlay the map title in the bottom right hand corner

$ composite -geometry 300x300+1520+700 1.png background.png scaled/1.png

The script also generates a file called list.txt which contains a list of the filenames and how many times to repeat them at 1 frame per second to match up with the orginal video. Using this list with mencoder to generate the video like this.

$ mencoder mf://@list.txt -mf w=1920:h=1080:fps=1:type=png -ovc copy -oac copy -o output.avi

Finally I used OpenShot to generate the composite video.

I’m going to have a look at using OpenStreetmap instead of Google Maps to avoid the rate limiting and also trying to generate some gauges for speed, cadence, altitude and heading.

1 The laptop I’m doing this on does actually have 2 graphics cards, but I’m using the better one to actually run the OS and it’s a royal pain to switch them round.

Timelapse photography

As I mentioned at the end of my last post I have been playing around with gphoto2 to create some time lapse videos of the assembly of one of my Christmas gifts.

I have played with making time lapse video before, when I set up my MMS CCTV system with motion I enabled a feature that creates a video from a sample image every 30 seconds. Motion uses the web cam (or other Video4Linux src) and all the web cams I had access to up at my folks are pretty low resolution so this wasn’t what I was looking for.

I did have my Canon 350D and the little point and shoot Canon A520 that lives on the end of my work bag so I thought I’d see what I could do with them. The 350D is 8 Megapixel and the A520 is 4 Megapixel so both will take a frame way bigger than 720p that I can crop down.

I had a bit of a look round and found an app called gphoto2 that claimed to be able to drive both cameras via the USB port to take images at a set interval. I plugged the 350D in and tried to get it to work but I kept getting the following error*:

*** Error ***
Sorry, your camera does not support generic capture
ERROR: Could not capture.
*** Error (-6: 'Unsupported operation') ***

So I swapped over to the A520 and things seamed to work fine. So I set it up on the tripod and fired off the following command:

[hardillb@bagend ~]$ gphoto2 --capture-image -I 120

This triggers the camera every 2 mins which I thought should be enough time to see some progress in each frame.

Apart from having to swap the batteries twice it all went rather well, I soon started to ignore the little beep from the camera as it took each shot. At the end I copied all the images off the SD card for processing. Each image started out at 2272×1704 so they would have been big enough to use in 1080p video but I decided to shrink them down to 720p.

The following little ImageMagik script resizes the images down and adds 2 black bars down the sides to pad it out to a full 16:9 720p frame size.

for x in `ls *.JPG`
   convert $x -resize 1280x720 -bordercolor black -border 160x0 resized/$x

The first bit -resize 1280×720 resized the original image down to 720 pixels high and the second bit -bordercolor black -border 160×0 adds on the 2 x 160 pixel wide black bars to pad the image up to the required 1280 pixels wide before writing a copy out to the resized directory.

And this mencoder line to stitch them together as a video with 2 frame per second so each second of video is equivalent to about 4 minutes of real time.

mencoder "mf://*.JPG" -mf fps=2 -o day1.avi -ovc lavc -lavcopts \

Here is a sample of the output

*I have since found this gphoto bug report that mentioned changing the camera’s USB mode from PTP mode to normal mode. After finding this setting on the camera I managed to get the 350D to work with gphoto2 as well.

Linux Photo-me-booth

I’ve spent the last few days on site at a bespoke coach builder’s helping to fit out a truck, this all came about after a chat over lunch at the end of last year. Kevin mentioned a project he was involved in to put some our lab demo’s into a truck that will be touring round a number of universities as part of the Smarter Planet initiative.

Side 1

As well as building portable versions of some demo’s there was a new bit. The plan was to have a sort of photo booth to take pictures of the visiting students and build a custom avatar for them to use at each station and upload to sites like Flickr and Facebook.

Since we were already doing some work for the truck we said we would have a look at doing this bit as well, about half an hour of playing that afternoon it looked like we should be able to put something together using a commodity webcam, Linux and some existing libraries.

First Pass

The first approach was to see if we could do what we needed in a reasonably simple web page. Using a streaming video server, the HTML 5 <video> tag and a bit of javascript we had a working prototype up and running very quickly.

The only problem was the lag introduced by the video encoding and the browser buffering, most of the time it was about 12 seconds, with a bit of tinkering we got it down to 5 seconds, but this was still far too long to ask somebody to pose for in order to just grab a single image.

Second Pass

So after getting so close with the last attempt I decided to have a look at a native solution that should remove the lag. I had a bit of a look round to see what options where available and I came across the following:

  • Video4Linux
    This is gives direct access to the video hardware connected to Linux
  • GStreamer
    This is a framework that allows you to build flows for interacting with media sources. This can be audio or video and from files as well as hardware devices.

As powerful as the Video4Linux API is it’s seamed a bit too heavy weight for what I was looking for. While looking into the GStreamer code I found it had a pre-built package that would do pretty much exactly what I wanted called CameraBin.

With a little bit of python it is possible to use the CameraBin module to show a Viewfinder and then write an image on request.

	self.camerabin = gst.element_factory_make("camerabin", "cam")
	self.sink = gst.element_factory_make("xvimagesink", "sink")
	src = gst.element_factory_make("v4l2src","src")
	self.camerabin.set_property("viewfinder-sink", self.sink)
	self.camerabin.set_property("video-source", src)
	self.camerabin.set_property("flicker-mode", 1)

Where self.sink is a Glade drawing area to use as a view finder and self.image_captured is a call back to execute when the image has been captured. To set the filename to save the image to and start the viewfinder run the following code.

	self.camerabin.set_property("filename", "foo.jpg")

To take a photo call the self.camerabin.emit(“capture-start”) method

The plan was for the avatar to be a silhouette on a supplied background, to make generating the silhouette easier the students will be standing in front of a green screen

Green screen

The Python Imaging Library makes manipulate the captured image and extract the silhouette and then build up the final image from the background, the silhouette and finally the text.

	image = Image.open(path)
	image2 = image.crop((150,80,460,450))
	image3 = image2.convert("RGBA")
	pixels = image3.load()
	size = image2.size;
	for y in range(size[1]):
		for x in range(size[0]):
			pixel = pixels[x,y]
			if (pixel[1] > 135 and pixel[0] < 142 and pixel[2] < 152):
				pixels[x,y] = (0, 255, 0, 0)
				pixels[x,y] = (0, 0, 0, 255)

	image4 = image3.filter(ImageFilter.ModeFilter(7))
	image5 = image4.resize((465,555))
	background = Image.open('facebook-background.jpg')
	text = Image.open('facebook-text.png')

The final result shown on one of the plasma screens in the truck.

Silhouette wall

As well as building Silhouette wall, ETS has provided a couple of other items to go on the truck

  1. See It Sign It

    This application is a text to sign language translation engine that uses 3D avatars to sign. There will be 2 laptops on the truck that can be used to have a signing conversation. There is a short video demonstration of the system hooked up to a voice to text system here: http://www.youtube.com/watch?v=RarMKnjqzZU

  2. Smarter Office

    This is an evolution of the Smarter Home section in the ETS Demo lab at Hursley. This uses a Current Cost power meter to monitor the energy used and feeds this to a Ambient Orb to visualise the information better. It also has a watch that can recognise different gestures which in turn can be used to turn things like the lamp and desk fan on and off and the amount of power used by these is reflected in the change in colour from the orb.

For details of where the truck will be visiting over the year, please visit the tours facebook page in the resources.