Tag Archives: video

Openstreetmap Video overlays

So as I mentioned in my last post I’ve been playing with generating map overlays for the cycling videos I’ve been making while out training. I’d run into a rate limiting problem when using Google Maps static map API.

To work round this I thought I’d see what I could do using Openstreetmap. Openstreetmap doesn’t have a static image API so I’m going to try and build something similar using LeafletJS and a way to grab images of the pages generated.

<html>
<head>
	<title>Maps</title>
	<link type="text/css" href="leaflet.css" rel="stylesheet"/>
	<script type="application/javascript" src="leaflet.js"></script>
	<style >
	#map { 
		height: 250px; 
		width: 250px;
	}
	</style>
</head>
<body>
	<div id="map"/>
	<script type="application/javascript">
function getUrlVars()
{
    var vars = [], hash;
    var hashes = window.location.href.slice(window.location.href.indexOf('?') + 1).split('&');
    for(var i = 0; i < hashes.length; i++)
    {
        hash = hashes[i].split('=');
        vars.push(hash[0]);
        vars[hash[0]] = hash[1];
    }
    return vars;
}

var args = getUrlVars();

var line = args["line"].split("|");

var last = line[(line.length - 1)];

var centre = [last.split(',')[0], last.split(',')[1]];

var map = L.map('map',{
	zoomControl: false,
	zoom: 15
});
map.setView(centre, 15);

L.tileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', 
	{
		maxZoom: 20,
	}).addTo(map);

var latlngs = [];

for (var i=0; i<line.length; i++) {
	latlngs.push(L.latLng(line[i].split(',')[0],line[i].split(',')[1]));
}

var polyline = L.polyline(latlngs, {color: 'red'}).addTo(map);
	</script
</body>
</html>

This generates the map tiles and overlays the route, but it’s as a web page, now I needed a way to convert this into a PNG image. There are two options, html2canvas or PhantomJS. I decided to go with PhantomJS first. The following loads and renders the page and then generates a PNG image.

var page = require('webpage').create();
var system = require('system');


page.onConsoleMessage = function(msg, lineNum, sourceId) {
  //console.log('CONSOLE: ' + msg + ' (from line #' + lineNum + ' in "' + sourceId + '")');
};

page.viewportSize = {
  width: 265,
  height: 250
};

var url = "file:///opt/share/playing/map-overlay/index.html?" + system.args[1];

console.log(url);

page.open(url, function(){
  setTimeout(function() {	
    page.render(system.args[2]);
    phantom.exit();
  },500);
});

The coordinates for the line are passed in on the command line along with the file name to write the file to.

test

phantomjs --local-to-remote-url-access=true --max-disk-cache-size=1024 --disk-cache=true map.js [Path] [file name]

Running PhantomJS with the disk cache enabled should keep the load on the Openstreetmap servers to a minimum but I’m also looking at how easy it is to set up my own tile server.

I can now feed this in to the scripts I spun up last time.

YouTube Preview Image

DIY Video overlay

I got myself a Garmin Virb Elite in the post Christmas sales, the plan was to use it while riding my bike and when snowboarding.

The camera will shoot in full 1080p HD and has a super wide angle lens to grab loads of what’s going on.

I finally managed to get out on my bike at the weekend and remembered to hook the camera up so it was hung under the handle bars. The raw video looks pretty good.

Having got some video I wanted to overlay some of the stats from my Garmin 810 cycling computer like location, speed, elevation, gradient and heart rate. Garmin provide some software called Virb Edit which will do all this, unfortunately it only runs on Windows or OSx. No problem I thought, I’ll just throw it in the same Windows 7 VM I use for Garmin Express, the app I use to upload my workouts. This was going well until I tried to view one of the videos I’d just imported and it complained about not having DirectX 10 support. This is the bit of Windows that handles all the multimedia stuff and for video tends to need access to a GPU to accelerate things. While it is possible to get things like this to work with a VM it is a lot of work and requires the machine your using to have 2 graphics cards1.

Since the standard software wasn’t going to work I thought I’d have a go at trying to build some scripts to do this with Linux. I decided to start with a simple map overlay. I’ve played with Google Map’s static maps API before so I ran up some simple NodeJS code to read a TCX file I generated from the FIT file created by the camera.

var tcx = require('tcx-js');
var http = require('https');
var fs = require('fs');

var parser = new tcx.Parser();
parser.parse_file("test.tcx");

var tail = [];


parser.activity.trackpoints.every(function(element, index, array){
	
	var filename = index + '.png';
	var url = "https://maps.googleapis.com/maps/api/staticmap?center=" + element.lat + "," + element.lng + "&zoom=16&size=300x300";
	if (index != 0) {
		url = url + '&path=color:0x0000ff80|weight:2|' + tail.join('|');
	}
	console.log(url);


	 (function(uri,fn){
	 	http.get(uri, function(res){
	 		//console.log("statusCode: ", res.statusCode);
 //  			//console.log("headers: ", res.headers);
	 		var data = '';
	 		res.setEncoding('binary');
	 		res.on('data', function (chunk){
	 			data += chunk;
	 		});

	 		res.on('end', function(){
	 			fs.writeFile(fn, data, 'binary', function(err) {
	 				if (err) {
	 					console.log(err);
	 				}
	 			});
	 		});
	 	});	
	 })(url, filename);


	tail.push(element.lat + "," + element.lng);
	if (tail.length >= 25) {
		tail.shift();
	}

	if (index != (array.length-1)) {
		var now = new Date(element.time);
		var then = new Date(array[index +1].time);

		var diff = Math.abs((then.getTime() - now.getTime() ) / 1000);
		//console.log('%d', diff);
		for (var i=0; i<diff; i++) {
			fs.appendFile('list.txt',filename + '\n',function(err){});
		}
	}

	return true;
	
});

This worked pretty well until I hit Google’s rate limit so only got the first 100 map titles.

I used the ImageMagick scripts to build a composite image of the map tile on transparent background.

Create an empty 1920×1800 image with a transparent background

$ convert -size 1920x1080 xc:transparent background.png

Overlay the map title in the bottom right hand corner

$ composite -geometry 300x300+1520+700 1.png background.png scaled/1.png

The script also generates a file called list.txt which contains a list of the filenames and how many times to repeat them at 1 frame per second to match up with the orginal video. Using this list with mencoder to generate the video like this.

$ mencoder mf://@list.txt -mf w=1920:h=1080:fps=1:type=png -ovc copy -oac copy -o output.avi

Finally I used OpenShot to generate the composite video.

I’m going to have a look at using OpenStreetmap instead of Google Maps to avoid the rate limiting and also trying to generate some gauges for speed, cadence, altitude and heading.

1 The laptop I’m doing this on does actually have 2 graphics cards, but I’m using the better one to actually run the OS and it’s a royal pain to switch them round.

Timelapse photography

As I mentioned at the end of my last post I have been playing around with gphoto2 to create some time lapse videos of the assembly of one of my Christmas gifts.

I have played with making time lapse video before, when I set up my MMS CCTV system with motion I enabled a feature that creates a video from a sample image every 30 seconds. Motion uses the web cam (or other Video4Linux src) and all the web cams I had access to up at my folks are pretty low resolution so this wasn’t what I was looking for.

I did have my Canon 350D and the little point and shoot Canon A520 that lives on the end of my work bag so I thought I’d see what I could do with them. The 350D is 8 Megapixel and the A520 is 4 Megapixel so both will take a frame way bigger than 720p that I can crop down.

I had a bit of a look round and found an app called gphoto2 that claimed to be able to drive both cameras via the USB port to take images at a set interval. I plugged the 350D in and tried to get it to work but I kept getting the following error*:

*** Error ***
Sorry, your camera does not support generic capture
ERROR: Could not capture.
*** Error (-6: 'Unsupported operation') ***

So I swapped over to the A520 and things seamed to work fine. So I set it up on the tripod and fired off the following command:

[hardillb@bagend ~]$ gphoto2 --capture-image -I 120

This triggers the camera every 2 mins which I thought should be enough time to see some progress in each frame.

Apart from having to swap the batteries twice it all went rather well, I soon started to ignore the little beep from the camera as it took each shot. At the end I copied all the images off the SD card for processing. Each image started out at 2272×1704 so they would have been big enough to use in 1080p video but I decided to shrink them down to 720p.

The following little ImageMagik script resizes the images down and adds 2 black bars down the sides to pad it out to a full 16:9 720p frame size.

#!/bin/sh
for x in `ls *.JPG`
do 
   convert $x -resize 1280x720 -bordercolor black -border 160x0 resized/$x
done

The first bit -resize 1280×720 resized the original image down to 720 pixels high and the second bit -bordercolor black -border 160×0 adds on the 2 x 160 pixel wide black bars to pad the image up to the required 1280 pixels wide before writing a copy out to the resized directory.

And this mencoder line to stitch them together as a video with 2 frame per second so each second of video is equivalent to about 4 minutes of real time.

mencoder "mf://*.JPG" -mf fps=2 -o day1.avi -ovc lavc -lavcopts \
vcodec=msmpeg4v2:vbitrate=800

Here is a sample of the output

YouTube Preview Image

*I have since found this gphoto bug report that mentioned changing the camera’s USB mode from PTP mode to normal mode. After finding this setting on the camera I managed to get the 350D to work with gphoto2 as well.

Linux Photo-me-booth

I’ve spent the last few days on site at a bespoke coach builder’s helping to fit out a truck, this all came about after a chat over lunch at the end of last year. Kevin mentioned a project he was involved in to put some our lab demo’s into a truck that will be touring round a number of universities as part of the Smarter Planet initiative.

Side 1

As well as building portable versions of some demo’s there was a new bit. The plan was to have a sort of photo booth to take pictures of the visiting students and build a custom avatar for them to use at each station and upload to sites like Flickr and Facebook.

Since we were already doing some work for the truck we said we would have a look at doing this bit as well, about half an hour of playing that afternoon it looked like we should be able to put something together using a commodity webcam, Linux and some existing libraries.

First Pass

The first approach was to see if we could do what we needed in a reasonably simple web page. Using a streaming video server, the HTML 5 <video> tag and a bit of javascript we had a working prototype up and running very quickly.

The only problem was the lag introduced by the video encoding and the browser buffering, most of the time it was about 12 seconds, with a bit of tinkering we got it down to 5 seconds, but this was still far too long to ask somebody to pose for in order to just grab a single image.

Second Pass

So after getting so close with the last attempt I decided to have a look at a native solution that should remove the lag. I had a bit of a look round to see what options where available and I came across the following:

  • Video4Linux
    This is gives direct access to the video hardware connected to Linux
  • GStreamer
    This is a framework that allows you to build flows for interacting with media sources. This can be audio or video and from files as well as hardware devices.

As powerful as the Video4Linux API is it’s seamed a bit too heavy weight for what I was looking for. While looking into the GStreamer code I found it had a pre-built package that would do pretty much exactly what I wanted called CameraBin.

With a little bit of python it is possible to use the CameraBin module to show a Viewfinder and then write an image on request.

<code>
	self.camerabin = gst.element_factory_make("camerabin", "cam")
	self.sink = gst.element_factory_make("xvimagesink", "sink")
	src = gst.element_factory_make("v4l2src","src")
	src.set_property("device","/dev/video0")
	self.camerabin.set_property("viewfinder-sink", self.sink)
	self.camerabin.set_property("video-source", src)
	self.camerabin.set_property("flicker-mode", 1)
	self.camerabin.connect("image-done",self.image_captured)
</code>

Where self.sink is a Glade drawing area to use as a view finder and self.image_captured is a call back to execute when the image has been captured. To set the filename to save the image to and start the viewfinder run the following code.

<code>
	self.camerabin.set_property("filename", "foo.jpg")
	self.camerabin.set_state(gst.STATE_PLAYING)
</code>

To take a photo call the self.camerabin.emit(“capture-start”) method

The plan was for the avatar to be a silhouette on a supplied background, to make generating the silhouette easier the students will be standing in front of a green screen

Green screen

The Python Imaging Library makes manipulate the captured image and extract the silhouette and then build up the final image from the background, the silhouette and finally the text.

&lt;code&gt;
	image = Image.open(path)
	image2 = image.crop((150,80,460,450))
	image3 = image2.convert("RGBA")
	pixels = image3.load()
	size = image2.size;
	for y in range(size[1]):
		for x in range(size[0]):
			pixel = pixels[x,y]
			if (pixel[1] > 135 and pixel[0] < 142 and pixel[2] < 152):
				pixels[x,y] = (0, 255, 0, 0)
			else:
				pixels[x,y] = (0, 0, 0, 255)

	image4 = image3.filter(ImageFilter.ModeFilter(7))
	image5 = image4.resize((465,555))
	background = Image.open('facebook-background.jpg')
	background.paste(image5,(432,173,897,728),image5)
	text = Image.open('facebook-text.png')
	background.paste(text,(0,0),text)
&lt;/code&gt;

The final result shown on one of the plasma screens in the truck.

Silhouette wall

As well as building Silhouette wall, ETS has provided a couple of other items to go on the truck

  1. See It Sign It

    This application is a text to sign language translation engine that uses 3D avatars to sign. There will be 2 laptops on the truck that can be used to have a signing conversation. There is a short video demonstration of the system hooked up to a voice to text system here: http://www.youtube.com/watch?v=RarMKnjqzZU

  2. Smarter Office

    This is an evolution of the Smarter Home section in the ETS Demo lab at Hursley. This uses a Current Cost power meter to monitor the energy used and feeds this to a Ambient Orb to visualise the information better. It also has a watch that can recognise different gestures which in turn can be used to turn things like the lamp and desk fan on and off and the amount of power used by these is reflected in the change in colour from the orb.

For details of where the truck will be visiting over the year, please visit the tours facebook page in the resources.

Resources