in-browser video splicing

This is a short post about one of the coolest technologies in Strobee – in-browser video splicing & editing. Editing and streaming video requires a lot of processing power, and introduces lots of latency, but here at Strobee we’re able to build custom video streams quickly and cheaply – inside every user’s browser. The secret is to do this in Javascript without the user even knowing that their web-browser is doing the video splicing (editing short clips together).In the old days, video editing happened entirely offline in a desktop app. I originally learnt to edit using Media100, but today most people use Adobe Premier, Apple iMovie, Microsoft Movie Maker or Sony Vegas. This is how people who need total control create video – people who want to fill the screen with thousands of options.

But these apps take a lot of time to learn, and most people don’t have time to do that just to make a holiday video – my Gran is never going to be able to use Movie Maker, however motivated she is. So we needed simpler video editing.

In answer to this, video editing platforms have become automated, and massively simplified, such as Magisto or WeVideo. These are online applications that enable easy video upload, and sharing from your phone, tablet or desktop.

The problem with these online video editors is that they aren’t as interactive as the old desktop apps –  you upload your footage to your provider in the cloud, and then wait to preview every action that you see. The waiting is caused by the expensive video splicing happening in the cloud, and causes a really ragged user experience – not what we want for Strobee at all:

With the advent of HTML 5 video, it’s possible to splice video in the browser. At Strobee we hijacked browser’s tools that were originally designed for variable quality streaming, to instead perform browser-side video splicing.

The way this works is that each bee (our short video clips) is stored as a static file on the server, and is requested by the browser. The browser uses our api to request file-names, based on the users’s preferences. The required static files are then requested from the server, and patches so they can be inserted into the video stream. This means that we can use all the usual static file caching techniques to reduce cost (Memcache, Cloudflare etc…), but give the user custom video streams:

Clips delivered straight to your phone, with much less expensive work done in cloud

It’s these custom streams that allows Strobee to give users a different video every-time they come to the site, even every-time they perform an action (fast-forward, repeat a bee, change the length of a bee, preview a user’s stories). Even better it means that the video never has to end – we can keep adding bees as fast as the user watches them, even recycling already downloaded clips to save bandwidth.

Strangely, the most complicated part has been creating a simple UI that people understand – letting people edit video while they think they are just navigating a friend-list.

One final feature that we’re proud of, is that the user can even download their video. This uses just the same process – the composition happens in the user’s browser, splicing the video in memory. We look forward to finding out how this scales – both on the server side (when a user requests all their bees as a single video file), and client side (when browser tries to build the user’s video in memory).