GoPro 4K and Twixtor - not a viable solution

The most cost effective way of shooting 4K footage today is with a GoPro Hero3 Black edition. With a name like that it should be shaped like a Stealth bomber or Darth Vader's helmet but in fact it looks like this.

The GoPro Hero3 Black

So not much to look at but it does record 4K footage. Actually not quite as the resolution is 3840x2160 not the 4096x2160 (or 4096x2304) generally accepted as 4K, but close enough. Not quite close enough either is the frame rate it records at, which in PAL setting is 12.5 frames per second, half of what is generally accepted and the result is (if the footage is played at normal speed) a pretty jumpy video. But a San Francisco company called Re:Vision Effects Inc. has produced a piece of software called Twixtor which can re-time your video and insert the missing frame through very clever interpolation. It is designed to allow normal speed footage to be slowed down but I thought I would shoot a 4K clip on my GoPro Hero3 Black and see whether Twixtor could make a nice smooth 25fps real speed version of it.

The test footage was shot on a tripod so most of the image was not changing from frame to frame. The girl on the swing is passing across a complex background of leafy trees and a solid tree trunk. The rope tied to the tire is quite thin and had a tendency to warp with some settings. I experimented with a number of settings using Twixtor in After Effects (this is the trial Twixtor software hence the red cross) and this is the best solution I could come up with frame interpolation set to nearest.

Twixtor settings

Here is the treated footage and a link to the 1080p YouTube version.

So to me it is quite smooth and at first look, fairly acceptable, but looking at it more carefully the girl looks like she has been rotoscoped into the video and around her border there is a lot of blurry and incorrect pixels. Here are two screen grabs of an original frame followed by an interpolated Twixtor frame. 

Original frame

Twixtor interpolated frame

And here are crops of the above images

Original frame cropped

Twixtor processed frame cropped

So unfortunately using Twixtor to interpolate the 12.5 frames per second of the GoPro Hero3 in 4K is not going to be the solution to videographers dreams. But all is not lost, because the Sony sensor within the GoPro Hero3 is capable of proper full frame (4096x2160) 4K recording at frame rates of up to 60fps in 10 bit mode and 48fps in 12 bit mode. So GoPro may just be teasing us with the current "gelded" version and the next release may make the Canon 1DX C look like an over priced paper weight. Well a man can dream can't he?

Sony IMX117CQT sensor details

4K codecs - the good, the big and the ugly. Part 2

The first part of this blog looked at the file sizes of a 1 second 4K video clip using various QuickTime codecs. This part looks at the image quality of the codecs that use an efficient lossy compression system. Some codecs can compress an image and uncompress it without any loss of information (lossless compression), but these produce extremely large file sizes and I have left them out of this test as they should, in theory, show no difference to the original.   

The codecs compared here are GoPro's Cineform, Avid's DNxHD, MJPEG A, MJPEG B, MPEG 4 and Photo JPEG. Here is the table of file sizes for the various versions I made:

File sizes of 4K 1 second clip

Here is the link to the 1 second clip at HD 1920x1080 size in the MPEG 4 codec at 90%.

I made 2 JPEG (100%) stills of the first frame of the clip. One is the full size 4K image and the other is a 640x360 pixel crop with the image zoomed up to 200%. The cropped image is the top of the tower with the crane by the side. All the images are shown below, but I also made a stack of the crops in Photoshop and sliced out the layers for a comparison. 

Quality of various codecs 90% quality. 200% zoom and cropped

I haven't included MJPEGB here because it is no different from MJPEGA, but both MJPEGs show a large variation in the original colour, lifting the gamma significantly. All the other codecs were consistent and matched the uncompressed version.


The original purpose of this test was to see if the Avid DNxHD codec was suitable for 4K use but when I saw how poor the quality was I looked at all the other codecs to see what was wrong. You can see from the sliced picture how blurred the image of the crane has become, but the MPEG4 version is as sharp as the Photo JPEG image below. I was surprised when I looked at the file size how much DNxHD was compressing the image because I hadn't seen such degradation when editing high definition material and the file size is larger than MPEG4. I rendered out this clip in 1920x1080 with the codecs again set to 90% quality. The DNxHD clip was 44 MB in size and the MPEG4 was only 14 MB in size.

To be precise the HD version of the DNxHD codec was 44,808 KB in size which is exactly the size of the 4K version, which gave the solution. The DNxHD codec is only designed for the 1920x1080 video space and when the image dimension goes beyond that the pixels are duplicated (or more) to fill in the gaps, hence why it is blurred and the file size doesn't increase.

I think this is a mistake and Avid should limit the dimension the codec can produce, like H.264 does. Or it should ignore file size and produce a similar quality of image that the other codecs show here. I assume Avid will come up with a native 4K codec very soon which can be used in the next generation of cameras such as the Blackmagic Design Cinema Camera and their edit tools.

Meanwhile the best solution for moving 4K videos around the web appears to be MPEG 4 because it scales well, reproduces the original colour and is the smallest of all the options. However I hope that the GoPro Cineform codec becomes more widely used (and free) as it is robust and platform agnostic. And of course we wait for the launch of H.265 which is likely to be the choice of the broadcast world.

For videographers producing 4K clips the only solution is to use a lossless codec 

Here are the JPEG images of all the test clips. Please download them to compare.

Cineform 90%

Cineform 90% crop with 200% zoom

Avid DNxHD 90%

Avid DNxHD 90% crop with 200% zoom

MPEG A 90%

MPEG A 90% crop with 200% zoom

MPEG 4 90%

MPEG 4 90% crop with 200% zoom

Photo JPEG 90%

Photo JPEG 90% crop with 200% zoom

4K codecs - the good, the big and the ugly. Part 1

There can be little doubt that there is going to be a huge increase in the demand for 4K footage over the next few years and much of that early demand will have to satisfied by stock material. Currently broadcasters can turn to an enormous library of SD and HD material compiled over decades of shooting, but that won't cut the mustard when intercut with newly acquired 4K footage.

Which is good news for those of us who are planning on making our own library of high quality 4K stock shots. One question I would love to answer is which flavour of 4K will be the choice of stock libraries and I really hope that the big boys can come to some sort of agreement here, because uploading four different versions of a 4K clip is not a quick process.

Now H.265 has been announced and I think this will become the bookies favourite even though its younger cousin H.264 is not currently stock libraries first choice for progressive footage. But efficient as H.265 is, it's unlikely to be a good format to edit with for reasons I won't go into here. This means any stock footage will have to be transcoded to another format or codec before editing.

So I ran some tests on existing codecs using a 4K clip from a time lapse within Adobe After Effects to see if any of them would be good enough to host on the web and edit with natively. Here is a still from the sequence. 

The One Tower, St Georges Wharf, Vauxhall, London

The crane on the tall building behind Vauxhall Bridge was unfortunately hit by a helicopter in January 2013 in case it looks familiar - you can see a 1080p version of the time lapse here. I chose this clip because of the complexity of the image with lots of vertical and horizontal lines and a lot of detail in the sky and water.

Here are the codec candidates:

Avid DNxHD - works on Mac and PC and the codec is free.
BlackMagic codec - great quality but probably too big and not widely used.
Cineform - again works on Mac and PC but you need the Premium version of Cineform Studio to create files and this is $299.
Photo JPEG - An old codec, with good compatibility. Currently the choice of many stock companies but not the most efficient of compression choices.
TIFF - Using LZW compression should give a perfect image but with file sizes to match. Also tried a QT using TIFF.
Animation - Not used so much any more but moving between editing and graphics tools is used to be the king. Huge differences in file size depending on the compression ratio.
MPEG A & B - Had their day I think and largely used only for interlaced  
ProRes - A really great codec but will not play well (if at all) on PCs, which really is a ridiculous situation. If it did this would be numero uno. I'm working on a PC so couldn't produce any test results. Pah!

I created a 4K (4096x2304) 1 second clip in each format at two or three compression settings (75%, 90% & 100% where possible) to look at file size and quality. Here are the results:

File sizes of  1 second 4K clip in various codecs 

The TIFF sequence adds up to 288 MB for the 25 frames. 

Unsurprisingly the uncompressed or lossless compression codecs are significantly larger than the lossy compressed versions. The embedded TIFF QTs were significantly larger than the sequence of stills using the LZW lossless compression and the TIFF sequence would still be a good way to exchange between editing and graphic systems.

The lossy compression codecs start with Cineform (100% quality) at 126 MB for a 1 second clip. Bearing in mind that the maximum  length of a clip on iStock is 30 seconds and you could end up with a files of 3.78 GB! At 75% quality it would be a mere 1.8 GB

The tests showed that DNxHD 444 10 bit was almost the same size at 100%, 90% and 75%. Somewhat surprisingly it made little difference on MJPEG codecs whether 75% or 90% compression was chosen. But Cineform at 100% jumped up to 126 MB - almost double the 90% file size, which shows remarkably efficient compression if the quality stands up.

Photo JPEG and MPEG4 also showed a big difference between 75% and 90% and a 30 second clip at the higher compression (75% quality) would be just over 1 GB in size.

In Part 2 of this blog I look at the picture quality of the smaller files to see whether they have a future as a 4K stock format.