Sneak preview inside the 5D Mark IV Dual Pixel RAW files¶
-- a1ex, september 2016The 5D Mark IV is now shipping, and sample images are already available on both camera review sites and blogs. In particular, this sample from kamerabild.se was picked by quite a few others who wanted to look at dual pixel raw data:
%%shell
wget -c http://www.kamerabild.se/sites/kamerabild.se/files/_91A0045.CR2
Intro¶
I've first noticed Laurent Clévy (you know him from http://lclevy.free.fr/cr2/) releasing a command-line tool that could read the two image streams from a dual pixel CR2 (craw2tool.exe). Then, shortly after, Hirax attempted to create depth maps from some dual pixel CR2 images, using Laurent's tool to decode the files 1 2 3. These experiments actually motivated me to look inside the dual pixel raw. I've downloaded the above sample, and after a bit of fiddling with octave, I've got this animation, and also noticed the secondary image from the dual pixel raw contains one extra stop of highlights.
After browsing around, it turns out others were even faster. For example, Iliah Borg from RawDigger already wrote about dual pixel raw and found out the extra stop of highlights as well (before me). His findings confirmed my theories, although I did not agree 100% with what he wrote. Planet Mitch also did a nice summary of the initial findings around dual pixel. And - guess what - although most third party raw converters did not open the 5D4 CR2 files, dcraw did!
When reading all this, I thought "Oh boy, I'm way too late to the party", and went back to my usual stuff.
After doing some more experiments with this sample raw file, I realized there's still a lot to be discovered about the dual pixel to cover, which is why I've started writing this.
Enough chit-chat, let's look at the files.
Loading dual pixel images in octave¶
You will need Octave 4.0 with 16-bit image support. The default installation on most Linux distros only opens 8-bit images, so it won't work. If you are on Ubuntu, you may use ppa:aalpo/octave-q1; otherwise, you may have to compile it from source. I'm not sure what's the situation on other operating systems - you tell me.
%%shell
# -E is undocumented; it outputs the raw image as with -D, also including optical black areas
dcraw -4 -E -s all -v _91A0045.CR2
As expected, we've got two PGM images from a dual pixel CR2. Let's open them in octave:
pkg load image
format compact
im0 = double(imread('_91A0045_0.pgm'));
im1 = double(imread('_91A0045_1.pgm'));
Let's look at them. The images are huge, so I'll show a downsampled version. For simplicity, I'll show just each Bayer channel.
Why sqrt? The image looks too dark without it.
I know, purists will scream. It's just a quick preview, nothing more.
% subtract black level and apply a "gamma" curve
black = 2048;
im0_disp = sqrt(max(im0-black, 0));
im1_disp = sqrt(max(im1-black, 0));
% show each Bayer channel for each sub-image
im0_channels = [ im0_disp(1:32:end,1:32:end) im0_disp(1:32:end,2:32:end);
im0_disp(2:32:end,1:32:end) im0_disp(2:32:end,2:32:end); ];
im1_channels = [ im1_disp(1:32:end,1:32:end) im1_disp(1:32:end,2:32:end);
im1_disp(2:32:end,1:32:end) im1_disp(2:32:end,2:32:end); ];
% join both images in a single figure
imshow([im0_channels im1_channels], []);
You have already noticed:
- the RGGB layout of the Bayer grid (same as with other Canon cameras)
- the first subimage has clipped sky details in the green and blue channels
- the second subimage looks darker and there are no clipped highlights
Let's try a color preview:
function imshow_bayer(im)
% half-res debayer: turn each RGGB cell into a RGB pixel
% not color-correct, but enough for a quick preview
rgb(:,:,1) = im(1:2:end,1:2:end);
rgb(:,:,2) = (im(2:2:end,1:2:end) + im(2:2:end,1:2:end)) / 2;
rgb(:,:,3) = im(2:2:end,2:2:end);
# black level, white balance and gamma correction (hardcoded)
# octave also expects image data to be from 0 to 1 when using floating point (double)rgb = sqrt(max(rgb - 2048, 0));
black = 2048;
white = 16383;
rgb = max(rgb-black, 0);
rgb(:,:,1) *= 2.1;
rgb(:,:,3) *= 1.4;
rgb = rgb / (white-black);
rgb = rgb.^(1/2.2);
# show a downsampled version
imshow(rgb(1:8:end,1:8:end,:))
end
% join both images in a single figure
imshow_bayer([im0 im1]);