70D - initial experiments on Dual Pixel RAW¶

                                                      -- a1ex, september 2018

August 25, me:

Just stumbled upon something interesting in the firmware. Not sure if nikfreak or anyone else tried that.

call("lv_daf_mode", 1); // "FACTORY:AB_OUT_LEFT"
call("lv_daf_mode", 2); // "FACTORY:AB_OUT_RIGHT"
call("lv_daf_mode", 0); // guess: default, both "channels"

I'd like to see some DNGs from LiveView (plain 1080p) after calling the above functions, without camera movement between the test shots if possible.

September 9, David Hugh:

Here's the test results! Pretty ähm.. interesting :P.

But almost like you expected, right? https://we.tl/t-BykEItcY9C

First dual pixel raw frames from LiveView¶

Since wetransfer links are going to expire, I've re-uploaded the files. Let's get them:

In [1]:
%%shell
wget -c -q https://a1ex.magiclantern.fm/bleeding-edge/70D/dual-pixel/wetransfer-32dddf.zip
unzip -o wetransfer-32dddf.zip

Archive:  wetransfer-32dddf.zip
Written using ZipTricks 4.6.0
 extracting: RAW-000.DNG             
 extracting: RAW-001.DNG             
 extracting: RAW-002.DNG             
 extracting: RAW-003.DNG

Background¶

Two years ago I've started to look into Dual Pixel RAW files from 5D Mark IV. Beware: unfinished experiments.

One of my goals was (and still is) to see how far out-of-focus images can be recovered or refocused, with my own algorithms. That wasn't exactly easy for an image processing n00b like me, but got some promising results on synthetic data (look at [17], [18], [22] and [23] in the above link). The other goal - if results are worth the effort, why not expand the concept to dual pixel raw video?

Oh well, so many interesting things to research, so little time...

Let's open the files!¶

Yes, the above DNGs were from LiveView, i.e. dual pixel raw video. Just 2 frames for now - Canon engineers called them "FACTORY:AB_OUT_LEFT" and "FACTORY:AB_OUT_RIGHT", whatever that means.

In [2]:
%%shell
dcraw RAW-*.DNG
montage RAW-000.ppm RAW-003.ppm RAW-001.ppm RAW-002.ppm -geometry 50%x50%+5+5 preview.jpg

preview

What the duck?!

Time for some pixel peeping in Octave!¶

You will need Octave 4.x with 16-bit image support.

First, a quick'n'dirty helper for opening raw images (DNG, CR2, whatever) through dcraw:

In [3]:
function im = read_raw(filename)
    # -E is undocumented; it outputs the raw image as with -D, also including optical black areas
    dcraw = 'dcraw -c -4 -E';
    system(sprintf('%s "%s" > tmp.pgm', dcraw, filename));
    im = double(imread('tmp.pgm'));
    system('rm tmp.pgm');
end

First image was created with default LiveView RAW settings in Canon firmware (i.e. what we use for raw video), second image was "FACTORY:AB_OUT_LEFT", third image -- "FACTORY:AB_OUT_LEFT", and last image -- back to defaults.

In [4]:
A = read_raw("RAW-000.DNG");
B = read_raw("RAW-001.DNG");
C = read_raw("RAW-002.DNG");
D = read_raw("RAW-003.DNG");
In [5]:
figure,imshow([A D])
figure,imshow([B C])

A bit too dark for my taste...

Let's apply some gamma correction (2 looks good enough for our purposes) and also normalize the images (trial and error).

In [6]:
a = sqrt(max(A - 2048, 0));
b = sqrt(max((B - 512) * 16, 0));
c = sqrt(max((C - 512) * 16, 0));
d = sqrt(max(D - 2048, 0));
imshow([a d; b c])