Vertical stripe fix for 5D Mark III H.264 files

What is this about?

Some users noticed a vertical pattern in 5D Mark III video files, first with regular raw video images. Some converters will correct this artifact automatically, so it wasn't a big issue lately.

After implementing the 3x crop mode, the artifact became visible in H.264 files as well. The previous algorithm that operates on Bayer raw data can no longer be used - it need to be adapted to H.264 files.

The effect is visible only in highlights, not in shadows. If you see vertical lines visible in shadows, that's a different issue, so you may stop reading here.

What's causing it?

The 5D Mark III appears to read out 8 columns at a time, in parallel. These 8 columns appear to have different amplifier gains, and that's why the effect is visible in highlights, but not in shadows.

Since:

  1. with regular (non-crop) mode, the artifact was visible in RAW, but not in H.264,
  2. with crop mode, the artifact is visible in H.264, but not in RAW,
  3. the crop mode only changes the sampled area on the sensor, and the pixel binning factors, but leaves all other parameters (resolution, Canon processing and so on) unchanged,

we may suspect that those stripes are present in the raw sensor data, corrected by Canon code in regular (non-crop) H.264, but for crop mode H.264, it would require a recalibration (and reverse engineering to figure out how to do that first). It is also possible that our vertical stripe artifact is amplified by the mismatched calibration.

Enough chit-chat, how to fix it?

You will need:

  • IPython notebook
  • octave (the one prepackaged for your operating system should be fine)
  • the 'image' package from octave-forge
  • octave_kernel for the IPython notebook
  • ffmpeg

Let's the sample files from here in octave. We'll extract a single frame from each clip, as uncompressed 8-bit ppm:

In [1]:
%%shell
# note: ffmpeg is very verbose, so I won't display the output here
ffmpeg -ss 0.96 -i 1080p\ Stripes\ 1.mov bad.ppm -y 2>ffmpeg.log
ffmpeg -i 1080p\ Stripes\ 2.mov ref.ppm -y 2>>ffmpeg.log

Octave code follows:

In [2]:
pkg load image
more off
In [3]:
bad = im2double(imread('bad.ppm'));
ref = im2double(imread('ref.ppm'));
figure,imshow(bad(1:5:end,1:5:end,:))
figure,imshow(ref(1:5:end,1:5:end,:))

warning: your version of GraphicsMagick limits images to 16 bits per pixel
warning: called from
    imformats>default_formats at line 256 column 11
    imformats at line 79 column 3
    imageIO at line 106 column 11
    imread at line 106 column 30

The user who reported the issue included a normal image and also a blank wall, that can be used to extract the vertical pattern. Nice!

Let's see a 1:1 crop from the bad image:

In [4]:
imshow(bad(1:400,1:800,:))

Doesn't look very nice.

Let's try to learn the pattern from the reference image (blank wall), from the green channel. Easier to work with grayscale, isn't?

In [5]:
% will stretch the image to make the pattern very obvious
ref_g = ref(:,:,2);
imshow(ref_g(1:400,1:800), [])

The reference blank wall is not completely uniform - let's filter out the low frequency components:

In [6]:
% filter out very low frequency components in the horizontal direction
im = ref_g;
imf = imfilter(im, ones(1,100)/100, 'replicate');
figure, imshow((im-imf)(1:400,1:800), [])

The image is more uniform now, but there are still a bunch of artifacts. Let's find out the gain for each column:

In [7]:
% get the vertical pattern
% we already know the pattern is actually a difference in column gains
% assume the filtered image is ideal, and extract per-column gain variations
gain = median(im ./ imf);
plot(gain)
axis tight

Let's put it together:

In [8]:
function cor = identify_pattern_gray(im)
  % filter out very low frequency components in the horizontal direction
  imf = imfilter(im, ones(1,100)/100, 'replicate');

  % get the vertical pattern
  % we already know the pattern is actually a difference in column gains
  % assume the filtered image is ideal, and extract per-column gain variations
  gain = median(im ./ imf);
  
  % expand the correction data (vector, one entry per column)
  % to match the size of the image
  cor = bsxfun