How can I read 10-bit Raw image? which contain RGB-IR data
<p>I want to know how extract the rgb image from my 10-bit raw (it has rgb-ir imagedata) data?</p>

<p>How can I read in Python or MATLAB?</p>

<p>The camera resolution at the time of capture were 1280x720:
indoor photo <a href="https://drive.google.com/file/d/0B0givAGTBMIweWMzbWFzd0xSQ2M/view?usp=sharing" rel="nofollow">Image for download</a> 
outdoor photo <a href="https://drive.google.com/drive/folders/0B0givAGTBMIwSkVJT1B5bHJoYkk?usp=sharing" rel="nofollow">Image 2 for download</a></p>

<p>camera Model: e-CAM40_CUMI4682_MOD </p>

<p>Thanks a lot </p>

<p>I used the following image processing stages:  </p>

<ul>
<li>Bayer mosaic color channel separation.</li>
<li>Linear stretching each color channel.</li>
<li>Simple white balance.</li>
<li>Replace IR color channel with green (convert image to standard Bayer format).</li>
<li>Restore Bayer mosaic.</li>
<li>Simple gamma correction.</li>
<li>Demosaic</li>
</ul>

<p>Instead of processing the IR color channel, I replaced it with green color channel.</p>

<p>According to the RGB image you added, I found the <a href="https://en.wikipedia.org/wiki/Color_filter_array" rel="nofollow">CFA</a> order.<br>
The CFA (color filter array) order is:</p>

<pre><code>B | G
-- --
IR| R
</code></pre>

<p>The following Matlab code process the image to RGB:  </p>

<pre class="lang-matlab prettyprint-override"><code>srcN = 1280;
srcM = 720;

f = fopen('image_raw.raw', 'r');

%Read as transposed matrix dimensions, and transpose the matrix.
%The reason for that, is that Matlab memory oreder is column major, and
%raw image is stored in row major (like C arrays).
I = fread(f, [srcN, srcM], 'uint16');
fclose(f);
I = I';

%Convert from range [0, 1023] range [0, 1] (working in double image format).
I = I/(2^10-1);

%Bayer mosaic color channel separation
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Assume input format is GBRG Bayer mosaic format.
%Separate to color components.
B = I(1:2:end, 1:2:end);
G = I(1:2:end, 2:2:end);
IR = I(2:2:end, 1:2:end);
R = I(2:2:end, 2:2:end);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Linear stretching each color channel.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Linear streatch blue color channel.
B = imadjust(B, stretchlim(B, [0.02 0.98]),[]);

%Linear streatch green channel.
G = imadjust(G, stretchlim(G, [0.02 0.98]),[]);

%Linear streatch red color channel.
R = imadjust(R, stretchlim(R, [0.02 0.98]),[]);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Simple white balance
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Median or R, G and B.
rgb_med = [median(R(:)), median(G(:)), median(B(:))];
rgb_scale = max(rgb_med)./rgb_med;

%Scale each color channel, to have the same median.
R = R*rgb_scale(1);
G = G*rgb_scale(2);
B = B*rgb_scale(3);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Restore Bayer mosaic.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Insert streached color channnels back into I.
I(1:2:end, 1:2:end) = B;
I(1:2:end, 2:2:end) = G;
%I(2:2:end, 1:2:end) = G; %Replace IR with Green.
I(2:2:end, 2:2:end) = R;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Replace IR with green - resize green to full size of image first.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
T = imresize(G, [srcM, srcN]); %T - temporary green, size 1280x720
I(2:2:end, 1:2:end) = T(2:2:end, 1:2:end); %Replace IR with Green.
I = max(min(I, 1), 0); %Limit I to range [0, 1].
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Simple gamma correction
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
gamma = 0.45;
I = I.^gamma;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Demosaic
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Convert to uint8 (range [0, 255]).
I = uint8(round(I*255));
RGB = demosaic(I, 'bggr');
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

imshow(RGB);
</code></pre>

<p>Result:</p>

<p><a href="http://i.stack.imgur.com/u3n4A.jpg" rel="nofollow"><img src="http://i.stack.imgur.com/u3n4A.jpg" alt="enter image description here"></a></p>

<p>Now the colors are normal...  </p>

<hr>

<p><strong>Outdoor image processing :</strong></p>

<p>Applying "indoor" processing on the outdoor image, gets the following result:  </p>

<p><a href="http://i.stack.imgur.com/wEqwt.jpg" rel="nofollow"><img src="http://i.stack.imgur.com/wEqwt.jpg" alt="enter image description here"></a></p>

<p>White trees is a sign for Near-IR spectrum penetration to R, G and B pixels (not only to IR pixels).<br>
The chlorophyll of the vegetation has high reflection in the Near-IR spectrum.  See: <a href="http://missionscience.nasa.gov/ems/08_nearinfraredwaves.html%E2%80%8C%E2%80%8B" rel="nofollow">http://missionscience.nasa.gov/ems/08_nearinfraredwaves.htmlââ</a>, and search it on Google.<br>
Subtracting IR from Red, Green and Blue color channels is required.  </p>

<hr>

<p>I used the following image processing stages:  </p>

<ul>
<li>Bayer mosaic color channel separation.</li>
<li><strong>Subtract IR "surplus" from Red, Green and Blue color channels.</strong></li>
<li>Linear stretching each color channel.</li>
<li>Simple white balance.</li>
<li>Restore Bayer mosaic.</li>
<li>Simple gamma correction.</li>
<li>Demosaic.</li>
<li>Resize RGB image to lower resolution.</li>
</ul>

<p>The following Matlab code process the outdoor image to RGB:  </p>

<pre class="lang-matlab prettyprint-override"><code>srcN = 1280;
srcM = 720;

f = fopen('ir_6.raw', 'r');

%Read as transposed matrix dimensions, and transpose the matrix.
%The reason for that, is that Matlab memory oreder is column major, and
%raw image is stored in row major (like C arrays).
I = fread(f, [srcN, srcM], 'uint16');
fclose(f);
I = I';

%Convert from range [0, 1023] range [0, 1] (working in double image format).
I = I/(2^10-1);

%Bayer mosaic color channel separation
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Assume input format is GBRG Bayer mosaic format.
%Separate to color components.
B = I(1:2:end, 1:2:end);
G = I(1:2:end, 2:2:end);
IR = I(2:2:end, 1:2:end);
R = I(2:2:end, 2:2:end);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Subtract IR "surplus" from R, G and B.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%The coefficients were tuned by trial and error...
ir_r = 1.3;  % 130% of IR radiation is absorbed by red pixels??? 
ir_g = 0.35; % 35% of IR radiation is absorbed by green pixels.
ir_b = 0.3;  % 30% of IR radiation is absorbed by blue pixels.

IR = imresize(IR, size(I)); %Resize IR to the size of I.
IR = max(min(IR, 1), 0); %Limit IR to range [0, 1] (because imresize values slightly outside the range of input).

R = R - IR(2:2:end, 2:2:end)*ir_r; %Subtract IR for R (IR scale coefficient is ir_r).
G = G - IR(1:2:end, 2:2:end)*ir_g; %Subtract IR for G (IR scale coefficient is ir_g).
B = B - IR(1:2:end, 1:2:end)*ir_b; %Subtract IR for B (IR scale coefficient is ir_b).

R = max(min(R, 1), 0); %Limit IR to range [0, 1] 
G = max(min(G, 1), 0);
B = max(min(B, 1), 0);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Linear stretching each color channel.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Linear streatch blue color channel.
B = imadjust(B, stretchlim(B, [0.02 0.98]),[]);

%Linear streatch green channel.
G = imadjust(G, stretchlim(G, [0.02 0.98]),[]);

%Linear streatch red color channel.
R = imadjust(R, stretchlim(R, [0.02 0.98]),[]);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Simple white balance
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Median or R, G and B.
rgb_med = [median(R(:)), median(G(:)), median(B(:))];
rgb_scale = max(rgb_med)./rgb_med;

%Scale each color channel, to have the same median.
R = R*rgb_scale(1);
G = G*rgb_scale(2);
B = B*rgb_scale(3);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Restore Bayer mosaic.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Insert streached color channnels back into I.
I(1:2:end, 1:2:end) = B;
I(1:2:end, 2:2:end) = G;
I(2:2:end, 2:2:end) = R;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Replace IR with green - resize green to full size of image first.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
T = imresize(G, [srcM, srcN]); %T - temporary green, size 1280x720
I(2:2:end, 1:2:end) = T(2:2:end, 1:2:end); %Replace IR with Green.
I = max(min(I, 1), 0); %Limit I to range [0, 1].
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Simple gamma correction
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
gamma = 0.45;
I = I.^gamma;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Demosaic
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Convert to uint8 (range [0, 255]).
I = uint8(round(I*255));
RGB = demosaic(I, 'bggr');
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

RGB = imresize(RGB, size(I)/2); %Shrink size of RGB image for reducing demosaic artifacts.

imshow(RGB);
</code></pre>

<p>Result is not so good, but it demonstrates the concept that IR channel can be subtracted from the Red Green and Blue channels.<br>
There is still work to be done...<br>
Result image:  </p>

<p><a href="http://i.stack.imgur.com/G0FzH.jpg" rel="nofollow"><img src="http://i.stack.imgur.com/G0FzH.jpg" alt="enter image description here"></a></p>

<p>Reason for the "false color" green patches:<br>
Saturated pixels in the Red color channel (saturated in the raw input), are not handled properly.<br>
Problem can by solved by reducing exposure (taking the shot with lower exposure time).  </p>

python matlab numpy image-processing