The best way to make black and white images

from deep infrared raw files

 

Most of digital cameras have a Bayer color filter array in front of the photosensitive sensor. The Bayer filter array is a repeating pattern of four colored filters. Half of the filters are green, one quarter is red and one quarter is blue. The reason for more green elements in Bayer color filter array is greater impact of green light on images, because human eye is more sensitive to green than to red or blue light. The Bayer filter allows only red, green or blue part of spectrum to reach photosensitive sensor, distinguishing colors on a camera’s sensors that can’t recognize colors.

Every pixel (the smallest element of an image) of color images contains three intensities of light for three elementary colors: red, green and blue. Sensor reading for every pixel behind a Bayer filter contains only one third of the necessary data, intensity of only one of the elementary colors. Data captured by camera’s sensor is like a mosaic; pixels with only red, green or blue color. Steps required to produce an image are in the sample bellow. All images are saved with scale 1:1 in lossless PNG file, no loss in quality as for JPG images. The best way to see real details in an image on screen is to look at image in its original size. For finer details you could go with double size (200%) without image enhancements, or without “smooth image when zoomed-in” option. Smoothing an image could blur differences when comparing two similar images.

The picture bellow contains five images. The first four images are showing the path from the camera’s sensor readings to the final color photograph. The fifth image is a conversion from color to black and white image. The first image marked with ‘R’ is a raw camera output, light intensities captured by camera’s sensor behind Bayer filter. Upper part of an image is black and white, with shades of gray as seen by sensor. The lower part of image is color-coded with Bayer filter colors (mosaic). The missing two color’s intensities are set to middle value of the whole image. This is more accurate than to set them to zero. Also setting missing colors to zero gives very dark image. The second image marked with ‘W’ is a raw camera output with white balance multipliers. Upper and lower parts of the ‘W’ are shown in the same way as for the ‘R’. The second image has accurate colors. It looks green because of the double number of green mosaic parts. The mesh like pattern over an image, inevitable by Bayer filter design, is obvious on both ‘R’ and ‘W’ images. To avoid this pattern and obtain a full color image, missing color data are reconstructed with various demosaicing algorithms. It is a guessing of missing data, interpolation of surrounding pixels of the corresponding colors, based on an assumption that the color of an area in the image is relatively constant. Result of interpolation is visible on the third image marked with ‘I’. This image looks dark because brightness perception of human eye is a power function, while sensor has a linear response to light.

Color RAW to BW.

The next modification of an image is to convert linear pixel values to the power function of the human eye. It is called gamma function. Proper full color image is marked with ‘G’. The last image marked with ‘D’ is the color image converted to black and white (desaturated).

All steps in the sample above are exact calculations except interpolation. The interpolation always adds some noise (errors) to an image. It could be very small, but always exist. The only way to eliminate this noise is not to use interpolation, which is the case with a camera without Bayer filter. A camera with Foveon sensor captures intensities of all three elementary colors for each pixel, works similar to a camera with color film. For black and white photography, cameras without Bayer filter exist. They work similar to cameras with black and white film. Aside of avoid interpolation, an additional advantage of those cameras is that they need one EV step less to make photographs, because there is no Bayer filter’s light loss.

Deep infrared images above 830 nanometers are monochromatic (gray scale). They look exactly the same, regardless of which part of the Bayer filter the light passes through. This allows us to skip interpolation and have better images!

The method described here will work with deep infrared filters equivalent to RG830 or even better RG850, but will not work well with standard infrared filter as RG715. Differences in spectral response (transmittance) of red, green and blue part of Bayer filter bellow 830 nanometers, will leave mesh like pattern on images with standard infrared or red filters.

Bellow is the same scene with 830 nanometers infrared pass filter, following the same steps as for color image. A custom white balance for infrared images was used. Bayer filter mesh pattern visible on raw image, vanishes on the image with custom white balance. Smaller differences in spectral response of red, green and blue part of Bayer filter for infrared part of spectrum, compared to visible part of spectrum causes mesh pattern on raw images less visible on infrared than on color images. Images marked with ‘I’ and ‘G’ are full color images. They look black and white as expected for correct white balance. The final desaturated image is a real black and white image. This last step is not necessary, but a real black and white image is smaller in size and easier to manipulate with. Small white balance imperfection adds a subtle orange cast to the image marked ‘G’, while desaturated image marked with ‘D’ is completely gray. Increasing contrast of an image, could add unwanted color cast on a color image, but not on a black and white image.

Infrared RAW to BW - interpolation.

A simpler way to produce a black and white infrared image is shown bellow. Starting with the same raw image, staying with shades of gray as seen by sensor, apply only white balance multipliers and gamma function. A simpler solution is usually a better solution. Bellow is a way to make a black and white infrared image in three instead of five steps. One of skipped steps is interpolation. Without interpolation noise, the final image should be better. The ‘R’ and ‘W’ images are shown, as before, with upper part black and white, and lower part as color-coded with Bayer filter colors. Interpolation is not necessary because deep infrared light pass through red, green and blue part of Bayer filter with the same light intensity loss. Image is monochromatic. No need to reconstruct missing colors. Image is a real gray scale image, no need for conversion to black and white too.

As expected, image marked with ‘G’ bellow, made without interpolation is a slightly better than image above marked with ‘D’ made with interpolation and desaturation.

Infrared RAW to BW - document.

To have a better look on image differences, bellow are differences calculated by subtracting pixel values of images made with those two methods. Brightness of a pixel is proportional to difference between pixels on two images; a black pixel means no difference. The first image on the left shows relative difference between image made with common five steps method with interpolation, and simple three steps method image without interpolation. Image is a real color image, not a color-coded image. Every pixel on this image has one color channel correct, and two interpolated channels, that could have noise (interpolation errors). This noise is always with one of color channels intensity equal to zero. All noise pixels have a saturated color, combination of two elementary colors. Another way to see differences is to convert image made with interpolation to gray scale image, and subtract it from the image made without interpolation. Resulting relative differences are in the middle. The last image on the right shows only interpolation relative errors, without color manipulations errors. White balance in five steps method was changed to “average color channels of the whole image”, the same as in three steps method. Type of conversion to gray was also changed to be the same as in three steps method. Default change to sRGB color space is skipped. This is a channel mixer, with nine coefficients, specific for each camera. Colors on images in sRGB color space will look the same regardless of type of camera after this conversion. For a monochrome infrared image this just adds noise. These three changes leave only interpolation errors when subtracting this image from three steps image.

You can recognize the shape of a wall on interpolation noise images. Interpolation noise is higher on edges. Errors from interpolation alone, compared to complete five steps method errors, are smaller (darker right image than the middle one) and less random. Bright pixels, errors of the interpolation on the image on the right side, are mostly where the edges are – the wall and branches.

Average absolute error was 4.3% of white pixel for common five steps method, and 4% for interpolation. Adaptive Homogeneity-Directed interpolation was used.

The best way to isolate interpolation errors completely accurate is to take an infrared image with a true monochromatic camera without Bayer filter, remove sensor’s data not captured by Bayer filter camera, and let interpolation algorithm to reconstruct missing data. It would be interesting to see these interpolation errors, but not having infrared camera without Bayer filter prevent me doing this.

Interpolation noise images.

Bellow are two images made from the same raw data. The first image on the top was made with, the second image on the bottom without interpolation, following five and three steps methods. Both images are scaled to fit on the screen. Images look the same. Differences exist, but it is hard to see them on images with reduced size.

Image with and without interpolation.

Bellow are the same two images converted to lossy JPG format with quality 40. Reducing image quality makes differences visible. After conversion, imperfections pop out on image with interpolation, but remain moderate on image without interpolation.

Image with and without interpolation, jpg quality 40.

Now we are going to compare a part of this image marked with green rectangle with 1:1 scale. Four different ways were used to make black and white images from a raw file. Images are sorted from the worst to the best looking image. The first image was made with interpolation with daylight (auto) white balance multipliers. Image bellow shows impact of wrong white balance when converting raw file to a gray scale image.

Sample-Interpolation-Auto-Luma.

The next image is a little better, the same five steps method with interpolation, described earlier, but with a correct custom white balance.

Sample-Interpolation-Custom-Luma.

The third image bellow was also converted to black and white (gray scale) with interpolation, but desaturation step was changed. On the third image all three color channels contribute to intensity of gray with an equal amount. The second image used definition of luma for conversion to black and white, default for the most of photo software. Brightness of a pixel with luma depends mostly on green channel. Luma is usually a better option for black and white conversion of color photographs, but one third of each color channel type of conversion gives better infrared images as you can see.

Sample-Interpolation-Custom-Average.

The best image bellow is made with simple three steps conversion without interpolation.

Sample-No_Interpolation-Average.

Another three samples shows differences in images with and without interpolation. On the left side is part of an image made with interpolation, and on the right side is the same part of image made without interpolation. Next image is part of the scene seen before, inside the yellow square. Picture on the right side, made without interpolation is clearer (not grainy), details are more distinct.

Compare Sample1.

 

Another example that skipping interpolation produces a better image is bellow.

Compare Sample 2.

 

The last example is a part of image, but this time enlarged, with scale 2:1. Enlargement helps to see fine differences. Letter “a” on the right image, made without interpolation looks better. Also light pole arches are looking more natural on the same image.

Compare Sample 3 - 200%

 

Photographers who changed a Bayer filter camera with a camera with Foveon sensor or with a dedicated black and white camera without Bayer filter, would very likely agree, that images made on a camera without Bayer filter (without interpolation) are better looking images. Differences in images are small, but clearly visible. For truly monochromatic, deep infrared images, Bayer filter is useless. Interpolation adds noise to an infrared image. Using method described here, gives almost the same result as if you have removed Bayer filter from your deep infrared camera. Only difference is that you can’t shoot one EV step faster.

The simplest way to make a deep infrared image without interpolation is with dcraw. Dcraw is an excellent free multiplatform raw file decoder made by David Coffin. It is used as backend software in lots of image converters and viewers. Source code is available on author’s web page. Dcraw is included in most Linux distributions. You can find and download dcraw.exe for Windows easily. Another option is to compile the source code. The whole installation of dcraw is a simple copy of executable file to a folder that is in the operating system path.

After have dcraw saved on computer, follow steps:

·         Open terminal (Command prompt)

·         Go to folder with raw deep infrared image(s)

·         Type:   dcraw -d -a name.raw

Options -d -a are case sensitive, no space between dash (minus sign) and option letter. Raw image file with extension is name.raw, for example DSC_1004.NEF.

Black and white image name.pgm will be created in the same folder. You can upload file name.pgm in your photo editing software for enhancement of image or file type conversion.

Option -d is for document mode, light intensity of pixels with gamma function, without interpolation.

Option -a is for averaging the whole image for white balance. This allows a correct black and white image, regardless on white balance set in camera.

This is enough for practical purpose, to make images without interpolation noise. If you want to try more option some variations are:

Use -w instead of -a for white balance set in camera. Use -6 option for 16 bit samples instead of default 8 bits. Use -T for tiff instead of pgm type of output file.

You can add -j -t 0 to ensure no rotation or stretching of raw sensors readings. This makes difference only with cameras with non-square sensor’s elements or with cameras with diagonal orientation of sensor’s elements as Fujifilm S2 Pro camera.

ICC profile file contains description of specific color space. It is unique for each camera type. You can find it in installed camera manufacturer’s software for raw file manipulation, or on software CD for your camera. It has an icm extension. To use ICC profile with dcraw add option: -p iccprofile.icm. Here iccprofile.icm is ICC profile file name. Usually output image will look better with profile, although it can cause mesh like pattern to appear with some files.

One of unique use of dcraw is to get unmodified (completely raw) sensor’s reading of a digital camera with: dcraw -D -4 -j -t 0 name.raw . This file is equivalent to upper part of image marked with ‘R’ in three steps method of getting infrared image from raw file described earlier. If you are interested, you could finish the next two steps by yourself: First repeat white balance multiplication on every 2 by 2 square of pixels (makes image W). Second step, apply gamma function on each pixel (makes image G). For the first step you can get Bayer filter pattern, along with other information with dcraw  -i -v name.raw . If you want to average all three colors, what is the best solution for deep infrared image, you don’t need to know Bayer filter pattern.

The last version of dcraw is from year 2018. Dcraw could fail to open raw images on some of digital cameras less than three years old (this was written in 2021).

If you prefer graphic software to command line dcraw, the only graphic software I know than can be used is Darktable. Photoshop don’t have any demosaicing controls, and can’t be used for this purpose. Settings in Darktable to have similar result to dcraw use are:

·         Demosaic = passthrough (monochrome)

·         White balance = camera

You can adjust picture in Darktable, or save it and adjust in other software. Disadvantage of using Darktable instead of dcraw is that Darktable don’t have Average white balance option. If you look at image with magnification and see mesh like pattern, you should manually correct white balance till mesh like pattern disappears. This requires fine tuning of two of white balance multipliers. Not having a proper white balance will void gains of not using interpolation.

If you like programming you can try LibRaw. It is a free and open-source library that could help you to start writing your own raw image software.

Final thoughts. If you only publish small infrared images on web, method you are going to use will not make a visible difference. If you print full size images, small but clearly visible improvement in infrared images is worth of using method described here.