Colorizing the Prokudin-Gorskii Photo Collection

By Jonathan Zakharov

Introduction

Sergei Mikhailovich Prokudin-Gorskii (1863-1944) [Сергей Михайлович Прокудин-Горский, to his Russian friends] was a man well ahead of his time. Convinced, as early as 1907, that color photography was the wave of the future, he won Tzar's special permission to travel across the vast Russian Empire and take color photographs of everything he saw including the only color portrait of Leo Tolstoy. And he really photographed everything: people, buildings, landscapes, railroads, bridges... thousands of color pictures! His idea was simple: record three exposures of every scene onto a glass plate using a red, a green, and a blue filter. Never mind that there was no way to print color photographs until much later -- he envisioned special projectors to be installed in "multimedia" classrooms all across Russia where the children would be able to learn about their vast country. Alas, his plans never materialized: he left Russia in 1918, right after the revolution, never to return again. Luckily, his RGB glass plate negatives, capturing the last years of the Russian Empire, survived and were purchased in 1948 by the Library of Congress. The LoC has recently digitized the negatives and made them available on-line.

The goal of this project is to restore these historic images by digitally aligning and combining the color channels. This report outlines the algorithm and techniques used for aligning and colorizing the images, presenting the results of these methods to some of the images in the Prokudin-Gorskii collection.

Algorithm

The process of colorizing the Prokudin-Gorskii images involves several distinct steps. Each step is designed to algorithmically guarantee the proper construction of each colored image. The greatest challenge is taking into account the slight offset spatially, angularly, and temporally between photos, creating the need to align the three separate channels to create a cohesive colored image. To address this, our algorithm goes as follows:

• Splitting the Image into Channels: The original digitized photographs consist of three vertically stacked black-and-white images. From top to bottom, these correspond to the blue, green, and red color channels. The first step is to split the image into its constituent parts.
• Aligning the Channels: Since the photos were taken sequentially, small shifts due to camera movement or changes in the environment are common. Successful alignment requires identifying the best spatial offset between the channels to minimize visual artifacts and create a seamless color image.
• Combining the Channels: Once the channels are aligned, the final step is to merge them into a single color image using the three color channels to construct the full-color photograph.

Step 1: Image Splitting

The raw image is split into its three channels: blue, green, and red (from top to bottom). Since each channel occupies one-third of the image height, given the image dimensions are \( H \times W \), we can easily divide the sections:

\[ I_B = I[0 : \frac{H}{3}], \quad I_G = I[\frac{H}{3} : \frac{2H}{3}], \quad I_R = I[\frac{2H}{3} : H] \]

Step 2: Alignment Using Pyramid

Images which are too large to be directly compared must be downscaled. The downscaled versions of the images are aligned first then at higher resolutions. Starting at the lowest resolution and iteratively refines the offset at each level. It is based on minimizing the L2 distance between the edge maps of the fixed and aligning images, computed using the Sobel filter. The error for a given offset The error for a given offset \( (\Delta x, \Delta y) \) is:

\[ Error = L_2(M_{\text{fixed}},\ \text{shift}(M_{\text{aligning}}, \Delta x, \Delta y)) \]

The L2 loss (or L2 error) between two images \( I_1 \) and \( I_2 \) is given by:

\[ L_2(I_1,\ I_2) = \sqrt{\sum_{x, y} \left( I_1(x, y) - I_2(x, y) \right)^2} \]

Here, \( I_1(x, y) \) and \( I_2(x, y) \) represent the pixel values at position \( (x, y) \) in the two images.

Step 3: Image Combination

After alignment, the channels are stacked to form the final color image:

\[ I_{\text{Colored}} = \left[ I_{Shifted\ R}, \, I_{Shifted\ G}, \, I_B \right] \]

Results

We tested our algorithm on a selection of images from the Prokudin-Gorskii collection. Below are some examples of the restored images, showcasing the effectiveness of our approach in bringing these black-and-white photographs to life in color:

Here's some more cool images from the collection:

Conclusion

The colorization of the Prokudin-Gorskii photo collection provides a unique opportunity to revisit early 20th-century Russia in color. Through the use of image processing techniques, such as pyramid alignment and edge-based matching, the original black-and-white negatives have been transformed into vivid and historically significant color photographs. Nonetheless, this project successfully showcases how modern technology can resurrect the lost colors of a bygone era.