Subsea Color Imaging for Photogrammetry

Efficient wide-area optical exploration with Image Mapping is the key to uncovering the ocean’s mysteries and is vital for documenting and managing the habitats, history, and critical infrastructure that we all rely upon.

One of the main challenges with observing the ocean is the trade-off between coverage and resolution in subsea instruments. While sonars can capture data across large areas, they often do not have the resolution required for gathering quantifiable and actionable data. Optical sensors like high-resolution stills cameras can provide localized visual data to see the fine details, but historically could not be extended to wide areas. Now the advent of efficient photogrammetry algorithms enables thousands of images to be combined into a wide area quantifiable 3D model that provides a complete understanding of the subsea environment.

Ocean surveyors have historically relied on subsea video cameras deployed from Remotely Operated Vehicles (ROVs) or Autonomously Operated Vehicles (AUVs) to gain a visual understanding of underwater targets for inspection and exploration. But the operating mechanism of video—high compression and blurry features—significantly limits its use for advanced image processing and photogrammetry. Stills Imaging offers fundamental advantages in that short-exposure images and uncompressed raw formats can provide the crisp features and high-dynamic-range information necessary for effective machine vision.

When an underwater vehicle conducts an optical survey for the purpose of photogrammetry, it is paramount that the imaging system provides both consistent and reliable image capture, and fully automated image correction. There are, therefore, three overarching challenges of such a system: effective color correction, even illumination, and consistent focus. With these in mind, Voyis endeavors to produce underwater imagery where the effects of the water medium are removed, enabling the direct use of standard terrestrial processing algorithms like photogrammetry and machine learning.

UNDERWATER COLOR CORRECTION

To understand the challenges with color image correction underwater, we must first understand the problem. Humans perceive color based on how different wavelengths of light (color channels) reflect off objects and reach our eyes. Underwater, the perceived color is altered because red wavelengths are absorbed more than blue and green, and standard cameras are not designed to account for this color shift. Corrections cannot just be done manually on single images; they must be globally consistent so that features can be matched throughout the image sequence as the camera’s perspective shifts. At each point of image capture, both a feature’s perceived color and brightness will change as its position relative to the camera and the light source changes. A solution must therefore balance color reproduction effectiveness and image-to-image consistency in its performance.

To address these colorization challenges, Voyis has developed a survey-grade color correction solution that leverages machine learning to efficiently train a correction model. By analyzing an initial image dataset, a model can be generated that accurately characterizes the color degradation and illumination patterns present in the imaging system. This can then be efficiently applied to correct each image in real-time without operator input, utilizing raw image data to maintain image quality.

EVEN ILLUMINATION

The next challenge is producing an even illumination across the entire camera field of view, a problem that is again due to water’s light absorption, with the amount of absorption growing with target distance. Since light rays directed towards the corners of the image must travel further through the water, more of it is absorbed, and the image corners are perceived as darker. 

To achieve effective photogrammetry, features in the image must be detected and matched across the entire field of view, both in the center of the image and in the corners. When these images are combined into a final model, image brightness must be consistent across both an individual image and between sequential images to avoid visible stitch lines.

Voyis imaging systems employ a two-pronged approach to combat uneven lighting, utilizing both physical and algorithmic lighting correction. LEDs inside the lights are individually oriented based on the selected camera separation to distribute the artificial light, biasing more to the corners of the image. The second prong is utilizing high dynamic range data—the ability of the camera to resolve a wide range of light intensities—to further optimize. A common example is resolving bright reflections off the target and weak reflections off the dull seabed.

Intelligent image enhancement algorithms can utilize this extra intensity information to effectively brighten dark areas of the image without any loss of quality.

CAMERA FOCUS

The final factor is consistent image capture. Many underwater camera systems utilize Auto Focus to continuously optimize image focus as the scene varies throughout a survey. When employed to survey complex structures, the focus algorithm will often lock onto different features in sequential image captures and noticeably adjust the overall camera focus. This focus inconsistency hinders both the ability of photogrammetry to match feature between frames and can also create focus variation artifacts in the 3D model. Moreover, when a lens’ focus changes, its parameters change, altering the camera’s calibration and reducing the accuracy of final photogrammetric 3D model.

Voyis has concluded that the use of a fixed focus lens is best suited for a survey-grade camera as it maintains a constant camera calibration and is more robust to a dynamic image scene. By finely tuning the optical system during calibration it is possible to provide both a consistent focus across the field of view, and an effective depth of field across typical imaging ranges.

UNDERWATER PHOTOGRAMMETRY

The combination of these unique solutions has produced a survey-grade color imaging system that delivers consistently corrected images that remove all aspects of the water medium—color, lighting, and distortions. The result is optical datasets that can be directly utilized in standard photogrammetry software packages to produce incredible wide-area image maps and 3D models that truly bring the ocean to life.

Innovations in subsea technology have made it possible to visually explore vast areas of our oceans more efficiency and with greater detail. It is by deeply understanding the core challenges of underwater optics, both the scientific and practical considerations, that we can push the limits of underwater imaging and provide a robust solution for surveyors to explore and understand the underwater world with extraordinary detail.

Through their seamless integration into subsea vehicles, this is ultimately what our Observer and Nova imaging solutions help users achieve, optimized color images for photogrammetry, machine learning, and survey visualization.

This story was originally published in ON&T Magazine's November 2022 issue. Click here to read more.