An ROV Vision System Without Compromise

The ocean is an enigmatic and uncharted realm that has long fascinated humanity. The emergence of autonomous underwater vehicles presents the opportunity to gain new insights and groundbreaking discoveries in this vast region, but getting there is just the first step.

One of the core challenges is effective underwater vision, the ability of the subsea platform to perceive its surroundings. Recent years have seen an increase in underwater exploration. From the RMS Titanic to Shackleton’s Endurance, advances in Work Class ROVs and the popularity of small ROVs have made these endeavors attainable. However, camera technology for interacting with the surroundings has not kept pace. Two-dimensional video streams with limited resolution remain the norm.

Cutting-edge vision systems can improve piloting and inspection effectiveness by enhancing situational awareness and spatial perception. Voyis' Discovery Vision System addresses the challenges of underwater vision head-on. The goal is to overcome the trade-off between piloting and 3D inspection cameras.

Discovery is a platform without compromise, leveraging edge computing to capture low-latency, enhanced piloting video and high-quality still image data for 3D machine vision. Voyis offers two versions: The Discovery Camera and the Discovery Stereo.

THE TRADE-OFF: STILLS OR VIDEO

An ROV survey operation requires a low-latency video stream for effective vehicle piloting, but conventional video cameras do not capture the high-quality stills images that are needed for 3D reconstruction. Conversely, inspection cameras capture stills images, but typically at a lower frame rate and higher latency than required for piloting. This trade-off is caused by limitations in the camera’s computing architecture that prevent the recording of both compressed video and the original raw image frames.

A video camera acquires image frames at around 25 FPS, which are encoded into a video stream. The process of video encoding introduces artifacts that render the data ineffective for 3D modelling. Furthermore, each video frame typically uses long shutter speeds to overcome limited sensor sensitivity and low power lighting. To capture a target in motion shutter speeds must be sufficiently low to ensure the target does not move during the exposure.

To address this complex problem, the Discovery camera utilizes an approach that Voyis calls “Stills Driven Video.” By utilizing machine vision cameras and a powerful onboard computing module, this trade-off between great video and images can be eliminated. Stills images are captured at 25 FPS with short exposure times, enhanced with edge-computing, and encoded to a low latency video stream. Then instead of discarding the original images, they are time tagged and saved for 3D reconstruction.

DESIGNED FOR MACHINE VISION

The core purpose of the Discovery platform is to enable effective underwater machine vision by acquiring the best possible input data. It seeks to address two core issues that limit standard cameras, a lack of depth of field and limited dynamic range.

Depth of field is the ability of the camera to focus simultaneously on near and far elements, producing a sharp image at both distances. Since 3D inspection cameras must be calibrated to generate undistorted images, they must have a fixed focus to maintain this state. With this goal in mind, the Discovery camera employs a large glass dome, and a specialized 4K lens, to ensure that focus is maintained across the operating range.

Dynamic range is the ability of a camera to resolve both bright and dark targets simultaneously. This is critical in subsea surveys because the light that travels to the corners of the image undergoes more absorption across the longer distance, producing poor illumination at the edges of the field of view. Machine vision algorithms must utilize the entire field of view to effectively track and identify image features. To overcome this, a sensor was chosen that captures 12-bit HDR raw images, providing an extended dynamic range that can be utilized by edge-computing for lossless light leveling. Synchronized high power

Nova Mini lights provide 125,000 lumens, either in short strobes or continuously on, which further improves the operating range and feature detection capabilities of the vision system.

These features ensure the pilot, and machine vision algorithms, can effectively utilize the entire ultra-wide 130ox130o field of view for situational awareness and tracking.

SITUATION & DEPTH PERCEPTION

The Discovery Stereo brings true depth perception and real-time 3D modelling to ROV platforms, while still maintaining a wide 75ox75o field of view and low latency video stream for piloting. This product employs a pair of calibrated cameras in a deep rated housing to deliver real-time, scaled 3D point clouds.

Images are accurately captured from both cameras simultaneously, with edge-computing enhancing, correcting, and analyzing the images to calculate 3D data in real-time. A stereo point cloud is generated using feature matches between a pair of images at a single point in time and single perspective. It can be displayed in a real-time rotatable 3D video stream to observe the changing scene and visualize the build-up of data as the operator “paints” the structure during the survey. This provides quantitative insight to the survey team, as well as real-time quality control on the survey coverage.

In addition to providing critical depth perception to ROV pilots, it also can help enable autonomous capabilities on newer robotic platforms. Since these depth maps can directly model the position of the vehicle and its manipulator relative to the target in real-time, it provides the needed feedback for manipulator autonomy, station keeping, or feature based navigation.

These point clouds are then refined and combined in post-processing to produce a complete and accurate colorized reconstruction of the underwater environment. The result has millimetric accuracy, delivering the potential to revolutionize methodologies for many underwater inspection applications.

SIMPLIFIED INTEGRATION

The Data Distribution Service (DDS) standard defines a data centric architecture where data is the primary and permanent asset. The Discovery is the first underwater camera product that fully supports the DDS architecture, both externally and internally, using its onboard edge computing hardware. When combined with its ROS2 interface, the platform supports simplified and reliable sensor integrations by vehicle manufacturers.

The result is open communication of all product data steams to unlimited endpoints, a significant step forward in how sensors will enable autonomy. By connecting the Discovery camera to the vehicle network, it will broadcast specific data topics, such as Video, Raw Images, Corrected Stills Images, IMU Data, and Depth Maps, enabling any computer on the platform to access the live data and control the product.

The culmination of these innovations in vision system technology has made it possible for ROV operators to explore our oceans with complete situational awareness, real-time 3D data, and an improved visual experience. It is by deeply understanding these core challenges, both the scientific and practical considerations, that Voyis can push the limits of underwater vision and provide a robust solution for operators navigate their environments and enable future autonomy.

For more information, visit: www.voyis.com.

This story was originally featured in ON&T Magazine's June 2023 issue. Click here to read more.