- Research article
- Open Access
Methods for quantitative studies of seafloor hydrothermal systems using 3D visual reconstructions
ROBOMECH Journalvolume 4, Article number: 22 (2017)
3D visual mapping of the seafloor has found applications ranging from environment monitoring and survey of marine minerals to underwater archaeology and inspection of modern artificial structures. However, the attenuation of light is significantly more pronounced in water than in air or in space, and so in order to obtain underwater images in colour, it is typically necessary to be within 2–3 m of the seafloor. In addition to the high risk of collision when operating underwater vehicles at such low altitudes, the limited area of the seafloor covered in each image means large area surveys require a significant investment of time. In this research, we aim to increase the efficiency of mapping large areas of the seafloor by developing an underwater imaging system that can take colour images at ranges of more than 10 m, so that each image can cover a larger area, together with the necessary algorithms to automatically process the data it obtains. The system was deployed to map artificial hydrothermal vents in Iheya North field using the ROV Hyper-Dolphin in October 2012. The surveyed area is of particular interest to the research community, as multiple artificial vent holes were drilled during a mission in 2010, which locally impacted the flow of hydrothermal fluids. In this paper, we describe the methods used to process the data that the imaging system obtains and demonstrate how the mapping data can be used in quantitative studies of the seafloor. Habitats of Shinkaia crosnieri squat lobsters, which are abundant in the hydrothermally active areas, are identified in the maps and their population density calculated, and the amount of hydrothermal deposits that have grown on the artificial vent is derived from the mapping data. The work demonstrates how 3D visual mapping can be applied to benthic biology and geological studies.
Visual mapping of the seafloor is a versatile tool for underwater research with applications including benthic habitat mapping [1,2,3], survey of mineral resources , and mapping of archaeological sites . In particular, the 3D imaging techniques developed in recent years can generate high resolution digital reconstructions of the seafloor [3, 4, 6, 7]. The addition of colour information enhances the representation of the seafloor compared to conventional bathymetric mapping techniques, and potentially allows for a greater diversity of classification and statistical analysis of features of interest. As such it also is a versatile tool for assessing and monitoring the human impact on the sea bottom environment.
The application of ecological survey and surveying slow changes in highly heterogeneous environments such as deep-sea hydrothermal vent fields can be broken down into the following three, independent points. The mapping system must:
Have sufficiently high resolution and colour fidelity to quantitatively measure physically small (centimetre order) features and correctly identify animals or deposits from their morphology and colour.
Achieve sufficient area coverage (in the order of hectares) to collect data that is representative of the targeted environment.
Be robust to changes in measurement conditions caused by the imperfect altitude control of the robotic platform due to rugged underwater terrains and underwater currents and the variable turbidity of seawater which affect colour attenuation.
Collecting meaningful ecological and geological data in hydrothermal environments over statistically relevant spatial scales requires all three of the above points to be fulfilled, which is not possible with existing underwater imaging systems and algorithms used to process their data. This study overcomes these technical challenges and realises such a system by extension of the long-baseline structured light and colour mapping algorithm described in  to use imagery from a separate, high-sensitivity colour camera synchronised with a high-power strobe. This extension allows us to achieve centimetre order resolution 3D bathymetric measurements and retain colour information through the use of the strobed images taken from significantly higher altitudes than was previously possible (10 m instead of 2 m). The ability to map and image from high altitudes at the necessary resolution and colour fidelity makes it possible to cover significantly larger areas than was previously possible (1000–10,000 m2/h) making it feasible now to map an entire hydrothermal site in a single autonomous underwater vehicle (AUV) or remotely operated vehicle (ROV) deployment.
The system was deployed in the hydrothermally active Iheya North field to map megabenthos  and hydrothermal deposits  around and on top of artificial hydrothermal vents. The Iheya North field is a kilometre scale active deep-sea hydrothermal field off the west coast of Okinawa, at a depth of approximately 1000 m . It was drilled during the Integrated Ocean Drilling Program (IODP) 331 “Deep Hot Biosphere” expedition during September and October 2010 [11, 12]. Surveys that followed the IODP 331 drilling campaign reported noticeable changes in the seafloor landscape and growth of mineral deposits near the drill site . In addition, significant increases in the amount of mega benthos were reported near the C0014 drill site which was previously not inhabited . While it is clear that the changes in the environment observed following a human drilling effort have direct relevance when considering the future deep-sea mining, the observations made were either qualitative in nature or highly localised and it is clear that there is a need to document these effects quantitatively over spatial scales that are sufficient to draw meaningful conclusions regarding the effect on the entire hydrothermal site in order to inform future decisions. In order to illustrate how this can be practically achieved, we introduce a method for achieving wide-area 3D visual reconstruction and demonstrate its application by quantitatively mapping the distribution of Shinkaia crosnieri squat lobsters, which is a representative species of megabenthos in the Iheya North Field and was reported in large numbers at the drill affected area , and also demonstrate how the growth of deposits in the drilled areas can be quantitatively determined from the high resolution bathymetry obtained by this technology.
During the NT12-27 cruise of the Japan Agency for Marine Science and Technology (JAMSTEC), the long range “SeaXerocks 2” mapping system developed in this work was mounted on the ROV Hyper-Dolphin to obtain 3D imagery of the seafloor between three artificial hydrothermal vents and the North Big Chimney (NBC) mound. 3D colour reconstructions of the seafloor have been generated using the mapping algorithm described in , that has been adapted for long range mapping using multiple synchronised flashes and a high power sheet laser for seafloor illumination and bathymetric measurements, respectively.
To date, surveys of this type were performed in a series of hierarchical stages that lose a significant amount of information at each stage. Typically, broad scale site surveys relied on acoustic methods, such as side-scan sonar and multibeam sonar surveys that can cover entire, multi-hectare hydrothermal sites. However the resolution of the data collected by these systems is in the order of 0.5–2 m and does not contain colour information. It is not possible to measure growth that takes place in these environments due to the slow rates (<1 m/year) and biota (<0.5 m) cannot be recognised. The information is augmented typically by small footprint sampling or imaging surveys of regions that are smaller than 1000 m2 in various, characteristic areas identified by the acoustic surveys. This type of data however, can only give statistical information about the relative abundance of animals and cannot be used for quantitative studies or for temporal comparisons on the scale of a entire site. The technique described is transformative from the viewpoint of its application to mapping hydrothermal sites because it gives biologists and geologists the ability to perform quantitative whole-site surveys from a single AUV or ROV deployment.
Mapping device and generation of 3D maps
The mapping method, illustrated in Fig. 1, is based on laser profiling extended to use colour. In this study, we extend the method described in our previous work  in order to achieve mapping from significantly higher altitudes than was previously possible. In our previous work, the algorithm assumed a single camera to capture both the laser line projection and the illuminated area of the seafloor. While this is efficient in terms of hardware requirements, it also means that the choice of camera and method for illumination become limited. In order to achieve mapping from higher altitudes, the requirements for collecting images of the laser projection and those required for imagery diverge. First, images of the laser profile must be collected at a fast frame rate to maintain a high resolution in the forward direction. Since the mapping altitude is high, it is desirable to have a large baseline between the laser and the camera observing the laser projection to achieve a high resolution in the vertical direction, which typically requires the camera and the laser to be at opposite ends of the vehicle. Since the ability to map from high altitudes also makes mapping possible over more rugged terrains, it is also desirable to tilt the camera towards the laser, in order to maintain a large range of altitudes over which the projection is inside the cameras’ field of view. The requirements for collecting high quality imagery are fundamentally different. First, the footprint of an illuminated image is large, and scales linearly with altitude. Therefore, for the large footprints made possible at high altitude, it is sensible to record images at a relatively slow repetition rate and use strobes delivering a more powerful impulse of light than continuous lights are capable of. Second, the rugged shape of the terrain means that the camera becomes susceptible to occlusions in areas with large profiles, and that shadows formed by the terrain can be detrimental. In order to minimise the effect of shadowing, it is desirable to have multiple strobes in an array with the camera central and looking vertically downwards. Therefore, unlike the camera used for laser detection, the best position to have the camera for imaging is in the centre of the vehicle, with the strobes located at the corners of the vehicle. In order to make this possible, the algorithm was extended to process imagery data from a dedicated camera and model the light path of any number of strobes located at arbitrary locations around the camera, to allow high colour fidelity reconstructions to be generated.
The mapping system that was used is shown in Fig. 2. It consists of high sensitivity cameras to collect seafloor images illuminated by four synchronised Xenon flashes, with a total illumination of \(\sim\)3 klms emitted per pulse, distributed around the camera, and a separate camera to capture the sheet laser projection at a high frame rate. The setup allowed us to make full use in exploiting the advantages offered by the modified mapping algorithm. The laser profiling system employs a 532 nm, 120 mW sheet laser, offset 1.027 m from the laser line camera (tilted) that captures images of the laser line projection at a rate of 15 frames per second. This enables the setup to acquire shape information from high altitudes. It is robust in moderately turbid deep-sea environments because there is no backscatter from continuous lights, which causes particles to appear blurred and larger than their actual dimensions, and the laser line provides a clear visual feature that is easy to automatically extract. The camera mounted vertically is synchronised with the flashes and captures images from which colour is applied to the 3D reconstruction.
Navigation data recorded from a depth sensor and a Doppler velocity log (DVL) with a compass and a tilt sensor are used in the algorithm to determine the relative position from which the images were taken. The surface vessel’s super short baseline (SSBL) acoustic positioning system is used to georeference the 3D map. The mapping device is rated to 3000 m depth and can be deployed by an AUV or ROV.
During mapping, the sheet laser projects a line perpendicular to the direction of motion on the seafloor. This is captured by a camera with a translational offset to the laser and is used to calculate the vertical profile of the illuminated section. As the vehicle moves, the seafloor is scanned and the set of images of the laser line projection allow the bathymetry of the scanned area to be generated. With the setup used during the cruise to the Iheya North field, the resolution along the track is 20 mm, across the track 7 and 56 mm in vertical direction when mapping from an altitude of 8 m at a forward speed of 30 cm/s. From 10 m altitude the resolution is 20 mm along track, 9 mm across track and 87 mm in vertical direction. While the resolution of the data obtained in the forwards direction does not change with altitude, the localisation accuracy of the seafloor measurements reduces with altitude due to the inherent uncertainty of the pitch and roll sensors on the underwater vehicle and inherent uncertainties in the calibration of the lens, both of which increase the position uncertainty of the point measured on the seafloor in Cartesian coordinates. While the cumulative effects of these are not negligible, both effects scale linearly with altitude and are in the order of centimetres at an altitude of 8 m.
Flashed photos are taken every 6 s, leading to an overlap of 75% at an altitude of 8 m, from which the colour is applied to each bathymetry point in order to obtain a coloured 3D digital representation of the seafloor. Data collected during each dive is processed off-line by the algorithm whose basic steps are illustrated in Fig. 3. Since the flash imagery from which colour information is retrieved and the images of the laser line projection are asynchronous, an algorithm was developed to associate colour information from each point of bathymetry based on the vehicle’s localisation estimate. The location of the camera with flashed imagery is determined from the vehicle’s position and the geometry of the camera setup. Based on this location and the camera calibrations, it is possible to find the projection of each pixel in the illuminated images on the seafloor, and so determine the colour information that corresponds to each point of bathymetry measured. The colour values are corrected for the effects of wavelength dependent attenuation and the beam pattern of the flashes. Since it is likely that the bathymetry point appears in more than one flashed image, a weighted average is computed, where values from images where the backprojection of the bathymetry point is closer to the centre of the image have a larger weight than corresponding pixels close to the border of the image in order to minimise the effects of uncertainty in the calibration and light attenuation model. The so generated coloured point cloud is then converted into a mesh using Delaunay triangulation and output as a ply file, which can be easily viewed and lends itself well to further processing of the data to directly extract the numerical information needed for scientific analysis. Based on measurements of known features in the 3D reconstruction, it can be determined that the dimensional accuracy is \(\sim\)12.5%.
Results from deployment at sea
On the NT12-27 cruise, mapping operations were performed using the ROV Hyper-Dolphin from the research vessel Natsushima for 1 h. Compressed images were sent up over a RS-232 data link for checking the image quality in real time. A total of 13,532 images were obtained along a 920 m trajectory from an average altitude of 7.7 m, resulting in a seafloor data acquisition rate of 8144 m\(^2\)/h at an average pixel resolution of 6.4 mm. This represents an order-of-magnitude increase in the area data acquisition rate compared to conventional colour imaging of the seafloor where images are obtained from an altitude of 2–3 m [1, 4]. The top-view of the 3D map generated from this data overlaid on an acoustic bathymetry map is shown in Fig. 4, with enlargements of the three artificial hydrothermal chimneys C0013E, C0014G and C0016B.
In the mapped area accumulations of hydrothermal deposits, as well as numerous S. crosnieri squat lobsters  are visible. Both the deposits and S. crosnieri, which feed on methane and sulfide reducing bacteria are indicators of hydrothermal activity. The mapping data was used to quantify both the volume of deposits as well as the number and distribution of S. crosnieri in the mapped area.
Mapping megabenthos distribution
Figure 5 shows the C0014G artificial hydrothermal vent and its surroundings. The reconstruction was generated from a single pass over the artificial chimney from an altitude of about 10 m and it covers an area of 37.5 m by 15.5 m. This was mapped in 2 min 40 s. The white hydrothermally altered region extends west of the vent for a distance of 24 m, and is densely populated by S. crosnieri, visible in enlargement A. Their appearance in the map is relatively small, but the species was determined from sampled individuals and they were also identified in ROV footage taken from closer range. It is also known that there are no other animals of similar size or colour living in this specific area. The depression seen 4.7 m west of the artificial vent, enlarged in detail view B, is drill hole C0014D/E made during the IODP 331 expedition and large numbers of S. crosnieri are also seen inside of it.
The S. crosnieri were identified in a top view of the 3D reconstruction and manually labelled. From this data the population density maps for the area around the natural and the three artificial hydrothermal vents were generated using a 2D Gaussian window with a standard deviation of 15 cm. Figure 6 shows the 3D map of chimney C0014G with the S. crosnieri population density overlaid. The total number of S. crosnieri found in the area was 3140 and the highest concentration was 349 individuals/m2. The map shows that the highest population densities are not necessarily found nearest the artificial vent but rather in the surrounding areas. The histogram of the number of S. crosnieri in relation to the distance from the mouth of the artificial chimney shown in Fig. 7 shows that the majority of S. crosnieri are found at a distance of 6–14 m from the chimney. There was a particularly high population density of S. crosnieri in and around the drill hole C0014D/E (feature B in Fig. 5) west of the artificial chimney. It is possible that there may be hydrothermal diffuse flows seeping up through the ground in these areas, which are more suitable for supporting sea life than the strong flow gushing out of the artificial vent [14, 16].
Figure 8 shows the top view of the artificial chimney C0016B with its surroundings and Fig. 9 shows the population density maps of S. crosnieri. There are significantly less S. crosnieri than around chimney C0014G (261 individuals at C0016B vs. 3140 at C0014G) and they are mainly found close to the base of the artificial chimney. The histogram shown in Fig. 10 also shows that the peak density is closer to the artificial chimney than in the case of C0014G.
Figure 11 shows the top view of a transect going over the NBC mound and Fig. 12 shows the population density map of S. crosnieri in this area. A total number of 5025 S. crosnieri were identified in the 3D reconstruction with a maximum density of 210 individuals/m2. The actual total population is likely to be larger, because some parts of the 3D reconstruction are not clearly visible due to saturation in some of the photos, in particular around the peak and the reconstruction only covers part of the chimney.
Around the artificial chimney C0013E, shown in the enlargement at the bottom left of Fig. 4, no S. crosnieri were found. Unlike C0014G and C0016B the vent is not active  and so it cannot support chemosynthetic primary production. Table 1 summarises the numerical data of the S. crosnieri population around the three artificial and the one natural hydrothermal vent (NBC).
Deposit volume and mass estimation
Among the three artificial hydrothermal vents, chimney C0016B has the largest amount of deposit growing on top of the artificial structure. Before the area was mapped during the NT12-27 cruise deposits at this location were sampled by the ROV, meaning that little deposit was left at the time of mapping. There are also some deposits on the seafloor directly next to the artificial chimney. This was not there when the chimney was installed in September 2010 and so it can be concluded that this has grown in the two year period between when the site was drilled and mapped .
An algorithm was developed that analyses the 3D outputs of the mapping system and calculates the volume of protrusion of a feature. It was used to compute the amount of deposit on the top plate of chimney C0016B. Figure 13 shows the elevation map of the top plate based on the mapping data. A photo of the top plate taken using the ROV forward looking camera is also shown. The deposit on the top plate has a volume of 0.032 ± 0.007 m3.
Based on hydrothermal deposits sampled by the ROV, the dry density was measured to be 2.6 g/cm3 with a standard deviation of 0.16 g/cm3. Similar values were reported in . Using the above-mentioned density measurements, the dry mass of deposits on the chimney top is estimated to be 83 ± 18 kg.
A long range 3D imaging system has been demonstrated that can generate 3D colour reconstructions of the seafloor from altitudes of more than 10 m. The data presented shows that the method allows the seafloor to be mapped both rapidly and with high resolution. The system is as a powerful tool to study the marine environment on hectare order scales. Entire vent colonies can potentially be mapped in a short time , and the resolution it achieves is sufficient to map deposits protruding by as little as a few centimetres from their surroundings. The applications to quantify megabenthos and deposit volumes illustrate the opportunities high resolution visual and bathymetric data offers scientific studies. It is also hoped that data from repeated surveys will help document the changes of the environment over a longer time span and so make it possible to better assess the impact of human drilling activities in deep-sea environments.
Future directions for the technology relate mainly to the interpretation of data and verification of measurement precision and repeatability. In particular, the data collection for this survey took 1 h and the processing computational time is in the order of a few hours. However, the manual labelling of biota took five person days to achieve and is the obvious bottleneck in the analysis. Methods to assist our interpretation of the data through partial automation can allow for more efficient interpretation and so push forward our understanding of deep-sea processes. With regard to the precision and repeatability of the method, it is necessary to perform repeat surveys and analyse the uncertainties introduced at the various stages of data collection, processing and interpretation. This type of data will be crucial for future, temporal studies in order to identify trends and changes in environments as the minimum unit of change will be strongly coupled to the repeatability of the method described.
autonomous underwater vehicle
Doppler velocity log
remotely operated vehicle
Ludvigsen M, Sortland B, Johnsen G, Singh H (2007) Applications of geo-referenced underwater photo mosaics in marine biology and archaeology. Oceanography 20(4):140–149. doi:10.5670/oceanog.2007.14
Pizarro O, Friedman A, Bryson M, Williams SB, Madin J (2017) A simple, fast, and repeatable survey method for underwater visual 3D benthic mapping and monitoring. Ecol Evol 7(6):1770–1782. doi:10.1002/ece3.2701
Johnson-Roberson M, Pizarro O, Williams SB, Mahon I (2010) Generation and visualization of large-scale three-dimensional reconstructions from underwater robotic surveys. J Field Robot 27(1):21–51. doi:10.1002/rob.20324
Bodenmann A, Thornton B, Ura T (2015) Generation of high-resolution three-dimensional reconstructions of the seafloor in colour using a single camera and structured light. J Field Robot 34(5):883–851. doi:10.1002/rob.21682
Singh H, Whitcomb L, Yoerger D, Pizarro O (2000) Microbathymetric mapping from underwater vehicles in the deep ocean. Comput Vis Image Underst 79(1):143–161. doi:10.1006/cviu.2000.0850
Pizarro O, Eustice R, Singh H (2009) Large area 3-d reconstructions from underwater optical surveys. IEEE J Ocean Eng 34(2):150–169
Garcia R, Campos R, Escartín J (2011) High-resolution 3D reconstruction of the seafloor for environmental monitoring and modelling. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems, San Francisco, CA, pp 58–65
Bodenmann A, Thornton B, Nakajima R, Yamamoto H, Ura T (2013) Wide area 3D seafloor reconstruction and its application to sea fauna density mapping. In: Proc. of MTS/IEEE oceans, San Diego, USA
Bodenmann A, Thornton B, Ura T (2014) Visual 3D mapping to measure hydrothermal deposit growth rates at a man-made deep sea vent. In: Proceedings of 2014 IEEE/OES autonomous underwater vehicles (AUV). IEEE, Oxford. doi:10.1109/AUV.2014.7054421
Masaki Y, Kinoshita M, Inagaki F, Nakagawa S, Takai K (2011) Possible kilometer-scale hydrothermal circulation within the Iheya-North field, mid-Okinawa Trough, as inferred from heat flow data. JAMSTEC Rep Res Dev 12:1–12. doi:10.5918/jamstecr.12.1
Expedition 331 Scientists (2010) Deep hot biosphere. IODP Preliminary Report 331. doi:10.2204/iodp.pr.331.2010
Takai K, Mottl MJ, Nielsen SHH, IODP expedition 331 scientists (2012) IODP expedition 331: strong and expansive subseafloor hydrothermal activities in the Okinawa Trough. Sci Dril 13:19–27. doi:10.2204/iodp.sd.13.03.2011
Kawagucci S, Miyazaki J, Nakajima R, Nozaki T, Takaya Y, Kato Y, Shibuya T, Konno U, Nakaguchi Y, Hatada K, Hirayama H, Fujikura K, Furushima Y, Yamamoto H, Watsuji T-O, Ishibashi J-I, Takai K (2013) Post-drilling changes in fluid discharge pattern, mineral deposition, and fluid chemistry in the Iheya North hydrothermal field, Okinawa Trough. Geochem Geophys Geosyst 14(11):4774–4790. doi:10.1002/2013GC004895
Nakajima R, Yamamoto H, Kawagucci S, Takaya Y, Nozaki T, Chen C, Fujikura K, Miwa T, Takai K (2015) Post-drilling changes in seabed landscape and megabenthos in a deep-sea hydrothermal system, the Iheya North field, Okinawa Trough. PLoS ONE 10(4):1–22. doi:10.1371/journal.pone.0123095
Watsuji T-O, Nakagawa S, Tsuchida S, Toki T, Hirota A, Tsunogai U, Takai K (2010) Diversity and function of epibiotic microbial communities on the galatheid crab, Shinkaia crosnieri. Microbes Environ 25(4):288–294. doi:10.1264/jsme2.ME10135
Tokeshi M (2011) Spatial structures of hydrothermal vents and vent-associated megafauna in the back-arc basin system of the Okinawa Trough, western Pacific. J Oceanogr 67(5):651. doi:10.1007/s10872-011-0065-9
Nozaki T, Ishibashi J-I, Shimada K, Nagase T, Takaya Y, Kato Y, Kawagucci S, Watsuji T, Shibuya T, Yamada R, Saruhashi T, Kyo M, Takai K (2016) Rapid growth of mineral deposits at artificial seafloor hydrothermal vents. Sci Rep 6:22163. doi:10.1038/srep22163
Thornton B, Bodenmann A, Pizarro O, Williams SB, Friedman A, Nakajima R, Takai K, Motoki K, Watsuji T-O, Hirayama H, Matsui Y, Watanabe H, Ura T (2016) Biometric assessment of deep-sea vent megabenthic communities using multi-resolution 3D image reconstructions. Deep Sea Res Part I Oceanogr Res Pap 116:200–219. doi:10.1016/j.dsr.2016.08.009
AB developed the 3D visual mapping and data extraction algorithms, designed and made the electronics of the SeaXerocks 2 mapping device, wrote the control software of the mapping device, operated the mapping device at sea, processed the data. BT designed the mapping device, wrote data recording software for the mapping device, assisted the development of the mapping and data extraction algorithms, planned the mapping route and operated the device at sea. RN assisted on the cruise and identified and marked the S. crosnieri in the 3D maps. TU assisted with the development of the algorithm, conception and design of missions to collect data and secured funding for developing the hardware and deploying it at sea. All authors read and approved the final manuscript.
This work was supported by the Japan Science and Technology CREST grant program for the “Establishment of core technology for the preservation and regeneration of marine biodiversity and ecosystems”, Grant Number JPMJCR11A2.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.