FathomNet collects images from MBARI and other institutions to create an expertly curated, publicly available underwater image training database. FathomNet will help unleash the power of artificial intelligence to accelerate ocean image processing. Photo: © 2020 MBARI
A new collaborative effort between MBARI and other research institutions is harnessing the power of artificial intelligence and machine learning to accelerate efforts to study the ocean.
In order to manage the impacts from climate change and other threats, researchers urgently need to learn more about ocean populations, ecosystems, and processes. As scientists and engineers develop advanced robots that can visualize marine life and environments to monitor changes in ocean health, they face a fundamental problem: the collection of images, video and other visual data far beyond the researchers’ ability to analyze.
FathomNet It is an open source image database that uses the latest data processing algorithms to help handle visual data backlogs. The use of artificial intelligence and machine learning will alleviate the bottleneck of underwater image analysis and accelerate critical research on ocean health.
“The big ocean needs big data. Researchers collect large amounts of visual data to monitor life in the ocean. How can we process all this information without automation? Machine learning provides a way forward, but these methods rely on huge data sets for training. FathomNet has been built To fill this gap,” said MBARI Principal Engineer Kakani Katiga.
Project founders Katiega and Katie Crove Bell (Ocean Discovery League) and Ben Woodward (CVision AI), along with members of the expanded FathomNet team, detailed the development of this new image database in a recent paper. Published in Scientific Reports.
Recent advances in machine learning allow for rapid, sophisticated analysis of visual data, but the use of AI in ocean research has been limited by the lack of a standard set of existing images that can be used to train machines to recognize and classify underwater objects. . and life. FathomNet addresses this need by compiling images from multiple sources to create an expertly curated, publicly available underwater image training database.
“In the past five years, machine learning has revolutionized the automated visual analysis landscape, driven in large part by massive sets of labeled data. ImageNet and Microsoft COCO are two standard datasets for terrestrial applications that machine learning and computer vision researchers flock to, but we haven’t even begun to Scratching the surface of the machine learning capabilities of underwater optical analysis”. Founder and CEO of Cvision AI and co-founder of FathomNet. “With FathomNet, we aim to provide a rich and interesting standard for engaging the machine learning community in a new field.”
Over the past 35 years, MBARI has recorded nearly 28,000 hours of deep-sea videos and collected more than 1 million deep-sea images. This set of visual data is explained in detail by research technicians at the MBARI Video Lab. MBARI’s video archive includes nearly 8.2 million annotations that record observations of animals, habitats, and objects. This rich data set is an invaluable resource for researchers at the Institute and collaborators around the world.
FathomNet includes a subset of the MBARI dataset, as well as assets from National Geographic and NOAA.
The National Geographic Society’s Exploration Technology Laboratory has published versions of its independent benthic landing platform, Deep Sea Camera System, since 2010, collecting more than 1,000 hours of video data from locations in all ocean basins and in a variety of marine habitats. These videos were subsequently assimilated into Cvision AI’s cloud-based collaborative analysis platform and annotated by subject specialists at the University of Hawaii and OceansTurn.
National Oceanic and Atmospheric Administration (NOAA) ocean exploration began collecting video data using a remotely operated vehicle system aboard the NOAA Ship Okeanos Explorer in 2010. More than 271 terabytes are archived and publicly accessible from the NOAA National Environmental Information Centers (NOAA). NCEI). Originally NOAA Ocean Exploration annotations were collectively sourced through volunteer participating scientists, and began supporting taxonomists in 2015 to add annotations to videos collected more comprehensively.
“FathomNet is a great example of how collaboration and community science can advance breakthroughs in how we recognize the ocean,” said Lonny Lundsten, senior research technician at MBARI’s Video Lab, co-author, and FathomNet team member.
As an open source web-based resource, other organizations can contribute to and use FathomNet in place of the traditional resource-intensive effort of visual data processing and analysis. MBARI has launched a pilot program to use machine learning models trained by FathomNet to annotate video captured by remotely operated underwater vehicles (ROVs). The use of AI algorithms reduced human effort by 81 percent and increased the tagging rate tenfold.
Machine learning models trained with FathomNet data also have the potential to revolutionize ocean exploration and monitoring. For example, equipping robotic vehicles with their cameras and improved machine learning algorithms could enable automated search and tracking of marine animals and other underwater objects.
“Four years ago, we envisioned using machine learning to analyze thousands of hours of ocean videos, but at the time, that was basically not possible due to a lack of annotated images. Now FathomNet will make that vision a reality, unleashing discoveries and enabling tools that Explorers, scientists and the public can use it to speed up the pace of ocean discovery.” -Founder.
As of September 2022, FathomNet contained 84,454 images, representing 175,875 locations from 81 separate groups for 2,243 concepts, with additional contributions continuing. FathomNet aims to obtain 1,000 independent observations of more than 200,000 animal species in a variety of shooting poses and conditions – ultimately more than 200 million total observations. For FathomNet to reach its intended goals, significant community participation – including high-quality contributions across a wide range of groups and individuals – and broad use of the database will be required.
“Although FathomNet is an API-based web-based platform where people can download labeled data to train new algorithms, we also want it to serve as a community where ocean explorers and enthusiasts of all backgrounds can contribute their knowledge and expertise and help solve challenges related to ocean visual data which would be impossible without widespread participation,” Kateja said.
To join the FathomNet community, visit fathomnet.org and follow FathomNet on Twitter.
Initial funding for FathomNet was provided by the National Geographic Society (#518018), the National Oceanic and Atmospheric Administration (NA18OAR4170105), and MBARI through the generous support of the David and Lucile Packard Foundation. Additional funding support was provided by the National Geographic Society (NGS-86951T-21) and the National Science Foundation (OTIC #1812535 & Convergence Accelerator #2137977).
About MBARI
MBARI (Monterey Bay Aquarium Research Institute) is a private, not-for-profit oceanographic research center founded by David Packard in 1987. MBARI’s mission is to advance marine science and technology to understand the changing ocean. Learn more at mbari.org.
FathomNet: A global image database to enable artificial intelligence in the oceanScientific Reports (open access)
Astrobiology, Extrasolar System, Exoplanet, Artificial Intelligence,