Cameras mounted on fishing boats combined with advanced machine learning can help better monitor legal and illegal catches, advocates say.
The numbers are shocking.
Overexploitation of the world's fish stocks is growing at an alarming rate, says the U.N. Food and Agriculture Organisation (FAO).
Nearly 90% are either at or exceeding sustainable capacity, and in less than 10 years, production is set to grow by 17%.
One of the keys to sustainable fisheries has been the employ of human monitors to watch what is being scooped up from the sea. In the United States, fishing boats are routinely accompanied by independent observers who track compliance with fishing regulations.
In other countries' waters, it's a whole different story, so those government and independent agencies hoping to halt overfishing are turning to some of the same digital tools that let social media sites recognize faces in photos.
In the region of the Pacific Ocean from Indonesia and the Philippines to Hawaii—the source of the majority of the world's tuna harvest—a mere 2% of fishing operations are watched by observers, says Mark Zimring, director of The Nature Conservancy's Indo-Pacific tuna program.
That makes it harder for scientists to understand the effects fishing is having on imperiled species in the area, and that means large numbers of valuable fish—including the yellowfin and bigeye tuna varieties found in sushi restaurants around the world—are being harvested illegally, he says.
But by effectively automating part of the job of the observer by using cameras to record what creatures are caught and sophisticated software to classify them by species, regulators would be able to get a fuller picture of legal harvests and detect unlawful operations.
Just as automation and machine learning have given internet companies detailed records and predictions of how users behave online, they can potentially enable scientists and government agencies to build similarly detailed models of the world's fisheries.
"Today it's estimated that what's called illegal, unreported, and unregulated fishing costs the region between a half billion dollars and $1.5 billion a year," Zimring says. And even low rates of accidental capture and killing of important predators such as sharks can have significant impacts on aquatic ecosystems, he says.
"We really need to understand, from a kind of science perspective, when at-risk species like sharks and turtles are being caught, and what really happens to them," he says.
The Nature Conservancy is working with governments in the region, including in Palau, Micronesia, the Marshall Islands, and the Solomon Islands, to implement alternative monitoring programs, capturing video footage of fishing vessels instead of placing observers on each boat. The group has equipped about a dozen boats with cameras so far, with plans to place more this year.
Footage is stored on hard drives that can be removed for analysis when boats come into port. But that method still produces huge amounts of raw video that need to be manually analyzed by hand. To make analyzing the footage more feasible, The Nature Conservancy is investigating ways to use machine learning techniques to process that video material.
"You've got to be able to translate that raw video data into useful information—that's where we think machine learning can really help," says Zimring. The group is sponsoring a $150,000 challenge on machine learning competition platform Kaggle, seeking algorithms that can optimally classify types of sea creatures caught by fishing boats.
Even if automated methods can't completely replace human analysis, they can still provide a big benefit if they can reduce the amount of footage people need to review, saving time and potentially making it possible to transmit the video material more efficiently, Zimring says.
Similarly, the National Oceanic and Atmospheric Administration (NOAA) is investigating using electronic monitoring in fisheries across the U.S. from the Atlantic to the Pacific. One pilot program is placing cameras on board smaller boats in Alaska that catch fish like cod and halibut.
"It's been difficult to get observers onto these boats, and we've been working with them to develop electronic monitoring as an alternative," says Chris Rilling, director of the Fisheries Monitoring and Analysis Division at NOAA's Alaska Fisheries Science Center. Right now, fishing boats participating in a voluntary pilot program are equipped with cameras and with hard drives that are periodically shipped to NOAA for human review, but the agency is developing sets of training data for machine learning algorithms to use, with an eye toward integrating automated classification around 2019.
"That training data set has to come from the fishery that you're worried about, because you need to understand the complexity of the species," says NOAA researcher Farron Wallace. And even if the machine learning algorithms aren't perfect, they can still provide valuable information, since scientists will be able to test the inherent uncertainty in the data they produce, he says.
NOAA is also experimenting with ways to electronically monitor fish populations while they're still underwater. One device debuted in 2015 uses a low-powered computer and sonar system attached to the seafloor to automatically survey fish populations.
"These instruments wake up and send out pings, and we measure the energy that's reflected, and that tells us about how many fish are there," says NOAA biologist Alex De Robertis. When the device's work is done, NOAA researchers send it a specific audio signal that causes it to detach from the sea floor using attached flotation devices to rise to the surface.
This summer, the agency plans to place some of the devices in a remote area of the Arctic usually difficult to access without expensive icebreaker ships during the winter, and leave them in place to track fish populations and movements after the winter freeze.
"I think the real strength of this is to look over long time periods," says De Robertis. "It's ice-covered and very hostile and very expensive to work in there in the winter time."