AI and video helping salmon data collection in Indigenous communities

In a groundbreaking collaboration among Canadian First Nations, government bodies, academic institutions, and conservation organizations, a pioneering tool known as “Salmon Vision” is emerging as a “game-changer” in the monitoring of salmon populations.

Fisheries in British Columbia have grappled with data scarcity for decades, which forces managers to base harvest numbers on early-season catch data rather than the actual salmon return figures. Moreover, shifting weather patterns, stream flows, and ocean conditions introduce greater variability in salmon returns, compounding the existing risks of overfishing already-vulnerable populations.

It’s a global problem facing fish monitoring everywhere, so the Canadian innovation is valuable.

“Without real-time data on salmon returns, it’s extremely difficult to build climate-smart, responsive fisheries,” says Dr Will Atlas, the Senior Watershed Scientist with the Wild Salmon Center and lead author of the new study in the journal Frontiers in Marine Science.

Atlas says Salmon Vision fills this crucial information void, offering a tool for First Nation fisheries managers and other organizations both in decision-making contexts and in remote rivers across salmon territory where on-the-ground data collection is logistically challenging and expensive.

The technology combines artificial intelligence with traditional fishing weir techniques.

Wild salmon play a vital role in the social-ecological systems of the Northeastern Pacific Rim, but they face unprecedented challenges due to climate change. Their annual migrations from the ocean to freshwater habitats are not just a natural spectacle but a source of sustenance and cultural identity for Indigenous communities, as well as a vital resource for coastal communities.

But declines in salmon abundance and productivity have been observed, making salmon returns increasingly unpredictable. To compound these challenges, mixed-stock fisheries that harvest from multiple populations indiscriminately pose risks to salmon-centred ecosystems.

Declines in salmon abundance and productivity have been observed, making salmon returns increasingly unpredictable.

The lack of comprehensive monitoring data, high data collection costs, and mixed-stock fishing practices exacerbate these challenges. As a result, there is a growing need for real-time monitoring and adaptive management tools to enhance fishery and ecosystem resilience.

“Basically, we started video monitoring on a project I’ve been working on with the Heiltsuk Integrated Resource Management Department in 2019,” says Atlas.

“But we had way more video than our technicians could reasonably review (5 months of continuous video) so we thought, this would be a really good use case for AI.”

Artificial intelligence (AI), including computer vision and deep learning, has revolutionised data processing and analysis, finding applications in various fields. In recent years, AI has been increasingly used in animal ecology, conservation, and marine monitoring.

However, these cutting-edge tools have not always been integrated with rural, remote, and marginalised communities, limiting their benefits outside mainstream centres of power. To address these issues, an interdisciplinary team was formed to develop a computer vision model for automated salmon identification and counting. This collaborative effort involved computer scientists, fishery and conservation experts, and Indigenous and non-Indigenous practitioners.

The goal was to automate the counting and identification of salmon using underwater videos from salmon counting weirs (structures placed in rivers or streams to monitor and count the number of salmon migrating upstream).

The team developed and tested two computer vision models – a multi-object tracker (MOT) and a species detection mode – that work in parallel to accurately and efficiently count and identify individual salmon passing through the weirs.

“Yes basically, we developed a computer-vision model using existing open-source model architectures (YOLO family of models). These models were trained, tested, and fine-tuned to optimise their performance at detection, tracking, and identification of salmon from videos,” says Atlas.

“One of the key challenges for computer-vision is limiting the amount of variability in the videos. Fortunately for us, Gitanyow Fishery Authority and the Skeena Fishery Commission had been running video monitoring projects for a few years so we were able to borrow the video camera systems and trap box design from them, which is what we’re now using around the North and Central Coast of British Columbia, creating a fairly standard set up for the computer-vision model.”

We’ve seen the promise of underwater video technology to help us literally see salmon return to rivers.

Dr Will Atlas

“In recent years, we’ve seen the promise of underwater video technology to help us literally see salmon return to rivers. That dovetails with what many of our First Nations partners are telling us: that we need to automate fish counting to make informed decisions while salmon are still running.”

The Salmon Vision pilot study annotated more than 500,000 individual video frames captured at Indigenous-run fish counting weirs on the Kitwanga and Bear Rivers in British Columbia’s Central Coast.  This model showcases impressive accuracy in identifying various salmon species, boasting mean average precision rates of 67.6% in tracking 12 different fish species as they traverse custom fish-counting boxes at the two weirs.

Notably, the accuracy soars above 90% and 80%t for coho (Oncorhynchus kisutch) and sockeye salmon (Oncorhynchus nerka), two key species targeted by First Nations, commercial, and recreational fishers.

“When we envisioned providing fast grants for projects focused on Indigenous futurism and climate resilience, this is the type of project that we hoped would come our way,” says Dr Keolu Fox, a professor at the University of California-San Diego, and one of several reviewers in an early crowdfunding round for the development of Salmon Vision.

“Our findings highlight the potential for computer vision to advance greater sustainability in salmon fisheries, however further work will be needed to put these tools in the hands of conservation practitioners and fishery managers,” the authors explained in their paper.

Thus, the Salmon Vision team is embarking on a trial implementation of automated counting in several rivers around the British Columbia North and Central Coasts in 2023, with the aim of providing reliable real-time count data by 2024. They hope their groundbreaking technology has the potential to transform the conservation and management of salmon populations and the sustainability of fisheries across the region, providing a powerful and timely solution to a long-standing challenge.

The team also believes in the use of this technology for other applications, with Atlas concluding: “I think the tools and insights we’re building could be applied to other challenges with training data and expert knowledge of what questions need to be answered by practitioners in those fields.

“I know for example, computer-vision is being tested to monitor bycatch, to estimate tree stand composition, to identify wildlife species in trap camera images, and to count and identify fish species from underwater surveys to name a few.

“We’re pretty focused on salmon monitoring, but computer-vision holds immense promise in conservation generally, especially as we move into an era of big data with the broader application of remote sensing, video, and other forms of continuous monitoring.”