Mathematical modelling: A dark art comes to light

You could say that mathematical modelling’s moment had arrived. The coronavirus had reached Australia, state health officials were quickly becoming the leaders we didn’t know we needed and a mathematician-turned-journalist named Casey Briggs was appearing daily on the national news. With a pandemic upon us, infectious disease modelling was cast into the public spotlight, warts and all.

But long before the pandemic emerged, mathematical models were used to understand, then simulate, complex systems. Virtual worlds were made a playground for testing different scenarios, policies and interventions to work out the right move to make, whether it be in conservation, clean energy or climate – or simpler still, to understand how flocking birds fly.

And yet as indispensable as modelling has proved to be in a pandemic, systems science hasn’t been readily adopted where arguably it’s needed most. “We just don’t see that same approach to mental health,” says Associate Professor Jo-An Occhipinti, head of Systems Modelling, Simulation and Data Science at the University of Sydney’s Brain and Mind Centre. Leading a research group focused on mental health and suicide prevention, Occhipinti develops models of healthcare systems to help decision makers “flatten the mental health curve” – and to think clearly in a crisis so that decisions about funding and essential services aren’t misplaced.

Modelling an illness

Infectious disease experts use two types of models to simulate the spread and control of a disease.

The first, called a system dynamics or compartmental model, groups people into buckets, based on health status or risk profile, and computes the rate of flow between each. People who are susceptible to contracting an infectious disease or developing a mental health condition (that’s one bucket), become infected or distressed (that’s another), seek care, then make a full recovery, remain chronically unwell or die with the disease or by suicide. With compartmental models, whole health systems can be recreated to predict how a crisis unfolds.

Behavioural trends of an entire population are quite predictable and can be described in mathematical terms

The other approach is to simulate a system from the ground up, by programming virtual people and watching what behavioural patterns emerge when they interact, in a so-called agent-based model. Though individual behaviours may be practically impossible to predict, behavioural trends of an entire population are quite predictable and can be described in mathematical terms. And just as a fluid can be described in two ways – as the bustling particles that bump along together or as a smooth, free-flowing substance – agent-based models and their compartmental counterparts simulate the same system two different ways.

Occhipinti’s team uses both kinds of models in their work, with upsides and drawbacks apiece. But modelling is not a dark art as people might think, says team member Dr Danya Rose, a population mathematician. In the past, Rose has developed agent-based models of how social dynamics within a community shapes human evolution and influences bushfire evacuations.

Rose says making a model – in this case, a map of the different pathways that people take through the mental healthcare system, as they bounce between services or in and out of care – is a process of paring down a system to its core components and rebuilding it one part at a time, adding mathematical formulas shaped by data insights that attempt to describe how it all moves.

“If we can represent the mechanisms [of a complex system] realistically, then we have a potentially useful model,” says Rose. “If we can tune it with the right data, to represent the population we want to model, we’ll get answers that aren’t entirely wrong.”

Rose says that the team is constantly checking that its assumptions remain true by testing that their model can reproduce historical data.

“We’re aware that no model is perfect – whatever that might mean – but by capturing the core essences of a dynamical system, we can get results that are faithful to it. And this faithfulness is tested through validation against historical data.”

Feeding into their simulations are streams of data on ED visits and hospital stays, specialist appointments and waitlist times, online support programs and community clinics. Research evidence describing risk factors for mental illness and suicidal behaviour also shapes the system, along with outcome data on the likely success of treatment interventions.

Taking it one step further, the team also invites people who work in or have sought mental health care, to refine their models with insights that can’t be gleaned from published papers or public datasets. “We’re not modelling an ideal mental health system, what we’re modelling is the reality of what happens on the ground,” says Occhipinti. “And that’s vitally important if we’re going to have a really valid and robust decision-support tool at the end.”

But the way these models are built, using demographic data from particular regions and figures on healthcare services in that area, means they’re not necessarily generalisable from one place to the next (unless adjusted with other data) because access to healthcare and factors impacting mental health vary. And where agent-based models can replicate individuals as a mixed bag, compartmental models can’t, though they can swiftly simulate the bulk flow of people through a healthcare system, tens of thousands of people at a time.

Put together, these models are a useful tool for health officials and policy makers, guiding them on where to focus finite resources, to get the biggest mental health boon for our collective buck, and how to rethink their investments. Running simulations can also help users appreciate what the rebound effect might be if a service were to be taken away or scaled back, and to recognise that funding a particular intervention, such as an awareness campaign, might actually create a bottleneck in another part of the system if the capacity isn’t there.

Crossing disciplinary borders

Dr Fabio Boschetti is an applied mathematician with CSIRO’s Ocean and Atmospheres division who has modelled networks of connectivity between fringing coral reefs that line the north Western Australia coast, near World Heritage-listed Ningaloo Reef.

One particular model, based on data from ecological surveys and designed to help natural resource managers prioritise which reefs to protect, involved tracking coral spores adrift on simulated ocean currents and computing the rise and fall of coral populations, which can be seeded by their spawning neighbours near and far. However, the model didn’t account for any diversity in coral species that grow at different rates, or seagrasses that can out-compete corals in the region.

What Boschetti and his colleagues found, much to their surprise, was that based on connectivity, the coral reefs that are important for maintaining a healthy ecosystem are not the same ones you might want to protect if you were expecting the region to be hit by a major disaster – say, a catastrophic marine heatwave – and wanted to aid the ecosystem’s recovery.

In presenting unseen solutions like this, models also can challenge long-held beliefs about the best approach to take when resources are limited.

In presenting unseen solutions like this, models also can challenge long-held beliefs about the best approach to take when resources are limited. Models developed by Occhipinti and her team simulating the NSW North Coast and Western Sydney, for example, showed that not all reductions in the number of hospital beds in psychiatric wards would result in increased suicides, if resources were reallocated to community-based programs and services.

“[Our] modelling has shown that … the ‘more is better’ assumption [for funding services] just actually doesn’t hold up,” says Occhipinti. Sometimes decision makers can make investments that work against each other, or a new program might siphon resources away from another service that would have greater impact, she says.

At the same time, mathematics is demystifying a crisis of another kind: misinformation.

Lewis Mitchell, an associate professor in applied mathematics at the University of Adelaide, started out modelling weather systems but has taken to investigating how bits of information are copied and spread online via social media.

“Human behaviour is absolutely the most challenging thing to predict,” says Mitchell. “It is truly complex in a way that weather and physics are not.” But that’s not to say it can’t be done.

When you treat information like a virus that people can contract and spread, and look at an entire population of social media users, as Mitchell does in his models, patterns start to emerge in the way people behave. These are understandable and describable using mathematical terms, he says. These fundamental mechanisms underpinning social interactions online Mitchell calls “the physics of online influence”.

Using something similar to an agent-based model, where some people are thought leaders and others are followers, Mitchell has shown how much information can be encoded about a person from their online friends. It turns out you only need to look at a person’s 10 closest friends to make a fair prediction about what that person is tweeting themselves.

A bigger question, however, is how to stop misinformation in its tracks. Not that mathematics can distinguish between facts and falsehoods: it’s just words and text in tweets and posts. But underlying social networks can be inferred from online behavioural patterns. It’s possible to ask questions with models about how rewiring these networks, to promote connections between echo chambers, or injecting other counter material might stop misinformation spreading, Mitchell says.

Except models can be terribly flawed. Some are even said to perpetuate systemic biases seen in society if their assumptions run unchecked.

“Certainly, with any model or machine learning algorithm, we need to be very cautious,” says Mitchell, “about how to implement it – if at all – and be very clear about the underlying assumptions of the model.”

There are many cases in complex systems science where simple models exacerbate existing biases in a similar way, he adds. In which case, modelling exposes some hard truths about ourselves and how we view the world.

On the flip side, if modellers are candid about their assumptions, writing them into equations and disclosing them in published papers, then models can provide an open arena for testing and interrogating whether those assumptions are reasonable, says Boschetti.

Either way, transparency is key. “If models remain black boxes, it’s easy for people to discount modelling,” says Occhipinti, who co-authored an opinion piece calling on researchers to openly share their model code during the COVID-19 crisis. “But if people are drawn into the debate, if we help people understand the inner workings of the model and challenge its assumptions, it draws people in, and it advances science.”

Predicting the future

One of the hardest questions to answer about the COVID pandemic is when it will end. In terms of impact on mental health, it will likely cast a long shadow.

Unemployment and financial insecurity mixed with unabated uncertainty continues after a year defined by social isolation. Along with domestic violence and homelessness, these are all factors that can turn up the dial on psychological distress.

When Occhipinti and her colleagues added these social drivers to their existing models, it made for some grim projections. Estimates made last August, based on economic outlooks at the time, suggested that psychological distress across Australia would peak by April 2022. But among young people that wave barrelling towards the healthcare system would be steeper and come sooner, with almost 60% of young Australians in distress by this year’s end.

Those estimates have since been revised, with observational data from 2020, and it looks like the peak may be behind us, as Australia fared far better than expected, Occhipinti says. “As new data comes in, it makes the models more robust over time.”

The modelling also showed that, in the same way that climate modelling makes projections about unspeakable futures and the best way to stop them happening, the single most effective way to avoid a mental health crisis and prevent suicides wasn’t anywhere to be found in the healthcare system. The best measure was to extend employment support schemes, such as JobKeeper, to alleviate the financial hardship that so often leads to psychological distress.

“We’ve shown that we can save time, resources and lives by directing investments to strategies that will deliver the greatest impact,” says Occhipinti.

But their work, focused on regional areas and expanded in this case to a national scenario, has been met with robust criticism, which the team have not shied away from.

Other researchers and mental health experts said that mental illness and suicide are multifaceted, fluctuating entities that are too complex to capture in models, let alone predict. They point to the real disparities between people and their risk of experiencing mental ill-health, which systems modelling overlooks, such as the colossal impact of the pandemic on women compared to men. They say that the available health system data is inadequate to create reliable models, and health records only ever capture data on people who seek medical care.

While these things are true, and similar limitations exist when modelling infectious disease, it doesn’t necessarily mean such models ought to be thrown out altogether. Dismissing population-level models of mental healthcare entirely, because of these realities, also misses the point of what the models are designed to do, says Occhipinti. Systems modelling aims to understand a complex system as a whole, and by its moving parts, to see how it behaves when people try to intervene.

“Models help create that clarity about how to act proactively, rather than reactively”

Jo-An Occhipinti

Though models are carefully constructed virtual realities providing estimates at best, without them we’d be fumbling around in the dark, she says. “Models help create that clarity about how to act proactively, rather than reactively,” Occhipinti says. “Otherwise, we’re going for the obvious solutions which aren’t necessarily the right solutions.

“We’ve tried that approach already [in population mental health research], and it hasn’t worked. In fact, it’s failed people that need good quality care.”

For Fabio Boschetti, it’s about recognising that models are always a numerical description of their makers’ view of reality, and will never provide perfect answers – but nor should we expect them to, he says. “The issue is not is the model right or wrong, but whether a model leads to better decisions than would be made without it.”