For startups and investors aiming to bring health technology to underserved populations, proof of impact should be everything: it’s what shows employers and payers that a new offering is worth investing in. be paid for, and what can help convince patients to try it.
But while the market is crowded with companies claiming their products dramatically improve health – especially for underserved groups – there is still no standard list of industry-wide metrics to fully assess them. . Some report the number of new appointments they have created among low-income or underrepresented populations; others offer new ways to get closer to how their app or wearable has specifically helped rural and urban patients with chronic conditions, for example.
“It’s a really big deal,” said Alex Lennox-Miller, a healthcare analyst at venture capital research firm CB Insights. “A big part of the future of health systems is care that happens outside of appointments – like we talk about patient engagement, like we talk about remote monitoring, like we talk about home care , these are the places where we have the worst means of measuring and the least visibility on their impact.
Measurement challenges go beyond issues of access and equity; Venture capital fund Rock Health recently developed its own “clinical robustness” score after finding that only a small portion of health tech companies even make claims about their clinical impact on patients.
And while investors are increasingly interested in moving from traditional financial metrics like market size and growth rate to numbers that reflect meaningful clinical impact — lowered A1c levels, for example — it’s “Still relatively early, so these metrics aren’t fully standardized yet,” said Sunny Kumar, a partner at GSR Ventures, which invests in early-stage health-tech companies.
Still, there’s debate over what exactly companies should be measuring to show they’re meaningfully reaching patients who may not have strong established relationships with traditional healthcare. “What is a clinical touch? Does a text message count, are they visits from nurses, is it only when they walk into the clinic? said Zen Chu, who leads MIT’s Hacking Medicine Initiative.
As companies are still experimenting with new measures, it is usually up to health systems or the employers purchasing these services to set their own criteria for fairness – whether reaching specific populations or incentivizing them. to stay active on the app or wearable device, Lenox-Miller said. “It’s almost more of a problem for investors, because ideally if you’re a buyer or a healthcare system looking at these solutions, you’ll have done the legwork up front to understand the problems you’re trying to solve.”
Investors, on the other hand, need to “be aware that these aren’t yet questions with hard answers and start paying attention to companies, to founders, who recognize this and say, ‘We’re figuring this out, we don’t.” have a solid answer,’ [and] a willingness to even recognize that this is an issue,” added Lenox-Miller.
If they don’t get it right, he said, companies making unfairness-based claims risk losing the trust of providers and payers aiming to reach these underserved groups. “What they do is try to fake it until they get it right and hope they give many answers before people start to realize they really don’t make a difference. It destroys the trust of providers, healthcare organizations and patients,” he said.
The lack of standards also presents an opportunity for companies to get creative, he added. Companies like Alphabet-spinout Cityblock, which largely focuses on urban Medicaid populations, “have the privilege of being able to almost begin to understand what some of these metrics should be and what those data points should look like,” did he declare.
Cityblock CEO Toyin Ajayi told STAT that while there are widely used measures for clinical improvement, such as those developed by the National Committee for Quality Assurance, measures such as engagement may require different references for different services or populations. But she argued that any company or provider serious about reaching underserved populations likely has a way of measuring how often patients interact with the service and how satisfied they are, as well as how often they meet with clinicians.
Instead of seeking an industry standard for these factors, she said, health-tech companies could develop their own metrics.
“If you’re looking to treat serious mental illness in populations of people with mental health issues and social needs, then we can all agree that a text message isn’t enough…if you’re looking to move the needle to get someone who has insurance, who has a primary care provider, to see their primary care provider for a preventative visit, and you know they’re engaged digitally, maybe a text message enough,” she said. “I agree that there is no standard measure. But I would say that maybe it’s not the right thing to do.”
Cityblock, for example, has different engagement criteria based on a patient’s health risk levels. “We’ve certainly built these kinds of measures that are very relevant to our understanding of our population, and the recognition that this one-size-fits-all isn’t going to be the right answer,” she said.
Some industry groups are concocting their own tools to better quantify the reach of health tech companies in underserved populations. The Digital Medicine Society helped build a “market opportunity model” that CEO Jennifer Goldsack said companies could use to carve out their market opportunities and revenue size based on variables such as age, race and ethnicity, income, education and disability status.
But she said health-tech companies and investors are often concerned about measuring factors like whether new patients are downloading or using apps instead of focusing more on improving their health. “As an industry, we are drilling on the wrong thing. We spend time with great consternation about new kinds of process, engagement, and retention metrics. How about whether it makes people better? she asked.
Health-tech companies facing growing economic uncertainty and tightening customer belts — including employers and payers — may find that clearer patient impact metrics will make or break their businesses, said Rock Health President Tom Cassels. “What I would expect to see is a reversion to the mean when it comes to locations, that is, ‘we can’t go in without evidence of life or patient outcomes, because our competitors have them. “”
This story has been updated with additional reports from Cityblock Health.
This story, part of a series on health technologies for underserved populations, was supported by the USC Annenberg Center for Health Journalism National Fellowship.