British startup wants to read your vital signs via video call

A British startup is trying to measure heart rate, respiratory rate and even blood pressure over a video call to reduce the need for patients to travel to a doctor’s office.

Zenicam founder Eirini Kateri hopes its artificial intelligence software will provide a rich set of real-time data, allowing doctors to make decisions remotely and ultimately improve access to healthcare in hard-to-reach areas.

Her first results seem promising. But experts warn the technology will face some major hurdles before it can reach a real doctor’s office.

Kateri first worked on an AI video pulse monitor as part of a PhD program at the University of Southampton in England. She told me she had a “light bulb moment” when a friend was sick and had trouble reaching a doctor during the pandemic.

“I thought: is that allowed? [get this information] remotely to help people who do not have access to doctors, or to help when doctors are overworked and overburdened?” she said.

Gemma Galdón-Clavell, founder and CEO of AI safety organization Eticas Foundation, told me that Zenicam was “certainly ambitious” and described the start-up’s focus on equity as “commendable”.

But she is skeptical that the technology can succeed in a space full of technical challenges.

“These systems generally don’t work. We have seen time and time again how facial and emotion recognition technologies usually fail,” she said. “And Zenicam tries to assess something even more nuanced: vital functions.”

Bias is one of the problems Kateri hopes to solve with her approach. Medical AI software can be prone to bias if it is trained on video and images of people who look relatively similar.

It’s a problem that exists across medicine, where research and technology tend to focus on white people.

In Britain, medical professionals are still missing and identifying symptoms that are sometimes different for different skin tones with devastating consequences. Earlier this year, a government report criticized medical devices that don’t work well on people of color.

Kateri believes she has found a way to ensure Zenicam is accurate regardless of a patient’s ethnicity or appearance. It is trained on both real and “synthetic data,” which it says can compensate for gaps in existing data.

This is a kind of ‘fake data’, based on real data that Zenicam has already collected, she told me. You can account for a wide range of different scenarios by “adding motion or other lighting or changing skin tone, scale, or color based on existing data.”

“I may have had a video of someone not wearing glasses,” she said. “I could artificially add occlusions to the face or cover certain areas to hide scenarios where someone was wearing them.”

To work in the medical field, Zenicam will have to prove that it can record vital signs in different environments, says Galdón-Clavell, who advises the United Nations and the EU on applied ethics and responsible AI.

It would have to deal with the fact that most patients will not be in a quiet, perfectly lit room and will not always have a stable internet connection.

By training her AI models on these variations, Katerini says she can accurately determine heart rate and other vital signs regardless of people’s physical appearance or the environment they’re calling from.

But Galdón-Clavell urged caution in using synthetic data as it risks “exacerbating existing problems.”

“You first need to have a good understanding of the real-world data and be able to assess what is missing in your existing data set,” she told me. “The synthetic data must be able to compensate for what is missing.”

This is “a growing specialty in the field” that requires serious expertise and supervision to work with properly, she added.

Nevertheless, Kateri told me that her results so far are promising. “For heart rate we have an accuracy within one beat per minute, for respiration within two breaths per minute and for blood pressure for diastolic and systolic we have an error as low as 5 mmHG for all physical manifestations,” she said.

Measuring blood oxygen levels and temperatures was “more challenging,” but Zenicam currently had an accuracy of around 70%.

The startup will need to provide rigorous scientific evidence to back up these claims if they want to get Zenicam into clinical use.

Because the software is designed to help healthcare professionals make more informed decisions during a video call, “it needs to be validated as a medical device,” Kateri said.

Currently, her team is integrating the AI ​​technology with videoconferencing systems “to demonstrate exactly how it will work and measure its accuracy for regulators.”

She thinks this will take at least a year and a half. In the meantime, she hopes to conduct tests on the device at the same time to validate the technology.

If it works, Kateri hopes she can eventually get the product into countries where access to a doctor is scarcer than in Britain.

“I’m most excited about the solution’s potential to be used globally, especially in low-coverage locations,” she told me. “Where we can help everyone, regardless of their background, have access and fair access to healthcare.”