In the first webinar in the 2021 #EvaluateDigiHealth series, leads for digital health companies critically discuss their experiences of navigating the evidence generation maze, including their challenges, achievements and failures.
Find out more about the series and sign up for upcoming webinars here.
Chair: Paul Wallace, Clinical Director Digital, Health Innovation Network; Professor Emeritus of Primary Care, UCL
Adam Hunter, Board Director, Phlo Digital Pharmacy
Elliott Engers, CEO and Co-Founder, Infinity Health
Charlotte Lee, UK Director, Big Health
We’ve summarised the key themes and questions from the discussion:
Are Randomised Controlled Trials (RCTs) the gold standard?
For Big Health, evidence was core to the development of their app, Sleepio, as the Co-Founder, Professor Espie, had 30 years of research experience. This meant he did not launch Sleepio until a RCT was completed and they now have now carried out over 12 RCTs.
Charlotte felt that any product with a claim to be made would require an RCT but understood that there were a myriad of other products for which RCTs would not be appropriate.
Charlotte explained that since their time on the DigitalHealth.London Accelerator programme they have been expanding their evidence generation focus outside of RCTs, to build up their real-world evidence base including cost savings.
The importance of the end user
Contrastingly, Adam from Phlo shared that their approach to evidence was very much based on patient experience. As an NHS-authorised pharmacy, the use of Phlo is based on patient choice so meeting the patients needs is vital. Adam explained that identifying the paint points for patients (such as an unease with receiving medication through the post) led to important changes in the product (the creation of central in city pharmacy hubs which could deliver medication within hours).
Elliot detailed Infinity Health’s initial “research lite” approach in which they carried out user testing but explained that it wasn’t until they were introduced to the DigitalHealth.London Generator as part of their time on the Accelerator programme that they realised the importance of health economics and evidence generation.
Growing data sets
Another challenge highlighted by Adam in the discussion was having a large enough data set, in Phlo’s case this meant enought patients signed up to their platform, in order to draw valuable insights. He shared that Phlo now has enough data in order look at completing in depth research and forming collaborations. Elliot also felt that different levels of evidence and research is possible at different stages, sharing that when first developing Infinity they had very little data but now they have enough to carry out more robust research. Evidence generation will depend on the type of product but also the timing – it’s about identifying right research for the right solution at the right time.
What evidence is right for investors and commissioners?
All panellists agreed on the importance of evidence when talking to investors. Adam noted that where a company is in their life cycle impacts on the evidence that investors want to see, with the key question changing over time from “Is it a good idea?” to “Can it scale?”.
When talking about their experience of providing evidence to commissioners, Elliot shared that generally commissioners do not know what evaluation is appropriate, pointing out that if the company themselves are struggling to ascertain this, it would be even more difficult for the commissioners.
Charlotte shared that they’d had varied experiences with commissioners with some Scottish commissioners they’d worked with having a clear process and understanding of their pathway, but English commissioners not generally having the same level of understanding.
The need for standards
It was clear from the discussion that all speakers agreed on the real need for evidence generation standards.
Charlotte highlighted the issues that Sleepio faced as a product that represents a different mode of treatment delivery and the resultant issues in ascertaining the level of evidence needed for such a new concept – should it be treated as a drug or like any other digital, MedTech product? She highlighted the need for a consensus across the board on standards and thresholds for evidence generation.
Elliot also felt that there was a lack of consensus on what evidence is needed and that the pattern he has seen is that more evidence means more rapid adoption. He highlighted the important role the DigitalHealth.London Generator played in providing signposting and advice, as well as a community of academics and economists.
The importance of collaboration
Early in the discussion, Adam flagged the importance of collaborations and the challenges of getting in front of the right person. Paul concluded the session by highlighting the value of connecting organisations and academia, and how the DigitalHealth.London Generation programme is aiming to achieve this.
Find out more about the Generator and future #EvaluateDigiHealth webinars here.