Watch back – #EvaluateDigiHealth Webinar: What can different academic disciplines bring to evaluating digital health?

This webinar, the second in the 2020 #EvaluateDigiHealth series, features leading academics with extensive experience of researching and evaluating digital health interventions in complex systems. The session explored the interdisciplinary nature of digital health evidence generation, including how theories and methods from different fields – such as human factors, applied health and computer science – can inform practice, from product development to real world implementation.

Find out more about the webinar series and sign up for upcoming webinars here.

Webinar overview

The webinar was chaired by Dr Jean Ledger, Research Fellow in the Department of Applied Health Research at University College London, supported by Dr Henry Potts, Associate Professor, Institute of Health Informatics at University College London.

The webinar featured a panel of experts:

• Professor Peter Buckle – Principal Research Fellow at Imperial College London
• Professor Elizabeth Murray – GP, Professor of eHealth and Primary Care and Clinical Director of the Institute of Healthcare Engineering at University College London
• Professor Ann Blandford – Professor of Human-Computer Interaction and Deputy Director (Digital) of the Institute of Healthcare Engineering at University College London.

Terminology is important

The panel discussed the importance of using the correct terminology. Different terms can mean different things to different people in the evidence generation space. The term “evaluation” for example, to an academic will likely mean formative evaluation (i.e. can people use it, do they find it engaging, can they use it to monitor their health) whereas clinicians might interpret it in a broader clinical context. This creates different perspectives on evidence, further widening the gap between industry, academia and healthcare.

Clinical trials are not the only approach

Peter shared that he believes Randomised Controlled Trials (RCTs) are overrated especially when the intervention has a subtle patient impact. He felt that the design of a trial depends on multiple factors and that an RCT isn’t necessarily the best option – it depends completely on the problem you’re tackling. The panel agreed that RCTs are not always useful in showing value and sometimes it is better to identify who you want to show value to (and understand what they find valuable) and picking a type of assessment accordingly.

The panel agreed that a clinical trial may be necessary at some point but it is important to undertake a number of other pieces of research to improve the intervention, and to have some evidence to show the intervention is useful, before embarking on a trial.

Academic and SME collaboration

The panellists discussed the important questions which they most commonly ask SMEs including:

• Has the solution been tested with the user?
• Does it have good user experience?
• Does it solve a big enough problem?
• Does the SME understand the system context and their customer’s workflow?

They highlighted that it is important to understand the care pathway as it is currently in practice (including it’s flaws). They agreed that they want SMEs to engage with them as early as possible, to make sure these questions are asked early on.

The panel discussed how having conversations around understanding what each partner finds valuable is crucial. An academic may focus on ensuring the evidence shows the product is useful and useable, a clinician might be more focussed on the spread and deployment.

They commented that academics are not only driven by the financial element of research, but that many are looking for a genuine partnership with mutual sharing and trust, an opportunity to make a difference and a compatible working dynamic.

They recommended that companies also think about different evaluation questions at different points in the development of their business. It is a long road and therefore it is not a one-off exercise.

Henry reflected at the end of the discussion that there are multiple networks emerging to help bring the ecosystem of digital health SMEs and academics together. The DigitalHealth.London Generator is one of these.

Recommended reading: WHO guide on evaluating digital health – a practical guidebook