With a current health secretary so openly enthusiastic to promote apps and digital “solutions” in the NHS it’s useful to check on what level of actual evidence is available on how useful the new technology and software really is.
It seems there is relatively little appetite to find out – perhaps because those marketing the new digital devices and technology are less than keen to have it thoroughly tested. Only recently Babylon deleted any reference on its website to a high-profile test of its controversial chatbot which had appeared to show it competing successfully against real doctors, after the validity of the test was debunked by a number of experts.
Now a new study by a team of German academics of research papers on the existence, use and benefits of digital technology in relation to nursing care has responded to the “lack of good empirical overviews of existing technologies”.
Negative evidence on Babylon diagnosis app goes missing
They have found few papers based on efficiency studies, and many studies based on “a low level of evidence”. The authors point out prior to their study:
“To the best of our knowledge, there is no review article that outlines the broad range of technologies developed to support formal and informal care, and no research findings are available that outline the existing evidence with respect to acceptance, effectiveness and efficiency for this broad field of technologies.”
The team conducted a review of research papers in German or English produced over a 7-year period up to March 2018. Their extensive online search led to analysing 715 full text articles from 69 countries.
The findings are interesting, but not entirely surprising given the current poor level of critical reporting and discussion of new technology.
Little evidence on cost effectiveness
Very few of the studies focused at all on costs of technologies, and very few included full economic evaluations: most studies categorized as “efficiency-studies” offered only simple cost analyses. Indeed while 60% of studies analysed aspects of the effectiveness of the technology, less than 6% analysed efficiency or included a cost analysis. Just 13 studies out of the 715 analysed cost-effectiveness. Only 4 offered a cost-benefit or cost-utility analysis.
There was also little focus in the research on digital support for informal carers: just 8% of papers considered this, while a vanishingly small number (less than 1%) saw children in need of care as a target group for digital solutions. Most of the studies were of technology for patients in need of care, or formal care givers.
The authors note that they found:
“large number of effectiveness studies with a focus on ICT, robots and sensors, and a large number of acceptance studies focusing on ICT, robots and EHR/EMR [electronic records].
“However, a large proportion of these studies has a low level of evidence …. Efficiency studies are very rare in general. This points to the low consideration of the relationship between benefits and costs of a technology, so far.”
The German team also note that the way their study had been organised made it less likely they would find any research papers critical of the new technology, almost all of which are to be found outside the mainstream of academic journals:
“We considered published scientific studies only, and no grey literature [research that is either unpublished or has been published in non-commercial form]. This review therefore tends to contain fewer publications with negative or neutral findings. Consequently, it can be assumed that there may be a bias towards promising technologies.”
Dear Reader,
If you like our content please support our campaigning journalism to protect health care for all.
Our goal is to inform people, hold our politicians to account and help to build change through evidence based ideas.
Everyone should have access to comprehensive healthcare, but our NHS needs support. You can help us to continue to counter bad policy, battle neglect of the NHS and correct dangerous mis-infomation.
Supporters of the NHS are crucial in sustaining our health service and with your help we will be able to engage more people in securing its future.
Please donate to help support our campaigning NHS research and journalism.
Comments are closed.