Number graphic

The future looks incredibly promising for analysis and analysts in the NHS.  I can’t remember a time when the environment looked so conducive.  Over the next few years, analysts will get plenty of opportunities to make a real difference to the NHS and the people it serves.  And at the moment, the NHS needs all the help it can get. 

But I’m a worrier.  I can’t help but think about the issues that might trip us up on the road to the better use of analysis.  Essentially, this boils down to hard conversations that need to take place between different groups of people that occupy the space in and around the use of analysis in the NHS.   

I see three sets of conversations that are needed: 

Conversation 1: analysts and decision-makers 

The first is between analysts and the users of their work. These might be senior managers, or clinicians or strategic decision makers.  In the NHS the default interaction between these two tribes is for managers to come to analysts with a data request: ‘I need data on X so that I can manage Y, problem / issue’.   

This has to change if we’re going to capitalise on the potential of analytics to improve services and outcomes.   

Managers need to take a step back, describing the problem, but not prescribing the analysis. And at the same time analysts must take a step forward. Senior managers and clinicians will always have the clearest view of the problems at hand. Identifying, prioritising and describing problems is the manager’s role.  But analysts should lead the conversation that structures those problems into a form that can be addressed by analysis.   

It is analysts, not managers, that are best placed to decide which datasets and which methods are best able to address an issue.  And – although this is a common area for development - it is analysts who have the skills to present results appropriately.   

This change will induce anxieties.  Managers will feel a loss of control and analysts will feel the weight of additional responsibility.  But if we don’t then we’ll always be limited by the types of analysis we do – because, understandably, managers do not have a full and rounded view of the myriad powerful analytical techniques that are available to us. 

Conversation 2: New and experienced analysts 

The second set of conversations is between the wonderful group of new analysts that have come to the NHS fresh from university over the past few years, and the old guard (like me).   

The emergence of data science as a discrete discipline means that our new recruits come to us as capable coders on day 1.  They can’t imagine doing an analysis without having git as a key part of their workflow.  Meanwhile many of the old hands are still getting to grips with these newer methods.   

The last change of this magnitude was in the late 1980s when computers started to become commonplace in offices.  There were analysts that knew how to operate a computer and those that did or would not.   

For us old-timers, this is an area where resistance is not only futile it’s counterproductive.  We need to get on board with best coding practices, workflow, version control etc.  It will feel odd for experienced analysts to be behind the curve on a key skill in our evolving field, but in practice it will be quicker and easier than many might expect.   

But the key insight here is that these new skills and ways of working are additive, they do not diminish or replace the other skills that are critical to effective analysis. Some elements of our craft can only be acquired over time: a rounded knowledge of the health service and its processes; problem structuring; experience of using the analytical techniques in real world settings; a thorough understanding of the innumerable datasets that exist in the NHS; effectively communicating the results of analysis. These skills are no less important given the emergence and formalisation of data science.   

Younger analysts need to know that their data science skills only have value when well targeted and deployed. Getting this right requires experience, for which there is no shortcut.   

Unfortunately, I’ve seen a lot of very clever data science recently that has been applied to answer questions that are, frankly, meaningless.  Because the skills and experience required to scope and structure the work up front has been missed.  So, younger analysts they need to trust their more experienced colleagues to help with this.  Meanwhile helping them – patiently - get to grips with the new technology. 

Conversation 3: Analysis and business intelligence 

The third conversation is possibly the most difficult. It might also be the most consequential.  It’s a conversation we’ve put off for years, but we can’t delay it any further.  It’s between analysts and our colleagues who provide business intelligence services.   

The primary function of business intelligence is to make data available to managers as quickly, efficiently, accurately and accessibly as possible.  Whereas analysts are there to provide answers to specific, usually complex business questions.  The outputs of analysis are rarely tables of numbers or a dashboard; they are insights, prescriptions for action, recommendations.   

Of course, these two disciplines have common ground, they both need good access to data to succeed.  But really that’s where the similarities end.  Because they have different objectives, they require very different skills, differ workflows, different mindsets, outlooks, attitudes, they operate on different timeframes and they require different environments to flourish.   

Yet in most NHS organisations, these two disciplines are conflated conceptually and resourced collectively. And blurring the distinction always ends the same way: the immediate imperatives for business intelligence crowd out the longer term, more strategic questions that would be answered by analysts.   

We need to think of these as two distinct disciplines and resource them separately.  Bringing them together undermines both.  We need space to develop our own paths, valuing and building the unique skills required for each.   

An analogy is with accountancy and economics, two disciplines that are ostensibly concerned with money, but with very different roles, methods, and histories.  Imagine the problems that would ensue if we got an economist to prepare our annual accounts.  And we probably wouldn’t want an accountant working on economic growth forecasts.   

We’re making the equivalent errors by conflating business intelligence and analysis every day in the NHS.  The solution is a clear separation of these two disciplines, so that excess demand for business intelligence does not result in scaling back of analysis. 

An ongoing dialogue in each local health system 

So these are the three potential fault lines that I worry about: between analysts and decision makers; between newly qualified and experienced analysts; and between analysts and business intelligence teams.  

Conversations are needed across all three. And these conversations can’t just take place once, nationally and at a single point in time. They need to be extended dialogues that occur in every local health system. 

This blog is adapted from a workshop on the future of healthcare analytics that formed part of our 2022 INSIGHT Festival.  Video available here….