Driving innovation: a case study using a simple evaluation tool
Professor Terry Young, Dr Fay Wilson, Alan MacDonald and Mr Simon Dodds describe for ICJ an evaluation tool that was used to support the development of a rapid access clinic in an existing health centre in Erdington.
Project management is simple: look after people, plan well, monitor effectively and intervene as needed. However, people, planning, monitoring and intervening are complicated, so good project managers mix in experience and tools they pick up over a lifetime.
Case study
The team from Badger (Birmingham and District GP Emergency Room) Group used a 6M Design® approach with three pre-implementation stages (map, measure, model – a prototype based on a simulation), and three phases of implementation (modify, monitor, maintain).
Once a working upgrade of the facility was in place, two reviews were conducted in January 2022, using a simulated set of patients (based on a typical case-mix) and putting this stream of virtual patients through the service to see how it was responding.
Across the two days, the workforce involved in the review included: heath care support workers; receptionists; and clinicians – around 8 people in all. A 4N framework was used for collecting and analysing the feedback.
To understand 4N, figure 2 (used with permission from SaaSoft) shows a creative journey into new territory – leaving behind what is already there and implementing what is desired. Specifically, 4N feedback uses the dimensions of time (present/future) and emotions (positive/negative) to create a chart which is split into quadrants, defined as:
- Nuggets (relates to the present and the things about which the stakeholders have positive feelings)
- Niggles (relating to the present but capturing features where the feelings are negative)
- Nice Ifs (similarly, relating to the future and positive feelings)
- No Nos (finally, relating to the future and negative feelings)
Niggles identify what hasn’t gone to plan to date. To grasp this feedback, an improvement tool called a Niggle-o-gram® was used, based on the familiar failure modes and effects analysis (FMEA). The list of niggles was graded based on three scores:
- Incidence: how often does this niggle occur (never, 0 to 9, always)?
- Impact: what is its effect (no impact, 0 to major impact, 9)
- Influence: how much can we do (nothing, 0 to 9, change it completely)
Using this scheme, niggles that nobody could do anything about were assigned low priority. In this case, the top 5 niggles came out as shown in table 1.
A more intuitive way of grasping the feedback is to produce a word cloud, weighting the size of words in each quadrant by their priority in the rankings.
Using the findings to address key issues
One tension to emerge was around staffing, since booking patients in at reception takes less time than appointments with the healthcare worker or clinical staff. With everything running absolutely smoothly, a single receptionist might have coped, but interruptions and sporadic other tasks meant that having two receptionists was important for safety.
In turn, this created a tension between having a workload that would utilise two receptionists, and the capacity for healthcare and clinical appointments. In the end, a system designed for 10-minute appointments was fine tuned to 12-minute appointments to get the best combination of reception and other staff usage.
A second issue was Covid safety in a waiting room that could only accommodate 4 patients. Again, from modelling and other analysis, this was too few to manage the overall capacity planned for the centre, so a new process was developed whereby patients arriving in their cars would use their cars as their waiting room (and the receptionist would contact them to walk straight to their appointment), freeing up the waiting room for the exclusive use of those who arrived in other ways.
In the end…
Using a 4N framework as part of the management of an expansion of a health centre, an existing service was quickly upgraded for rapid access Covid patients, and tested robustly for capacity, throughput and safety.
Tools such as this are critical to the success of any agile or pop-up service. The good news is that the 4N approach is easy to understand, straightforward to implement and bridges the worlds of experience and clinical quality.
About the authors
Mr Simon Dodds, MA, MS, FRCS
Mr Simon Dodds is a general surgeon at University Hospitals Birmingham NHS Foundation Trust. He studied medicine and digital systems engineering before following a career in general and then vascular surgery. In 1999, he was appointed as a consultant surgeon at Good Hope Hospital in North Birmingham and applied his skills as an engineer and a clinician in the redesign of the vascular surgery clinic and the leg ulcer service.
In 2004, the project was awarded a national innovation award for service improvement. This experience led to the design, development, and delivery of the Health Care Systems Engineering (HCSE) programme.
Alan MacDonald, BSc
Alan studied at Nottingham Trent University and has a BSc (Hons) in Biomedical Science.
He worked for the Badger Group as an Out-Of-Hours primary care team leader and later became a data analyst. Since the start of the COVID-19 pandemic in March 2020, he became directly involved with the development of a multi-lane drive through Covid Referral Centre at the NEC.
He has been instrumental in the deployment of other temporary drive through clinics across Birmingham. He has also been actively part of the original team who were successful in applying this concept to the first purpose-built drive through clinic in the UK
He is frequently involved in new & novel projects within the out of hours primary care sector and is currently studying Health Care Systems Engineering.
Dr Fay Wilson, MBChB, FRCGP
Fay trained in Birmingham and has practiced there as a GP there since 1985. Her extensive national and local portfolio includes: NHS HA Non Exec, GMC fitness to practise chair, and associate postgraduate dean at Health Education West Midlands. She has served on the council of the BMA and other bodies. Fay brings people together to develop new models of care, a notable success being the Birmingham Multifund co-operative, a pioneering nurse-led walk-in centre and a prototype GP provider-at-scale ahead of its time in the mid-1990s.
Dr Wilson is medical director and co-founder of Badger, a GP social enterprise since 1996 providing out of hours and urgent primary care. COVID-19 introduced her to systems engineering, new people and new ways of thinking. Her ambition for the last decade has been to slow down.
Prof Terry Young, BSc, PhD, FBCS
After 16½ years as a research Engineer, Divisional Manager and Business Development Director, Terry became a professor at Brunel University London for 17 years.
He has a BSc in Electronic Engineering and Physics, a PhD in laser spectroscopy both from the University of Birmingham, UK.
His research has been in health technology, health services, and information systems. He has taught information system management, project management and e-Business.
His awards include the Operational Research Society’s Griffiths Medal, 2021, for analysing the return simulation methods offer when used to improve healthcare services.
Prof Young set up Datchet Consulting in 2018 to support innovation on the borders of academia, health and industry, of which the project reported here is an excellent example.
