Editor’s Note: This post is one of two pieces on the topic of procedural competence. Read the other piece here.
By: Martin Pusic, MD, PhD, assistant professor, Emergency Medicine, and director, Division of Learning Analytics, New York University School of Medicine Institute for Innovations in Medical Education, New York, New York.
When I read Vaisman and Cram’s thoughtful Perspective on academic faculty procedural competence, I agreed with most of what they had to say. Academic faculty are certainly having to adapt to a myriad of dislocations as our health systems adapt to new realities. What doesn’t change is the buck stops with the academic attending, which can be problematic for all the reasons they list. However, there is one facet of their argument with which I take a different perspective and that is the idea that there is a paucity of data on procedural success. Anyone who has done a procedure note in a modern electronic health record (EHR) knows that the data is available and in spades. This will only increase given that the incorporation of digital photos and videos is only a matter of time—why would I write down “we used 1% xylocaine without epinephrine to locally infiltrate the laceration” when my head camera can easily stream this to the EHR? Sure there are issues to be worked out, but soon we’ll be swimming in all the data we could possibly want about our procedural successes or lack thereof.
What will we do with all this procedural data? That’s where there is plenty of hope for steady improvement in not only academic faculty procedural success but procedural success in general. Consider a (currently) more cutting edge example: the professional hockey goaltender. It used to be that there was a paucity of information on how a goaltender was performing. The key metric was the goals against average defined as the number of goals allowed per game. Problems with that metric included the fact that it was more dependent on how the whole team played and that there were only ~50 data points per season. Enter better data collection in the form of counts of shots per game: the save percentage is the proportion of shots that goalie stops. This is a metric that is more purely down to the goalie and there are more than 1,500 data points per season. Of the top 30 goaltenders during the 2015–2016 National Hockey League season, the difference between the best (Carey Price, 93.4%) and the “worst” (a tie amongst four goaltenders, 91.8%) is a miniscule 1.6%,1 a product of fierce, transparent competition and a relentless focus on deliberate practice. Contrast this with, say, infant lumbar puncture success rates where success rates can range from 80% down to 36%, depending on the report.2 More recently, the goalie metric has been refined further to the adjusted save percentage which takes into account shot difficulty, meaning that the video of EVERY shot is rated as to difficulty of the save and then incorporated into the metric. Imagine that technique being applied to medical procedures! Anders Ericsson3 has called medical educators out on our slowness to adopt video review into our educational practices, though we have indeed started (see the ground-breaking studies by Birkmeyer et al4).
Orienting metrics, at both process and outcome levels, allow us to better decide who does the procedure, when, and with what support. Such metrics can be used for both quality improvement and educational goals—indeed such a fusion is a positive development in and of itself. We don’t yet all have the feedback resources that professional hockey goaltenders do, but when we’re doing something more important than stopping a hockey puck, isn’t it time we started thinking about fine-grained data in the same way?
- National Hockey League. Statistics. http://www.nhl.com/stats/player?reportType=season&report=goaliesummary&season=20152016&gameType=2&sort=savePctg&aggregate=0&filter=gamesPlayed,gte,10&pos=G. Accessed July 25, 2016.
- Shafer S, Rooney D, Schumacher R, House JB. Lumbar punctures at an academic level 4 NICU: Indications for a new curriculum.Teach Learn Med. 2015;27: 205–207.
- Ericsson KA. Necessity is the mother of invention: Video recording firsthand perspectives of critical medical procedures to make simulated training more effective.Acad Med. 2014;89:17–20.
- Birkmeyer JD, Finks JF, O’Reilly A, et al. Surgical skill and complication rates after bariatric surgery. N Engl J Med.2013;369:1434–1442.