Editor’s Note: For more from Dr. Pugh, tune in September 11, 2014 for her TEDMED talk on the use of sensor technology for assessment.
By: Carla M. Pugh, MD, PhD, associate professor of surgery and industrial & systems engineering, vice-chair of education and patient safety, and clinical director of the UW Simulation Program, University of Wisconsin
In his study in the August issue of Academic Medicine, Watson describes his analysis of hand motion patterns using a machine learning algorithm. The study, which uses engineering metrics, inspires cross-disciplinary collaboration between medicine and engineering. Breaking down the traditional silos is critical in facilitating new discoveries and advancing science. Recently, faculty in the department of surgery at the University of Wisconsin held a “speed-dating” event with faculty in the college of engineering. The resulting collaborations spanned the gamut of clinical, basic science, and education research, reinforcing the wealth of opportunities that exist in all aspects of clinical medicine. The collaborative projects featured new diagnostic technology for colorectal screening; pattern recognition software to automate image processing, and microscopic scaffolding for embedding nanomaterials into vascular conduits.
Watson used machine learning techniques to explore the metrics of hands-on performance. Such surgical performance metrics are still in their infancy and remain a fertile and exciting area for research. I agree with Watson that a lot of work in this area must be done and that the focus of this research should include more than economy of motion and task completion times. Based on my own experiences with sensor and motion tracking technology, I believe that a multi-variable approach will eventually provide a way forward in unfolding the true complexity of hands-on skills and allow for explicit performance measures that enable deliberate practice.
From a practical standpoint, the next step is to convert the engineering metrics to a tangible learning opportunity as it is unlikely that learners can use the numerical metrics alone for deliberate practice. One possibility that has great potential is to use the machine learning metrics (and other variables) as a tool for tagging videos. If numerical performance metrics can be used to target a specific time point in a task when a learner’s performance differs from that of an expert, the learner then can focus on that part of the task during training. Olympians and other athletes use this method to train. Video reviews and instant replays capture important learning moments and allow for focused remediation and performance improvement. Having this capability in medicine will provide a whole new meaning to self-assessment and life-long learning. I think we are definitely ready!
Beyond hands-on skills, motion tracking technology can be used to better understand surgical planning and decision making. Our lab is currently using motion tracking during complex tasks to look at idle time–the periods without movement. Our preliminary results show that the amount of idle time increases for novices, as their planning is not yet automated. In addition, idle time is longer for tasks with a higher error risk.
Overall, I believe that advances in our understanding of performance metrics will allow for an automated, quantifiable learning experience that will revolutionize training for beginners and practitioners at every level. Once these measures are available and well integrated into daily practice, the discussions around quality will take on a new form and greatly advance our ability to produce top notch, elite professionals who provide the best care.