A Crowdsourcing Model for Creating Preclinical Medical Education Study Tools

By Hansen C. Bow, PhD, third-year medical student, Johns Hopkins University School of Medicine

About two years ago, Jon Dattilo, Andrea Jonas, and I started talking about how we could more effectively study our preclinical material. Both Jon and Andrea are classmates of mine at the Johns Hopkins University School of Medicine. We noticed that among our peers, studying mainly consisted of highlighting texts or PowerPoint slides, making outlines, or writing summaries. We recognized that these study methods were static–our review before tests would basically involve rereading the materials without any form of substantive interaction with what we were studying. We also noticed that small groups of students would get together and quiz each other before the section exams–a dynamic and interactive study method but one that is inefficient with large groups of people.

Based on these observations, we decided to take a different approach to studying. Instead of the usual highlighting and outlining, we created question and answer sets that related to each organ system block in the Genes to Society basic science course. We describe this process in more detail in our June article in Academic Medicine. Building on the capabilities of Google Drive spreadsheets, we created these lists in such a way that changes and additions could be instantaneously viewed by anyone with a link to the spreadsheet. By sharing these resources with our entire medical school class, we worked together with a community of our peers to create, edit, and provide commentary on over 16,000 question and answer sets. From a brief survey, we learned that about half of our class used the resources. We also found that the average test scores for our class rose compared to those for the previous class (who didn’t have access to the question and answer sets). We hope that our novel question and answer sets and flashcard program will serve as a model for collaborative learning.

Click on the images of our question and answer sets and flashcards below for more information.

Screenshot of the Google Drive spreadsheet, including links to the various question and answer sets for the gastrointestinal/liver section of Genes to Society basic science course. From here, students can access lists of question and answer sets that their classmates created. Anyone can modify this list and add new links to new resources.

Screenshot of the Google Drive spreadsheet, including links to the various question and answer sets for the gastrointestinal/liver section of Genes to Society basic science course. From here, students can access lists of question and answer sets that their classmates created. Anyone can modify this list and add new links to new resources.

 

Screenshot of sample questions and answers for the gastrointestinal/liver section of the Genes to Society basic science course. By studying with resources in the form of questions and answers, students can quickly identify the topics that they need to study and can more efficiently spend their study time.

Screenshot of sample questions and answers for the gastrointestinal/liver section of the Genes to Society basic science course. By studying with resources in the form of questions and answers, students can quickly identify the topics that they need to study and can more efficiently spend their study time.

 

Screenshot of a question flashcard. Users input an Excel spreadsheet or PowerPoint presentation (as we did in this example) into our Java software. The software then converts the data it into an interactive quiz. Pressing the down arrow reveals the answer, pressing the left arrow indicates that the user got the answer incorrect, and pressing the right arrow indicates that the user got it correct.

Screenshot of a question flashcard. Users input an Excel spreadsheet or PowerPoint presentation (as we did in this example) into our Java software. The software then converts the data it into an interactive quiz. Pressing the down arrow reveals the answer, pressing the left arrow indicates that the user got the answer incorrect, and pressing the right arrow indicates that the user got it correct.

Screenshot of an answer flashcard. Users input an Excel spreadsheet or PowerPoint presentation (as we did in this example) into our Java software. The software then converts the data it into an interactive quiz. Pressing the down arrow reveals the answer, pressing the left arrow indicates that the user got the answer incorrect, and pressing the right arrow indicates that the user got it correct.

Screenshot of an answer flashcard. Users input an Excel spreadsheet or PowerPoint presentation (as we did in this example) into our Java software. The software then converts the data it into an interactive quiz. Pressing the down arrow reveals the answer, pressing the left arrow indicates that the user got the answer incorrect, and pressing the right arrow indicates that the user got it correct.

Related Posts

5 Comments

  1. Kemi Tomobi
    June 27, 2013 at 6:03 PM

    Thanks for sharing this finding, especially at the Johns Hopkins University, an institution with a reputation for being very competitive.  I am glad that the PASS/FAIL system is in effect, it is true that programs with a pass/fail program are more collaborative than those that are not.  The crowdsourcing model helps to deal with the large amounts of information one has to learn in the preclinical years.  It is “effective.” This method does the right things right in terms of studying, and in encouraging a shared effort, and as a result, the overall class averages were higher than the previous class.  This proves that “strength in numbers” of the right people -especially with a wide range of talents – can move mountains.

    I hope this sends the message that medical school is not the time to be cutthroat – the most competitive part of the road to medicine is the application process for medical school.  Once in medical school, people should learn to be more collaborative, this is god for each individual student, and this also makes the school look good.

    A reservation I have about this study is that it is limited to the courses that are pass/fail.  I would like to see a follow up study or different study for courses that are scaled on honors/high pass/pass/marginal pass/fail or use the letter A-E system, where the stakes are higher.  Would all students at least high pass or get As and Bs if the crowdsourcing method was used?  Would more students participate?

    Another reservation I have that is not likely to be published in any study is that how does crowdsourcing affect standardized test results like STEP 1?  Does crowdsourcing increase the pass rate for STEP 1? STEP 2 CK? CS?

    And what about the clinical years? For some, basic science is not as challenging as the clinical years are. There is a lot more information to take in during the clinical years, and one has much less control over schedule, and much less free time.  Would students be more encouraged to collaborate during the clinical years, a crucial period of interest to residency program directors?  Again, if such collaboration was fostered in the clinical years, this would help each individual student, and it would also help the institution’s reputation.

    Otherwise it is great article, and great initiative in this topic at the Johns Hopkins University School of Medicine.

    ________________________________

    • Andrea Jonas
      June 27, 2013 at 10:47 PM

      Kemi,
      Thank you for your thoughtful response to our study, and we certainly agree that collaboration is a key element to success in medicine, both during medical school and beyond. We believe that the effects of working together not only makes the medical school experience much more positive, as you point out, but can also inculcate a generation of physicians who prioritize team work and actively foster an intellectual community.
      As for some of the questions and points you raise, we agree that it would be wonderful to have data on the effects of crowdsourcing in an educational system that used tiered grades. It is possible that our study only had the positive effects that it did because of the “lower stakes” circumstances of Pass/Fail grading. All preclinical coursework at Johns Hopkins is strictly pass/fail, which does not present much opportunity at our institution to explore your idea. One exciting possibility would be to partner with another institution that does implement tiered grading to use our model of crowdsourcing and study its effects. This would provide one opportunity to study the effectiveness of collaboration in a different educational environment.
      As for Step 1 studying, our collaborators at Johns Hopkins did generate a number of flashcard resources for the class of 2014, though we have not assessed any affects on average Step 1 score or Step 1
      passing rate as of yet.
      It would be very interesting and exciting to bring collaborative studying to the clinical years. You correctly point out the difficulties of developing strong study routines while on the wards. This only heightens the value of bring collaborative studying to this time, which is such a rich period in our development as doctors. One possibility, however, is that crowdsourced flashcards may not be the optimal strategy to bring collaboration into the crucial clinical years. Other possibilities might include generating material such as case presentations, disease management seminars, and other clinically relevant material. This is something we could look into generating for the upcoming third year class of medical students. Finally, we could assess whether participating in crowdsourced flashcards changed students approaches to collaborative learning during the clinical years. It would be interesting to learn whether getting experience with collaborative learning in the preclinical years, which are pass/fail, in any way shaped their approach to, or their willingness to participate in collaborative learning during the clinical (tiered grading system) years.
      Overall, we’re very excited to see the ongoing positive effects of collaborative learning play out in medical education, and are happy to see the degree of enthusiasm and support these types of projects have generated at all levels of medical education.
      Thank you,
      Andrea Jonas

  2. A Crowdsourcing Model for Creating Preclinical Medical Education ... - MEU India
    June 30, 2013 at 7:45 AM

    […] By Hansen C. Bow, PhD, third-year medical student, Johns Hopkins University School of Medicine. About two years ago, Jon Dattilo, Andrea Jonas, and I started talking about how we could more effectively study our …Read More […]

  3. Kamrudeen Mohammed
    July 3, 2013 at 12:38 PM

    This is an excellent initiative; I can see potential benefits beyond pre-clinical years. Spaced repetition certainly has its values.
    Could I ask for a link to an easy method of learning to use the Java software for this particular purpose?

    Kamrudeen Mohammed

  4. Hansen Bow
    July 28, 2013 at 3:52 PM

    Hi Kamrudeen,

    You can download a copy of the Java software here:
    https://dl.dropboxusercontent.com/u/18898723/FC_V1.3.jar

    The instructions are the following:
    1. Click on the link to the left to download the program.
    2. Create a new file folder (e.g. on your desktop)
    3. Put the program and either an excel or powerpoint file in that folder
    4. Double-click the program to run
    5. Down = reveal answer; Left = wrong; Right = correct; Backspace = go back”

    I hope that helps. Please let me know if you have additional questions.

    Hansen

%d bloggers like this: