Innovations in Education Scholarship and Scholarly Publishing: A Conversation with Lauren Maggio, PhD, MS

On By amroundsIn AM Podcast, AM Podcast Transcript

On this episode of the Academic Medicine Podcast, Lauren Maggio, PhD, MS(LIS), joins host Toni Gallo to discuss open science and innovations in education scholarship and scholarly publishing, including the role of AI. As the new editor-in-chief of MedEdPORTAL, Lauren also shares what makes the journal unique, her advice for authors, and her vision for the future.

This episode is now available through Apple Podcasts, Spotify, and anywhere else podcasts are available.

A transcript is below.

Read the articles discussed in this episode:

Check out these MedEdPORTAL resources:

  • Author Development Program, including monthly author Q&A sessions sign up
  • Become a reviewer: Email your inquiry and CV to mededportal@aamc.org. CVs will be vetted and if there is demonstrated alignment between the reviewer’s experience and expertise and the journal’s mission and focus, a reviewer account will be created. There is always a need for reviewers who are skilled in curricular design and evaluation and/or who have expertise in diversity, equity, and inclusion.
Lauren Maggio, the episode's guest, with the following text: the episode's title, Innovations in Education Scholarship and Scholarly Publishing, her institution, the University of Illinois College of Medicine at Chicago, and her title, Editor-in-Chief, MedEdPORTAL

Transcript

Toni Gallo:

Welcome to the Academic Medicine Podcast. I’m Toni Gallo. I am joined today by Dr. Lauren Maggio, who is a long-time Academic Medicine author, reviewer, and a former member of the journal’s editorial board. Lauren also started her term as editor-in-chief of MedEdPORTAL, the AAMC’s peer-reviewed journal of teaching and learning resources, at the beginning of this month.

In our conversation today, we’ll talk about Lauren’s research and scholarship that’s looking at open science and innovations in publishing, as well as some of her work on measuring and recognizing educators’ educational activities. Lauren has published much of this scholarship in Academic Medicine. The links to those articles are in the notes for today’s episode, and you can read all of those on academicmedicine.org. Definitely check out the papers that are on topics of interest to you. You can learn a lot more.

We’ll finish our conversation today with Lauren’s vision for MedEdPORTAL as she takes over the role of editor-in-chief, and we’ll get into the future of scholarly publishing and some of what Lauren sees happening there. So with that, I’ll ask you to introduce yourself for our listeners.

Lauren Maggio:

Great. Thanks Toni, and thanks for having me on today. As you said, I’m Lauren Maggio and I am a full professor at the University of Illinois Chicago in the Department of Medical Education. In that role, I’m also the department’s director of research, so I am thinking about research all the time. Also, as you mentioned, I’m the new editor-in-chief for MedEdPORTAL.

I’m excited today to talk about my research, as you mentioned and I mentioned, I think a lot about research. And in my research, what I like to do is blend my background in information science with my PhD in health professions education to get a sense of how we create information, how we disseminate it, and how we use it, and following that path has afforded me the ability to look at many different pieces of our scholarly process in HPE. So my new role is such an exciting opportunity to continue to look at a new journal with a unique mission through my lens as an information scientist and health professions educator.

Toni Gallo:

Thanks so much for being on the podcast today. I’m looking forward to our conversation. I thought we could start with some background for listeners. There’s lots of terms out there in the publishing world, whether it’s preprints or open science or open access, and I think there’s maybe a bit of confusion about how those terms are different and what they actually mean. And so could you just maybe go through some of these big trends in publishing or some of the big pieces and how you define them and think about them in terms of your scholarship?

Lauren Maggio:

Sure, I’m happy to do that. I’m going to actually start off with preprints because I get that question a lot. As someone who has the opportunity to work with a lot of early career researchers and students, there’s been a lot of interest in preprints.

So a preprint is the author’s version of a manuscript that is deposited to a preprint server, so a resource like bioRxiv or medRxiv. You deposit your manuscript to that website before peer review, which is the critical part, and that manuscript is now visible or that preprint is now visible to the world. You can get feedback. It comes with a digital object identifier or a DOI, and it can be listed on grants.

The key things are it is your copy that you’ve made public before peer review. I’ve started probably for the past four or five years. Anything I first author, I will move to one of the preprint servers so I can get it out there early.

We have a few pieces in Academic Medicine that I’ve had the opportunity to write more about what a preprint is and the benefits and some of the potential cautions that you should keep in mind. We’ve also started to do research around preprints in HPE or in medical education. And we are finding that our community is increasingly depositing preprints.

Now, preprints fit into that ecosystem of open science. I think of open science kind of as an umbrella of many different approaches to doing science. So a component of that is open access. And when we think about open access, that is the ability to freely read journal articles. But open science is actually broader than that. It also has to do with depositing your data, having a protocol of your research. A lot of it really comes down to transparency, about being open about what you are doing. So I think of that open science as the umbrella. Open access comes under that because we’re usually thinking about it in relation to journals and journal articles.

We also could talk about open data, which is when you deposit your data sets, making them freely accessible for someone to reuse, rerun. There’s also open code, which would be the same thing you would use it to rerun and make use of that code. We did a few Last Pages actually, so you could check those out. We did a Last Page on what is open access and what does that mean for your research. Again, looking at pros and cons of making your work open access. And then we have another piece that we looked at Creative Commons license because that is another component that comes under that umbrella of open science and those are licenses to help you think about how you want people to be able to reuse or remix your work.

Toni Gallo:

Could you walk us through… You’re a first author on a research project. How do you think about all of these things as part of the publishing process? What might that look like for you?

Lauren Maggio:

Yeah, and it varies by project. However, it always starts off by having a conversation with my team early. It’s really important that you ensure that everyone’s on board, they’re feeling comfortable with what’s going to happen. So early on, there might be a discussion about, “Do we want to deposit a study protocol?”

So let’s just use the example of a review. If we were doing a review, some of the review guidelines now require you to deposit a protocol of your work, which basically is your action plan of how you see that project laying out. So we discuss if we were going to do a protocol.

We might also have a conversation about, which journal do we want to publish this in. Do we want it to be in an open access journal? How important is that to us? We would also have the conversation of are we going to do a preprint of this piece of scholarship, because while the majority of journals do allow for preprints, not all of them do. So we’d want to have a thought about, “Okay, what are our target journals?” And I say target journals because it’s really hard to get your first pick now. Sometimes you kind of fall down the list. So again, it would happen early. We’d have a lot of those conversations.

And then we would come back to that conversation throughout the process because as we work through the materials, we would also be making sure we were designing our code so that it could be put up on GitHub so other people could run it. We’d be thinking about any of our data that comes out, how do we want to package it up so that we can put it on a repository like Zenodo. So all of the steps that we go through, you have to really think ahead of time, “How can I make sure that the products of my research are going to be usable?” And that’s a little bit different than how we sometimes think of the end part as the product.

We actually think about different parts of the process as being that product, and I really try to put those products out into a publicly accessible form where other people can use them. So an example, I’m working on a review right now. It created a gigantic data set. We got to look at a sliver of it. I realized that I’m never going to be able to look at everything in that data set. So we have published the data set–the articles–so that other people can go and look at that data set and cut it in different ways. And that was a decision that we made very early on, and we had to come to that decision as a team to decide that was really what we wanted to do.

Toni Gallo:

I thought it was interesting when you talked about all of these different components now are really part of your scholarship. It’s not just that final paper that gets published in the journal. It’s all of these different components from the code to the data set and all of that. How did your thinking about all of this evolve? Was this something early on that you started working on or have you sort of changed how you’ve thought about what scholarship really is?

Lauren Maggio:

Yeah, one of the thing is that I’ve been really lucky to get the opportunity to collaborate with people in the scholarly publishing field. Oftentimes we forget in our HPE world that there’s a whole other field out there that just studies scholarly communication and publishing. And so some of my collaborative work with those groups, it’s really opened my eyes to the different products that are produced and then thinking about some of these best practices for making sure our science is open and transparent.

Toni Gallo:

I think this feeds a little bit into then the work that you’re doing around educational activities and recognizing educational activities. That’s a big part of faculty members’ work is their teaching. And it’s not always that research study that they publish. And you’ve done some work around how do we measure all of those pieces and recognize all of the different components. So maybe you could talk a little bit about some of that work that you’ve done and maybe what innovations are happening in this space?

Lauren Maggio:

That’s a great question. One of the things I also wanted to mention is that when I do a preprint or I deposit code or have a data set, all of that stuff goes on my CV. And increasingly as you’re looking at some of the templates for different universities, you are seeing places on our P&T templates for putting in that content.

So in a positive way, it’s also being recognized by our universities as something that they want us to be doing, something they want us to put back out into the world. I am very privileged. I get to work with a lot of early career researchers and students, so I get to often think alongside them about promotion and tenure and what counts. And I think you hit the nail on the head when you said there is the research study, and of course that still counts, but there’s a lot of other forms of scholarship that also count and are really important. Many of us work as educators, so it’s important to have that scholarship of teaching and learning and for our educators to be able to get credit for that.

And I have to say that was one of the reasons I was attracted to MedEdPORTAL. I think it’s unique in that it is this journal of off-the-shelf educational activities and materials that you can use, you can implement directly into your classroom. And it’s also an open access, free to read, free to publish journal. That is a rarity that is incredibly valuable. And then if you take it that next level to not only can you read it for free, you can take it and implement it, and there’s no other journal that really has resources like that.

Toni Gallo:

Could you talk a little bit about that aspect of MedEdPORTAL? The “you can take a resource that’s freely available and use it in your own teaching.” How is that possible? Can you talk about the publishing model?

Lauren Maggio:

Yeah, so it is different from what we would think about in a journal when we see a research article. So to walk you through what you would see if you came to MedEdPORTAL, so if you access MedEdPORTAL, and you can do that through Medline, you can do that through Scopus or you can come directly to the website. The first thing you would see is something we call the ESR. And that is laid out more like a journal article with your intro, methods, results, and discussion. And really what that does is give you a high level view of what the educational activity or resource is. It also has evaluation of that resource so you can get a sense of how it worked.

But underpinning that ESR are a number of appendices and some have a lot of appendices, and these are really your materials that you’re going to implement. So it could be PowerPoint slides that have not only the content on the PowerPoint slides, but we’ll also have faculty notes to help you work through that deck. We have a lot of cases. There’s cases for standardized patients. I’ve been amazed by how much simulation and case content is in there. There’s also sometimes videos. There’s articulate files. And that content in those appendices is what you can take and you can use in your own classroom.

Additionally, we have a Creative Commons attribution license, which means you can use those materials. You would give attribution to the authors that created it, but you can also update those materials for your particular context, which as an educator myself, it takes so long to create educational materials. If you can find something that’s even a starter point for what you want to do in your classroom, that’s a huge gift so that you don’t have to spend that amount of time. So I’ve been amazed. I’ve been very pleased with how the authors, many, if not all of them, are educators themselves and they just get that. They get how they need to make the materials so that they can then be used.

Toni Gallo:

So I want to talk a little bit about your vision for MedEdPORTAL as you start your term as editor. Congratulations. And where do you see the journal fitting in the field? You’ve talked a little bit about that, of its unique role. But yeah, how are you thinking about the journal and its future? MedEdPORTAL turned 20, and so this is kind of the next chapter. How are you thinking about all of that as you start?

Lauren Maggio:

Yeah, it’s a big year for MedEdPORTAL. I’ve been tracking this throughout my career, so it’s so exciting to see it reach 20. Someone said we’re out of the tweens and we’re moving into adulthood. I really do see it sitting beside all of the journals in HPE as a partner or an adjunct to the research studies, again, that is really valuable for those educators that are implementing the work.

And in some cases we have people, they will have their MedEdPORTAL submission, they will continue to run it and then they move and they put their future study into a research article. So I think there’s lots of opportunity for the journals to work together and there to be good synergy. I know that past editors have worked very closely with Academic Medicine and there’s been lots of collaborative education. There’s been a lot of thinking about how resources can be shared between the journals. So I think there’s a lot of synergy between the two sister publications.

Looking to the future, I think the future is incredibly bright. I had the opportunity to twin for a little while with Grace Huang, the previous editor in chief. Every time I would look at the materials, I would just say, “Wow. Grace is already doing a great job. And so I feel like I’m in a good place to now think about the future.” And one of the places I want to continue to build is related to our education program. We already have materials and initiative around education for authors and for editors. And I’d like to take that to another level. We’ve been thinking about potentially partnering with professional associations and graduate programs to work along some of their mentored projects so that MedEdPORTAL is on their radar early as they begin to design their projects. And then not only is it on their radar, but they know what it would take to make a submission that would pass the bar for MedEdPORTAL.

Also, right now, I’ve been very impressed. We do have a global footprint. I’d like to make it larger, and again, partnering with different organizations that are already working in this space. As someone who works in a program that grants Master’s in Health Professions Education at UIC and previously at Uniformed Services University, I’m always impressed by the innovative projects that many of our students bring to these programs. And we work alongside them for months to develop the curricula or to develop the intervention. And oftentimes they had been looking straight ahead at that research article, but I think there’s this real opportunity for us to work with them not only to do the research article, but to make sure they are able to package up their materials so that we can use them.

I saw too many educational interventions kind of fall to the cutting room floor because you only get so much space in a research article. You don’t have all of that ability to lay out the appendices and really get those celebrated. So I’m looking forward to doing increased education, growing the global footprint. Also looking forward as we’re seeing technology is continually shifting in terms of file formats, in terms of different educational technology innovation that many of our amazing faculty are implementing in their classrooms. So I want to ensure that we’re moving ahead in that space.

And lastly, I would say I’m interested in also looking out to that broader scholarly communications environment to understand what’s happening at other journals, what are some of the innovations. Oftentimes when I talk to people, whether they’re clinical journal editors or social science journal editors, we’re all having similar problems. So there’s a real opportunity for us to learn from each other, to look out into the field, and to borrow some of those practices inside MedEdPORTAL. Obviously tailored to our unique nature, but that opportunity to not reinvent the wheel is valuable.

Toni Gallo:

You mentioned author development being a big piece of the mission of MedEdPORTAL. Do you have advice for authors or possible authors who might be listening, anything that kind of high level that folks should be thinking about? When they should come to MedEdPORTAL, what they should think about for their submissions? It’s, as we’ve said, not your typical research paper, so what advice might you have?

Lauren Maggio:

Yeah, one of the things I would say, similar to anything that you’re going to write up, make sure you can build the case for your resource. And the “so what?” Why should I care or why should someone care about this resource?

The other thing I would say, and this again, good practice for any publication you’re doing, but is to really document and make sure you’re saving all of your materials. Because at some point when you go to write it up, you’re going to be like, “Oh, well, why did we make that decision to make it online? Or why did we decide to break up the video into four versus three?” We want those details because those are the details that’s going to help someone implement. So track everything.

And then also as you build your PowerPoints, as you build your videos, think about how you would describe to a friend or a colleague what you were thinking as you designed that. So I think we often have facilitator guides which really walk you through all the details, including what was the setting, who are the learners. I just read a piece and you had to order Cornish hens and bottles and all these things. And all of that is built in there, but again, that’s really to make it so that you can take it off the shelf. Some of these things we just forget in the course of a project, so I would encourage you to document them early.

Another piece that I’ve been seeing a lot of would be also around the evaluation. So often we are so busy. And that when we put an intervention in place, we don’t often think about how we would evaluate that. And so I would encourage you to think early about how you might do that kind of evaluation. Keeping in mind that good evaluation takes often way more time than we think it might. So again, talk to people early. You might want to talk to experts in your evaluation office to think about how you might look at the efficacy of what you’ve done.

And the last thing, and this is one of my favorite things and I’ve been pushing on this in submissions, is tell me where it didn’t go perfect. I’ve implemented things and you realize like, “Oh wow, it was really hard to get the faculty on board to do this,” or, “Oh, it took me forever to do those videos, so I decided to do a different way.” We want to hear those places where it got a little bit sticky so that we can help the person who’s going to implement that work get past that. So we want to hear not only where things were difficult or sticky, we also want to hear how you overcame that or if you didn’t overcome that, because again, we don’t want people to fall into that trap.

So that is a little bit different. I know we’re all geared to think about how do we say we have the best intervention. And I’m sure your intervention’s great, but I’m sure hit some roadblocks along the way. And those I think are a really nice addition to MedEdPORTAL. And so I’ve been encouraging authors to put that upfront. It actually goes… I said upfront, but it goes in the discussion. But to include that kind of commentary and just honesty, because at the end of the day, we want our colleagues to succeed when they implement the submission.

Toni Gallo:

Thank you. That’s great advice. For listeners, think about that as you’re working on your next MedEdPORTAL submission and tell your friends too when they’re working on theirs.

Lauren Maggio:

Can I just make one more plug? So we offer every month, I want to say it’s the third or fourth Thursday of the month, it’s on the MedEdPORTAL website, we have kind of an Ask Me Anything where we have our education editors and associate editors that rotate through. And we invite authors to come and ask those questions that they might be having about a submission. I found that that’s been a great opportunity. We get great questions. We bring that back. We think about how we can begin to optimize things for authors. And hopefully we get people unstuck or further down the path.

So if you’re interested in MedEdPORTAL and you’re working on a piece or thinking about working on a piece, please feel free to jump into some of those office hours. I think you’ll find it very valuable.

Toni Gallo:

And we can put the link to sign up for those in the notes for this episode. So check that out.

My last question is around looking to the future of scholarly publishing. Are there certain things that you are excited about, that you’re watching, that you’re concerned about? How are you thinking about the future and where our field is going?

Lauren Maggio:

So the place my mind went first was the thing that keeps me up at night, which is around peer review. As we know, peer review is incredibly valuable. It’s important for our field. I think we’re feeling pain across all of the journals in terms of getting reviewers. And I completely respect the art and the gift that peer review is, and we’re getting peer reviews from volunteers across our community, but it is getting harder and harder to secure peer reviewers such that we’re going out to many multiple requests to get the two that we need.

I have not heard about the silver bullet. Editor-in-chiefs talk about this all the time. I think a lot of us are really concerned about peer review. We have seen some approaches out in the field. There’s things like Publons so that you can get credit for your peer reviews. There’s some journals that provide a discount for their publisher’s books. Some journals have thought about how do we incentivize monetarily? Nothing has worked great yet, but this is definitely an area. And I know at MedEdPORTAL there’s been a lot of work done to also do mentored reviews, to do training for people that are learning how to be reviewers. And I would say collectively across HP, all of the journals starting to think in this way and do some amazing training exercises for people that want to review.

Toni Gallo:

Have you thought at all about the role of AI in publishing? I know AI is everywhere in medical education and everyone’s thinking about it and trying out different things. How have you been thinking about AI when it comes to publishing and reviewing and your own scholarship?

Lauren Maggio:

Yeah, that’s a great question. So first and foremost, as an author, I think about it in relation to the COPE guidelines, that it’s something that should be declared. And I have to say, journals have done a great job where there is a place in the journal system, you’re often asked to put in your cover letter. So I think the use of AI is fine. I do think it needs to be declared and it should be clear where that’s been used.

For me, things get fuzzier when I start to think about it in the peer review process. Of course, when we are peer reviewing people’s work, that is their content. So we have to be very careful about uploading it into any kind of a system where we’re letting it go. So I would caution against that. I don’t think that’s our right to put things out to have the AI write our peer review.

I think in my own work, I’ve used it several times, again, declaring it. And I have to say, at first we were a little bit nervous like, “Oh, are the journals going to push back? Is it going to be a problem?” And I was writing a paper with a group of full professors and we’re like, “Let’s just try. Let’s just see. Let’s use it and see if they push back on us.” And fortunately, they didn’t. We used it to help us with the discussion. We declared it, and it was no problem.

So I think there is a role. I’ve found that I’ve been using it sometimes to help with an abstract or to help with the discussion and then polish things up. I do a lot of knowledge syntheses as well. I think we’re seeing that move into the knowledge syntheses space in terms of it helping us to extract data. So I think there’s going to be a lot of use for AI. I think we just have to think about how do we want to be AI literate in terms of understanding what you’re getting back from the AI.

And again, back to the topic of open science, I think a lot of it comes down to the transparency. And that’s what I’ve felt for many of the HPE editors, is, please declare it, be honest. And they’ve used the COPE guidelines and been very clear about it.

Toni Gallo:

So before we wrap up, any final thoughts or anything else you want to share with listeners about open science, around MedEdPORTAL? Just anything else?

Lauren Maggio:

Yeah, I would say in terms of open science, please familiarize yourself with the concept. I think sometimes people get confused, like you said, about what it is and understanding it’s an umbrella term. You don’t have to go all in it first. You might decide, “Okay, I’m going to try to do a preprint here, or perhaps I’m going to deposit my data for this project.” You can try out pieces. Like I said, don’t have to go all in. There might be parts or particular projects that you feel more comfortable participating or using open science practices, and that’s okay.

I would also say, increasingly, our journals are getting more comfortable with it. They understand it. They’re making places in their online systems where you can declare these different things. And so I think slowly we’re going to see increased practice of some of these open science practices coming into our journals. And I would say I’ve really enjoyed it.

Just as a plug for open science, as we deposit code or data, people from many different fields end up using some of our products, and that’s just been so gratifying to be able to engage with people throughout the world across different fields and give them that ability to start from our data.

Toni Gallo:

Well, thanks so much for being on the podcast today. I appreciate it. And I would encourage our listeners to check out the different papers that Lauren talked about. The links are in the notes for today’s episode. And if you are not familiar with MedEdPORTAL, definitely check out that journal as well and consider submitting your work if you’re doing teaching scholarship. So thanks very much.

Lauren Maggio:

Great. Thank you, Toni.

Toni Gallo:

From Academic Medicine’s website, you can access the latest articles and our archive dating back to 1926, as well as additional content like free eBooks and article collections.

Subscribe to Academic Medicine through the subscription services link under the Journal Info tab, or visit shop.lww.com and enter Academic Medicine in the search bar.

Be sure to follow us and interact with the journal staff on X, formerly Twitter, at @AcadMedJournal and on LinkedIn at Academic Medicine Journal. Subscribe to this podcast anywhere podcasts are available. Be sure to leave us a rating and a review when you do. Let us know how we’re doing. Thanks so much for listening.