Editor’s Note: The following post is part of a series of Peer Reviewer Resources written by some of Academic Medicine‘s top peer reviewers. Read other peer review posts.

By: Carol-anne Moulton, MD, FRACS, MEd, PhD, Department of Surgery, University of Toronto, and Priyanka Patel, MSc, Wilson Center, University Health Network, University of Toronto

This is a tough task. Let us say that off the bat. We have been involved in qualitative research for a long time now and the complexity of it never ceases to amaze us…so there is no “how to” guide that will suit all qualitative research.

Having said that, we think there are some guiding principles that can help us begin to understand the rigor of qualitative research and consequently the review process.

1. Question/Purpose: This should be clearly stated, as in all research studies. There are generally no hypotheses statements in qualitative research as we are not testing but rather exploring. Ideally, the questions are framed around how and why type questions, rather than how often, is there a difference, or what are the factors type questions.
2. Rationale of study: We like to make sure that the study was built upon a well justified and referenced rationale. It may not be our area of study but we think it is important for the authors to provide rationale for their study by building up the arguments from the literature. Theories or pre-existing frameworks that informed the research question should be described up front. Some work claims to be atheoretical. Traditionally, grounded theorists claimed their work to be atheoretical, but nowadays many grounded theorists are acknowledging being informed by particular perspectives, frameworks, or theories. This should be made explicit.
3. Methodology described: What type of research was this? Ethnography? Grounded theory? Phenomenology? Discourse analysis? It’s important that the researchers describe their research journey in a clear and detailed enough way to give the readers an understanding for how the analyses evolved. This should include an explanation of why the methodological approach was used, as well as the key principles from the methodology that guided the study.
4. Epistemology: Researchers come from all paradigms and it is important to identify within which paradigm the authors are situated. Sometimes they might state deliberately “We have used constructivist grounded theory,” but it might be a matter of reading between the lines to figure it out. If from the positivist paradigm, authors might use the terms valid or verified to imply they are making statements of truth. The paradigm helps us understand what the authors mean by “truth” and informs how they went about creating knowledge and constructing meaning from their results.
5. Context described satisfactorily: Qualitative research is not meant to imply generalizability. In fact, we celebrate the importance of context. We recognize that the phenomena we study are often different in meaningful ways when taken to a different context. For example, the experiences of physicians coping with burnout may be unique based on specialty and/or institution (i.e. type of systems-level support available, differing demands in academic or community institutions). A good qualitative study should therefore describe sufficient details of context (i.e. physical, cultural, social, and/or environmental context) in which the research was conducted to allow the reader to make judgments of whether the results might be transferable to another (possibly their own) setting.
6. Data collection and analysis: Do they provide enough information to understand the collection and analysis process? As reviewers, we often ask ourselves whether the data collection and analyses are clear and detailed enough for us to gain a sense of how the analysis of the phenomena evolved. For example, who made up the research team? Because most knowledge is viewed as a co-construction between researcher and participants, each individual (e.g. a sociologist versus a surgeon) will analyze the results differently, but both meaningfully, based on their unique position and perspective.
7. Sampling strategies: These are very important to understand whether the question was aligned with the data collection process. The sample reflects the type of results achieved and helps the reader understand from which perspective the data was collected. Some common sampling strategies include theoretical sampling and negative case sampling. Researchers may theoretically sample by selecting participants that in someway inform their understanding of an emergent theme or idea. Negative case sampling may be used to search for instances that may challenge the emergent patterns from the data for the purpose of refining the analysis. Negative case sampling is used to ensure that the researchers are not specifically selecting cases that confirm their findings.
8. Analysis elevated beyond description: Results might be descriptive in nature (e.g. “One surgeon felt upset and isolated after he experienced a hernia complication in his first month of independent practice”) or they might be elevated to create more abstract concepts and ideas removed from the primary dataset (e.g. characterizing the phases of surgeons’ reactions to complications). In either case, the researcher should ensure that the way they present their findings are aligned with principles of the methodology used.
9. Proof of an iterative process: Qualitative research is usually done in an iterative manner where ideas and concepts are built up over time and occur through cycles of data collection and data analysis. This is demonstrated through statements like “Our interview template was altered over time to reflect the emergent ideas through the analysis process,” or “As we became interested in this concept, we began to sample for…”.
10. Reflexivity: This is tough to understand, especially for those of us who come from the positivist paradigm where it is of utmost importance to “prove” that the results are “true” and untainted by bias. The aim of qualitative research is to understand meaning rather than assuming that there is a singular truth or reality. A good qualitative researcher recognizes that the way they make sense of and attach meaning to the data is partly shaped by the characteristics of the researcher (i.e. age, gender, social class, ethnicity, professional status, etc.) and the assumptions they hold. The researcher should make explicit the perspectives they are coming from so that the readers can interpret the data appropriately. Consider a study exploring the pressures surgical trainees experience in residency conducted by a staff surgeon versus a non-surgical anthropologist. You can imagine the findings may differ based on the types of questions the two interviewers decide to ask, what they each find interesting or important, or how comfortable the resident feels discussing sensitive information with an outsider (anthropologist) as opposed to an insider (surgeon). We like to see that a researcher has reflected on how her or his unique position, preconceptions, and biases influenced the findings.