In 2015, the PhD faculty at the University of Illinois at Chicago, College of Nursing launched a new, PhD-level course, Developing Literature Reviews. The aim of the course was to provide intensive instruction to PhD students on conducting rigorous literature reviews. The course objective set by the faculty was to have each PhD student prepare a draft of a literature review manuscript suitable for publication submission. In 2016, the course became a requirement for all PhD students and is offered each fall over a 16-week semester. The author was asked to develop and teach the course since its implementation in 2015.
Pursuant to the course objective, students are required to complete a literature review that uses an explicit, rigorous, and reproducible (e.g., structured) methodological process such as those used in integrative, systematic, or scoping literature reviews. The requirement to have students complete a structured literature review was set based on the author's own scientific experience with completing literature reviews (Hershberger, 2004; Hershberger, 2016; Hershberger & Pierce, 2010), the trajectory toward increased methodological rigor and transparency when conducting literature reviews (Higgins et al., 2019), the high level of evidence generated by structured reviews (Gregory & Denniss, 2018; Wormald & Evans, 2018), and the influence of literature reviews on future studies, PhD dissertations, clinical practice, health care interventions, and health care policy (Higgins et al., 2019).
Development of the Template and Other Course Materials
The author developed course materials such as reading lists, lectures, small-group projects, and class discussions, along with other novel strategies, to aid PhD students in achieving the course objective. These pedagogic materials and strategies are used throughout the semester to provide foundational knowledge about the appropriate steps and components for completing structured literature reviews. For example, instructional texts (Cooper et al., 2019; Garrard, 2017; Higgins et al., 2019), informative journal articles (Haddaway et al., 2015; Spurlock, 2019; Sutton et al., 2019; Whittemore et al., 2014), and expert recommendations (Cochrane Collaboration, n.d.; Joanna Briggs Institute, n.d.; Ottawa Hospital Research Institute & University of Oxford, 2015) are incorporated throughout the semester to facilitate student understanding and comprehension regarding all aspects of literature reviews. Novel course strategies include collaborating with a journal editor to allow for active student engagement in a journal manuscript peer review while under direct faculty supervision (Monsivais, 2016), inviting journal editors as guest lecturers to speak directly with students about developing and submitting literature reviews to professional journals, and engaging former students who have disseminated their literature reviews to discuss facilitators and challenges to dissemination.
Considering the efficient use of student and faculty time and a need to communicate consistent course requirements to foster student learning and successful literature review dissemination, a literature review manuscript template was created. The design of the template accommodates the wide range of literature review types (e.g., integrative, systematic, scoping) often completed by PhD nursing students. At the same time, the template provides concise and consistent information to streamline the completion of each student's literature review, regardless of review type. In developing the template, the author considered the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Ottawa Hospital Research Institute & University of Oxford, 2015) and recommendations from other methodological experts (Moher et al., 2009; National Heart, Lung, and Blood Institute, n.d.-a).
A unique yet valuable feature of the template is that its design also aims to simulate the peer-review process. Using an iterative course-assignment process, students are required to submit sections of their developing literature review manuscript incrementally throughout the semester for faculty evaluation and feedback. With each manuscript section submission, students receive constructive feedback and recommendations for improving the manuscript from course faculty, which is incorporated into the template document. Students are expected to revise their manuscript and address the faculty's recommendations using an appropriate professional response detailed within the template. Incorporating the requirement of students to provide professional evaluation responses is consistent with simulating the peer-review process and provides students with an opportunity for a hands-on experience under direct faculty guidance.
Template Sections and Components
The template consists of eight sections that correspond with the main sections of structured literature review manuscripts (e.g., Abstract, Methods, Results). Each section contains a corresponding table where essential components are listed. The components serve to communicate specific requirements that must be satisfied for course completion and for each section of the manuscript. They also serve as benchmarks for faculty evaluation of each section to ensure the components are clearly defined and consistently evaluated. Each section is denoted within the template by a corresponding table, and the faculty evaluations and student responses are in an open-ended format. Each of the eight template sections and components are described below, along with examples of course pedagogy. The Results section of the template and an exemplar of the faculty and student responses within the table that simulates the peer-review process is provided in Table 1. The full template with additional exemplars of iterative faculty evaluations and student responses to the peer-review simulation is shown in Table A (available in the online version of this article).
Template Exemplar: Results Section With Faculty Evaluation and Student Response
Literature Review Manuscript TEMPLATE [Title Page]
Title Page Section
A title for the literature review manuscript and a list of coauthors, which includes the student's PhD faculty advisor, are required components. Students are expected to work closely with their advisor for topical expertise. Methodological strategies, which are provided in the course, along with their advisor's topical expertise, ensure a necessary mix of skills and knowledge to conduct a high-quality review (Lasserson et al., 2019). Authorship information from course readings and lectures is presented in the course, which is guided by recommendations from the International Committee of Medical Journal Editors (n.d.). Selecting a target journal for the manuscript (Gennaro, 2019; Likis, 2018) is also a required component. Students are instructed to incorporate and comply with the author guidelines of the target journal for their manuscript. Thus, if the author guidelines limit the number of words in a title, students are required to abide by the guidelines when drafting the title.
Common abstract headings (e.g., Problem/Background, Purpose, Methods) are provided in the template to allow students to comprehend typical abstract headings. However, students are instructed to comply with the author guidelines required by the target journal for abstract headings, as well as the number of words allowed in the abstract. Course instruction pertains to developing a review abstract that is an unbiased representation of the full manuscript and links directly to the sections within the manuscript (Beller et al., 2013). Lectures provide examples of various abstracts from several published reviews. Class discussion is encouraged by identifying similarities and differences across abstract requirements from the students' target journals. A component of the abstract section is the identification of keywords. The importance of keywords is discussed, and a short class exercise demonstrating the benefits of linking keywords to medical subject headings is provided.
Introduction and Background Section
The template stipulates main components of the Introduction and Background section, including (a) a brief summary of relevant literature (including any prior literature reviews) and background knowledge, (b) identification of the gaps in prior knowledge to address the rational for completing the literature review, (c) use of appropriate supporting references (e.g., primary sources, current or classic literature), and (d) a purpose statement or review question that concludes the section. The page limit is set at one to two pages because of the variability in the length of the Introduction and Background section required by journals. Class lectures and discussions provide insight on the components of the Introduction and Background section, such as summarizing relevant literature from a broad to specific approach (Bahadoran et al., 2018; Oermann & Hays, 2019). The Introduction and Background section of published reviews are also dissected into components (e.g., summary of background knowledge, purpose statement) during course lectures to provide examples of the essential components and to stimulate in-class discussion and learning.
The Methods section requires five components students must address in one to four paragraphs. The five components are (a) providing one or more methodological references for guiding the review, (b) detailing the search and retrieval strategies, (c) describing the inclusion and exclusion criteria along with the scientific rational for the criteria, (d) providing information about how the data analysis and synthesis was conducted, and (e) describing the quality appraisal tool used in the review. Because the Methods section is vital to transparency for any scientific project, it is critical that students provide clear and accurate information about each of the five components in their literature review manuscripts. Multiple in-class discussions, lectures, individual and group projects, and required readings aid in the students understanding of the components. For example, information on methodological techniques that guide specific literature review types are provided in lectures and course readings (Arksey & O'Malley, 2005; Higgins et al., 2019; Peters et al., 2015; Whittemore & Knafl, 2005). Additional readings and lectures instruct students about search, selection, and retrieval strategies (Atkinson et al., 2015; Paez, 2017; Spurlock, 2019) and the construction of a PRISMA flow diagram to visually map out the identification and selection process (Moher et al., 2009). A health sciences librarian who is highly skilled in reviews provides individual and group sessions to students regarding comprehensive electronic search and retrieval strategies. Lectures covering how to select and use quality appraisal tools (e.g., Crowe et al., 2012; Des Jarlais et al., 2004; National Heart, Lung, and Blood Institute, n.d.-b) that align with review types to systematically appraise the quality of included studies are provided.
Four components are required for the Results section, including (a) a minimum of one paragraph that describes the overall characteristics of the studies included in the review; (b) description of the themes, patterns, or other subsections of the findings from the analysis and synthesis that address the purpose or research question(s); (c) a literature review table that details characteristics and/or variables from each of the identified studies in the review; and (d) a brief description of the findings from the selected quality appraisal tool(s). Students are required to submit three to five pages devoted to addressing these four components. Required readings (Cooper, 2017; Garrard, 2017; Higgins et al., 2019) targeted individual and group exercises, and in-class discussions with exemplars aid in instruction and comprehension of the analytic processes necessary to formulate the Results section. Because students often find the analysis and synthesis to be challenging, lectures that detail specific analytic processes such as delineating data into data “types” (i.e., numerical, categorical, or narrative) are used as an effective pedagogic strategy (Dunn Lopez et al., 2020). This strategy includes lectures and discussions that provide examples of types of data along with the appropriate analysis and synthesis. Specifically, descriptive statistics or pooled statistical methods are appropriate for numerical data; sorting, filtering, and summing of categories are appropriate for categorical data; and content analysis or other similar procedures are appropriate for narrative or textual data (Dunn Lopez et al., 2020). Another effective strategy is to require students to bring their developing Results section to class. An intensive hands-on experience is then undertaken where students' data serve as further exemplars for data types and analyses and syntheses activities. Students are required to complete a peer review of each other's work after they complete the Results section.
The template highlights five components of a well-written Discussion section, including (a) provide a succinct statement of how the findings address the purpose of the review or the research question(s) and how these findings impact the field (e.g., the “So what?” question); (b) detail what the findings mean by stating how the findings add to the current scientific literature (e.g., the “What does it mean?” question); (c) provide a short limitations paragraph; (d) describe how the findings guide future research, and if appropriate, can impact clinical practice; and (e) finish with a short summary paragraph. Lectures, course readings, and class discussions are critical to helping students understand the components of the Discussion section. Course instruction provides guidance about the components such as succinctly addressing the purpose/review question in the Discussion section (Ghasemi et al., 2019); crafting a discourse of the findings within the current scientific milieu (Kearney, 2017); including supporting literature and, when appropriate, contradictory literature (Höfler et al., 2018; Jenicek, 2006); avoiding overgeneralizations (Conn, 2017); writing the limitations subsection (Conn, 2017; Kearney, 2017); and creating a summary or concluding paragraph (Ghasemi et al., 2019). Student-to-student peer review following the completion of the Discussion section is optional but encouraged, even if it is completed after the course has officially ended.
Course lectures, discussions, and readings pertain to selecting appropriate references to support statements and claims within the manuscript (Oermann & Hays, 2019). The template does not specify a standard citation style (e.g., American Psychological Association format). Rather, students are to use the style guide directed by the target journal. This requirement enables students to be better prepared for manuscript submission versus managing incongruent target journal citation styles requirements.
To measure student satisfaction, the students' standardized course evaluations administered by the University of Illinois at Chicago, College of Nursing were analyzed using descriptive statistics. In addition, the number of students who were successful in disseminating manuscripts was recorded and tallied by the author. Student responses to standardized course evaluation questions such as “Overall quality of instruction was high” ranged from 4.06 to 4.62 over 5 years (mean = 4.35; 5-point Likert-scale, where 0 = strongly disagree and 5 = strongly agree). Open-ended student responses to standardized questions were also positive. Typical responses were: “This is a great course that all in the PhD program should take. I definitely learned the process of literature review and hope to publish my paper,” and “Learning to write a high-quality scientific literature review was of tremendous benefit as a graduate student. Building a literature review [manuscript] throughout the semester is a great and useful way to approach the class.”
To date, 18 students have successfully published integrative and systematic literature reviews as first author in a wide range of scientific journals. Two students have published more than one literature review. One student published an integrative review during the PhD program and a meta-analysis after graduating. Eleven students have presented their peer-reviewed abstracts at international, national, or regional scientific conferences as first authors. These numbers reflect submission to professional journals during the student's PhD program (n = 16), after graduation (n = 1), or during a postdoctoral fellowship (n = 1) as one student enrolled in the course during their fellowship. All abstract submissions occurred during the student's PhD program (n = 11).
Discussion and Conclusion
The use of a well-crafted manuscript template and the peer-review simulation is one pedagogic strategy that can facilitate student learning, completion, and dissemination success in the modern, time-pressured academic environment for both PhD students and course faculty. Other course strategies such as required readings, lectures, class discussions, and individual and group projects provide support for student learning and engagement. It is worth noting the course is offered in a research-intensive environment and the students' PhD faculty advisors, who often serve as coauthors, are highly skilled scientists. Therefore, the success of PhD students in other academic settings, even with the pedagogy and template described in this article, may vary. Despite this limitation and although the template and course instruction has focused on PhD students, Doctor of Nursing Practice students would likely benefit from similar pedagogy and a modified template that places translation of evidence into practice as a priority.