Evidence-based practice is an essential program outcome in health care professional schools. Finding evidence to answer clinical questions is a critical competency for all health professionals to be evidence users (Melnyk, Gallagher-Ford, Long, & Fineout-Overholt, 2014; Melnyk et al., 2018). Academic librarians and health science librarians have expertise in information literacy skills and are well poised to teach these critical skills to nursing students, nurses, and other health care professional students and clinicians. These skills include, but are not limited to, selecting appropriate sources, constructing effective searches, defining keywords, and optimizing search strategies for specific databases. Skills and competencies for instruction are defined by the American Library Association's Framework for Information Literacy for Higher Education (2015) and Information Literacy Competency Standards for Nursing (2013).
Choosing the most relevant databases is essential to retrieve evidence to answer a focused clinical question (Cooper, Booth, Varley-Campbell, Britten, & Garside, 2018; Fineout-Overholt, Berryman, Hofstetter, & Sollenberger, 2019; McElhinney, Taylor, Sinclair, & Holman, 2016; McGrath, Brown, & Samra, 2012). Understanding the relationship of PubMed® to MEDLINE® also informs health care professionals, students, and clinicians in their selection of resources for evidence. Moreover, selecting and navigating databases to locate evidence can be a challenging and time-consuming process with a steep learning curve.
Thus, the authors set out to answer the question, “what is the best strategy to teach database selection?” Knowing that each clinical question is unique and that the availability of databases within academic institutions and within hospital settings is uneven, the authors wanted to know if the most relevant and well-matched databases for the clinical question might be effective for teaching how to locate the best available research. Selecting the most relevant databases to answer a clinical question can be taught with a specific search strategy, resulting in low recall with high precision.
Conversely, using all available databases to locate evidence can be taught with a sensitive search strategy. A sensitive search strategy would result in high recall and low precision, providing the health care professional or student many articles to evaluate. The authors desired to compare the two strategies to determine whether a specific search strategy (strategy A) might be as effective as a sensitive search strategy (strategy B) in finding evidence to answer the same focused clinical question.
In an undergraduate 4-year Bachelor of Science degree in nursing program, where a stand-alone, semester-long evidence-based practice course is taught, students are shown how to conduct a systematic review using a sensitive search strategy (strategy B) to locate the best available evidence. Search strategy B involves searching multiple databases in a systematic way using keywords, synonyms, and Boolean operators to locate evidence (Bramer, de Jonge, Rethlefsen, Mast, & Kleijnen, 2018; DeLuca et al., 2008). This search strategy increases the number of abstracts retrieved (high recall), thereby increasing the screening effort (Halladay, Trikalinos, Schmid, Schmid, & Dahabreh, 2015). A shortcoming of this strategy is the identification of many citations that are irrelevant (low precision) (Betrán, Say, Gülmezoglu, Allen, & Hampson, 2005). However, this strategy is beneficial to locate information that is unique or on a topic for which not much has been reported.
The alternate strategy is the specific search strategy (strategy A). A specific search strategy will return items more pertinent (high precision) to answering the clinical question. A shortcoming of using search strategy A is low recall which may result in missed results. None the less, the more focused the clinical question, a specific search strategy may produce all the evidence on the topic (Mileham, 2019).
A review of the literature suggests support for using the specific search strategy while confirming that more than one database needs to be searched. Searching only MEDLINE may result in missed studies (Beyer & Wright, 2013), as well as lead to bias in the results (Gerberi & Marienau, 2017; Halladay et al., 2015; Vassar et al., 2017). Halladay et al. (2015) found that searching databases beyond PubMed and EMBASETM produced a modest gain to answer their clinical questions; likewise, Kwon, Powelson, Wong, Ghali, and Conly (2014) found that searching EMBASE, PubMed, and CINAHL® were sufficient. Similarly, Hartling et al. (2016) conducted a cross-sectional study to assess the effect of searching select databases on the results of meta-analyses. They found that most of the relevant studies were identified in three databases.
The authors (one academic librarian and one nursing faculty member) conducted and compared the results of two search strategies—specific search strategy (strategy A) and sensitive search strategy (strategy B)—to answer the same focused clinical question: “In graduate students (P), how does practicing self-care interventions (I) compared to those not practicing self-care interventions (C) affect perceived stress (O) during graduate school (T)” (Stillwell, Vermeesch, & Scott, 2017, p. 508), with the aim of identifying the most effective and efficient search strategy to answer the clinical question.
The authors replicated a published systematic review (Stillwell et al., 2017) that used a specific search strategy to answer the aforementioned PICOT (Population, Intervention, Comparison, Outcome, Time) question using a sensitive search strategy. Both search strategies used (a) the same inclusion criteria (e.g., self-care was the intervention, graduate students were the population, and perceived stress, as measured by the Perceived Stress Scale [Cohen, Kamarck, & Mermelstein, 1983] was the outcome); (b) searched the same date range, and (c) included studies that were conducted in the United States, published in English, peer reviewed, and quantitative in design, and had institutional review board approval.
The systematic review by Stillwell et al. (2017) used search strategy A and included three databases. Each database was searched separately to optimize the use of a thesaurus and subject-specific vocabulary. These database features were used to identify the population of interest. To ensure search repeatability, keywords were used for all other elements of the PICOT question. The authors were confident that search strategy A would produce relevant results because the databases selected closely aligned with elements of the clinical question. The three databases used were EBSCO CINAHL Plus with Full Text, EBSCO MEDLINE, and EBSCO PsycINFO®.
The second systematic review, conducted by the authors, used a sensitive search strategy (strategy B). It included 12 health science databases: EBSCO Alt Health Watch, EBSCO CINAHL Plus with Full Text, EBSCO Health Source: Nursing Academic Edition, EBSCO MEDLINE, EBSCO PsycINFO, EBSCO SPORTDiscus, Ovid® Nursing and Biomedical Journals, ProQuest® Nursing & Allied Health, ProQuest PILOTS, ProQuest Public Health, PubMed, and Wiley Cochrane Library. The number of studies identified using search strategy B was then compared with the number of studies identified with search strategy A.
Comparing the results obtained from the two search strategies, the authors found that search strategy B produced more than twice the number of articles to screen as was produced by search strategy A. Although the recall was greater for search strategy B, the precision of search strategy A resulted in fewer irrelevant articles to screen. The precision of a search is defined as “the proportion of relevant studies identified by a search strategy expressed as a percentage of all studies (relevant and irrelevant) identified by that method” (Khan, Kunz, Kleijnen, & Antes, 2011, p. 190). The precision of each search strategy was calculated. Search strategy B had a precision of 0.08%, identifying 11,362 potential articles with nine being relevant, whereas search strategy A was twice as precise (0.16%), identifying 5,108 potential articles with eight being relevant (Figure 1).
Comparison of results from a sensitive and specific search. From “Interventions to reduce perceived stress among graduate students: A systematic review with implications for evidence-based practice,” by S. Stillwell, A. Vermeesch, and J. Scott, 2017, Worldviews on Evidence Based Nursing, 14, 507–513. Copyright 2017 by John Wiley & Sons. Adapted with permission.
However, search strategy B identified one additional study not found with search strategy A. The additional study (Greeson, Toohey, & Pearce, 2015) was appraised by the authors and was found to be valid and reliable. The authors evaluated and graded the additional study and found that the outcome in the Greeson et al. (2015) study corroborated the outcome of the Stillwell et al. (2017) systematic review in that practicing self-care interventions reduced perceived stress in graduate students.
When implementing a practice change, it is important to search relevant databases as indicated so that studies are not missed (Fineout-Overholt et al., 2019). Therefore, the authors contend that a critical element of instruction with nursing students and clinicians, as well as with other health care professionals, is the selection of relevant databases before a search is undertaken. Aligning each clinical question to the databases available is of primary importance and the time spent doing so is efficient and effective. Reading descriptions of databases, consulting the librarian, reviewing LibGuides with the most relevant resources for nursing would all be helpful ways to determine appropriate databases to search. Databases are expensive and those subscribed to will vary in academic institutions and clinical settings. Every PICOT question is distinct, and as the literature shows, the number of databases needed to find the best available evidence depends on the question and available research.
Part of database selection is knowing which databases aggregate the research of specific disciplines. Using a subject database when appropriate, such as PsycINFO, is vital to locate relevant articles (Hartling et al., 2016; Stevinson & Lawlor, 2004; Suarez-Almazor, Belseck, Homik, Dorgan, & Ramos-Remus, 2000; Woodman et al. 2010), thus increasing precision of the search. Given that the outcome of the PICOT question used in this comparison was stress and the intervention included mind–body stress reduction behavior, selecting a subject database such as PsycINFO was key. The PsycINFO database was effective in locating five of the nine relevant articles with both the specific and sensitive search strategies. Further, the PsycINFO database produced high precision (56%); however, not all relevant articles were identified, highlighting the fact that more than one database is necessary to find all relevant evidence.
The authors recognized that if the Stillwell et al. (2017) systematic review had used the PubMed database instead of MEDLINE, the Greeson et al. (2015) study would have been identified. The authors learned that PubMed is a more current database than MEDLINE (Duffy et al., 2016; Katchamart, Faulkner, Feldman, Tomlinson, & Bombardier, 2011; National Library of Medicine, 2018; Rice et al., 2016). In fact, a delay of a median of 10.5 months may exist from the time a citation is entered in PubMed and when it appears in MEDLINE (Cooper et al., 2018; Duffy et al., 2016). The specific search strategy would have produced the same studies to answer the focused clinical question if PubMed instead of MEDLINE had been searched.
From a review of the health science literature, this is the only study to the authors' knowledge that compares the same clinical question using both a specific search strategy and a sensitive search strategy. A strength of this comparison is the use of the same search time frame, definitions of terms, inclusion criteria, and similar search structures. The results of the comparison corroborate the conclusions drawn by Beyer and Wright (2013), Ogilvie, Hamilton, Egan, and Petticrew (2005), Hartling et al. (2016), and Kwon et al. (2014) that for focused clinical questions, searching beyond the most relevant health science databases plus a relevant subject specific database does not produce noteworthy additional evidence. But as those researchers noted, generalizability to all clinical questions is limited. Although our comparison leads to an increased awareness of the importance of database selection, a replication of a sensitive and specific search strategy to answer a different health care–related PICOT question that includes PubMed and a subject specific database, if warranted, is recommended.
Contributing to the existing body of knowledge of database selection, the comparison of the sensitive search to the specific search gives support to instructing nursing students and clinicians, and other health care professionals in the importance of aligning the available and relevant databases to the clinical question. Knowing how to create an appropriate PICOT question to guide the search and how to align the databases to the question requires instructional emphasis. Additionally, health care and nurse educators should ensure that students and clinicians understand the importance of searching PubMed for a focused clinical question. The authors recommend that time may be well spent in collaboration with an academic or clinical librarian or a clinical or academic nurse educator refining the focused clinical question and determining relevant databases prior to the search. Future studies comparing a sensitive search strategy with a specific search strategy to answer the same clinical PICOT question would extend the knowledge on effective and efficient search strategies for evidence-based practice.
- American Library Association. (2013). Information literacy competency standards for nursing. Retrieved from http://www.ala.org/acrl/standards/nursing
- American Library Association. (2015). Framework for information literacy for higher education. Retrieved from http://www.ala.org/acrl/standards/ilframework
- Betrán, A.P., Say, L., Gülmezoglu, A.M., Allen, T. & Hampson, L. (2005). Effectiveness of different databases in identifying studies for systematic reviews: Experience from the WHO systematic review of maternal morbidity and mortality. BMC Medical Research Methodology, 5(1), 6. https://doi.org/10.1186/1471-2288-5-6 PMID: doi:10.1186/1471-2288-5-6 [CrossRef]15679886
- Beyer, F.R. & Wright, K. (2013). Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Information and Libraries Journal, 30(1), 49–58 Retrieved from https://doi.org/doi:10.1111/hir.12009. https://doi.org/10.1111/hir.12009 PMID: doi:10.1111/hir.12009 [CrossRef]23413793
- Bramer, W.M., de Jonge, G.B., Rethlefsen, M.L., Mast, F. & Kleijnen, J. (2018). A systematic approach to searching: An efficient and complete method to develop literature searches. Journal of the Medical Library Association, 106(4), 531–541. https://doi.org/10.5195/JMLA.2018.283 PMID: doi:10.5195/JMLA.2018.283 [CrossRef]30271302
- Cohen, S., Kamarck, T. & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385–396. https://doi.org/10.2307/2136404 PMID: doi:10.2307/2136404 [CrossRef]6668417
- Cooper, C., Booth, A., Varley-Campbell, J., Britten, N. & Garside, R. (2018). Defining the process to literature searching in systematic reviews: A literature review of guidance and supporting studies. BMC Medical Research Methodology, 18(1), 85. https://doi.org/10.1186/s12874-018-0545-3 PMID: doi:10.1186/s12874-018-0545-3 [CrossRef]30107788
- DeLuca, J.B., Mullins, M.M., Lyles, C.M., Crepaz, N., Kay, L. & Thadiparthi, S. (2008). Developing a comprehensive search strategy for evidence based systematic reviews. Evidence Based Library and Information Practice, 3(1), 3–32. https://doi.org/10.18438/B8KP66 doi:10.18438/B8KP66 [CrossRef]
- Duffy, S., de Kock, S., Misso, K., Noake, C., Ross, J. & Stirk, L. (2016). Supplementary searches of PubMed to improve currency of MEDLINE and MEDLINE In-Process searches via Ovid. Journal of the Medical Library Association, 104(4), 309–312. https://doi.org/10.3163/1536-5050.104.4.011 PMID: doi:10.3163/1536-5050.104.4.011 [CrossRef]27822154
- Fineout-Overholt, E., Berryman, D., Hofstetter, S. & Sollenberger, J. (2019). Finding relevant evidence to answer clinical questions. In Melnyk, B. & Fineout-Overholt, E. (Eds.), Evidence-based practice in nursing & healthcare: A guide to best practice (4th ed., pp. 40–70). Wolters Kluwer Health.
- Gerberi, D. & Marienau, M.S. (2017). Literature searching for practice research. American Association of Nurse Anesthetists Journal, 85(3), 195–204 PMID:31566556
- Greeson, J.M., Toohey, M.J. & Pearce, M.J. (2015). An adapted, four-week mind-body skills group for medical students: Reducing stress, increasing mindfulness, and enhancing self-care. Explore (New York, N.Y.), 11(3), 186–192. https://doi.org/10.1016/j.explore.2015.02.003 PMID: doi:10.1016/j.explore.2015.02.003 [CrossRef]
- Halladay, C.W., Trikalinos, T.A., Schmid, I.T., Schmid, C.H. & Dahabreh, I.J. (2015). Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. Journal of Clinical Epidemiology, 68(9), 1076–1084. https://doi.org/10.1016/j.jclinepi.2014.12.017 PMID: doi:10.1016/j.jclinepi.2014.12.017 [CrossRef]26279401
- Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, D.M. & Vandermeer, B. (2016). The contribution of databases to the results of systematic reviews: A cross-sectional study. BMC Medical Research Methodology, 16(1), 127. https://doi.org/10.1186/s12874-016-0232-1 PMID: doi:10.1186/s12874-016-0232-1 [CrossRef]27670136
- Katchamart, W., Faulkner, A., Feldman, B., Tomlinson, G. & Bombardier, C. (2011). PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. Journal of Clinical Epidemiology, 64(7), 805–807. https://doi.org/10.1016/j.jclinepi.2010.06.004 PMID: doi:10.1016/j.jclinepi.2010.06.004 [CrossRef]
- Khan, K., Kunz, R., Kleijnen, J. & Antes, G. (2011). Systematic reviews to support evidence medicine: How to review and apply findings of healthcare research (2nd ed.). New York, NY: CRC Press.
- Kwon, Y., Powelson, S.E., Wong, H., Ghali, W.A. & Conly, J.M. (2014). An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks. Systematic Reviews, 3, 135. https://doi.org/10.1186/2046-4053-3-135 PMID: doi:10.1186/2046-4053-3-135 [CrossRef]25387523
- McElhinney, H., Taylor, B., Sinclair, M. & Holman, M.R. (2016). Sensitivity and specificity of electronic databases: The example of searching for evidence on child protection issues related to pregnant women. Evidence-Based Midwifery, 14(1), 29–34.
- McGrath, J.M., Brown, R.E. & Samra, H.A. (2012). Before you search the literature: How to prepare and get the most out of citation databases. Newborn and Infant Nursing Reviews, 12(3), 162–170. https://doi.org/10.1053/j.nainr.2012.06.003 doi:10.1053/j.nainr.2012.06.003 [CrossRef]
- Melnyk, B.M., Gallagher-Ford, L., Long, L.E. & Fineout-Overholt, E. (2014). The establishment of evidence-based practice competencies for practicing registered nurses and advanced practice nurses in real-world clinical settings: Proficiencies to improve healthcare quality, reliability, patient outcomes, and costs. Worldviews on Evidence-Based Nursing, 11(1), 5–15. https://doi.org/10.1111/wvn.12021 PMID: doi:10.1111/wvn.12021 [CrossRef]24447399
- Melnyk, B.M., Gallagher-Ford, L., Zellefrow, C., Tucker, S., Thomas, B., Sinnott, L.T. & Tan, A. (2018). The first U.S. study on nurses' evidence-based practice competencies indicates major deficits that threaten healthcare quality, safety, and patient outcomes. Worldviews on Evidence-Based Nursing, 15(1), 16–25. https://doi.org/10.1111/wvn.12269 PMID: doi:10.1111/wvn.12269 [CrossRef]
- Mileham, P. (2019). Finding sources of evidence. In Schmidt, N. A. & Brown, J. M. (Eds.), Evidence–based practice for nurses: Appraisal and application of research (4th ed., pp. 92–129). Burlington, MA: Jones & Bartlett Learning.
- National Library of Medicine. (2018). MEDLINE, PubMed, and PMC (PubMed Central): How are they different. Retrieved from https://www.nlm.nih.gov/bsd/difference.hmtl
- Ogilvie, D., Hamilton, V., Egan, M. & Petticrew, M. (2005). Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go?Journal of Epidemiology and Community Health, 59(9), 804–808. https://doi.org/10.1136/jech.2005.034181 PMID: doi:10.1136/jech.2005.034181 [CrossRef]16100321
- Rice, D.B., Kloda, L.A., Levis, B., Qi, B., Kingsland, E. & Thombs, B.D. (2016). Are MEDLINE searches sufficient for systematic reviews and meta-analyses of the diagnostic accuracy of depression screening tools? A review of meta-analyses. Journal of Psychosomatic Research, 87, 7–13. https://doi.org/10.1016/j.jpsychores.2016.06.002 PMID: doi:10.1016/j.jpsychores.2016.06.002 [CrossRef]27411746
- Stillwell, S.B., Vermeesch, A.L. & Scott, J.G. (2017). Interventions to reduce perceived stress among graduate students: A systematic review with implications for evidence-based practice. Worldviews on Evidence-Based Nursing, 14(6), 507–513. doi:10.1111/wvn.12250 [CrossRef]28795775
- Stevinson, C. & Lawlor, D.A. (2004). Searching multiple databases for systematic reviews: Added value or diminishing returns?Complementary Therapies in Medicine, 12(4), 228–232. https://doi.org/10.1016/j.ctim.2004.09.003 PMID: doi:10.1016/j.ctim.2004.09.003 [CrossRef]
- Suarez-Almazor, M.E., Belseck, E., Homik, J., Dorgan, M. & Ramos-Remus, C. (2000). Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Controlled Clinical Trials, 21(5), 476–487. https://doi.org/10.1016/S0197-2456(00)00067-2 PMID: doi:10.1016/S0197-2456(00)00067-2 [CrossRef]11018564
- Vassar, M., Yerokhin, V., Sinnett, P.M., Weiher, M., Muckelrath, H., Carr, B. & Cook, G. (2017). Database selection in systematic reviews: An insight through clinical neurology. Health Information and Libraries Journal, 34(2), 156–164. https://doi.org/10.1111/hir.12176 PMID: doi:10.1111/hir.12176 [CrossRef]28383159
- Woodman, J., Harden, A., Thomas, J., Brunton, J., Kavanagh, J. & Stansfield, C. (2010). Searching for systematic reviews of the effects of social and environmental interventions: A case study of children and obesity. Journal of the Medical Library Association, 98(2), 140–146. https://doi.org/10.3163/1536-5050.98.2.006 PMID: doi:10.3163/1536-5050.98.2.006 [CrossRef]20428279