Performance of conceptual framework elements for the retrieval of qualitative health literature: a case study
DOI:
https://doi.org/10.5195/jmla.2021.1150Keywords:
systematic searching, qualitative reviews, PubMed, CINAHL, conceptualization modelsAbstract
Objective: A growing volume of studies address methods for performing systematic reviews of qualitative studies. One such methodological aspect is the conceptual framework used to structure the review question and plan the search strategy for locating relevant studies. The purpose of this case study was to evaluate the retrieval potential of each element of conceptual frameworks in qualitative systematic reviews in the health sciences.
Methods: The presence of elements from conceptual frameworks in publication titles, abstracts, and controlled vocabulary in CINAHL and PubMed was analyzed using a set of qualitative reviews and their included studies as a gold standard. Using a sample of 101 publications, we determined whether particular publications could be retrieved if a specific element from the conceptual framework was used in the search strategy.
Results: We found that the relative recall of conceptual framework elements varied considerably, with higher recall for patient/population (99%) and research type (97%) and lower recall for intervention/phenomenon of interest (74%), outcome (79%), and context (61%).
Conclusion: The use of patient/population and research type elements had high relative recall for qualitative studies. However, other elements should be used with great care due to lower relative recall.
References
Green J, Thorogood N. Qualitative methods for health research. Sage; 2018. ISBN 1526448807.
Campbell F, Weeks L, Booth A, Kaunelis D, Smith A. A scoping review found increasing examples of rapid qualitative evidence syntheses and no methodological guidance. J Clin Epidemiol. 2019 Nov;115:160–71. DOI: https://doi.org/10.1016/j.jclinepi.2019.05.032.
Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev. 2016 May 04;5:74. DOI: https://doi.org/10.1186/s13643-016-0249-x.
Noyes J, Booth A, Flemming K, Garside R, Harden A, Lewin S, Pantoja T, Hannes K, Cargo M, Thomas J. Cochrane Qualitative and Implementation Methods Group guidance paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018 May;97:49–58. DOI: https://doi.org/10.1016/j.jclinepi.2017.06.020.
Noyes J, Booth A, Cargo M, Flemming K, Garside R, Hannes K, Harden A, Harris J, Lewin S, Pantoja T, Thomas J. Cochrane Qualitative and Implementation Methods Group guidance series-paper 1: introduction. J Clin Epidemiol. 2018 May;97:35–38. DOI: https://doi.org/10.1016/j.jclinepi.2017.09.025.
Harris JL, Booth A, Cargo M, Hannes K, Harden A, Flemming K, Garside R, Pantoja T, Thomas J, Noyes J. Cochrane Qualitative and Implementation Methods Group guidance paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018 May;97:39–48. DOI: https://doi.org/10.1016/j.jclinepi.2017.10.023.
Harden A, Thomas J, Cargo M, Harris J, Pantoja T, Flemming K, Booth A, Garside R, Hannes K, Noyes J. Cochrane Qualitative and Implementation Methods Group guidance paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018 May;97:70–78. DOI: https://doi.org/10.1016/j.jclinepi.2017.11.029.
Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane Qualitative and Implementation Methods Group guidance paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018 May;97:79–85. DOI: https://doi.org/10.1016/j.jclinepi.2017.10.022.
Frandsen TF, Gildberg FA, Tingleff EB. Searching for qualitative health research required several databases and alternative search strategies: a study of coverage in bibliographic databases. J Clin Epidemiol. 2019 Oct;114:118–24. DOI: https://doi.org/10.1016/j.jclinepi.2019.06.013.
Rogers M, Bethel A, Abbott R. Locating qualitative studies in dementia on MEDLINE, EMBASE, CINAHL, and PsycINFO: A comparison of search strategies. Res Synth Methods. 2018 Dec;9(4):579–86. DOI: https://doi.org/10.1002/jrsm.1280.
Wong SS, Wilczynski NL, Haynes RB, Hedges T. Developing optimal search strategies for detecting clinically relevant qualitative studies in MEDLINE. Stud Health Technol Inform. 2004;107(Pt 1):311–6.
Wagner M, Rosumeck S, Kuffmeier C, Doring K, Euler U. A validation study revealed differences in design and performance of MEDLINE search filters for qualitative research. J Clin Epidemiol. 2020 Apr;120:17–24. DOI: https://doi.org/10.1016/j.jclinepi.2019.12.008.
Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol. 2018 Jan 10;18(1):5. DOI: https://doi.org/10.1186/s12874-017-0468-4.
Saini M, Shlonsky A. Systematic synthesis of qualitative research (Pocket guide to social work research methods). New York: Oxford University Press; 2012. p. xiv, 208.
Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012 Oct;22(10):1435–43. DOI: https://doi.org/10.1177/1049732312452938.
Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579. DOI: https://doi.org/10.1186/s12913-014-0579-0.
Booth A. Clear and present questions: formulating questions for evidence based practice. Libr Hi Tech. 2006;24(3):355–68. DOI: https://doi.org/10.1108/07378830610692127.
Pearson A. Evidence-based review in policy and practice, an issue of nursing clinics. E-book. Elsevier Health Sciences; 2014. ISBN 0323326633.
Dosenovic S, Jelicic Kadic A, Vucic K, Markovina N, Pieper D, Puljak L. Comparison of methodological quality rating of systematic reviews on neuropathic pain using AMSTAR and R-AMSTAR. BMC Med Res Methodol. 2018 May 8;18(1):37. DOI: https://doi.org/10.1186/s12874-018-0493-y.
Noyes J, Booth A, Cargo M, Flemming K, Harden A, Harris J, Garside R, Hannes K, Pantoja T, Thomas J. Qualitative evidence. In: Higgins JPT, Thomas J, Chandler J, Cumptson M, Li T, Page MJ, Welch VA, eds. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane, 2021. .
Booth A, Moore G, Flemming K, Garside R, Rollins N, Tuncalp O, Noyes J. Taking account of context in systematic reviews and guidelines considering a complexity perspective. BMJ Glob Health. 2019;4(Suppl 1):e000840. DOI: http://dx.doi.org/10.1136/bmjgh-2018-000840.
Petticrew M, Knai C, Thomas J, Rehfuess EA, Noyes J, Gerhardus A, Grimshaw JM, Rutter H, McGill E. Implications of a complexity perspective for systematic reviews and guideline development in health decision making. BMJ Glob Health. 2019;4(Suppl 1):e000899. DOI: http://dx.doi.org/10.1136/bmjgh-2018-000899.
Booth A, Noyes J, Flemming K, Moore G, Tunçalp Ö, Shakibazadeh E. Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ Global Health. 2019;4(Suppl 1):e001107. DOI: https://doi.org/10.1136/bmjgh-2018-001107.
Aromataris E, Munn Z, eds. Joanna Briggs Institute reviewer's manual [Internet]. The Joanna Briggs Institute; 2017. p. 299. Available from <https://reviewersmanual.joannabriggs.org/>.
Considine J, Shaban RZ, Fry M, Curtis K. Evidence based emergency nursing: designing a research question and searching the literature. Int Emerg Nurs. 2017 May;32:78–82. DOI: https://doi.org/10.1016/j.ienj.2017.02.001.
Frandsen TF, Bruun Nielsen MF, Lindhardt CL, Eriksen MB. Using the full PICO model as a search tool for systematic reviews resulted in lower recall for some PICO elements. J Clin Epidemiol. 2020;127:69–75. DOI: https://doi.org/10.1016/j.jclinepi.2020.07.005.
Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. J Med Libr Assoc. 2018;106(4):420–31. DOI: https://doi.org/10.5195/jmla.2018.345.
Ames HM, Glenton C, Lewin S. Parents' and informal caregivers' views and experiences of communication about routine childhood vaccination: a synthesis of qualitative evidence. Cochrane Database Syst Rev. 2017;2(2):CD0111787. DOI: https://doi.org/10.1002/14651858.cd011787.pub2.
Saada A, Lieu TA, Morain SR, Zikmund-Fisher BJ, Wittenberg E. Parents’ choices and rationales for alternative vaccination schedules: a qualitative study. Clin Pediatr (Phila). 2015;54(3):236–43. DOI: https://doi.org/10.1177/0009922814548838.
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page ML, Welch VA, eds. Cochrane handbook for systematic reviews of interventions. Version 6.0 (updated July 2019). Cochrane; 2019 [rev. Jul 2019]. Available from .
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016 Jul;75:40–6. DOI: https://doi.org/10.1016/j.jclinepi.2016.01.021.
Behrend D. CINAHL complete. The Charleston Advisor. 2020;22(2):26–9.
Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104. DOI: https://doi.org/10.1186/s13643-015-0069-4.
Mohammed MA, Moles RJ, Chen TF. Meta-synthesis of qualitative research: the challenges and opportunities. Int J Clin Pharm. 2016;38(3):695–704. DOI: https://doi.org/10.1007/s11096-016-0289-2.
Moseley AM, Elkins MR, Herbert RD, Maher CG, Sherrington C. Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy. J Clin Epidemiol. 2009;62(10):1021–30. DOI: https://doi.org/10.1016/j.jclinepi.2008.09.018.
Biocic M, Fidahic M, Cikes K, Puljak L. Comparison of information sources used in Cochrane and non‐Cochrane systematic reviews: A case study in the field of anesthesiology and pain. Res Synth Methods. 2019;10(4):597–605. DOI: https://doi.org/10.1002/jrsm.1375.
Gholizadeh M, Amir-Behghadami M, Janati A. Systematic reviews: are they actually well conducted and reported in accordance with PRISMA? Bull Emerg Trauma. 2020;8(1):51. DOI: https://dx.doi.org/10.29252%2Fbeat-080110.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108. DOI: https://doi.org/10.1111/j.1471-1842.2009.00848.x.
