PubMed's core clinical journals filter: redesigned for contemporary clinical impact and utility

Objective: The Core Clinical Journals (CCJ) list, produced by the U.S. National Library of Medicine (NLM), has been used by clinicians and librarians for half a century for two main purposes: narrowing a literature search to clinically useful journals and identifying high priority titles for library collections. After documentation of low usage of the existing CCJ, a review was undertaken to assess current validity, followed by an update to current clinical needs. Methods: As the subject coverage of the 50-year-old list had never been evaluated, the CCJ committee began its innovative step-wise approach by analyzing the existing subject scope. To determine whether clinical subjects had changed over the last half-century, the committee collected data on journal usage in hospitals and medical facilities, adding journal usage from Morning Report blogs recording the journal article citations used by physicians and residents in response to clinical questions. Patient-driven high-frequency diagnoses and subjects added contextual data by depicting the clinical environment. Results: The analysis identified a total of 80 subjects and selected 241 journals for the updated Clinical Journals filter, based on actual clinical utility of each journal. Discussion: These data-driven methods created a different framework for evaluating the structure and content of this filter. It is the real-world evidence needed to highlight CCJ clinical impact and push clinically useful journals to first page results. Since the new process resulted in a new product, the name warrants a change from Core Clinical Journals (CCJ) to Clinically Useful Journals (CUJ). Therefore, the redesigned NLM Core Clinical Journals/AIM set from this point forward will be referred to as Clinically Useful Journals (CUJ). The evidence-based process used to reframe evaluation of the clinical impact and utility of biomedical journals is documented in this article.


INTRODUCTION
For fifty years, from 1970-2020, clinicians and librarians used either the Abridged Index Medicus (AIM) or Core Clinical Journals (CCJ) list for two main purposes: narrowing a literature search to clinically important journals and identifying high-priority titles for medical library collections. Originally developed as a manageable subset of 100 important titles from the 2,300 Englishlanguage journals then indexed in Index Medicus, now known as MEDLINE (both produced by the U.S. National Library of Medicine (NLM), the Abridged Index Medicus (AIM) list aided clinicians seeking to limit their searches to clinically oriented articles. Investigation revealed that there was no record of the methods used to produce the 1970 AIM list other than published reports of the professions involved (librarians, physicians, editors) and that it was designed for practicing physicians [1]. In 1979, 26 journals were added (along with one in 1978), and eight titles were deleted, resulting in 119 indexed journals [2]. NLM automated and renamed it the Core Clinical Journals filter to augment PubMed.gov in 2001 [3]. Subsequently, one title was removed to leave 118 journals on the CCJ list [4].
The impetus for the current update was the 2014 research and subsequent article demonstrating the CCJ filter's low usage, recall and precision for clinical searching [5], specifically the findings that only 30% of clinically used articles were from the CCJ filter and only 16% of the journals were represented in Core titles. In 2015, the Medical Library Association (MLA) convened an Ad Hoc practitioners and hospital librarians who require access to essential clinical literature (Appendix A).
The Ad Hoc committee was cochaired by Michele Klein-Fedyshin, MSLS, BSN, RN, and Andrea M. Ketchum, MS, MLIS, and included members from the Hospital Library and Nursing and Allied Health Sections of MLA, as well as other specialties, such as Medical Informatics. Only five members were journal "Selectors," although the committee included liaisons from the MLA Board and NLM. The committee recognized that health care professionals need to conduct efficient, yet effective searches. By evaluating a broader variety of health care professions, hospital/outpatient/office environments, and patient ages and conditions, the new list would be valuable to all clinical practitioners.

METHODOLOGY
Since the subject coverage of the 50-year-old list had never been evaluated, the committee formulated a data-driven, step-wise approach to the update, selecting subjects first and journals second. This approach generated two questions: 1.
Do NLM subject headings represented by the existing CCJ list align with current clinical practice? 2.
What journals currently indexed in MEDLINE best meet newly aligned CCJ subject headings-defined usage and practice criteria?

Initial Data Gathering Steps of Subject and Journal Selection Process
Lacking any previous methodology for the process, the Committee considered data sources, scope, and usage statistics for both subject and journal selection criteria. A totally new, data-driven approach incorporating clinical Journal Usage (JU) and Patient-Driven Count (PDC) indicators was developed to demonstrate actual clinical activity and journal use.  [8], Healthy People 2020 Objectives [9] for national contemporary health concerns, and topic frequency data for requested alerts in Medscape [10] (Appendix C).
Thus, real-world evidence incorporating practical journal usage by a wide variety of institutions and professionals correlated with national U.S. clinical data and 2020 health goals ultimately produced two tools: JUs paired with PDCs to rank updated clinical subjects, and a "clinical utility score" to indicate a journal's current clinical usefulness. Together, they enabled the subject and journal selection process.

SUBJECT REVIEW
The Committee Evaluated CCJ Subject Coverage First MEDLINE journals are indexed with a simplified list of 125 NLM Broad Subject Headings [11], which serve to aggregate several separate MeSH headings. One or more Broad Subject Headings are provided in the PubMed bibliographic record for every MEDLINE journal, and many are assigned multiple Broad Subject Headings. After the JU and PDC counts were gathered, the data were correlated by the statistician for all 125 Broad Subject Headings assigned across CCJ journals. The correlation of JU and PDC data divided those Subject Headings into nine paired high, middle and low count groups.

Assigning Selectors and Creating Uniform Criteria
To facilitate the selection of journal titles for each subject, the committee co-chairs organized the subjects into logical groupings so that the same person covered similar subjects (e.g., psychiatry and psychology).

Candidate Journal Selection
Two versions of a Candidate Journal Worksheet applicable across all subjects were created: a working version to use for collecting comparative data for each subject, and a final version with just the selected journal titles. Candidate journal data were recorded on the subject worksheet, and journals in each subject were ranked. Figure 2 depicts a Candidate Journal Worksheet.

Table 3 Candidate Journal Worksheet
The following "ground rules" for candidate journals were set: 1.
The journal title must be currently indexed in MEDLINE; 2.
Numeric ranking among our clinically used titles derived from JU counts was critical; 3.
Must be indexed to the Broad Subject Heading under consideration;

4.
The maximum number of journals allocated for that subject could not be exceeded; fewer recommended journals were allowable if usage data did not support reaching that number.
A detailed description of the calculations used to determine subject coverage and select journals for the resulting subjects is in Appendix D. A decision toolkit consisting of Candidate Journal Worksheets, Journal Usage counts, and PALS ranking helped all selectors use common factors when picking journal titles for each assigned subject. Of note, only Ad Hoc CCJ committee members who were not NLM or MLA staff could select journals.

Final Clinically Useful Journals List and Recommendations
The new Clinically Useful Journals list adds journals for the 33 new subjects. Coverage expanded in some of the 47 existing subjects. Figure 1 graphically depicts these changes.

Figure 1 Subjects, Journals Analyses
The new analysis and selection process resulted in a 241journal product named Clinically Useful Journals (CUJ). The full alphabetical list of journals comprising the new CUJ is displayed below in Table 4.  [14] and MEDLINE's growth from 2,300 to 5,288 indexed journals, the CUJ proposed list of 241 journals represents a core collection that is slightly less than MEDLINE's growth. The 241 selections are less than the 341 covered in five primary care review services, such as ACP Journal Club [15] or DynaMed [15,16]; and it is less than the 250 medical journals included in the NEJM Journal Watch series [17].

Highest Clinical Impact Journals
Although the use of a journal-limited filter could eliminate relevant articles, CUJ's data-driven journal selections can also filter to the most highly clinically useful journals. The resulting Clinically Useful Journals (CUJ) list reflects real-world evidence encompassing national discharge data, U.S. health goals for 2020, and actual journal usage by a wide variety of institutions and professionals across the country. Among the total usage of the over 1,600 journals analyzed, journals assigned to the new CUJ list accounted for about 85% of the usage. In addition to a oneclick clinical limit, institutions may use the data-driven list to create customized searches in PubMed.gov for their institutional providers. Implementation will enable future evaluation of the scope and utility of the new list.

Sustainability
The committee does not anticipate that the subjects covered by CUJ require frequent, regular review. At least every 15 years should be an adequate subject reevaluation schedule, although journals may warrant closer scrutiny.
The Evidence-based Usage Model process could be more automated. Some journal vendors are able to provide usage statistics to hospital libraries. As more hospital libraries automate their journal lists and receive electronic usage reports, data could be solicited from hospital libraries and furnished to future researchers to update the CUJ. This would automatically supply the JU counts needed, and Patient Driven Counts, some publicly available via AHRQ, won't be needed until subjects are reviewed again.
The CUJ Update Flow Chart ( Figure 2) below illustrates an automation process for the CUJ.

Figure 2 CUJ Automated Update Process
With a more automated method to collect clinical journal usage and assign subjects to it, the process could be streamlined. It may be possible to create a ranked list with minimal manual handling.

IMPLEMENTATION
To complete its charge, the committee made the following recommendations to MLA: To enable its immediate application, the CUJ list has been translated into a PubMed search strategy and is available in Appendix H.
The potential to sort search results by "Best Match" in the current PubMed as one of several sorting options, may be enhanced by the CUJ subset. "Best Match" sorted results limited to the CUJ subset can limit results to the journals most likely to be clinically impactful. While the rigorously designed CUJ filter adds another step, it can strengthen the clinical relevance of a standard PubMed search by using a filter defined by clinical journal usage data. With over 80% of searchers' clicks occurring among the top 20 citations or first page of results returned [24], clinician search efficiency and satisfaction may be increased by applying the CUJ filter to "Best Match" results.

LIMITATIONS
The Proportion method may be imperfect, but it was the most data-driven method to approximate the original ratio of CCJ journals to the total number of journals within MEDLINE. Although it included growth in non-clinical journals, over the decades some preclinical sciences became relevant to clinical queries. Medical Genetics is one example. Note that some journals retained NLM broad subjects beyond the scope of CUJ's new set of 80 subjects reflecting the relevance of an expanded focus.
The data collected on clinical use of journals were from North American library members of the Medical Library Association. Thus, the high use journals included English language titles, with many international journals represented. World-wide application of the journals selected might be limited in non-English speaking countries. Military, veterans, and indigenous populations may be underrepresented. Preclinical and animal subjects were omitted.

CONCLUSIONS
Search efficiency is very important to librarians and clinicians alike. It is vital to have one universally available filter, whether logged in to a personal account or not, to automatically reduce retrieval to a focused, manageable set. By determining the most important subjects for clinical application using JUs and PDCs, and then selecting journals based on actual clinical usage, the new list is strongly oriented to the most clinically useful journals. Thus, the new CUJ list is optimized for clinical inquiries and offers evidence-selected journals to practitioners. Research shows that freely available PubMed/MEDLINE is frequently searched to answer clinical questions, second only to directly searching within journal issues [20,25]. Having a CUJ filter to limit to those journals most frequently accessed for clinical queries would add a powerful tool.
Hospital librarians and physicians have applied essentially the same CCJ filter for 50 years. Research revealed gaps in journal coverage that are addressed by this newly updated, evidence-based CUJ filter. An updated list of journals reflecting actual clinical usage should result in search retrievals more applicable to realworld clinical questions and a broader range of healthcare practitioners. Implementing the new CUJ could benefit the entire healthcare community by encompassing high use journals for medical and mental conditions.