Figure 1 Recognition of pattern change in blue and white references, indicating a duplicate that should not be removed; the references in the black boxes are updates and should all be removed
Wichor Bramer, Paul Bain
doi: http://dx.doi.org/10.5195/jmla.2017.183
Received 01 January 2017: Accepted 01 February 2017
Performing, writing, and publishing a systematic review take a long time. In a cohort of journal-published systematic reviews, Cochrane reviews, and health technology assessment reports, the median time lag between the stated last search date and publication was 61 weeks (interquartile range, 33–87 weeks) [1]. In the same cohort of reviews, 7% were out of date at the time of publication [2]. More recently, an examination of 182 systematic reviews performed at Erasmus Medical Centre showed that the median time between the first search and the appearance of the resulting review in PubMed was 89 weeks (interquartile range, 63–126 weeks).
To maximize the currency of a review, an update of the search is recommended before submission for publication. The Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards requires: “Rerun or update searches for all relevant databases within 12 months before publication” [3]. Many handbooks and guidelines for performing systematic reviews state that search strategies should be updated regularly to keep track of newly added references on the topic [4–6].
Recent guidance from an international panel of authors, editors, clinicians, statisticians, information specialists, other methodologists, and guideline developers considered various aspects of updating reviews, including efficient searching. Such efficiencies included refinements based on the yield of the original search and incorporation of technological advances in searching. Garner et al. provided practical guidance on refining the original search in their appendix 2 [7].
The Cochrane handbook mentions in chapter 3.4.2.1 (“Re-executing the search”) using the last date of the original search as the beginning date for the update, which is common practice, but chapter 6.4.12 (“Updating searches”) does not describe a clear method [8]. The date that the record became accessible through searching, rather than the publication date, is the relevant field for updating. The user can choose from the thesaurus date (i.e., the date that the thesaurus terms were added), the date of the last metadata change, or the date of entry into the database. For example, the National Library of Medicine recommends using the Create date (CRDT in PubMed) field for its databases [7]. The MeSH date field (MHDA in PubMed), which is the date the Medical Subject Headings (MeSH) terms were added to the record, also has some advantages. However, some interfaces, such as Web of Science, do not provide record dates that could guide updating. In such cases, searchers can use publication date and a safe overlapping period, resulting in extensive duplication with records retrieved in the original search.
Complicating matters further, the search may have been modified since the last search date. For instance, new words may have been added to the original search strategy, based on relevant terms found in studies included in the original review. These novel terms need to be searched in all the databases that were queried in the original search from the original starting date, thus requiring even more complicated search structures and date ranges. Hence, to many authors, updating a search can seem to be a complicated and uncertain task. It need not be.
The authors have developed a method for updating existing reviews that uses EndNote reference management software [9]. The technique uses two EndNote files: one containing the current results as they are downloaded from the complete set of databases, as if it were a first search; and one with the results of the previous or original search. By subtracting records found in the original search from the current results through EndNote’s customizable de-duplication feature, only records that were not screened in the original search will remain in the library. Another group has previously alluded to a similar process: “download all references from the update search and directly de-duplicate them with the references from the initial search (e.g., using Endnote)” [10].
Here, we describe the process in step-by-step detail. The steps will be identical in any recent version of EndNote. The method was first developed with the Microsoft Windows version of EndNote X3 and has been fairly consistent until the current version, X8. The same steps are also applicable for Macintosh editions by replacing the standard Windows controls with the corresponding Macintosh controls: for instance, command-A instead of control-A or command-click instead of control-click.
The initial search requires no extra action. We do recommend thorough de-duplication in EndNote by the process described in an earlier article by Bramer et al. [11]. This earlier article also describes how substantial differences in the way single articles are represented in various databases can be resolved. It describes how page numbers from MEDLINE and the Cochrane Library can be completed, turning abbreviated pagination (e.g., 1008–12) into full format pagination (e.g., 1008–1012) as is used in other databases. The earlier article also recommends importing full journal titles instead of abbreviations. If these steps are followed, the method described in this article will be easier to follow. The following description will still be effective if the libraries were de-duplicated using other sequences in EndNote but will require additional care to ensure that novel articles are not eliminated.
The first steps in the updating process serve to create an EndNote file containing all results from the new or current search as they are retrieved on the new date. This search’s date should now be considered the last date searched, and the number of records retrieved from this search will be used in the published PRISMA flow chart. In the process, a compressed library (.enlx file) can serve as an archive and be used in subsequent updates to identify and remove previously screened records.
Records from the original search have already been screened and need not be seen again. To remove these records from the current search results, the old records are added to the EndNote Library containing the records from the new search. By means of duplicate detection, matching pairs of records (one from the original search and the other from the new search) can be identified and removed, leaving only new records that have yet to be screened.
If the original search was executed in the recent past and used the same methods for import into EndNote, the Copied References group should be nearly empty after this step. If the original search was performed long before the update took place or the initial search had been performed by another searcher, possibly using different export methods and interfaces, there may be a large number of remaining unmatched references from the original search in the Copied References group. If so, follow two extra steps of the method described in Bramer et al. [11]. Step 3: 2–5 can be repeated with the Title, Volume, Pages fields and then with the Author, Volume, Pages fields; however, in these cases, references without page numbers should be assessed for duplicates, and true duplicates should be manually selected. To do this, after step 3: 4, click on the Pages column heading to sort by page number. References with page numbers can be deleted without manual assessment. If the Pages column heading is not visible in the reference table, go to Edit > Preferences > Display Fields, and select the Pages field in one of the columns.
In the next steps, the new results are compared with the original results on title. This requires quick manual assessment to prevent removing two new, independent search results that share the exact title.
|
|
||
|
Figure 1 Recognition of pattern change in blue and white references, indicating a duplicate that should not be removed; the references in the black boxes are updates and should all be removed |
||
A consequence of this method is that the reference numbers and totals required for the PRISMA flow chart might not match, as the sum of the novel and old records might not equal the contents of the new EndNote library. This discrepancy can be attributed to several normal database activities: updates to individual records (such as added volume and page numbers or a changed publication year after an article appeared in print), global changes to controlled vocabulary, recent addition of older material to the database, or the dropping out of high relevance ranking in Google Scholar. Additionally, records with publication dates that antedate the previous search will appear among the novel records. This can be the result of both changes to controlled vocabulary. Revising and adding to the search strategy will also result, appropriately, in retrieving novel records from before the date of the previous search as well. Hence, the most reliable method of correcting the numbers for publication in the PRISMA flowchart is backward correction.
Typically, the total number of unique records retrieved is reported both in the results section of a systematic review and in the PRISMA flow chart detailing the search and subsequent screening. For updated searches, the most appropriate number to report is the total number of records remaining in the updated search after removing duplicates. This number represents the minimum number of records screened for all searches. For dates of coverage, authors need only report the beginning date of coverage for each database and the date of the latest update, as the numbers of records are accurate for all searches at this point in time. The total number of full-text articles read will be the sum of the articles read in all the previous searches plus the number of articles read in current search. The number of articles excluded based on title abstract, for which no specific reason has to be given, is the total number of records remaining in the updated search minus the total number of full-text articles read. The numbers of records or articles excluded for the specific reasons in the full-text review stage can be summed over the previous and updated searches.
The records described in step 4: 9 are records that had been found during the original search but that were not found in the update search. This situation can occur if a search strategy during the update is narrowed compared to the original search strategy or if certain databases that had been searched initially were not searched during the update. If the search strategy has been narrowed, it is necessary to assure that the included references from the original search results are still retrieved by the current searches.
We provide a method to simply subtract previously screened results from updated systematic review searches. By using this method, searchers spare themselves the time and effort required to reconcile update dates across platforms. Our method is described for use with EndNote bibliographic management software, although it might conceivably be adapted to other bibliographic managers. However, for optimal implementation of this method, such a program requires customizable settings for duplicate detection and the option to remove both duplicate references. We are unaware of a bibliographic manager that is as flexible as EndNote in this regard. Most software uses a standard de-duplication algorithm and is set to merge duplicate references instead of removing them.
There is no consistent, widely accepted method of updating searches. Our method suggests a simple standard for both carrying out and reporting updates that requires giving only the total number of records from the inception of the original search to the date of the last update, along with any revisions to the search itself. In reporting only these details, authors give an accurate representation of the state of the database and the response to the query on the date of the last search. The only information that is lost is the number of records retrieved and screened in previous searches that did not match records in the updated query. Rerunning and de-duplicating the updated search obviates the confusing task of choosing and reporting a date field for the beginning date of the new search (e.g., publication date, thesaurus date, creation date).
The authors thank Margaret Sampson, AHIP, from the Children’s Hospital of Eastern Ontario, Ottawa, Canada, for her input on earlier drafts of this article.
1 Sampson M, Shojania KG, McGowan J, Daniel R, Rader T, Iansavichene AE, Ji J, Ansari MT, Moher D. Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol. 2008 Aug;61(8):755–62.
2 Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? a survival analysis. Ann Intern Med. 2007 Aug 21;147(4):224–33.
3 Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D. Methodological standards for the conduct of new Cochrane intervention reviews [Internet]. The Cochrane Library; 2013 [updated 2 Dec 2013; cited 20 Jan 2017]. <http://editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/public/uploads/MECIR_conduct_standards%202.3%2002122013_0.pdf>.
4 Moher D, Tsertsvadze A, Tricco A, Eccles M, Grimshaw J, Sampson M, Barrowman N. When and how to update systematic reviews. Cochrane Database Syst Rev. 2008 Jan 23;(1):MR000023.
5 Shojania KG, Sampson M, Ansari MT, Ji J, Garritty C, Rader T, Moher D. Updating systematic reviews. Rockville, MD: Agency for Healthcare Research and Quality (US); 2007 Sep.
6 Stovold E, Beecher D, Foxlee R, Noel-Storr A. Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram. Syst Rev. 2014 May 29;3:54.
7 Garner P, Hopewell S, Chandler J, MacLehose H, Akl EA, Beyene J, Chang S, Churchill R, Dearness K, Guyatt G, Lefebvre C, Liles B, Marshall R, Martínez García L, Mavergames C, Nasser M, Qaseem A, Sampson M, Soares-Weiser K, Takwoingi Y, Thabane L, Trivella M, Tugwell P, Welsh E, Wilson EC, Schünemann HJ; Panel for updating guidance for systematic reviews (PUGs). When and how to update systematic reviews: consensus and checklist. BMJ (Clinical Research Ed). 2016 Jul 20;354:i3507. (Erratum. BMJ. 2016 Sep 6;354:i4853.)
8 Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions: Wiley Online Library; 2008.
9 Thomson Reuters. EndNote X7. 2015.
10 European Network for Health Technology Assessment. Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness [Internet]. The Network [cited 23 May 2017]. <http://www.eunethta.eu/sites/5026.fedimbo.belgium.be/files/2015-07-13_Guideline_Information_Retrieval_final.pdf>.
11 Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 2016 Jul;104(3):240–3. DOI: http://dx.doi.org/10.3163/1536-5050.104.3.014.
12 Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. J Med Libr Assoc. 2017 Jan;105(1):84–7. DOI: http://dx.doi.org/10.5195/jmla.2017.111.
Wichor Bramer, w.bramer@erasmusmc.nl, Biomedical Information Specialist, Medical Library, Erasmus MC–Erasmus University Medical Centre, Rotterdam, The Netherlands
Paul Bain, pbain@hms.harvard.edu, Countway Library of Medicine, Harvard University Medical School, Boston, MA
Articles in this journal are licensed under a Creative Commons Attribution 4.0 International License.
This journal is published by the University Library System of the University of Pittsburgh as part of its D-Scribe Digital Publishing Program and is cosponsored by the University of Pittsburgh Press.
Journal of the Medical Library Association, VOLUME 105, NUMBER 3, July 2017