Tertiary drug information sources for treatment and prevention of COVID-19

Objective: To evaluate tertiary drug information databases in terms of scope, consistency of content, and completeness of COVID-19 drug information. Methods: Five electronic drug information databases: Clinical Pharmacology, Lexi-Drugs, AHFS DI (American Hospital Formulary Service Drug Information), eFacts and Comparisons, and Micromedex In-Depth Answers, were evaluated in this cross-sectional evaluation study, with data gathered from October 2021 through February 2022. Two study investigators independently extracted data (parallel extraction) from each resource. Descriptive statistics were primarily used to evaluate scope (i.e., whether the resource addresses use of the medication for treatment or prevention of COVID-19) and completeness of content (i.e., whether full information is provided related to the use of the medication for treatment or prevention of COVID-19) based on a 10-point scale. To analyze consistency among resources for scope, the Fleiss multi-rater kappa was used. To analyze consistency among resources for type of recommendation (i.e., in favor, insufficient evidence, against), a two-way mixed effects intraclass coefficient was calculated. Results: A total of 46 drug monographs, including 3 vaccination monographs, were evaluated. Use of the agents for treatment of COVID-19 was most frequently addressed in Lexi-Drugs (73.9%), followed by eFacts and Comparisons (71.7%), and Micromedex (54.3%). The highest overall median completeness score was held by AHFS DI followed by Micromedex, and Clinical Pharmacology. There was moderate consistency in terms of scope (kappa 0.490, 95% CI 0.399-0.581, p<0.001) and recommendations (intraclass correlation coefficient 0.518, 95% CI 0.385-0.651, p<0.001). Conclusion: Scope and completeness results varied by resource, with moderate consistency of content among resources.


INTRODUCTION
The coronavirus disease 2019 (COVID-19) pandemic presented many challenges to healthcare providers.Knowledge of the biology and pathophysiology of SARS-CoV-2 remains in its infancy, leading to frequent changes to recommended treatment options.With over 1 million US deaths since the onset of the pandemic, with a casefatality ratio of 1.1%, the public health impact of COVID-19 remains high [1].Since the beginning of the pandemic, the pressure to release information quickly and the global nature of the pandemic has challenged researchers' ability to conduct and complete rigorous, generalizable clinical trials [2,3].Thus, clinicians have often had to rely on the off-label use of drugs or await the release of study drugs for clinical use through the FDA Expanded Access Programs or Emergency Use Authorization.As of October 2022, 13 treatments for COVID-19 were being used off-label for emergency use while only 2 were FDAapproved [4].However, repurposing drugs also demands a thorough process for determining the dose, treatment duration, and dosing regimen [5].These parameters are conventionally studied in phase II clinical trials [6].The absence of such trials for COVID-19 has made it difficult to establish cause and effect relationships and has further complicated the COVID-19 drug development process.
Especially during the early months of the COVID-19 pandemic, there was a lack of scientific evidence.This lack negatively impacted the public health response [7].Tertiary, point-of-care, drug information databases are considered reliable sources of scientific evidence.We sought to evaluate these databases in terms of scope, consistency of content, and completeness of COVID-19 drug information.Specifically, we assessed the five major compendia, which are considered the most important online drug information resources: Clinical Pharmacology, Lexi-Drugs, AHFS DI (American Hospital Formulary Service Drug Information), eFacts and Comparisons, and Micromedex In-Depth Answers [8,9,10,11,12].

METHODS
The five electronic drug information databases were evaluated in this cross-sectional evaluation study, with data gathered from October 2021 through February 2022.Study methods, including endpoints and statistics, were developed based on a similar previous study evaluating tertiary drug information database content for off-label uses [13].The resources were selected based on the American Association of Colleges of Pharmacy (AACP) Basic Resources for Pharmacy Education list and have been included in previous studies [14,15,16,17].AHFS-DI was accessed through the Lexicomp database.A sample of medications was developed by extracting medications acknowledged by three major clinical practice guidelines: the National Institute for Health and Care Excellence (NICE), National Institutes of Health (NIH), and the Surviving Sepsis Campaign [18,19,20].The medications did not need to be recommended in the guidelines to be included in the sample, in order to account for medications that had not been positively recommended in guidelines.To ensure that the list of drugs was as inclusive as possible, the sample was then reviewed by a subject matter expert in infectious diseases who recommended several additional medications.
Two study investigators independently extracted data from each of the five resources using a standard data collection form prepared in Microsoft® Excel®.A third investigator then verified and reviewed any divergence in the data through discussion with the original investigators.The five resources were searched using either the generic name of each medication from the preidentified sample of agents, or the brand name if more than one brand was available for the same generic medication.Data were collected by searching through each drug monograph for information specifically related to COVID-19.In each resource, any information referencing indication (or lack thereof), dosage, route, and use in special populations (i.e., pediatrics, pregnancy, and geriatrics) for COVID-19 were extracted for review.
Descriptive statistics were primarily used to evaluate scope (i.e., whether the resource addresses use of the medication for treatment or prevention of COVID-19) and completeness (i.e., whether full information is provided related to use of the medication for treatment or prevention of COVID-19).For scope, the percentage of medications covered by the resource out of the total sample was calculated.For completeness, 1-point was awarded for presence of each of the ten listed completeness factors (itemized in Table 3) for each drug for a total possible score of 10 points.Aggregate results were described using median and interquartile range (IQR).
To group references into tiers according to scope and completeness, the highest scoring reference for each was compared to the next highest scoring reference, sequentially, until the difference was statistically significant using an alpha of 0.05.A sequential, tiers-based approach was selected to identify resources that had similar performance in each endpoint and to distinguish them from those resources that had higher or lower performance at each endpoint.Since the same sample of medications was evaluated in each resource, the McNemar test was used to compare scope (matched pair related categorical data) and the Wilcoxon signed-rank test was used to compare completeness (matched pair related ordinal data) [21].To analyze consistency among resources for scope, the Fleiss multi-rater kappa was used.To analyze consistency among resources for type of recommendation (i.e., in favor, insufficient evidence, against), an intraclass coefficient was calculated using two-way mixed effects and measuring absolute agreement.Statistics were computed using IBM SPSS Statistics version 28.

DISCUSSION
Overall, our study provided insight regarding the common drug information tertiary resources' scope, completeness, and consistency of content for COVID-19 drug information.Lexi-Drugs and eFacts and Comparisons were mostly likely to acknowledge or address the use of a medication for treatment or prevention of COVID-19, whereas AHFS DI and Micromedex provided more complete information.Consistency was reported as moderate agreement among all 5 databases.Specifically, Lexi-Drugs and eFacts and Comparisons together had greater coverage of the 46 COVID-19 medications (consistency of scope) and Clinical Pharmacology, Micromedex, and AHFS DI were more likely to recommend in favor of use (consistency of content).
Factors that contributed to low completeness scores included a lack of reported measures of statistical significance and effect size, and a lack of guidance for use in special populations, specifically geriatrics and pregnancy.More quantitative information from primary literature would help clinicians and health care providers make more nuanced treatment decisions that consider the clinical significance and risk-benefit ratio of the medication, not just whether the study was successful.These points may be helpful for information professionals to share with trainees, to help them identify when direct use of primary literature may be necessary to answer a clinical question.With many of these medications being new to market, it is expected that information guiding use in older adults and pregnant patients will emerge with time.In the meantime, with many COVID-19 clinical trials having substantial geriatric representation, the information provided for non-geriatric adults may be able to be extrapolated to geriatric patients.
Similar to the results of our study, Lexi-Drugs (Lexicomp) has been one of the top drug information tertiary resources in scope and consistency in a variety of analyses.In a study from 2007, Lexicomp was able to answer 82.9% of 158 drug information questions [15].It is important to note that an update to this study would be an area for future research.Specifically, when assessing drug interactions in drugs of abuse, Lexicomp had the highest percent of interaction pair entries (43.4%) compared to the other databases [16].In addition, the severity of interaction consistency was high in Lexicomp (83.1%).Similar scope score and consistency results were reported for drug-ethanol interactions: 84.9% and 75.6% respectively [17].Lexicomp was also found to have the highest sensitivity and negative predictive value in regard to detecting oral oncolytic drug interactions [22].In studies specifically assessing pharmacogenomic drug information, Lexicomp was found to include the most information for drugs assessed compared to the other compendia [23,24].The results of our study and previous studies, which have consistently reported Lexicomp as one of the top resources, can assist heath sciences librarians in determining whether the cost of Lexicomp is justifiable.When conducting a search for new COVID-19 treatment options, Lexicomp was found to be the best resource to consult first due to the high scope scores (73.9%).
In contrast, eFacts and Comparisons has not typically held high scope scores in previous studies (15.1-69.8%)[16,17].However, the findings of our study are not surprising since both databases are now owned by Wolters Kluwer and are increasingly becoming more visually and contextually similar.This finding also highlights that the quality and completeness of resources may vary over time and by subject matter.
When comparing our completeness results to other studies, Micromedex has also been consistently high.In the study from 2007, Micromedex had a completeness score of 97.0% [15].In the studies assessing drug interactions in drugs of abuse and drug-ethanol interactions, Micromedex had completeness scores of 5 out of 5 in both cases [16,17].Most notably, a similar recent study found that Micromedex had among the highest scores for scope and completeness looking at information for off-label uses and infectious disease information [13,25].Conversely, compared to Clinical Pharmacology and Lexicomp, Micromedex reported fewer drug-drug interactions for assessed antiviral medications in a recent study [26].These results, in combination with assessment of user needs and other comparable resources in the collection, will assist health sciences librarians in assessing the appropriateness of continued support of Micromedex despite price increases.Micromedex's completeness score makes this resource a critical second option to search when conducting COVID-19 research.When end users are seeking a resource that provides comprehensive information on off-label uses, this study and others suggest Micromedex may be a good option for information professionals to recommend.
To our knowledge, this is the first study of its kind to assess COVID-19 drug information scope, completeness, and consistency of content in commonly used databases.These results are important to inform practitioners of the best tertiary resources to consult for COVID-19 treatment information in this ever-evolving landscape and may also provide guidance in future pandemics or situations of rapidly evolving medication information.Additionally, health science librarians and drug information pharmacists would benefit from this contribution to the literature as it illustrates resources that provide wellreferenced information during a health crisis and are most reliable when conducting research.When educating medical personnel in the use of resources, librarians can utilize the results of our study to illustrate the importance of checking more than one resource, given lack of consistency among resources.In addition, librarians can highlight Micromedex and AHFS as the best options for completeness of information and Lexicomp as the best resource in terms of scope for information related to COVID-19.When conducting a literature search for an end user, all resources provide helpful context, but Micromedex and AHFS appear to most consistently cite and discuss primary literature that can be used as a starting point.Some strengths of our study include the thoughtful and rational approach for determining our treatment sample.Clinical practice guidelines were consulted to determine which COVID-19 medications should be searched in the databases.As a result, our sample was comprehensive at the time of generation and even included dietary supplements.All five major compendia databases were included in our study.Data collection was conducted by two independent investigators and discrepancies were assessed by a third investigator.
Our study had a few notable limitations.First, during the COVID-19 pandemic, information about these medications was constantly being updated.As a result, the information obtained from the resources may have changed since our analysis.AHFS-DI is updated monthly whereas Micromedex, Lexi-Drugs, eFacts and Comparisons, and Clinical Pharmacology are updated daily [27,28,29,30].This could have affected the performance of AHFS-DI in our study.The approach to assessing completeness of the information was used from previous studies but was not validated, though there is not currently a fitting, validated alternative.If a validated alternative becomes available, these resources should be reassessed using a validated measure.Additionally, differences in language were discovered in the databases.For example, some resources (e.g., Micromedex) tended to provide their own recommendations, based on the literature, whereas others simply summarized what was being stated in the literature without critical appraisal or assessment (e.g., Clinical Pharmacology, AHFS DI).Consistency of scope and recommendation results suggest that some databases (e.g., Clinical Pharmacology) may be more likely to acknowledge medications being used for COVID-19 when that use is recommended, versus addressing other medications that may be used despite low levels of evidence.
Overall, our study will assist healthcare providers in determining the best resources for COVID-19 treatment information and provide health science librarians with justification for resource funding.While no resource is perfect, Lexi-Drugs (scope) and AHFS (completeness) were found to provide the most information regarding medications used to treat COVID-19.Our study highlights the importance of always checking more than one resource when responding to a drug information question, especially during the COVID-19 pandemic.If only one resource is used, important information could be missed, and patient care could be affected.

Table 1
Sample of medications acknowledged by COVID-19 clinical practice guidelines

Table 2
Scope results by resource