Original Investigation

Patient-based benefit-risk assessment of medicines: development, refinement, and validation of a content search strategy to retrieve relevant studies

Hiba El Masri¹, Treasure M. McGuire², Christine Dalais³, Mieke van Driel⁴, Helen Benham⁵, Samantha A. Hollingworth⁶

doi: http://dx.doi.org/10.5195/jmla.2022.1306

Volume 110, Number 2: 185-204
Received 05 2021: Accepted 11 2021

ABSTRACT

Introduction:

Poor indexing and inconsistent use of terms and keywords may prevent efficient retrieval of studies on the patient-based benefit-risk assessment (BRA) of medicines. We aimed to develop and validate an objectively derived content search strategy containing generic search terms that can be adapted for any search for evidence on patient-based BRA of medicines for any therapeutic area.

Methods:

We used a robust multistep process to develop and validate the content search strategy: (1) we developed a bank of search terms derived from screening studies on patient-based BRA of medicines in various therapeutic areas, (2) we refined the proposed content search strategy through an iterative process of testing sensitivity and precision of search terms, and (3) we validated the final search strategy in PubMed by firstly using multiple sclerosis as a case condition and secondly computing its relative performance versus a published systematic review on patient-based BRA of medicines in rheumatoid arthritis.

Results:

We conceptualized a final search strategy to retrieve studies on patient-based BRA containing generic search terms grouped into two domains, namely the patient and the BRA of medicines (sensitivity 84%, specificity 99.4%, precision 20.7%). The relative performance of the content search strategy was 85.7% compared with a search from a published systematic review of patient preferences in the treatment of rheumatoid arthritis. We also developed a more extended filter, with a relative performance of 93.3% when compared with a search from a published systematic review of patient preferences in lung cancer.

Keywords: patient-based benefit-risk assessment; benefit-risk assessment; attribute development; patient preference; prescription drug; risk assessment/methods; databases; bibliographic; information storage and retrieval/methods; information storage and retrieval/standards; Medical Subject Headings; terminology as topic; reproducibility of results.

INTRODUCTION

Medicines are used for a known therapeutic benefit including cure, delaying disease progression, relieving symptoms, or preventing comorbidities. Every medicine also carries a risk of side effects, from minor to severe. A thorough understanding of both benefits and risks of every pharmacotherapeutic intervention should underpin the effective medicine management cycle of prescribing, dispensing, administering, and monitoring of effect [1]. Balancing benefits and risks is also a key step in the decision-making process of regulatory authorities and in developing guidelines by professional societies [2].

The methodical and regular review of the efficacy and safety parameters of a medicine is called a benefit-risk assessment or analysis (BRA) or benefit-risk ratio evaluation. BRA is primarily an exercise that balances two dimensions: the dimension of benefit that includes not only therapeutic efficacy but also improvement of quality of life, and the dimension of risks that consists of the safety profile of the given medicine and the potential risk of unintended adverse events anticipated on the basis of the mechanism of action [3]. The dimension of cost could also be embedded in this analysis [4].

Despite the use of quantitative and structured approaches to evaluate the benefits and risks of medicines, the BRA remains heavily influenced by the value judgments of clinical experts [2]. This professional BRA judgment does not necessarily correlate with patients' evaluations. In fact, there is growing evidence that the trade-offs between benefits and risks made by patients differ significantly from clinical experts [5]. Furthermore, patients with chronic conditions cautiously and deliberately reassess the benefits and risks of their treatments at multiple milestones in their disease journey and may overemphasize the potential risks of their current medicines or overestimate the benefits of new treatments [6]. The concept of a patient-based BRA of medicines has recently emerged and is attracting the attention of regulatory authorities who acknowledge the importance of incorporating patients' perspectives into their decision-making processes [7, 8].

There has been a steady increase in studies eliciting patients' preferences and perspectives on their medicines [9, 10]. The growing evidence about patients' perspectives in general—and patient-based BRA in particular—is likely to inform clinical and regulatory decision-making [11]. There are barriers, however, to comprehensive and efficient searching for this evidence, including poor indexing and the inconsistent use of terms to denote either patient preferences or patient-based BRA [12]. Moreover, the capacity of most methods used to accurately elicit patients' perspectives depends on the researcher selecting a medicine's attributes and the patient's understanding the proposed choices [13]. A robust search strategy with predictable performance parameters is needed to rigorously generate and develop BRA attributes.

While some search strategies have been proposed to identify the literature on patients' knowledge, views, and preferences about their health and health care [14–16], we could not retrieve any search strategy to selectively identify how patients balance the benefits and risks of their medicines. The former search strategies produce a wide search yield, containing studies on patients' perspectives in various areas such as information and knowledge needs, communication and social support, appraisal of symptom severity, comorbidity management, hospitalization, and prevention and screening tests. We aimed to create and validate a targeted, objectively derived content search strategy for patient-focused BRA of medicines that would detect studies on patients' perspectives on medicines' attributes, assessments of adverse event severity, the importance of avoiding side effects, and other treatment characteristics such as routes of administration, frequency of treatment, inconvenience caused by scheduled treatment, and costs. The measures of this validated and objectively derived search strategy would be calculated. Search strategy performance would be predictable, unlike subjectively derived strategies that are mainly based on the authors' expertise, with their methodologies not consistently reproducible. The proposed content search strategy would contain generic search terms, rather than terms specific to a given therapeutic area, allowing it to be used as a search filter for patient-based BRA of medicines in any therapeutic area.

METHODS

We developed a content search strategy to retrieve studies on patient-based BRA of medicines using an iterative process: (1) developing a search term bank, (2) refining the search strategy, and (3) validating the final strategy (Figure 1). This content search strategy was constructed and validated in PubMed. PubMed is a free resource supporting the search and retrieval of biomedical and life sciences literature and is one of the most commonly used medical databases [15].

Figure 1

Flow diagram depicting the iterative process of developing a search term bank, refining the search strategy, and then validating the final strategy

Developing a bank of candidate search terms

We compiled a list of search terms (Medical Subject Heading [MeSH] terms, other indexed terms in title or abstract, and free text) from two sources: (1) existing search strategies used to retrieve studies on patients' knowledge and information needs, experiences, views, and preferences [14–16] and (2) the MeSH database in PubMed [17]. The MeSH thesaurus is a controlled and hierarchically organized vocabulary established by the National Library of Medicine for indexing and searching biomedical and health-related information [17].

We combined these terms into a subjectively derived search filter, or hedge, that was developed based on several sources: the authors' subject knowledge, expertise of our librarian searcher, existing search strategies, the MeSH database, and a thesaurus (Appendix 1) [18]. We ran an initial search (using OR) in PubMed in July 2020. Although the patient was at the core of our initial search strategy, we expected it would yield a large number of articles under the broad scope of patients' preferences in all aspects of health and illness, including preferences related to the attributes of their medicines. The initial search yielded 790,674 articles. For practical purposes, we manually screened the first 5,000 articles (by order of appearance) retrieved by this combined list of search terms without limiting our search by the nature of the medical or health condition, language, or date of publication. We also used Best Match rather than chronological order as the active filter in PubMed to enable relevant older citations to be retrieved. However, our a priori decision was to continue beyond this citation limit if insufficient relevant papers were retrieved or search term saturation was not achieved. We first screened at the level of title and abstract, to determine which articles discussed patient-based BRA of medicines.

We identified seventy relevant papers on patients' preferences for treatment attributes in different therapeutic areas, published in many journals (Appendix 2). The studies were deemed relevant if they elicited patient-based BRA of medicines using quantitative and or qualitative approaches. The identified studies covered many therapeutic categories and medical conditions, mostly chronic in nature (e.g., cardiology, endocrinology, neurology, nephrology, dermatology, immunology, rheumatology, gastroenterology, oncology, and other therapeutic areas). We extracted controlled vocabulary (i.e. author-designated keywords and MeSH terms plus subheadings) and free text words from abstracts and full texts of the seventy identified papers on patient-based BRA of medicines (Appendix 2). The resulting collection of terms, including all terms from the hedge, formed our bank of candidate search terms.

Refining the search strategy

In developing methodological search strategies, experts define the gold standard (also called reference standard or reference set) as a set of relevant records against which the search strategy is tested and validated to determine its performance parameters [18]. Through an iterative process and two rounds of discussions, two authors (HM and SH) screened and tested the bank of candidate search terms to optimize the effectiveness and efficiency of the search without making the search yield too narrow. The analytical approach to refine the terms to be included in the content search strategy was based on frequency of occurrence, often used by information specialists and experts to develop and validate objectively derived search strategies [19–21]. We set the minimum threshold of frequency of occurrence for an individual term to be included by multiple testing of various combinations to obtain optimal performance measures, in particular sensitivity, precision, and specificity. The sensitivity or recall rate is defined as the number of relevant records in the gold standard retrieved by the search strategy as a proportion of the total number of records in the gold standard. The precision or positive predictive value (PPV) is the number of relevant records retrieved by the search strategy as a proportion of the total number of records retrieved. The specificity is the number of records that are not relevant and are not retrieved by the search strategy as a proportion of the total number of nonrelevant records [18]. Equations used to calculate performance parameters are given in Table 1.

Search syntax applied	Relevant articles	Nonrelevant articles	Total
Content search strategy + (Search A) Search strategy combined (using AND) with the MS search string*	a True positives	b False positives	a + b
Content search strategy – (Search B) Use of the MS search string* alone	c False negatives	d True negatives	c + d

We performed our term frequency analysis in two rounds: in the first round we retained the terms with sensitivity higher than 1%, and in the second round we retained the terms with precision greater than 2%. We set the minimum thresholds of frequency of occurrence by multiple testing of various combinations to obtain optimal performance. We aimed for a minimum of 80% for sensitivity and 20% for precision. The balance between sensitivity and precision was determined with the thresholds 1% and 2%, with consensus among all authors. For an extended search strategy, we aimed for a minimum of 90% for sensitivity at the cost of any precision. In the extended search strategy, we included all the terms that surpassed the 1% sensitivity threshold (Appendix 3).

In this refining exercise, we chose type 2 diabetes mellitus (T2DM) as a case condition to test the performance of each search term. We chose T2DM for four reasons: (1) T2DM is highly prevalent and associated with significant excess risk in cardiovascular morbidity and mortality [22], (2) treatment can be with oral or parenteral glucose-lowering agents, with various device models available, (3) newer oral-glucose-lowering agents are suggested to have better cardiovascular effects than long-established glucose-lowering agents [23, 24], and (4) clinical recommendations stress the importance of a personalized and patient-centered treatment approach [25]. There are many studies exploring patient-based preferences of medicines for treating T2DM [26, 27], which makes it a suitable area to test the performance of our compiled search terms.

For terms relevant to T2DM and its management, we used this search string:

Diabetes Mellitus / therapy[MeSH] OR Diabetes Mellitus, Type 2 / drug therapy[MeSH] OR Diabetes Mellitus, Type 2 / therapy[MeSH] OR Hypoglycemic Agents / administration & dosage[MeSH] OR Hypoglycemic Agents / therapeutic use[MeSH] OR Hypoglycemic Agents / adverse effects[MeSH] OR Hypoglycemia / prevention & control[MeSH] OR Blood Glucose / drug effects[MeSH] OR Glucagon-Like Peptides / therapeutic use[MeSH] OR Insulin / therapeutic use[MeSH] OR Insulin / administration & dosage[MeSH].

To determine a gold standard reference set for performance measurements in this refining exercise, we combined—in PubMed—all candidate search terms (using OR) with the T2DM search string (given above) using AND. We screened on title and abstract level all retrieved records for relevant studies on patient-based BRA of medicines in T2DM; this constituted our gold standard reference set. We then tested in PubMed, one by one, each of the terms in our bank by combining them (using AND) with the T2DM search string.

We removed redundant terms that retrieved irrelevant studies. We added synonyms and alternative terms to the search terms. We applied truncation techniques to selected search terms—in other words, a deliberate shortening of a search term by adding a wildcard character (e.g., *) to retrieve other variants of the word due to differences in descriptions, language, or spelling [28, 29]. Terms with a sensitivity value higher than 1% were retained in the first round of refining. We further optimized the search by using Boolean operators (OR; AND) [30] to combine the search terms from the developed bank and reflect the two components of the patient-focused BRA concept (i.e., the patient and the balance between benefit and risk including all attributes of the medicines). In the second round we retained terms with satisfactory sensitivity (higher than 1%) and precision value greater than 2%. We reached the proposed final version of the search strategy by consensus among authors. We also put all the terms retained in the first round (sensitivity higher than 1%) into an extended content search strategy (Appendix 3).

Validating the search strategy

We validated the approach at two levels. First, we validated the search strategy in a case condition and computed its performance parameters. Second, we tested its relative performance by comparing it to the search approach in a published systematic review.

Validating the search strategy in a case condition

We validated the proposed final search strategy in PubMed by conducting a search of existing literature on patient-based BRA of disease-modifying drugs used to treat multiple sclerosis (MS) as a case condition. The choice of MS was based on (1) factors related to the disease, where patients experience significant pain and disability during their disease journey and may face multiple relapses and remissions, and (2) factors related to the newer, more-effective treatment options, which are also associated with increased risks of severe adverse events [31]. These two factors make MS patients' perceptions of benefits and risks of their medicines crucial in the shared decision-making process of managing their condition [32, 33]. The complex risk-benefit profiles of MS treatment options and the greater involvement of patients in their treatment decisions than in other diseases have made MS a rich area to explore patient-based BRA of medicines [34].

We used this search string:

Multiple sclerosis OR Multiple Sclerosis [MeSH Terms] OR Multiple Sclerosis / therapy [MeSH Terms] OR Multiple Sclerosis, Relapsing-Remitting / drug therapy [MeSH Terms].

We ran the search in PubMed in August 2020, with (using AND) and without the proposed search strategy and applied a ten-year filter for the publication date (from August 2010 to August 2020). Search A was based on the search string of MS combined with the developed search strategy (using AND) and search B was based on the search string of MS alone. We screened all records in search A on title and abstract level. We considered a study as relevant if the outcomes of interest included the preferences of MS (any subtype) patients toward their medicines, ideally specified according to clinical or convenience attributes describing various treatment scenarios. We did not exclude any study based on the methodology (quantitative or qualitative, empirical or review). We categorized articles as either relevant (true positives) or not relevant (false positives).

We then ran the search for articles on MS and its treatment, using the above search string alone, with the same ten-year filter for the publication dates (search B). We screened all records in search B on title and abstract level. The false negatives were considered the relevant articles that were only detected without adding the search strategy (detected only by applying the string to retrieve articles on MS and its treatment—search B). These were the relevant articles that the search strategy failed to detect. The remaining articles retrieved without the use of the search strategy were the true negatives. Because there is no established standard to ensure identifying all relevant articles on MS and its treatment [15]—thus identifying all false negatives—we compared our list of “relevant detected” and “relevant not detected” articles to those included in two published systematic reviews on patients' preferences for risks and benefits of disease-modifying drugs in MS [34, 35]. All articles listed in the two reviews and published on or after August 2010 (Appendix 4) were already included in our list of “relevant detected” and “relevant not detected” articles. Assuming that the pool of identified papers is as near as possible to the total number of relevant papers, we computed the sensitivity, specificity, precision, and number needed to read (NNR) of the content search strategy (Table 1) [36, 37].

Testing the relative performance of the search strategy by replicating a published systematic review

Systematic reviews aim to comprehensively locate and appraise research on a particular question, using structured and replicable procedures at each step of the process [38]. The approach and corresponding search strategies adopted in published systematic reviews to identify literature on patient-based BRA of medicines are considered the gold standard in this field [39]. As there are no established criteria to assess risk of bias or the methodological quality of patient preference studies [40], we adopted the checklist constructed by Eiring and colleagues [41] to choose quality systematic reviews for chronic diseases of interest. The checklist consisted of thirty-one quality criteria within five main domains: (1) external validity of the study, (2) quality of construct representation, (3) minimization of the risk of construct-irrelevant variance, (4) quality of reporting and analysis, and (5) other aspects that may strengthen or weaken the study.

We compared the performance of our developed content search strategies to the results of a published systematic review on patient preferences for disease-modifying antirheumatic drug (DMARD) treatment in rheumatoid arthritis (RA) [42]. In this systematic review, thirty-one studies had an overall quality ranging from medium to high, and four had a low overall quality (Appendix 5).

We combined the developed strategy (using AND) with the following search terms:

Rheumatoid arthritis OR RA OR rheumatic diseases OR disease-modifying antirheumatic drugs OR DMARDs OR antirheumatic agents OR Arthritis, Rheumatoid / drug therapy [MeSH] OR Arthritis, Rheumatoid / therapy [MeSH] OR Antirheumatic Agents / therapeutic use [MeSH] OR Biological Products / therapeutic use [MeSH].

We ran the search in PubMed in December 2020. We calculated the relative performance of the content search strategy in PubMed by dividing the number of articles included in the systematic review and detected by the search strategy by the total number of articles included in the systematic review and cited in PubMed [14].

Testing the relative performance of the extended search strategy by replicating a published systematic review

We compared the performance of the extended content search strategy to the results of a published systematic review on patient preferences for lung cancer treatment [43]. In this systematic review, fourteen studies had an overall quality ranging from medium to high, and one study had a low overall quality (Appendix 3).

We combined the extended content search strategy (using AND) with the following search terms:

Lung Neoplasm [MeSH] OR Antineoplastic Agents [MeSH] OR lung cancer [tiab]

We ran the search in PubMed in July 2021. We calculated the relative performance of the content search strategy in PubMed by dividing the number of articles included in the systematic review and detected by the extended search strategy by the total number of articles included in the systematic review.

RESULTS

Developing a bank of search terms and refining the search strategy

We conceptualized a final search strategy to retrieve studies on patient-based BRA after a series of refining rounds. The search terms are grouped into two domains: (1) domain of the patient and (2) domain of the BRA of medicines (Table 2). The reproducible search strategy can also be found in Appendix 6. Terms within each domain were combined using the Boolean operator OR, and the two domains were combined using the operator AND. Several search entries were directly sourced from the MeSH database, such as Patient Preference, Choice Behavior, Benefit Risk Assessment, and Risk Assessment. There were no specific entries corresponding to patient-based risk assessment or patient-based BRA below the corresponding terms or other related terms in the MeSH hierarchy. Most search terms included in the BRA domain were in the form of free text extracted from our review of selected studies of patients' understanding and preferences for risks and benefits of their medicines. In contrast, the terms compiled in the patient domain derived mostly from MeSH terms and subheadings in existing search strategies. In the initial search, these strategies identified articles with relatively high sensitivity and acceptable precision with a focus on patients' views and perceptions of all aspects of their disease and health management.

We further refined the search strategy when we added the concept of medicine or treatment to the approach. Many retrieved studies discussed patients' participation in physical activity and screening activities rather than eliciting patients' preferences about their medicines. Most studies were retrieved using the term “participation,” so we deemed terms related to participation (patient's participation, user's participation) as redundant (sensitivity of each term below 1%). Similarly, most of the studies retrieved using the term “patient satisfaction” (not treatment satisfaction) focused on patients' assessment of health services, programs, and use of devices (sensitivity below 1%), so we removed this search term.

We included in our initial bank of search terms as many entries that identified articles relevant to patients' preferences and choice behavior as possible (e.g., user, user's, users, consumers, individuals, based, focused, centered), given the inconsistencies in the nomenclature and indexing about patients' preferences and perspectives on medicines. We noticed during the refinement stage that the term “patient” was more often used in studies examining the BRA of medicines, but the terms “user” and “consumer” were more often used in studies about devices and services. We excluded the latter terms in the final search strategy (sensitivity of each term below 1%). We used truncation on selected free-text entries to reduce the number of terms (e.g., accepta* covered acceptance, acceptability, and acceptable). The simultaneous use of both the MeSH term and the corresponding free text yielded more results than either alone. This was the rationale for the final strategy containing double entries to be searched in “All fields OR in MeSH.”

The refining component was based on the search terms identified in the screening of seventy studies focused on patient-based BRA of medicines (Appendix 2) from various therapeutic areas. We derived search terms from fifty-seven different journals with different indexing requirements and use of terms. The refining component comprised an iterative process of testing each term for sensitivity and precision. We used T2DM as a case condition. We included terms in the final strategy if they had acceptable sensitivity (more than 1% in the first round) and precision (higher than 2% in the second round). The individual precision varied widely across included search terms as well as the number of total articles retrieved per term. For example, stated preference*[tiab] had the highest precision (70.0%) but with seven relevant articles out of ten retrieved, whereas Patient Preference*[MeSH] had the second highest (33.3%) with 67 relevant articles out of 201 retrieved. Tradeoff*[All fields], willingness[tiab], and Choice Behavior[MeSH] also had high individual precision: 24.4% (20 relevant articles of 82 retrieved), 15.2% (44 relevant articles of 289 retrieved), and 9.7% (20 relevant articles of 207 retrieved), respectively. The rest of the included terms had lower precisions, with substantially higher numbers of articles retrieved like attribute*[tiab] (4.6%, 46 relevant articles of 998 retrieved); Health Knowledge, Attitudes, Practice [MeSH] (3.0%, 57 relevant articles of 1,875 retrieved); and Benefit Risk Assessment [MeSH] (2.0%, 46 relevant articles of 2,266). The combination of terms with Boolean operators enabled us to optimize the performance of the overall content search strategy.

While refining the compiled search terms, we found selected entries (free text and controlled vocabulary) with passable sensitivity (more than 1%) but very low precision (less than 2%). These entries, if used together, increased the sensitivity of the search strategy but decreased the precision. These search terms were in both domains: (a) the patient (e.g., treatment satisfaction, perception, Patient Acceptance of Health Care [MeSH]), and (b) the BRA of medicines (e.g., discontinuation, medication belief*, patient-relevant benefit, Risk [MeSH], Treatment Outcome [MeSH]). We built an extended search strategy containing the final refined strategy combined with all these additional terms (Appendix 3).

Validation of the content search strategy

Validating the search strategy in MS as a case condition

We validated the final search strategy by combining it (using AND, Filter applied: in the last 10 years) with the relevant search terms for MS in PubMed. Search A returned 304 results: 63 relevant (true positives) and 241 not relevant to patient-based BRA of DMARDs (false positives). A search using only the relevant search terms for MS (Filter applied: in the last 10 years—search B) returned 43,901 hits including 75 relevant (which included 12 false negatives) and 43,825 true negatives. False negatives are the relevant articles that the search strategy failed to detect. All relevant articles (true positives and false negatives) are listed with their corresponding keywords and MeSH terms in Table 3. The sensitivity of the content search strategy was 84.0%, the specificity was 99.4%, and the accuracy was 99.4%. The PPV was 20.7% and the NNR was 4.8 studies (Table 4).

Relative performance of the content search strategy

The systematic review on patient preferences for DMARD treatment in RA included 36 unique studies [42]. We used 35 studies to benchmark the output of the content search strategy as one study was not cited in PubMed [44]. Our search combining the developed content search strategy with the search syntax relevant to RA yielded 30 of the 35 studies: an 87.5% relative performance of the content search strategy. All articles in the systematic review (detected and not detected by the content search strategy) are listed with their corresponding keywords and MeSH terms in Appendix 5.

Relative performance of the extended search strategy

The systematic review on patient preferences for lung cancer treatment included fifteen unique studies [43]. We used all the included studies to benchmark the output of the extended search strategy, as all were listed in PubMed. The search combining the extended content search strategy with the search syntax relevant to lung neoplasm detected fourteen of the fifteen studies. This equates to 93.3% relative performance for the extended filter. All articles in the systematic review (detected and not detected by the extended content search strategy) are listed with their corresponding keywords and MeSH terms in Appendix 3 (Table 2). The reproducible extended search strategy can also be found in Appendix 6.

DISCUSSION

We developed a high-performance, objectively derived search strategy for patient-focused BRA of medicines. This search filter had high sensitivity for studies about patients' preferences and perspectives on the benefits and risks of their medicines, with excellent specificity and accuracy, and acceptable precision. Our proposed content search strategy was more targeted to retrieve studies on how patients perceive the effectiveness of their medicines and whether the potential benefits outweigh the harms when compared with another search filter designed to identify existing literature on patients' knowledge, views, and values [15]. The main advantage of our search strategy is that it has been empirically developed and validated based on clearly defined, pragmatic, and reproducible methods [20]. The use of such strategies could minimize the time, biases, and potential obstacles associated with those subjectively derived, such as the need for multiple search queries to make the search sufficiently wide and the subsequent need to restrict when the search yield is too broad [20, 45].

Patient-centered care encompasses shared decision-making, support for self-management, patient information, patient empowerment, care planning, the integration of medical and nonmedical care, and good communication between clinicians and patients [46]. It is also important to understand patients' preferences in all aspects of their disease management. It is therefore important to efficiently collect evidence on the priorities and perspectives of patients regarding their medicines for prescribing, developing clinical guidelines, or making decisions in health technology assessment [39, 47].

We detected the search terms mostly used in studies relevant to patient-focused BRA of medicines by developing a bank of search terms and iteratively refining these terms. Although the terms for the domain of the patient are relatively consistent, those terms relevant to the domain of benefits and risks of medicines are variable. Most search terms in the domain were extracted as free text from the screened articles. The MeSH terms of Benefit Risk Assessment and Risk Assessment were neither precise nor sensitive in retrieving articles pertinent to patient-based BRA of medicines. Specific entries corresponding to patient-based risk assessment or patient-based benefit-risk assessment were not found below these terms in the MeSH hierarchy. It seems that the concept of patient-based BRA has not yet been identified under the larger concept of risk assessment or BRA in general.

Despite the relative consistency in the use of keywords and MeSH terms in the domain of patients and their preferences, we did not incorporate any of these terms in the search syntax of our validation case of MS—with and without the proposed content search strategy. The main reason was to detect the largest possible number of publications that address patient-based BRA of drug therapy in MS. For example, the use of certain terms (e.g, Patient Preference [MeSH] OR Patient Preference / psychology [MeSH Subheading] OR Patient Preference / statistics & numerical data [MeSH Subheading] OR Decision Making [MeSH] OR Choice Behavior [MeSH] OR Health Knowledge, Attitudes, Practice [MeSH]) in the syntax for MS would have precluded retrieving articles where the keywords used to describe patient preference were respondents' attitude, patient engagement, or opinion of individuals. Screening of the many publications related to MS and its treatment in the last ten years allowed us to detect more precisely the true and—more importantly—the false negatives of the proposed content search strategy. The benchmarking of the pool of articles included in our validation versus those included in two published systematic reviews on patient-based BRA of medicines in MS permitted an optimal calculation of the performance parameters. Testing our content search strategy against a completed systematic review permitted us to calculate the relative performance of our generic search strategy versus the sophisticated and thorough research approach adopted in systematic reviews. It also allowed us to test it in a different case condition.

The success of an empirically derived content search strategy is estimated by the generalizability of the gold standard adopted in its validation. Despite being costly and time-consuming, hand-searching is still viewed as the method of choice [20]. The manual search based on MS syntax alone (43,901 hits) enabled generation of a reference set of seventy-five articles, which represented the gold standard in the validation. Nevertheless, this reference set was limited to one disease, and this might have introduced a bias in the calculation of the performance parameters. To counter this limitation, we performed a second validation based on extracted references from a relevant systematic review in another therapeutic area, RA. The systematic review–based quasi-gold standards have been increasingly used as an alternative approach for a gold standard in search filter development and validation [20].

A key point in developing search strategies is striking a balance between the sensitivity and specificity for the intended end users. It is hard to determine when a search strategy is completed, as there are no fixed measures or criteria for performance parameters for this purpose [20]. Moreover, increasingly extensive strategies may be more prone to errors. We proposed a high-performing strategy and a more extended one with higher sensitivity but where more articles would need to be read. The different performance levels of those two strategies will cater to different end users and their particular information needs.

Strengths and limitations

This is the first study, to our knowledge, to develop a content search strategy that contains generic keywords and MeSH terms and subheadings that retrieve published data on patient-based BRA of medicines with high sensitivity, specificity, and accuracy. It can be used to identify the evidence on patient preferences and perspectives on their medicines in any therapeutic area when combined (using the Boolean operator AND) with search entries pertinent to a given disease or condition. This research-based search strategy can replace subjectively derived and unvalidated strategies currently used in the field of patient-based BRA, which rely more on expert opinions and are regarded as methodologically weak in standard biomedical evidence hierarchies [48, 49].

The main study limitation is that the search strategy was designed and validated using only one database, PubMed. All search strategies must be developed for a specific database [18], due to disparities in metadata sets and discrepancies in the use of controlled vocabularies, search syntax, truncation, and proximity operators [48]. This content search strategy could be readapted, and its performance reexamined in other large bibliographic databases (e.g., Embase).

Implications for practice

This search strategy for identifying patient-based BRA of medicines can be used with confidence by not only information specialists but also clinicians or regulators with limited bibliographic search skills in a wide range of clinical areas where there is a need or wish to integrate patients' perspectives in the assessment of medicines. Its metric-based performance can inform prospective users of the retrieval rate they can expect in their search. An extended search strategy is also proposed to be used for a more exhaustive search. The methods used to develop and validate the search filter can also be replicated in other complex search domains.

FUNDING

This research was not supported by any funding.

CONFLICT OF INTEREST

The authors declare they have no conflicts of interest.

AVAILABILITY OF DATA

There is no additional data related to this manuscript. Endnote libraries can be shared upon request.

AUTHORS' CONTRIBUTIONS

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by HEM, SH, MVD, CD, and TM. The first draft of the manuscript was written by HEM, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

SUPPLEMENTAL FILES

Appendix 1: Search terms forming the hedge

Appendix 2: Articles screened for building and refining a search term bank with corresponding keywords and MeSH terms

Appendix 3: Extended content search strategy and testing its relative performance by replicating a published systematic review

Appendix 4: Articles included in two systematic reviews on patient preferences for attributes of disease-modifying drugs in multiple sclerosis

Appendix 5: Testing the relative performance of the content search strategy by replicating a published systematic review

Appendix 6: Reproducible search strategies on patient-based benefit-risk assessment of medicines in PubMed

REFERENCES

1. Stowasser DA, Allinson YM, O'Leary M. Understanding the medicines management pathway. JPPR. 2004;34(4):293–6.

2. Muhlbacher AC, Juhnke C, Beyer AR, Garner S. Patient-focused benefit-risk analysis to inform regulatory decisions: the European Union perspective. Value Health. 2016;19(6):734–40.

3. Curtin F, Schulz P. Assessing the benefit: risk ratio of a drug-randomized and naturalistic evidence. Dialogues Clin Neurosci. 2011;13(2):183–90.

4. Eichler HG, Abadie E, Raine JM, Salmonson T. Safe drugs and the cost of good intentions. N Engl J Med. 2009;360(14):1378–80.

5. Harrison M, Milbers K, Hudson M, Bansback N. Do patients and health care providers have discordant preferences about which aspects of treatments matter most? Evidence from a systematic review of discrete choice experiments. BMJ Open. 2017;7(5):e014719.

6. El Masri H, Hollingworth SA, van Driel M, Benham H, McGuire TM. Real-world questions and concerns about disease-modifying antirheumatic drugs (DMARDs): a retrospective analysis of questions to a medicine call center. BMC Rheumatol. 2020;4:27.

7. Gabriel SE, Normand SLT. Getting the methods right — the foundation of patient-centered outcomes research. N Engl J Med. 2012;367(9):787–90.

8. Postmus D, Mavris M, Hillege HL, Salmonson T, Ryll B, Plate A, Moulon I, Eichler H-G, Bere N, Pignatti F. Incorporating patient preferences into drug development and regulatory decision making: results from a quantitative pilot study with cancer patients, carers, and regulators. Clin Pharmacol Ther. 2016;99(5):548–54.

9. Marsh K, Lanitis T, Neasham D, Orfanos P, Caro J. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. PharmacoEconomics. 2014;32(4):345–65.

10. Clark MD, Determann D, Petrou S, Moro D, de Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32(9):883–902.

11. Facey K, Boivin A, Gracia J, Hansen HP, Lo Scalzo A, Mossman J, Single A. Patients' perspectives in health technology assessment: a route to robust evidence and fair deliberation. Int J Technol Assess Health Care. 2010;26(3):334–40.

12. Dirksen CD. The use of research evidence on patient preferences in health care decision-making: issues, controversies and moving forward. Expert Rev Pharmacoecon Outcomes Res. 2014;14(6):785–94.

13. Coast J, Al-Janabi H, Sutton EJ, Horrocks SA, Vosper AJ, Swancutt DR, Flynn TN. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730–41.

14. Selva A, Sola I, Zhang Y, Pardo-Hernandez H, Haynes RB, Martinez Garcia L, Navarro T, Schünemann H, Alonso-Coello P.. Development and use of a content search strategy for retrieving studies on patients' views and preferences. Health Qual Life Outcomes. 2017;15(1):126.

15. Wessels M, Hielkema L, Weijden T. How to identify existing literature on patients' knowledge, views, and values: the development of a validated search filter. J Med Libr Assoc. 2016;104:320–4.

16. Greenhalgh T, Humphrey C, Woodard F. User involvement in health care. Hoboken, NJ: Wiley-Blackwell/BMJ Books; 2011.

17. Zhang L, Jiang S, Ren C, Gao C. Is long-term warfarin therapy necessary in Chinese patients with atrial fibrillation after bioprosthetic mitral valve replacement and left atrial appendage obliteration? Heart Surg Forum. 2015;18(1):E11–16.

18. Jenkins M. Evaluation of methodological search filters–a review. Health Info Libr J. 2004;21(3):148–63.

19. Glanville J, Bayliss S, Booth A, Dundar Y, Fernandes H, Fleeman ND, Foster L, Fraser C, Fry-Smith A, Golder S, Lefebvre C, Miller C, Paisley S, Payne L, Price A, Welch K. So many filters, so little time: the development of a search filter appraisal checklist. J Med Libr Assoc. 2008;96(4):356–61.

20. Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Syst Rev. 2012;1:19.

21. Pols DH, Bramer WM, Bindels PJ, van de Laar FA, Bohnen AM. Development and validation of search filters to identify articles on family medicine in online medical databases. Ann Fam Med. 2015;13(4):364–6.

22. Sarwar N, Gao P, Seshasai SR, Gobin R, Kaptoge S, Di Angelantonio E, Ingelsson E, Lawlor DA, Selvin E, Stampfer M, Stehouwer CDA, Lewington S, Pennells L, Thompson A, Sattar N, White IR, Ray K, Danesh J. Diabetes mellitus, fasting blood glucose concentration, and risk of vascular disease: a collaborative meta-analysis of 102 prospective studies. Lancet. 2010;375(9733):2215–22.

23. Zelniker TA, Wiviott SD, Raz I, Im K, Goodrich EL, Bonaca MP, Mosenzon O, Kato E, Cahn A, Furtado R, Bhatt DL 1, Leiter LA, McGuire DK, Wilding J, Sabatine MS.. SGLT2 inhibitors for primary and secondary prevention of cardiovascular and renal outcomes in type 2 diabetes: a systematic review and meta-analysis of cardiovascular outcome trials. Lancet. 2019;393(10166):31–39.

24. Fitchett D, Inzucchi SE, Cannon CP, McGuire DK, Scirica BM, Johansen OE, Sambevski S, Kaspers S, Pfarr E, George JT, Zinman B. Empagliflozin reduced mortality and hospitalization for heart failure across the spectrum of cardiovascular risk in the EMPA-REG OUTCOME trial. Circulation. 2019;139(11):1384–95.

25. Cosentino F, Grant PJ, Aboyans V, Bailey CJ, Ceriello A, Delgado V, Federici M, Filippatos G, Grobbee DE, Hansen TB, Huikuri HV, Johansson I, Jüni P, Lettino M, Marx M, Mellbin LG, Östgren CJ, Rocca B, Roffi M, Sattar N, Seferović PM, Sousa-Uva M, Valensi P, Wheeler DC, ESC Scientific Document Group. 2019 ESC Guidelines on diabetes, pre-diabetes, and cardiovascular diseases developed in collaboration with the EASD. Eur Heart J. 2020;41(2):255–323.

26. Joy SM, Little E, Maruthur NM, Purnell TS, Bridges JF. Patient preferences for the treatment of type 2 diabetes: a scoping review. Pharmacoeconomics. 2013;31(10):877–92.

27. McSharry J, McGowan L, Farmer AJ, French DP. Perceptions and experiences of taking oral medications for the treatment of Type 2 diabetes mellitus: a systematic review and meta-synthesis of qualitative studies. Diabet Med. 2016;33(10):1330–8.

28. Ho GJ, Liew SM, Ng CJ, Hisham Shunmugam R, Glasziou P. Development of a search strategy for an evidence based retrieval service. PloS One. 2016;11(12):e0167170.

29. Lessick S. The Medical Library Association essential guide to becoming an expert searcher: proven techniques, strategies, and tips for finding health information. J Med Libr Assoc. 2009 Apr;97(2):153–55.

30. Webster AC, Cross NB, Mitchell R, Craig JC. How to get the most from the medical literature: searching the medical literature effectively. Nephrology (Carlton). 2010;15(1):12–19.

31. Montalban X, Gold R, Thompson AJ, Otero-Romero S, Amato MP, Chandraratna D, Clanet M, Comi G, Derfuss T, Fazekas F, Hartung HP, Havrdova E, Hemmer B, Kappos L, Liblau R, Lubetzki C, Marcus E, Miller DH, Olsson T, Pilling S, Selmaj K, Siva A, Sorensen PS, Sormani MP, Thalheim C, Wiendl H, Zipp F. ECTRIMS/EAN Guideline on the pharmacological treatment of people with multiple sclerosis. Mult Scler. 2018;24(2):96–120.

32. Cocco E, Caoci A, Lorefice L, Marrosu MG. Perception of risk and shared decision making process in multiple sclerosis. Expert Rev Neurother. 2017;17(2):173–80.

33. Eskyte I, Manzano A, Pepper G, Pavitt S, Ford H, Bekker H, Chataway J, Schmierer K, Meads D, Webb E, Potrata B. Understanding treatment decisions from the perspective of people with relapsing remitting multiple sclerosis: a critical interpretive synthesis. Mult Scler Relat Disord. 2019;27:370–7.

34. Reen GK, Silber E, Langdon DW. Multiple sclerosis patients' understanding and preferences for risks and benefits of disease-modifying drugs: a systematic review. J Neurol Sci. 2017;375:107–22.

35. Webb EJD, Meads D, Eskyte I, King N, Dracup N, Chataway J, Ford HL, Marti J, Pavitt SH, Schmierer K, Manzano A. A systematic review of discrete-choice experiments and conjoint analysis studies in people with multiple sclerosis. Patient. 2018;11(4):391–402.

36. Haynes RB, Wilczynski NL. Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey. BMJ (Clinical research ed). 2004;328(7447):1040.

37. Hoogendam A. Evaluation of PubMed filters used for evidence-based searching: validation using relative recall. J Med Libr Assoc. 2009;97(3):186–93.

38. Littell JH, Corcoran J, Pillai V. Systematic reviews and meta-analysis. New York: Oxford University Press; 2008.

39. Mühlbacher A, Bethge S. First and foremost battle the virus: eliciting patient preferences in antiviral therapy for hepatitis c using a discrete choice experiment. Value Health. 2016;19(6):776–87.

40. Yu T, Enkh-Amgalan N, Zorigt G. Methods to perform systematic reviews of patient preferences: a literature survey. BMC Med Res Methodol. 2017;17(1):166.

41. Eiring Ø, Landmark BF, Aas E, Salkeld G, Nylenna M, Nytrøen K. What matters to patients? A systematic review of preferences for medication-associated outcomes in mental disorders. BMJ Open. 2015;5(4):e007848

42. Durand C, Eldoma M, Marshall DA, Bansback N, Hazlewood GS. Patient preferences for disease-modifying antirheumatic drug treatment in rheumatoid arthritis: a systematic review. J Rheumatol. 2020;47(2):176–87.

43. Sugitani Y, Sugitani N, Ono S. Quantitative preferences for lung cancer treatment from the patients' perspective: a systematic review. Patient. 2020;13(5):521–36.

44. Ferraz MB, Pinheiro GR, Helfenstein M, Albuquerque E, Rezende C, Roimicher L, Brandao L, Silva SC, Pinheiro GC, Atra E. Combination therapy with methotrexate and chloroquine in rheumatoid arthritis. A multicenter randomized placebo-controlled trial. Scand J Rheumatol. 1994;23(5):231–6.

45. Damarell RA, Tieman JJ, Sladek RM. OvidSP Medline-to-PubMed search filter translation: a methodology for extending search filter range to include PubMed's unique content. BMC Med Res Methodol. 2013;13:86.

46. Scholl I, Zill JM, Harter M, Dirmaier J. An integrative model of patient-centeredness - a systematic review and concept analysis. PloS One. 2014;9(9):e107828.

47. Muhlbacher AC, Juhnke C. Patient preferences versus physicians' judgement: does it make a difference in healthcare decision making? Appl Health Econ Health Policy. 2013;11(3):163–80.

48. Bak G, Mierzwinski-Urban M, Fitzsimmons H, Morrison A, Maden-Jenkins M. A pragmatic critical appraisal instrument for search filters: introducing the CADTH CAI. Health Info Libr J. 2009;26(3):211–9.

49. ISSG Search Filter Resource [Internet]. Glanville J, Lefebvre C, Manson P, Robinson S and Shaw N, editors. York (UK): The InterTASC Information Specialists' Sub-Group; 2006 [cited 15 March 2021]. <http://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home>.

50. Abolfazli R, Elyasi A, Javadi MR, Gholami K, Torkamandi H, Amir-Shahkarami M, Etemadifar M, Nasr Z. Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta. Iran J Neurol. 2014;13(3):160–7.

51. Arenson K. Freedom to be altruistic: allowing for risk/knowledge ratios in decisions concerning multiple sclerosis research. Open Med. 2011;5(1):e26–27.

52. Arroyo R, Sempere AP, Ruiz-Beato E, Prefasi D, Carreño A, Roset M, Maurino J. Conjoint analysis to understand preferences of patients with multiple sclerosis for disease-modifying therapy attributes in Spain: a cross-sectional observational study. BMJ Open. 2017;7(3):e014433.

53. Barone DA, Singer BA, Merkov L, Rametta M, Suarez G. Survey of US patients with multiple sclerosis: comparison of the new electronic interferon beta-1b autoinjector (BETACONNECT™) with mechanical autoinjectors. Neurol Ther. 2016;5(2):155–67.

54. Bauer B, Brockmeier B, Devonshire V, Charbonne A, Wach D, Hendin B. An international discrete choice experiment assessing patients' preferences for disease-modifying therapy attributes in multiple sclerosis. Neurodegener Dis Manag. 2020.

55. Bayas A, Mäurer M. Teriflunomide for the treatment of relapsing-remitting multiple sclerosis: patient preference and adherence. Patient Prefer Adherence. 2015;9:265–74.

56. Beckmann H, Augustin M, Heesen C, Poettgen J, Blome C. Benefit evaluation in multiple sclerosis relapse treatment from the patients' perspective - Development and validation of a new questionnaire. Mult Scler Relat Disord. 2019;28:256–61.

57. Bichuetti DB, Franco CA, Elias I, Mendonça ACR, Carvalho LFD, Diniz DS, Tur C, Tintoré M, de Oliveira EML. Multiple sclerosis risk perception and acceptance for Brazilian patients. Arq Neuropsiquiatr. 2018;76(1):6–12.

58. Boeru G, Milanov I, De Robertis F, Kozubski W, Lang M, Rojas-Farreras S, Tomlinson M. ExtaviJect® 30G device for subcutaneous self-injection of interferon beta-1b for multiple sclerosis: a prospective European study. Med Devices (Auckl). 2013;6:175–84.

59. Bottomley C, Lloyd A, Bennett G, Adlard N. A discrete choice experiment to determine UK patient preference for attributes of disease modifying treatments in Multiple Sclerosis. J Med Econ. 2017;20(8):863–70.

60. Brown H, Gabriele S, White J. Physician and patient treatment decision-making in relapsing-remitting multiple sclerosis in Europe and the USA. Neurodegener Dis Manag. 2018;8(6):371–6.

61. Bruce JM, Bruce AS, Catley D, Lynch S, Goggin K, Reed D, Lim SL, Strober L, Glusman M, Ness AR, Jarmolowicz DP. Being kind to your future self: probability discounting of health decision-making. Ann Behav Med. 2016;50(2):297–309.

62. Bruce JM, Bruce AS, Lynch S, Thelen J, Lim SL, Smith J, Catley D, Reed DD, Jarmolowicz DP. Probability discounting of treatment decisions in multiple sclerosis: associations with disease knowledge, neuropsychiatric status, and adherence. Psychopharmacology (Berl). 2018;235(11):3303–13.

63. Bruce JM, Jarmolowicz DP, Lynch S, Thelen J, Lim SL, Smith J, Catley D, Bruce AS. How patients with multiple sclerosis weigh treatment risks and benefits. Health Psychol. 2018;37(7):680–90.

64. Carlin CS, Higuera L, Anderson S. Improving patient-centered care by assessing patient preferences for multiple sclerosis disease-modifying agents: a stated-choice experiment. Perm J. 2017;21:16–102.

65. Ceuninck van Capelle A, Meide HV, Vosman FJH, Visser LH. A qualitative study assessing patient perspectives in the process of decision-making on disease modifying therapies (DMT's) in multiple sclerosis (MS). PLoS One. 2017;12(8):e0182806.

66. Col NF, Solomon AJ, Springmann V, Garbin CP, Ionete C, Pbert L, Alvarez E, Tierman B, Hopson A, Kutz C, Morales IB, Griffin C, Phillips G, Ngo LH. Whose preferences matter? A patient-centered approach for eliciting treatment goals. Med Decis Making. 2018;38(1):44–55.

67. Col NF, Solomon AJ, Springmann V, Ionete C, Alvarez E, Tierman B, Kutz C, Morales IB, Griffin C, Ngo LH, Jones DE, Phillips G, Hopson A, Pbert L. Evaluation of a novel preference assessment tool for patients with multiple sclerosis. Int J MS Care. 2018;20(6):260–7.

68. de Seze J, Borgel F, Brudon F. Patient perceptions of multiple sclerosis and its treatment. Patient Prefer Adherence. 2012;6:263–73.

69. Fernández O, Duran E, Ayuso T, Hernández L, Bonaventura I, Forner M. Treatment satisfaction with injectable disease-modifying therapies in patients with relapsing-remitting multiple sclerosis (the STICK study). PLoS One. 2017;12(10):e0185766.

70. Fox RJ, Salter A, Alster JM, Dawson NV, Kattan MW, Miller D, Ramesh S, Tyry T, Wells BW, Cutter G. Risk tolerance to MS therapies: survey results from the NARCOMS registry. Mult Scler Relat Disord. 2015;4(3):241–9.

71. Frost N, Freeman J, Brixner D, Mort J, Clem J, Ngorsuraches S. Patients' preferences and willingness-to-pay for disease-modifying therapies. Mult Scler Relat Disord. 2019;35:55–60.

72. Garcia-Dominguez JM, Muñoz D, Comellas M, Gonzalbo I, Lizán L, Polanco Sánchez C. Patient preferences for treatment of multiple sclerosis with disease-modifying therapies: a discrete choice experiment. Patient Prefer Adherence. 2016;10:1945–56.

73. Glanz BI, Greeke E, LaRussa A, Stuart F, Rintell DJ, Chitnis T, Healy BC. Risk attitudes and risk perceptions in individuals with multiple sclerosis. Mult Scler J Exp Transl Clin. 2016;2:2055217316665406.

74. Goodwin E, Green C, Hawton A. What difference does it make? A comparison of health state preferences elicited from the general population and from people with multiple sclerosis. Value Health. 2020;23(2):242–50.

75. Heesen C, Kleiter I, Meuth SG, Krämer J, Kasper J, Köpke S, Gaissmaier W. Benefit-risk perception of natalizumab therapy in neurologists and a large cohort of multiple sclerosis patients. J Neurol Sci. 2017;376:181–90.

76. Heesen C, Kleiter I, Nguyen F, Schäffler N, Kasper J, Köpke S, Gaissmaier W. Risk perception in natalizumab-treated multiple sclerosis patients and their neurologists. Mult Scler. 2010;16(12):1507–12.

77. Hincapie AL, Penm J, Burns CF. Factors associated with patient preferences for disease-modifying therapies in multiple sclerosis. J Manag Care Spec Pharm. 2017;23(8):822–30.

78. Hofmann A, Stellmann JP, Kasper J, Ufer F, Elias WG, Pauly I, Repenthin J, Rosenkranz T, Weber T, Köpke S, Heesen C, MS Network Hamburg. Long-term treatment risks in multiple sclerosis: risk knowledge and risk perception in a large cohort of mitoxantrone-treated patients. Mult Scler. 2013;19(7):920–5.

79. Jarmolowicz DP, Bruce AS, Glusman M, Lim SL, Lynch S, Thelen J, Catley D, Zieber N, Reed DD, Bruce JM. On how patients with multiple sclerosis weigh side effect severity and treatment efficacy when making treatment decisions. Exp Clin Psychopharmacol. 2017;25(6):479–84.

80. Jarmolowicz DP, Reed DD, Bruce AS, Catley D, Lynch S, Goggin K, Lim SL, Strober L, Glusman M, Norouzinia AN, Bruce JM. Using EP50 to forecast treatment adherence in individuals with multiple sclerosis. Behav Processes. 2016;132:94–9.

81. Jarmolowicz DP, Reed DD, Bruce AS, Lynch S, Smith J, Bruce JM. Modeling effects of side-effect probability, side-effect severity, and medication efficacy on patients with multiple sclerosis medication choice. Exp Clin Psychopharmacol. 2018;26(6):599–607.

82. Köpke S, Kern S, Ziemssen T, Berghoff M, Kleiter I, Marziniak M, Paul F, Vettorazzi E, Pöttgen J, Fischer K, Kasper J, Heesen C. Evidence-based patient information programme in early multiple sclerosis: a randomised controlled trial. J Neurol Neurosurg Psychiatry. 2014;85(4):411–8.

83. Kremer IE, Evers SM, Jongen PJ, van der Weijden T, van de Kolk I, Hiligsmann M. Identification and prioritization of important attributes of disease-modifying drugs in decision making among patients with multiple sclerosis: a nominal group technique and best-worst scaling. PLoS One. 2016;11(11):e0164862.

84. Kremer IEH, Evers S, Jongen PJ, Hiligsmann M. Comparison of preferences of healthcare professionals and MS patients for attributes of disease-modifying drugs: a best-worst scaling. Health Expect. 2018;21(1):171–80.

85. Lee Mortensen G, Rasmussen PV. The impact of quality of life on treatment preferences in multiple sclerosis patients. Patient Prefer Adherence. 2017;11:1789–96.

86. Lin PJ, Saret CJ, Neumann PJ, Sandberg EA, Cohen JT. Assessing the value of treatment to address various symptoms associated with multiple sclerosis: results from a contingent valuation study. Pharmacoeconomics. 2016;34(12):1255–65.

87. Lizán L, Comellas M, Paz S, Poveda JL, Meletiche DM, Polanco C. Treatment adherence and other patient-reported outcomes as cost determinants in multiple sclerosis: a review of the literature. Patient Prefer Adherence. 2014;8:1653–64.

88. Lynd LD, Henrich NJ, Hategeka C, Marra CA, Mittmann N, Evans C, Traboulsee AL. Perspectives of patients with multiple sclerosis on drug treatment: a qualitative study. Int J MS Care. 2018;20(6):269–77.

89. Lynd LD, Traboulsee A, Marra CA, Mittmann N, Evans C, Li KH, Carter M, Hategekimana C. Quantitative analysis of multiple sclerosis patients' preferences for drug treatment: a best-worst scaling study. Ther Adv Neurol Disord. 2016;9(4):287–96.

90. Mansfield C, Thomas N, Gebben D, Lucas M, Hauber AB. Preferences for multiple sclerosis treatments: using a discrete-choice experiment to examine differences across subgroups of US patients. Int J MS Care. 2017;19(4):172–83.

91. McDonnell GV, Marriott JJ. Patient-perceived risks of MS DMTs: problems of communication and risk management? Neurology. 2019;92(14):647–8.

92. McGinley MP, Cola PA, Fox RJ, Cohen JA, Corboy JJ, Miller D. Perspectives of individuals with multiple sclerosis on discontinuation of disease-modifying therapies. Mult Scler. 2019:1352458519867314.

93. Mendel R, Traut-Mattausch E, Frey D, Bühner M, Berthele A, Kissling W, Hamann J. Do physicians' recommendations pull patients away from their preferred treatment options? Health Expect. 2012;15(1):23–31.

94. Miller CE, Karpinski M, Jezewski MA. Relapsing-remitting multiple sclerosis patients' experience with natalizumab: a phenomenological investigation. Int J MS Care. 2012;14(1):39–44.

95. Poulos C, Kinter E, van Beek J, Christensen K, Posner J. Preferences of patients with multiple sclerosis for attributes of injectable multiple sclerosis treatments in the United Kingdom and France. Int J Technol Assess Health Care. 2018;34(4):425–33.

96. Poulos C, Kinter E, Yang JC, Bridges JF, Posner J, Gleiβner E, Mühlbacher A, Kieseier B. A discrete-choice experiment to determine patient preferences for injectable multiple sclerosis treatments in Germany. Ther Adv Neurol Disord. 2016;9(2):95–104.

97. Poulos C, Kinter E, Yang JC, Bridges JF, Posner J, Reder AT. Patient preferences for injectable treatments for multiple sclerosis in the United States: a discrete-choice experiment. Patient. 2016;9(2):171–80.

98. Poulos C, Wakeford C, Kinter E, Mange B, Schenk T, Jhaveri M. Patient and physician preferences for multiple sclerosis treatments in Germany: a discrete-choice experiment study. Mult Scler J Exp Transl Clin. 2020;6(1):2055217320910778.

99. Rahimi F, Rasekh HR, Abbasian E, Peiravian F, Etemadifar M, Ashtari F, Sabzghabaee AM, Amirsadri MR. Patient preferences for Interferon-beta in Iran: a discrete choice experiment. PLoS One. 2018;13(3):e0193090–100.

100. Rath L, Vijiaratnam N, Skibina O. Assessing understanding of individual risk and symptoms of progressive multifocal leukoencephalopathy in patients prescribed natalizumab for multiple sclerosis. Intern Med J. 2017;47(2):194–9.

101. Rosato R, Testa S, Oggero A, Molinengo G, Bertolotto A. Quality of life and patient preferences: identification of subgroups of multiple sclerosis patients. Qual Life Res. 2015;24(9):2173–82.

102. Salamonsen A. Mind the gap! Lay and medical perceptions of risks associated with the use of alternative treatment and conventional medicine. Forsch Komplementmed. 2015;22(1):24–29.

103. Salter AR, Marrie RA, Agashivala N, Belletti DA, Kim E, Cutter GR, Cofield SS, Tyry T. Patient perspectives on switching disease-modifying therapies in the NARCOMS registry. Patient Prefer Adherence. 2014;8:971–9.

104. Sempere AP, Vera-Lopez V, Gimenez-Martinez J, Ruiz-Beato E, Cuervo J, Maurino J. Using a multidimensional unfolding approach to assess multiple sclerosis patient preferences for disease-modifying therapy: a pilot study. Patient preference and adherence. 2017;11:995–9.

105. Shingler SL, Swinburn P, Ali S, Perard R, Lloyd AJ. A discrete choice experiment to determine patient preferences for injection devices in multiple sclerosis. J Med Econ. 2013;16(8):1036–42.

106. Syed M, Rog D, Parkes L, Shepherd GL. Patient expectations and experiences of multiple sclerosis interferon β-1a treatment: a longitudinal, observational study in routine UK clinical practice. Patient Prefer Adherence. 2014;8:247–55.

107. Thach AV, Brown CM, Herrera V, Sasane R, Barner JC, Ford KC, Lawson KA. Associations between treatment satisfaction, medication beliefs, and adherence to disease-modifying therapies in patients with multiple sclerosis. Int J MS Care. 2018;20(6):251–9.

108. Thakur K, Manuel L, Tomlinson M. Autoinjectors for administration of interferon beta-1b in multiple sclerosis: patient preferences and the ExtaviPro™ 30G and Betacomfort (®) devices. Pragmat Obs Res. 2013;4:19–26.

109. Tourbah A, Papeix C, Tourniaire P, Rerat K, Meite M, Durand B, Lamy F, Chouette I, Mekies C, ESGILE Study Group. Reasons for switching to fingolimod in patients relapsing-remitting multiple sclerosis in France: the ESGILE study. Mult Scler Relat Disord. 2020;46:102433.

110. Tur C, Tintoré M, Vidal-Jordana Á, Bichuetti D, Nieto González P, Arévalo MJ, Arrambide G, Anglada E, Galán I, Castilló J, Nos C, Río J, Martín MI, Comabella M, Sastre-Garriga J, Montalban X. Risk acceptance in multiple sclerosis patients on natalizumab treatment. PLoS One. 2013;8(12):e82796.

111. Turčáni P, Mašková J, Húska J. Real-world treatment patterns of disease modifying therapy (dmt) for patients with relapse-remitting multiple sclerosis and patient satisfaction with therapy: results of the non-interventional SKARLET study in Slovakia. Patient Prefer Adherence. 2020;14:1129–35.

112. Utz KS, Hoog J, Wentrup A, Berg S, Lämmer A, Jainsch B, Waschbisch A, Lee DH, Linker RA, Schenk T. Patient preferences for disease-modifying drugs in multiple sclerosis therapy: a choice-based conjoint analysis. Ther Adv Neurol Disord. 2014;7(6):263–75.

113. Visser LA, Louapre C, Uyl-de Groot CA, Redekop WK. Patient needs and preferences in relapsing-remitting multiple sclerosis: a systematic review. Mult Scler Relat Disord. 2020;39:101929.

114. Visser LH, van der Zande A. Reasons patients give to use or not to use immunomodulating agents for multiple sclerosis. Eur J Neurol. 2011;18(11):1343–9.

115. Volpicelli Leonard K, Robertson C, Bhowmick A, Herbert LB. Perceived treatment satisfaction and effectiveness facilitators among patients with chronic health conditions: a self-reported survey. Interact J Med Res. 2020;9(1):e13029.

116. Wicks P, Brandes D, Park J, Liakhovitski D, Koudinova T, Sasane R. Preferred features of oral treatments and predictors of non-adherence: two web-based choice experiments in multiple sclerosis patients. Interact J Med Res. 2015;4(1):e6.

117. Wilkie DD, Solari A, Nicholas R. Initiating disease-modifying treatments in multiple sclerosis: measuring the decision process using decisional conflict and decisional regret scales. Mult Scler J Exp Transl Clin. 2019;5(1):2055217319833006.

118. Wilson L, Loucks A, Bui C, Gipson G, Zhong L, Schwartzburg A, Crabtree E, Goodin D, Waubant E, McCulloch C. Patient centered decision making: use of conjoint analysis to determine risk-benefit trade-offs for preference sensitive treatment choices. J Neurol Sci. 2014;344(1–2):80–87.

119. Wilson LS, Loucks A, Gipson G, Zhong L, Bui C, Miller E, Owen M, Pelletier D, Goodin D, Waubant E, McCulloch C. Patient preferences for attributes of multiple sclerosis disease-modifying therapies: development and results of a ratings-based conjoint analysis. Int J MS Care. 2015;17(2):74–82.

120. Zimmer A, Bläuer C, Coslovsky M, Kappos L, Derfuss T. Optimizing treatment initiation: effects of a patient education program about fingolimod treatment on knowledge, self-efficacy and patient satisfaction. Mult Scler Relat Disord. 2015;4(5):444–50.

Hiba El Masri, 1 h.elmasri@uqconnect.edu.au, PhD Candidate, School of Pharmacy, The University of Queensland, Woolloongabba, QLD, Australia

Treasure M. McGuire, 2 t.mcguire@uq.edu.au, Faculty of Health Sciences and Medicine, Bond University, Robina, QLD, Australia, Mater Pharmacy, Mater Health, Raymond Tce, South Brisbane, QLD, Australia

Christine Dalais, 3 c.dalais@library.uq.edu.au, University Library, The University of Queensland, Brisbane, QLD, Australia

Mieke van Driel, 4 m.vandriel@uq.edu.au, Primary Care Clinical Unit, Faculty of Medicine, The University of Queensland, Brisbane, QLD, Australia

Helen Benham, 5 h.benham@uq.edu.au, Department of Rheumatology, Princess Alexandra Hospital, Brisbane, QLD, Australia

Samantha A. Hollingworth, 6 s.hollingworth@uq.edu.au, School of Pharmacy, The University of Queensland, Brisbane, QLD, Australia

Copyright © 2022 Hiba El Masri, Treasure M. McGuire, Christine Dalais, Mieke van Driel, Helen Benham, Samantha A. Hollingworth

This work is licensed under a Creative Commons Attribution 4.0 International License.

Journal of the Medical Library Association, VOLUME 110, NUMBER 2, April 2022