Surveys and Studies

Characteristics of personal health information management groups: findings from an online survey using Amazon’s mTurk

Sujin Kim, PhD, Jeffrey T. Huber, PhD

doi: http://dx.doi.org/10.5195/jmla.2017.312

Received February 2016: Accepted March 2017

ABSTRACT

Objective:

The study characterized three groups with different levels of familiarity with personal health information management (PHIM) in terms of their demographics, health knowledge, technological competency, and information sources and barriers. In addition, the authors examined differences among PHIM groups in subjective self-ratings and objective test scores for health literacy.

Methods:

A total of 202 survey participants were recruited using Amazon’s Mechanical Turk (mTurk) service, a crowdsourcing Internet service. Using K-means clustering, three groups with differing levels of familiarity with PHIM were formed: Advanced, Intermediate, and Basic.

Results:

The Advanced group was the youngest, and the Basic group contained the highest proportion of males, whereas the Intermediate group was the oldest and contained the fewest males. The Advanced group was significantly more likely to engage in provider- or hospital-initiated PHIM activities such as emailing with providers, viewing test results online, and receiving summaries of hospital visits via email or websites than the other groups. The Basic group had significantly lower information management skills and Internet use than the other groups. Advanced and Basic groups reported significant differences in several information barriers. While the Advanced group self-reported the highest general literacy, they scored lowest on an objective health literacy test.

Conclusions:

For effective personal health records management, it is critical to understand individual differences in PHIM using a comprehensive measure designed to assess personal health records–specific activities. Because they are trained to perform an array of information management activities, medical librarians or patient educators are well positioned to promote the effective use of personal health records by health consumers.

INTRODUCTION

Personal health information management (PHIM) refers to both the practices and activities that individuals perform to collect, organize, find or refind, use, and share personal health information needed to fulfill health-related tasks and roles [1, 2]. These activities involve an entire spectrum of information management activities that have been conventionally performed by well-trained, experienced information professionals, such as librarians or archivists, for large collections and diverse patron groups. With the advance of consumer health technology and the patient empowerment movement, health consumers are now being asked to perform these challenging activities despite their inexperience.

Making the situation more difficult, PHIM is a special case of personal information management that is associated with multiple interactions among varying users (e.g., patients, providers, insurance companies); complex health information and systems (e.g., labs, medications, insurance); and advanced health information technology tools (e.g., personal health records, personal health devices) [3–5].

Various consumer health technologies claim to facilitate consumers’ information management, but in reality, these technologies are still in their infancy stage [6, 7]. For example, personal health records (PHRs), as hubs of health care–related documents, are thought to be a promising way to promote patients’ engagement in their health care. Whereas PHIM refers to the activities and processes that patients use to manage their health information, PHRs represent a web technology solution to PHIM. Despite the growing PHR adoption rate in major hospitals, not all features are available or utilized. Patients’ favored features of PHRs include scheduling appointments, messaging securely with health care providers, and requesting prescription refills [8–10]. Underutilized features include viewing laboratory test results, updating health conditions, reading educational materials, and synchronizing external health devices and cell phone applications for monitoring health-related activities [11].

Unlike professionally controlled medical records or health information, the successful use of patient-centered PHRs is closely related to an individual’s level of health literacy. Lester and colleagues found that patients’ understanding of medical records and legal liability are barriers to successful PHIM and PHR use [12]. Previous studies also reported that demographic characteristics of individuals with limited health literacy—including being elderly, chronically ill, an ethnic minority, or male or having a low socioeconomic status or education level—are somewhat related to poor PHIM practices [13, 14]. However, these results are limited and inconclusive due to small sample sizes, restricting their generalizability and ability to accurately characterize individual variations in PHIM.

Over the past years, literacy scales have been developed to assess individuals’ health literacy levels, but none have focused on health literacy in the context of PHIM. Conventional health literacy assessments, such as the Rapid Estimate of Adult Literacy in Medicine and the Test of Functional Health Literacy in Adults, do not adequately measure PHIM skills because they do not assess technology skills or diverse information management activities [15]. Moreover, these conventional scales are mostly based on self-reported surveys that ask how much respondents know or are aware of given information or technologies. Therefore, their relationship to objective measurements, such as direct test scores, is not known. Most importantly, conventional health literacy scales do not distinguish between different aspects of health literacy or highlight areas for further improvement.

In early electronic health record (EHR) system design and adoption, medical librarians actively engaged in providing custom links to the medical literature to meet individual patient needs. Similar to EHR systems, librarians in large academic hospitals noted that PHRs could be used as a new vehicle for delivering quality health information and could be a focus of library instruction [16]. Indeed, a role of librarians in teaching best practices in PHIM is critical, as Tang and colleagues identified lack of knowledge of PHRs as a major barrier to successful PHIM [11].

Although the body of literature on health literacy and PHR systems has grown considerably over the last decade [11], little research has been conducted to characterize different levels PHIM activities and their relationship to health literacy [17]. Therefore, the main objective of this study was to characterize different PHIM groups in terms of their demographics, health knowledge, technology competency, and information sources and barriers.

METHODS

Participants

The authors recruited participants using a relatively low incentive via Amazon’s Mechanical Turk (mTurk) service, a crowdsourcing Internet service in which individuals participate in surveys or subject-tagging activities that require human intelligence. Despite some concerns about recruiting subjects using mTurk or other crowd-sourcing services [18], previous studies support the use of the mTurk service as a valid survey research method that produces results similar to other reliable recruitment methods [19–23]. Using the mTurk service, we assumed that participants were US residents (based on ownership of a US bank account), had at least a 95% task approval rate for their previous assignments, and were able to complete the survey in English. Non-US residents were excluded because our measures might not be valid for non-English-speaking, non-US participant samples. The initial survey participants received a $1.50 incentive. These respondents were then asked to participate in an additional test for another $1.50 incentive.

Measures

We first administered a PHIM survey composed of questions about demographics (e.g., age, gender, ethnicity, education, and income), health knowledge and technology competency, utilization of information sources, information barriers (e.g., cost, knowledge, language), and self-reported health literacy. In addition, 9 items measuring the participant’s familiarity with PHIM activities were included: (1) making appointments with a provider by email or on a website, (2) seeing a provider use a computer or handheld device to look up test results or other information, (3) emailing a provider’s office and receiving an answer to a medical question, (4) looking for test results on a website supplied by a provider, (5) receiving a summary of hospital visits by email or on a website, (6) reading hospital websites about health-related information, (7) requesting copies of medical records, (8) searching websites to find answers to health-related questions, and (9) posting or answering health-related questions on websites.

The overall reliability coefficient (Cronbach’s alpha) for this set of 9 PHIM activity–related questions was 0.849, which indicates a high level of internal consistency in this study sample. A total of 202 responses were included for analysis after responses that were from duplicate Internet protocol (IP) addresses and/or were missing answers to more than 25% of the questions were excluded.

For those who completed the initial PHIM survey, we then administered a Research Triangle Initiative (RTI) health literacy test [24, 25], which is composed of 25 items grouped into 5 subscales for different health literacy areas: (1) identifying and understanding health-related text (i.e., print-prose); (2) interpreting information and/or data in the form of tables, charts, pictures, symbols, maps, and videos (i.e., print-document); (3) completing computations (i.e., print-quantitative); (4) making inferences based on the information presented or applying information to a specific scenario (i.e., oral); and (5) utilizing the Internet/computer to obtain health information (i.e., Internet). Although the RTI test was not designed to measure PHIM literacy specifically, it measures multiple aspects of health literacy. A total of 139 respondents (68.8% of the initial survey sample) participated in the RTI test.

Except for demographic data on age, gender, and ethnicity, study variables were measured using Likert scales ranging from 1 (“strongly disagree” or “unsatisfactory”) to 7 (“strongly agree” or “highly satisfactory”). For education and income variables, we used numerical scales representing 7 education levels: 1: middle school graduate (or equivalent), 2: high school graduate (or equivalent), 3: some college (1–4 years, no degree), 4: associate’s degree, 5: bachelor’s degree (BA, BS, etc.), 6: master’s degree (MA, MS, etc.), and 7: professional or doctorate degree (MD, JD, PhD, etc.); and 7 income levels: 1: Less than $9,999, 2: $10,000–$24,999, 3: $25,000–$49,999, 4: $50,000–$69,999, 5: $70,000–$99,999, 6: $125,000–$149,999, and 7: more than $150,000.

Data analysis

The K-means clustering technique was used to identify three groups of individuals with different levels of the nine PHIM activities. Clustering began with the construction of initial PHIM group centers to assign individuals to a predefined number of clusters based on the distance of the nine PHIM activities from each of the cluster centers. The locations of cluster centers were updated based on the mean values of cases in each cluster until any reassignment of cases made the clusters more internally variable. We predefined three cluster memberships representing Advanced, Intermediate, and Basic levels of PHIM activities based on previous health literacy assessments [26, 27].

After individuals were assigned to one of the three groups via clustering, we tested for differences among the three groups using analysis of variance (ANOVA) and Tukey post-hoc analyses. Explanatory variables for the group differences included demographics (e.g., age, gender, ethnicity, education, and income); health knowledge (e.g., health care experience, content knowledge, and health care familiarity); technology competency (e.g., general computing, information management, and Internet use); utilization of information sources (e.g., health care professionals, family, friends, insurance, pharmacy, websites, schools, libraries, and government); and information barriers (e.g., financial issues, knowledge, language, human resources, culture, and superfluous information). We also performed Pearson’s correlations between RTI test scores and self-reported literacy ratings across the three PHIM groups.

RESULTS

Tables 1, 2, and 3 provide complete lists of and details about variables.

Table 1 Demographic characteristics and personal health information management (PHIM) activities

Table 2 Health knowledge and technology competency

Table 3 Health information sources and barriers

Demographic characteristics and personal health information management (PHIM) activities

Three groups of individuals were formed based on their familiarity with PHIM activities: Advanced (n=68, 34%), Intermediate (n=79, 39.5%), and Basic (n=55, 26.5%) (Table 1). There were significant differences between groups in overall PHIM activity score as well as all individual PHIM activity scores as determined by one-way ANOVAs. In general, post-hoc tests showed that the Advanced group was more likely to engage in provider- or hospital-initiated PHIM activities such as emailing with providers, viewing test results online, and receiving summaries of hospital visits via email or website than the Intermediate and Basic groups (p<0.05). However, there were no significant differences between the Intermediate and Basic groups in these activities (p>0.05). Also, Advanced and Intermediate groups were more likely to engage in individual-initiated activities such as searching websites for health information and posting or answering health-related questions on websites than the Basic group (p<0.05), whereas there were no significant differences between Advanced and Intermediate groups (p>0.05).

Among the demographic variables that were analyzed, age and gender significantly distinguished the three groups, whereas race, education, and income did not. The Advanced group was the youngest and the Basic group contained the highest proportion of males, whereas the Intermediate group was the oldest and contained the lowest proportion of males.

Health knowledge and technology competency

We next examined differences among the 3 PHIM groups in their health knowledge and technology competency (Table 2). We found significant differences among groups in all 3 health knowledge domains as determined by one-way ANOVA. In general, post-hoc tests showed that the Intermediate group had significantly higher health care experience and content knowledge than the Advanced and Basic groups (p<0.05). We found no significant difference among the 3 groups in general computing (p>0.05), but the difference among groups in information management and Internet use was statistically significant (p<0.05). Post-hoc tests showed that the Basic group had significantly lower information management skills and Internet use than the Advanced and Intermediate groups (p<0.05).

Information sources and barriers

We also examined differences among the 3 PHIM groups in their information sources and barriers (Table 3). We found significant differences among groups in the total information source score and most individual information sources as determined by one-way ANOVAs. In general, post-hoc tests revealed significant differences in individual information sources between Advanced and Basic groups and between Intermediate and Basic groups (p<0.05), whereas there were no differences between Advanced and Intermediate groups (p>0.05). Information sources such as websites or health care professionals were most frequently used, whereas institution- or organization-based information sources—such as those from governments, schools, work, libraries, or patient education centers—were less likely to be used.

We also found significant differences among groups in total information barrier score and most individual information barriers as determined by one-way ANOVAs. Post-hoc tests revealed significant differences between Advanced and Basic groups in four issues: “I do not have sufficient insurance coverage”; “I do not have any people with whom to discuss my health issues”; “I have a low budget for keeping my health information”; and “My providers do not offer electronic copies of my health records” (p<0.05).

Self-reported general literacy and objectively measured health literacy

For analysis of health literacy, we included 152 survey respondents (75.25%) out of 202 total survey respondents who answered and took the 11 general literacy questions and the RTI literacy test (Table 4). Interestingly, the Advanced group self-reported the highest general literacy but scored the lowest on an objective health literacy test. We found a significant difference among groups in self-reported aspects of general literacy as determined by one-way ANOVAs. Post-hoc tests showed that the Basic group self-reported lower overall general literacy than the Intermediate or Advanced groups (p<0.05), whereas there was no difference between Intermediate and Advanced groups (p>0.05). Concerning individual self-reported general literacy items, post-hoc tests generally showed significant differences between the Advanced and Basic groups (p<0.05).

Table 4 Self-reported and Research Triangle Initiative (RTI) health literacy test scores

The RTI health literacy test was administered to objectively measure health literacy. We found a significant difference among groups in overall RTI test score, with post-hoc tests showing that the Intermediate group scored higher than the Basic and Advanced groups (p<0.05). Considering the 5 subscales, post-hoc tests revealed that the Intermediate group scored higher than the Advanced group on the print-document subscale (p<0.05), whereas there were no group differences for the other subscale items. We then computed correlation coefficients between self-reported general literacy and RTI subscale scores. We found only some weak correlations between self-reported general literacy and objective PHIM literacy (Table 5).

Table 5 Correlations between self-rated and RTI health literacy test subscale scores

DISCUSSION

In this study, we used the K-means clustering technique to form three distinct PHIM groups—Advanced, Intermediate, and Basic—based on participants’ familiarity with nine distinct PHIM activities, and we characterized the groups in terms of their demographics, health knowledge, technology competency, and information sources and barriers. Furthermore, we examined differences among PHIM groups in self-rated and objective test-based health literacy.

We formed three PHIM groups based on their familiarity with PHIM-related activities. The formation of three groups is a common approach when using health literacy instruments to assess an individual’s capability in managing health-related activities [26, 27]. We confirmed that the three PHIM groups as formed by K-means clustering showed significant differences in their PHIM-related activities. Interestingly, participants in the Advanced group gave high ratings for institution-supported PHIM experiences, whereas those in the Intermediate and Basic groups gave low ratings for these activities (i.e., emailing with providers, accessing test results online, making appointments online). These activities are only possible through provider-initiated PHR applications, or so-called tethered PHR applications, which are connected to hospitals’ EHR systems.

The 2009 US Health Information Technology for Economic and Clinical Health Act created a meaningful use incentive program to provide financial support to health care providers and health systems adopting EHR technologies, which has significantly increased PHR adoption [28]. With the meaningful use program, more hospitals provide access to patient portals containing comprehensive medical records. Thus, patients (or their caregivers) with hospital-supported PHR access will have more opportunities to improve their PHIM skills than those without access to such systems. Although not a direct implication of our study findings, our results could indirectly imply that untethered, consumer-driven patient portals are not sufficient unless hospitals (or physicians’ offices) input their records into these patient portals.

We found that the three PHIM groups showed distinct demographic profiles in terms of age and gender, whereas the other demographic characteristics were similar across groups. The demographic profiles associated with different levels of health literacy are not yet conclusive. In a recent comprehensive review, the Agency for Healthcare Research and Quality reports the existence of ethnicity-based health literacy disparities [274], yet we found no significant differences in ethnicity among different PHIM groups, perhaps due to the fact that our sample was overwhelmingly Caucasian. Therefore, health literacy disparities among distinct demographic groups should be further studied for stronger evidence of existing links.

We found that the Intermediate group was highly knowledgeable, the most experienced, and relatively familiar with health information content and health care services. This result was somewhat related to an earlier finding that the Intermediate group was the oldest and most educated [29]. Consistent with previous literature reporting that older adults are more likely to have health issues and more experience with managing chronic health conditions than younger adults [30], we found that the older adults in the Intermediate group showed the greatest knowledge, experience, and familiarity with various health activities and health care services. However, the Intermediate group did not exhibit the highest level of familiarity with PHIM activities. In other words, if someone is highly competent in health issues, this does not guarantee a high level of PHIM literacy.

Additionally, we assessed differences among the three PHIM groups in general computing and Internet activities. We found no significant differences among groups in general computing skills. However, the Intermediate group scored slightly higher than the other two groups on certain technology-related Internet activities, such as evaluating health resources found on the Internet; using a computer to check news, weather, or sports; using a computer for banking, paying bills, or shopping; and using a computer to search for information. Considering the differences in age between groups, our results are consistent with those of Pak and colleagues, who found that older people are more likely to adopt some forms of technology than younger people [31].

Not surprisingly, the reported information sources and barriers were consistent with health and technology knowledge-related ratings. In other words, individuals in the Advanced PHIM group reported more exposure to information sources and fewer information barriers. Interestingly, the most popular information sources across the three groups were websites, followed by health care professionals. However, information provided by libraries and patient education centers were not fully utilized by the study participants. Therefore, a new approach to expanding medical libraries or patient education centers is needed. For example, Huber and colleagues proposed expanding a patient navigator program by including library and information science professionals who are “tasked with the selection and dissemination of understandable, relevant, culturally appropriate information.” Likewise, medical librarians could work with PHR teams or patient navigator teams to help solicit relevant personal health documents from patients, perform as mediated searchers, and train users of PHR systems [32, 33].

The Advanced PHIM group gave the highest self-ratings for all eleven general literacy scales. However, the Advanced group scored lowest on the objective RTI health literacy test, whereas the Intermediate group scored the highest. This finding implies that individuals who are relatively experienced and more knowledgeable about health care have greater health literacy, whereas those who rate themselves highly in PHIM-related activities have lower health literacy. In other words, perhaps younger individuals have more confidence with technology-related tasks but lack health knowledge and experience. Among the five RTI subscales, most participants scored low in Internet searching and quantitative literacy. Likewise, in a previous health literacy study, numeracy was reported as most problematic and challenging compared with other literacy skills [29].

In addition, all three PHIM groups self-rated their Internet searching skills highly but scored lowest on the Internet subscale of the RTI test. Based on the mean subscale scores, most participants did not answer half of the Internet-related questions correctly. Considering the high reliance on Internet searching skills due to current Internet-based PHR applications, Internet literacy should be improved to meet competency requirements for the effective use of PHR applications. Also, Internet-related literacy should be included in PHR literacy scales and should be an important consideration with developing personalized PHR training programs.

Our study has some limitations. First, the recruitment of participants from a crowd-sourced Internet service might introduce sampling bias, especially as these heavy Internet users might not accurately reflect the general public and their level of health literacy. Second, although the RTI test objectively measures diverse aspects of health literacy, it does not contain PHIM-specific measurement items. Multiple aspects of health literacy items covering not only health knowledge, but also diverse information management activities should be considered in future PHIM literacy or PHR usability studies [34].

ACKNOWLEDGMENTS

The University of Kentucky Office of Project Management and Analytics & Technologies mainly supported the project. In addition, the project was partially supported by the National Center for Advancing Translational Sciences, UL1TR000117, and the National Center for Advancing Translational Sciences, UL1TR000117. The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institute of Health (NIH).

REFERENCES

1. Jones W. Personal information management. Ann Rev Inf Sci Technol. 2007;41(1):453–504.
  

2. Pratt W, Unruh K, Civan A, Skeels MM. Personal health information management. Commun ACM. 2006 Jan;49(1):51–5.
  

3. Jones W, Bruce H. A report on the NSF-sponsored workshop on personal information management, Seattle, WA; 2005 [Internet]. Jan 2007 [cited 15 Oct 2014]. <http://pim.ischool.washington.edu/final%20PIM%20report.pdf>.

4. Agarwal R, Khuntia J. Personal health information and the design of consumer health information technology: background report. (Prepared by Insight Policy Research under contract no. 290-07-10072-1) [Internet]. Agency for Healthcare Research and Quality publication no. 09-0075-EF. Rockville, MD: The Agency; 2009 [cited 10 Oct 2015]. <https://healthit.ahrq.gov/sites/default/files/docs/citation/09-0075-EF.pdf>.

5. Agarwal R, Anderson C, Zarate J, Ward C. If we offer it, will they accept? factors affecting patient use intentions of personal health records and secure messaging. J Med Internet Res. 2013 Feb;15(2):e43. DOI: http://dx.doi.org/10.2196/jmir.2243.
      

6. Fabozzi N. Market disruption imminent as hospitals and physicians aggressively adopt patient portal technology [Internet]. Frost & Sullivan; 2013 [cited 13 Oct 2014]. <http://www.frost.com/prod/servlet/press-release.pag?docid=285477570>.

7. Centers for Medicare & Medicaid Services. A record of progress on health information technology [Internet]. The Centers; 2013 [cited 15 Oct 2014]. <https://www.cms.gov/Newsroom/MediaReleaseDatabase/Fact-sheets/2013-Fact-sheets-items/2013-04-23.html>.

8. Nazi KM. Veterans’ voices: use of the American Customer Satisfaction Index (ACSI) survey to identify My HealtheVet personal health record users’ characteristics, needs, and preferences. J. Am Med Inform Assoc. 2010 Mar;17(2):203–11. DOI: http://dx.doi.org/10.1136/jamia.2009.000240.
      

9. Sarkar U, Lyles CR, Parker MM, Allen J, Nguyen R, Moffet HH, Schillinger D, Karter AJ. Use of the refill function through an online patient portal is associated with improved adherence to statins in an integrated health system. J Med Care. 2014 Mar;52(3):194–201. DOI: http://dx.doi.org/10.1097/MLR.0000000000000069.
  

10. Colorado State Health Exchange. Trend report: patient portal adoption on the rise [Internet]. The Exchange; 2013 [cited 2 Aug 2017].

11. Tang PC, Ash JS, Bat DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc. 2006 Mar–Apr;13(2):121–6. Epub 2005 Dec 15.
  

12. Lester M, Boateng S, Studeny J, Coustasse A. Personal health records: beneficial or burdensome for patients and healthcare providers? Perspectives Health Inf Manag. 2016 Spring;13:1–12.

13. Institute of Medicine. Health literacy: a prescription to end confusion. Washington, DC: National Academies Press; 2004. p. 31.

14. Grossman JM, Zayas-Cabán T, Kemper N. Information gap: can health insurer personal health records meet patients’ and physicians’ needs? Health Aff (Millwood). 2009 Mar/Apr;28(2):377–89. DOI: http://dx.doi.org/10.1377/hlthaff.28.2.377.
  

15. Altin SV, Finke I, Kautz-Freimuth S, Stock S. The evolution of health literacy assessment tools: a systematic review. BMC Public Health. 2014;14:1207. DOI: http://dx.doi.org/10.1186/1471-2458-14-1207.
      

16. Jones DA, Shipman JP, Plaut DA, Selden CR. Characteristics of personal health records: findings of the Medical Library Association/National Library of Medicine Joint Electronic Personal Health Record Task Force. J Med Libr Assoc. 2010 Jul;98(3):243–9. DOI: http://dx.doi.org/10.3163/1536-5050.98.3.013.
      

17. Lober WB, Zierler B, Herbaugh A, Shinstrom SE, Stolyar A, Kim EH, Kim Y. Barriers to the use of a personal health record by an elderly population. AMIA Annu Symp Proc. 2006:514–8.

18. Snow R, O’Connor B, Jurafsky D, Ng AY. Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing; Association for Computational Linguistics; Oct 2008; Honolulu, HI. p. 254–63.

19. Mason W, Suri W. Conducting behavioral research on Amazon’s Mechanical Turk. Beh Resav Methods. 2012 Mar;44(1):1–23.
  

20. Horton JJ, Rand DG, Zeckhauser RJ. The online laboratory: conducting experiments in a real labor market. Exp Econ. 2011 Sep;14(3):399–425.
  

21. Mims C. How Mechanical Turk is broken. MIT Technol Rev [Internet]. 2010 [cited 25 Oct 2014]. <https://www.technologyreview.com/s/416966/how-mechanical-turk-is-broken/>.

22. Rand DG. The promise of Mechanical Turk: how online labor markets can help theorists run behavioral experiments. J Theor Biol. 2012 Apr;299:172–9.
  

23. McCormack L, Bann C, Squiers L, Berkman ND, Squire C, Schillinger D, Ohene-Frempong J, Hibbard J. Measuring health literacy: a pilot study of a new skills-based instrument. J Health Commun. 2010;15(suppl 2):51–71. DOI: http://dx.doi.org/10.1080/10810730.2010.499987.
    

24. Squiers L, Peinado S, Berkman N, Boudewyns V, McCormack L. The health literacy skills framework. Health Commun. 2012;17(suppl 3):30–54. DOI: http://dx.doi.org/10.1080/10810730.2012.713442.
  

25. Davis TC, Long SW, Jackson RH. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med. 1993:25(6):391–5.
  

26. Bostock S, Steptoe A. Association between low functional health literacy and mortality in older adults: longitudinal cohort study. BMJ. 2012 Mar 15;344:e1602.
      

27. Segall N, Saville JG, L’Engle P, Carlson B, Wright MC, Schulman K, Tcheng JE. Usability evaluation of a personal health record. AMIA Annu Symp Proc. 2011:1233–42.

28. International Organization for Standardization. ISO/DTR 14292 health informatics-personal health records: definition, scope, and context [Internet]. The Organization; 2012 [cited 13 Sep 2014]. <https://www.iso.org/obp/ui/#iso:std:iso:tr:14292:ed-1:v1:en>.

29. Kim S, Abner E. Predictors affecting personal health information management skills. Inform Health Soc Care. 2016;41(3):211–29.
    

30. Pak R, Price MM, Thatcher J. Age-sensitive design of online health information: comparative usability study. J Med Internet Res. 2009 Nov 16;11(4):e45.
      

31. Price MM, Pak R, Müller H. Older adults’ perceptions of usefulness of personal health records. Univ Access Inf Soc. 2013 Jun;12(2):191–204
  

32. Huber JT, Shapiro RM, Burke HJ, Palmer A. Enhancing the care navigation model: potential roles for health sciences librarians. J Med Libr Assoc. 2014 Jan;102(1):55–61. DOI: http://dx.doi.org/10.3163/1536-5050.102.1.011.
      

33. Huber JT, Shapiro RM, Gillaspy ML. Top down versus bottom up: the social construction of the health literacy movement. Libr. Q. 2012 Oct;82(4):429–51.
  

34. Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc. 2009 Sep–Oct;16(5):660–9.
      

(Return to Top)

Sujin Kim, PhD (corresponding author), sujinkim@uky.edu, Associate Professor, Division of Biomedical Informatics, College of Medicine and School of Information Science, College of Communication and Information, University of Kentucky, 725 Rose Street, 208H Multidisciplinary Science Building, Lexington, KY 40536-0082

Jeffrey T. Huber, PhD, jeffrey.huber@uky.edu, Professor, School of Information Science, 323 Little Library Building, University of Kentucky, Lexington, KY 40506-0224

Articles in this journal are licensed under a Creative Commons Attribution 4.0 International License.

This journal is published by the University Library System of the University of Pittsburgh as part of its D-Scribe Digital Publishing Program and is cosponsored by the University of Pittsburgh Press.

Journal of the Medical Library Association, VOLUME 105, NUMBER 4, October 2017