Health websites on COVID-19: are they readable and credible enough to help public self-care?
DOI:
https://doi.org/10.5195/jmla.2021.1020Keywords:
Coronavirus, Comprehension, Anxiety, Cyberchondria, Patient Portals, Public Health Informatics, Medical Informatics, Health Education, Internet-Based InterventionAbstract
Objective: There are concerns about nonscientific and/or unclear information on the coronavirus disease 2019 (COVID-19) that is available on the Internet. Furthermore, people’s ability to understand health information varies and depends on their skills in reading and interpreting information. This study aims to evaluate the readability and creditability of websites with COVID-19-related information.
Methods: The search terms “coronavirus,” “COVID,” and “COVID-19” were input into Google. The websites of the first thirty results for each search term were evaluated in terms of their credibility and readability using the Health On the Net Foundation code of conduct (HONcode) and Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), Gunning Fog, and Flesch Reading Ease Score (FRE) scales, respectively.
Results: The readability of COVID-19-related health information on websites was suitable for high school graduates or college students and, thus, was far above the recommended readability level. Most websites that were examined (87.2%) had not been officially certified by HONcode. There was no significant difference in the readability scores of websites with and without HONcode certification.
Conclusion: These results suggest that organizations should improve the readability of their websites and provide information that more people can understand. This could lead to greater health literacy, less health anxiety, and the provision of better preventive information about the disease.References
World Health Organization (WHO). Coronavirus [Internet]. Geneva, Switzerland: The Organization [cited 2 Mar 2020]. <https://www.who.int/health-topics/coronavirus>.
World Health Organization. Coronavirus disease (COVID-19) pandemic [Internet]. Geneva, Switzerland: The Organization; 2020 [cited 2 Mar 2020]. <https://www.who.int/emergencies/diseases/novel-coronavirus-2019>.
World Health Organization (WHO). Coronavirus disease 2019 (COVID-19). situation report - 100 [Internet]. Geneva, Switzerland: The Organization; 29 Apr 2020 [cited 30 Apr 2020]. <https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200429-sitrep-100-covid-19.pdf?sfvrsn=bbfbf3d1_6>.
Heymann DL, Shindo N. COVID-19: what is next for public health? Lancet [Internet]. 2020 Feb 22;395(10224):542–5. DOI: http://dx.doi.org/ 10.1016/S0140-6736(20)30374-3.
Steckelberg JM. What is MERS-CoV, and what should I do? [Internet]. Mayo Foundation for Medical Education and Research; 2018 [cited 20 Dec 2019]. <http://www.mayoclinic.org/diseases-conditions/sars/expert-answers/what-is-mers-cov/faq-20094747>.
Choudhury SM, Arora T, Alebbi S, Ahmed L, Aden A, Omar O, Taheri S. How do Qataris source health information? PLoS One. 2016;11(11):e0166250. DOI: http://dx.doi.org/10.1371/journal.pone.0166250.
Beck F, Richard JB, Nguyen-Thanh V, Montagni I, Parizot I, Renahy E. Use of the Internet as a health information resource among French young adults: results from a nationally representative survey. J Med Internet Res. 2014 May 13;16(5):e128. DOI: http://dx.doi.org/10.2196/jmir.2934.
Simou E. Health information sources: trust and satisfaction. Int J Healthc. 2015;2(1). DOI: http://dx.doi.org/10.5430/ijh.v2n1p38.
Alduraywish SA, Altamimi LA, Aldhuwayhi RA, AlZamil LR, Alzeghayer LY, Alsaleh FS, Aldakheel FM, Tharkar S. Sources of health information and their impacts on medical knowledge perception among the Saudi Arabian population: cross-sectional study. J Med Internet Res. 2020 Mar 19;22(3):e14414. DOI: http://dx.doi.org/10.2196/14414.
Atkinson NL, Saperstein SL, Pleis J. Using the Internet for health-related activities: findings from a national probability sample. J Med Internet Res. 2009 Feb;11(1):e4. DOI: http://dx.doi.org/10.2196/jmir.1035.
Hamzehei R, Ansari M, Rahmatizadeh S, Valizadeh-Haghi S. Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease. Online J Public Health Inform. 2018 Dec 30;10(3):e221. DOI: http://dx.doi.org/10.5210/ojphi.v10i3.9544.
Ansari M, Hamzehei R, Valizadeh-Haghi S. Persian language health websites on Ebola disease: less credible than you think? J Egypt Public Health Assoc. 2020 Dec;95(1):2. DOI: http://dx.doi.org/10.1186/s42506-019-0027-4.
Ritchie L, Tornari C, Patel PM, Lakhani R. Glue ear: how good is the information on the World Wide Web? J Laryngol Otol. 2016 Feb;130(2):157–61. DOI: http://dx.doi.org/10.1017/S0022215115003230.
Rahmatizadeh S, Valizadeh-Haghi S. Evaluating the trustworthiness of consumer-oriented health websites on diabetes. Libr Philos Pract. 2018 Spring;(1786).
Rahmatizadeh S, Valizadeh-Haghi S, Kalavani A, Fakhimi N. Middle East respiratory syndrome on health information websites: how much credible they are? Libr Philos Pract. 2019 Sep 14;(2885).
Valizadeh-Haghi S, Rahmatizadeh S. Evaluation of the quality and accessibility of available websites on kidney transplantation. Urol J. 2018 Sep 26;15(5):261–5. DOI: http://dx.doi.org/10.22037/uj.v0i0.4252.
Eagle S. Diseases in a flash!: an interactive, flash-card approach. 14th ed. Philadelphia, PA: F.A. Davis; 2011. 796 p.
Sokic N. Hypochondria in the age of COVID-19 [Internet]. Healthing.ca; 2020 [cited 13 Mar 2020]. <https://www.healthing.ca/diseases-and-conditions/coronavirus/hypochondria-in-the-age-of-covid-19>.
Starcevic V. Cyberchondria: challenges of problematic online searches for health-related information. Psychother Psychosom [Internet]. 2017 May;86(3):129–33. DOI: http://dx.doi.org/10.1159/000465525.
Anderson J, Rainie L. The future of truth and misinformation online. Pew Research Center. 2017.
Boyer C. When the quality of health information matters: Health on the Net is the quality standard for information you can trust [Internet]. Health On the Net Foundation; 2013 [cited 19 Mar 2018]. <http://www.hon.ch/Global/pdf/TrustworthyOct2006.pdf>.
Valizadeh-Haghi S, Moghaddasi H, Rabiei R, Asadi F. Health websites visual structure: the necessity of developing a comprehensive design guideline. Arch Adv Biosci.2017 Sep 26;8(4):53–9. DOI: http://dx.doi.org/10.22037/jps.v8i4.18175.
Nyilasy G. Fake news in the age of COVID-19. Pursuit [Internet]. University of Melbourne [cited 12 Jun 2020]. <https://pursuit.unimelb.edu.au/articles/fake-news-in-the-age-of-covid-19>.
The Conversation. False information fuels fear during disease outbreaks: there is an antidote. The Conversation [Internet]. 9 Feb 2020 [cited 12 Jun 2020]. <https://theconversation.com/false-information-fuels-fear-during-disease-outbreaks-there-is-an-antidote-131402>.
Starcevic V, Baggio S, Berle D, Khazaal Y, Viswasam K. Cyberchondria and its relationships with related constructs: a network analysis. Psychiatr Q. 2019 Sep;90(3):491–505. DOI: http://dx.doi.org/10.1007/s11126-019-09640-5.
Mahase E. Coronavirus: COVID-19 has killed more people than SARS and MERS combined, despite lower case fatality rate. BMJ. 2020 Feb 18;368:m641. DOI: http://dx.doi.org/10.1136/bmj.m641.
Chasiotis A, Wedderhoff O, Rosman T, Mayer AK. Why do we want health information? the goals associated with health information seeking (GAINS) questionnaire. Psychol Heal. 2020 Mar;35(3):255–74. DOI: http:/dx.doi.org/10.1080/08870446.2019.1644336.
Khazaal Y, Chatton A, Cochand S, Hoch A, Khankarli MB, Khan R, Zullino DF. Internet use by patients with psychiatric disorders in search for general and medical informations. Psychiatr Q. 2008 Dec;79(4):301–9. DOI: http://dx.doi.org/10.1007/s11126-008-9083-1.
Calvo MG, Carreiras M. Selective influence of test anxiety on reading processes. Br J Psychol. 1993 Aug;84(pt. 3):375–88. DOI: http://dx.doi.org/10.1111/j.2044-8295.1993.tb02489.x.
Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, Kanouse DE, Muñoz JA, Puyol JA, Lara M, Watkins KE, Yang H, McGlynn EA. Health information on the Internet: accessibility, quality, and readability in English and Spanish. J Am Med Assoc. 2001 May 23;285(20):2612–21. DOI: http://dx.doi.org/10.1001/jama.285.20.2612.
National Work Group on Literacy and Health; Weiss BD; University of Texas Health Science Center. Communicating with patients who have limited literacy skills. J Fam Pract. 1998 Feb;46(2):168–76.
Albright J, de Guzman C, Acebo P, Paiva D, Faulkner M, Swanson J. Readability of patient education materials: implications for clinical practice. Appl Nurs Res. 1996 Aug;9(3):139–43. DOI: http://dx.doi.org/10.1016/s0897-1897(96)80254-0.
Weiss BD. Health literacy: a manual for clinicians [Internet]. Chicago, IL: American Medical Association Foundation; 2003 [cited 6 Nov 2020]. <http://lib.ncfh.org/pdfs/6617.pdf>.
Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Community Health. 2005 Jun;30(3):213–9. DOI: http://dx.doi.org/10.1007/s10900-004-1959-x.
Raj S, Sharma VL, Singh AJ, Goel S. Evaluation of quality and readability of health information websites identified through India’s major search engines. Adv Prev Med. 2016:1–6. DOI: http://dx.doi.org/10.1155/2016/4815285.
Fahimuddin FZ, Sidhu S, Agrawal A. Reading level of online patient education materials from major obstetrics and gynecology societies. Obstet Gynecol. 2019 May;133(5):987–93. DOI: http://dx.doi.org/10.1097/AOG.0000000000003214.
Eltorai A, Ghanian S, Adams C, Born C, Daniels A. Readability of patient education materials on the American Association for Surgery of Trauma website. Arch Trauma Res. 2014 Apr 30;3(1):e18161. DOI: http://dx.doi.org/10.5812/atr.18161.
Manchaiah V, Dockens AL, Flagge A, Bellon-Harn M, Azios JH, Kelly-Campbell RJ, Andersson G. Quality and readability of English-language Internet information for tinnitus. J Am Acad Audiol. 2019 Jan;30(1):31–40. DOI: http://dx.doi.org/10.3766/jaaa.17070.
Vives M, Young L, Sabharwal S. Readability of spine-related patient education materials from subspecialty organization and spine practitioner websites. Spine (Phila Pa 1976). 2009 Dec 1;34(25):2826–31. DOI: http://dx.doi.org/10.1097/BRS.0b013e3181b4bb0c.
Helitzer D, Hollis C, Cotner J, Oestreicher N. Health literacy demands of written health information materials: an assessment of cervical cancer prevention materials. Cancer Control. 2009 Jan;16(1):70–8. DOI: http://dx.doi.org/10.1177/107327480901600111.
Badarudeen S, Sabharwal S. Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America web sites. J Bone Joint Surg Am. 2008 Jan;90(1):199–204. DOI: http://dx.doi.org/10.2106/JBJS.G.00347.
Google Trends. Coronavirus [Internet]. Google; 2020 [cited 13 Mar 2020]. <https://trends.google.com/trends/explore?cat=45&date=today 1-m&q=%2Fm%2F01cpyy>.
StatCounter. Global stats: browser, OS, search engine including mobile usage share [Internet]. Search Engine Market Share Worldwide; 2020 [cited 5 Mar 2020]. <https://gs.statcounter.com/search-engine-market-share>.
eBiz. Top 15 best search engines [Internet]. eBiz MBA, eBusiness guide; Sep 2019 [cited 19 Oct 2019]. <http://www.ebizmba.com/articles/search-engines>.
iProspect. Blended search results study 2008 [Internet]. iProspect; 2008 [cited 10 Sep 2017]. <http://www.iprospect.com>.
Rayess H, Zuliani GF, Gupta A, Svider PF, Folbe AJ, Eloy JA, Carron MA. Critical analysis of the quality, readability, and technical aspects of online information provided for neck-lifts. JAMA Facial Plast Surg. 2017 Mar 1;19(2):115–20. DOI: http://dx.doi.org/10.1001/jamafacial.2016.1219.
Varela-Centelles P, Ledesma-Ludi Y, Seoane-Romero JM, Seoane J. Information about oral cancer on the Internet: our patients cannot understand it. Br J Oral Maxillofac Surg. 2015 Apr;53(4):393–5. DOI: http://dx.doi.org/10.1016/j.bjoms.2015.01.020.
Hadden K, Prince LY, Schnaekel A, Couch CG, Stephenson JM, Wyrick TO. Readability of patient education materials in hand surgery and health literacy best practices for improvement. J Hand Surg Am. 2016 Aug 1;41(8):825–32. DOI: http://dx.doi.org/10.1016/j.jhsa.2016.05.006.
Royal KD, Erdmann KM. Evaluating the readability levels of medical infographic materials for public consumption. J Vis Commun Med. 2018 Jul 3;41(3):99–102. DOI: http://dx.doi.org/10.1080/17453054.2018.1476059.
Alamoudi U, Hong P. Readability and quality assessment of websites related to microtia and aural atresia. Int J Pediatr Otorhinolaryngol. 2015 Feb;79(2):151–6. DOI: http://dx.doi.org/10.1016/j.ijporl.2014.11.027.
Brigo F, Otte WM, Igwe SC, Tezzon F, Nardone R. Clearly written, easily comprehended? the readability of websites providing information on epilepsy. Epilepsy Behav. 2015 Mar 1;44:35–9. DOI: http://dx.doi.org/10.1016/j.yebeh.2014.12.029.
Grewal P, Alagaratnam S. The quality and readability of colorectal cancer information on the Internet. Int J Surg. 2013;11(5):410–3. DOI: http://dx.doi.org/10.1016/j.ijsu.2013.03.006.
Chi E, Jabbour N, Aaronson NL. Quality and readability of websites for patient information on tonsillectomy and sleep apnea. Int J Pediatr Otorhinolaryngol. 2017 Jul;98:1–3. DOI: http://dx.doi.org/10.1016/j.ijporl.2017.04.031.
MedlinePlus. Choice rev online. National Library of Medicine; 2006 Aug 1;43(12):43Sup-0363.
Readability Formulas. Contact us [Internet]. My Byline Media [cited 19 Oct 2019]. <http://www.readabilityformulas.com/contactus.php>.
Maskell K, McDonald P, Paudyal P. Effectiveness of health education materials in general practice waiting rooms: a cross-sectional study. Br J Gen Pract. 2018 Dec;68(677):e869–76. DOI: http://dx.doi.org/10.3399/bjgp18X699773.
Cherrez-Ojeda I, Felix M, Mata VL, Vanegas E, Gavilanes AWD, Chedraui P, Simancas-Racines D, Calderon JC, Ortiz F, Blum G, Plua A, Gonzalez G, Moscoso G, Morquecho W. Preferences of ICT among patients with chronic kidney disease undergoing hemodialysis: an Ecuadorian cross-sectional study. Healthc Inform Res. 2018 Oct;24(4):292–9. DOI: http://dx.doi.org/10.4258/hir.2018.24.4.292.
Ng MK, Mont MA, Piuzzi NS. Analysis of readability, quality, and content of online information available for “stem cell” injections for knee osteoarthritis. J Arthroplasty. 2020 Mar;35(3):647–51.e2. DOI: http://dx.doi.org/10.1016/j.arth.2019.10.013.
Perruzza D, Jolliffe C, Butti A, McCaffrey C, Kung R, Gagnon L, Lee P. Quality and reliability of publicly accessible information on laser treatments for urinary incontinence: what is available to our patients? J Minim Invasive Gynecol. Nov–Dec 2020;27(7):1524–30. DOI: http://dx.doi.org/10.1016/j.jmig.2020.01.001.
Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: caveant lector et viewor—let the reader and viewer beware. JAMA. 1997 Apr 16;277(15):1244–5. DOI: http://dx.doi.org/10.1001/jama.1997.03540390074039.
Charnock D. The DISCERN handbook: quality criteria for consumer health information on treatment choices. Abingdon, Oxon, UK: Radcliffe Medical Press; 1998.
Health on the Net Foundation. The commitment to reliable health and medical information on the Internet [Internet]. The Foundation; 2017 [cited 10 Apr 2020]. <https://www.hon.ch/HONcode/Patients/Visitor/visitor.html>.
Health on the Net Foundation. HONcode toolbar [Internet]. The Foundation [cited 3 Feb 2020]. <https://www.hon.ch/en/tools.html#honcodeextension>.
Señor IC, Fernández-Alemán JL, Toval A. Are personal health records safe? a review of free web-accessible personal health record privacy policies. J Med Internet Res. 2012 Aug 23;14(4):e114. DOI: http://dx.doi.org/10.2196/jmir.1904.
Saraswat I, Abouassaly R, Dwyer P, Bolton DM, Lawrentschuk N. Female urinary incontinence health information quality on the Internet: a multilingual evaluation. Int Urogynecol J. 2016 Jan;27(1):69–76. DOI: http://dx.doi.org/10.1007/s00192-015-2742-5.
Khazaal Y, Fernandez S, Cochand S, Reboh I, Zullino D. Quality of web-based information on social phobia: a cross-sectional study. Depress Anxiety. 2008;25(5):461–5. DOI: http://dx.doi.org/10.1002/da.20381.
Grohol JM, Slimowicz J, Granda R. The quality of mental health information commonly searched for on the Internet. Cyberpsychology Behav Soc Netw. 2014 Apr;17(4):216–21. DOI: http://dx.doi.org/10.1089/cyber.2013.0258.
Quesenberry AC. Plain language for patient education. J Consum Health Internet. 2017 Apr 3;21(2):209–15. DOI: http://dx.doi.org/10.1080/15398285.2017.1311611.
Drake BF, Brown KM, Gehlert S, Wolf LE, Seo J, Perkins H, Goodman MS, Kaphingst KA. Development of plain language supplemental materials for the Biobank informed consent process. J Cancer Educ. 2017 Dec;32(4):836–44. DOI: http://dx.doi.org/10.1007/s13187-016-1029-y.
US Department of Health & Human Services. Writing for the web [Internet]. The Department; 2019 [cited 9 Nov 2019]. <https://www.usability.gov/how-to-and-tools/methods/writing-for-the-web.html>.
Martinez S. Consumer literacy and the readability of health education materials. Perspect Commun Disord Sci Cult Linguist Divers Popul. 2011;18(1):20–6. DOI: http://dx.doi.org/10.1044/cds18.1.20.
Castro-Sánchez E, Spanoudakis E, Holmes AH. Readability of Ebola information on websites of public health agencies, United States, United Kingdom, Canada, Australia, and Europe. Emerg Infect Dis. 2015 Jul;21(7):1217–9. DOI: http://dx.doi.org/10.3201/eid2107.141829.
Mcinnes N, Haglund BJAA. Readability of online health information: implications for health literacy. Inform Health Soc Care. 2011 Dec 18;36(4):173–89. DOI: http://dx.doi.org/10.3109/17538157.2010.542529.
van Deursen AJAM, van Dijk JAGM. Using the Internet: skill related problems in users’ online behavior. Interact Comput. 2009 Dec;21(5–6):393–402. DOI: http://dx.doi.org/10.1016/j.intcom.2009.06.005.
Zuchowski KA, Sanders AE. Readability and implications for health literacy: a look at online patient-oriented disease information by Alzheimer’s disease research centers. Alzheimer’s Dement. 2018 Jul 1;14(7):P1380. DOI: http://dx.doi.org/10.1016/j.jalz.2018.06.2011.
American Cancer Society. Cancer information on the Internet [Internet]. The Society; 2016 [cited 20 Feb 2018]. <http://www.cancer.org/cancer/cancer-basics/cancer-information-on-the-internet.html>.
Khazaal Y, Chatton A, Cochand S, Zullino D. Quality of web-based information on cocaine addiction. Patient Educ Couns. 2008 Aug;72(2):336–41. DOI: http://dx.doi.org/10.1016/j.pec.2008.03.002.
Ostry A, Young ML, Hughes M. The quality of nutritional information available on popular websites: a content analysis. Health Educ Res. 2008;23(4):648–55. DOI: http://dx.doi.org/10.1093/her/cym050.
Yi YJ, Stvilia B, Mon L. Cultural influences on seeking quality health information: an exploratory study of the Korean community. Libr Inf Sci Res. 2012 Jan;34(1):45–51. DOI: http://dx.doi.org/10.1016/j.lisr.2011.06.001.
Koo K, Yap RL. How readable is BPH treatment information on the Internet? assessing barriers to literacy in prostate health. Am J Mens Health. 2017 Mar;11(2):300–7. DOI: http://dx.doi.org/10.1177/1557988316680935.
