Abstract

Summary: Two hundred and twenty-four adults evaluated three preference-sensitive online decision aids related to their personal self-reported health status. Respondents were recruited in 2009, and user review was conducted online outside of a research or clinical setting. The majority of respondents had some college education, were white, and were middle aged. The three decision aids tested (statins [n = 70], aspirin [n = 97], and MRI [n = 57]) have been developed through a rigorous, iterative, expert medical review process; are evidence based; and are written in plain language. The results of general linear model repeated measures analyses were statistically significant for pre-post changes in user knowledge and for between-subject differences according to health issue. Post hoc comparisons for the results of one-way analysis of variance for eight dimensions of usability show that users of the MRI decision aid, compared with the other two user groups, felt that they had learned more, that the tool had helped clear up their feelings about the issue, and that they were more inclined to talk with their physician about their health issue.
Keywords: Decision aids, consumer, patient participation, shared decision making, statins, aspirin, MRI, cholesterol, low back pain.
Citation: Magee K, Cabinaw J, Reis J. Consumer responses to online decision aids for 3 preference-sensitive health problems. J Participat Med. 2013 Jul 31; 5:e31.
Published: July 31, 2013.
Competing Interests: Katy Magee was involved in the development of content for Healthwise Decision Points as an employee at the time the research was conducted, and is still employed by Healthwise in this capacity. Julie Cabinaw had a role in the development and sales of Healthwise products as a prior employee of the company. Janet Reis provided the analyses reported in this paper as part of her work as an independent consultant to Healthwise. None of the authors received additional compensation from Healthwise beyond their regular salary or fee for service.

Introduction

Decision aids for preference-sensitive conditions are regarded as important tools for helping patients arrive at decisions best suited to their individual needs and interests. People with preference-sensitive conditions face choices for treatment or management of their health with more than one reasonable option and with a given choice carrying potential positive and negative consequences. Studies on decision aids used with patients and their physicians have shown that patients have a higher level of satisfaction with the care received and decisions made. For some conditions, they opt for less invasive procedures following the use of a decision aid.[1] Most recently, decision aids have been embodied as part of the Patient Protection and Affordable Care Act in section 3506, Program to Facilitate Shared Decision Making and section 3022, Medicare Shared Savings Program.[2]

The internet is a good environment to provide and test decision aids. According to a 2012 survey, 72% of the U.S. adult population had used the internet to look for online health information during the previous year. Thirty-five percent of the respondents sought specific diagnostic information for themselves or someone else.[3] An earlier survey revealed that 66% look for information about specific diseases or medical problems and 56% search for information about particular medical treatments or procedures.[4]

Numerous investigations have been conducted on the quality of the health information available online. The information has been found to vary widely according to quality and accessibility.[5][6] Beyond data on frequency of use and general categories of information, however, data are scant on how consumers view the quality or usability of health information that they discover in their online searches outside the domains of a research study or clinical setting. This article summarizes user responses to decision aids for three preference-sensitive conditions. Internet users qualified for the study by virtue of their self-reported health conditions and signed up with a survey marketing company to participate in use of online health tools. Decision Points (DPs), the tools used in this study, were prepared by Healthwise, a 501(c)(3) nonprofit health education company.

The preference-sensitive conditions were selected, recognizing that patients and their providers face choices regarding the timing and course of treatment. On the “medical side” for each decision is a body of knowledge regarding short- and long-term health outcomes. On the patient preference side is a combination of quality-of-life questions linked to specific treatments or drugs, concerns about the invasiveness and costs of a given procedure, and time required for a procedure or drug to have a potentially positive effect.

Table 1 presents information provided in the Healthwise DP for statin use. This format is used for all of the 165 Healthwise DPs. The comparison of treatment options guides both juxtaposition of personal choices and the knowledge questions asked at the conclusion of the DP. The key points for this patient preference-sensitive condition (ie, whether or not to take statins to lower cholesterol) are the immediate pros and cons of taking a daily medication, and the longer term health trade-offs of controlling cholesterol through medication or lifestyle changes.

Table 1. Statin Decision Point: Compare Options

Statins for control of high cholesterol are known to have relatively few side effects and are effective for many people. For individuals averse to taking daily medications, lifestyle changes may suffice to control cholesterol levels. However, alterations in exercise, diet, and other lifestyle choices may not be enough to lower cholesterol levels. Timely use of statins may be important, particularly for patients with diabetes who are at increased risk for cardiovascular problems.[7][8][9][10][11] Results from a randomized trial with 98 diabetic patients found that a decision aid about statin drugs resulted in immediate increased knowledge. It also resulted in significantly better adherence to statin drugs at 3 months post intervention for those receiving the decision aid (OR, 3.4; 95% CI, 1.5-7.5).[12]

The other two health topics tested in this study, aspirin for low back pain and MRI for low back pain, have no information yet available on consumer reactions to decision aids. Aspirin is known to reduce the chance of a heart attack or stroke, but it also increases the risk of internal bleeding. A healthy lifestyle is an alternative approach to reducing risk and thus is a choice other than taking aspirin daily or every other day.[13][14][15] MRIs are not accepted as a standard test for determining the cause of low back pain, and most people with this type of pain recover over time. MRIs are not only expensive but can lead to unnecessary testing.[16][17]

Methods

Development of Decision Aids

Healthwise DPs cover 165 topics. Each DP goes through the following product development cycle. The need for a new DP is determined by an expert medical team composed of two to four licensed physicians and content specialists. Factors contributing to the designation of a DP topic include the prevalence of the decision, degree to which the issue is preference-sensitive because of the equivocal nature of the evidence for treatment, the range of negative and positive outcomes related to a specific condition, and the amount of control the patient has in determining the quality of the outcome for his or her health for that specific condition.

Once a health issue has been chosen for a DP, a medical writer is assigned to develop a work plan, which takes 2 to 4 weeks to complete. The writer has expertise in writing in plain language at the sixth-grade level. All writers are guided by a standard protocol for design and placement of content, construction of the interactive elements of a DP, and creation of knowledge questions used to assess the user’s understanding of the key issues for the health topic covered by the DP. Content is focused on three or four key ideas distilled from the pertinent medical literature and organized around a balance of what a physician would want his or her patient to understand and what matters most to the patient. Videos, pictures, or other visual illustrations are directly embedded and therefore immediately accessible to the user.

A draft work plan is shared with the expert medical team and reviewed for content and structure. The draft DP goes through several rounds of review with editors, the medical team, and an outside specialist.

Each DP has six elements. Element 1 (Get the Facts) contains a distillation of pertinent medical information with an expandable Frequently Asked Questions pull-down menu for users interested in additional information. Elements 2 (Compare Options), 3 (Your Feelings), and 4 (Your Decision) address the key dimensions of shared decision making. The “Your Feelings” element gives the user an interactive slider tool to rate the importance of three or four reasons a patient or consumer might pursue a certain course of action for the health condition in question versus not pursuing that course. These preference questions are arrived at through review of the pertinent literature on adherence and side effects, clinical experience, and pilot testing with users. Element 4 (Your Decision) asks the user to assess the direction in which he or she is leaning by using an interactive slider tool. Element 5 (Quiz Yourself) queries the user on three questions key to the decision being made and level of confidence in the decision on a 5-point scale. Each of these “knowledge questions” receives a correct (coded 2) versus incorrect rating (coded 1) or an “I am not sure” (coded 0). These knowledge questions were asked before and after participants used the decision aid in the study reported here. The sixth and final element of the DP is a printable, one-page summary of the user’s responses to questions about preferences.

All Healthwise DPs meet the International Patient Decision Aids Standards.[18] The format of the DPs has gone through iterative usability and functionality testing; all DPs are based on the Healthwise Knowledgebase, which is updated quarterly by impartial experts and are URAC-accredited (Utilization Review Accreditation Commission) .

Participant Recruitment

Participants were recruited through Zoomerang, a commercial, web-based survey service (now combined with Survey Monkey). Study participants were recruited for a 2-week block, beginning in mid-August 2009 for the statins and aspirin decision aids and beginning in late September 2009 for the MRI decision aid. After being designated as eligible for participation, participants were introduced to an online session with a statement about the importance of honest and anonymous feedback and that the average length of time was 20 minutes to complete the survey.

All participants agreed to a standard statement about personal privacy and use of their responses for research purposes (Zoomerang/Survey Monkey privacy policy, available at http://www.surveymonkey.com/mp/policy/privacy-policy/). Consent was gained a second time with users responding to the question “Do you agree to the uses of your survey responses as noted above?”

As part of the sign-up protocol, potential participants were screened by Zoomerang for background characteristics (e.g. minimum age, to assure they were not employed as a health professional and health status via self-report of diagnosis). The initial pool of users was then sent to a Healthwise website where another validation of health and and employment status was completed. Completed survey responses were checked for time spent on the website and for patterns of responses. User responses lasting less than 3 minutes and those showing a pattern of response bias were eliminated from the data file. For the current study, 3% of users were eliminated prior to analysis.

Data Analysis

Questions about knowledge were analyzed with a mixed design with one “within-subjects” factor (pre-post) and one “between-group factor” (health condition). The IBM SPSS Statistics 21 general linear model with repeated measures was used for the analysis.[19] The multi-variate results for Pillai’s trace (V) and related statistics are reported.[20] Since there were only two repeated data points per participant, the question of sphericity did not apply.[21] A first step in the analysis was to test the contribution of four socio-demographic factors to pre-post changes in the knowledge questions. Gender and educational level were statistically significant (P = .01) for the second knowledge question.

Descriptors of the DPs were analyzed as dependent measures with a one-way analysis of variance with health condition as a three-level factor. Post hoc comparisons for significant “between-subjects” main effects were analyzed with the Bonferroni test.[22] As with the knowledge questions, the four socio-demographic factors were first analyzed as potential predictors of differences on the descriptive variables. Of these factors, race/ethnicity was significant (P ≤ .05) for five of the eight descriptors. The last set of questions on user assessment of the DP was analyzed with Pearson’s chi-square test for the association between health condition and these questions.

Results

Table 2 summarizes the socio-demographic and health characteristics of the respondents according to the three health conditions. Across the three health issues, the majority of respondents had some college education and were white. Respondents to the MRI DP were younger (69% were 44 years of age or younger) than the respondents in the other two groups, with users of the statin and aspirin decisions aids skewed toward 45 years of age and older. Respondents to the statin and aspirin decision aids were approximately balanced for gender. More women than men volunteered to review the decision aid for MRI (78% female).

Table 2. Socio-demographic and Health Characteristics According to Health Condition

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Statistically significant “within-subject” effects (pre-post knowledge) and “between-subjects” effects (health issue) were found for the three general linear model repeated measures analyses (Table 3). In all cases, there were positive changes in average pre-post knowledge scores, although the change for the third knowledge question for statins was very small. This change was not significant in a separate paired t-test for the participants using the statin DP. Small effect sizes (.04 to .06) were observed.

Table 3. Preference Questions According to Health Condition and Pre-Post Changes in Knowledge

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Participants were also asked to rate the DP on a 5-point scale according to the following descriptors: taught something new, options understood, clear about benefits and side effects, confidence in decision, intend to discuss with physician, have enough support, tool helped clear up feelings, and tool was helpful. Lastly, participants were asked on a 2-point scale about intentions to comply with their physician’s advice, whether they found the DP complete and would be willing to use other tools, and whether they would recommend the tools to others.

As shown in Table 4, three statistically significant effects were found in the one-way analysis of variance tests on user ratings of DPs. (“I learned something I didn’t already know,” “I intend to discuss the health issue with my doctor,” and “This tool helped clear up my feelings.”) In each case of the statistically significant effect, users of the MRI DP rated this DP higher than users in the other two groups rated their respective DPs. This rating was significantly higher than the ratings for the aspirin, as determined by post hoc Bonferroni tests. In terms of patterns of user satisfaction, 22 (92%) of the 24 average ratings presented in Table 4 were rated as 3 (neutral) or greater, and 18 (75%) of these average ratings were 3.5 or greater.

Table 4. User Assessment of Decision Point According to Health Condition

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
No statistically significant association was found between the health issue and the users’ agreement that they would be interested in using similar tools (67% and more agreed) or that they would recommend use of their DP to friends or family members facing this decision (70% and more agreed). The association between health issue and intentions to do what the user’s doctor recommended was significant (P = .02) with 84% of MRI users agreeing with this statement compared with 60% of the statin and aspirin users (chi-square = 7.89).

Discussion

The results of this online evaluation of three decision aids for preference-sensitive health decisions provide baseline feedback on the direction and magnitude of the learning that occurred for each decision aid and on user assessment of the value of the decision aid in thinking through dimensions of choices to be made.

User feedback from this study can be put into two contexts. First, the results add to the understanding of how consumers evaluate the health information they find online. The little evidence available on this question speaks to the importance of ongoing and detailed assessments of how consumers react to what they find in their searches. The Pew Research Center reported that 50% of respondents in 2008 and 65% of respondents in 2011 believed that the medical advice or information found on the Internet was of no help, and 31% and 24%, respectively, found the information to be of minor or moderate help.[23]` In contrast, the overall positive pattern of responses from the study reported here speaks to the potential benefits of providing such tools online if consumers find the information credible.

Health care reform offers a second context for considering the implications of the user feedback obtained for the DPs. The aforementioned 3022 section of the Patient Protection and Affordable Care Act explicitly calls for use of shared decision making as part of the eligibility requirements for Accountable Care Organizations participating in the Medicare shared savings program. Decision support tools and shared decision making methods are options for operationalizing a necessary and measureable level of patient engagement.[2] A mandate to bolster patient engagement raises questions about the complexity of patients’ responses to a shared decision making experience and questions of where best to place interactive health information in the sequence of patient care. Each of these questions is discussed, in turn, after recognition of study limitations.

One limitation concerns the lack of a control group. This study used the participants as a baseline against themselves with no comparison to other individuals who had not used a DP. Inclusion of a control group in further testing of the DPs would strengthen the internal validity of the results obtained. Having comparative data on the knowledge questions would be particularly useful. Generalizations of the results of this study to actual consumer or patient behavior are not possible because of reliance on short-term changes in self-report about participants’ intentions to consult with their physician and abide by their physician’s direction. People’s intentions, however, are widely regarded as a proxy for behavior,[24] and the volunteers for this study were carefully screened before being given access to the online tools. Additionally, the feedback obtained was not in the context of a clinical trial or clinical setting. On the other hand, most decisions about health and medical treatments are ultimately made outside the immediate clinical setting. Finally, the effect sizes observed for the changes in knowledge were very small by conventional standards.[25]

The benefit of using an online decision aid is seen in the confirmation of the percentage of respondents who felt they learned something new and had their decision making clarified through use of the DP tool. The complexity of the process of decision making is seen in the different profiles of assuredness and intentions for each health issue. Across the three health topics covered in the DPs tested, there were significant differences in user perceptions of what was learned, intentions for interaction with the user’s physician, perceived support for the choice to be made, and feelings about that choice.

The potential of online health information and decision-making tools to spur conversations with health care professionals raises the question as to where and when to present such tools to patients in the course of their care. To date, most decision aids have been tested in clinical settings under the direction of physicians.[1] This level of clinical control speaks to the opportunities to create environments in which patients and their health care providers can engage in participatory dialogues. However, many clinic workflows preclude extension of the time available for patient-provider dialogues, and clinicians may require fairly extensive training in the use of decision aids.[26] The widespread use of the Internet to find health information and the likelihood of this pattern increasing in the future suggests that clinicians should consider when their patients might best benefit from referral to trusted Internet sites and when, in the course of care, physicians and their health care teams would be available to review patient feedback from such information sources.

Considerably more evidence is needed on how consumers are using the internet to inform themselves about health issues and to arrive at decisions regarding the health care they receive. Clinicians may find the task of procuring such information overwhelming both because of the time required and because of the potential complexities involved in interpreting consumer feedback as seen in the different profiles of consumer reaction for each of the three health issues presented through a decision aid. On the positive side, however, decision aid tools such as the Healthwise DPs have the benefit of being vetted for the accuracy of the information summarized and for the capability of electronically capturing user feedback to questions of knowledge, intentions to act, and personal preferences for care. Internet-supported health information may be one of the keys to meaningfully expanding patients’ participation in their care, especially given the impetus to expand consumer engagement under the national health care reform activities.

References

1. Stacey D, Bennett CL, Col, NF, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2011;Oct 5(10):CD011431. ↩
2. Patient Protection and Affordable Care Act of 2010, HR 3590, 111th Cong, 2nd Sess (2010). ↩
3. Fox S, Duggan M. Health Online 2013. Pew Research Center. Available at: http://pewinternet.org/Reports/2013/Health-online.aspx. Accessed January 20, 2013. ↩
4. Fox S. Health Topics 2011. Pew Research Center. Available at: http://www.pewinternet.org/Reports/2011/HealthTopics.aspx. Accessed January 20, 2013. ↩
5. Banzi R, Liberati A, Moschetti I, et al. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res. 2010;12(3):e26. ↩
6. Gibbons MC, Wilson RF, Samal L, et al. Impact of consumer health informatics applications. Evid Rep Technol Assess (Full Rep). 2009;Oct(188):1-546. ↩
7. Drugs for lipids. Treat Guidel Med Lett. 2011;9(103):13-20. ↩
8. Tonelli M, Lloyd A, Clement F, et al. Efficacy of statins for primary prevention in people at low cardiovascular risk: a meta-analysis. Can Med Assoc J. 2011;183(16):e1189-e1202. ↩
9. Alberton M, Wu P, Druyts E, Briel M, Mills EJ. Adverse events associated with individual statin treatments for cardiovascular disease: an indirect comparison meta-analysis. QJM. 2012;105(2):145-157. ↩
10. Bonovas S, Filioussi K, Tsavaris N, Sitaras NM. Statins and cancer risk: a literature-based meta-analysis and meta-regression analysis of 35 randomized controlled trials. J Clin Oncol. 2006;24(30):4808-4817. ↩
11. Mills EJ, Wu P, Chong G, et al. Efficacy and safety of statin treatment for cardiovascular disease: a network meta-analysis of 170,255 patients from 76 randomized trials. QJM. 2011;104(2):109-124. ↩
12. Weymiller AJ, Montori VM, Jones LA, et al. Helping patients with type 2 diabetes mellitus make treatment decisions: Statin choice randomized trial. Arch Intern Med. 2007;167:1076-1082. ↩
13. U.S. Preventive Services Task Force. Aspirin for the Prevention of Cardiovascular Disease, 2009. Available at: http://www.uspreventiveservicestaskforce.org/uspstf/uspsasmi.htm. Accessed January 20, 2013. ↩
14. Paikin JS, Eikelboom JW. Aspirin. Circulation. 2012;125(10):e439-e442. ↩
15. Taylor F, Ward K, Moore T, et al. Statins for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev. 2011;Jan 19(8):CD004816. ↩
16. Davis PC, Wippold FJ, II, Cornelius RS, et al. ACR Appropriateness Criteria: Low Back Pain, 2011. American College of Radiology. Available at: http://www.acr.org/~/media/ACR/Documents/AppCriteria/Diagnostic/LowBackPain.pdf. Accessed July 31, 2013. ↩
17. Chou R, Qaseem A, Owens DK, et al. Diagnostic imaging for low back pain: advice for high-value health care from the American College of Physicians. Ann Intern Med. 2011;154(3):181-189. ↩
18. Elwyn G, O’Connor A, Stacey D, et al. Developing a quality criteria framework for patient decision aids: Online international Delphi consensus process. BMJ. 2006;333 (7565):417. ↩
19. IBM SPSS Statistics 21 [Software package]. Version 21. Chicago: IBM Software Group; 2012. ↩
20. Field A. Discovering Statistics Using IBM SPSS Statistics. 4th ed. Los Angeles: Sage; 2013. ↩
21. Tabachnick BG, Fidell LS. Using Multivariate Statistics. 6th ed. Boston: Pearson Education; 2012. ↩
22. Maxwell SE. Pairwise multiple comparisons in repeated measures designs. J Ed Stat. 1980; 5(3):269-287. ↩
23. Fox S. The Social Life of Health Information, 2011. Pew Research Center. Available at: http://pewinternet.org/~/media/Files/Reports/2011/PIP_Social_Life_of_Health_Info.pdf. Accessed January 20, 2013. ↩
24. Fishbein M, Triandis H, Kanfer F, et al. Factors influencing behavior and behavior change. In: Baum A, Revenson T, Singer J, eds. Handbook of Health Psychology. 2nd ed. Mahwah, NJ: Erlbaum; 2001:3-16. ↩
25. Cohen J. Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press; 1977. ↩
26. Friedberg MW, Van Busum K, Wexler R, Bowen M, Schnieder EC. A demonstration of shared decision making in primary care highlights barriers to adoption and potential remedies. Health Aff. 2013;32(2):268-275. ↩

Copyright: © 2013 Katy Magee, Julie Cabinaw, and Janet Reis. Published here under license by The Journal of Participatory Medicine. Copyright for this article is retained by the authors, with first publication rights granted to the Journal of Participatory Medicine. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution 3.0 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.