Enhancing In-Clinic Doctor-Patient Communication in Real Time through Adaptive Best-worst Conjoint (ABC) Analysis
|
If you are the presenter of this abstract (or if you cite this abstract in a talk or on a poster), please show the QR code in your slide or poster (QR code contains this URL). |
Abstract
Background: We develop a new tool to enhance clinical care by measuring individual patient treatment preferences in real time. The results are reported instantaneously and used to enhance the patient-doctor discussion of alternative treatments. The new tool we have developed, Adaptive Best-worst Conjoint (ABC) Analysis, improves upon traditional Choice-Based Conjoint Analysis (CBC) in terms of efficiency, speed and software requirements (ABC works in Excel). A multi-year clinical study of ABC versus Ratings Scale (RS) and Time Trade Off (TTO) is being conducted through the NIH and initial results show promise over RS and TTO, with both patients and doctors expressing appreciation for enhanced self-learning and communication quality.
Conjoint analysis measures treatment preferences by defining each treatment as a “bundle†of attribute levels, for example varying treatment characteristics and outcomes, and asking patients to choose one such bundle over another based on their individual priorities. Using a balanced, orthogonal experimental design, individual-level attribute preferences can be estimated based on the choices made by each patient. Each individual’s “utility faction†is reported and summarized graphically on a single sheet of paper, and is shared with both doctor and patient to enhance communication and learning.
In CBC, patients see four treatment options-at-a-time, and select the best option. CBC produces only three paired comparisons (1st Choice A > option B, 1st Choice A > option C, 1st Choice A > option D). ABC asks patients to choose not only the best of the four options, but also the worst, and therefore with the identical four options generates five of the six possible paired comparisons (A > B, A > C, A > D, B > D, and C > D; only B is not compared to C). So ABC is 66% more “efficient†than CBC even without adaptive questioning.
ABC’s inherent efficiency advantage over CBC is further enhanced through adaptive questioning based on transitivity of preference. That is, we assume that if treatment A is preferred to treatment B, and if B > E, then A is also > E, even though A and E may never be compared directly. Such transitivity of preference may resolve even more paired comparisons than direct questioning. For example, given the 16-full-profile conjoint stimuli in our experiment, there are 16 x 15 / 2 = 120 possible paired comparisons. The ABC method has required 11 to 17 tasks, with an average of 12 best-worst tasks, to estimate a utility function with 9 parameters. The 12 tasks each directly resolve 5 paired comparisons (though some redundancy is likely, so they may not all be unique pairs), so direct questioning resolves at most 60 pairs (5 x 12). The remaining 60+ paired comparisons are resolved through transitivity.
The adaptive element of ABC derives from choosing the four stimuli to show in the next best-worst task based on maximizing the number of unresolved pairs given all prior responses. The method seems to be particularly effective in clinic.
Objectives: We apply the ABC best-worst method to full-profile conjoint stimuli, and further improve questioning efficiency by employing adaptive questioning. To highlight the method and its benefits, we test the effectiveness of the approach in clinic for a prostate cancer treatment application. With this approach, a 9-parameter utility function is estimated in real time at the individual level with an average of 12 tasks (11-17 range) completed in 10 minutes (7-20 minute range).
The primary problems with current conjoint methods such as CBC include: (1) they require expensive proprietary software such as Sawtooth's CBC, (2) are not run in real time nor do they provide instantaneous utility estimation, and (3) they take too long per respondent to estimate individual utility (they are really designed for population estimates), e.g. 18-24+ choice tasks to estimate 9 parameters. ABC solves these problems by (1) running in Excel, (2) reporting immediately and (3) cutting the questioning burden by more than half.
Methods: The entire application has been built as an Excel spreadsheet file using macros. Respondents are first taught about attributes and levels, then primed with a sample task, then guided through 11-15 such choice tasks between four full profiles at a time. The full profiles can be text- or graphics-based. In the background, the Excel software tracks every possible paired comparison and records all of the direct comparisons as well as calculating all of the transitive ones. The paired comparisons are converted into scores for each full-profile card, and a utility function estimated based on these scores. A report is automatically generated at the end of the process so that the individual can learn about his or her preferences. The report serves as the basis of discussion for post-biopsy meeting between patient and medical professional.
Clinical settings impose many constraints on time, space and resources. ABC addresses these issues by being very fast (~ 10 minutes), easy-to-implement (uses simple PC running Excel), and automated (Excel-based PC macro “interviews†the patient so clinic personnel can be freed up).
Results: Presently (April 2012): The presentation will highlight results about:
• Internal consistency: How many of the paired comparisons are consistent with the estimated utility function? Initial results: 60%-70%
• Estimation: a comparison of regression versus other methods such as LINMAP; the need or lack thereof for population-level analysis: Similar performance, with ABC nearly matching the predictive accuracy of the state-of-the-art Hierarchical Bayesian approach that exploits population data and cannot be run in real time.
• Respondent reactions to this system: Favorable and very appreciative from both patient and doctor.
We will also address some of the method’s limitations:
– The lack of intensity of preference measurements, i.e. A is assumed to be preferred to B by the same amount as B is preferred to C
– The issue of estimating rank ordered data using multiple linear regression rather than more advanced estimation methods
Conclusions:
Adaptive Best-worst Conjoint (ABC) has two efficiency advantages over traditional choice-based conjoint:
(1) directly resolving more paired preference comparisons per choice task and
(2) indirectly resolving additional paired preference comparisons through transitivity of preference.
The total number of tasks required of each survey subject can be further reduced through adaptive questioning by taking into account all prior responses and focusing on unresolved preference pairs.
The ABC method is currently being evaluated for use in other medical contexts including co-morbidity, in the context of depression, and to help with the treatment of chronic urinary tract infections. The presentation will discuss the medical and clinical contexts for which ABC is most appropriate, and those for which it is not.
The method can be implemented in low cost software (e.g. Excel macros) and is able to generate virtually instantaneous feedback to enhance discussions between patients and healthcare professionals.
The primary benefits of ABC in the clinical setting over existing methods are:
(1) improved estimation of patient preferences and priorities
(2) real-time clinical usability
(3) enhanced self-learning for patients and improved communication between doctor and patient
(4) Enhanced patient satisfaction with ultimate treatment choice
Conjoint analysis measures treatment preferences by defining each treatment as a “bundle†of attribute levels, for example varying treatment characteristics and outcomes, and asking patients to choose one such bundle over another based on their individual priorities. Using a balanced, orthogonal experimental design, individual-level attribute preferences can be estimated based on the choices made by each patient. Each individual’s “utility faction†is reported and summarized graphically on a single sheet of paper, and is shared with both doctor and patient to enhance communication and learning.
In CBC, patients see four treatment options-at-a-time, and select the best option. CBC produces only three paired comparisons (1st Choice A > option B, 1st Choice A > option C, 1st Choice A > option D). ABC asks patients to choose not only the best of the four options, but also the worst, and therefore with the identical four options generates five of the six possible paired comparisons (A > B, A > C, A > D, B > D, and C > D; only B is not compared to C). So ABC is 66% more “efficient†than CBC even without adaptive questioning.
ABC’s inherent efficiency advantage over CBC is further enhanced through adaptive questioning based on transitivity of preference. That is, we assume that if treatment A is preferred to treatment B, and if B > E, then A is also > E, even though A and E may never be compared directly. Such transitivity of preference may resolve even more paired comparisons than direct questioning. For example, given the 16-full-profile conjoint stimuli in our experiment, there are 16 x 15 / 2 = 120 possible paired comparisons. The ABC method has required 11 to 17 tasks, with an average of 12 best-worst tasks, to estimate a utility function with 9 parameters. The 12 tasks each directly resolve 5 paired comparisons (though some redundancy is likely, so they may not all be unique pairs), so direct questioning resolves at most 60 pairs (5 x 12). The remaining 60+ paired comparisons are resolved through transitivity.
The adaptive element of ABC derives from choosing the four stimuli to show in the next best-worst task based on maximizing the number of unresolved pairs given all prior responses. The method seems to be particularly effective in clinic.
Objectives: We apply the ABC best-worst method to full-profile conjoint stimuli, and further improve questioning efficiency by employing adaptive questioning. To highlight the method and its benefits, we test the effectiveness of the approach in clinic for a prostate cancer treatment application. With this approach, a 9-parameter utility function is estimated in real time at the individual level with an average of 12 tasks (11-17 range) completed in 10 minutes (7-20 minute range).
The primary problems with current conjoint methods such as CBC include: (1) they require expensive proprietary software such as Sawtooth's CBC, (2) are not run in real time nor do they provide instantaneous utility estimation, and (3) they take too long per respondent to estimate individual utility (they are really designed for population estimates), e.g. 18-24+ choice tasks to estimate 9 parameters. ABC solves these problems by (1) running in Excel, (2) reporting immediately and (3) cutting the questioning burden by more than half.
Methods: The entire application has been built as an Excel spreadsheet file using macros. Respondents are first taught about attributes and levels, then primed with a sample task, then guided through 11-15 such choice tasks between four full profiles at a time. The full profiles can be text- or graphics-based. In the background, the Excel software tracks every possible paired comparison and records all of the direct comparisons as well as calculating all of the transitive ones. The paired comparisons are converted into scores for each full-profile card, and a utility function estimated based on these scores. A report is automatically generated at the end of the process so that the individual can learn about his or her preferences. The report serves as the basis of discussion for post-biopsy meeting between patient and medical professional.
Clinical settings impose many constraints on time, space and resources. ABC addresses these issues by being very fast (~ 10 minutes), easy-to-implement (uses simple PC running Excel), and automated (Excel-based PC macro “interviews†the patient so clinic personnel can be freed up).
Results: Presently (April 2012): The presentation will highlight results about:
• Internal consistency: How many of the paired comparisons are consistent with the estimated utility function? Initial results: 60%-70%
• Estimation: a comparison of regression versus other methods such as LINMAP; the need or lack thereof for population-level analysis: Similar performance, with ABC nearly matching the predictive accuracy of the state-of-the-art Hierarchical Bayesian approach that exploits population data and cannot be run in real time.
• Respondent reactions to this system: Favorable and very appreciative from both patient and doctor.
We will also address some of the method’s limitations:
– The lack of intensity of preference measurements, i.e. A is assumed to be preferred to B by the same amount as B is preferred to C
– The issue of estimating rank ordered data using multiple linear regression rather than more advanced estimation methods
Conclusions:
Adaptive Best-worst Conjoint (ABC) has two efficiency advantages over traditional choice-based conjoint:
(1) directly resolving more paired preference comparisons per choice task and
(2) indirectly resolving additional paired preference comparisons through transitivity of preference.
The total number of tasks required of each survey subject can be further reduced through adaptive questioning by taking into account all prior responses and focusing on unresolved preference pairs.
The ABC method is currently being evaluated for use in other medical contexts including co-morbidity, in the context of depression, and to help with the treatment of chronic urinary tract infections. The presentation will discuss the medical and clinical contexts for which ABC is most appropriate, and those for which it is not.
The method can be implemented in low cost software (e.g. Excel macros) and is able to generate virtually instantaneous feedback to enhance discussions between patients and healthcare professionals.
The primary benefits of ABC in the clinical setting over existing methods are:
(1) improved estimation of patient preferences and priorities
(2) real-time clinical usability
(3) enhanced self-learning for patients and improved communication between doctor and patient
(4) Enhanced patient satisfaction with ultimate treatment choice
Medicine 2.0® is happy to support and promote other conferences and workshops in this area. Contact us to produce, disseminate and promote your conference or workshop under this label and in this event series. In addition, we are always looking for hosts of future World Congresses. Medicine 2.0® is a registered trademark of JMIR Publications Inc., the leading academic ehealth publisher.
This work is licensed under a Creative Commons Attribution 3.0 License.