Validity and Reliability of the EHealth Analysis and Steering Instrument (eASI)
|
If you are the presenter of this abstract (or if you cite this abstract in a talk or on a poster), please show the QR code in your slide or poster (QR code contains this URL). |
Abstract
Background
eHealth services can contribute to individuals’ self-management, i.e., performing lifestyle related activities and decision-making, to maintain a good health or to mitigate the effect of an (chronic) illness on their health. But how effective are these services? The eHealth Analysis and Steering Instrument (eASI) aims at answering this question. It surveys how eHealth services score on 35 items representing three dimensions (i.e., Utility, Usability and Content), which are theoretically related to the effectiveness of eHealth. However, there are no data on its validity and reliability.
Objectives
To assess the construct and predictive validity and inter-rater reliability of eASI.
Methods
We found a total of 16 eHealth services supporting self-management published in the literature, whose effectiveness was evaluated in a Randomized Controlled Trial and the service itself was available for rating. Participants (N=16) rated these services with the eASI. We analyzed the correlation of eASI items with the underlying three dimensions (construct validity), the correlation between the eASI score and the eHealth services’ effect size observed in the RCT (predictive validity), and the inter-rater agreement.
Results
Three items did not fit with the other items and dimensions and were removed from the eASI; four items replaced from the Utility to the Content dimension. The inter-rater reliabilities of the dimensions and the total score were moderate (Total, κ = .53, and Content, κ = .55) and substantial (Utility, κ .69, and Usability, κ = .63). The adjusted eASI explained variance in the eHealth services’ effect sizes (R2 = .31, P < .001), as did the dimensions Utility (R2 = .49, P < .000) and Usability (R2 = .18, P = .021). Usability explained variance in the effect size on health outcomes (R2 = .13, P = .028).
Conclusions
After removing three items and replacing four items to another dimension, eASI has a good construct validity and predictive validity. The eASI scales are moderately to highly reliable. Due to a small pool of available eHealth services, it is advised to reevaluate the eASI in the future with more services.
eHealth services can contribute to individuals’ self-management, i.e., performing lifestyle related activities and decision-making, to maintain a good health or to mitigate the effect of an (chronic) illness on their health. But how effective are these services? The eHealth Analysis and Steering Instrument (eASI) aims at answering this question. It surveys how eHealth services score on 35 items representing three dimensions (i.e., Utility, Usability and Content), which are theoretically related to the effectiveness of eHealth. However, there are no data on its validity and reliability.
Objectives
To assess the construct and predictive validity and inter-rater reliability of eASI.
Methods
We found a total of 16 eHealth services supporting self-management published in the literature, whose effectiveness was evaluated in a Randomized Controlled Trial and the service itself was available for rating. Participants (N=16) rated these services with the eASI. We analyzed the correlation of eASI items with the underlying three dimensions (construct validity), the correlation between the eASI score and the eHealth services’ effect size observed in the RCT (predictive validity), and the inter-rater agreement.
Results
Three items did not fit with the other items and dimensions and were removed from the eASI; four items replaced from the Utility to the Content dimension. The inter-rater reliabilities of the dimensions and the total score were moderate (Total, κ = .53, and Content, κ = .55) and substantial (Utility, κ .69, and Usability, κ = .63). The adjusted eASI explained variance in the eHealth services’ effect sizes (R2 = .31, P < .001), as did the dimensions Utility (R2 = .49, P < .000) and Usability (R2 = .18, P = .021). Usability explained variance in the effect size on health outcomes (R2 = .13, P = .028).
Conclusions
After removing three items and replacing four items to another dimension, eASI has a good construct validity and predictive validity. The eASI scales are moderately to highly reliable. Due to a small pool of available eHealth services, it is advised to reevaluate the eASI in the future with more services.
Medicine 2.0® is happy to support and promote other conferences and workshops in this area. Contact us to produce, disseminate and promote your conference or workshop under this label and in this event series. In addition, we are always looking for hosts of future World Congresses. Medicine 2.0® is a registered trademark of JMIR Publications Inc., the leading academic ehealth publisher.
This work is licensed under a Creative Commons Attribution 3.0 License.