Analysis of Response Time in Electronic PRO Instrument: How Could It Be Used to Refine the Analysis of Health-Related Quality of Life Data?
|
If you are the presenter of this abstract (or if you cite this abstract in a talk or on a poster), please show the QR code in your slide or poster (QR code contains this URL). |
Abstract
Background: Electronic patient-reported outcomes (PRO) measures can be used to assess health-related quality of life in clinical settings. They also provide auxiliary information such as time to complete a survey or time spent on each item of a questionnaire.
Objective: The aim of this study was to evaluate response time in relation with item content and item severity on an electronic questionnaire assessing health-related quality of life specific of HIV patients.
Methods: Data were collected during the validation of the ePROQOL-HIV questionnaire (8 dimensions, 43 Likert-type items in total) on internet. Sociodemographic data and individual timestamps were available for 54 electronic records. The sample of HIV patients had the following characteristics: 36% females, mean age 47 years, 72% of secondary or university education level. Items were characterized by their length (number of characters), difficulty and spread of response categories thresholds as estimated from an item response model for polytomous items (Partial Credit Model). Analysis was carried out on two dimensions of the questionnaire: physical health and symptoms (PHS, 9 items, Cronbach alpha=0.885), and emotional distress (ED, 4 items, Cronbach alpha=0.869). Pearson correlations with associated 95% CIs were used to summarize linear associations between numerical variables; ANOVA F-tests were used to summarize the effect of a categorical predictor on a continuous outcome. Statistical analyses were done using R software with the eRm package.
Results: Few items exhibited reversed thresholds for the extreme response categories (e.g., ‘rarely’ was less likely to be endorsed than ‘never’), although item fit statistics remained in the acceptable range (INFIT and OUTFIT indices between 0.7 and 1.3). Average standard errors for person location parameters were 0.44 (PHS) and 0.63 (ED). Average response time (RT, in seconds) for the PHS dimension (12.1s) was twice that observed for the ED dimension (6.5s). It was positively correlated with item length (r=0.600, [0.342;0.940]) and negatively correlated with item parameters (r=-0.595, [-0.880;0.008]), which means that more severe items were associated with longer RTs. However, the spread of item thresholds did not correlate with item characteristics. Average RT for each dimension did not correlate with person location on the latent trait, nor with study level of participants (ANOVA, p=0.188 for PHS and p=0.273). However, a significant effect of gender was found for the ED dimension, with women showing higher RTs (9.2 ± 4.4s) compared to men (6.3 ± 3.9s, p=0.016). No such effect was observed for the PHS dimension.
Conclusions: The present findings suggest that items dealing with more severe impact of HIV can also affect RT, besides item length. Interestingly, although the ED dimension was composed of shorter items implying faster reading, women did take more time to answer those questions, which suggests that they might pay more attention to their emotional state than men. Such analysis could be useful in questionnaire’s development, or in field trials in association with cognitive debriefing.
Objective: The aim of this study was to evaluate response time in relation with item content and item severity on an electronic questionnaire assessing health-related quality of life specific of HIV patients.
Methods: Data were collected during the validation of the ePROQOL-HIV questionnaire (8 dimensions, 43 Likert-type items in total) on internet. Sociodemographic data and individual timestamps were available for 54 electronic records. The sample of HIV patients had the following characteristics: 36% females, mean age 47 years, 72% of secondary or university education level. Items were characterized by their length (number of characters), difficulty and spread of response categories thresholds as estimated from an item response model for polytomous items (Partial Credit Model). Analysis was carried out on two dimensions of the questionnaire: physical health and symptoms (PHS, 9 items, Cronbach alpha=0.885), and emotional distress (ED, 4 items, Cronbach alpha=0.869). Pearson correlations with associated 95% CIs were used to summarize linear associations between numerical variables; ANOVA F-tests were used to summarize the effect of a categorical predictor on a continuous outcome. Statistical analyses were done using R software with the eRm package.
Results: Few items exhibited reversed thresholds for the extreme response categories (e.g., ‘rarely’ was less likely to be endorsed than ‘never’), although item fit statistics remained in the acceptable range (INFIT and OUTFIT indices between 0.7 and 1.3). Average standard errors for person location parameters were 0.44 (PHS) and 0.63 (ED). Average response time (RT, in seconds) for the PHS dimension (12.1s) was twice that observed for the ED dimension (6.5s). It was positively correlated with item length (r=0.600, [0.342;0.940]) and negatively correlated with item parameters (r=-0.595, [-0.880;0.008]), which means that more severe items were associated with longer RTs. However, the spread of item thresholds did not correlate with item characteristics. Average RT for each dimension did not correlate with person location on the latent trait, nor with study level of participants (ANOVA, p=0.188 for PHS and p=0.273). However, a significant effect of gender was found for the ED dimension, with women showing higher RTs (9.2 ± 4.4s) compared to men (6.3 ± 3.9s, p=0.016). No such effect was observed for the PHS dimension.
Conclusions: The present findings suggest that items dealing with more severe impact of HIV can also affect RT, besides item length. Interestingly, although the ED dimension was composed of shorter items implying faster reading, women did take more time to answer those questions, which suggests that they might pay more attention to their emotional state than men. Such analysis could be useful in questionnaire’s development, or in field trials in association with cognitive debriefing.
Medicine 2.0® is happy to support and promote other conferences and workshops in this area. Contact us to produce, disseminate and promote your conference or workshop under this label and in this event series. In addition, we are always looking for hosts of future World Congresses. Medicine 2.0® is a registered trademark of JMIR Publications Inc., the leading academic ehealth publisher.

This work is licensed under a Creative Commons Attribution 3.0 License.