Pierre Pluye*, McGill University, Montreal, Canada Roland Grad, McGill University, Montreal, Canada Carol Repchinsky, Canadian Pharmacists Association, Ottawa, Canada Barbara Jovaisas, Canadian Pharmacists Association, Montreal, Canada Charo Rodriguez, McGill University, Montreal, Canada Gillian Bartlett, McGill University, Montreal, Canada Janique Johnson-lafleur, McGill University, Montreal, Canada Barbara Farrell, University of Ottawa, Ottawa, Canada Dyan Tuft-conrad, Canadian Pharmacists Association, Ottawa, Canada Dino Mastroguiseppe, Canadian Pharmacists Association, Ottawa, Canada France Légaré, Université Laval, Québec, Canada Carmen Loiselle, McGill University, Montreal, Canada Jon Salsberg, McGill University, Montreal, Canada Martin Dawes, McGill University, Montreal, Canada Ann Macaulay, McGill University, Montreal, Canada Patrick Cohendet, HEC Montreal, Montreal, Canada Track: Research Presentation Topic: Usability and human factors on the web Presentation Type: Oral presentation Submission Type: Single Presentation Building: MaRS Centre, 101 College Street, Toronto, Canada Room: CR2 Date: 2009-09-18 01:30 PM – 03:00 PM Last modified: 2009-08-13

If you are the presenter of this abstract (or if you cite this abstract in a talk or on a poster), please show the QR code in your slide or poster (QR code contains this URL).

Background: Management studies show that the timely handling of user feedback benefits organizations and sustains relationships with users (Strauss & Hill, 2001). However, no studies have examined the integration of health professionals’ feedback into the management of electronic knowledge resources when feedback is stimulated by a validated evaluation method. The Canadian Pharmacists Association (CPhA) publishes e-Therapeutics+®, an electronic textbook for treatment recommendations, and other pharmaceutical databases. Before the implementation of the ‘Information Assessment Method’ (IAM), CPhA received little feedback (on average one per week) from a free-text comment box only accessible when users were logged into e-Therapeutics+®. Since dissatisfaction rarely leads to formal complaint, e.g., negative written comments on e-Therapeutics+®, a method such as IAM might help users to provide more feedback. Taking feedback into account is crucial since it represents the ‘tip of the iceberg’, i.e., other instances of dissatisfaction that were not expressed.

Objective: The purpose of the CPhA-McGill collaborative action research project was to explore whether feedback obtained from health professionals using IAM could be integrated into the management of e-Therapeutics+®.

Methods: Based on a seven-year research and development program, IAM is a validated method for systematically assessing the relevance, cognitive impact, use and expected health benefits of information objects. Following a participatory research approach, a web-based survey on highlights (selected content from a chapter in e-Therapeutics+®) was conducted in a push (email) and a pull (login & search) context over a 20-week period. When participants read a highlight, they were asked to rate the content using the IAM pop-up questionnaire that includes a comment box. Respondents could then enter a contest to win a small prize. In PUSH, 36 highlights were emailed to 6,500 CPhA members (mostly pharmacists working in hospital, community or primary care, and pharmacy students). In PULL, e-Therapeutics+® subscribers made 220,898 logins (nurses, pharmacists and physicians). All push and pull IAM comments were reviewed to identify constructive feedback, defined as comments requiring further investigation. Weekly constructive feedback reports were sent to CPhA.

Results: Responses varied depending on highlights and respondents. PUSH: Of 234,000 emails, 25.6% were opened, and 1.4% led CPhA members to complete a questionnaire (n=3,198). Of 120 constructive feedbacks, 15 led to some change in e-Therapeutics+® content: additional reference (n=4), more information (n=6), and rewording (n=5). PULL: 238 completed questionnaires were collected (on average 11.9 per week), leading to nine constructive feedbacks, but none lead to change. This increase of feedback was well-received by the editors, but increased their workload.

Conclusion: Results show the implementation of IAM stimulated constructive feedback in push and pull context. Feedback from the push context allowed the CPhA to make modifications designed to improve e-Therapeutics+®. This study suggests the implementation of IAM and the integration of constructive feedback into information management of electronic knowledge resources may renew organizational routines.