Categories
Uncategorized

Aligning implementation science with practice by co-designing an evaluation tool

Background:

The Alberta SPOR SUPPORT Unit Knowledge Translation Platform (AbSPORU KT Platform) works with health science researchers and practitioners to incorporate implementation science principles into research and quality improvement initiatives taking place in Alberta’s health care system. Incorporating implementation evaluation metrics into evaluation plans are of particular interest to those seeking implementation support because of the recognized importance of capturing successful implementation, as well as ways to improve ongoing implementation efforts.

A commonly used framework to evaluate health innovations in Alberta is The Alberta Quality Matrix for Health (AQMH) developed by the Health Quality Council of Alberta. The matrix includes six dimensions of health service quality adapted to the Alberta health care context: acceptability, accessibility, appropriateness, effectiveness, efficiency, and safety. Several of these dimensions overlap with Proctor et al.’s recommended taxonomy of implementation outcomes: acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability.

To help bridge these approaches, the KT Platform has brought together a team of intended users, content experts, and health system partners to co-design a user-friendly tool to help people incorporate implementation outcomes, such as those recommended by Proctor et al., into evaluations based on the six dimensions of health service quality captured in the AQMH.

 

Methods:

To create this tool, our team followed the Successive Approximation Model from the field of instructional design, which outlines an iterative, participatory design and development process that focuses on end users’ experiences, engagement, and motivation. All stages of this co-design process were shaped by and carried out in collaboration with team members as the project unfolded:

  • Set the Stage – Determine end users’ needs, gaps, and collect relevant background information.
  • Savvy Start Feedback Loops – Using the information gathered in Set the Stage, bring together the co-design team to identify the problem, other potential users, their contexts, brainstorm solutions, carry out rapid prototyping and feedback loops, and select one prototype for further development.
  • Prototype Development and Testing – Build the selected prototype in an iterative manner based on ongoing user testing and feedback.
  • Evaluate – Carry out an ongoing evaluation to ensure both the co-design process and final product meet team members’ and end users’ expectations.

 

Results:

The co-design activities outlined here have enabled our team to develop a web-based, interactive evaluation tool prototype that helps users incorporate implementation outcomes into AQMH-based evaluations. To date, 16 individuals have participated in usability and feedback surveys. This feedback has been used to make continuous improvements to the prototype. Interviews are now underway with a subset of usability participants to better understand the relevance of the tool to their projects and how they might incorporate the tool into their day-to-day work. Additionally, the ongoing evaluation will enable the KT Platform to learn about effective co-design and development processes for producing tools like this one, with the goal of informing future projects.

 

Conclusions:

The development of this tool will enable our Platform to address a need identified by health science researchers and practitioners seeking implementation evaluation support.

Poster

 

Leave a Reply

Your email address will not be published. Required fields are marked *