The experience of using an integrated knowledge translation approach to develop and implement an audit and feedback system in Ontario maternal-newborn hospitals

Authors: Reszel J, Dunn S, Weiss D, Graham ID, Sprague A, and Walker M, on behalf of the Maternal Newborn Dashboard team

Presenter: Jessica Reszel

Institution: Ottawa Hospital Research Institute and Better Outcomes Registry & Network (BORN) Ontario

 

Background: 

Integrated knowledge translation (IKT) involves an ongoing relationship between researchers and knowledge users for the purpose of engaging in a mutually beneficial project to support decision-making. We used an IKT approach to co-create and implement a new innovation, the Maternal Newborn Dashboard (MND), an electronic audit and feedback system with six key performance indicators (KPI) for all Ontario maternal-newborn hospitals. As one part of our mixed-methods study, we evaluated our use of an IKT approach to co-create and implement the MND, and the effect on end-user uptake.

 

Methods: 

We drew from two data sources. Stakeholder survey: We developed and administered an electronic survey to stakeholders involved in MND development (e.g., KPI selection, benchmark setting, dashboard design/testing, implementation/post-implementation). The survey included closed- and open-ended questions to assess respondents’ perceptions of the process used to develop and implement the dashboard and identify areas for improvement. Case study: We conducted a case study (including interviews and focus groups) with a purposeful sample of diverse Ontario maternal-newborn hospital personnel who had varying success improving rates on MND KPIs. Qualitative content analysis was used to code and categorize the data. The use of these two data sources allowed comparison of perspectives between those directly involved in MND development/implementation (survey respondents) and those that were not (case study participants).

 

Results: 

Nineteen stakeholders completed the survey (response rate=54%), including those involved in KPI selection and benchmark setting (n=11), dashboard design and testing (n=7), and dashboard implementation (n=13). In addition, 107 stakeholders from 14 hospitals participated in the case study, primarily maternal-newborn healthcare providers (n=63) and leaders (n=31). All survey respondents (n=19/19) directly involved in MND development/implementation were satisfied with the process used and most agreed that all essential stakeholders were involved in KPI selection and benchmark setting (n=10/11), design and testing (n=6/7), and implementation (n=8/13). These stakeholders perceived that end-user engagement and communication was good and the MND was feasible to implement and disseminate to users, with some ideas for improvement. Some of our case study participants not directly involved in MND development/implementation demonstrated a misunderstanding of the process used to select the KPIs and set benchmarks for the MND. For example, some end-users were not aware of why KPIs were selected and/or erroneously believed that only researchers were involved in creating the MND, with inadequate input from clinicians. This resulted in some individuals/teams doubting the credibility of the MND and subsequently not initiating any clinical practice changes on the targeted KPIs.

 

Conclusions: 

Despite using a large and diverse interprofessional provincial committee and a highly coordinated communication strategy, some end-users lacked knowledge about the MND and the development process, which limited uptake in clinical practice. Key recommendations for an IKT approach to enhance the acceptability and uptake of future health-system level innovations include: (1) finding new and feasible ways to engage a broader group of end-users in the co-creation process, and (2) enhancing transparent communications to non-participants of the original IKT process.

 

 

View Slides