We will conduct a 42-month mixed methods health services project with the following study objectives:

1. To design a theory-based and knowledge user-informed intervention strategy to implement two provincial pediatric emergency CPs into practice in community EDs.

2. To evaluate the effectiveness of this implementation strategy, using a cluster randomized controlled trial (cRCT) design, through measurement of relevant process and clinical outcomes in community EDs in Ontario, Canada.

3. To conduct a process evaluation of the implementation strategy.

4. To conduct an economic analysis of implementation costs and benefits.

The individual hospital sites and their ED teams will be the focus of this study. Participants at each site will include ED staff and physicians, as well as hospital administrators with responsibility for the ED. Clinical outcomes for pediatric patients (defined as <18 years of age) with either asthma or gastroenteritis will also be studied.

Given the hospital context and involvement of the full ED team, CP implementation is a complex process requiring an informed and well-designed intervention strategy. To address the busy and competing pressures on EDs, an optimal intervention will be effective, practical, and feasible, yet with a strong evidence base for the design.

Control sites will continue with standard care, without additional materials. We will inform sites at recruitment that relevant pediatric patient data will be collected for ED presentations of asthma and V&D in the pre- and post-implementation periods. We will monitor relevant activities, such as self-implementation of any protocols that may relate to the control CP, within the process evaluation. While it is not possible to blind site partners to their allocated intervention, we will endeavor to conceal their allocation in the control arm. Control sites will not receive any CP documents. We will minimize potential co-interventions at recruitment by selecting only sites that do not have either related pediatric CPs in place. Based on recent studies of ED-based protocols 52 53 , most community EDs in Ontario do not have an asthma or V&D CP in place.

Because the number of cluster sites is relatively small, simple unrestricted randomization would not be sufficient to ensure balance between the study arms. Pairing of hospitals would improve balance but may reduce study power and precision due to loss of degrees of freedom associated with a matched analysis. We will therefore implement a minimization procedure 54 to ensure overall balance based on three important covariates: annual pediatric visits, location (urban/rural), and recent experience with any ED process improvement initiatives (yes/no). A statistician not associated with the study will use a computerized algorithm to identify all possible allocations that meet the balancing constraints and one of the allocations will be randomly selected 55 .

As shown in Table 2, the project is divided into five discrete phases. Following a nine-month preparation phase, the implementation phase will take place over the subsequent nine-month period. We have found that an open-ended implementation period is likely to lead to delays at various stages and might also complicate our data collection processes. For these reasons, we have designated a specific nine-month implementation period, with negotiated interim target dates, for all sites. Completed implementation will be defined when the following have been done: site-customization and committee approvals for the intervention CP, the CP is ready for use in the ED, delivery of at least two educational workshops, and promotion of the e-learning module through a central website. Our project coordinator will communicate regularly to ensure negotiated target deadlines are met. Should there be incomplete implementation at any site, reasons for this will be documented and explored further in the post-implementation interviews.

As shown in Table 3, we have planned a comprehensive evaluation approach including process and clinical outcomes. These are described in detail below.

Using a standardized tracking form, site visits will be conducted two months after completed implementation to assess awareness, accessibility and ease of use of the CP. Findings at each site will be compared with pre-implementation site visits.

Post-implementation interviews will be conducted to gain a richer understanding of perceptions, team dynamics, and other relevant issues occurring at the individual and organizational levels. We will conduct post-implementation interviews at one-half of the sites for each intervention, for a total of 8 KI interviews and eight focus groups with up to ten ED provider participants each. Additional interviews will be conducted if data saturation is not achieved. To identify readiness for change and barriers and facilitators at the organizational level, one administrator per site will be selected for KI interviews. Interviews will be audiotaped and transcribed. At each focus group site, up to 10 ED health professionals with direct patient involvement will be invited to participate in a one-hour focus group session. The focus group moderator will record field note observations. A court reporter will do real-time transcription at all focus groups, a method that yields more complete and rapidly available records of greater fidelity 56 . Using the TDF as a coding framework, two coders will independently analyze all transcripts and field notes. Data collection and analysis will follow an iterative and concurrent process 49 .

Given the nature of our study, both clinical and process outcomes are important to evaluate. Process outcomes relate to use of the CP for relevant patients. Clinical outcomes relate to adherence with key interventions as recommended by each pathway as well as patient specific outcomes. Table 4 presents a description of the specific outcome measures for this study.

Clinical outcomes will be measured through data abstracted from patient records and administrative databases in the 9-month pre and post-intervention periods. Prospective audits of the pre-implementation data are not possible as this period pre-dates the trial beginning. The post-implementation period can be clearly identified and prospective audit collection will therefore be done. Following completed implementation, we have scheduled a three-month settling in period before post-intervention audits will commence.

Using a previously successful approach with community ED studies 57 , we will ask local medical records departments to pull relevant charts using International Classification of Diseases-10 (ICD-10) codes for all diagnoses related to our index conditions (asthma and V&D) during the defined study periods. Chart auditors will review all records to ensure eligibility criteria are met. All retrieved and eligible patient charts will be audited. Based on data from previous work, and pediatric asthma visits in Ontario EDs 58 , we estimate an average of 85 Asthma and 115 V&D patient charts per site during each nine-month pre- and post-intervention period. Therefore, at each site 200 charts will be audited in each period, for a study total of 6,400 chart audits. In the event of multiple visits for the same patient, data will only be collected for the index visit. Four health record auditors will be trained to abstract data from patient records and directly enter these into a secure online database. To assess inter-rater agreement, pairs of auditors will initially each abstract the same 100 charts (50 asthma, 50 V&D). Agreement will be measured with a kappa coefficient and further training will be done until a kappa >0.8 is achieved. A data dictionary will be created to guide the chart auditors and ensure standardized data collection procedures. Auditors will be blinded to the study aims, study design, and group allocation.

Health administrative databases available at the Institute for Clinical Evaluative Sciences (ICES) will be used to study the impact of being on a CP on overall ED length of stay (LOS), hospital admissions, return visits within 72 hours following ED discharge for non-admitted patients, and death. Appropriate procedures will be used to link health administrative and patient specific data. The relevant databases are National Ambulatory Care Reporting System (NACRS), containing data from all Ontario EDs, and vital statistics data. Only data elements known to be reliable and valid will be used 59 . Use of NACRS data allows efficient capture of return visits, irrespective of whether the patient returns to the index hospital. These data also allow evaluation of shift-level characteristics, such as mean triage adjusted total LOS of all other patients in the ED on the same shift, which is an important contextual variable that may be associated with whether a CP was used and can be used to adjust for the LOS of the study patients. We will model whether the intervention had an impact on EDLOS, hospital admissions, return visits within 72 hours, and death; and whether ED busyness is associated with patients not being put on a pathway 66 . We will control for age, gender, Canadian Triage Acuity Scale (CTAS) rating, and EDLOS for other similar rated CTAS patients.

We conducted sample size calculations for our primary clinical outcome measures using standard formulas for comparing two proportions in cluster-randomized trials 60 . We conducted separate calculations for each CP and selected the larger of the two requirements as our target sample size. We assumed an intracluster correlation coefficient (ICC) of 0.01. This was the largest ICC based on preliminary data from a similar, not yet published study, which showed ICCs for dichotomous outcomes ranging from 0.001 to 0.01. Based on preliminary data, we assumed proportions of 10% in the control arm. We anticipated average numbers of asthma and V&D patients per hospital over the nine-month study duration of 85 and 110 respectively. After applying 10% inflation to account for cluster size imbalances, and allowing for attrition of one hospital per arm, eight hospitals are required in each study arm to yield 90% power to detect an absolute difference of 10% between study arms (10% in the control arm versus 20% in the intervention arm) using a two-sided test at the 5% level of significance. At recruitment, hospitals will be asked for written commitment to the study. Regardless of whether full implementation at that site has been completed, all relevant patient charts will be audited as per the protocol described. Therefore, barring exceptional circumstances, it is unlikely that hospitals will be lost to follow-up.

Throughout the project, we will track costs for all activities required for successful CP implementation at each site. This will include costs for: document customization and committee approval processes at each site, production of forms, posters and educational aides, preparation and delivery of workshops, and staff participation at these presentations. Additional costs to be tracked include refinement of web-based resources and access, travel and orientation for site champions and opinion leaders, telecommunication support, and other incidental activities. The hourly wage will be assumed to be the provincial mean for health professionals and administrators as per Ontario collective bargaining agreements, and $150 per hour for physician time.

In the final economic evaluations, additional costs and outcomes will be synthesized in an incremental cost-effectiveness ratio (ICER) comparing the intervention versus control. This will result in two cost-effectiveness analyses (CEAs) relating to each CPs. The ICER for each CEA will be expressed as the incremental costs per proportion of change related to the primary clinical outcome for that CP.

In the final analysis, because an economic evaluation is always surrounded with uncertainty 64 the robustness of the ICER will be checked for sample uncertainty using non-parametric bootstrapping. The bootstrapped cost-effectiveness ratios will be subsequently plotted in a cost-effectiveness plane, in which the vertical line reflects the difference in costs and the horizontal line reflects the difference in effectiveness. The choice of treatment depends on the maximum cost, known as the ceiling ratio, that society is prepared to pay for a gain in effectiveness. Therefore, the bootstrapped ICERs will also be depicted in a cost-effectiveness acceptability curve 65 showing the probability that the intervention is cost-effective using a range of ceiling ratios.

Ethical approval has been granted for this study at the coordinating hospital (Children’s hospital of Eastern Ontario Research Ethics Board), and we are seeking site ethics approval at each hospital as their study participation is confirmed. Informed consent will be sought for interviews and focus group meetings. All research data will be stored on a secure server and will be transmitted through a secure virtual private network (VPN). It will then be securely deleted from the laptop hardrive using the software Eraser (http://www.eraser.heidi.ie). This trial is registered with ClinicalTrials.gov (http://www.clinicaltrials.gov/NCT01815710).

We are currently in the preparation phase, enrolling sites and seeking local ethics approvals.