During the clinical research course, surgeon researchers from the United States, Canada, Mexico, and Cuba presented lectures on study protocol, surgical research methodology, and navigation of the publication process (Figure 1). Surgeons presented the one-day training immediately prior to a course on orthopedic trauma. All slides were in Spanish, and live interpreters translated for those presenters who lectured in English. In addition, each participant received a 60-page syllabus, also translated into Spanish.

No existing and validated tool specifically addressed the content of our course, which included instruction on fundamental elements of EBM incorporated into instruction on practical aspects of carrying out clinical research and preparing work for publication. Therefore, established quality guidelines (Shaneyfelt et al., 2006) were used to design our own assessment tool.

Evaluation instruments for education in EBM are diverse. Multiple authors conclude that no one tool would be appropriate to evaluate any proposed EBM training intervention—rather a tool designed to evaluate knowledge acquisition in the domains of intended knowledge gain should be thoughtfully selected (Straus et al., 2004; Shaneyfelt et al., 2006). Straus et al. proposed a conceptual framework for the evaluation of EBM that advised considering a physician’s “mode of practice” of EBM and choosing an evaluation instrument that reflects the different learners, interventions, and outcomes expected (Straus et al., 2004).

A recent review of instruments available for the evaluation of evidence-based practice suggested that the highest quality instruments were characterized by “interrater reliability (if applicable), objective (non-self-reported) outcome measures, and multiple (> = 3) types of established validity evidence (including evidence for discriminative validity)” (Shaneyfelt et al., 2006). Ultimately, an instrument was designed specific to the course, using the quality guidelines established by Shaneyfelt et al. and focusing on fundamental knowledge acquisition, appropriate for a one-day educational event. This was adapted in part from the questionnaire designed by Sprague and colleagues, who described the development and evaluation of a short training course in EBM for a surgeon audience in North America. Using a pre- and post-course questionnaire scheme, they reported a statistically significant increase in knowledge acquired based on relative increase in score as well as participant satisfaction with the training (Sprague et al., 2012). The instrument they employed was fundamentally based on the highly reliable and validated Fresno Test (Ramos et al., 2003; Thomas and Kreptul, 2015). The Fresno Test is a performance-based measure that was designed to evaluate a broad range of EBM knowledge and skills. Sprague and colleagues adapted it to their group’s more narrow goals and requirements, and the instrument was altered to apply to our training event.

By defining the learners (surgeons, physicians, etc.), intervention (education on clinical research and EBM), and the outcome (objective questionnaire results), our tool was consistent with the Straus conceptual model (Straus et al., 2004). A demographic survey, which specifies gender, age, and profession was also included. Once participants were separated into subgroups, it could then be determined which subgroups entered the course more knowledgeable in clinical research and EBM principles than others, and which subgroups benefited the most from the one-day course (as indicated by an increase in test scores). Pre-course and post-course questionnaires were intended to highlight objective (and not self-reported) measures, and the self-reported levels of knowledge were only present on the pre-test. In total, there were 18 multiple choice questions, eight of which referred to a specific clinical scenario case study, and 10 relating to clinical research design and methodology.

Participants received a printed pre-test upon entering the conference area, and these were collected before commencement of proceedings. Post-tests were handed out at the conclusion of the course and collected as participants exited the conference area. The outcome measure was defined as change in number of questions answered correctly on the post-test compared with the number answered correctly on the pre-test; thus, it was objective. Responsive validity was ensured by comparing individual participants’ scores from before and after the intervention. Discriminative validity was maintained by our ability to stratify participants based on their reported level of expertise in EBM and clinical research.

Descriptive statistics were performed using means and proportions. The distribution of scores pre- and post-course were assessed for normality. The paired Student’s t-test was used to assess the difference in means of pre- and post-course scores for statistical significance, assuming an assumption of normal distribution of scores. The data were further analyzed for further trends and hypothesis setting. The data were stratified based on professional level (surgeon, physician, and other health professional) as well as by experience with EBM and by experience in clinical research, rated by participants based on six-point scales. All analyses were conducted using STATA version 12.0 (College Station, TX, USA).

This study was submitted to the University of California, San Francisco’s Human Research Protection Program (IRB number: 11-06489) and was certified as exempt from Institutional Review Board consideration under Exempt Category 2: Use of educational tests, surveys, interviews, or observations of public behavior.

As required by this body, participants were provided a document with details regarding risks and benefits of the study. This document informed them that completion of the survey was considered consent to participate. The Human Research Protection Program did not require a signature on this document.

Ninety six of 102 registered attendees (94.1%) completed both the pre-course and post-course questionnaires. Eight participants completed only one of the two instruments and were, therefore, dropped from the analysis. Attendees were asked beforehand to complete the questionnaires only if they planned on staying throughout the entirety of the course. Surveys were collected from 104 total attendees, indicating that at least two other non-registered attendees elected to participate in the questionnaires.

Though the course was aimed at a surgeon audience, professionals from a number of other healthcare disciplines also attended and completed the surveys. Due to the high participation rate among attendees, it is improbable that certain demographic groups were more likely to complete the survey than others. Thus, in terms of who completed the survey, there was little to no selection bias. The mean age of participants was 44.6 years (SD: 11.8 years) and 52% were male. 46% identified themselves as surgeons, 17% as physicians, 23% as other professionals (nurses, physical therapists, etc.), and 15% did not respond to this question (Figure 1; Table 1). Self-reported exposure to EBM and clinical research on the part of this group was low to moderate; 46% of participants judged their EBM experience to be “poor” or “very poor,” and 38% described it as “good” or “average.” Similarly, 30% of participants judged their clinical research knowledge to be “poor” or “very poor,” while 53% described it as “good” or “average.”

The mean pre-test score was 43.9% (95% CI: 41.1–46.6%). The mean post-test score was 59.3% (95% CI: 56.5–62.1%). Despite the relatively low mean post-test score of 59.3%, there were no clear trends in what questions participants answered correctly or incorrectly. Overall, participants demonstrated a significant increase in post-course questionnaire scores compared with pre-course scores. 68% of participants improved by at least 20% relative their pre-test scores, and 35% of participants improved by at least 50% relative to their pre-test scores.

All professional groups demonstrated relative score increases (Table 2; Figure 2). At 57.9% (CI: 43.1–72.7%), non-surgeon, non-physician professionals showed the highest relative score increase. Physicians showed the lowest relative increase at 31.6% (CI: 10.2–53.0%). Scores increased at all EBM and clinical research experience levels. There were no clear patterns relative to changes in score based on pre-course self-assessment of experience with EBM (Figure 3; Table 3) or clinical research experience (Figure 4; Table 4).

The strengths of this study included a design that utilized a pre- and post-course questionnaire. As these were matched for each participant, a more robust statistical analysis was possible. Published guidelines were engaged to develop a high-quality instrument specific to the course content, participants, and structure. Potential contamination was limited by implementing both parts of our instrument on the same day as the course—immediately prior to its start and before participants departed from the final lecture.

Though our intended audience was orthopedic surgeons, many professionals from other health disciplines also registered for and benefited from the course. It would seem that non-physician, non-surgeon participants benefited the most, although their relatively higher change in score is the result of a lower pre-test mean. The intervention appears on the whole to have raised all participants to a common, higher level of understanding of the fundamentals of clinical research as presented in the course.

Limitations to our study include the possibility that an unknown proportion of participants were unable to be present for all portions of the course. In addition, the degree of effort exercised by participants in completing the questionnaires could not be assessed. Also, because of the nature of this course, with distance and logistics as significant barriers, an instrument was not designed to test the knowledge retention of the attendees over time. However, other similarly structured 1-day courses in pediatric life-support and partner abuse suggest that trainees retain increased knowledge for at least 3 to 4 months after the course (Durojaiye and O’Meara, 2002; Madden et al., 2015). Finally, it is unknown whether the gain in knowledge and skills that were measured will translate into an increase in clinical research or improvement in patient outcomes. Other studies have shown that teaching EBM knowledge and research skills can change health care professional behavior (Straus et al., 2005), and international health organizations, including the World Health Organization (WHO), the Council on Health Research for Development, and the Global Forum on Health Research, have consistently emphasized the critical importance of health research capacity development for the success of health systems in LMICs (Commission on Health Research for Development, 1990; Global Forum for Health Research and Davey, 2004; Lansang and Dennis, 2004; Changing Mindsets, 2008).

The lack of quality data on the burden of global orthopedic trauma and the availability of care to manage trauma-related injuries impede the establishment of policies that prevent avoidable morbidity and mortality (Elliott et al., 2015). It is most appropriate that those who live and practice in the affected communities conduct the research, collect the data, and apply the results to their practice (Gosselin et al., 2009).

A 2004 Bulletin of the WHO states: “A combination of short-term and long-term strategies, directed at individual, institutional, and country levels are necessary to develop a sustainable system of health research.” (Lansang and Dennis, 2004). The authors believe that directed coursework is an effective short-term strategy to improve clinical investigation. Following this course, academic and clinical research collaborations developed between the course supporters, including UCSF, CIMEQ, AMOT, and the Osteosynthesis and Trauma Care Foundation (OTCF). It was felt that the course enabled this collaboration, most importantly by improving the understanding of EBM and the ability of the Cuban orthopedic surgeons to access more established and resourced international partners. Subsequent to this course, there are plans to include Cuban surgeons in large prospective studies through McMaster University and the University of California, San Francisco. While not a direct indication of the long-term efficacy of the course, Cuban surgeons who participated in this work have become active members of the recently established international research consortium, Associación de Cirujanos Traumatologicos en las Americas (ACTUAR) that supports clinical research across the Americas (Chomsky-Higgins et al., 2017).

Based on the results of this work and successful experience with this course, the authors plan to offer additional clinical research courses to medical professionals in other countries in Central and South America. Similar results are anticipated, and the attendees of these courses are expected to not only improve their ability to better interpret the existing literature but also engage with local, regional, and international clinical research efforts. Ultimately, the authors hope to empower surgeons to create their own an international Latin American network, where the most relevant questions for their patient populations can be addressed.