Improving Confidence in Obstetric Skills with Basic and Advanced Life Support in Obstetrics® Training in Ethiopia: A Pre/post Study

Basic Life Support in Obstetrics (BLSO®) is a novel, simulation-based, emergency obstetrics training program for health care workers who perform deliveries infrequently, such as paramedics and nurses. Advanced Life Support in Obstetrics (ALSO®) is intended for regular maternity care providers such as physicians and certified nurse midwives. This paper describes a pre/post study of 111 learners in two ALSO and two BLSO courses taught in Addis Ababa, Bonga, and Jimma, Ethiopia in October, 2012. The study objective was to assess the effectiveness of ALSO and BLSO training programs to improve Ethiopian health care workers’ confidence in managing common obstetrical problems. A previously-validated survey tool was given immediately preand post-course and six months post-course. The survey tool included demographic questions and Likert scales to self-assess comfort managing 13 common obstetric emergencies including postpartum hemorrhage, maternal resuscitation, and preeclampsia. Ninety-seven ALSO and BLSO learners completed preand immediate post-training questionnaires. Nineteen ALSO learners completed the 6 month post-training questionnaire. Immediately post-course, participants in ALSO and BLSO courses reported increased comfort in all 13 areas assessed. ALSO participants had increased comfort at 6 months in all areas except breech delivery. Participation in a novel emergency obstetrics training program increases Ethiopian health workers’ self-assessed confidence in addressing common obstetrical emergencies. This increase in confidence persists in most topic areas for at least six months. This paper describes the first formal evaluation of BLSO and contributes to the growing body of evidence regarding the value of ALSO training in a lowand middle-income country. Corresponding Author: Ann E. Evensen, MD Associate Professor, Department of Family Medicine University of Wisconsin School of Medicine and Public Health, 100 North Nine Mound Road, Verona, WI Ann.evensen@uwmf.wisc.ed Office phone 608 845 9531 Office fax 608 845 8684


Introduction
Fifty percent of global maternal deaths occur in six countries, including Ethiopia, which had a maternal mortality ratio of 420 per 100,000 live births in 2013. 1,2 Poor access to qualified health care providers, safe and timely emergency transport, medications, blood products, and other medical supplies contribute to this tragic statistic. Shortages of personnel and supplies can be most acute in rural areas such as the Southern Nations, Nationalities, and Peoples' Region (SNNPR), located in southwestern Ethiopia. The SNNPR has an estimated 1 doctor per 55,000 people and 1 midwife per 10,000 expected births, and the maternity care skills of these birth attendants may be inadequate to identify and appropriately treat obstetric emergencies. 3,4 Clinicians working in remote, poorly-supplied environments are the ones who could most benefit from learning effective strategies to safely treat patients or stabilize and transport them to higher levels of care.
In 2012, Peace Corps volunteers conducted a needs assessment to better understand the obstetric needs of community members and hospital staff in Bonga, a small, rural community in SNNPR. Emergency obstetrics training was identified, as a top priority (unpublished, see Appendix 1). The needs assessment and resulting courses were facilitated by Peace Care, an innovative U.S. nonprofit organization.
Peace Care brings together local community and health system leaders, Peace Corps volunteers, and US academic institutions with academic institutions in low-and middle -income-countries (LMICs) to strengthen health services, build capacity, and fund implementation research. 5 Advanced Life Support in Obstetrics (ALSO®) and Basic Life Support in Obstetrics (BLSO®) were chosen as the curricula to fill the emergency obstetrics knowledge and skills gap. ALSO courses have been taught in 62 countries, including many low-resource settings. 6 BLSO is a new course designed for health care workers who perform deliveries infrequently, such as paramedics and nurses. Following pilot courses in India in 2011, it has been taught in Ethiopia, Mexico and the US.
Incorporating low-fidelity simulation, case discussion, and didactics, the courses teach standard protocols for identifying and managing obstetrical emergencies. The training requires active participation by attendees and is conducted over 2-3 days (ALSO) or 1-2 days (BLSO) using a low student-to-teacher ratio. A practical final examination evaluates competence in managing obstetric emergencies such as postpartum hemorrhage and shoulder dystocia. The ALSO/BLSO Instructor Course is a one-day faculty development course that incorporates many aspects of the "train the trainer" model found to be beneficial in a recent systematic review. 7 Previous studies evaluating the effectiveness of ALSO training have documented improved participant confidence in managing obstetrical emergencies. 8,9 However, these studies surveyed clinicians practicing in high-resource communities. Because clinical training is expensive, it is worthwhile to evaluate a curriculum's effectiveness to determine if the costs are reasonable, especially for resource-poor communities. Improved participant self-confidence in managing emergencies is one desirable outcome of a potentially valuable curriculum.
The purpose of this paper is to lay the groundwork for rigorous data collection on the efficacy of emergency obstetrics training programs. Collecting data on clinician-and patient-oriented outcomes should be a priority to aid governments and potential funders in evaluating training programs. Our study objective was to assess the effectiveness of ALSO and BLSO training programs in improving Ethiopian health care workers' confidence in managing common obstetrical problems. We hypothesized that the ALSO and BLSO training would increase Ethiopian health care workers' self-confidence in emergency obstetrical care for at least 6 months after course completion.

Materials and Methods
This was a pre/post study evaluating selfassessed comfort with managing common obstetrical problems. In October, 2012, there were four courses taught at three sites in Ethiopia (Table 1 lists  Representatives from the health centers, county health departments, and health posts identified individuals for training, which is common practice in rural Ethiopia. ALSO course participants in Jimma were clinicians from the surrounding region of Oromia and were recruited through the Department of Obstetrics and Gynecology at Jimma University Specialty Hospital. Government health officials in Bonga recruited two or three midwives and/or health officers from each surrounding county health center for the ALSO course and one or two health extension workers (HEWs) from the smaller village health posts for the BLSO course. HEWs are Ethiopian women with 10 th grade education who have taken extra training in health promotion and basic management of common medical problems including pregnancy. One or two HEWs staff health posts serving a population of approximately 5000 persons. The majority of the participants in Addis Ababa were the first class of government-sponsored emergency medicine technician (EMT) trainers who were to be stationed in remote sites to provide EMT training in rural Ethiopia. The remainder of Addis participants were community-based nurses from Yetebon Hospital in rural SNNPR.

Data sources
The study evaluates self-assessed comfort with management of common obstetrical problems by ALSO and BLSO participants. A written questionnaire was used to collect data from ALSO and BLSO training course participants at three time points: (a) prior to the ALSO and BLSO courses, (b) immediately after participants completed ALSO and BLSO courses, and (c) six months post-course. This survey tool had been previously validated. 9 See Appendix 2 for the survey tool. The questionnaire was administered in person prior to and immediately following training. Six month post-course questionnaires were distributed by mail or in person. Rates of successful course completion by participants were recorded. The questionnaire included: 1) seven demographic questions, 2) thirteen items that documented level of comfort in managing obstetric emergencies using a 4-point Likert scale ranging from 1 (terrified) to 4 (comfortable), 3) one question regarding participants' likelihood to practice obstetrics in the future, and 4) course evaluation questions (four in immediate post-course survey, two in 6-month survey).
ALSO training covers 13 emergency obstetric topics and procedures (see Table 4). The questionnaire asked participants to rate their self-confidence in managing these topics and procedures. BLSO training covers eight of the 13 topics in the ALSO course. BLSO includes very limited discussion of preterm labor, first trimester complications, and dysfunctional labor. BLSO does not include any training in vacuum-or forcepsassisted delivery. However, the same study questionnaire was given to ALSO and BLSO trainees so the 5 topics that are not covered in the BLSO course could be used as internal controls (i.e., we did not expect to see improvements in self-confidence in these topic areas for the BLSO trainees).
Likert scale values were based on values used in prior studies but the term "coping" was changed to the term "ok" because of anticipated translation difficulties. Of note, English is formally used in medical training schools in Ethiopia. Ethiopian physicians are typically fluent in both spoken and written English. Non-physician Ethiopian health care workers vary in English proficiency, and written proficiency is often higher than verbal proficiency. The Bonga BLSO participants were HEWs with a 10 th grade education and had a low level of English proficiency. The written questionnaires were in English but were translated orally by Ethiopian interpreters into Amharic, Ethiopia's official language. The participants were able to ask clarifying questions.
An identical questionnaire was administered to participants immediately post-course. In addition to the questionnaire items described above, the post-course questionnaire solicited qualitative feedback. Six months after the course, a similar questionnaire to the postcourse questionnaire was attempted (without the two questions soliciting qualitative course feedback). Six months after the courses we were able to reach participants from the Bonga ALSO training using Peace Corps volunteers or a government courier.
Nonresponders were not sent reminders or repeat surveys. Jimma ALSO course, Addis BLSO course, and Bonga BLSO course participants were unreachable as they were stationed in areas of Ethiopia far from study personnel. We did not have sufficient funds or personnel to reach them six months post-course.
The study was approved by the Northwestern University Internal Review Board in Chicago, IL on 8/31/2012 under reference number STU00066311. Prior to taking the survey, participants were informed in both English and Amharic that the study surveys were voluntary, anonymous, and would not affect their performance in the ALSO or BLSO course itself. Written participant consent was obtained on IRB-approved consent forms prior to completion of the pre-course study survey. Mobile telephone numbers were used for subject identification. The main outcome measure was self-assessed comfort with management of 13 common obstetric emergencies. Results from self-assessed comfort questions were summarized using frequencies and descriptive statistics. Mean differences between pre-, immediately post-, and 6 months post-course surveys were examined using a paired samples t test at the α = .05 significance level. Data were analyzed using SPSS® Version 22 for Mac. Only participants who completed both pre-and post-course tests are included in the analysis ( Table 2 lists survey participation rates and Table 3 lists demographic characteristics of participants).

Qualitative feedback:
Qualitative findings were used to supplement interpretation of quantitative responses. Subjective feedback was solicited from the learners after each course. Additionally, each of the team members involved with the project had the opportunity to provide feedback. This data was not analyzed formally but is incorporated into the Discussion section below.

Results
Participants: Table  3 summarizes demographic characteristics for participants in the ALSO and BLSO courses who completed both pre-and immediate posttests, as well as in aggregate. Participants were diverse in their prior training (physicians, health officers, health extension workers, and nurses). Participants estimated that their annual delivery volume was high (more than 30% are delivering more than 60 patients per year). Table 2 summarizes the course participation and survey completion rates, and ALSO participants had baseline self-assessed comfort ratings that ranged from a low of 2.2 for shoulder dystocia and forceps delivery to a high of 3.2 for neonatal resuscitation. BLSO participants had baseline self-assessed comfort ratings from a low of 2.0 for shoulder dystocia to a high of 2.9 first trimester bleeding. Immediately post-course, ALSO participants had mean increases in self assessed comfort ranging from 0.3 for first trimester bleeding to 0.9 for shoulder dystocia on a 4-point Likert scale. BLSO participants had mean increases in self assessed comfort ranging from 0.5 for maternal resuscitation and trauma to 1.5 for shoulder dystocia on a 4-point Likert scale. Participants' self-assessed level of comfort managing obstetrical emergencies increased immediately after ALSO and BLSO courses in all 13 topic areas. ALSO course participants who completed the 6 month post-course survey showed a sustained and significant increase in self-confidence in in 12 of 13 topics. There was no change in confidence with management of breech delivery at six months post-ALSO course. There was no six month post-course data available for BLSO participants due to difficulties in reaching them at their rural postings as described in Data Sources, above.

Participant Course Evaluation and Practice Plans:
Participants were asked questions regarding their overall evaluation of the training course and their practice plans. One hundred percent of combined course participants who completed evaluations (N=97) would recommend the course to others. Ninety-three percent are planning to practice obstetrics, one percent is not planning to practice obstetrics, and six percent are undecided.
Ninety-five percent thought the course increased their likelihood of practicing obstetrics, two percent thought the course decreased their likelihood of practicing obstetrics, and three percent were undecided.

Discussion
The purpose of the study was to assess the effectiveness of ALSO and BLSO training programs in improving Ethiopian health care worker's confidence in managing common obstetrical problems. The results This table specifies the survey participation rates of ALSO and BLSO participants before the course, immediately after, and six months following the course. This table lists the 13 emergency obstetrics topics covered in ALSO training and the mean improvement seen in participants' self-assessed confidence in management of these topics immediately after and six months after ALSO training. Statistical significance of improvements is included. PROM = premature rupture of membranes, SD = standard deviation  show increased comfort in managing most common obstetric emergencies following ALSO and BLSO training. Additionally, the 19 participants who provided six month post-course feedback reported increased comfort for at least six months post-ALSO training. The study is consistent with findings of previous studies of clinicians in the United States and Australia documenting increased comfort in managing obstetric emergencies after taking an ALSO course and better performance than untrained clinicians in an objective structured clinical examination (OSCE)-style assessment of shoulder dystocia management six months post-training. [8][9][10][11] Because previous studies only surveyed clinicians practicing in high-resource communities, this study is important because it is the first to evaluate BLSO in any setting and the first to evaluate ALSO participant self-assessed comfort in a LMIC. Training clinicians is expensive in terms of time, opportunity costs, and course expenses (per diem, food, simulation equipment, printed materials, etc.). Thus, it is imperative to demonstrate a curriculum's effectiveness to determine if the costs are worthwhile and reasonable. This study is a critical first step in identifying training programs that are affordable and effective in underresourced settings.
Increased self-assessed comfort was found immediately post-course in all 13 topics covered in the ALSO course. The largest increase was seen with the topic of shoulder dystocia. This was not surprising as it is taught using a simple mnemonic that is practiced on mannequins repeatedly by every course participant. Similarly, BLSO trainees noted greatest improvement in shoulder dystocia (see below). Six months post-course, increased self-assessed comfort was found in all 13 topics except breech delivery. Possible explanations may be a low volume of breech deliveries encountered postcourse and/or that this method of delivery is more complex and riskier than other vaginal deliveries. This procedure may require more real-world experience before a learner feels confident.
The BLSO curriculum incorporates eight of the 13 topics covered in ALSO. Increased self-assessed comfort was found immediately post-course in every topic, including those that were not covered extensively in the BLSO curriculum (vacuum and forceps deliveries are not covered in BLSO, and limited teaching is done regarding first trimester bleeding, dysfunctional labor, and preterm delivery). We chose to provide the full 13topic survey to the BLSO participants so that the topics not covered thoroughly by the BLSO course could be used to support the survey's validity (i.e., we did not expect self-assessed comfort to increase in these areas). However, the data did show increased self-assessed comfort in all areas. This may reflect a recall bias where participants score test items more favorably because they are more familiar with the test questions. 12 Other explanations are also possible. Before the course, the BLSO participants uniformly assessed themselves as 'terrified' or 'scared'; BLSO training may have bolstered their overall confidence in providing maternity care. Another explanation for the uniform increase in confidence may be a reflection of the difficulties that the Bonga BLSO participants, the majority of whom were HEWs with 10 th grade education, may have had understanding the survey questions. Verbally translating the survey into Amharic and giving the opportunity to ask questions may not have been sufficient to overcome literacy issues. A different study design may be needed for this group of learners.
We found that self-assessed comfort increased in 12 of 13 topic areas from immediately pre-ALSO course to six month post-ALSO course. This is not consistent with prior studies of clinical skills retention that have demonstrated that skills typically decay over time. 13 However, these prior studies have also shown that competency is more likely to be retained in personnel who practice their skills in actual clinical environments. Sub-analysis of surveys from Bonga ALSO course participants showed that more than 70 percent of attendees estimated that they delivered 30 or more patients annually. This large number of maternity patients likely provided significant clinical experience for the Bonga ALSO course participants in the six months following training.
Increased self-assessed comfort with managing emergencies was consistent with the objective measure of successful course completion. Of the 111 participants, only two participants who took the final evaluation did not pass (11 participants could not be tested due to work-related schedule conflicts).
The quantitative results are strengthened by the qualitative results, which provide us with a deeper understanding of the great interest in and appreciation for the training. From a Bonga ALSO course student: "Enables me to save pregnant ladies with high risk and infants too. Words fail me to express how useful it is." Regardless of where the trainees are providing maternity care-hospital, health center, or the emerging network of Ethiopian ambulances-this material is immediately relevant. Ninety-five percent of combined ALSO/BLSO attendees indicated that taking the course increased their likelihood of practicing obstetrics.
Attendees and Peace Corps Volunteers appreciated the pre-course preparation done to ensure that the content was appropriate for rural Ethiopian practitioners. From a student in the Bonga ALSO course: "This course is very important and it considered our country's context, the practical was very good, it makes people confident." The practical hands-on style of this training is not typical. Ethiopian medical training is traditionally done by a lecturer speaking to a nonparticipatory audience due to large numbers of trainees and the limited available educational and human resources From a Peace Corps volunteer: "I would like to highlight the importance of combining lecture-based material with hands-on, real-world practice…the participants were asked to think creatively and critically, which is definitely missing in the Ethiopian education system." With implementation of the "train the trainer" program, including donation of several pelvic and neonatal mannequins, local medical professionals are now able to continue this valuable training with little additional cost. From a Peace Corps volunteer: "The efforts will be exponentially increased as the trainees become the trainers in their work places."

Limitations of the Study
Our conclusions must be interpreted with acknowledgement of potential biases due to the methods of participant recruitment and the study design. The majority of course attendees were likely willing participants and perhaps actively sought out this training. Self-selection could create a study population that is more motivated to retain skills or more confident in managing obstetric emergencies at baseline. Other limitations inherent in self-assessment include the potential for underestimating or overestimating one's skills. Our study lacked objective assessment of skills prior to the course and at the six month post course, which may be particularly important to incorporate in the future for low literacy leaners. The survey had a low response rate at six months. To overcome these limitations, future studies should include serial evaluations including OSCE-style skill assessments, a longer time-frame to determine skills decay over time, and a greater effort to overcome barriers to longitudinal study participation.

Conclusion
Participation in a novel, low-fidelity simulation training program increased Ethiopian health workers' self -assessed confidence in addressing common obstetrical emergencies. This increase in confidence persisted in most topic areas for at least six months as assessed by a pre/post study. Though there were methodological limitations, this study is an important first step in identifying effective and relevant skills training for clinicians in low-resource settings.
Future assessments of emergency obstetric training should address objective outcomes such as clinician's retention of critical skills and patient-oriented outcomes such as rates of postpartum hemorrhage. This study contributes to the important field of primary health care implementation research. This field of research is needed to evaluate potential interventions, their outcomes, and contextual influences 14 The capacity of LMIC local health systems is often inadequate to support even basic primary health care services. Thus, there is ample opportunity for innovative collaborative efforts, such as the community-Peace Corps-academic partnership described in this study, to conduct critical quality improvement initiatives and promote research focusing on local primary health care priorities. 5