---

Abstract

Background: Ensuring accurate and complete emergency medical services (EMS) patient documentation is vital for the safe transition of patient care. Objectives: This study examined whether a quality improvement (QI) project focused on documentation via checklists can improve the inclusion of key documentation criteria on electronic patient care reports (ePCRs) in a collegiate-based EMS system. Methods: This retrospective study analyzed the ePCRs of Georgetown Emergency Response Medical Services before and after the revision of documentation checklists as part of the agency’s continuous QI process. Reports for calls in which patients presented with any of the following chief complaints were analyzed: chest pain, abdominal pain, seizure/syncope, and head/neck trauma. Results were reported as the percentage of required elements noted in the checklists that were documented. Results: Over a 2-year period (2013-2015), 373 charts were analyzed. Following the adoption of the updated documentation checklists, there were statistically significant increases in the overall inclusion of required documentation elements for all studied chief complaints: chest pain (69.29% to 81.31%); abdominal pain (70.65% to 80.82%), head trauma (80.47% to 86.39%); seizure/syncope (78.3% to 84.67%); all p<0.05. Conclusions: Checklists are a potential tool to improve EMS documentation as part of an agency’s continuous QI process in a collegiate-based EMS setting.

---

Introduction

Accurate, complete, and reliable EMS documentation is important for effective communication and the safe transition of patient care.¹ Inadequate prehospital documentation has been associated with increased in-hospital morbidity and mortality.² Prehospital care reports are valuable tools to assess and improve the quality of care. Proper documentation enables EMS agencies to collect and trend data to maximize operational performance, focus training and continued education efforts, provide membership feedback, and adjust medical protocols. Quality improvement (QI) strategies have been shown to enhance EMS performance and documentation.³ For collegiate-based EMS (CBEMS) organizations, confronted with rapid membership turnover and smaller call volumes, QI review processes may be especially critical in ensuring adequate documentation.

The Georgetown Emergency Response Medical Service (GERMS) is a basic life support (BLS) CBEMS agency in Washington, DC that serves Georgetown University and the surrounding community. GERMS is composed entirely of undergraduate student volunteers and responds to approximately 900 EMS calls annually. In 2014, a longstanding set of documentation checklists to assist providers with writing electronic patient care reports (ePCRs) were revised as part of the agency’s continuous quality improvement (QI) process. The agency’s student leaders and medical directors collaborated to revise the documentation checklists when a needs analysis found opportunities for improvement in the performance and documentation of appropriately-focused physical exams. 

Previously used checklists noted some criteria for commonly encountered chief complaints, but were outdated, inconsistent, and incomplete. The revised checklists incorporated more essential documentation elements for a wider range of chief complaints. Criteria for patient history, assessment, and intervention were organized separately in an easy-to-read format. GERMS has a QI committee that reviews each ePCR to measure compliance with current guidelines. Trends in patient documentation are shared at general membership meetings to highlight problems and areas for improvement. Prior to the adoption of these revised checklists, GERMS leadership oriented the membership during an annual Fall semester training workshop. The checklists were placed in the medical bags carried by members.

Objectives

The study aimed to determine if and to what extent QI measures such as documentation checklists can improve inclusion of key documentation criteria on ePCRs in a collegiate-based EMS system.

Methods

In this retrospective chart review, the investigators analyzed GERMS ePCRs captured via emsCharts (Warrendale, PA). Charts written by providers from August 1, 2013, to July 31, 2015 were reviewed and included for analysis if the reported chief complaint was seizure/syncope, abdominal pain, chest pain, or head/neck trauma. These chief complaints were specifically analyzed as they were identified as likely higher medico-legal risk categories by the GERMS QI committee. The stratification of risk was based on a consensus between GERMS’ medical director, QI officer, QI committee, and similar QI initiatives in the metropolitan region. Two reviewers (A.A. and B.M.) then independently scored each patient report according to the revised documentation standard implemented on June 1, 2014. The score was calculated as a percentage of chief complaint-based criteria that were correctly documented. For study analysis, each chart’s score was the average between the two reviewers, with a kappa statistic used to determine overall agreement. 

The required documentation elements for the four chief complaints analyzed are listed in Table 1. Documentation elements for each chief complaint are divided into history, assessment, and interventions. For each element, the reviewers were instructed to add 1 point if the providers documented the specific element in the chart narrative or 0 points if omitted. A score was then calculated as the percentage of total possible points. For this study, the reviewers examined only the free-text narrative portion of the chart.

Table 1. Expanded documentation guidelines (for chief complaints analyzed). 

Patient intervention criteria (in Table 1) were considered to be met if either performance of the intervention was specifically documented or an explanation for non-intervention was provided. For example, a patient with suspected head or neck trauma may not require immobilization if specifically noted in the chart to be inappropriate by the agency’s immobilization patient care protocols or online medical direction.

To assess the impact of the QI initiative on EMS documentation quality, ePCRs were divided into two groups (pre-intervention and post-intervention) based on date of submission. ePCRs completed prior to June 1, 2014 were assigned to the pre-intervention group while subsequent reports were considered post-intervention. This date marked the introduction of the updated documentation checklists to the GERMS membership.

For each chief complaint and study group, the scores of the individual ePCRs were averaged to compare overall documentation pre- and post-intervention. Documentation frequency of individual criteria were subsequently calculated and compared between the two study groups. Two-sample t tests were used to compare the two study groups with significance defined as P < 0.05. Inter-rater reliability was tested using the kappa statistic (Microsoft Excel, Redmond, WA). This study was reviewed and approved by the Georgetown University Institutional Review Board.

Results

A total of 373 charts met inclusion criteria and were analyzed in this study. A breakdown of the number of ePCRs per chief complaint analyzed in either the pre- or post-intervention phases is noted in Table 2.

Table 2. Number of electronic patient care reports (ePCRs) meeting inclusion criteria, specified by chief complaint.

Following the adoption of the updated documentation checklist, there were statistically significant increases in the overall inclusion of required documentation elements for all studied chief complaints: chest pain (69.29% to 81.31%, P = 0.006); abdominal pain (70.65% to 80.82%, P < 0.001); head trauma (80.47% to 86.39%, P < 0.001); seizure/syncope (78.3% to 84.67%, P = 0.012), as noted in Figure 1.

Figure 1. Compliance with required documentation elements.

Within each chief complaint, only some documentation elements showed statistically significant increases in frequency of documentation pre- and post-intervention (Table 3). The documentation of last oral intake for patients with abdominal pain slightly decreased after implementation of the revised checklists (P = 0.044).

Table 3. Comparison of frequency of documentation of required elements (Table 1) prior to and after the implementation of the expanded documentation guidelines. 

The study investigators’ inter-rater reliability was generally high, as noted by most kappa statistics being greater than 0.8 for these criteria (Table 3, Appendix S1). Kappa was less than 0.8 for a few elements, including: weakness/fatigue, location of pain, and radiation of pain (abdominal pain); nitroglycerin administration and call ahead to receiving facility (chest pain); pain on palpation and ALS dispatched (head trauma).

Discussion

Many barriers to effective EMS documentation exist. While the verbal handoff between EMS and emergency department providers is undoubtedly critical to patient care, it is often inaccurate, incomplete, or not heard. Information is lost amidst ongoing distractions, attempts to expedite care, and the need for provider-multitasking. Moreover, only about half of verbal information has shown to be retained by receiving ED staff even when structured handoff models are used.⁴ Prehospital written documentation ensures pertinent EMS information is communicated appropriately and available to the entire treatment team.

EMS providers receive little training on proper documentation. Producing a useful and relevant EMS chart is more complex than typing a narrative and checking boxes; learning how to collect and synthesize information, determine relevance, and recall necessary elements requires practice. Only 1.5 hours of the standard 110-hour EMT curriculum is specifically allocated to documentation.⁵ For collegiate-based EMS programs like GERMS, high turnover rates necessitate continuous training and reinforcement of documentation best-practices. The relatively low call volumes of CBEMS limits provider experience caring for and documenting certain chief complaints. Training time is also limited since members are typically full-time undergraduate students. Even on shift, members are often balancing academics and other extracurricular activities. Without adequate time and experience, explicit written instruction on documentation is invaluable.  

This study supports documentation checklists as an effective strategy to improve prehospital documentation in collegiate-based EMS. Within medicine, checklists are effective memory aids during stressful, time-sensitive situations such as cardiac resuscitation and rapid sequence intubation.^6-8 Aviation checklists have long been used to safeguard against inevitable human error.^9,10 Like aviation pilots, EMS providers are not immune to human error. Fatigue, stress, frequent distractions, and cognitive overload undoubtedly impact documentation quality. Delays in charting from long patient transfer times and multiple dispatches are frequent and lend to recall bias and documentation errors. A 2017 prospective, observational study revealed how documentation from memory can introduce significant error. Following completion of a simulated scenario, ten paramedic volunteers completed an ePCR before using a body-worn camera to check for accuracy and make changes. Serious documentation errors were made such as omission of patient suicidality or presence of weapons on scene.¹¹

While documentation guidelines are often incorporated into the EMS organization’s standard operating procedures (SOPs), checklists enhance guideline adherence.^12,13 However, the introduction of a checklist alone to improve documentation is likely insufficient. Orientation of such checklists to the prehospital providers who will be using them is a vital aspect of ensuring compliance.^14,15

In this study, inclusion of key documentation criteria significantly improved with the QI initiative and checklist revision. This suggests that the checklist revision and QI initiative increased provider compliance to current guidelines. Little inter-rater variability supports checklist-use as a reliable tool to assess documentation trends and provide performance-based feedback. The study’s analysis of the free-text narrative portion of the chart, which may result in more subjectivity, may explain areas of poor inter-rater variability.

Limitations

This study is limited by its retrospective study design. The data is also derived from a single agency, potentially limiting generalizability. 

Several uncontrolled variables may have influenced our results as well. Each year, GERMS leadership undergoes a complete turnover and new directors of training, continuing education, and operations are elected. Crew officers, who oversee patient care and documentation, also frequently change. In October 2014, a new medical director was appointed to serve GERMS and supervised the QI efforts – developed jointly with the GERMS leadership and both the incoming and outgoing medical directors – which were ultimately implemented in June 2014. After implementation, more focused attention was given to accurate, complete documentation. The scores of individual members were tracked providing an increased incentive to adhere to the documentation checklist. Areas where consistent deficits were identified were addressed with group-wide training.  

Lastly, although our study measured the quantitative improvement of documentation of checklist items, additional qualitative measures of PCR quality (e.g., clarity of narrative) were not evaluated in this study.

Conclusions

In this retrospective chart review, standardization and expansion of an existing set of documentation criteria was followed by an increase in both adherence to overall documentation standards and frequency of documentation of individual elements. Collegiate-based EMS agencies may consider checklists as a potential tool to improve the thoroughness of documentation.

Acknowledgements

The authors acknowledge the dedicated members of Georgetown Emergency Response Medical Service. 

References

1. Hagiwara MA, Magnusson C, Herlitz J, et al. Adverse events in prehospital emergency care: a trigger tool study. BMC Emerg Med. 2019;19(1):14. https://doi.org/10.1186/s12873-019-0228-3
2. Laudermilch DJ, Schiff MA, Nathens AB, Rosengart MR. Lack of emergency medical services documentation is associated with poor patient outcomes: a validation of audit filters for prehospital trauma care. J Am Coll Surg. 2009;210(2):220–227. https://doi.org/10.1016/j.jamcollsurg.2009.10.008
3. Joyce SM, Dutkowski KL, Hynes T. Efficacy of an EMS quality improvement program in improving documentation and performance. Prehosp Emerg Care. 1997;1(3):140-4. https://doi.org/10.1080/10903129708958807
4. Talbot R. Bleetman A. Retention of information by emergency department staff at ambulance handover: do standardised approaches work? Emerg Med J. 2007;24(8):539-542. https://doi.org/10.1136/emj.2006.045906
5. U.S. Department of Transportation National Highway Traffic Safety Administration. EMT-Basic: National Standard Curriculum. https://www.ems.gov/pdf/education/Emergency-Medical-Technician/EMT_Basic_1996.pdf. Accessed 25 May 2019.
6. Bloomer R, Burns BJ, Ware S. Improving documentation in prehospital rapid sequence intubation: investigating the use of a dedicated airway registry form. Emerg Med J. 2013;30(4):324-6. https://doi.org/10.1136/emermed-2011-200715
7. Fee-Mulhearn AL, DePriest J, Teleron A. A novel ACLS team leader checklist implemented to improve resuscitation efforts. Resuscitation. 2013;84(9):e115. https://doi.org/10.1016/j.resuscitation.2013.03.002
8. Mommers L, Keogh S. SPEEDBOMB: a simple and rapid checklist for Prehospital Rapid Sequence Induction. Emerg Med Australas. 2015;27(2):165-8. https://doi.org/10.1111/1742-6723.12363
9. Helmreich RL. On error management: lessons from aviation. BMJ. 2000;320(7237):781-5. https://doi.org/10.1136/bmj.320.7237.781
10. McCune K. Reducing error, improving safety. Doctors could certainly take lessons from aviation. BMJ. 2000;321(7259):506.
11. Ho JD, Dawes DM, McKay EM., et al. Effect of body-worn cameras on EMS documentation accuracy: a pilot study. Prehosp Emerg Care. 2017;21(2):263-271. https://doi.org/10.1080/10903127.2016.1218984
12. Chen C, Kan T, Li S, Qiu C, Gui L. Use and implementation of standard operating procedures and checklists in prehospital emergency medicine: a literature review. Am J Emerg Med. 2016;34(12):2432-2439. https://doi.org/10.1016/j.ajem.2016.09.057
13. Ko HC, Turner TJ, Finnigan MA. Systematic review of safety checklists for use by medical care teams in acute hospital settings–limited evidence of effectiveness. BMC Health Serv Res. 2011;11:211. https://doi.org/10.1186/1472-6963-11-211
14. Kerner T, Schmidbauer W, Tietz M, Marung H, Genzwuerker HV. Use of checklists improves the quality and safety of prehospital emergency care. Eur J Emerg Med. 2017;24(2):114-119. https://doi.org/10.1097/MEJ.0000000000000315
15. Ross DW, Schullek JR, Homan MB. EMS triage and transport of intoxicated individuals to a detoxification facility instead of an emergency department. Ann Emerg Med. 2013;61(2):175-84. https://doi.org/10.1016/j.annemergmed.2012.09.004

Supplementary Materials

Appendix S1: Complete version of Table 3 – Comparison of frequency of documentation of required elements prior to and after the implementation of the expanded documentation guidelines.

---

Author & Article Information

---

Avery S. Alatis, MD is an emergency medicine (EM) resident physician at Georgetown/MedStar Washington Hospital Center, assistant medical director for Georgetown Emergency Response Medical Services (GERMS), and previously served GERMS as a crew chief and vice president for education. Brian V. Monahan, MD is a general surgery resident physician at Temple University and previously served GERMS as a crew chief and captain. Allyson D. Raymond, MD is an EM resident physician at Temple University and previously served GERMS as a crew chief and vice president for education. Korin B. Hudson, MD, CAQ-SM is an associate professor of EM at Georgetown University School of Medicine and MedStar Georgetown University Hospital and has previously served as medical director for GERMS. Julie T. Vieth, MBChB is the director of education for EM and interim chair of the Emergency Department at Canton-Potsdam Hospital and has previously served as medical director for GERMS. Jose V. Nable, MD, NRP is an assistant professor of EM at Georgetown University School of Medicine and MedStar Georgetown University Hospital, in addition to serving as medical director for GERMS.

---

Author Affiliations: From Georgetown University / MedStar Washington Hospital Center Emergency Medicine Residency Program, Washington, DC, USA (A.S.A.); Georgetown Emergency Response Medical Service, Georgetown University, Washington, DC, USA (A.S.A., B.V.M., A.D.R., K.B.H., J.T.V., J.V.N.); Georgetown University School of Medicine, Washington, DC, USA (A.S.A., B.V.M., A.D.R., K.B.H., J.V.N.); Temple University General Surgery Residency Program, Philadelphia, PA, USA (B.V.M.); Temple University Emergency Medicine Residency Program, Philadelphia, PA, USA (A.D.R.); St. Lawrence Health System, Canton-Potsdam Hospital, Potsdam, NY, USA (J.T.V.); Department of Emergency Medicine, MedStar Georgetown University Hospital, Washington, DC, USA (K.B.H., J.V.N).
Address for Correspondence: Jose V. Nable, MD, MS, NRP. 3800 Reservoir Rd NW, Department of Emergency Medicine, G-CCC.,Washington, DC 20007.
Email: Jose.Nable@georgetown.edu
Conflicts of Interest/Funding Sources: By the JCEMS Submission Declaration Form, all authors are required to disclose all potential conflicts of interest and funding sources. All authors declared that they have no conflicts of interest. All authors declared that they did not receive funding to conduct the research and/or writing associated with this work.
Authorship Criteria: By the JCEMS Submission Declaration Form, all authors are required to attest to meeting the four ICMJE.org authorship criteria: (1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND (2) Drafting the work or revising it critically for important intellectual content; AND (3) Final approval of the version to be published; AND (4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Prior Presentation: Preliminary data from this study were presented in abstract form at Georgetown University School of Medicine’s Student Research Day on April 23, 2018 in Washington, DC, USA.
Submission History: Received May 28, 2019; accepted for publication February 2, 2020.
Published Online: February 28, 2020.
Published in Print: February 28, 2020 (Volume 3: Issue 1)
Reviewer Information: In accordance with JCEMS editorial policy, Original Research manuscripts undergo double-blind peer-review by at least two independent reviewers. JCEMS thanks the anonymous reviewers who contributed to the review of this work. 
Copyright: © 2020 Alatis, Monahan, Raymond, Hudson, Vieth & Nable. This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International (CC BY 4.0) License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. The full license is available at: https://creativecommons.org/licenses/by/4.0/
Electronic Link: https://doi.org/10.30542/JCEMS.2020.03.01.5