|Year : 2018 | Volume
| Issue : 10 | Page : 1323-1329
Comparing the interpretation of emergency department computed tomography between emergency physicians and attending radiologists: A multicenter study
R Guven1, Ali Haydar Akca2, C Caltili3, MI Sasmaz2, EK Kaykisiz1, S Baran4, L Sahin5, A Ari6, G Eyupoglu1, V Kirpat7
1 Department of Emergency Medicine, Bitlis State Hospital, Bitlis, Turkey
2 Department of Emergency Medicine, Faculty of Medicine, Yuzuncu Yil University, Van, Turkey
3 Department of Emergency Medicine, University of Health Sciences Bagcilar Training and Research Hospital, Istanbul, Turkey
4 Department of Emergency Medicine, Necip Fazil City Hospital, Kahramanmaras, Turkey
5 Department of Emergency Medicine, Bingöl State Hospital, Bingöl, Turkey
6 Department of Emergency Medicine, University of Health Sciences, Dr. Ersin Arslan Training and Research Hospital, Gaziantep, Turkey
7 Department of Emergency Medicine, University of Health Sciences, Van Training and Research Hospital, Van, Turkey
|Date of Acceptance||08-Jun-2018|
|Date of Web Publication||8-Oct-2018|
Dr. R Guven
Department of Emergency Medicine, Bitlis State Hospital, Bitlis
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Introduction: Computed tomography (CT) interpretation in the emergency department is one of the vital issues that should be carried out rapidly and accurately. The objective of this study was to examine the interpretation accuracy of emergency physicians (EPs) regarding CT scans at the emergency department for traumatic and nontraumatic purposes. Materials and Methods: The study that was carried out as a prospective, observational study was completed at four centers during 1 year. Results: Accuracy ratios of CT interpretations of EP regarding cranial injuries, thoracic injuries and vertebral spine injuries are above 95% in addition to the fact that the concordance's with the final result are perfect, the concordance with the final results of the CT interpretations of EP for abdominal injuries was moderate (<0.75). Accuracy ratios of the CT interpretations of EP for nontrauma patients were above 90% for brain hemorrhage and chest injuries and that the concordance with the final results was perfect (≥0.75). The CT interpretation accuracy rates of EP for spontaneous pneumothorax and aortic aneurysm/aortic dissection cases were 100%. CT interpretation rate of EP for pulmonary embolism was 89.4%, whereas the level of concordance with the final results was moderate (<0.75). Whereas the CT interpretation accuracy rates of EP for nontraumatic abdominal injuries varied between 83.3% and 93.1%, their levels of concordance with the final results were moderate (<0.75). Conclusions: The CT interpretations for abdominal traumatic patients in addition to pulmonary embolism and acute nontraumatic abdominal injuries should be carried out more carefully.
Keywords: Attending radiologists, computed tomography interpretation, emergency physicians, nontrauma emergencies, trauma emergencies
|How to cite this article:|
Guven R, Akca AH, Caltili C, Sasmaz M I, Kaykisiz E K, Baran S, Sahin L, Ari A, Eyupoglu G, Kirpat V. Comparing the interpretation of emergency department computed tomography between emergency physicians and attending radiologists: A multicenter study. Niger J Clin Pract 2018;21:1323-9
|How to cite this URL:|
Guven R, Akca AH, Caltili C, Sasmaz M I, Kaykisiz E K, Baran S, Sahin L, Ari A, Eyupoglu G, Kirpat V. Comparing the interpretation of emergency department computed tomography between emergency physicians and attending radiologists: A multicenter study. Niger J Clin Pract [serial online] 2018 [cited 2019 Mar 21];21:1323-9. Available from: http://www.njcponline.com/text.asp?2018/21/10/1323/242839
| Introduction|| |
Many patients apply to the emergency departments for traumatic or nontraumatic purposes., Rapid and accurate diagnosis has critical importance for this patient group. Radiological imagines are frequently used as an approach to such patients and computed tomography (CT) is one of them.
CT is used as a golden standard diagnostic tool especially for the evaluation of trauma patients. The diagnostic value of CT is very high in the diagnosis of nontrauma patients as well and it is mostly used to show vascular pathologies. Even though it plays such a critical role at emergency departments, it is still not clear who will read CTs and how they will be interpreted even in developed countries. Even though there are advanced communication systems today such as WhatsApp in addition to advanced imaging systems such as picture archiving and communication system, emergency physicians (EPs) still experience problems regarding the interpretation of tomographies at emergency departments. These problems may be listed as failure to reach the on-call radiologist, failure to read the CT on time even when the on-call radiologist is reached, not documenting the reports in an official manner, very low accuracy ratios for reports prepared by off-site reporters and the fact that other clinical departments make hospitalization and discharge decisions based only on., EPs who mostly have to manage critical patients by themselves may also sometimes be left alone with regard to CT interpretation. Hence, we evaluated the accuracy ratio of the interpretations of CTs by EP in this multi-centered prospective study.
| Materials and Methods|| |
Subjects and study period
This multi-centered, progressive observational study was carried out during the dates of June 10, 2016 – May 26, 2017 at times when the participating EPs were on-call. The study was carried out at a total of 4 central hospitals. Approval was acquired from the local ethics committee of the University of Health Sciences in order to carry out the study.
Exclusion criteria were determined as; patients below the age of 15, patients for whom the interpretations of the EP or the out of duty attending radiologist have not been included in the study form or have been noted down incompletely, patients who left the hospital without approval or patients who were transferred (since it would be difficult to have access to the final result) in addition to patients who were accepted from an outside center.
All centers that participated in the study were central hospitals of the related provinces which accept both trauma and nontrauma patients. Two of the centers that participated in the study used Toshiba 16-Slice tomography, one used Philips 16-Slice tomography and another used General Electric Healthcare 4 Slice (This center also used Toshiba 16-Slice tomography for angiographic images) CTs.
EPs in our country have 1 month of radiology rotation during their education and interpreting CT scans makes up a significant portion of this period. In addition, there are also regular theoretical courses at emergency congresses and symposiums.
Data collection and clinical outcomes of patient
CT images of the patients who applied to the emergency service with traumas or nontraumatic causes were first interpreted by the EP. This preliminary interpretation was noted down by the EP on the study form in the “interpretation of the EP” section. The clinical information of the patient was simultaneously forwarded to the antimicrobial resistance AR via phone/WhatsApp. AR accessed the images of the patient via digital imaging and communications in medicine viewer and forwarded his/her interpretation to the EP. The EP noted down the interpretation of the AR in the “interpretation of the AR” section of the study form. The study form including the clinical information of the patient along with the interpretations of the EP and AR were given to the team (final result team) comprised by an EP and an attending radiologist. The final result team examined the current tomography image of the patient in addition to other information of the patient (information on hospitalization if any, surgery notes, pathology results if any) thus deciding on the final diagnosis.
Classification was made for the trauma patients in the form as “yes” or “no” thus accepting diagnoses that are in accordance with the final result as “compatible” and contrary results as “discrepancy.” Anatomic localization of intracavity hemorrhage or classification differences in solid organ damage were not accepted as discrepancies.
Classification was made in the form for nontrauma patients as “yes” or “no” thereby accepting diagnoses that are in accordance with the final result as “compatible” and contrary results as “discrepancy.” Differences in intracerebral hemorrhage or localization, major/minor differences in pulmonary emboli risk classification, differences in aorta dissection type were not accepted as discrepancies.
Statistical analyses were performed using SPSS 22.0 (Armonk, NY, USA: IBM Corp.) and MedCalc Version 17.6 (MedCalc Software, Ostend, Belgium). Traumatic and nontraumatic CT interpretations of EP in addition to sensitivity, specificity, positive predictive value, negative predictive value, area under curve, accuracy rate, false negative, false positive and κ value levels for the AR were calculated according to the final result. The diagnosis compliance ratio (κ value) was classified according to Fleiss. Accordingly, κ ≥ 0.75 were accepted as perfect compliance, values between 0.40 and 0.75 were accepted as moderate compliance whereas values <0.40 were accepted as weak compliance.
| Results|| |
CTs of 1008 patients were taken throughout the study. Of these patients, 87 were excluded due to insufficient follow-up (transfer or leaving the hospital without approval), 52 patients were excluded due to insufficiencies of the preliminary interpretation of the EP or the attending radiologist and 81 patients were excluded because they were in the pediatric age group. Thus, a total of 788 patients were included in the study with 483 trauma patients (77.0% [n = 372] male and 23.0% [n = 151] female) in addition to 305 nontrauma patients (49.5% [n = 154] male and 50.5% [n = 151] female). The median age of the trauma patients who participated in the study was 39 (25%–75%, 26–54) whereas the median age of the nontrauma patients was 61 (25%–75%, 40–76). A total of 1598 CT scans were carried out on 788 patients throughout the study. The types of CT scans were as follows; a total of 1028 CT scans for trauma patients (280 cranial CT, 276 chest CT, 232 abdominal CT, 240 vertebral spine CT) and a total of 570 CT scans for nontrauma patients (92 cranial CT, 67 chest CT, 95 pulmonary CT angiography, 258 abdominal CT, 58 contrast-enhanced thoracoabdominal CT).
The most frequent reasons for application to the emergency service for trauma patients were falling from heights-crashing with 49.1% (n = 237) and traffic accidents with 43.5% (n = 210) which were followed by stabbing with 2.9% (n = 14), gunfire wounds with 2.3% (n = 11) and other injuries with 2.3% (n = 11).
Whereas it was observed when the reasons for emergency service application of nontrauma patients were examined that majority was due to abdominal-side pain with 48.8% (n = 149) which was followed by shortness of breath with 16.4% (n = 50), impaired consciousness with 15.1% (n = 46), chest pain with 5.9% (n = 18), head ache-loss of strength with 4.9% (n = 15) and nausea-vomiting with 3.9% (n = 12).
The CT interpretation sensitivity and specificity levels of EP were above 90% for traumatic injuries excluding abdominal injuries. Whereas the CT interpretation sensitivity ratios varied between 60% and 80% for abdominal injuries and the specificity ratios were above 95% [Table 1]. Whereas the κ values of EP in abdominal injuries are at moderate (0.40–0.75) levels, the agreement level of EP with the final result for other regions of the body was at a perfect level (≥0.75). It was determined that the agreement between the CT interpretations of AR for trauma patients with the final result were perfect (≥0.75) [Table 2].
|Table 1: Sensitivity, specificity, positive predictive value, negative predictive value and area under curve ratios of the interpretations of emergency physicians and attending radiologists according to the final result for trauma patient computed tomographys|
Click here to view
|Table 2: Accuracy rate, false negative, false positive and inter.rater agreement (κ value) ratios of the interpretations of emergency physicians and attending radiologists according to the final result for trauma patient computed tomographys|
Click here to view
CT interpretation sensitivity and specificity levels of EP in nontraumatic injuries excluding abdominal injuries and pulmonary embolism were above 90%. The CT interpretation sensitivity ratios in abdominal injuries vary between 60% and 80% with specificity ratios above 95%. The CT interpretation sensitivity ratio of EP for pulmonary embolism was 74.1% and the specificity ratio was 95.6% [Table 3]. The interpretations of EP regarding CT scans due to nontraumatic pathologies were concluded that the agreement between the final result and brain hemorrhage, spontaneous pneumothorax, thoracic other pathologic findings, cholecystitis/thickened gallbladder wall/cholelithiasis and aortic aneurysm/aortic dissection diagnoses were perfect (κ ≥ 0.75) (κ = 0.933;κ = 1.000; κ = 0.881;κ = 0.775;κ =1.000; respectively). The agreement between the final result and the EP diagnoses regarding pulmonary embolism, acute appendicitis, kidney and urinary tract stones, acute pancreatitis and abdominal other pathologic findings was at a moderate (0.40–0.75) level (κ = 0.729; κ = 0.627; κ = 0.719; κ = 0.630; κ = 0.588; respectively). The agreements of AR diagnoses with the final result were at a perfect level (κ ≥ 0.75) for all nontraumatic pathologies [Table 4].
|Table 3: Sensitivity, specificity, positive predictive value, negative predictive value and area under curve ratios of the interpretations of emergency physicians and attending radiologists according to the final result for non-trauma patient computed tomographys|
Click here to view
|Table 4: Accuracy rate, false negative, false positive and inter-rater agreement (κ value) ratios of the interpretations of emergency physicians and attending radiologists according to the final result for non-trauma patient computed tomographys|
Click here to view
| Discussion|| |
CT has become one of the most important diagnostic tools in modern emergency medicine understanding since it is effective, rapid and noninvasive. Kocher et al. carried out a retrospective study covering the years of 1996–2007 in which it was determined that the CT use in emergency services has increased by 330%.
When problems in accessing the radiologists and sloppy reports due to lack of official reporting are added to the current technical problems experienced in developing countries, the EP has to manage the patient according to his/her CT interpretation. In addition, reporting of CT scans at emergency departments vary from country to country and there is no standard approach. CTs of patients in European and North American countries are reported by off-site reporters who never see the patients. This may lead to erroneous CT reports. Autopsy studies have put forth that the discordance between the final result and radiological reporting may be as high as 20%. EPs who experience radiological reporting problems carried out various team studies recently for interpreting CT scans at emergency departments.
Head CT interpretation ratios between emergency medicine residents and emergency medicine physicians were examined in a study  in which radiologists were accepted as golden standard. A total of 544 head CT scans were examined in this study in which 35 (6.4%) false negatives and 53 (9.7%) false positive ratios were determined for emergency medicine residents whereas 74 (13.6%) false negatives and 12 (2.2%) false positive ratios were determined for emergency medicine physicians. Even though false negative ratios in nontraumatic head injuries were similar in our study, false positive ratios were determined to be much lower. The fact that our study had better false positive ratios in comparison with the results of the before mentioned study published in 2013 indicates that the CT interpretation skills of EP have increased.
Thoracic traumas are the 3rd most frequent reason for traumas in polytrauma patients after head and extremity traumas and the 2nd most frequent cause of death after head traumas. Kartal et al. carried out a study on emergency multi-traumatic patients in which the interpretation accuracies of on-call radiologists and EP were compared as a result of which EP had high accuracy ratios in chest injuries. Similarly, the kappa agreement levels between the final result and the EP interpretations as well as the agreement levels were high for traumatic chest injuries in our study.
CT, which has been used on trauma patients since 1970, is able to put forth the existence of organ injuries and intraabdominal hemorrhage with a much higher sensitivity thanks to technological developments. When the abdominal CT interpretation ratios of EP were examined in the study by Kartal et al., it was observed that sensitivity varied between 50% and 88% for solid organ injuries (liver, spleen, kidney), specificity varied between 97% and 100%, whereas sensitivity was 70% for abdominal hemorrhage and specificity was 97%. A moderate agreement level was obtained in the same study for solid organ injuries and intra-abdominal hemorrhages. We obtained lower sensitivity, similar κ values (moderate) in comparison with this study. The fact that EP received training from radiology clinic instructors before the study by Kartal et al. may be related with the higher sensitivity ratios they obtained in comparison with our study.
CT can place a higher ratio of diagnoses for vertebral spine fractures in comparison with plain radiography and this ratio is around 97%–100%. CT interpretation sensitivity ratios of EP were similar in our study for both cervical spinal injuries and thoracolumbar spine injuries and their agreement levels with the final result were perfect.
Even though the initial diagnostic tool for nontraumatic chest injuries is chest X-ray, chest CT may change the leading diagnosis and management decision at rates of 35%. It was observed in our study that chest CT is an important diagnostic tool for nontraumatic pulmonary injuries and that the CT interpretation accuracy rates were high.
Pulmonary embolism which is one of the life-threatening cases is among vascular emergencies that requires rapid diagnosis and treatment. Hochhegger et al. carried out a study in which they examined 123 CT pulmonary angiographies and compared the interobserver agreement ratios between radiology residents and EP for the Detection of Pulmonary Embolism. Whereas it was determined in this study that accepts thoracic radiologists as the golden standard that there was a very good agreement between radiology residents and thoracic radiologists, a fair-moderate agreement was determined between thoracic radiologist and EP. The agreement levels of EP with the final result were moderate in our study whereas this level was very good for AR as was the case in the study by Hochhegger et al. The fact that the agreement levels of EP in our study were lower than those for AR was due to high false negative ratios. Even though we did not verify this, almost all false negatives of the EP who participated in the study were pulmonary embolism due to subsegmental or peripheral pulmonary arteries.
Non-traumatic abdominal pains make up 5%–10% of the causes of applying to emergency services. Kang et al. carried out a study on the accuracy of CT interpretations of emergency medicine residents for nontraumatic acute abdomen patients. Whereas the discrepancy ratio of emergency resident preliminary reports was 16.7%, this ratio was determined as 12.2% for radiology residents.
When the nontraumatic CT interpretation discrepancy ratios determined in our study were examined, the largest difference between EP and AR was determined in abdominal CT's. Even though the nontraumatic abdominal CT accuracy ratios of AR were lower in comparison with other regions of the body, we observed that this ratio decreased significantly for EP. The reason for this may be due to the fact that EP use ultrasound sonography more actively for acute abdominal cases in recent years. that they do not give the required importance to nontraumatic abdominal CT interpretation as well as the difficulties related with abdominal anatomy.
CT devices with 16 slice detectors were used in our study for patients with aortic dissection suspicions. We came across a total of 8 Aortic aneurysm/Aortic dissections one of which was traumatic with 7 nontraumatic. EP were able to place correct diagnoses to all these patients.
The fact that consultants decided on hospitalization or discharge depending on the interpretation of the attending radiologist and the inability to carry out prospective follow-ups of patients discharged from the emergency department were the fundamental limitations of our study. Except this, this study has more several limitations. The first one is that there was no discrimination of CT-based classification of solid organ injuries and other traumatic injuries. The second one is that the working experience of EP or AR were different from each other.
| Conclusions|| |
In conclusion, the CT interpretation skills of EPs were perfect in all body injuries excluding traumatic abdominal injuries, nontraumatic abdominal injuries and pulmonary embolism. Whereas the CT interpretation levels of radiologists were better in comparison with those of the EPs. Standard solutions should be put into effect including EPs, radiologists and hospital administrators for minimizing errors in the interpretation of CTs at emergency departments where critical patients are managed.
We would like to thank the AR and radiology technicians at all centers where the study was carried out for their support and contributions. In addition, we would also like to put forth our gratitude to emergency service auxiliary health staff (nurses, paramedics, health clerks) for their support in ensuring that the study is carried out smoothly.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Zhang E, Hung SC, Wu CH, Chen LL, Tsai MT, Lee WH, et al.
Adverse event and error of unexpected life-threatening events within 24hours of ED admission. Am J Emerg Med 2017;35:479-83.
Chang BP, Cain D, Mitroff SR. Emergency department crowding associated with differences in CXR interpretations between emergency physicians and radiologists. Am J Emerg Med 2017;35:793-4.
Berdahl CT, Vermeulen MJ, Larson DB, Schull MJ. Emergency department computed tomography utilization in the United States and Canada. Ann Emerg Med 2013;62:486-94.e3.
Rose JS, Levitt MA, Porter J, Hutson A, Greenholtz J, Nobay F, et al.
Does the presence of ultrasound really affect computed tomographic scan use? A prospective randomized trial of ultrasound in trauma. J Trauma 2001;51:545-50.
Akgün V, Battal B. CT Angiography in Intracranial Vascular Pathologies. Turk Klin J Radiol Spec Top 2012;5:78-85.
Howlett DC, Drinkwater K, Frost C, Higginson A, Ball C, Maskell G, et al.
The accuracy of interpretation of emergency abdominal CT in adult patients who present with non-traumatic abdominal pain: Results of a UK national audit. Clin Radiol 2017;72:41-51.
DeCorato DR, Kagetsu NJ, Ablow RC. Off-hours interpretation of radiologic images of patients admitted to the emergency department: Efficacy of teleradiology. AJR Am J Roentgenol 1995;165:1293-6.
Brady AP. Error and discrepancy in radiology: Inevitable or avoidable? Insights Imaging 2017;8:171-82.
Fleiss JL. Statistical Methods for Rates and Proportions. 2nd
ed. New York, NY: John Wiley & Sons Inc.; 1981.
Larson DB, Johnson LW, Schnell BM, Salisbury SR, Forman HP. National trends in CT use in the emergency department: 1995-2007. Radiology 2011;258:164-73.
Kocher KE, Meurer WJ, Fazel R, Scott PA, Krumholz HM, Nallamothu BK, et al.
National trends in use of computed tomography in the emergency department. Ann Emerg Med 2011;58:452-62. e3.
Mills AM, Raja AS, Marin JR. Optimizing diagnostic imaging in the emergency department. Acad Emerg Med 2015;22:625-31.
Wachter RM. Why diagnostic errors don't get any respect – And what can be done about them. Health Aff (Millwood) 2010;29:1605-10.
Arhami Dolatabadi A, Baratloo A, Rouhipour A, Abdalvand A, Hatamabadi H, Forouzanfar M, et al.
Interpretation of computed tomography of the head: Emergency physicians versus radiologists. Trauma Mon 2013;18:86-9.
Safari S, Baratloo A, Negida AS, Sanei Taheri M, Hashemi B, Hosseini Selkisari S, et al.
Comparing the interpretation of traumatic chest X-ray by emergency medicine specialists and radiologists. Arch Trauma Res 2014;3:e22189.
Kartal ZA, Kozacı N, Çekiç B, Beydilli İ, Akçimen M, Güven DS, et al.
CT interpretations in multiply injured patients: Comparison of emergency physicians and on-call radiologists. Am J Emerg Med 2016;34:2331-5.
Rizzo AG, Steinberg SM, Flint LM. Prospective assessment of the value of computed tomography for trauma. J Trauma 1995;38:338-42.
Ensrud KE, Schousboe JT. Clinical practice. Vertebral fractures. N
Engl J Med 2011;364:1634-42.
Gezer NS, Balcı P, Tuna KÇ, Akın IB, Barış MM, Oray NÇ, et al.
Utility of chest CT after a chest X-ray in patients presenting to the ED with non-traumatic thoracic emergencies. Am J Emerg Med 2017;35:623-7.
Fedullo PF, Tapson VF. Clinical practice. The evaluation of suspected pulmonary embolism. N
Engl J Med 2003;349:1247-56.
Hochhegger B, Alves GR, Chaves M, Moreira AL, Kist R, Watte G, et al.
Interobserver agreement between radiologists and radiology residents and emergency physicians in the detection of PE using CTPA. Clin Imaging 2014;38:445-7.
Kamin RA, Nowicki TA, Courtney DS, Powers RD. Pearls and pitfalls in the emergency department evaluation of abdominal pain. Emerg Med Clin North Am 2003;21:61-72, vi.
Kang MJ, Sim MS, Shin TG, Jo IJ, Song HG, Song KJ, et al.
Evaluating the accuracy of emergency medicine resident interpretations of abdominal CTs in patients with non-traumatic abdominal pain. J Korean Med Sci 2012;27:1255-60.
Laméris W, van Randen A, van Es HW, van Heesewijk JP, van Ramshorst B, Bouma WH, et al.
Imaging strategies for detection of urgent conditions in patients with acute abdominal pain: Diagnostic accuracy study. BMJ 2009;338:b2431.
[Table 1], [Table 2], [Table 3], [Table 4]