If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
AUC comparisons did not reveal significant differences between the scoring tools.
•
EMSE-EAL-40 tended towards highest diagnostic accuracy.
•
END-IT-3 showed most balanced sensitivity-specificity ratio.
Abstract
Purpose
Several scoring tools have been developed for the prognostication of outcome after status epilepticus (SE). In this study, we compared the performances of STESS (Status Epilepticus Severity Score), mSTESS (modified STESS), EMSE-EAL (Epidemiology-based Mortality Score in Status Epilepticus- Etiology, Age, Level of Consciousness) and END-IT (Encephalitis-NCSE-Diazepam resistance-Image abnormalities-Tracheal intubation) in predicting in-hospital mortality after SE.
Method
Data collected retrospectively from a cohort of 287 patients with SE were used to calculate STESS, mSTESS, EMSE-EAL, and END-IT scores. The differences between the scores’ performances were determined by means of area under the ROC curve (AUC) comparisons and McNemar testing.
Results
The in-hospital mortality rate was 11.8%. The AUC of STESS (0.628; 95% confidence interval (CI), 0.529–0.727) was similar to that of mSTESS (0.620; 95% CI, 0.510–0.731), EMSE-EAL (0.556; 95% CI, 0.446–0.665), and END-IT (0.659; 95% CI, 0.550–0.768; p > .05 for each comparison) in predicting in-hospital mortality. STESS with a cutoff of 3 was found to have lowest specificity and number of correctly classified episodes. EMSE-EAL with a cutoff at 40 had highest specificity and showed a trend towards more correctly classified episodes while sensitivity tended to be low. END-IT with a cutoff of 3 had the most balanced sensitivity-specificity ratio.
Conclusions
EMSE-EAL is as easy to calculate as STESS and tended towards higher diagnostic accuracy. Adding information on premorbid functional status to STESS did not enhance outcome prediction. END-IT was not superior to other scores in prediction of in-hospital mortality despite including information of diagnostic work-up and response to initial treatment.
Status epilepticus represents a challenging condition in which therapy needs to be balanced between both the risks of over- and undertreatment. While the first may lead to iatrogenic harm [
]. Early knowledge on the prognosis of an SE episode might help differentiate patients in need of aggressive therapy from those in whom a conservative approach is justifiable [
]. To date, four prediction tools have been created aiming to allow for prognosis after SE based on different sets of prognosticators: 1) Status Epilepticus Severity Score (STESS) [
]. In this study, these four scores were compared in an attempt to evaluate their predictive accuracy for in-hospital mortality in a cohort of SE patients.
] and in line with previous studies, SE was defined as clinical and/or electroencephalographic evidence of seizure activity for ≥5 min or as series of seizures with incomplete interictal clinical recovery [
]. SE secondary to hypoxic encephalopathy was excluded from this study, so were recurrent SE episodes. The outcome measure was death during hospital stay.
2.2 Score calculations
STESS and mSTESS were calculated as proposed by their developers [
]. Regarding EMSE, Leitinger et al. assessed models including six variables for their prognostic value in SE and found highest performance in a score including four domains: etiology (E; grouped into 15 categories), age (A; stratified in 10-year intervals), comorbidities (C), and EEG data (E) (=EMSE-EACE), while level of consciousness (L) and duration of SE (D) did not increase the diagnostic value of the models [
]. Because of partly incomplete data on comorbidities and/or EEG in our patients, we chose to apply EMSE-EAL in the present study. In patients with competing SE etiologies, the most severe one according to EMSE was considered for score calculations. Patients with underlying etiology not represented in EMSE were not assigned an EMSE-EAL score. In terms of the END-IT score, the item “Diazepam resistance” was replaced by “refractoriness to a first line of medication”, particularly as 1) diazepam is not the benzodiazepine of first choice in the treatment of SE in our institution and 2) -despite generally accepted guidelines- not all patients receive a benzodiazepine as first SE treatment [
]. With regards to cerebral imaging, cerebral microangiopathy, amyloid angiopathy, and generalized atrophy were not interpreted as imaging lesions responsible for an SE episode. Patients without imaging data did not receive an END-IT score.
2.3 Statistical analysis
Statistical analyses were performed using IBM SPSS Statistics 22.0 (http://www.spss.com), GraphPad Prism 7.0 (www.graphpad.com) and Medcalc 17.9.7 (www.medcalc.org). Two sided p values of less than 0.05 were considered statistically significant. For each analyzed score, a receiver operating characteristics (ROC) curve was generated. The resulting areas under the curves (AUCs) were then compared to assess score performances using the method by Hanley and McNeil [
]. The ROC curves were furthermore used to determine optimal cutoff values for in-hospital mortality via the Youden index. Based on the identified cutoff values, sensitivity, specificity, positive and negative predictive values (PPV, NPV), and the rate of correctly classified episodes were calculated for each score. Given controversial results in the current literature, for the STESS these calculations were performed for the cutoff points 3 and 4, regardless of which one was the identified optimal cutoff value. McNemar test was used to compare sensitivities, specificities, and the rates of correctly classified cases [
]. Only patients in whom the respective scores could be calculated were included into the pairwise statistical comparisons of two scoring tools.
3. Results
3.1 Study cohort
We identified 362 SE episodes in our databases in the 8 year period from 2007 to 2014. After exclusion of recurrent episodes, 287 cases remained for final analysis. Table 1 and Fig. 1 give an overview of the patient cohort. Data on age, premorbid modified Rankin Scale (mRS) score, underlying SE etiology, and history of previous seizures were complete, thus all patients could be given STESS and mSTESS scores. In 16/287 (5.6%) episodes, the underlying SE etiology was not represented in the EMSE. This concerned patients with SE provoked by systemic infection (n = 9), patients suffering from progressive neurodegenerative disease (n = 5), and SE associated with application of contrast agents or chemotherapy (n = 2). These patients could therefore not be assigned EMSE-EAL scores. A total of 34/287 (11.8%) patients lacked imaging data in our electronic database because they either did not receive imaging or because it was performed in another hospital before patients were transferred to our institution. Therefore, the END-IT score could only be calculated in the remaining 253 patients. Of those 101/253 (39.9%) received MRI imaging and the remaining 152/253 (60.1%) CT. 29/287 (10.1%) patients were not administered a benzodiazepine as the first line of therapy. In most of these cases (n = 17) the first AED applied was levetiracetam, and a few patients received phenytoin, valproic acid, lacosamide, or anesthetic AEDs as first-line therapy.
Table 1Overview of study cohort.
Total cohort (n = 287)
Demographics
Female gender
165 (57.5%)
Age on admission, y
71 (58–79)
Premorbid mRS
3 (1–4)
Status epilepticus characteristics
Etiology
Acute symptomatic
160 (55.7%)
Remote symptomatic
59 (20.6%)
Progressive symptomatic
42 (14.6%)
Unknown
26 (9.1%)
History of seizures
156 (54.4%)
Encephalitis
21 (7.3%)
Worst seizure type before initiation of treatment
Simple partial or complex partial
135 (47.0%)
Generalized convulsive
131 (45.6%)
NCSE in coma
21 (7.3%)
Neuroradiological data
Cerebral imaging findings available
253 (88.2%)
Treatment
Tracheal intubation
90 (31.4%)
Refractoriness to 1st and 2nd line treatment
112 (39.0%)
Outcome
In-hospital mortality
34 (11.8%)
Values are n (%) or median (interquartile range). Abbreviations: mRS, modified Rankin Scale; NCSE, Nonconvulsive Status Epilepticus in Coma.
Abbreviations: SE, Status epilepticus; STESS, Status Epilepticus Severity Score; mSTESS, modified STESS; EMSE-EAL, Epidemiology-based Mortality Score in Status Epilepticus − Etiology, Age, Level of Consciousness; END-IT, Encephalitis-NCSE-Diazepam resistance-Image abnormalities-Tracheal intubation.
3.2 Comparison of AUCs for prediction of in-hospital mortality
The statistical comparison of STESS and mSTESS included all patients (n = 287), while comparison of EMSE-EAL and END-IT with STESS and mSTESS was performed in the subset of patients with available EMSE-EAL (n = 271) and END-IT scores (n = 253) respectively. The comparison of EMSE-EAL with END-IT included patients with available scores for those two scoring tools (n = 240). The ROC curves for the prediction of in-hospital mortality are depicted in Fig. 2. AUCs were similar for STESS (0.628; 95% CI, 0.529–0.727), mSTESS (0.620; 95% CI, 0.510–0.731), END-IT (0.659; 95% CI, 0.550–0.768), and EMSE-EAL (0.556; 95% CI, 0.446–0.665) with p > .05 for each comparison.
Fig. 2Receiver operating characteristic curves of STESS, mSTESS, EMSE-EAL, and END-IT for the prediction of in-hospital mortality.
Abbreviations: STESS, Status Epilepticus Severity Score; mSTESS, modified STESS; EMSE-EAL, Epidemiology-based Mortality Score in Status Epilepticus – Etiology, Age, Level of Consciousness; END-IT, Encephalitis-NCSE-Diazepam resistance-Image abnormalities-Tracheal intubation; AUC, area under the curve; CI, confidence interval
3.3 Comparison of score performance using optimal cutoff points for in-hospital mortality
The optimal cutoff points for prediction of in-hospital mortality were 5 for mSTESS, 3 for END-IT, and 40 for EMSE-EAL. With regards to the STESS, the cutoff points 3 and 4 yielded practically identical Youden indices. Fig. 3 gives a graphical overview of performances of the scores using these cutoff values (i.e., STESS-3, STESS-4, mSTESS-5, END-IT-3, and EMSE-EAL-40). Pairwise comparisons using McNemar’s test revealed STESS-3 to show significantly lower specificity than the other scores (n = 287: 45.8 vs 68.8% (STESS-4), vs 67.2% (mSTESS-5); n = 271: 47.3 vs 77.0% (EMSE-EAL-40); n = 253: 43.8 vs 64.3% (END-IT-3); see Table 2). Similarly, STESS-3 correct classification rates were significantly lower than those of all other scores (n = 287: 49.1 vs 66.7% (STESS-4), vs 65.9% (mSTESS-5); n = 271: 50.2 vs 72.3% (EMSE-EAL-40); n = 253: 47.4 vs 64.4% (END-IT-3)). EMSE-EAL-40 showed the significantly highest specificity (n = 271: 77.0 vs 47.3% (STESS-3), vs 68.6% (STESS-4), vs 66.9% (mSTESS-5); n = 240: 75.1 vs 63.4% (END-IT-3)) and either had or tended towards highest rates of correct classification (n = 271: 72.3 vs 50.2% (STESS-3), vs 66.4% (STESS-4), vs 65.3% (m-STESS-5); n = 240: 71.3 vs 63.8% (END-IT-3)). END-IT-3 showed the most balanced sensitivity-specificity ratio. The positive and negative predictive values of all scores were very similar.
Fig. 3Performance characteristics for prediction of in-hospital mortality.
Values for sensitivities, specificities, positive/negative predictive values, and rate of correct classification rates are for patients in whom the respective scores could be calculated (STESS-3, STESS-4, mSTESS: n = 287; EMSE-EAL: n = 271; END-IT: n = 253).
Abbreviations: STESS, Status Epilepticus Severity Score; mSTESS, modified STESS; EMSE-EAL, Epidemiology-based Mortality Score in Status Epilepticus – Etiology, Age, Level of Consciousness; END-IT, Encephalitis-NCSE-Diazepam resistance-Image abnormalities-Tracheal intubation;
Table 2Pairwise comparisons of sensitivities, specificities, and correct classification rates.
A) Sensitivities (%)
STESS-3
73.5 vs 50.0 (p=.008)a
STESS-4
73.5 vs 55.9 (p = .146)a
50.0 vs 55.9 (p = .727)a
mSTESS-5
71.9 vs 37.5 (p=.003)b
50.0 vs 37.5 (p = .219)b
53.1 vs 37.5 (p = .125)b
EMSE-EAL-40
75.9 vs 65.5 (p = .549)c
51.7 vs 65.5 (p = .424)c
55.2 vs 65.5 (p = .549)c
40.7 vs 66.7 (p = .092)d
END-IT-3
B) Specificities (%)
STESS-3
45.8 vs 68.8 (p<.001)a
STESS-4
45.8 vs 67.2 (p<.001)a
68.8 vs 67.2 (p = .672)a
m-STESS-5
47.3 vs 77.0 (p<.001)b
68.6 vs 77.0 (p=.010)b
66.9 vs 77.0 (p=.007)b
EMSE-EAL-40
43.8 vs 64.3 (p=.002)c
67.4 vs 64.3 (p = .530)c
65.6 vs 64.3 (p = .844)c
75.1 vs 63.4 (p=.003)d
END-IT-3
C) Correct classification rates (%)
STESS-3
49.1 vs 66.7 (p<.001)a
STESS-4
49.1 vs 65.9 (p<.001)a
66.7 vs 65.9 (p < .896)a
m-STESS-5
50.2 vs 72.3 (p<.001)b
66.4 vs 72.3 (p = .056)b
65.3 vs 72.3 (p=.045)b
EMSE-EAL-40
47.4 vs 64.4 (p<.001)c
65.6 vs 64.4 (p = .845)c
64.4 vs 64.4 (p = 1.000)c
71.3 vs 63.8 (p = .057)d
END-IT-3
Comparisons should be read from left to right. Values are (A) sensitivities (%), (B) specificities (%), (C) correct classification rates (%) of the column-defining vs the row-defining scoring tools with the respective p-values from McNemar test in parentheses. Values are for the subsets of patients with scores available for the two scoring tools compared (a) n = 287, b) n = 271, c) n = 253, d) n = 240). Statistically significant values (p < .05) are expressed in bold.
Abbreviations: STESS, Status Epilepticus Severity Score; mSTESS, modified STESS; EMSE-EAL, Epidemiology-based Mortality Score in Status Epilepticus − Etiology, Age, Level of Consciousness; END-IT, Encephalitis-NCSE-Diazepam resistance-Image abnormalities-Tracheal intubation.
In the present study, we examined scoring tools for the prediction of in-hospital mortality in SE. We compared AUCs and used optimal cutoff values to calculate and assess the sensitivity, specificity, and the rate of correctly classified episodes for each score. Several aspects of our findings require discussion.
STESS was the first score developed for the prediction of in-hospital mortality in SE and it includes overall fewer and/or less detailed clinical information than all the scoring tools that were introduced afterwards. However, comparison of AUCs did not show it to perform worse than the newer scores. In particular, we did not find adding information on premorbid functional status, as proposed in mSTESS, to increase prognostic performance. That our results here are not in line with those of Gonzalez-Cuevas et al. might be explained by the overall higher premorbid mRS we observed in our cohort (3 vs 1 mRS points) [
]. Although potentially indicative of frailty and as such a rather negative predictor of outcome, higher premorbid mRS can also point toward remote symptomatic SE etiologies which, in comparison to acute ones, positively impact outcome as they are associated with lower risk of treatment refractoriness [
]. Therefore, in SE higher premorbid mRS may not necessarily correlate with mortality.
END-IT differs from all other scores as it includes data not readily available at the beginning of an SE episode but requiring diagnostic work-up and information on treatment response. This extra information did, however, not lead to superiority of this score over the others, which is surprising, particularly as previous studies found significant impact of imaging findings and treatment refractoriness on outcome in SE [
]. There seem to be several reasons for why the score did not perform better in our patients. First, END-IT was created based on a cohort of very young ICU patients with approximately a third having SE caused by encephalitis and many requiring mechanical ventilation [
]. Therefore, the score does not ideally apply to a hospital based Central European SE cohort with a median age almost 50 years higher and encephalitis underlying SE in only 7.3% of cases. Second, the imaging item of END-IT does not take into account the acuteness of cerebral lesions. This reduces the value of this prognosticator because remote lesions causing SE lead to a higher score although they may not necessarily represent negative outcome predictors [
While the AUCs were similarly low for all scores, comparison of test performances using optimized cutoff values for discrimination of survivors and nonsurvivors yielded differences deserving further discussion, particularly with regards to the STESS.
Rossetti et al. initially proposed that STESS could be used to guide therapy in SE and that patients with values of below 3 should probably not be treated aggressively as in those the risk of therapy may outweigh its benefits. However, uncertainty exists as to which cutoff point best differentiates survivors from nonsurvivors. Studies yielded controversial results, and while Rossetti et al. initially proposed a cutoff at 3, others found higher performance of STESS-4 [
]. In our cohort, while having high sensitivity, STESS-3 showed significantly lower specificity and worse overall diagnostic accuracy compared to STESS-4 and all the other scores. Therefore, our results favor the use of 4 as STESS cutoff for prediction of in-hospital mortality.
In addition to the problems regarding cutoff values, STESS was furthermore reported to carry the risk of a ceiling effect [
], particularly in prediction of mortality of older patients with no history of previous seizures, as these per se are likely to score high on STESS. Risks of such an effect were claimed to be lower for EMSE, but due to its complexity, the score’s practicability in clinical routine was questioned [
]. Mainly because of partly incomplete data, we chose to use a modified version of EMSE including 3 variables which are all also represented in STESS, however categorized in greater detail in EMSE. Still, EMSE-EAL can be calculated almost as easily as STESS, and given its high overall predictive accuracy, the score may represent a reasonable STESS alternative. However, EMSE-EAL-40 sensitivity tended to be lower compared to the other scores and we found that several SE etiologies are not represented in the EMSE. This category may therefore require expansion. The optimal cutoff we found was somewhat higher than in a study from Argentina [
], possibly reflecting higher chance of surviving hospital stay in our cohort (88.2 vs 72%). Further studies are needed to identify the best EMSE-EAL cutoff value for SE cohorts from Europe.
With the exception of END-IT, all scores examined in this study showed more or less pronounced imbalance between sensitivity and specificity. However, despite its favorable sensitivity-specificity ratio, END-IT was not found to have higher overall diagnostic accuracy.
Our study has several limitations including the retrospective single-center design. Not all patients could be assigned EMSE-EAL and END-IT scores and therefore statistical calculations were not all performed on the same exact cohort. For 34 patients we lacked data on cerebral imaging which may have influenced our findings. We used a modified END-IT version and replaced refractoriness to diazepam by refractoriness to any first line of medication. Furthermore, END-IT was originally designed to predict outcome at 3 months after discharge, while we looked at mortality during hospital stay. Moreover, the END-IT cohort only included convulsive SE, while in our study all semiologies were considered. The version of EMSE we assessed in this study differs from the one proposed by the EMSE developers and our findings therefore only apply for EMSE-EAL which, of note, the original investigators found to have a lower PPV than EMSE-EACE [
To conclude, STESS-3 was associated with the significantly lowest rates of correct outcome prediction. When using 4 as cutoff for in-hospital mortality prognostication, however, STESS did not perform worse than the other scores. We found no benefit of adding premorbid functional status to STESS. EMSE-EAL-40 tended to yield the best rate of correctly classified episodes and might therefore be considered a reasonable alternative to STESS-4. Despite including information on cerebral imaging and response to treatment, END-IT did not perform better than the other scores, probably as a result of lacking differentiation between acute and remote cerebral lesions and an emphasis on variables not ideal to assess outcome in a hospital based Central European SE cohort.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Authors’ contributions
CR and DM: study concept and design, acquisition, analysis and interpretation of data, statistical analysis, manuscript drafting. HBH study concept and supervision, data interpretation, manuscript drafting. RUK, MIS, JAS, TM, HMH: data acquisition, critical revision for important intellectual content.
The present work was performed in partial fulfillment of the requirements for obtaining the degree “Dr. med.” for RUK.
All authors have read the manuscript, agreed with the contents, and approved the final version of the manuscript.
Conflicts of interest statement
The authors report no conflicts of interest.
References
Sutter R.
Marsch S.
Fuhr P.
Kaplan P.W.
Ruegg S.
Anesthetic drugs in status epilepticus: risk or rescue? A 6-year cohort study.
30820-8&id=gr1.jpg){kind=link}
30820-8&id=gr2.jpg){kind=link}
30820-8&id=gr3.jpg){kind=link}