Advertisement

Seizure detection devices for use in antiseizure medication clinical trials: A systematic review

Open ArchivePublished:February 13, 2019DOI:https://doi.org/10.1016/j.seizure.2019.02.007

      Highlights

      • Seizure detection devices may be useful for reporting ASM trial primary endpoints.
      • The highest performing devices are SPEAC (GTCs) and a wireless EEG device (FIAs).
      • Some devices are also capable of reporting secondary endpoints in ASM trials.
      • Recommendations are put forth to address limitations in device validation studies.
      • A framework is suggested to streamline testing before device use in ASM trials.

      Abstract

      Objective

      This study characterizes the current capabilities of seizure detection device (SDD) technology and evaluates the fitness of these devices for use in anti-seizure medication (ASM) clinical trials.

      Methods

      Through a systematic literature review, 36 wireless SDDs featured in published device validation studies were identified. Each device’s seizure detection capabilities that addressed ASM clinical trial primary endpoint measurement needs were cataloged.

      Results

      The two most common types of seizures targeted by ASMs in clinical trials are generalized tonic-clonic (GTC) seizures and focal with impaired awareness (FIA) seizures. The Brain Sentinel SPEAC achieved the highest performance for the detection of GTC seizures (F1-score = 0.95). A non-commercial wireless EEG device achieved the highest performance for the detection of FIA seizures (F1-score = 0.88).

      Discussion

      A preliminary assessment of device capabilities for measuring selected ASM clinical trial secondary endpoints was performed.
      The need to address key limitations in validation studies is highlighted in order to support future assessments of SDD fitness for ASM clinical trial use. In tandem, a stepwise framework to streamline device testing is put forth. These suggestions provide a starting point for establishing SDD reporting requirements before device integration into ASM clinical trials.

      Keywords

      1. Introduction

      The evaluation of anti-seizure medication (ASM) efficacy during clinical trials hinges on collecting accurate and consistent patient-reported outcome measures (PROMs) [
      • Fisher R.S.
      Bad information in epilepsy care.
      ,
      • Blachut B.
      • Hoppe C.
      • Surges R.
      • Elger C.
      • Helmstaedter C.
      Subjective seizure counts by epilepsy clinical drug trial participants are not reliable.
      ]. The FDA relies on patient-reported seizure frequency, often recorded in an electronic or paper diary format, as the primary endpoint for assessing and approving ASMs [
      • Katz R.
      Issues in clinical trial design from the FDA perspective.
      ,
      • Ben-Menachem E.
      • Sander J.W.
      • Privitera M.
      • Gilliam F.
      Measuring outcomes of treatment with antiepileptic drugs in clinical trials.
      ].
      Many patients and their caregivers, however, are neither able to provide reliable seizure counts nor able to recollect important seizure characteristics, including seizure length or type [
      • Ferastraoaru V.
      • Goldenholz D.M.
      • Chiang S.
      • Moss R.
      • Theodore W.H.
      • Haut S.R.
      Characteristics of large patient‐reported outcomes: where can one million seizures get us?.
      ]. One study showed that adults with epilepsy failed to report over 85% of their seizures during the night, and up to 50% of their seizures during the day [
      • Hoppe C.
      • Poepel A.
      • Elger C.E.
      Epilepsy: accuracy of patient seizure counts.
      ]. As a result, seizure frequency is often underreported, which may present a significant barrier for assessing ASM efficacy [
      • Ferastraoaru V.
      • Goldenholz D.M.
      • Chiang S.
      • Moss R.
      • Theodore W.H.
      • Haut S.R.
      Characteristics of large patient‐reported outcomes: where can one million seizures get us?.
      ].
      Introducing seizure detection devices (SDDs) into ASM clinical trials may help address these patient and caregiver-based seizure reporting challenges [
      • Ryvlin P.
      • Beniczky S.
      Seizure detection and mobile health devices in epilepsy: update and future developments.
      ]. SDD technology continues to advance, with the FDA issuing 510(k) clearances for two wearable SDDs [,
      • Medical Devices; Neurological Devices
      Classification of the non-electroencephalogram physiological signal based seizure monitoring system.
      ]. Recent research has also resulted in the publication of several validation studies that assess device detection performance and additional reporting capabilities [
      • Szabó C.Á
      • Morgan L.C.
      • Karkar K.M.
      • Leary L.D.
      • Lie O.V.
      • Girouard M.
      • et al.
      Electromyography-based seizure detector: preliminary results comparing a generalized tonic-clonic seizure detection algorithm to video-EEG recordings.
      ,
      • Onorati F.
      • Regalia G.
      • Caborni C.
      • Picard R.
      Improving convulsive seizure detection by exploiting data from outpatient settings using the Embrace wristband.
      ].
      In this review, we investigate the potential of SDDs to complement or replace the role of PROMs in ASM clinical trials. Subsequently, we suggest steps to streamline the testing of SDDs before their clinical trial adoption.

      2. Methods

      2.1 Evaluating SDD performance and reporting capabilities

      We conducted a systematic literature review, following PRISMA guidelines, to determine which devices could best fulfill ASM clinical trial reporting needs [
      • Ben-Menachem E.
      • Sander J.W.
      • Privitera M.
      • Gilliam F.
      Measuring outcomes of treatment with antiepileptic drugs in clinical trials.
      ,
      • Moher D.
      • Liberati A.
      • Tetzlaff J.
      • Altman D.G.
      • Group T.P.
      Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
      ]. Each device was evaluated in terms of its primary endpoint measurement capabilities.

      2.2 Literature search strategy

      The systematic review examined articles from the PUBMED database. Literature search terms included “seizure detection device” OR “wearable seizure device” OR “seizure alarm” OR “seizure monitor.” This search yielded 2262 publications.
      The following criteria were employed to refine these search results. Only device validation studies published between January 2003 and February 2018 were examined, as the goal of this review was to capture the present state of seizure detection. Included studies also required a participant cohort of at least three patients with seizures (PWS), as smaller device validation studies may not produce meaningful or generalizable results for ASM clinical trials. Additionally, only studies with sufficient data to calculate device F1-score were included to ensure that device performance could be standardized and compared.
      Articles were subsequently removed from consideration using the following criteria. Studies of invasive or non-mobile seizure detection modalities including video, intracranial, and wired EEG were excluded, as they remain largely impractical to implement in an ambulatory or outpatient ASM clinical trial setting. Validation studies for classification algorithms without an accompanying device were also excluded, as the accuracy and reliability of the algorithm may be device dependent and may lack immediate stand-alone utility in ASM clinical trials. Outpatient device validation studies were excluded if mobile EEG utilizing the 10–20 surface electrode placement system was not used as a standard for performance assessment. Finally, articles that profiled devices that exclusively detect psychogenic nonepileptic seizures (PNES) were not considered.
      In total, 38 studies met our inclusion and exclusion criteria. 30 of these studies were identified from our literature search, while 8 studies were selected by examining the references of the initial 30 publications and other SDD reviews (Fig. 1).
      Fig. 1
      Fig. 1This flowchart depicts the search and filtering process of device validation studies. The initial search yielded 2262 publications of which 38 were selected for inclusion in this review.
      Multiple studies of the same device were separately evaluated if each study had a unique patient population or assessed the detection of a different seizure type. When a publication assessed more than one SDD, and the performance data of each device was provided, their reporting capabilities were individually cataloged.
      Reviewed SDDs were divided into commercial and non-commercial categories, as commercial devices are immediately purchasable for ASM clinical trial use while non-commercial devices, at the time of this publication, may require additional production or licensing steps before they are available for large-scale use.

      2.3 Evaluating SDD primary endpoint measurement capabilities

      The performance of each device was calculated from seizure detection outcomes during use by patients with seizures (PWS). As a first step, we tabulated the true positive (TP) detections, false positive (FP) detections, and false negative (FN) detections of each device.
      We employed these values to assess SDD performance in terms of three statistics: precision, recall, and F1-score. Precision quantifies the likelihood that a positive detection is a true seizure, while recall quantifies the fraction of seizures successfully detected. F1-score evenly weights precision and recall, thereby enabling the standardized evaluation of device performance through a single, unified metric [
      • Powers D.M.
      Evaluation: from precision, recall, and F-measure to TOC, informedness, markedness & correlation.
      ,
      • Bidwell J.
      • Khuwatsamrit T.
      • Askew B.
      • Ehrenberg J.A.
      • Helmers S.
      Seizure reporting technologies for epilepsy treatment: a review of clinical information needs and supporting technologies.
      ]. These statistics were calculated as shown below [
      • Powers D.M.
      Evaluation: from precision, recall, and F-measure to TOC, informedness, markedness & correlation.
      ,
      • Bidwell J.
      • Khuwatsamrit T.
      • Askew B.
      • Ehrenberg J.A.
      • Helmers S.
      Seizure reporting technologies for epilepsy treatment: a review of clinical information needs and supporting technologies.
      ].
      Precision      TP/(TP + FP)
      (1)


      Recall      TP/(TP + FN)
      (2)


      F1-score      2 x (Precision x Recall)/(Precision + Recall)
      (3)


      True positive (TP); False positive (FP); False negative (FN)

      3. Results

      3.1 Commercial SDD performance

      The majority of the studies included in Table 1, which shows the performance data from the validation studies of commercial devices, were conducted in an Epilepsy Monitoring Unit (EMU)-setting. One study was conducted in an outpatient setting [
      • Kjaer T.W.
      • Sorensen H.B.D.
      • Groenborg S.
      • Pedersen C.R.
      • Duun-Henriksen J.
      Detection of paroxysms in long-term, single-channel EEG-monitoring of patients with typical absence seizures.
      ].
      Table 1Commercial Device Performance.
      PaperModalitySeizure TypeCommercial NamePWS
      Patients with Seizures.
      PrecisionRecallF1-score
      Jeppesen et al. [
      • Jeppesen J.
      • Beniczky S.
      • Fuglsang Frederiksen A.
      • Sidenius P.
      • Johansen P.
      Modified automatic R-peak detection algorithm for patients with epilepsy using a portable electrocardiogram recorder.
      ]
      ECGNSePatch71.001.001.00
      Szabo et al. [
      • Szabó C.Á
      • Morgan L.C.
      • Karkar K.M.
      • Leary L.D.
      • Lie O.V.
      • Girouard M.
      • et al.
      Electromyography-based seizure detector: preliminary results comparing a generalized tonic-clonic seizure detection algorithm to video-EEG recordings.
      ]
      sEMGGTCSPEAC110.950.950.95
      Sareen et al. [
      • Sareen S.
      • Sood S.K.
      • Gupta S.K.
      An automatic prediction of epileptic seizures using cloud computing and wireless sensor networks.
      ]
      Wireless EEGNSEmotiv Epoch50.950.940.94
      Sullivan et al. [
      • Sullivan J.
      SmartWatch clinical study report.
      ]
      InertialGTCSmartWatch150.880.990.93
      Kjaer et al. [
      • Kjaer T.W.
      • Sorensen H.B.D.
      • Groenborg S.
      • Pedersen C.R.
      • Duun-Henriksen J.
      Detection of paroxysms in long-term, single-channel EEG-monitoring of patients with typical absence seizures.
      ]
      Wireless EEGAbsenceActiwave60.870.980.92
      Beniczky et al. [
      • Beniczky S.
      • Polster T.
      • Kjaer T.W.
      • Hjalgrim H.
      Detection of generalized tonic–clonic seizures by a wireless wrist accelerometer: a prospective, multicenter study.
      ]
      InertialGTCEpi-Care Free200.810.900.85
      Kramer et al. [
      • Kramer U.
      • Kipervasser S.
      • Shlitner A.
      • Kuzniecky R.
      A novel portable seizure detection alarm system: preliminary results.
      ]
      InertialNSEpiLert150.710.910.80
      Onorati et al. [
      • Onorati F.
      • Regalia G.
      • Caborni C.
      • Migliorini M.
      • Bender D.
      • Poh M.-Z.
      • et al.
      Multicenter clinical assessment of improved wearable multimodal convulsive seizure detectors.
      ]
      MultimodalNSE3/E4220.510.950.66
      Velez et al. [
      • Velez M.
      • Fisher R.S.
      • Bartlett V.
      • Le S.
      Tracking generalized tonic-clonic seizures with a wrist accelerometer linked to an online database.
      ]
      InertialGTCSmartWatch100.430.920.59
      Narechania et al. [
      • Narechania A.P.
      • Garić I.I.
      • Sen-Gupta I.
      • Macken M.P.
      • Gerard E.E.
      • Schuele S.U.
      Assessment of a quasi-piezoelectric mattress monitor as a detection system for generalized convulsions.
      ]
      PressureGTCEmfit130.430.890.58
      Fulton et al. [
      • Fulton S.
      • Poppel K.V.
      • McGregor A.
      • Ellis M.
      • Patters A.
      • Wheless J.
      Prospective study of 2 bed alarms for detection of nocturnal seizures.
      ]
      MultimodalGTCMedpage: MP5151.000.110.20
      Halford et al. [
      • Halford J.J.
      • Sperling M.R.
      • Nair D.R.
      • Dlugos D.J.
      • Tatum W.O.
      • Harvey J.
      • et al.
      Detection of generalized tonic-clonic seizures using surface electromyographic monitoring.
      ]
      sEMGGTCSPEAC610.061.000.12
      Lockman et al. [
      • Lockman J.
      • Fisher R.S.
      • Olson D.M.
      Detection of seizure-like movements using a wrist accelerometer.
      ]
      InertialNSSmartWatch60.030.880.06
      Carlson et al. [
      • Carlson C.
      • Arnedo V.
      • Cahill M.
      • Devinsky O.
      Detecting nocturnal convulsions: efficacy of the MP5 monitor.
      ]
      MultimodalGTCMedpage: MP560.030.630.06
      Fulton et al. [
      • Fulton S.
      • Poppel K.V.
      • McGregor A.
      • Ellis M.
      • Patters A.
      • Wheless J.
      Prospective study of 2 bed alarms for detection of nocturnal seizures.
      ]
      InertialNSMedpage: ST2151.000.020.04
      Onorati et al. [
      • Onorati F.
      • Regalia G.
      • Caborni C.
      • Picard R.
      Improving convulsive seizure detection by exploiting data from outpatient settings using the Embrace wristband.
      ]
      MultimodalNSEmbrace40.021.000.03
      a Patients with Seizures.
      The performance of six commercial devices exceeded that of daytime (F1-score = 0.81) and nighttime (F1-score = 0.25) patient reports [
      • Szabó C.Á
      • Morgan L.C.
      • Karkar K.M.
      • Leary L.D.
      • Lie O.V.
      • Girouard M.
      • et al.
      Electromyography-based seizure detector: preliminary results comparing a generalized tonic-clonic seizure detection algorithm to video-EEG recordings.
      ,
      • Bidwell J.
      • Khuwatsamrit T.
      • Askew B.
      • Ehrenberg J.A.
      • Helmers S.
      Seizure reporting technologies for epilepsy treatment: a review of clinical information needs and supporting technologies.
      ,
      • Kjaer T.W.
      • Sorensen H.B.D.
      • Groenborg S.
      • Pedersen C.R.
      • Duun-Henriksen J.
      Detection of paroxysms in long-term, single-channel EEG-monitoring of patients with typical absence seizures.
      ,
      • Jeppesen J.
      • Beniczky S.
      • Fuglsang Frederiksen A.
      • Sidenius P.
      • Johansen P.
      Modified automatic R-peak detection algorithm for patients with epilepsy using a portable electrocardiogram recorder.
      ,
      • Sareen S.
      • Sood S.K.
      • Gupta S.K.
      An automatic prediction of epileptic seizures using cloud computing and wireless sensor networks.
      ,
      • Sullivan J.
      SmartWatch clinical study report.
      ,
      • Beniczky S.
      • Polster T.
      • Kjaer T.W.
      • Hjalgrim H.
      Detection of generalized tonic–clonic seizures by a wireless wrist accelerometer: a prospective, multicenter study.
      ].
      ePatch (ECG) was the highest performing commercial SDD (F1-score = 1.00) [
      • Jeppesen J.
      • Beniczky S.
      • Fuglsang Frederiksen A.
      • Sidenius P.
      • Johansen P.
      Modified automatic R-peak detection algorithm for patients with epilepsy using a portable electrocardiogram recorder.
      ], SPEAC (sEMG, midline bicep) was the highest performing GTC detection device (F1-score = 0.95) [
      • Szabó C.Á
      • Morgan L.C.
      • Karkar K.M.
      • Leary L.D.
      • Lie O.V.
      • Girouard M.
      • et al.
      Electromyography-based seizure detector: preliminary results comparing a generalized tonic-clonic seizure detection algorithm to video-EEG recordings.
      ], and Actiwave (wireless EEG) was the highest performing absence SDD (F1-score = 0.92) [
      • Kjaer T.W.
      • Sorensen H.B.D.
      • Groenborg S.
      • Pedersen C.R.
      • Duun-Henriksen J.
      Detection of paroxysms in long-term, single-channel EEG-monitoring of patients with typical absence seizures.
      ].
      Three devices, SPEAC, Smartwatch, and Medpage MP5, were evaluated across multiple patient populations in separate validation studies. Each of these devices exhibited different performance statistics across their validation studies. SPEAC was evaluated in two different studies, with one resulting in an F1-score of 0.95 (PWS = 11) and the other [
      • Szabó C.Á
      • Morgan L.C.
      • Karkar K.M.
      • Leary L.D.
      • Lie O.V.
      • Girouard M.
      • et al.
      Electromyography-based seizure detector: preliminary results comparing a generalized tonic-clonic seizure detection algorithm to video-EEG recordings.
      ], in an F1-score of 0.12 (PWS = 61) [
      • Halford J.J.
      • Sperling M.R.
      • Nair D.R.
      • Dlugos D.J.
      • Tatum W.O.
      • Harvey J.
      • et al.
      Detection of generalized tonic-clonic seizures using surface electromyographic monitoring.
      ]. Smartwatch was featured in three different validation studies with each reporting a different F1-score: 0.93 (PWS = 15) [
      • Sullivan J.
      SmartWatch clinical study report.
      ], 0.59 (PWS = 10) [
      • Velez M.
      • Fisher R.S.
      • Bartlett V.
      • Le S.
      Tracking generalized tonic-clonic seizures with a wrist accelerometer linked to an online database.
      ], and 0.06 (PWS = 6) [
      • Lockman J.
      • Fisher R.S.
      • Olson D.M.
      Detection of seizure-like movements using a wrist accelerometer.
      ]. Medpage MP5 was evaluated in two validation studies, which yielded F1-scores of 0.20 (PWS = 15) and 0.06 (PWS = 6) [
      • Fulton S.
      • Poppel K.V.
      • McGregor A.
      • Ellis M.
      • Patters A.
      • Wheless J.
      Prospective study of 2 bed alarms for detection of nocturnal seizures.
      ,
      • Carlson C.
      • Arnedo V.
      • Cahill M.
      • Devinsky O.
      Detecting nocturnal convulsions: efficacy of the MP5 monitor.
      ].
      None of the reviewed commercial devices detected FIA seizures, focal seizures with awareness, atonic seizures, clonic seizures, or myoclonic seizures. Many validation studies of commercial devices, including ePatch [
      • Jeppesen J.
      • Beniczky S.
      • Fuglsang Frederiksen A.
      • Sidenius P.
      • Johansen P.
      Modified automatic R-peak detection algorithm for patients with epilepsy using a portable electrocardiogram recorder.
      ], did not delineate SDD performance by seizure type, and instead reported overall device detection performance without further specification. Such devices were assigned an “NS” (not-specified) designation when describing the type of seizures they detected.

      3.2 Non-commercial SDD performance

      All the studies included in Table 2, which documents non-commercial device performance data, were conducted in an EMU-setting.
      Table 2Non-Commercial Device Performance.
      PaperModalitySeizure TypePWS
      Patients with Seizures.
      PrecisionRecallF1-score
      Cogan et al. [
      • Cogan D.
      • Nourani M.
      • Harvey J.
      • Nagaraddi V.
      Epileptic seizure detection using wristworn biosensors.
      ]
      MultimodalNS30.881.000.94
      Kusmakar et al. [
      • Kusmakar S.
      • Muthuganapathy R.
      • Yan B.
      • O’Brien T.J.
      • Palaniswami M.
      Gaussian mixture model for the identification of psychogenic non-epileptic seizures using a wearable accelerometer sensor.
      ]
      InertialNS160.881.000.94
      Ahmed et al. [
      • Ahmed A.
      • Ahmad W.
      • Khan M.J.
      • Siddiqui S.A.
      • Cheema H.M.
      A wearable sensor based multi-criteria-decision-system for real-time seizure detection.
      ]
      MultimodalNS40.960.900.93
      Zibrandtsen et al. [
      • Zibrandtsen I.C.
      • Kidmose P.
      • Christensen C.B.
      • Kjaer T.W.
      Ear-EEG detects ictal and interictal abnormalities in focal and generalized epilepsy – a comparison with scalp EEG monitoring.
      ]
      Wireless EEGFIA80.850.920.88
      Borujeny et al. [
      • Borujeny G.T.
      • Yazdi M.
      • Keshavarz-Haddad A.
      • Borujeny A.R.
      Detection of epileptic seizure using wireless sensor networks.
      ]
      InertialNS30.850.850.85
      Nijsen et al. [
      • Nijsen T.M.E.
      • Cluitmans P.J.M.
      • Arends J.B.A.M.
      • Griep P.A.M.
      Detection of subtle nocturnal motor activity from 3-D accelerometry recordings in epilepsy patients.
      ]
      InertialNS70.651.000.79
      Cuppens et al. [
      • Cuppens K.
      • Karsmakers P.
      • Van de Vel A.
      • Bonroy B.
      • Milosevic M.
      • Luca S.
      • et al.
      Accelerometry-based home monitoring for detection of nocturnal hypermotor seizures based on novelty detection.
      ]
      InertialFIA70.600.950.74
      Massé et al. [
      • Massé F.
      • Bussel M.V.
      • Serteyn A.
      • Arends J.
      • Penders J.
      Miniaturized wireless ECG monitor for real-time detection of epileptic seizures.
      ]
      ECMNS30.700.750.72
      Van de Vel et al. [
      • Van de Vel A.
      • Cuppens K.
      • Bonroy B.
      • Milosevic M.
      • Van Huffel S.
      • Vanrumste B.
      • et al.
      Long-term home monitoring of hypermotor seizures by patient-worn accelerometers.
      ]
      InertialFIA70.580.960.72
      Luca et al. [
      • Luca S.
      • Karsmakers P.
      • Cuppens K.
      • Croonenborghs T.
      • Van de Vel A.
      • Ceulemans B.
      • et al.
      Detecting rare events using extreme value statistics applied to epileptic convulsions in children.
      ]
      InertialFIA50.530.850.65
      Van Elmpt et al. [
      • Van Elmpt W.J.C.
      • Nijsen T.M.E.
      • Griep P.A.M.
      • Arends J.B.A.M.
      A model of heart rate changes to detect seizures in severe epilepsy.
      ]
      ECGNS100.500.900.64
      Milošević et al. [
      • Milošević M.
      • Van de Vel A.
      • Bonroy B.
      • Ceulemans B.
      • Lagae L.
      • Vanrumste B.
      • et al.
      Automated detection of tonic-clonic seizures using 3-D accelerometry and surface electromyography in pediatric patients.
      ]
      InertialGTC70.480.820.61
      Conradsen et al. [
      • Conradsen I.
      • Beniczky S.
      • Hoppe K.
      • Wolf P.
      • Sorensen H.B.D.
      Automated algorithm for generalized tonic–clonic epileptic seizure onset detection based on sEMG zero-crossing rate.
      ]
      sEMGGTC110.411.000.58
      Kusmakar et al. [
      • Kusmakar S.
      • Karmakar C.K.
      • Yan B.
      • O’Brien T.J.
      • Muthuganapathy R.
      • Palaniswami M.
      Detection of generalized tonic-clonic seizures using short length accelerometry signal.
      ]
      InertialGTC120.410.950.57
      Milošević et al. [
      • Milošević M.
      • Van de Vel A.
      • Bonroy B.
      • Ceulemans B.
      • Lagae L.
      • Vanrumste B.
      • et al.
      Automated detection of tonic-clonic seizures using 3-D accelerometry and surface electromyography in pediatric patients.
      ]
      sEMGGTC70.450.730.55
      Osorio et al. [
      • Osorio I.
      • Manly B.F.J.
      Is seizure detection based on EKG clinically relevant?.
      ]
      ECGNS810.380.860.53
      Poh et al. [
      • Poh M.-Z.
      • Loddenkemper T.
      • Reinsberger C.
      • Swenson N.C.
      • Goyal S.
      • Sabtala M.C.
      • et al.
      Convulsive seizure detection using a wrist-worn electrodermal activity and accelerometry biosensor.
      ]
      MultimodalGTC70.350.940.51
      Nijsen et al. [
      • Nijsen T.M.E.
      • Aarts R.M.
      • Johan B.A.
      • Cluitmans P.J.M.
      Automated detection of tonic seizures using 3-D accelerometry.
      ]
      InertialTonic180.350.830.49
      Nijsen et al. [
      • Nijsen T.M.E.
      • Aarts R.M.
      • Cluitmans P.J.
      • Griep P.A.
      Time-frequency analysis of accelerometry data for detection of myoclonic seizures.
      ]
      InertialMyoclonic360.350.800.49
      Bruijne et al. [
      • De Bruijne G.R.
      • Sommen P.C.W.
      • Aarts R.M.
      Detection of epileptic seizures through audio classification.
      ]
      Audio (Screams)NS170.300.980.46
      Dalton et al. [
      • Dalton A.
      • Patel S.
      • Chowdhury A.R.
      • Welsh M.
      • Pang T.
      • Schachter S.
      • et al.
      Development of a body sensor network to detect motor patterns of epileptic seizures.
      ]
      InertialNS50.280.910.43
      Beniczky et al. [
      • Beniczky S.
      • Conradsen I.
      • Henning O.
      • Fabricius M.
      • Wolf P.
      Automated real-time detection of tonic-clonic seizures using a wearable EMG device.
      ]
      sEMGGTC200.220.940.36
      Gu et al. [
      • Gu Y.
      • Cleeren E.
      • Dan J.
      • Claes K.
      • Van Paesschen W.
      • Van Huffel S.
      • et al.
      Comparison between scalp EEG and behind-the-ear EEG for development of a wearable seizure detection system for patients with focal epilepsy.
      ]
      Wireless EEGNS120.150.810.25
      Vandecasteele et al. [
      • Vandecasteele K.
      • De Cooman T.
      • Gu Y.
      • Cleeren E.
      • Claes K.
      • Paesschen W.V.
      • et al.
      Automated epileptic seizure detection based on wearable ECG and PPG in a hospital environment.
      ]
      ECGFIA110.020.700.04
      Bruijne et al. [
      • De Bruijne G.R.
      • Sommen P.C.W.
      • Aarts R.M.
      Detection of epileptic seizures through audio classification.
      ]
      Audio (Lips)NS170.020.980.04
      Vandecasteele et al. [
      • Vandecasteele K.
      • De Cooman T.
      • Gu Y.
      • Cleeren E.
      • Claes K.
      • Paesschen W.V.
      • et al.
      Automated epileptic seizure detection based on wearable ECG and PPG in a hospital environment.
      ]
      PPGFIA110.010.320.02
      a Patients with Seizures.
      Five non-commercial devices achieved higher F1-scores than daytime (F1-score = 0.81) and nighttime (F1-score = 0.25) patient reports [
      • Bidwell J.
      • Khuwatsamrit T.
      • Askew B.
      • Ehrenberg J.A.
      • Helmers S.
      Seizure reporting technologies for epilepsy treatment: a review of clinical information needs and supporting technologies.
      ,
      • Cogan D.
      • Nourani M.
      • Harvey J.
      • Nagaraddi V.
      Epileptic seizure detection using wristworn biosensors.
      ,
      • Kusmakar S.
      • Muthuganapathy R.
      • Yan B.
      • O’Brien T.J.
      • Palaniswami M.
      Gaussian mixture model for the identification of psychogenic non-epileptic seizures using a wearable accelerometer sensor.
      ,
      • Zibrandtsen I.C.
      • Kidmose P.
      • Christensen C.B.
      • Kjaer T.W.
      Ear-EEG detects ictal and interictal abnormalities in focal and generalized epilepsy – a comparison with scalp EEG monitoring.
      ,
      • Ahmed A.
      • Ahmad W.
      • Khan M.J.
      • Siddiqui S.A.
      • Cheema H.M.
      A wearable sensor based multi-criteria-decision-system for real-time seizure detection.
      ,
      • Borujeny G.T.
      • Yazdi M.
      • Keshavarz-Haddad A.
      • Borujeny A.R.
      Detection of epileptic seizure using wireless sensor networks.
      ].
      The highest performing non-commercial SDDs were a multimodal device (F1-score = 0.94) [
      • Cogan D.
      • Nourani M.
      • Harvey J.
      • Nagaraddi V.
      Epileptic seizure detection using wristworn biosensors.
      ], and an inertial wrist-worn device (F1-score = 0.94) [
      • Kusmakar S.
      • Muthuganapathy R.
      • Yan B.
      • O’Brien T.J.
      • Palaniswami M.
      Gaussian mixture model for the identification of psychogenic non-epileptic seizures using a wearable accelerometer sensor.
      ]. An inertial limb-worn device was the highest performing GTC SDD (F1-score = 0.60) [
      • Milošević M.
      • Van de Vel A.
      • Bonroy B.
      • Ceulemans B.
      • Lagae L.
      • Vanrumste B.
      • et al.
      Automated detection of tonic-clonic seizures using 3-D accelerometry and surface electromyography in pediatric patients.
      ]. A wireless EEG device was the highest performing FIA SDD (F1-score = 0.88) [
      • Zibrandtsen I.C.
      • Kidmose P.
      • Christensen C.B.
      • Kjaer T.W.
      Ear-EEG detects ictal and interictal abnormalities in focal and generalized epilepsy – a comparison with scalp EEG monitoring.
      ]. Only one SDD, an arm-worn inertial device, detected myoclonic seizures (F1-score = 0.49) [
      • Nijsen T.M.E.
      • Aarts R.M.
      • Cluitmans P.J.
      • Griep P.A.
      Time-frequency analysis of accelerometry data for detection of myoclonic seizures.
      ]. Another arm-worn inertial system detected tonic seizures (F1-score = 0.49) [
      • Nijsen T.M.E.
      • Aarts R.M.
      • Johan B.A.
      • Cluitmans P.J.M.
      Automated detection of tonic seizures using 3-D accelerometry.
      ].
      No two non-commercial device validation studies reviewed the same SDD for the detection of a single seizure type.
      None of the reviewed non-commercial devices detected absence seizures, focal seizures with awareness, atonic seizures, or clonic seizures. In many non-commercial device validation studies, SDD performance by seizure type is also not reported.
      A comparison of commercial and non-commercial device performance is shown in Fig. 2. The highest performing commercial devices predominantly detected GTC seizures or were given the NS designation. In contrast, the highest performing non-commercial devices detected FIA seizures or were also given the NS designation.
      Fig. 2
      Fig. 2The reviewed commercial and non-commercial SDDs were cumulatively able to detect five different seizure types. The size of the circles is proportional to the square-root of the respective study’s population size. Devices were labeled as NS (non-specified) if their validation study did not catalog the seizure type they detected. 17 devices were designated NS, 13 detected GTC seizures, 5 detected FIA seizures, and 1 device detected tonic, myoclonic, and absence seizures respectively.

      4. Discussion

      Reviewed devices employed a range of seizure detection modalities including inertial, sEMG, ECG, and multimodal techniques. The highest performing SDD is an ECG-based wearable (ePatch) [
      • Jeppesen J.
      • Beniczky S.
      • Fuglsang Frederiksen A.
      • Sidenius P.
      • Johansen P.
      Modified automatic R-peak detection algorithm for patients with epilepsy using a portable electrocardiogram recorder.
      ], with an F1-score of 1.00. The top-performing GTC SDD is an EMG band (SPEAC) [
      • Szabó C.Á
      • Morgan L.C.
      • Karkar K.M.
      • Leary L.D.
      • Lie O.V.
      • Girouard M.
      • et al.
      Electromyography-based seizure detector: preliminary results comparing a generalized tonic-clonic seizure detection algorithm to video-EEG recordings.
      ], with an F1-score of 0.95. The top FIA SDD is a wireless EEG device [
      • Zibrandtsen I.C.
      • Kidmose P.
      • Christensen C.B.
      • Kjaer T.W.
      Ear-EEG detects ictal and interictal abnormalities in focal and generalized epilepsy – a comparison with scalp EEG monitoring.
      ], with an F1-score of 0.88. Eleven devices outperformed patient daytime and nighttime self-reporting F1-scores (6 commercial and 5 non-commercial devices). No SDDs were specifically able to detect focal seizures with awareness, atonic seizures, or clonic seizures at the time of this review.

      4.1 Evaluating SDD capabilities for measuring sample secondary endpoints

      In addition to reporting seizures, many SDDs are also capable of collecting secondary outcome measures that can help fulfill additional ASM clinical trial information needs. For example, certain SDDs can measure supplementary endpoints including seizure duration and ictal period patient vital signs, which may enable further assessment of the impact of ASMs on patient quality of life (QOL) [
      • Ben-Menachem E.
      • Sander J.W.
      • Privitera M.
      • Gilliam F.
      Measuring outcomes of treatment with antiepileptic drugs in clinical trials.
      ,
      A trial of two fixed doses of ZX008 (Fenfluramine HCl) as an adjunctive therapy in children and young adults with dravet syndrome.
      ,
      Fed bioequivalence study of CBZ formulations.
      ,
      Efficacy of naloxone in reducing postictal central respiratory dysfunction in patients with epilepsy.
      ,
      Safety, and tolerability of epidiolex in patients (ages 1–19 years) with intractable epilepsy.
      ].
      A search for the secondary outcome measures used in past and ongoing ASM clinical trials cataloged in the United States National Library of Medicine database (clinicaltrials.gov), employing the search terms “Anti-Seizure Medication” OR “Anti-Epileptic Drug” OR “Anti-Convulsant Drug” OR “Anti-Seizure Medication Clinical Trials” OR “Anti-Epileptic Drug Clinical Trials” OR “Anti-Convulsant Drug Clinical Trials”, generated 3205 clinical trials. The hundreds of secondary endpoints employed across these trials differed according to study design and clinical trial stage.
      Since it would be difficult to determine an SDD’s ability to quantify each of the endpoints that the search revealed, a preliminary analysis was performed to assess whether the reviewed SDDs could detect a small selection of potential secondary endpoints.
      In this exploratory analysis, we assessed the ability of devices to measure the following endpoints: patient-self reported medication compliance [
      • Prospective A.
      Observational study on the effectiveness of new antiepileptic drugs as first bitherapy in the daily clinical practice.
      ], patient self-reported ASM-related adverse effects [
      A long-term extension study of E2007 in patients with refractory partial seizures uncontrolled with other anti-epileptic drugs (AEDs).
      ], time to nth seizure [
      A study to assess the safety and efficacy of lacosamide versus placebo (a pill without active medication) in patients with idiopathic generalised epilepsy who are already taking anti-epileptic medications.
      ], seizure length [
      A trial of two fixed doses of ZX008 (Fenfluramine HCl) as an adjunctive therapy in children and young adults with dravet syndrome.
      ], temperature [
      Fed bioequivalence study of CBZ formulations.
      ], heart rate [
      Safety, and tolerability of epidiolex in patients (ages 1–19 years) with intractable epilepsy.
      ], and pulse oximetry (SpO2) [
      Efficacy of naloxone in reducing postictal central respiratory dysfunction in patients with epilepsy.
      ]. In addition, we made the following assumptions regarding SDD reporting abilities: if a device had a linked eDiary, we assumed that this component enabled the recording of patient self-reported drug compliance and adverse events. We also assumed that device-linked eDiaries were able to document the time until an ictal event. SDDs were designated as able to measure seizure length if they explicitly reported ictal duration without requiring post hoc analysis. Devices that featured ECG or Photoplethysmography (PPG) sensors were considered to be capable of detecting patient heart rate. Additionally, devices with built-in sensors to measure patient temperature or SpO2 were considered to be capable of reporting these endpoints.
      Fig. 3 provides a visual representation of our exploratory analysis, showing that the reviewed commercial SDDs excelled at enabling the recording of patient-reported qualitative measurements including medication compliance, while the reviewed non-commercial SDDs performed well in vital detection, specifically in heart rate measurement. Reviewed devices that were unable to detect any of the sample secondary endpoints were excluded from Fig. 3.
      Fig. 3
      Fig. 315 reviewed SDDs were capable of detecting at least one of the selected secondary endpoints in the exploratory analysis. 7 of the 9 commercial devices were able to record patient self-reported qualitative information through an autonomously populated (linked) eDiary. All six of the included non-commercial devices could detect at least one of the sample patient vital measurements.

      4.2 Shortcomings of device validation studies

      The following design limitations in several device validation studies should be addressed to enable improved assessment of SDD endpoint reporting capabilities. These limitations may have affected the results of our review as they could impact the accuracy of the tabulated device performances.

      4.2.1 Limited generalizability

      Device performance results from reviewed validation studies may not be generalizable to ASM clinical trials due to four key considerations: the studies were small in size, they included highly-specific patient populations, they were conducted over a short period of time, and they did not evaluate their device across multiple patient cohorts. For example, several device validation studies included less than five patients in their participant cohort [
      • Cogan D.
      • Nourani M.
      • Harvey J.
      • Nagaraddi V.
      Epileptic seizure detection using wristworn biosensors.
      ,
      • Ahmed A.
      • Ahmad W.
      • Khan M.J.
      • Siddiqui S.A.
      • Cheema H.M.
      A wearable sensor based multi-criteria-decision-system for real-time seizure detection.
      ,
      • Borujeny G.T.
      • Yazdi M.
      • Keshavarz-Haddad A.
      • Borujeny A.R.
      Detection of epileptic seizure using wireless sensor networks.
      ], and recorded performance for less than five days [
      • Cogan D.
      • Nourani M.
      • Harvey J.
      • Nagaraddi V.
      Epileptic seizure detection using wristworn biosensors.
      ]. Other studies only included adult patients with impaired mental faculties [
      • Nijsen T.M.E.
      • Cluitmans P.J.M.
      • Arends J.B.A.M.
      • Griep P.A.M.
      Detection of subtle nocturnal motor activity from 3-D accelerometry recordings in epilepsy patients.
      ]. Due to the limited participant cohort of these studies and their abbreviated length, their device performance assessments may have limited translatability to ASM clinical trials, which often include hundreds of patients and can last longer than twelve weeks per cohort [
      • Prospective A.
      Observational study on the effectiveness of new antiepileptic drugs as first bitherapy in the daily clinical practice.
      ,
      • Biton V.
      • Rogin J.B.
      • Krauss G.
      • Abou-Khalil B.
      • Rocha J.F.
      • Moreira J.
      • et al.
      Adjunctive eslicarbazepine acetate: a pooled analysis of three phase III trials.
      ,
      • Zadeh W.W.
      • Escartin A.
      • Byrnes W.
      • Tennigkeit F.
      • Borghs S.
      • Li T.
      • et al.
      Efficacy and safety of lacosamide as first add-on or later adjunctive treatment for uncontrolled partial-onset seizures: a multicentre open-label trial.
      ]. Even though technologies such as ePatch (F1-score: 1.00) performed very well in limited testing [
      • Jeppesen J.
      • Beniczky S.
      • Fuglsang Frederiksen A.
      • Sidenius P.
      • Johansen P.
      Modified automatic R-peak detection algorithm for patients with epilepsy using a portable electrocardiogram recorder.
      ], further studies with larger, more diverse participant pools over a longer time period are needed to better evaluate promising technologies for use in ASM clinical trials. Conducting follow-up studies which include multiple patient cohorts may enable obtaining more reproducible performance statistics. Furthermore, a multiple cohort study could facilitate quantification of variance in device performance and more extensive characterization of device usability.

      4.2.2 Limited outpatient testing

      Additionally, most devices were only evaluated in an EMU setting. While inpatient testing enables the use of a video-EEG (vEEG) benchmark against which to assess device performance, this setting may not accurately reflect device detection performance in outpatient “real-world” settings [
      • Rizvi S.A.
      • Téllez Zenteno J.F.
      • Crawford S.L.
      • Wu A.
      Outpatient ambulatory EEG as an option for epilepsy surgery evaluation instead of inpatient EEG telemetry.
      ]. For instance, everyday ambulatory activities can cause an increase in the false alarm rates of inertial and multimodal devices over the rates measured in an inpatient trial [
      • Cuppens K.
      • Lagae L.
      • Ceulemans B.
      • Huffel S.V.
      • Vanrumste B.
      Detection of nocturnal frontal lobe seizures in pediatric patients by means of accelerometers: a first study.
      ]. Additionally, outpatient trials can enable the evaluation of SDD performance consistency over a longer time period than can inpatient EMU-based studies. If designed appropriately, mobile-EEG benchmarked outpatient trials could address the above concerns, and also help answer logistical, maintenance, and implementation questions that ASM clinical trial administrators may pose.

      4.2.3 Lack of secondary endpoint sensor validation

      Validation studies typically did not document or evaluate the accuracy of device-enabled secondary endpoint measurements. The accuracy of these measurements should be assessed to ensure that device detection features are robust and are potentially employable in ASM clinical trials.

      4.2.4 Incomplete experimental description

      Validation studies frequently did not delineate crucial components of their experimental design. For example, the seizure types that an SDD detected were often not specified [
      • Jeppesen J.
      • Beniczky S.
      • Fuglsang Frederiksen A.
      • Sidenius P.
      • Johansen P.
      Modified automatic R-peak detection algorithm for patients with epilepsy using a portable electrocardiogram recorder.
      ,
      • Cogan D.
      • Nourani M.
      • Harvey J.
      • Nagaraddi V.
      Epileptic seizure detection using wristworn biosensors.
      ]. In fact, 17 of the reviewed devices were given the NS designation. Future validation studies should document device detection performance by seizure type.
      In addition, differentiation between diurnal and nocturnal seizure detection performance may be helpful. Since patient self-reported seizure detection performance, a salient benchmark for device performance, significantly varies between day (F1-score = 0.81) and nighttime (F1-score = 0.25) [
      • Bidwell J.
      • Khuwatsamrit T.
      • Askew B.
      • Ehrenberg J.A.
      • Helmers S.
      Seizure reporting technologies for epilepsy treatment: a review of clinical information needs and supporting technologies.
      ], device detection performance should be tabulated separately for both daytime and nighttime settings.

      4.2.5 Inconsistent statistics for reporting device performance

      Currently, a unified metric to assess seizure detection performance is not employed. While the majority of validation studies employ recall (see Eq. (2)) to quantify the fraction seizures successfully detected, there is no consensus metric to quantify device overreporting. Some validation studies employ specificity (TN/FP + TN; TN: True Negative), while others utilize precision (see Eq. (1)). Employing precision instead of specificity facilitates the use of F1-score as a unified metric for the assessment of device performance [
      • Powers D.M.
      Evaluation: from precision, recall, and F-measure to TOC, informedness, markedness & correlation.
      ].
      Precision may be a preferable measure for quantifying device overreporting, as specificity includes the number of TNs in its tabulation [
      • Powers D.M.
      Evaluation: from precision, recall, and F-measure to TOC, informedness, markedness & correlation.
      ]. Since periods of non-seizure activity are longer than the length of seizures, TNs far outnumber the number of FPs, FNs, and TPs. As a result, a device can have a high specificity value, despite reporting a high number of false positives. Thus, a metric that does not employ TNs may better capture the tendency of a device to overreport seizures. One validation study demonstrated the quantitative impact of employing precision instead of specificity. While the assessed multimodal device had a reported specificity of 0.90, this measure belied the precision of the device, which was calculated to be 0.03 [
      • Carlson C.
      • Arnedo V.
      • Cahill M.
      • Devinsky O.
      Detecting nocturnal convulsions: efficacy of the MP5 monitor.
      ].
      F1-score is the harmonic mean of recall and precision (Eq. (3)) [
      • Powers D.M.
      Evaluation: from precision, recall, and F-measure to TOC, informedness, markedness & correlation.
      ]. F1-score is utilized in this review as it enables SDD assessment by equally weighting device tendency to correctly report seizure occurrence and its propensity to underreport seizure frequency. As demonstrated above, F1-score may provide improved device performance assessment as compared to other unified metrics which employ TNs in their calculation, including the Matthews Correlation Coefficient and Youden’s J Statistic [
      • Boughorbel S.
      • Jarray F.
      • El-Anbari M.
      Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric.
      ,
      • Youden W.
      Index for rating diagnostic tests.
      ]. Future validation studies should tabulate device performance for each seizure type using F1-score.

      4.2.6 Lack of subclinical seizure reporting

      The majority of SDD validation studies do not profile the propensity for their device to detect seizures that present without clinical correlates (subclinical seizures), or characterize the ability of their device to differentiate between subclinical and clinical seizures. This shortcoming can negatively impact the resulting performance of devices in ASM clinical trials where the primary endpoint is the reduction of clinical seizure frequency. In order to address concerns about SDD overreporting of clinical seizures, validation studies should report device detection statistics for clinical and subclinical seizures separately. This differentiation can also enable the better quantification of device performance for ASM clinical trials where the reduction of subclinical seizure frequency is an endpoint [
      Assess safety and efficacy of levetiracetam (LEV; Keppra) for seizure prevention.
      ].
      It is therefore important for validation studies to employ larger and more diverse patient cohorts, evaluate devices in both inpatient and outpatient settings over a longer duration, specify the accuracy of additional endpoint detecting sensors and algorithms used, as well as report performance for seizure-type specific daytime and nighttime use through F1-score.

      4.3 Streamlining testing: a device validation framework

      Establishing a standardized approach to assess devices may help streamline the device validation process before initial ASM clinical trial adoption, and eventual registration endpoint measurement use [
      • Lee J.W.
      Real-time non-EEG convulsive seizure detection devices: they work; now what?.
      ]. We suggest a framework with three overarching phases of device testing and integration: inpatient validation, outpatient validation, and experimental ASM clinical trial validation (Fig. 4). Each of these three stages should include a pilot phase followed by a larger device study with multiple patient cohorts, in order to obtain generalizable and reproducible results. This structure may also help address several of the shortcomings with existing device validation studies that were profiled above.
      Fig. 4
      Fig. 4Our proposed validation framework aims to streamline the testing of devices from inception to eventual primetime use in ASM clinical trials. The first step, inpatient validation, facilitates the initial evaluation of seizure detection performance and secondary endpoint measurement accuracy against a gold-standard vEEG benchmark. Subsequently, the outpatient validation step enables the assessment of device performance in a “real-world” setting. In the third step, SDDs may be introduced into open-label arms of early-stage clinical trials and small cohorts of ASM safety and efficacy studies. This step allows the final assessment of device-based endpoint detection in a clinical trial setting before primetime usage. After passing through each of these stages, SDDs will be more prepared for ASM pivotal trial use in a primary and secondary endpoint detection capacity.
      The first stage includes validating devices in an inpatient EMU setting. This setting enables device assessment against an epileptologist-interpreted video-EEG [
      • Seneviratne U.
      • Cook M.J.
      • D’Souza W.J.
      Electroencephalography in the diagnosis of genetic generalized epilepsy syndromes.
      ]; due to the lack of intracranial EEG-benchmarked validation studies, we chose this as the 1.00 F1-score (gold standard) seizure detection benchmark for reporting on each seizure type, including subclinical seizures. Inpatient testing can also facilitate ground truth evaluation of a device’s secondary sensing capabilities.
      The second stage includes validating devices in an outpatient setting, which provides the opportunity to test devices against eDiary-tracked PROMs and mobile-EEG reports. This setting more closely mimics the “real-world” environment in which ASM clinical trials are conducted, enabling device usability and durability testing. Outpatient device testing may also provide more generalizable false-positive data, especially for inertial devices, as patients may perform more frequent and varied ambulatory movements [
      • Xu Y.
      • Nguyen D.
      • Mohamed A.
      • Carcel C.
      • Li Q.
      • Kutlubaev M.A.
      • et al.
      Frequency of a false positive diagnosis of epilepsy: a systematic review of observational studies.
      ]. While outpatient ambulatory trials face challenges in study design and administration, and often lack a performance benchmark, they may be necessary to evaluate SDD utility for ASM clinical trials [
      • Faulkner H.J.
      • Arima H.
      • Mohamed A.
      The utility of prolonged outpatient ambulatory EEG.
      ,
      • Seneviratne U.
      • Mohamed A.
      • Cook M.
      • D’Souza W.
      The utility of ambulatory electroencephalography in routine clinical practice: a critical review.
      ,
      • Lawley A.
      • Evans S.
      • Manfredonia F.
      • Cavanna A.E.
      The role of outpatient ambulatory electroencephalography in the diagnosis and management of adults with epilepsy or nonepileptic attack disorder: a systematic literature review.
      ]. In fact, SDDs that have received 510(k) clearance have only been approved for use during “periods of rest” [,
      • Medical Devices; Neurological Devices
      Classification of the non-electroencephalogram physiological signal based seizure monitoring system.
      ], further underscoring the need for this stage of testing. Final outpatient device studies should also be extended in duration to enable SDD performance and usability evaluation over the typical length of an ASM clinical trial.
      The third stage includes introducing high-performing devices to experimental ASM clinical trials. This stage may include device testing within open-label arms of early clinical trials and in small cohorts of preliminary ASM safety studies [
      • Beniczky S.
      • Ryvlin P.
      Standards for testing and clinical validation of seizure detection devices.
      ]. As device performance and outpatient usage are better characterized, SDDs can be introduced into select cohorts of active pivotal trials.

      4.4 ASM clinical trial adoption: suggestions for device evaluation

      As part of our device validation framework, below, we suggest a set of preliminary device evaluation steps to ASM clinical trial administrators in order to help them assess device fitness for experimental clinical trial introduction.

      4.4.1 Assessing the SDD regulatory landscape

      During the due diligence process before experimentally introducing SDDs to ASM clinical trials, trial administrators should review device developer interactions with regulatory bodies including the FDA and EMA, from device inception through the design and execution of advanced testing phases. During this review, trial administrators should focus their attention on understanding regulator assessments of a device’s primary endpoint reporting abilities, secondary endpoint reporting abilities, and the logistics regarding a device’s clinical trial introduction. As a second step, trial administrators may also need to directly interact with these regulatory bodies to profile the necessary clearances and label claims that would enable primetime device usage in ASM pivotal trials.

      4.4.2 Primary endpoint reporting evaluation

      SDD performance should exceed that of patient reports for each seizure type, in order for devices to improve ASM clinical trial endpoint measurement. The F1-score of patient seizure detection may vary across seizure classifications, based on factors including frequency, type of onset, and loss of patient consciousness [
      • Hoppe C.
      • Poepel A.
      • Elger C.E.
      Epilepsy: accuracy of patient seizure counts.
      ]. Accordingly, performance benchmarks for device introduction may practically need to reflect the corresponding difficulty of detecting each seizure type. GTC seizure detection is most widely explored [
      • van Andel J.
      • Thijs R.D.
      • de Weerd A.
      • Arends J.
      • Leijten F.
      Non-EEG based ambulatory seizure detection designed for home use: what is available and how will it influence epilepsy care?.
      ,
      • Ulate-Campos A.
      • Coughlin F.
      • Gaínza-Lein M.
      • Fernández I.S.
      • Pearl P.L.
      • Loddenkemper T.
      Automated seizure detection systems and their effectiveness for each type of seizure.
      ,
      • Jory C.
      • Shankar R.
      • Coker D.
      • McLean B.
      • Hanna J.
      • Newman C.
      Safe and sound? A systematic literature review of seizure detection methods for personal use.
      ], and many devices are capable of high-performance reporting. FIA seizure detection is considered to be more difficult than GTC seizure reporting [
      • Hoppe C.
      • Poepel A.
      • Elger C.E.
      Epilepsy: accuracy of patient seizure counts.
      ], which may contribute to the lack of high performing FIA SDDs. In fact, none of the commercial devices examined in this review were capable of FIA seizure detection. As a result, we suggest a lower performance benchmark for a device to be considered for trial introduction for FIA seizure reporting, as compared to that for GTC seizure reporting.

      4.4.3 Secondary endpoint selection and reporting evaluation

      The accuracy of a device’s secondary endpoint reporting capabilities should also be evaluated by ASM clinical trial administrators since these features enable additional opportunities to monitor a patient's condition and QOL throughout a trial [
      • Ben-Menachem E.
      • Sander J.W.
      • Privitera M.
      • Gilliam F.
      Measuring outcomes of treatment with antiepileptic drugs in clinical trials.
      ]. As the secondary reporting capabilities of interest are highly dependent on the design of the ASM clinical trial in which devices are integrated, we suggest that a device’s secondary endpoint measurement abilities are assessed once a target clinical trial is identified.
      eDiaries should also be evaluated by trial administrators, as patients use these applications in ASM clinical trials to track important secondary endpoints including pre-ictal precipitating factors, recent changes in medication, missed doses, and adverse effects [
      • French J.A.
      Improving clinical trial efficiency: is technology the answer?.
      ]. Well-designed device-linked eDiaries with notifications may aid in increasing participant engagement with their device, potentially leading to more frequent and higher quality patient-reporting [
      • Fisher R.S.
      • Blum D.E.
      • DiVentura B.
      • Vannest J.
      • Hixson J.D.
      • Moss R.
      • et al.
      Seizure diaries for clinical research and practice: limitations and future prospects.
      ]. ASM clinical trial administrators should, therefore, assess whether device-linked eDiaries are designed to allow for seamless data visualization and analysis, through a secure and compliant portal [
      • Fisher R.S.
      • Blum D.E.
      • DiVentura B.
      • Vannest J.
      • Hixson J.D.
      • Moss R.
      • et al.
      Seizure diaries for clinical research and practice: limitations and future prospects.
      ].

      4.4.4 Evaluating the logistics of device implementation

      Trial administrators may also need to assess whether device developers have the necessary infrastructure to support device use and patient education during a trial. These administrators should also vet SDD form factor, software interface, battery life, and ease of use before device introduction [
      • Patel A.D.
      • Moss R.
      • Rust S.W.
      • Patterson J.
      • Strouse R.
      • Gedela S.
      • et al.
      Patient-centered design criteria for wearable seizure detection devices.
      ]. Assessing these details could ensure increased participant engagement with their SDD throughout a trial [
      • Ozanne A.
      • Johansson D.
      • Hällgren Graneheim U.
      • Malmgren K.
      • Bergquist F.
      • Alt Murphy M.
      Wearables in epilepsy and Parkinson’s disease-a focus group study.
      ]. Additionally, trial administrators may need to consider privacy concerns associated with device-based monitoring in a participant’s home, as well as financial considerations including cohort-wide device cost and maintenance fees [
      • Tovar Quiroga D.F.
      • Britton J.W.
      • Wirrell E.C.
      Patient and caregiver view on seizure detection devices: a survey study.
      ].

      4.5 Additional clinical trial applications of SDDs

      SDDs can also improve endpoint reporting in anti-seizure device (ASD) clinical trials. The most common ASDs employ targeted brain stimulation to reduce the occurrence and severity of seizures, often in refractory epilepsy patient populations [
      • Izadi A.
      • Ondek K.
      • Schedlbauer A.
      • Keselman I.
      • Shahlaie K.
      • Gurkoff G.
      Clinically indicated electrical stimulation strategies to treat patients with medically refractory epilepsy.
      ,
      • Fisher R.S.
      Therapeutic devices for epilepsy.
      ]. Prominent ASDs include a responsive neurostimulation system (RNS), vagus nerve stimulation (VNS) devices, and deep brain stimulation (DBS) devices [
      • Izadi A.
      • Ondek K.
      • Schedlbauer A.
      • Keselman I.
      • Shahlaie K.
      • Gurkoff G.
      Clinically indicated electrical stimulation strategies to treat patients with medically refractory epilepsy.
      ]. Similarly to ASM efficacy, ASD effectiveness is currently evaluated by utilizing PROMs to assess changes in seizure frequency [
      • Fisher R.S.
      • Blum D.E.
      • DiVentura B.
      • Vannest J.
      • Hixson J.D.
      • Moss R.
      • et al.
      Seizure diaries for clinical research and practice: limitations and future prospects.
      ]. Since PROMs are often inaccurate, SDDs may also offer a superior alternative for ASD trial primary endpoint reporting in both inpatient and outpatient settings [
      • Ferastraoaru V.
      • Goldenholz D.M.
      • Chiang S.
      • Moss R.
      • Theodore W.H.
      • Haut S.R.
      Characteristics of large patient‐reported outcomes: where can one million seizures get us?.
      ,
      • Hoppe C.
      • Poepel A.
      • Elger C.E.
      Epilepsy: accuracy of patient seizure counts.
      ]. Additionally, because several secondary endpoints in ASD trials mirror those in ASM trials, SDDs may enable better quantification of these measures as well.
      SDDs are also being developed for PNES detection and the differentiation of PNES from epileptic seizures [
      Detecting PNES with single-channel sEMG.
      ]. These SDDs may have utility in trials that aim to assess the ability of treatments, including cognitive behavioral therapy, to reduce PNES frequency [
      Detecting PNES with single-channel sEMG.
      ,].

      4.6 Future work

      This review identifies the highest performing seizure detection technologies and provides suggestions for streamlining the evaluation of device reporting capabilities. Trial administrator-specified F1-score benchmarks for the detection of each seizure type may enable more complete screening of device fitness before ASM clinical trial adoption.
      The development of SDDs that can differentiate subclinical seizures from clinical seizures, as well as the creation of more devices that can detect FIA seizures, focal seizures with awareness, absence seizures, atonic seizures, clonic seizures, tonic seizures, and myoclonic seizures will help increase the potential of SDD use in a wider range of ASM clinical trials.
      Follow-up studies on device validation study methods and guidelines, as well as future work on outpatient trial logistics, can help encourage improved testing practices while achieving a streamlined validation framework [
      • Beniczky S.
      • Ryvlin P.
      Standards for testing and clinical validation of seizure detection devices.
      ]. It is also crucial to explore the logistics of device integration and maintenance in ASM clinical trials. As the seizure detection field advances and the above device validation shortcomings are addressed, the tabulation of inclusion and exclusion criteria for ASM clinical trials which employ SDDs for endpoint detection is also critical.
      Studies that compare the evaluation of SDDs across each of their different use cases, including in caregiver assistance at a patient’s home and a decision support capacity in the clinic, may also add information.
      Commentary on the SDD regulatory landscape, including device 510(k) clearance and additional label claims that could be necessary to enable device usage in a registration endpoint measurement capacity, may better elucidate the pathways to primetime SDD clinical trial adoption.

      4.7 Limitations of this systematic review

      While the search criteria of this review thoroughly covered published validation studies, they did not encompass unpublished device performance data. Additionally, publication bias may have been introduced to our results since several commercial device companies directly conducted their own validation studies.
      Inherent information bias may also exist within individual published studies. In studies of SDDs that detect non-specified seizure types, the authors may have elected not to include device performance by individual seizure type, as the device may not have maintained consistent performance across each classification. In response, we included a separate designation, NS (not specified), to label these studies.
      In some device validation studies, authors may not have included false alarm rate (FAR) and chosen only to report specificity; such papers were excluded from this review since we could not employ F1-score to evaluate or compare their device’s performance.

      5. Conclusion

      This systematic review investigated the fitness of SDDs for the measurement of ASM clinical trial endpoints. We cataloged the seizure detection performance of these devices to quantify their primary endpoint reporting capabilities. The device with the highest GTC seizure detection performance was SPEAC (F1-score = 0.95). A non-commercial wireless EEG device achieved the highest performance for FIA seizure detection (F1-score = 0.88). An exploratory analysis was then completed to assess the ability of the reviewed SDDs to measure a set of sample secondary endpoints. Commercial devices with secondary endpoint detection capabilities were most successful in recording patient self-reported qualitative information, with seven devices containing a device-linked eDiary. Non-commercial devices were more effective in the measurement of patient vitals, as six of these devices could detect at least one of the sample vitals.
      We subsequently described the key shortcomings of reviewed device validation studies and asserted the benefit of employing F1-score as a unified metric for evaluating device performance. Finally, we suggested a framework to facilitate streamlined device evaluation before SDD adoption in ASM clinical trials.

      Author contributions

      All authors meaningfully participated in the creation and writing of the manuscript.
      Abhinav Kurada led the development of the concept and research design of the study, helped interpret acquired data, and led drafting, reviewing, and editing of the manuscript.
      Tarun Srinivasan led the literature search, manuscript revision, study design process and data interpretation as well as participated indrafting, reviewing, and editing of the manuscript.
      Sarah Hammond participated in the reviewing and editing of the manuscript.
      Adriana Ulate-Campos participated in the reviewing and editing of the manuscript.
      Jonathan Bidwell supervised the study conceptualization and development, the interpretation of data from the literature, and the reviewing and editing the manuscript.

      Disclosure of conflicts of interest

      Abhinav Kurada received compensation from Takeda Pharmaceuticals while conducting a portion of this study.Tarun Srinivasan reports no disclosures.Sarah Hammond reports no disclosuresAdriana Ulate-Campos reports no disclosures.Jonathan Bidwell reports no disclosures.

      Funding

      This review had no funding.

      Acknowledgment

      The authors would like to thank Megan Bendsen-Jensen for her assistance with figure design.

      References

        • Fisher R.S.
        Bad information in epilepsy care.
        Epilepsy Behav E&B. 2017; 67: 133-134https://doi.org/10.1016/j.yebeh.2016.10.022
        • Blachut B.
        • Hoppe C.
        • Surges R.
        • Elger C.
        • Helmstaedter C.
        Subjective seizure counts by epilepsy clinical drug trial participants are not reliable.
        Epilepsy Behav E&B. 2017; 67: 122-127https://doi.org/10.1016/j.yebeh.2016.10.036
        • Katz R.
        Issues in clinical trial design from the FDA perspective.
        Epilepsia. 2003; 44: 9-15https://doi.org/10.1046/j.1528-1157.44.s7.7.x
        • Ben-Menachem E.
        • Sander J.W.
        • Privitera M.
        • Gilliam F.
        Measuring outcomes of treatment with antiepileptic drugs in clinical trials.
        Epilepsy Behav E&B. 2010; 18: 24-30https://doi.org/10.1016/j.yebeh.2010.04.001
        • Ferastraoaru V.
        • Goldenholz D.M.
        • Chiang S.
        • Moss R.
        • Theodore W.H.
        • Haut S.R.
        Characteristics of large patient‐reported outcomes: where can one million seizures get us?.
        Epilepsia Open. 2018; 3: 364-373https://doi.org/10.1002/epi4.12237
        • Hoppe C.
        • Poepel A.
        • Elger C.E.
        Epilepsy: accuracy of patient seizure counts.
        Arch Neurol. 2007; 64: 1595-1599https://doi.org/10.1001/archneur.64.11.1595
        • Ryvlin P.
        • Beniczky S.
        Seizure detection and mobile health devices in epilepsy: update and future developments.
        Epilepsia. 2018; 59: 7-8https://doi.org/10.1111/epi.14088
      1. 510(k) Premarket Notification. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm?ID=K172935 [Accessed 24 July 2018].

        • Medical Devices; Neurological Devices
        Classification of the non-electroencephalogram physiological signal based seizure monitoring system.
        Fed. Regist. 2017; ([Accessed 24 July 2018])
        • Szabó C.Á
        • Morgan L.C.
        • Karkar K.M.
        • Leary L.D.
        • Lie O.V.
        • Girouard M.
        • et al.
        Electromyography-based seizure detector: preliminary results comparing a generalized tonic-clonic seizure detection algorithm to video-EEG recordings.
        Epilepsia. 2015; 56: 1432-1437https://doi.org/10.1111/epi.13083
        • Onorati F.
        • Regalia G.
        • Caborni C.
        • Picard R.
        Improving convulsive seizure detection by exploiting data from outpatient settings using the Embrace wristband.
        ([Accessed 25 April 2018])
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • Group T.P.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6: e1000097https://doi.org/10.1371/journal.pmed.1000097
        • Powers D.M.
        Evaluation: from precision, recall, and F-measure to TOC, informedness, markedness & correlation.
        J Mach Learn Technol. 2011; 2: 37-63https://doi.org/10.9735/2229-3981
        • Bidwell J.
        • Khuwatsamrit T.
        • Askew B.
        • Ehrenberg J.A.
        • Helmers S.
        Seizure reporting technologies for epilepsy treatment: a review of clinical information needs and supporting technologies.
        Seizure. 2015; 32: 109-117https://doi.org/10.1016/j.seizure.2015.09.006
        • Kjaer T.W.
        • Sorensen H.B.D.
        • Groenborg S.
        • Pedersen C.R.
        • Duun-Henriksen J.
        Detection of paroxysms in long-term, single-channel EEG-monitoring of patients with typical absence seizures.
        IEEE J Transl Eng Health Med. 2017; 5: 2000108https://doi.org/10.1109/JTEHM.2017.2649491
        • Jeppesen J.
        • Beniczky S.
        • Fuglsang Frederiksen A.
        • Sidenius P.
        • Johansen P.
        Modified automatic R-peak detection algorithm for patients with epilepsy using a portable electrocardiogram recorder.
        Conf proc annu int conf IEEE eng med biol soc IEEE eng med biol soc annu conf 2017. 2017; : 4082-4085https://doi.org/10.1109/EMBC.2017.8037753
        • Sareen S.
        • Sood S.K.
        • Gupta S.K.
        An automatic prediction of epileptic seizures using cloud computing and wireless sensor networks.
        J Med Syst. 2016; 40: 226https://doi.org/10.1007/s10916-016-0579-1
        • Sullivan J.
        SmartWatch clinical study report.
        2013 ([Accessed 25 April 2018])
        • Beniczky S.
        • Polster T.
        • Kjaer T.W.
        • Hjalgrim H.
        Detection of generalized tonic–clonic seizures by a wireless wrist accelerometer: a prospective, multicenter study.
        Epilepsia. 2013; 54: e58-e61https://doi.org/10.1111/epi.12120
        • Halford J.J.
        • Sperling M.R.
        • Nair D.R.
        • Dlugos D.J.
        • Tatum W.O.
        • Harvey J.
        • et al.
        Detection of generalized tonic-clonic seizures using surface electromyographic monitoring.
        Epilepsia. 2017; 58: 1861-1869https://doi.org/10.1111/epi.13897
        • Velez M.
        • Fisher R.S.
        • Bartlett V.
        • Le S.
        Tracking generalized tonic-clonic seizures with a wrist accelerometer linked to an online database.
        Seizure. 2016; 39: 13-18https://doi.org/10.1016/j.seizure.2016.04.009
        • Lockman J.
        • Fisher R.S.
        • Olson D.M.
        Detection of seizure-like movements using a wrist accelerometer.
        Epilepsy Behav E&B. 2011; 20: 638-641https://doi.org/10.1016/j.yebeh.2011.01.019
        • Fulton S.
        • Poppel K.V.
        • McGregor A.
        • Ellis M.
        • Patters A.
        • Wheless J.
        Prospective study of 2 bed alarms for detection of nocturnal seizures.
        J Child Neurol. 2013; 28: 1430-1433https://doi.org/10.1177/0883073812462064
        • Carlson C.
        • Arnedo V.
        • Cahill M.
        • Devinsky O.
        Detecting nocturnal convulsions: efficacy of the MP5 monitor.
        Seizure. 2009; 18: 225-227https://doi.org/10.1016/j.seizure.2008.08.007
        • Kramer U.
        • Kipervasser S.
        • Shlitner A.
        • Kuzniecky R.
        A novel portable seizure detection alarm system: preliminary results.
        J Clin Neurophysiol. 2011; 28: 36-38https://doi.org/10.1097/WNP.0b013e3182051320
        • Onorati F.
        • Regalia G.
        • Caborni C.
        • Migliorini M.
        • Bender D.
        • Poh M.-Z.
        • et al.
        Multicenter clinical assessment of improved wearable multimodal convulsive seizure detectors.
        Epilepsia. 2017; 58: 1870-1879https://doi.org/10.1111/epi.13899
        • Narechania A.P.
        • Garić I.I.
        • Sen-Gupta I.
        • Macken M.P.
        • Gerard E.E.
        • Schuele S.U.
        Assessment of a quasi-piezoelectric mattress monitor as a detection system for generalized convulsions.
        Epilepsy Behav E&B. 2013; 28: 172-176https://doi.org/10.1016/j.yebeh.2013.04.017
        • Cogan D.
        • Nourani M.
        • Harvey J.
        • Nagaraddi V.
        Epileptic seizure detection using wristworn biosensors.
        Conf proc annu int conf IEEE eng med biol soc IEEE eng med biol soc annu conf 2015. 2015; : 5086-5089https://doi.org/10.1109/EMBC.2015.7319535
        • Kusmakar S.
        • Muthuganapathy R.
        • Yan B.
        • O’Brien T.J.
        • Palaniswami M.
        Gaussian mixture model for the identification of psychogenic non-epileptic seizures using a wearable accelerometer sensor.
        2016 38th annu. int. conf. IEEE eng. med. biol. soc. EMBC 2016, IEEE, institute of electrical and electronics engineers. 2016; : 1006-1009https://doi.org/10.1109/EMBC.2016.7590872
        • Zibrandtsen I.C.
        • Kidmose P.
        • Christensen C.B.
        • Kjaer T.W.
        Ear-EEG detects ictal and interictal abnormalities in focal and generalized epilepsy – a comparison with scalp EEG monitoring.
        Clin Neurophysiol. 2017; 128: 2454-2461https://doi.org/10.1016/j.clinph.2017.09.115
        • Ahmed A.
        • Ahmad W.
        • Khan M.J.
        • Siddiqui S.A.
        • Cheema H.M.
        A wearable sensor based multi-criteria-decision-system for real-time seizure detection.
        Conf proc annu int conf IEEE eng med biol soc ieee eng med biol soc annu conf 2017. 2017; : 2377-2380https://doi.org/10.1109/EMBC.2017.8037334
        • Borujeny G.T.
        • Yazdi M.
        • Keshavarz-Haddad A.
        • Borujeny A.R.
        Detection of epileptic seizure using wireless sensor networks.
        J Med Signals Sens. 2013; 3 (10.24098859): 63-68
        • Milošević M.
        • Van de Vel A.
        • Bonroy B.
        • Ceulemans B.
        • Lagae L.
        • Vanrumste B.
        • et al.
        Automated detection of tonic-clonic seizures using 3-D accelerometry and surface electromyography in pediatric patients.
        IEEE J Biomed Health Inform. 2016; 20: 1333-1341https://doi.org/10.1109/JBHI.2015.2462079
        • Nijsen T.M.E.
        • Aarts R.M.
        • Cluitmans P.J.
        • Griep P.A.
        Time-frequency analysis of accelerometry data for detection of myoclonic seizures.
        IEEE Trans Inf Technol Biomed. 2010; 14: 1197https://doi.org/10.1109/TITB.2010.2058123
        • Nijsen T.M.E.
        • Aarts R.M.
        • Johan B.A.
        • Cluitmans P.J.M.
        Automated detection of tonic seizures using 3-D accelerometry.
        4th European conference of the international federation for medical and biological engineering. 2009: 188-191 (10.1007%2F978-3-540-89208-3_47)
        • Nijsen T.M.E.
        • Cluitmans P.J.M.
        • Arends J.B.A.M.
        • Griep P.A.M.
        Detection of subtle nocturnal motor activity from 3-D accelerometry recordings in epilepsy patients.
        IEEE Trans Biomed Eng. 2007; 54: 2073-2081https://doi.org/10.1109/TBME.2007.895114
        • Cuppens K.
        • Karsmakers P.
        • Van de Vel A.
        • Bonroy B.
        • Milosevic M.
        • Luca S.
        • et al.
        Accelerometry-based home monitoring for detection of nocturnal hypermotor seizures based on novelty detection.
        IEEE J Biomed Health Inform. 2014; 18: 1026-1033https://doi.org/10.1109/JBHI.2013.2285015
        • Massé F.
        • Bussel M.V.
        • Serteyn A.
        • Arends J.
        • Penders J.
        Miniaturized wireless ECG monitor for real-time detection of epileptic seizures.
        ACM Trans Embed Comput Syst. 2013; 12 (1–102:21): 102https://doi.org/10.1145/2485984.2485990
        • Van de Vel A.
        • Cuppens K.
        • Bonroy B.
        • Milosevic M.
        • Van Huffel S.
        • Vanrumste B.
        • et al.
        Long-term home monitoring of hypermotor seizures by patient-worn accelerometers.
        Epilepsy Behav E&B. 2013; 26: 118-125https://doi.org/10.1016/j.yebeh.2012.10.006
        • Luca S.
        • Karsmakers P.
        • Cuppens K.
        • Croonenborghs T.
        • Van de Vel A.
        • Ceulemans B.
        • et al.
        Detecting rare events using extreme value statistics applied to epileptic convulsions in children.
        Artif Intell Med. 2014; 60: 89-96https://doi.org/10.1016/j.artmed.2013.11.007
        • Van Elmpt W.J.C.
        • Nijsen T.M.E.
        • Griep P.A.M.
        • Arends J.B.A.M.
        A model of heart rate changes to detect seizures in severe epilepsy.
        Seizure. 2006; 15: 366-375https://doi.org/10.1016/j.seizure.2006.03.005
        • Conradsen I.
        • Beniczky S.
        • Hoppe K.
        • Wolf P.
        • Sorensen H.B.D.
        Automated algorithm for generalized tonic–clonic epileptic seizure onset detection based on sEMG zero-crossing rate.
        IEEE Trans Biomed Eng. 2012; 59: 579-585https://doi.org/10.1109/TBME.2011.2178094
        • Kusmakar S.
        • Karmakar C.K.
        • Yan B.
        • O’Brien T.J.
        • Muthuganapathy R.
        • Palaniswami M.
        Detection of generalized tonic-clonic seizures using short length accelerometry signal.
        Conf proc annu int conf IEEE eng med biol soc IEEE eng med biol soc annu conf 2017. 2017; : 4566-4569https://doi.org/10.1109/EMBC.2017.8037872
        • Osorio I.
        • Manly B.F.J.
        Is seizure detection based on EKG clinically relevant?.
        Clin Neurophysiol. 2014; 125: 1946-1951https://doi.org/10.1016/j.clinph.2014.01.026
        • Poh M.-Z.
        • Loddenkemper T.
        • Reinsberger C.
        • Swenson N.C.
        • Goyal S.
        • Sabtala M.C.
        • et al.
        Convulsive seizure detection using a wrist-worn electrodermal activity and accelerometry biosensor.
        Epilepsia. 2012; 53: e93-e97https://doi.org/10.1111/j.1528-1167.2012.03444.x
        • De Bruijne G.R.
        • Sommen P.C.W.
        • Aarts R.M.
        Detection of epileptic seizures through audio classification.
        4th European conference of the international federation for medical and biological engineering. 2009; : 1450-1454https://doi.org/10.1007/978-3-540-89208-3_344
        • Dalton A.
        • Patel S.
        • Chowdhury A.R.
        • Welsh M.
        • Pang T.
        • Schachter S.
        • et al.
        Development of a body sensor network to detect motor patterns of epileptic seizures.
        IEEE Trans Biomed Eng. 2012; 59: 3204-3211https://doi.org/10.1109/TBME.2012.2204990
        • Beniczky S.
        • Conradsen I.
        • Henning O.
        • Fabricius M.
        • Wolf P.
        Automated real-time detection of tonic-clonic seizures using a wearable EMG device.
        Neurology. 2018; 90: e428-e434https://doi.org/10.1212/WNL.0000000000004893
        • Gu Y.
        • Cleeren E.
        • Dan J.
        • Claes K.
        • Van Paesschen W.
        • Van Huffel S.
        • et al.
        Comparison between scalp EEG and behind-the-ear EEG for development of a wearable seizure detection system for patients with focal epilepsy.
        Sensors. 2017; 18: 29https://doi.org/10.3390/s18010029
        • Vandecasteele K.
        • De Cooman T.
        • Gu Y.
        • Cleeren E.
        • Claes K.
        • Paesschen W.V.
        • et al.
        Automated epileptic seizure detection based on wearable ECG and PPG in a hospital environment.
        Sensors. 2017; 17: 2338https://doi.org/10.3390/s17102338
      2. A trial of two fixed doses of ZX008 (Fenfluramine HCl) as an adjunctive therapy in children and young adults with dravet syndrome.
        2018 ([Accessed 25 April 2018])
      3. Fed bioequivalence study of CBZ formulations.
        2018 ([Accessed 20 July 2018])
      4. Efficacy of naloxone in reducing postictal central respiratory dysfunction in patients with epilepsy.
        2018 ([Accessed 20 July 2018])
      5. Safety, and tolerability of epidiolex in patients (ages 1–19 years) with intractable epilepsy.
        2018 ([Accessed 20 July 2018])
        • Prospective A.
        Observational study on the effectiveness of new antiepileptic drugs as first bitherapy in the daily clinical practice.
        2018 ([Accessed 20 July 2018])
      6. A long-term extension study of E2007 in patients with refractory partial seizures uncontrolled with other anti-epileptic drugs (AEDs).
        2018 ([Accessed 20 July 2018])
      7. A study to assess the safety and efficacy of lacosamide versus placebo (a pill without active medication) in patients with idiopathic generalised epilepsy who are already taking anti-epileptic medications.
        2018 ([Accessed 20 July 2018])
        • Biton V.
        • Rogin J.B.
        • Krauss G.
        • Abou-Khalil B.
        • Rocha J.F.
        • Moreira J.
        • et al.
        Adjunctive eslicarbazepine acetate: a pooled analysis of three phase III trials.
        Epilepsy Behav. 2017; 72: 127-134https://doi.org/10.1016/j.yebeh.2017.04.019
        • Zadeh W.W.
        • Escartin A.
        • Byrnes W.
        • Tennigkeit F.
        • Borghs S.
        • Li T.
        • et al.
        Efficacy and safety of lacosamide as first add-on or later adjunctive treatment for uncontrolled partial-onset seizures: a multicentre open-label trial.
        Seizure. 2015; 31: 72-79https://doi.org/10.1016/j.seizure.2015.07.001
        • Rizvi S.A.
        • Téllez Zenteno J.F.
        • Crawford S.L.
        • Wu A.
        Outpatient ambulatory EEG as an option for epilepsy surgery evaluation instead of inpatient EEG telemetry.
        Epilepsy Behav E&B Case Rep. 2013; 1: 39-41https://doi.org/10.1016/j.ebcr.2013.01.001
        • Cuppens K.
        • Lagae L.
        • Ceulemans B.
        • Huffel S.V.
        • Vanrumste B.
        Detection of nocturnal frontal lobe seizures in pediatric patients by means of accelerometers: a first study.
        2009 annu. int. conf. IEEE eng. med. biol. soc. 2009; : 6608-6611https://doi.org/10.1109/IEMBS.2009.5332557
        • Boughorbel S.
        • Jarray F.
        • El-Anbari M.
        Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric.
        PLoS One. 2017; 12: e0177678https://doi.org/10.1371/journal.pone.0177678
        • Youden W.
        Index for rating diagnostic tests.
        Am Cancer Soc. 1949; 3: 32-35https://doi.org/10.1002/1097-0142
      8. Assess safety and efficacy of levetiracetam (LEV; Keppra) for seizure prevention.
        2019 ([Accessed 26 January 2019])
        • Lee J.W.
        Real-time non-EEG convulsive seizure detection devices: they work; now what?.
        Epilepsy Curr. 2018; 18: 164-166https://doi.org/10.5698/1535-7597.18.3.164
        • Seneviratne U.
        • Cook M.J.
        • D’Souza W.J.
        Electroencephalography in the diagnosis of genetic generalized epilepsy syndromes.
        Front Neurol. 2017; 8: 499https://doi.org/10.3389/fneur.2017.00499
        • Xu Y.
        • Nguyen D.
        • Mohamed A.
        • Carcel C.
        • Li Q.
        • Kutlubaev M.A.
        • et al.
        Frequency of a false positive diagnosis of epilepsy: a systematic review of observational studies.
        Seizure. 2016; 41: 167-174https://doi.org/10.1016/j.seizure.2016.08.005
        • Faulkner H.J.
        • Arima H.
        • Mohamed A.
        The utility of prolonged outpatient ambulatory EEG.
        Seizure. 2012; 21: 491-495https://doi.org/10.1016/j.seizure.2012.04.015
        • Seneviratne U.
        • Mohamed A.
        • Cook M.
        • D’Souza W.
        The utility of ambulatory electroencephalography in routine clinical practice: a critical review.
        Epilepsy Res. 2013; 105: 1-12https://doi.org/10.1016/j.eplepsyres.2013.02.004
        • Lawley A.
        • Evans S.
        • Manfredonia F.
        • Cavanna A.E.
        The role of outpatient ambulatory electroencephalography in the diagnosis and management of adults with epilepsy or nonepileptic attack disorder: a systematic literature review.
        Epilepsy Behav E&B. 2015; 53: 26-30https://doi.org/10.1016/j.yebeh.2015.09.032
        • Beniczky S.
        • Ryvlin P.
        Standards for testing and clinical validation of seizure detection devices.
        Epilepsia. 2018; 59: 9-13https://doi.org/10.1111/epi.14049
        • van Andel J.
        • Thijs R.D.
        • de Weerd A.
        • Arends J.
        • Leijten F.
        Non-EEG based ambulatory seizure detection designed for home use: what is available and how will it influence epilepsy care?.
        Epilepsy Behav E&B. 2016; 57: 82-89https://doi.org/10.1016/j.yebeh.2016.01.003
        • Ulate-Campos A.
        • Coughlin F.
        • Gaínza-Lein M.
        • Fernández I.S.
        • Pearl P.L.
        • Loddenkemper T.
        Automated seizure detection systems and their effectiveness for each type of seizure.
        Seizure. 2016; 40: 88-101https://doi.org/10.1016/j.seizure.2016.06.008
        • Jory C.
        • Shankar R.
        • Coker D.
        • McLean B.
        • Hanna J.
        • Newman C.
        Safe and sound? A systematic literature review of seizure detection methods for personal use.
        Seizure. 2016; 36: 4-15https://doi.org/10.1016/j.seizure.2016.01.013
        • French J.A.
        Improving clinical trial efficiency: is technology the answer?.
        Epilepsia Open. 2017; 2: 121-122https://doi.org/10.1002/epi4.12042
        • Fisher R.S.
        • Blum D.E.
        • DiVentura B.
        • Vannest J.
        • Hixson J.D.
        • Moss R.
        • et al.
        Seizure diaries for clinical research and practice: limitations and future prospects.
        Epilepsy Behav E&B. 2012; 24: 304-310https://doi.org/10.1016/j.yebeh.2012.04.128
        • Patel A.D.
        • Moss R.
        • Rust S.W.
        • Patterson J.
        • Strouse R.
        • Gedela S.
        • et al.
        Patient-centered design criteria for wearable seizure detection devices.
        Epilepsy Behav E&B. 2016; 64: 116-121https://doi.org/10.1016/j.yebeh.2016.09.012
        • Ozanne A.
        • Johansson D.
        • Hällgren Graneheim U.
        • Malmgren K.
        • Bergquist F.
        • Alt Murphy M.
        Wearables in epilepsy and Parkinson’s disease-a focus group study.
        Acta Neurol Scand. 2018; 137: 188-194https://doi.org/10.1111/ane.12798
        • Tovar Quiroga D.F.
        • Britton J.W.
        • Wirrell E.C.
        Patient and caregiver view on seizure detection devices: a survey study.
        Seizure. 2016; 41: 179-181https://doi.org/10.1016/j.seizure.2016.08.004
        • Izadi A.
        • Ondek K.
        • Schedlbauer A.
        • Keselman I.
        • Shahlaie K.
        • Gurkoff G.
        Clinically indicated electrical stimulation strategies to treat patients with medically refractory epilepsy.
        Epilepsia Open. 2018; 3: 198-209https://doi.org/10.1002/epi4.12276
        • Fisher R.S.
        Therapeutic devices for epilepsy.
        Ann Neurol. 2012; 71: 157-168https://doi.org/10.1002/ana.22621
      9. Detecting PNES with single-channel sEMG.
        2019 ([Accessed 26 January 2019])
      10. Treatment outcomes of CBT for PNES.
        2019 ([Accessed 26 January 2019])