If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Stichting Epilepsie Instellingen Nederland (SEIN), P.O. Box 540, 2130 AM, Heemstede, the NetherlandsDepartment of Neurology and Clinical Neurophysiology, Leiden University Medical Center (LUMC), Leiden, the Netherlands
Stichting Epilepsie Instellingen Nederland (SEIN), P.O. Box 540, 2130 AM, Heemstede, the NetherlandsDepartment of Neurology and Clinical Neurophysiology, Leiden University Medical Center (LUMC), Leiden, the NetherlandsUCL Queen Square Institute of Neurology, London, UK
Department of Health Services Research, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, the NetherlandsCentre for Economic Evaluation and Machine Learning, Trimbos Institute, Utrecht, the Netherlands
We evaluated the cost-utility and cost-effectiveness of a SDD: the NightWatch.
•
Two months NightWatch use decreased caregivers’ stress levels, QALYs were unchanged.
•
Two months of NightWatch implementation showed a decrease in mean costs of €775.
•
Cost-effectiveness probability was 72% for NightWatch at a €50.000 cost-effectiveness threshold.
Abstract
Purpose
We performed an economic evaluation, from a societal perspective, to examine the cost-utility and cost-effectiveness of a wearable multimodal seizure detection device: NightWatch.
Methods
We collected data between November 2018 and June 2020 from the PROMISE trial (NCT03909984), including children aged 4–16 years with refractory epilepsy living at home. Caregivers completed questionnaires on stress, quality of life, health care consumption and productivity costs after two-month baseline and two-month intervention with NightWatch. We used costs, stress levels and quality-adjusted life years (QALYs) to calculate incremental cost-effectiveness ratios (ICERs). Missing items were handled by mean imputation. Sensitivity analyses were performed to examine the robustness of the results including bootstrap sampling.
Results
We included 41 children (44% female; mean age 9.8 years, standard deviation (SD) 3.7 years). Total societal costs of the baseline period (T1) were on average €3,238 per patient, whereas after intervention (T2) this reduced to 2,463 (saving €775). The QALYs were similar between both periods (mean QALY 0.90 per participant, SD at T1 0.10, SD at T2 0.13). At a ceiling ratio of €50.000, NightWatch showed a 72% cost-effective probability. Univariate sensitivity analyses, on the perspective and imputation method, demonstrated result robustness.
Conclusion
Our study suggests that NightWatch might be a cost-effective addition to current standard care for children with refractory epilepsy living at home. Further research with an additional target group for a large timeframe may support the findings of this research.
SUDEP: Sudden unexpected death in epilepsy ; PROMISE: Promoting implementation of seizure detection devices in epilepsy care ; CSI: Caregiver Strain Index; iMTA MCQ: Institute for Medical Technology Assessment Medical Consumption Questionnaire; iMTA PCQ: Institute for Medical Technology Assessment Productivity Costs Questionnaire ; QoL: Quality of life ; ICER: Incremental cost-effectiveness ratio ; QALY: Quality-adjusted life year; CEAC: Cost-effectiveness acceptability curve ; CE: Cost-effectiveness; CHEERS: Consolidated Health Economic Evaluation Reporting Guidelines; SEIN: Stichting Epilepsie Instellingen Nederland; SDD: Seizure dectection device ; Hr-QoL: Health-Related Quality of Life
]. Having (generalised or focal to bilateral) tonic-clonic seizures, particularly if nocturnal and unattended, constitutes the most significant SUDEP risk factor [
]. This poses an opportunity for seizure detection devices (SDDs), which might lower the morbidity and mortality risk in epilepsy and potentially reduce the burden [
Automated seizure detection using wearable devices: a clinical practice guideline of the international league against epilepsy and the international federation of clinical neurophysiology.
]. A previous prospective multicenter, video-controlled cohort study demonstrated good performance of NightWatch in adults, with 86% sensitivity and a median false alarm rate of 0.25 per person per night [
]. Yet economic studies addressing the cost-effectiveness of NightWatch and other SDDs are still lacking. Since no studies were found on this subject, this study aims to fill in that gap. As resources are scarce, evidence-based decisions on costs and effects are increasingly important in current health care decision-making [
]. This is a pressing question as SDDs rapidly emerged in epilepsy care while costs of these devices are substantial and often not reimbursed, thus causing health inequality. We, therefore, aimed to perform an economic evaluation from a societal perspective to examine whether implementation of NightWatch is preferable over usual care in terms of costs, effects and utilities.
2. Methods
This study followed Dutch guidelines for economic evaluations [
We used data from a prospective multicenter home-based implementation study,the Promoting implementation of seizure detection devices in epilepsy care (PROMISE) trial; NCT03909984. PROMISE included 60 children aged 4–16 years with at least one major nocturnal motor seizure per week, living at home and treated at a tertiary epilepsy center in the Netherlands (SEIN, Kempenhaeghe or University Medical Center Utrecht). Background information from the children and caregivers participating in the PROMISE study was extracted from the PROMISE database (Table 1).
Table 1Demographic characteristics of study particpants.
Baseline characteristics (N=41)
N
%
Characteristics of children
Female
18
44
Mean age
9.8 (SD 3.7)
-
Mean age at seizure onset
2.8 (SD 3.3)
-
Epilepsy etiology
Genetic
15
37
Structural
11
27
Unknown
15
37
Learning disability
29
71
Number of ASMs at start study
None
1
3
One
7
17
Two
11
27
Three
14
34
Four
5
12
Five
3
7
Characteristics of caregivers
Female
33
81
Mean age
40.9 (SD 6.2)
-
Marital status (living together)
28
68
Paid work
31
76
Mean no. of working hours/week
28.3 (SD 8.3)
-
*N: number; SD: standard deviation; ASMs: antiseizure medications.
The economic evaluation was executed from a societal perspective. This perspective accounts for both directs costs (i.e. health care costs) and indirect costs (i.e. lost productivity costs). The PROMISE study consisted of a two-month baseline period without any SDD used (comparator), followed by a two-month period with NightWatch use at home (intervention). Data for our analysis was collected between November 2018 and June 2020. The Research Ethics Committee of University Medical Center Utrecht approved the study (PROMISE: NL62995.041.17). The study devices and equipment were provided free of charge by the company that developed NightWatch (LivAssured). LivAssured had no role in the study design, analysis, or decision to submit for publication.
2.1.3 Outcomes
Caregivers from the PROMISE study were asked to complete online questionnaires before the baseline period (T0), at the end of the baseline period (T1) and the end of the intervention period (T2). T0 included questions on baseline characteristics of the child and the caregiver. We used validated questionnaires to measure caregiver's stress (Caregiver Strain Index [CSI]), quality of life (EQ-5D-5L), medical consumption (Institute for Medical Technology Assessment Medical Consumption Questionnaire [iMTA MCQ]) and productivity (Institute for Medical Technology Assessment Productivity Costs Questionnaire [iMTA PCQ]) at T1 and T2. The iMTA MCQ and iMTA PCQ were specifically adjusted to the care situation of a child with epilepsy; the iMTA MCQ covered questions about the medical consumption of the child and the caregiver, while the other questionnaires focused only on the caregiver. We asked the caregiver that took primary care of the child to complete all questionnaires. An English version of the CSI and EQ-5D-5L, and the adjusted Dutch version of the iMTA MCQ and iMTA PCQ can be found in the Supplementary material.
2.2 Data analyses
2.2.1 Missing data
Missing items at T1 or T2 were handled by mean imputation, consisting of the mean score of the non-missing data [
]. At T1 data of two participants was missing (5% of the total study population). At T2 data of fifteen participants was missing (37% of the total study population).
2.2.2 Effectiveness
The effectiveness of the intervention, compared to the baseline period, was measured by the CSI questionnaire on caregiver's stress (Supplementary material 1). Individual CSI scores were calculated by adding up all questions answered with ‘yes’ (1 point per question).
2.2.3 Utility
The EQ-5D-5L questionnaire on caregiver's quality of life (QoL) [
] was used to measure the utility of the intervention, compared to the baseline period. The five dimensions of the EQ-5D-5L questionnaire were summed into a health state, with the help of the Dutch EQ-5D-5L utility values (Supplementary material 2) [
Zorginstituut Nederland Ziektelast in de praktijk: De theorie en praktijk van het berekenen van ziektelast bij pakketbeoordelingen (Burden of disease in practice: calculating the burden of disease in package assessments).
The iMTA MCQ (Supplementary material A.3,B.3) and the iMTA PCQ (Supplementary material 4) were included to measure the societal costs. A bottom-up approach was used to estimate the health care costs; information on each element of used service was multiplied by an appropriate unit cost (reference cost) and summed to provide overall costs [
]. The cost prices of respite care were calculated by comparing the cost prices of different respite care providers, and taking the average cost price [
For irregular working days, an average working day of 8 hours is assumed; GP: General practitioner; GGZ: Geestelijke gezondheidszorg [mental healthcare].
37.62
Hourly wage informal care
15.16
For irregular working days, an average working day of 8 hours is assumed; GP: General practitioner; GGZ: Geestelijke gezondheidszorg [mental healthcare].
Statistical analyses were performed using SPSS V.27. We used non-parametric bootstrapping (1000 replications) to test for statistical differences in costs between the intervention and the baseline period. Microsoft Excel 2016 was used to quantify the uncertainty around the incremental cost-effectiveness ratio (ICER; 5000 bootstrap replications). The ICER represents the costs of an additional quality-adjusted life year (QALY) gained, and was used to estimate the cost-utility of the intervention compared to usual care. ICERs were estimated by dividing the incremental costs by the incremental quality-adjusted life-year (QALY). The bootstrapped cost-effectiveness ratios were presented in a cost-effectiveness plane. The choice to implement the intervention depended on the maximum amount of money society is prepared to pay for a gain in QALYs (willingness-to-pay), determined as the ‘threshold’. As previously estimated in a Swedish study, we used a threshold (ceiling ratio) of €50,000 for refractory epilepsy per QALY gained [
]. We constructed a cost-effectiveness acceptability curve (CEAC) and calculated the incremental costs per responder to show the probability of a cost-effective intervention at different thresholds.
2.2.6 Sensitivity analysis
We performed three one-way sensitivity analyses to check the potential influence of base-case assumptions on the study findings. (1) To analyze the influence of our choice of perspective on the costs, we performed the data analysis from a health care perspective instead of a societal perspective [
]. (2) We tested a different imputation method (i.e. individual mean imputation), which replaces missing data by the individual mean score of a complete answered questionnaire at an earlier or later moment. (3) To test whether the mean imputation method was an appropriate way to handle missing data, all missing data (n = 17) were excluded from the analysis.
3. Results
We collected data from the PROMISE trial, including 60 participants, between November 2018 and June 2020, data from 41 participants was available for analysis. There were no statistically significant differences in characteristics (mean age, mean age at seizure onset, epilepsy etiology, learning disability (yes/no), number of anti-seizure medications at start study) between the dropped-out (N = 19) and included participants (N = 41), so no baseline corrections were performed.
3.1 Total resource use and total societal costs
Total societal costs of the baseline period were on average €3238 per patient (Table 3), whereas after intervention this reduced to €2463. During baseline, the health care costs (child and caregiver) accounted for 90% (€2910) of the total costs, compared to 91% (€2250) during the intervention. The productivity costs were respectively 10% (€328) and 9% (€212) (Table 3).
Table 3Bootstrapped mean of the QALY, stress, and costs (€) per participant during the baseline period and the intervention.
Fig. 1A illustrates the cost-utility analysis' cost-effectiveness (CE) plane from a societal perspective, representing the uncertainty surrounding the costs per QALY ratio. Based on the cost-utility analysis, the NightWatch was a cost-effective treatment compared to usual care alone (95% CI €19,387 - €28,182). The NightWatch is less expensive than usual care alone and equally effective in terms of QALYs (Table 3).
Fig. 1Cost-effectiveness planes and cost-effectiveness acceptability curves for the NightWatch intervention.
*Fig. A.1. Cost-effectiveness plane, costs per QALY
The horizontal axis represents the additional effects in Quality Adjusted Life Years [QALY] of the intervention (NightWatch) compared to baseline (usual care) (0); The vertical axis represents the additional costs of the intervention compared to baseline (€ -775,49); The blue dots represent the bootstrapped Incremental Cost-effectiveness Ratios [ICERs].
**Fig. B.1. Cost-effectiveness acceptability curve, costs per QALY
The horizontal axis represents the ceiling ratio/threshold (for refractory epilepsy this is € 50.000); The vertical axis represents the threshold/willingness to pay; The blue line represents the probability of NightWatch being cost-effective (72% at a ceiling ratio of € 50.000).
***Fig. C.1. Cost-effectiveness plane, costs per stress score
The horizontal axis represents the additional effects in stress of the intervention compared to baseline (-0,91); The vertical axis represents the additional costs of the intervention compared to baseline (€ -775,49); The blue dots represent the bootstrapped Incremental.
****Fig. D.1. Cost-effectiveness acceptability curve, costs per stress score
The hrizontal axis represents the ceiling ratio/threshold; The vertical axis represents the threshold/willingness to pay; The definition of the clinical outcomes, in this case stress levels, differs per study, we could not determine the ceiling ratio (threshold) for NightWatch. Therefore, it is not possible to interpret the probabilities of NightWatch being cost-effective in terms of costs and stress.
The incremental costs divided by the incremental effect (score on the CSI) resulted in an ICER of €846 per patient. The uncertainty analysis of this ICER is presented in a CE plane in Fig. 1C. Most ICERs lie in the dominant southeast quadrant (82%), indicating that the NightWatch is less expensive and more effective compared to usual care (95% CI €376–€7946).
3.2.3 Sensitivity analyses
Results from the sensitivity analyses are provided in Table 3. Looking at the costs per QALY from a health care perspective, instead of a societal perspective, the probability of NightWatch being cost-effective decreased by 2%. Using the individual mean imputation method, the cost-effectiveness probabilities of NightWatch decreased to 46%. This method resulted in higher caregivers’ stress levels (8.02 vs. 7.11) and higher costs (3223 vs. 2463) during the intervention period, compared to the mean imputation method. By removing incomplete cases cost-effectiveness probabilities of NightWatch decreased to 33%. This method resulted in lower caregivers’ stress levels (7.00 vs. 8.02) during the baseline period and higher stress levels (8.02 vs. 7.11) during the intervention period, compared to the mean imputation method. Also, costs decreased (2504 vs. 3238) during the baseline period using this method. From both a societal perspective and a healthcare perspective, most of the savings occur in healthcare costs (i.e. €659).
4. Discussion
4.1 Study findings
Our cost-utility and cost-effectiveness analysis suggests that a two-months intervention with NightWatch saves costs, reduces stress, and is equally effective in terms of QALYs, compared to usual care without an SDD.
4.2 Generalisability
We could not compare our results directly to others, as comparable studies are lacking. Some reports of the impact of wearables on caregivers’ HR-QoL are available [
]. The caregiver burden scores from our study (mean QALY 0.90) were similar to the previously reported EQ-5D-5L scores of 86 caregivers of children with epilepsy (mean QALY 0.88) [
]. Compared with non-users, SDD users were significantly more likely to have been impacted by epilepsy in multiple HR-QoL domains. 80% of caregivers using an SDD (20% of total) reported a reduction in anxiety following SDD deployment. Of note, the SDD usage tended to be skewed toward younger age, and caregivers with higher-income, reflecting health care inequality. In-depth interviews with caregivers from the PROMISE study revealed that the amount of assurance NightWatch could offer, strongly depended on the ability to reduce their protective behavior as well as their resilience to handle the potential extra burden of care (e.g. due to false alarms or technical problems) [
The total price of NightWatch (€1500) is on the higher end of the spectrum compared to other SDDs. Yet, according to recently published standards, NightWatch’ level of performance evidence is relatively high, and validation in adults support accurate detection of major nocturnal motor seizures [
Automated seizure detection using wearable devices: a clinical practice guideline of the international league against epilepsy and the international federation of clinical neurophysiology.
The high probability of NightWatch being cost-effective (72%) found in our study might encourage NightWatch implementation. These results should, however, be interpreted with caution due to the small sample size and short time period. The cost-effectiveness of NightWatch was mainly due to the decrease in costs during the intervention, while effects on stress and QoL were less pronounced. Alternatively, the NightWatch is already manifesting its potential positive impact within this time frame but may be outweighed by alarm fatigue, thus resulting in unaltered levels of parental stress and QALY's. Although the EQ-5D-5L is an extensively validated questionnaire often used for the assessment of QoL in health technology assessment studies, it might not be discriminative enough to measure an effect in our study. The relatively small sample size might be another explanation for the lack of gain in QoL found in this study. Also, within this short time horizon it is uncertain whether the potential costs associated with the seizures are accurately captured. Another important unknown is the long-term retention rate (due to alarm fatigue) and the impact of NightWatch on SUDEP prevention, as this could significantly affect the cost-effectiveness. We speculate that alarm fatigue may vary over time particularly in periods with high parental care burden [
]. We lack prospective long-term data to monitor the impact of NightWatch or any other SDD on survival. A retrospective analysis in two residential units demonstrated that the center with the lowest grade of supervision had the highest incidence of SUDEP [
]. The significant contrast between sites was due to a central acoustic system, with only a minority of participants using additional SDDs. More economic evaluations on different SDDs could be helpful to get more insight in probabilities to improve the financial accessibility to SDDs. The overall burden for caregivers of children with epilepsy cannot be fully alleviated, but the use of SDDs such as NightWatch could decrease the burden. Another limitation of our short-term evaluation is that we could not study how much medication up titration NightWatch may create. NightWatch implementation may unveil a higher than previously reported seizure frequency and, in turn, impact epilepsy management. Despite these limitations, we found an evident effect in cost-effectiveness during the short time horizon and sensitivity analyses demonstrated result robustness. For further research we suggest to expand the time horizon and sample size to identify the long-term effects of SDD intervention, like SUDEP, visits the emergency room and alarm fatigue.
We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.
Funding source
This work was supported by the Netherlands Organization for Health Research and Development (ZonMw) [Project number: 446001009]; EpilepsieNL and Health Holland [Project number: 40-41200-98-9335]; and the 'Christelijke Vereniging voor de Verpleging van Lijders aan Epilepsie'.
Declarations of Competing Interest
AE, AvW and SMAE have no disclosures to report. RDT received research support from Medtronic, the Human Measurement Models Programme co-funded by Health∼Holland, Top Sector Life Sciences & Health and ZonMw under grant agreement 114025101 ([email protected]), Michaal J Fox Foundation and received fees as speaker or consultant from Theravance Biopharma, Arvelle, Medtronic, Zogenix, UCB, NewLife Wearables and Novartis. LivAssured, the company developing the NightWatch device, has obtained an exclusive license to implement or use the data in the future for commercial purposes or in commercial enterprises in exchange for a percentage of the revenue for the institutions (SEIN, University Medical Center Utrecht and Kempenhaeghe). The Dutch Tele-Epilepsy Consortium will receive more research funds from the institutes as a consequence of this license receive money if NightWatch turns to be profitable. None of the authors has financial interests in LivAssured nor has received or will receive income from future sales of the NightWatch. The funding sources had no role in the study design, analysis, or decision to submit for publication.
Acknowledgments
We would like to thank all children and their caregivers for participating in the PROMISE study and all the Dutch TeleEpilepsy Consortium members for their contribution to this study. We are grateful to Prof. J.W. Sander for critically reviewing the manuscript.
Automated seizure detection using wearable devices: a clinical practice guideline of the international league against epilepsy and the international federation of clinical neurophysiology.
Ziektelast in de praktijk: De theorie en praktijk van het berekenen van ziektelast bij pakketbeoordelingen (Burden of disease in practice: calculating the burden of disease in package assessments).
For irregular working days, an average working day of 8 hours is assumed; GP: General practitioner; GGZ: Geestelijke gezondheidszorg [mental healthcare].
00182-0&id=gr1.jpg){kind=link}