Advertisement

Performance of the GAD-7 in adults with dissociative seizures

Open AccessPublished:November 22, 2022DOI:https://doi.org/10.1016/j.seizure.2022.11.011

      Highlights

      • We validated the GAD-7 anxiety scale in adults with dissociative seizures (DS).
      • We found high internal consistency for the GAD-7 in adults with DS.
      • Factor Analysis indicated one main factor and measurement invariance for the GAD-7.
      • GAD-7 has weak diagnostic utility for generalised anxiety disorder in adults with DS.
      • Clinical correlations with the GAD-7 support a fear-avoidance treatment model for DS.

      Abstract

      Purpose

      Little is known about the accuracy of the GAD-7, a self-report anxiety measure, in detecting generalised anxiety disorder (GAD) in people with dissociative seizures (DS). We evaluated the reliability, validity and uniformity of the GAD-7 using a diagnosis of GAD on the Mini-International Neuropsychiatric Interview as a reference.

      Methods

      We assessed 368 adults with DS at the pre-randomisation phase of the CODES trial. Factor analysis for categorical data assessed GAD-7 uniformity. Diagnostic accuracy was assessed by estimating the area under the curve (AUC). We evaluated discriminant validity, reviewed data on convergent validity and calculated internal consistency. We explored correlations between GAD-7 scores and monthly DS frequency, frequency of severe seizures and measures of behavioural and emotional avoidance.

      Results

      Internal consistency of the GAD-7 was high (α = 0.92). Factor analysis elicited one main factor and general measurement invariance. Diagnostic accuracy was fair (AUC = 0.72) but the best balance of sensitivity and specificity occurred at a cut-off of ≥12 and still had a specificity rate of only 68%. Discriminant and convergent validity were good. GAD-7 scores correlated positively with DS frequency, severe seizure frequency, behavioural and emotional avoidance (all p < 0.001).

      Conclusion

      Findings regarding internal consistency and factor structure parallel previous psychometric evaluations of the GAD-7. Correlations between GAD-7 scores and DS occurrence/severity and avoidance are evidence of the concept validity of GAD-7 and provide further support for a fear-avoidance treatment model for DS. However, the utility of the GAD-7 as a diagnostic instrument for generalised anxiety disorder is limited in patients with DS.

      Keywords

      1. Introduction

      Dissociative seizures (DS) are commonly described as involuntary behaviours, movements and sensations which strongly resemble epileptic seizures or syncope but cannot be explained by these or other medical disorders [
      • Lesser R.P.
      Psychogenic seizures.
      ,
      • Reuber M.
      Psychogenic nonepileptic seizures: answers and questions.
      ,
      • Brown R.J.
      • Reuber M.
      Psychological and psychiatric aspects of psychogenic non-epileptic seizures (PNES): a systematic review.
      ]. DS are characterised by reduced self-control and typically involve impairment of awareness. The International Classification of Diseases-10 (ICD-10) classified them as dissociative (conversion) convulsions and the recent ICD-11 guidelines place them in the spectrum of ‘dissociative neurological symptom disorder’ [
      World Health Organisation
      The ICD-10 classification of mental and behavioural disorders.
      ,

      World Health Organisation. International classification of diseases for mortality and morbidity statistics (11th Revision); 2018.

      ].
      Despite ongoing uncertainties about the aetiology, it is well accepted that psychological factors are likely to be relevant in DS. Patients with DS have consistently been shown to have a higher incidence of psychiatric disorders, including anxiety disorders (11–50%), compared to the general population [
      • Brown R.J.
      • Reuber M.
      Psychological and psychiatric aspects of psychogenic non-epileptic seizures (PNES): a systematic review.
      ]. Previous studies also report increased levels of anxiety and depression in patients with DS compared to epilepsy (e.g., [
      • Goldstein L.H.
      • Mellers J.D.
      Ictal symptoms of anxiety, avoidance behaviour, and dissociation in patients with dissociative seizures.
      ,
      • Reuber M.
      • Howlett S.
      • Khan A.
      • Grunewald R.A.
      Non-epileptic seizures and other functional neurological symptoms: predisposing, precipitating, and perpetuating factors.
      ,
      • Moore P.M.
      • Baker G.A.
      Non-epileptic attack disorder: a psychological perspective.
      ,
      • Hixson J.D.
      • Balcer L.J.
      • Glosser G.
      • French J.A.
      Fear sensitivity and the psychological profile of patients with psychogenic nonepileptic seizures.
      ,
      • Dimaro L.V.
      • Dawson D.L.
      • Roberts N.A.
      • Brown I.
      • Moghaddam N.G.
      • Reuber M.
      Anxiety and avoidance in psychogenic nonepileptic seizures: the role of implicit and explicit anxiety.
      ]) as well as heightened fear sensitivity [
      • Hixson J.D.
      • Balcer L.J.
      • Glosser G.
      • French J.A.
      Fear sensitivity and the psychological profile of patients with psychogenic nonepileptic seizures.
      ] and agoraphobic avoidance [
      • Goldstein L.H.
      • Mellers J.D.
      Ictal symptoms of anxiety, avoidance behaviour, and dissociation in patients with dissociative seizures.
      ,
      • Dimaro L.V.
      • Dawson D.L.
      • Roberts N.A.
      • Brown I.
      • Moghaddam N.G.
      • Reuber M.
      Anxiety and avoidance in psychogenic nonepileptic seizures: the role of implicit and explicit anxiety.
      ]. Similarities in anxiety-related symptomatology between DS and panic disorder have also been identified [
      • Vein A.M.
      • Djukova G.M.
      • Vorobieva O.V.
      Is panic attack a mask of psychogenic seizures?–A comparative analysis of phenomenology of psychogenic seizures and panic attacks.
      ]. Many DS patients experience autonomic arousal, as well as somatic and cognitive symptoms during their attacks, characteristics which also occur during a panic attack. However, these experiences are often not attributed to panic or paroxysmal anxiety by patients with DS [
      • Goldstein L.H.
      • Mellers J.D.
      Ictal symptoms of anxiety, avoidance behaviour, and dissociation in patients with dissociative seizures.
      ]. Consistent with this, a study by Dimaro et al. [
      • Dimaro L.V.
      • Dawson D.L.
      • Roberts N.A.
      • Brown I.
      • Moghaddam N.G.
      • Reuber M.
      Anxiety and avoidance in psychogenic nonepileptic seizures: the role of implicit and explicit anxiety.
      ] demonstrated a strong relationship between implicit anxiety scores and DS frequency, suggesting that anxiety plays a key role in the aetiology. The study also highlighted a correlation between a measure of explicit anxiety (the Spielberger State-Trait Anxiety scale [
      • Spielberger C.D.
      State-trait anxiety inventory.
      ]) and experiential avoidance (the Multidimensional Experiential Avoidance Questionnaire [
      • Gámez W.
      • Chmielewski M.S.
      • Kotov R.
      • Ruggero C.J.
      • Watson D.
      Development of a measure of experiential avoidance: the multidimensional experiential avoidance questionnaire.
      ]) in people with DS. Indeed, the fear-avoidance model [
      • Lang P.J.
      ,
      • Mowrer O.H.
      Learning theory and behaviour.
      ] has been used to underpin treatment of adults with DS [
      • Chalder T.
      Non-epileptic attacks: a cognitive behavioural approach in a single case with a four-year follow-up.
      ,
      • Goldstein L.H.
      • Deale A.C.
      • Mitchell-O'Malley S.J.
      • Toone B.K.
      • Mellers J.D
      An evaluation of cognitive behavioral therapy as a treatment for dissociative seizures: a pilot study.
      ,
      • Goldstein L.H.
      • Chalder T.
      • Chigwedere C.
      • Khondoker M.R.
      • Moriarty J.
      • Toone B.K.
      • et al.
      Cognitive-behavioral therapy for psychogenic nonepileptic seizures: a pilot RCT.
      ,
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Reuber M.
      • et al.
      Cognitive behavioural therapy for adults with dissociative seizures (CODES): a pragmatic, multicentre, randomised controlled trial.
      ] whereby a person's activities / behaviours and experiences are modified through fear of having DS and lifestyles become increasingly restricted; in the treatment of DS by our group [
      • Goldstein L.H.
      • Robinson E.J.
      • Pilecka I.
      • Perdue I.
      • Mosweu I.
      • Read J.
      • et al.
      Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
      ] dealing with avoidance behaviour is a key aspect.
      The Generalized Anxiety Disorder Assessment 7-item scale (GAD-7) is a practical, self-report instrument widely used to screen for anxiety in primary care and research settings [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Lowe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ,
      • Löwe B.
      • Decker O.
      • Müller S.
      • Brähler E.
      • Schellberg D.
      • Herzog W.
      • et al.
      Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population.
      ]. It has also been used to assess anxiety in patients with DS (e.g., [
      • Valente K.D.
      • Alessi R.
      • Baroni G.
      • Marin R.
      • Dos Santos B.
      • Palmini A
      The COVID-19 outbreak and PNES: the impact of a ubiquitously felt stressor.
      ,
      • Abe C.
      • Denney D.
      • Doyle A.
      • Cullum M.
      • Adams J.
      • Perven G.
      • et al.
      Comparison of psychiatric comorbidities and impact on quality of life in patients with epilepsy or psychogenic nonepileptic spells.
      ,
      • Novakova B.
      • Harris P.R.
      • Rawlings G.H.
      • Reuber M.
      Coping with stress: a pilot study of a self-help stress management intervention for patients with epileptic or psychogenic nonepileptic seizures.
      ,
      • Rawlings G.H.
      • Brown I.
      • Reuber M.
      Predictors of health-related quality of life in patients with epilepsy and psychogenic nonepileptic seizures.
      ]). Despite its wide applications and use, the psychometric properties of the GAD-7 have not yet been examined in detail in people with DS; neither has the optimal cut-off score for an anxiety disorder been determined. The original validation article proposed a cut-off score of ≥10 to be the optimal predictor of clinically relevant GAD diagnoses [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Lowe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ], and this is widely applied in clinical services. Spitzer et al. [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Lowe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ] suggested that the GAD-7 might assume utility in assessing symptom severity and measuring change across time.
      Our previous study exploring an optimal PHQ-9 cut-off score for the detection of clinically relevant depression in patients with DS [
      • Baldellou Lopez M.
      • Goldstein L.H.
      • Robinson E.J.
      • Vitoratou S.
      • Chalder T.
      • Carson A.
      • et al.
      Validation of the PHQ-9 in adults with dissociative seizures.
      ] found that optimal cut-offs should be higher than those typically used for the scale. In this study we set out to determine whether the existing GAD-7 cut-offs are suitable for DS patients, or whether different cut-off scores may be more appropriate for this patient population.
      The aims of this study were to
      • (a)
        assess the psychometric properties of the GAD-7 in patients with DS by examining its structural validity and reliability
      • (b)
        investigate the diagnostic accuracy of the GAD-7 in ascertaining anxiety symptom levels in DS patients likely to indicate the diagnosis of a current clinical anxiety disorder. We used the GAD-7 data from the CODES (COgnitive behaviour therapy vs standardized medical care for adults with Dissociative non-Epileptic Seizures) multi-centre randomised controlled trial ([19, 20] and the Mini - International Neuropsychiatric Interview [
        • Sheehan D.V.
        • Lecrubier Y.
        • Sheehan K.H.
        • Amorim P.
        • Janavs J.
        • Weiller E.
        The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.
        ] as the diagnostic reference
      • (c)
        explore clinical correlates of GAD-7 scores in people with DS, guided by Dimaro et al's [
        • Dimaro L.V.
        • Dawson D.L.
        • Roberts N.A.
        • Brown I.
        • Moghaddam N.G.
        • Reuber M.
        Anxiety and avoidance in psychogenic nonepileptic seizures: the role of implicit and explicit anxiety.
        ] methodology, on the assumption that the GAD-7 would, as in its original standardisation, prove to have good internal consistency (reliability). As indicated above, Dimaro et al. [
        • Dimaro L.V.
        • Dawson D.L.
        • Roberts N.A.
        • Brown I.
        • Moghaddam N.G.
        • Reuber M.
        Anxiety and avoidance in psychogenic nonepileptic seizures: the role of implicit and explicit anxiety.
        ] had investigated the relationships between anxiety and DS frequency and between anxiety and avoidance behaviour. Therefore, we examined the relationship between our current measure of explicit anxiety and seizure frequency. We also explored the correlation between GAD-7 scores and the Avoidance of People Places and Situations, a measure of behavioural avoidance used in the CODES study [
        • Goldstein L.H.
        • Robinson E.J.
        • Mellers J.D.C.
        • Stone J.
        • Carson A.
        • Chalder T.
        • et al.
        Psychological and demographic characteristics of 368 patients with dissociative seizures: data from the CODES cohort.
        ], and the Beliefs About Emotions Scale [
        • Rimes K.A.
        • Chalder T.
        The beliefs about emotions scale: validity, reliability and sensitivity to change.
        ] which may be viewed as a measure of emotional and, potentially, experiential avoidance. By undertaking these exploratory analyses, we set out to find further supportive evidence for the contribution of anxiety and avoidance behaviours to DS.

      2. Methods

      2.1 Participants

      Data were collected from 368 adults with DS who were recruited to the CODES trial, a pragmatic multi-centre randomised control trial that evaluated clinical outcomes and health service use following dissociative seizure specific cognitive behavioural therapy plus standardised medical care, vs standardised medical care alone. Data were collected between January 2015 and May 2017, following the provision of informed consent and prior to randomisation of participants. Details of inclusion and exclusion criteria and the two-stage recruitment process can be found elsewhere [
      • Goldstein L.H.
      • Robinson E.J.
      • Pilecka I.
      • Perdue I.
      • Mosweu I.
      • Read J.
      • et al.
      Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
      ,
      • Goldstein L.H.
      • Mellers J.D.
      • Landau S.
      • Stone J.
      • Carson A.
      • Medford N.
      • et al.
      Cognitive behavioural therapy vs standardised medical care for adults with dissociative non-epileptic seizures (CODES): a multicentre randomised controlled trial protocol.
      ].

      2.2 Measures

      The Mini-International Neuropsychiatric Interview (M.I.N.I.; version 6) [
      • Sheehan D.V.
      • Lecrubier Y.
      • Sheehan K.H.
      • Amorim P.
      • Janavs J.
      • Weiller E.
      The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.
      ] was used in this study to provide a reference for a clinical diagnosis of Generalised Anxiety Disorder (GAD). This structured interview is used to identify psychiatric diagnoses according to DSM-IV criteria and also the ICD-10 [
      World Health Organisation
      The ICD-10 classification of mental and behavioural disorders.
      ]. All diagnoses except one (Antisocial Personality Disorder) are from Axis I. There are 17 modules which enquire about the symptoms for major psychiatric diagnoses, one of which is GAD. The reliability and validity of the M.I.N.I. have been described [
      • Lecrubier Y.
      • Sheehan D.V.
      • Weiller E.
      • Amorim P.
      • Bonora I.
      • Sheehan K.H.
      The mini international neuropsychiatric interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI.
      ,
      • Sheehan D.V.
      • Lecrubier Y.
      • Harnett Sheehan K.
      • Janavs J.
      • Weiller E.
      • et al.
      The validity of the mini international neuropsychiatric interview (MINI) according to the SCID-P and its reliability.
      ]. Those administering the M.I.N.I. in the current study were, as previously described [
      • Baldellou Lopez M.
      • Goldstein L.H.
      • Robinson E.J.
      • Vitoratou S.
      • Chalder T.
      • Carson A.
      • et al.
      Validation of the PHQ-9 in adults with dissociative seizures.
      ], postgraduate research assistants with varied professional backgrounds and prior experience in mental health research but who had received a full-day's training from a neuropsychiatrist within the CODES project team in administering the M.I.N.I. The prevalence of the different psychiatric diagnoses identified using the M.I.N.I. in the 368 trial participants has been described elsewhere (e.g., [
      • Goldstein L.H.
      • Robinson E.J.
      • Pilecka I.
      • Perdue I.
      • Mosweu I.
      • Read J.
      • et al.
      Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
      ,
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Chalder T.
      • et al.
      Psychological and demographic characteristics of 368 patients with dissociative seizures: data from the CODES cohort.
      ]).
      The General Anxiety Disorder Assessment-7 (GAD-7) [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Lowe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ] is used widely to assess the presence of generalised anxiety and its severity. The original standardisation study in a clinical population indicated good reliability (Cronbach's alpha = 0.92; test-retest-reliability intraclass correlation = 0.83) and it has been shown to have good levels of criterion, construct, factorial and procedural validity [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Lowe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ]. Higher levels of anxiety are represented by higher scores. Scores of ≥ 10 indicate high likelihood of GAD and it has been suggested that scores of 5, 10 and 15 might be used to represent mild, moderate and severe levels of generalised anxiety. The cut-off score of ≥10 was reported to optimise sensitivity (89%) and specificity (82%) when compared to a mental health practitioner diagnosis using DSM-IV [
      American Psychiatric Association
      Diagnostic and statistical manual of mental disorders (4th Edition- DSM-IV).
      ] diagnostic criteria and a structured psychiatric interview.
      Dissociative seizure frequency was measured in terms of monthly seizure frequency (captured from a seizure diary or a single question – see Goldstein et al. [
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Reuber M.
      • et al.
      Cognitive behavioural therapy for adults with dissociative seizures (CODES): a pragmatic, multicentre, randomised controlled trial.
      ,
      • Goldstein L.H.
      • Robinson E.J.
      • Pilecka I.
      • Perdue I.
      • Mosweu I.
      • Read J.
      • et al.
      Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
      ] for further explanation of calculation of the measure of DS frequency). In addition, participants recorded the number of severe seizures they had experienced over this time, either via seizure diaries or via a single question.
      The Avoidance of People Places and Situations Scale (APPS) is a three-item scale designed for the CODES study [
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Chalder T.
      • et al.
      Psychological and demographic characteristics of 368 patients with dissociative seizures: data from the CODES cohort.
      ] to measure the avoidance of people, activities and situations due to fear of having a seizure. The specific questions were: “How much have you avoided specific situations (e.g. being out in public alone, social gatherings, using public transport) for fear of having a seizure?”; “How much have you avoided other people for fear of having a seizure?” and “How much have you avoided specific activities (e.g. physical exertion, bathing unsupervised) for fear of having a seizure?”. It was used in this study as a measure of behavioural avoidance. Each item was rated 0 (“never avoid”) to 10 (“always avoid”) with a total possible score of 30. The internal consistency of this scale is good (Cronbach's alpha = 0.83) [
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Chalder T.
      • et al.
      Psychological and demographic characteristics of 368 patients with dissociative seizures: data from the CODES cohort.
      ].
      The Belief About Emotions Scale (BES) [
      • Rimes K.A.
      • Chalder T.
      The beliefs about emotions scale: validity, reliability and sensitivity to change.
      ] was used here as a measure of emotional avoidance. It is a 12-item scale that measures beliefs about the unacceptability of either experiencing or expressing negative emotions. Example items include “If I am having difficulties it is important to put on a brave face”, “I should not let myself give in to negative feelings” and “It would be a sign of weakness to show my emotions in public”. The scale has good internal consistency (Cronbach's alpha (α) for patients with Chronic Fatigue Syndrome = 0.91, for controls α = 0.88) and validity; the scale has shown sensitivity to change [
      • Rimes K.A.
      • Chalder T.
      The beliefs about emotions scale: validity, reliability and sensitivity to change.
      ]. Total scores range from 0 to 72 and higher scores reflect stronger self-reported beliefs about the unacceptability of negative emotions.
      All of these measures (including the M.I.N.I. and the GAD-7) were administered to participants on the same occasion, following the giving of informed consent and prior to randomisation into the treatment phase of the CODES trial.
      In addition, demographic and clinical information was collected about participants that related to their age, gender, dissociative seizure history and semiology, education and employment status, relationships and previous help-seeking for mental health conditions.

      2.3 Statistical analysis

      2.3.1 Structural validity and invariance

      As the data were categorical the weighted least squares estimator method (WLSMV) [

      Muthén B., du Toit S.H.C., Spisic D. Robust inference using weighted least squares and quadratic estimating equations in latent variable modelling with categorical and continuous outcomes. (Technical report). Retrieved from https://www.statmodel.com/download/Article_075.pdf; 1997.

      ] was used to assess the dimensionality (structural validity) of the GAD-7.
      Exploratory factor analysis (EFA) was first conducted on a random split half of the data. Confirmatory factor analysis (CFA) was then conducted on the other half.
      In EFA, the Guttman-Kaiser criterion (the number of eigenvalues above 1 [
      • Guttman L.
      Some necessary conditions for common-factor analysis.
      ,
      • Kaiser H.F.
      The application of electronic computers to factor analysis.
      ]) and Parallel Analysis (the number of eigenvalues from the sample that are larger than the average of 1000 bootstrapped samples; [
      • Horn J.L.
      A rationale and test for the number of factors in factor analysis.
      ]) were both used to determine how many factors to extract. The latter was computed in the GAD-7 data by using the R package ‘random.polychor.pa’ for categorical data [

      Presaghi F., Desimoni M. Random polychor PA: a parallel analysis with polychoric correlation matrices. R package version 1.1.4-3. 2019.

      ]. Cattell's [
      • Cattell R.B.
      The scree test for the number of factors.
      ] scree plot was used to visualize the results. CFA was used to confirm the fit of the EFA suggested solutions.
      The multiple indicator multiple cause (MIMIC) model [

      Muthén B., du Toit S.H.C., Spisic D. Robust inference using weighted least squares and quadratic estimating equations in latent variable modelling with categorical and continuous outcomes. (Technical report). Retrieved from https://www.statmodel.com/download/Article_075.pdf; 1997.

      ,
      • Joreskog K.G.
      • Goldberger A.S.
      Estimation of a model with multiple indicators and multiple causes of a single latent variable.
      ] was used to evaluate the measurement invariance in relation to the gender of the participants. If the direct effect of gender on any item is significant, then measurement non-invariance (bias) with respect to gender is evident for that item.
      Relative and absolute fit indices of the models were computed to determine how many factors to retain and to assess the model fit to the data. The goodness of fit indices included the relative chi-square (Relative χ2: values <5 suggest a close fit) [
      • West S.G.
      • Taylor A.B.
      • Wu W.
      Model fit and model selection in structural equation modeling.
      ], the Root Mean Square Error of Approximation (RMSEA 90% CI: values <0.05 suggest close fit; values <0.08 indicate adequate fit) [
      • Hu L.T.
      • Bentler P.M.
      Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural equation modeling: a.
      ], the Comparative Fit Index (CFI: values > 0.95 are required for close fit) [
      • Hu L.T.
      • Bentler P.M.
      Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural equation modeling: a.
      ], the Tucker-Lewis Index (TLI: values >0.95 indicate close fit) [
      • Bentler P.M.
      • Bonett D.G.
      Significance tests and goodness of fit in the analysis of covariance structures.
      ], and the Standardized Root Mean Square Residual (SRMR: <0.05 are required for good fit) [
      • Hu L.T.
      • Bentler P.M.
      Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural equation modeling: a.
      ].

      2.3.2 Reliability

      The reliability of the GAD-7 in terms of internal consistency was assessed through Cronbach's alpha [
      • Cronbach L.J.
      Coefficient alpha and the internal structure of tests.
      ,
      • McDonald R.P.
      Test theory: a unified treatment.
      ] and McDonald's Omega [
      • McDonald R.P.
      Test theory: a unified treatment.
      ] (α and ω values above 0.7 were acceptable). Cronbach's alpha if item deleted (AID), inter-item correlations (IIC; satisfactory values between 0.3 and 0.8) and item total correlations (ITC; satisfactory item total correlation is indicated by values between 0.3 and 0.8) were also used to assess internal consistency [
      • Nunnally J.C.
      • Bernstein I.H.
      Psychometric theory.
      ].

      2.3.3 Validity

      Discriminant validity was assessed by comparing GAD-7 scores for participants who did or did not obtain a diagnosis of GAD on the M.I.N.I.
      For this task we also created the receiver operating characteristic curves (ROC) [
      • Hanley J.A.
      • McNeil B.J.
      The meaning and use of the area under a receiver operating characteristic (ROC) curve.
      ] and we also performed ROC regression analysis [
      • Janes H.
      • Longton G.
      • Pepe M.
      Accommodating covariates in ROC analysis.
      ] to assess the diagnostic accuracy of the GAD-7 in the presence of covariates (gender and age). The discriminative validity of each score is evaluated by the area under the curve (AUC) where values 0.9–1 indicate very good validity, 0.8–0.9 good validity, 0.7–0.8 fair validity, 0.6–0.7 poor validity and 0.5–0.6 failed to provide evidence for validity. We also report on Youden's index [
      • Youden W.J.
      Index for rating diagnostic tests.
      ], which varies between zero and 1 (perfect test without false negatives or false positives).
      Evidence of convergent validity of the scale, for example with other measures of mood, distress and quality of life in this population have been presented elsewhere [
      • Baldellou Lopez M.
      • Goldstein L.H.
      • Robinson E.J.
      • Vitoratou S.
      • Chalder T.
      • Carson A.
      • et al.
      Validation of the PHQ-9 in adults with dissociative seizures.
      ].
      Analyses were conducted in MPLUS v1.8 [
      • Muthén L.K.
      • Muthén B.O.
      Mplus user’s guide.
      ] and STATA 17.0 (StataCorp. 2021). Complete data were available for most of the analysis. In the rare occasion of one or two responses missing (out of 368), listwise deletion was used.

      3. Results

      3.1 Participants

      In our sample of 368 adults with DS, 266 (72%) were women. The sample's median age was 35 years. The median age at DS onset was 29 years (IQR 19, 42) and modal age at onset was 19 years. The median length of time for which participants had experienced their DS prior to diagnosis in the CODES trial was 3 years (IQR 1, 8); mean duration of the disorder prior to diagnosis in the CODES trial was 6.2 (sd 8.8) years. Over half of the participants (n = 195; 53%) had received their diagnosis using video-encephalography (video-EEG). Where video-EEG had not been undertaken, diagnosis was achieved based on consensus [
      • Goldstein L.H.
      • Robinson E.J.
      • Pilecka I.
      • Perdue I.
      • Mosweu I.
      • Read J.
      • et al.
      Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
      ]. In total, 130 (36%) had predominantly hypokinetic and 236 (65%) participants had predominantly hyperkinetic DS as rated by their clinician; this information was unavailable for two participants. Over half the sample (53%) were married or cohabiting, as opposed to being single, widowed or divorced. The majority of the sample (56%) had completed secondary or vocational education. Only 33% were currently in employment or education and there were high rates of receipt of state financial disability benefits (87%) by those aged <65 years, irrespective of whether or not they were working. Forty percent of the total sample reported having a carer who, in 53% of the cases, was their partner. The majority of the sample (241; 65%) reported they had previously sought assistance for a mental health problem. Further details about the sample can be found elsewhere [
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Reuber M.
      • et al.
      Cognitive behavioural therapy for adults with dissociative seizures (CODES): a pragmatic, multicentre, randomised controlled trial.
      ,
      • Goldstein L.H.
      • Robinson E.J.
      • Pilecka I.
      • Perdue I.
      • Mosweu I.
      • Read J.
      • et al.
      Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
      ,
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Chalder T.
      • et al.
      Psychological and demographic characteristics of 368 patients with dissociative seizures: data from the CODES cohort.
      ]. As shown in Table 1 the mean GAD-7 score in this sample was 9.8 (sd 6.2). In addition, the mean APPS score was 17.6 (sd 9.0) and the mean BES score was 41.9 (sd 16.9).
      Table 1Descriptive indices, associations with age, factor analysis loadings, and reliability indices of the GAD-7 (N = 368).
      Itemmean (sd)Median (min -max)Age rho/rLoadings EFA (CFA)ITCAID
      GAD-7_01 Feeling nervous, anxious, or on edge1.4 (1.0)1 (0–3)−0.0750.87 (1.00)0.770.90
      GAD-7_02 Not being able to stop or control worrying1.4 (1.1)1 (0–3)−0.0580.93 (1.08)0.820.90
      GAD-7_03 Worrying too much about different things1.5 (1.1)1 (0–3)−0.132*0.88 (1.02)0.780.90
      GAD-7_04 Having trouble relaxing1.7 (1.1)2 (0–3)0.0020.79 (0.92)0.770.90
      GAD-7_05 Being so restless that it is hard to sit still1.1 (1.1)1 (0–3)0.0380.66 (0.76)0.640.91
      GAD-7_06 Become easily annoyed or irritable1.6 (1.1)2 (0–3)−0.105*0.74 (0.86)0.690.91
      GAD-7_07 Feeling afraid as if something awful might happen1.1 (1.1)1 (0–3)−0.0390.82 (0.95)0.730.91
      GAD-7 total9.8 (6.2)10 (0–21)−0.061α=0.917, ω = 0.916
      rho = Spearman correlation coefficient; ITC = Item-total correlation; AID = Alpha if item deleted *p < 0.05; **p < 0.01.
      While full details about M.I.N.I. diagnoses are reported elsewhere [
      • Goldstein L.H.
      • Robinson E.J.
      • Pilecka I.
      • Perdue I.
      • Mosweu I.
      • Read J.
      • et al.
      Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
      ,
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Chalder T.
      • et al.
      Psychological and demographic characteristics of 368 patients with dissociative seizures: data from the CODES cohort.
      ] the five most frequently identified M.I.N.I.-confirmed diagnoses were agoraphobia (45%), major depressive disorder (31%), generalised anxiety disorder (29%), posttraumatic stress disorder (23%) and social anxiety disorder (20%).

      3.2 Structural validity and invariance

      Exploratory factor analysis for the GAD-7 was performed on the first random half of the sample (N = 194) and CFA on the second (N = 174). The sample size was adequate for each model, as only seven items were involved and a one-to-two factor solution was anticipated [
      • Kyriazos T.A.
      Applied psychometrics: sample size and ssmple power considerations in factor analysis (EFA, CFA) and SEM in general.
      ].
      The sample correlation matrix that emerged had only one eigenvalue above one (4.9) with the second being as low as 0.6. As depicted in the scree plot (Fig. 1), both parallel analysis and Guttman-Kaiser criteria suggested retaining one factor. The goodness of fit indices indicated acceptable fit (sample 1: rel χ2=3.7, RMSEA=0.12 90% CI (0.085, 0.15), CFI=0.99, TLI=0.98, SRMR=0.059). Increasing the number of factors to two produced a second factor mainly consisting of secondary cross-loadings. Therefore, the one factor solution was accepted and confirmed in CFA (sample 2: rel χ2=7.2, RMSEA=0.19 90% CI (0.156, 0.225), CFI=0.99, TLI=0.98, SRMR=0.047). All items had very strong loadings on the underlying trait (Table 1).
      According to the MIMIC model in the full sample (rel χ2=2,9, RMSEA=0.072 90% CI (0.054, 0.091), CFI=0.99, TLI=0.99, SRMR=0.044), two significant direct effects were present with respect to age, adjusted for gender. In particular, negative age effects were present on the GAD-7_03 item (Worrying too much about different things; de=−0.025, p = 0.008) and item GAD-7_06 (Become easily annoyed or irritable; de=−0.015, p = 0.027). For gender (adjusted for age) no significant direct effects emerged.

      3.3 Reliability and validity

      The internal consistency of the GAD-7 was satisfactory with both alpha and omega being higher than 0.9 (α = 0.917, ω = 0.916). No problematic items occurred (Table 1). Age correlated negatively and weakly with only two items (i.e., lower age was associated with higher anxiety scores on these measures; Table 1) and no gender differences occurred (p > 0.1 in all cases).
      According to ROC curve analysis, the diagnostic accuracy of the GAD-7 total score was fair (AUC = 0.72, 95% CI (0.67, 0.79); Fig. 2). Neither participants’ age nor gender affected the AUC significantly. Table 2 presents the sensitivity and the specificity of the GAD-7 scores at different levels. According to Youden's J criterion, a cut-off of 12 should be used as it allows for the largest J index. At this value, we observe an acceptable balance between the sensitivity and the specificity of the tool (approximately 68%). Above the value of 12 the sensitivity increases and below this value the specificity of the GAD-7 increases.
      Fig. 2
      Fig. 2Receiver operating characteristic curve for the GAD-7 total score, in relation to the M.I.N.I. diagnosis of GAD.
      Table 2Sensitivity, specificity and Youden's index for GAD-7 in relation to M.I.N.I. diagnosis of GAD.
      GAD-7 total scoresSpecificitySensitivity / True Positive RateFalse Positive Rate / (1-specificity)Youden's Index
      10.090.990.910.08
      20.130.990.870.12
      30.180.970.820.15
      40.280.940.720.22
      50.330.910.670.24
      60.380.890.620.27
      70.440.890.560.33
      80.50.830.500.34
      90.550.790.450.34
      100.580.760.420.34
      110.630.710.370.34
      120.680.680.320.35
      130.700.570.300.28
      140.750.510.250.26
      150.80.440.200.24
      160.860.370.140.23
      170.90.290.100.19
      180.930.230.070.17
      190.950.200.050.15
      200.960.190.040.14
      210.970.110.030.08
      Discriminant validity was evident in that those who were diagnosed with GAD on the M.I.N.I. scored significantly higher on the GAD-7 than those without this diagnosis (GAD present: mean = 13.3 (5.4) vs GAD absent mean = 8.4 (5.9), t = 7.599, df = 216.9, p < 0.0011, Cohen's d = 0.84 95% CI (0.61,1.1).

      3.4 Clinical correlates

      Higher GAD-7 total scores were associated with increased DS frequency, greater frequency of severe seizures, higher levels of behavioural avoidance (Avoidance of People Places and Situations – individual items and total score) and greater emotional avoidance (BES total score) (Table 3).
      Table 3Clinical correlates of the GAD-7: seizure frequency, seizure severity, behavioural avoidance and emotional avoidance.
      VariableGAD-7
      Pearson rp-valueN
      Baseline monthly seizure frequency0.213<0.001368
      Baseline monthly severe seizure frequency0.225<0.001368
      Avoidance of People Places and Situations_01 How much have you avoided specific situations for fear of having a seizure?0.291<0.001368
      Avoidance of People Places and Situations_02 How much have you avoided other people for fear of having a seizure?0.418<0.001368
      Avoidance of People Places and Situations_03 How much have you avoided specific activities for fear of having a seizure?0.250<0.001368
      Avoidance of People Places and Situations Overall total avoidance score0.383<0.001368
      Beliefs About Emotions Scale0.452<0.001367

      4. Discussion

      The psychometric evaluation of the GAD-7 shows that, in our sample, the GAD-7 has good structural validity. Analysis revealed a single factor solution, measurement invariance with respect to gender (adjusted for age), and only minimal effects of age (adjusted for gender). In addition, the GAD-7 shows good internal consistency (reliability) in the current sample. The findings of a single factor solution, general measurement invariance and good internal consistency are similar to results found in previous validation studies in clinical and general populations [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Lowe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ,
      • Löwe B.
      • Decker O.
      • Müller S.
      • Brähler E.
      • Schellberg D.
      • Herzog W.
      • et al.
      Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population.
      ].
      However, while there was evidence of discriminant validity when comparing GAD-7 scores for those with/without GAD diagnosis on the M.I.N.I., the ROC analysis showed an AUC < 0.8. Even allowing for the optimal value of Youden's Index, a cut-off score of 12 still yielded suboptimal sensitivity and specificity, since a score of 12 would be likely to correctly identify only 68% of people with likely GAD and 32% would be wrongly categorised as likely to have GAD. The standard cut-off score of 10 based on previous studies would yield better sensitivity but worse specificity, and the correct classification levels at all cut offs are considerably lower in our DS patient population than those obtained in the original validation of the GAD-7 [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Lowe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ]. While previous comparisons were made against different reference standards this indicates that, in DS populations, GAD-7 can be used to measure the extent of anxiety symptoms but that it is not an effective screening test for GAD, and the quoted standard cut-off score of ≥10 should not be used to identify patients with likely GAD.
      Evidence for convergent validity of GAD-7 in our population comes from the previous publication [
      • Baldellou Lopez M.
      • Goldstein L.H.
      • Robinson E.J.
      • Vitoratou S.
      • Chalder T.
      • Carson A.
      • et al.
      Validation of the PHQ-9 in adults with dissociative seizures.
      ] of correlations with scores on the CORE-10 [
      • Barkham M.
      • Bewick B.
      • Mullin T.
      • Gilbody S.
      • Connell J.C.
      • Cahill J.
      • et al.
      The CORE-10: a short measure of psychological distress for routine use in the psychological therapies.
      ] (0.791, p < 0.001), SF-12v2 Mental Component Summary [
      • Ware Jnr, J.E.
      • Kosinski M.
      • Turner-Bowker D.M.
      • Gandek B.
      How to score version 2 of the SF-12 health survey (With a supplement documenting version 1).
      ] (−0.712 p < 0.001), ED-5Q-5L [
      • Herdman M.
      • Gudex C.
      • Lloyd A.
      • Janssen M.
      • Kind P.
      • Parkin D.
      • et al.
      Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L).
      ] anxiety/depression subscale (0.675 p < 0.001) and the PHQ-9 depression scale [
      • Kroenke K.
      • Spitzer R.
      The PHQ-9: a new depression and diagnostic severity measure.
      ] (0.779, p < 0.001). Thus, across the present and previous studies, we have evidence for structural and convergent validity of the GAD-7 in the DS population, using the GAD-7 as a dimensional measure of symptom severity rather than diagnostic certainty.
      GAD-7 scores correlated with seizure frequency, frequency of severe seizures, and behavioural and experiential avoidance. However, we note that the correlations are mostly small in size, despite achieving statistical significance. The relationship between seizure frequency and GAD-7 supports findings from Dimaro et al. [
      • Dimaro L.V.
      • Dawson D.L.
      • Roberts N.A.
      • Brown I.
      • Moghaddam N.G.
      • Reuber M.
      Anxiety and avoidance in psychogenic nonepileptic seizures: the role of implicit and explicit anxiety.
      ] using different measures of anxiety symptoms and DS burden. Results also support a relationship between anxiety and behavioural and emotional avoidance in people with DS. We also, in a further analysis, found a correlation between behavioural and experiential avoidance in this study (r = 0.236, p < 0.001). The correlation reported here between the currently-used measure of anxiety and the BES supports the original validation of the BES. This previous study showed a correlation between anxiety and BES scores in people with Chronic Fatigue Syndrome but not in healthy controls [
      • Rimes K.A.
      • Chalder T.
      The beliefs about emotions scale: validity, reliability and sensitivity to change.
      ]. The finding of a correlation between the GAD-7 and BES in our sample of adults with DS supports Rimes and Chalder's [
      • Rimes K.A.
      • Chalder T.
      The beliefs about emotions scale: validity, reliability and sensitivity to change.
      ] suggestion that the beliefs measured by the BES may actually represent a transdiagnostic vulnerability factor that can contribute to a range of clinical presentations. They posit that people with the kind of emotional difficulties reflected by the BES may have developed these negative beliefs and emotional avoidance as a consequence of having experienced negative reactions or a lack of sympathy from others. In people with DS, frequent accounts of abuse history and family dysfunction, and also stigmatisation by health professionals, may be relevant here. Rimes & Chalder further suggest that these negative beliefs about emotion may then act to perpetuate the person's symptoms or restricted functioning even if they did not contribute to symptom development. Although the current results are correlational, so cannot imply direction of the relationship, these findings, along with the suggestions of Rimes and Chalder, add further support to use of fear-avoidance model in treatment. Furthermore, Whitfield et al. [
      • Whitfield A.
      • Walsh S.
      • Levita L.
      • Reuber M.
      Catastrophising and repetitive negative thinking tendencies in patients with psychogenic non-epileptic seizures or epilepsy.
      ] have identified a correlation between GAD-7 scores and measures of repetitive negative thinking (worry, rumination and perseverative thinking) and catastrophising in adults with DS; this interrelationship may further maintain fear (and other aspects of emotional avoidance) in people with DS. Addressing negative thinking is an aspect of the CBT intervention used in the CODES trial [
      • Goldstein L.H.
      • Robinson E.J.
      • Mellers J.D.C.
      • Stone J.
      • Carson A.
      • Reuber M.
      • et al.
      Cognitive behavioural therapy for adults with dissociative seizures (CODES): a pragmatic, multicentre, randomised controlled trial.
      ,
      • Goldstein L.H.
      • Robinson E.J.
      • Pilecka I.
      • Perdue I.
      • Mosweu I.
      • Read J.
      • et al.
      Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
      ].
      Our study has a number of limitations. The eligibility criteria for participants in CODES trial may limit generalisability of our findings as the study may have excluded people with more extreme psychopathology and also those people unwilling to engage in psychological treatment. The GAD-7 is a self-report measure (as are the measures of avoidance and seizure occurrence) which may be prone to response bias. However, the present analyses use patients’ baseline scores prior to randomisation so our findings were unaffected by treatment arm allocation.
      The M.I.N.I. was used for pragmatic reasons in this study to limit assessment burden for participants. A structured or semi-structured assessment by experienced psychiatrists might have led to different results and enabled us to calculate cut-off scores with better sensitivity and specificity. In addition, as noted with respect to our previous evaluation of the PHQ-9 [
      • Baldellou Lopez M.
      • Goldstein L.H.
      • Robinson E.J.
      • Vitoratou S.
      • Chalder T.
      • Carson A.
      • et al.
      Validation of the PHQ-9 in adults with dissociative seizures.
      ], the GAD-7 is a continuous measure whereas the M.I.N.I. provides a categorical definition of GAD. The original validation of the GAD-7 [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.
      • Lowe B.
      A brief measure for assessing generalized anxiety disorder: the GAD-7.
      ] that yielded high sensitivity and specificity scores reported the assessment of participants by a mental health professional (social worker or clinical psychologist) who undertook structured psychiatric interviews using the SCID, modified with additional questions to assess in greater detail some of the GAD diagnostic criteria. In the CODES trial it was not feasible to use the SCID or have more detailed evaluations of possible GAD undertaken by clinicians.
      Nonetheless, the study has a number of strengths. It benefitted from a large sample of closely studied adults with DS drawn from multiple centres in the UK thereby removing bias that might be associated with recruiting participants from a small number of centres. In addition to the rigorous psychometric approach adopted here, it also adds to the knowledge base through its use of additional measures of experiential avoidance and behavioural avoidance.

      5. Conclusions

      While we confirmed a single factor structure, general measurement invariance, good internal consistency and evidence of discriminant and convergent validity of the GAD-7 in our sample of adults with DS, the GAD-7 does not appear to offer a stringent means of diagnosing GAD in adults with DS even though it can be used to measure symptom severity. Correlations between the GAD-7 and measures of behavioural and emotional avoidance lend support to a fear-avoidance model that can be employed in the psychological treatment of people with DS. Its convergent validity and correlations with clinically relevant measures suggest that GAD-7 is a useful dimensional measure of anxiety, even if it is only provides a fair estimation of categorical anxiety disorder.

      Ethical publication statement

      We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this report is consistent with those guidelines. Ethical approval was granted by the London-Camberwell St Giles NRES Committee (reference number 13/LO/1595).

      Declaration of Competing Interest

      AC reports being a paid editor of the Journal of Neurology, Neurosurgery and Psychiatry, and is the director of a research programme on functional neurological disorders; he gives independent testimony in Court on a range of neuropsychiatric topics (50% pursuer, 50% defender). MR is the paid Editor-in-Chief of Seizure : European Journal of Epilepsy and receives authorship fees from Oxford University Press in relation to a number of books about dissociative seizures. JS reports independent expert testimony work for personal injury and medical negligence claims, royalties from UpToDate for articles on the functional neurological disorder and runs a free non-profit self-help website, www.neurosymptoms.org. The remaining authors have no conflicts of interest to declare.

      Acknowledgments

      This paper describes independent research funded by the National Institute for Health and Care Research (Health Technology Assessment programme, 12/26/01, COgnitive behavioural therapy v. standardised medical care for adults with Dissociative non-Epileptic Seizures: A multicentre randomised controlled trial (CODES)). This study also represents independent research part-funded (LHG, SV and TC) by the National Institute for Health and Care Research (NIHR) Maudsley Biomedical Research Centre at the South London and Maudsley NHS Foundation Trust and King's College London. JS is supported by an NHS Scotland NHS Research Scotland (NRS) Career Fellowship and JS and AC also acknowledge the financial support of NRS through the Edinburgh Clinical Research Facility. MR benefitted from the support of the NIHR Sheffield Biomedical Research Centre (Translational Neuroscience). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

      References

        • Lesser R.P.
        Psychogenic seizures.
        Neurology. 1996; 46: 1499-1507
        • Reuber M.
        Psychogenic nonepileptic seizures: answers and questions.
        Epilepsy Behav. 2008; 12: 622-635
        • Brown R.J.
        • Reuber M.
        Psychological and psychiatric aspects of psychogenic non-epileptic seizures (PNES): a systematic review.
        Clin Psychol Rev. 2016; 45: 157-182
        • World Health Organisation
        The ICD-10 classification of mental and behavioural disorders.
        Clinical description and diagnostic guidelines. World Health Organisation, Geneva1992
      1. World Health Organisation. International classification of diseases for mortality and morbidity statistics (11th Revision); 2018.

        • Goldstein L.H.
        • Mellers J.D.
        Ictal symptoms of anxiety, avoidance behaviour, and dissociation in patients with dissociative seizures.
        J Neurol Neurosurg Psychiatry. 2006; 77: 616-621
        • Reuber M.
        • Howlett S.
        • Khan A.
        • Grunewald R.A.
        Non-epileptic seizures and other functional neurological symptoms: predisposing, precipitating, and perpetuating factors.
        Psychosomatics. 2007; 48: 230-238
        • Moore P.M.
        • Baker G.A.
        Non-epileptic attack disorder: a psychological perspective.
        Seizure. 1997; 6: 429-434
        • Hixson J.D.
        • Balcer L.J.
        • Glosser G.
        • French J.A.
        Fear sensitivity and the psychological profile of patients with psychogenic nonepileptic seizures.
        Epilepsy Behav. 2006; 9: 587-592
        • Dimaro L.V.
        • Dawson D.L.
        • Roberts N.A.
        • Brown I.
        • Moghaddam N.G.
        • Reuber M.
        Anxiety and avoidance in psychogenic nonepileptic seizures: the role of implicit and explicit anxiety.
        Epilepsy Behav. 2014; 33: 77-86
        • Vein A.M.
        • Djukova G.M.
        • Vorobieva O.V.
        Is panic attack a mask of psychogenic seizures?–A comparative analysis of phenomenology of psychogenic seizures and panic attacks.
        Funct Neurol. 1994; 9: 153-159
        • Spielberger C.D.
        State-trait anxiety inventory.
        Corsini Encyclopedia of Psychology. 2022; : 1
        • Gámez W.
        • Chmielewski M.S.
        • Kotov R.
        • Ruggero C.J.
        • Watson D.
        Development of a measure of experiential avoidance: the multidimensional experiential avoidance questionnaire.
        Psychol Assess. 2011; 23: 692-713
        • Lang P.J.
        Shilen J.M. Research in Psychotherapy. Vol III. American Psychological Association, Washington DC1968
        • Mowrer O.H.
        Learning theory and behaviour.
        Wiley, New York1960
        • Chalder T.
        Non-epileptic attacks: a cognitive behavioural approach in a single case with a four-year follow-up.
        Clin Psychol Psychother. 1996; 3: 291-297
        • Goldstein L.H.
        • Deale A.C.
        • Mitchell-O'Malley S.J.
        • Toone B.K.
        • Mellers J.D
        An evaluation of cognitive behavioral therapy as a treatment for dissociative seizures: a pilot study.
        Cogn Behav Neurol. 2004; 17: 41-49
        • Goldstein L.H.
        • Chalder T.
        • Chigwedere C.
        • Khondoker M.R.
        • Moriarty J.
        • Toone B.K.
        • et al.
        Cognitive-behavioral therapy for psychogenic nonepileptic seizures: a pilot RCT.
        Neurology. 2010; 74: 1986-1994
        • Goldstein L.H.
        • Robinson E.J.
        • Mellers J.D.C.
        • Stone J.
        • Carson A.
        • Reuber M.
        • et al.
        Cognitive behavioural therapy for adults with dissociative seizures (CODES): a pragmatic, multicentre, randomised controlled trial.
        Lancet Psychiatry. 2020; 7: 491-505
        • Goldstein L.H.
        • Robinson E.J.
        • Pilecka I.
        • Perdue I.
        • Mosweu I.
        • Read J.
        • et al.
        Cognitive-behavioural therapy compared with standardised medical care for adults with dissociative non-epileptic seizures: the CODES RCT.
        Health Technol Assess. 2021; 25: 1-144
        • Spitzer R.L.
        • Kroenke K.
        • Williams J.B.
        • Lowe B.
        A brief measure for assessing generalized anxiety disorder: the GAD-7.
        Arch Intern Med. 2006; 166: 1092-1097
        • Löwe B.
        • Decker O.
        • Müller S.
        • Brähler E.
        • Schellberg D.
        • Herzog W.
        • et al.
        Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population.
        Med Care. 2008; 46: 266-274
        • Valente K.D.
        • Alessi R.
        • Baroni G.
        • Marin R.
        • Dos Santos B.
        • Palmini A
        The COVID-19 outbreak and PNES: the impact of a ubiquitously felt stressor.
        Epilepsy Behav. 2021; 117107852
        • Abe C.
        • Denney D.
        • Doyle A.
        • Cullum M.
        • Adams J.
        • Perven G.
        • et al.
        Comparison of psychiatric comorbidities and impact on quality of life in patients with epilepsy or psychogenic nonepileptic spells.
        Epilepsy Behav. 2020; 102106649
        • Novakova B.
        • Harris P.R.
        • Rawlings G.H.
        • Reuber M.
        Coping with stress: a pilot study of a self-help stress management intervention for patients with epileptic or psychogenic nonepileptic seizures.
        Epilepsy Behav. 2019; 94: 169-177
        • Rawlings G.H.
        • Brown I.
        • Reuber M.
        Predictors of health-related quality of life in patients with epilepsy and psychogenic nonepileptic seizures.
        Epilepsy Behav. 2017; 68: 153-158
        • Baldellou Lopez M.
        • Goldstein L.H.
        • Robinson E.J.
        • Vitoratou S.
        • Chalder T.
        • Carson A.
        • et al.
        Validation of the PHQ-9 in adults with dissociative seizures.
        J Psychosom Res. 2021; 146110487
        • Sheehan D.V.
        • Lecrubier Y.
        • Sheehan K.H.
        • Amorim P.
        • Janavs J.
        • Weiller E.
        The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.
        J Clin Psychiatry. 1998; 59: 22-33
        • Goldstein L.H.
        • Robinson E.J.
        • Mellers J.D.C.
        • Stone J.
        • Carson A.
        • Chalder T.
        • et al.
        Psychological and demographic characteristics of 368 patients with dissociative seizures: data from the CODES cohort.
        Psychol Med. 2021; 51: 2433-2445
        • Rimes K.A.
        • Chalder T.
        The beliefs about emotions scale: validity, reliability and sensitivity to change.
        J Psychosom Res. 2010; 68: 285-292
        • Goldstein L.H.
        • Mellers J.D.
        • Landau S.
        • Stone J.
        • Carson A.
        • Medford N.
        • et al.
        Cognitive behavioural therapy vs standardised medical care for adults with dissociative non-epileptic seizures (CODES): a multicentre randomised controlled trial protocol.
        BMC Neurol. 2015; 15: 98
        • Lecrubier Y.
        • Sheehan D.V.
        • Weiller E.
        • Amorim P.
        • Bonora I.
        • Sheehan K.H.
        The mini international neuropsychiatric interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI.
        Eur Psychiatry. 1997; 12: 224-231
        • Sheehan D.V.
        • Lecrubier Y.
        • Harnett Sheehan K.
        • Janavs J.
        • Weiller E.
        • et al.
        The validity of the mini international neuropsychiatric interview (MINI) according to the SCID-P and its reliability.
        Eur Psychiatry. 1997; 12: 232-241
        • American Psychiatric Association
        Diagnostic and statistical manual of mental disorders (4th Edition- DSM-IV).
        American Psychiatric Association, Washington DC1994
      2. Muthén B., du Toit S.H.C., Spisic D. Robust inference using weighted least squares and quadratic estimating equations in latent variable modelling with categorical and continuous outcomes. (Technical report). Retrieved from https://www.statmodel.com/download/Article_075.pdf; 1997.

        • Guttman L.
        Some necessary conditions for common-factor analysis.
        Psychometrika. 1954; 19: 149-161
        • Kaiser H.F.
        The application of electronic computers to factor analysis.
        Educ Psychol Meas. 1960; 20: 141-151
        • Horn J.L.
        A rationale and test for the number of factors in factor analysis.
        Psychometrika. 1965; 30: 179-185
      3. Presaghi F., Desimoni M. Random polychor PA: a parallel analysis with polychoric correlation matrices. R package version 1.1.4-3. 2019.

        • Cattell R.B.
        The scree test for the number of factors.
        Multivariate Behav Res. 1966; 1: 245-276
        • Joreskog K.G.
        • Goldberger A.S.
        Estimation of a model with multiple indicators and multiple causes of a single latent variable.
        J Am Stat Assoc. 1975; 70: 631-639
        • West S.G.
        • Taylor A.B.
        • Wu W.
        Model fit and model selection in structural equation modeling.
        in: Handbook of Structural Equation Modeling. The Guilford Press, New York, US2012: 209-231
        • Hu L.T.
        • Bentler P.M.
        Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural equation modeling: a.
        Multidiscip J. 1999; 6: 1-55
        • Bentler P.M.
        • Bonett D.G.
        Significance tests and goodness of fit in the analysis of covariance structures.
        Psychol Bull. 1980; 88: 588-606
        • Cronbach L.J.
        Coefficient alpha and the internal structure of tests.
        Psychometrika. 1951; 16: 297-334
        • McDonald R.P.
        Test theory: a unified treatment.
        Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US1999
        • Nunnally J.C.
        • Bernstein I.H.
        Psychometric theory.
        3rd ed. McGraw-Hill, New York1994
        • Hanley J.A.
        • McNeil B.J.
        The meaning and use of the area under a receiver operating characteristic (ROC) curve.
        Radiology. 1982; 143: 29-36
        • Janes H.
        • Longton G.
        • Pepe M.
        Accommodating covariates in ROC analysis.
        Stata J. 2009; 9: 17-39
        • Youden W.J.
        Index for rating diagnostic tests.
        Cancer. 1950; 3: 32-35
        • Muthén L.K.
        • Muthén B.O.
        Mplus user’s guide.
        8th ed. Muthén & Muthén, Los Angeles, CA1998–2022
        • Kyriazos T.A.
        Applied psychometrics: sample size and ssmple power considerations in factor analysis (EFA, CFA) and SEM in general.
        Psychology. 2018; 09: 25
        • Barkham M.
        • Bewick B.
        • Mullin T.
        • Gilbody S.
        • Connell J.C.
        • Cahill J.
        • et al.
        The CORE-10: a short measure of psychological distress for routine use in the psychological therapies.
        Couns Psychother Res. 2013; 13: 3-13
        • Ware Jnr, J.E.
        • Kosinski M.
        • Turner-Bowker D.M.
        • Gandek B.
        How to score version 2 of the SF-12 health survey (With a supplement documenting version 1).
        QualityMetric Incorporated., Lincoln, RI2002
        • Herdman M.
        • Gudex C.
        • Lloyd A.
        • Janssen M.
        • Kind P.
        • Parkin D.
        • et al.
        Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L).
        Qual Life Res. 2011; 20: 1727-1736
        • Kroenke K.
        • Spitzer R.
        The PHQ-9: a new depression and diagnostic severity measure.
        Psychiatr Ann. 2002; 32: 509-521
        • Whitfield A.
        • Walsh S.
        • Levita L.
        • Reuber M.
        Catastrophising and repetitive negative thinking tendencies in patients with psychogenic non-epileptic seizures or epilepsy.
        Seizure. 2020; 83: 57-62