Abstract
Purpose
Neuropsychology has become an essential diagnostic tool for epilepsy-related cognitive comorbidities and treatment evaluation. However, a lack of resources may prevent routine neuropsychological assessments outside specialized epilepsy centers. Computerized testing appears to offer a time- and cost-effective approach to assess cognitive functions in patients with epilepsy. Moreover, the technical advances of computerized tests provide interesting tools to address specific diagnostic questions around epilepsy. This review is intended: (1) to outline the advantages and disadvantages of computerized testing, (2) to delineate its indications and fields of application, and (3) to give an overview of available tools that have been applied in epilepsy or antiepileptic drug research.
Method
Iterative review of computer-based neuropsychological assessment batteries previously applied in clinical epileptological settings or antiepileptic drug trials.
Results
Among nine reviewed computer tests merely three were explicitly devised for epilepsy and showed sensitivity to clinical parameters like focus lateralization or localization or the presence/absence of epileptiform activity. Concurrent validity with established measures was demonstrated for two of these three tests. Some sensitivity to antiepileptic pharmacotherapy was reported for seven of all nine reviewed test batteries.
Conclusion
Additional studies are needed to demonstrate the sensitivity and specificity of computerized neuropsychological tests to epilepsy and treatment related variables. In most clinical scenarios exclusive computerized testing cannot substitute a thorough neuropsychological examination in patients with epilepsy at present.
Keywords
1. Introduction
Neuropsychology provides evidence-based methods to answer clinical questions regarding epilepsy and its treatment.
1
Apart from its role in the assessment of cognitive dysfunction in relation to the type of epilepsy and the underlying pathology, neuropsychology plays an increasing role in the individual quality and outcome control of all kinds of medical interventions, be they invasive (e.g. epilepsy surgery, radio- or stereotactic surgery, deep brain stimulation), semi-invasive (e.g. vagal nerve stimulation) or non-invasive (pharmacological treatment).2
, 3
As a consequence of the increasing dissemination and acceptance of neuropsychological diagnostics for monitoring CNS (central nervous system) diseases and treatment outcomes, there is the understandable interest in brief, cost effective, and easy-to-apply assessment and screening tools. Technical advances during the last decade now offer a unique possibility to transfer neuropsychological assessment into the world of computers, laptops, tablets, handheld devices and mobile phones. The transfer to and the utilization of such novel testing platforms is only just beginning. Nevertheless there has been an increasing interest in computerized diagnostics in patients with epilepsy.
The quality of neuropsychological results – and this holds true in equal measure for computerized as well as classical paper-pencil tests – depends on the validity of the neuropsychological instruments applied. The chosen measures and testing methods need to be sensitive and specific in regard to the intended field of application and the underlying clinical question. Beyond the three classical test criteria of “objectivity”, “reliability” and “validity”, the following additional criteria have to be considered: the quality and range of the normative data provided, the availability of parallel test versions and information about practice effects due to repeated test administration, i.e. test-retest norms. Furthermore, the effort in terms of time (duration of administration and scoring) and costs (test materials and staff) are relevant. Finally, the best test is useless if it is not available in the patient's language or if it is not valid in a particular patient group because of cultural variations.
4
2. The pros and cons of computerized assessments
Computerized neuropsychological tests usually have a high objectivity in terms of administration and scoring. A high level of test objectivity is a required but not sufficient precondition for high test-reliability. The same is true for the relation between test reliability and test validity. In contrast to paper-pencil testing, computer tests allow for the exact assessment of reaction times in milliseconds. This is particularly advantageous when evaluating processes of attention (e.g. alertness/speed of information processing). Furthermore, computerized tests can make use of enhanced interactive and multimedia-based question styles or virtual realities. They do not require stationary hardware since they can potentially be used on mobile phones or handheld computers. The possibility of a continuous and instant registration of cognitive performance can be utilized for adaptive testing, i.e. the tailoring of the task difficulty/demands to the individual performance level. The possibility of randomly selecting items from a pool of stimuli instead of using fixed parallel versions is an advantage when multiple reassessments are required. Moreover, computer tests are usually easy to administer or even self-administrable. Although the application thus does not necessarily require neuropsychological expertise, computerized testing without neuropsychological supervision bears the risk of false interpretations. Fully computerized tests must be differentiated from computer-guided or -aided tests. Whereas “fully computerized” implies computerized task presentation and automated response registration, computer-guided or -aided testing uses the computer either for stimuli presentation or as an electronic log sheet where the examiner records the patient's reactions/answers. Therefore computer-guided or -aided testing is by definition not self-administrable and always requires an examiner.
If the registration of results is computerized, computer tests can provide a fully automated calculation and presentation of the findings (in terms of raw data as well as standardized values related to normative data), which can be included in printed reports or electronic patient files automatically. This is time-efficient and also allows clinicians to provide instantaneous feedback to the patient investigated. In the context of scientific studies, the possibility of an electronic data export facilitates the import into statistical software packages. A major advantage of computerized testing, especially for the field of epilepsy, is the possibility of time-locked coregistration of cognitive processes and physiological measures. By this means the impact of interictal epileptiform discharges on cognitive performance can, for example, be analyzed. Apart from concomitant EEG-analyses, computerized tests can also be applied during functional brain imaging studies (e.g. fMRI). Finally, although the up-front equipment cost of computerized testing procedures may be higher, there may be savings in the long run since paper-pencil tests require constant restocking of log sheets.
Computerized neuropsychological tests may also have some disadvantages. Self-administrable computerized tests dramatically reduce the interaction between examiner and patient. Therefore a potentially important source of information is lost. Another disadvantage can be seen in interface problems, e.g. behavioral responses are reduced to reactions via mouse, keyboard or joystick. For example, the exact position of the index finger (e.g. the distance between finger and key) may be important for the measurements of reaction times in milliseconds. The increasing availability of more intuitive touchscreens may attenuate such reservations. Future and advanced techniques of video-monitoring, language recognition or movement registration may expand the variety of behavioral expressions. Principally, individual differences in regard to the familiarity with computer devices may affect the affinity to and the performance in computerized tests.
Another issue is that some well-validated traditional paper-pencil tests require written or spoken responses (e.g. free verbal memory recall), patient drawings or manual constructions (e.g. using blocks). This cannot be easily converted into a computerized test – at least not without extensive validation of new patient-test interaction methods. Finally, some computer test batteries require extra hardware (e.g. push-button, touchscreen) or have special software requirements (e.g. in terms of the operating system used). Differences in screen sizes and contrast/clarity may affect results and it can be difficult to adjust the audio volume when presenting sounds or speech.
3. Indications for computerized tests
For which diagnostic questions in the field of epilepsy are computer tests suitable? The use of a computerized test battery appears indicated in the following situations:
- (1)Computerized testing may provide an objective screening for cognitive problems in case of self-reported cognitive deficits. However, subjective complaints always call for a concomitant screening for depressed mood, because of the robust finding that subjective cognitive complaints often reflect mood rather than objective performance.5,6When objective cognitive problems become evident and more detailed information is required a subsequent more comprehensive standard neuropsychological assessment may be indicated.
- (2)Because of the high objectivity and time-efficient method of administration, computerized tests seem to be an ideal approach if frequent repeated testing is required, for instance to monitor the effect of medical interventions or of acute changes in the course of the disease.
- (3)In conjunction with co-registered EEG-data, computer tests can be used to assess the impact of interictal discharges on cognition and vice versa.
- (4)Postictal computerized testing may provide information on negative symptoms and the dynamics of seizure-related cognitive dysfunctions. The differential postictal recovery of material-specific memory functions in temporal lobe epilepsies has been shown to be related to the lateralization of the seizure onset.7
- (5)The use of computerized naming (frontal and temporal functions) and line-bisection tests (parietal functions) during invasive EEG-monitoring or awake surgery can facilitate the surgeon's decision to excise or spare particular areas of the brain.
- (6)Computerized tests are ideal for experimental assessments in cognitive neuroscience. Experiments performed on a group level under controlled conditions, however, may not be suitable for routine diagnostics and for individual level analysis since they lack standardization with normative data.
- (7)To date there are no standardized computer tests which have been used in patients with epilepsy outside the clinic or outpatient setting. However, a first attempt of assessing cognitive and behavioral effects of antiepileptic pharmacotherapy8demonstrated the potential of computerized tests to assess patients outside institutional settings in specific situations, at specific locations or times.
For all suggested indications it is essential that the chosen measures are valid in regard to the diagnostic target.
9
4. Which cognitive domains are best captured by computerized testing
Almost all of the cognitive domains examined by traditional neuropsychological tests can also be assessed by computerized testing procedures. The cognitive domains which are best examined using computer-based methods are those where computers have advantages over routine paper-pencil or face-to-face examinations. In the first line these are all functions for which time or precision are critical, both in terms of stimulus presentation and the required reaction/behavior. This concerns the domains of attention and decision making (reaction or decision times), perception (signal detection, threshold measurements, visual field assessment), hemispheric language or motor dominance (dichotic listening, tachistoscopy, finger tapping), visuo-spatial orientation and navigation (visual scanning, neglect tests, mazes, virtual reality), working memory as well as span and supraspan tasks (n-back tasks, Corsi block tapping), learning and memory (pools of stimuli for retrieval of items following random selection or a given algorithm; object location tasks), language functions (response times along with word finding difficulties and/or naming problems), and social cognition (Theory of Mind).
However, while this list gives an idea about possible applications of computer tests, it is not meant to imply that there is sufficient published evidence about tasks which can be selected for valid routine computerized diagnostics at present. Often computerized tasks are used to address only a single scientific question (see surface or intracranial cognitive event-related potentials in epilepsy), in some cases the validity and reliability of the tasks per se are in question (see value of dichotic listening or tachistoscopy for functional brain organization), or the tasks are in an experimental stage of development or have not yet been evaluated in epilepsy (see psycho-physics).
5. Overview of available computer tests
Given the potential advantages of computerized testing it is surprising how little use of such tests has, so far, been made in the field of epilepsy, and that it has not become an integral part of standard or routine diagnostics.
10
, 11
The following section comprises brief descriptions of computer tests available which have been applied as main cognitive measure in published studies in the fields of epilepsy or antiepileptic drug research. The computerized batteries have primarily been identified via literature search in PubMed (http://www.ncbi.nlm.nih.gov/pubmed/?term=(epilep*[title]+cognit*+computer*)+OR+(antiepileptic+drug*+cognit*+computer*)). In addition, surveys of neuropsychological praxis in epilepsy were screened for citations of computerized tests.
10
, 11
, 12
Experimental cognitive paradigms (e.g. those used exclusively in functional imaging experiments) were not considered, because of the lack of clinical utility due to missing or insufficient normative data. A related and rather complementary review by Wild et al.13
provides an overview of 18 computerized assessment tools in the field of dementia. With just two tests (CANTAB, CNTB) the overlap of this former and our present review is minimal.The questions which have been addressed by computerized neuropsychological tests in the field of epilepsy-related to screening for deficits (i.e. whether cognitive impairments are evident or not, or to objectify subjective complaints),
14
to assess the cognitive effects of antiepileptic drugs in patients or healthy subjects,15
to evaluate interictal or postictal cognitive change,- Hessen E.
- Lossius M.I.
- Reinvang I.
- Gjerstad L.
Influence of major antiepileptic drugs on attention, reaction time, and speed of information processing: results from a randomized, double-blind, placebo-controlled withdrawal study of seizure-free epilepsy patients receiving monotherapy.
Epilepsia. 2006; 47: 2038-2045
7
, 16
, - Ebus S.
- Arends J.
- Hendriksen J.
- van der Horst E.
- de la Parra N.
- Hendriksen R.
- Santegoeds E.
- Boon P.
- Aldenkamp B.
Cognitive effects of interictal epileptiform discharges in children.
European Journal of Paediatric Neurology: EJPN: Official Journal of the European Paediatric Neurology Society. 2012; 16: 697-706
17
or the effects of acute interventions (e.g. vagal nerve stimulation,18
interruption of nonconvulsive status epilepticus 19
, 20
).Table 1 provides a comparative overview of cognitive functions which authors have claimed to assess by use of the respective computer test. Table 2 summarizes and compares the requirements, features and test criteria of these tests.
Table 1Overview of the cognitive domains which are assessed by the different computerized batteries.
| CALCAP | CANTAB | CCTE | CDR | CNT | CNTB | FePsy | NeuroCog FX | TAP | |
|---|---|---|---|---|---|---|---|---|---|
| Attention | |||||||||
| Alertness | ✓ | ✓ | – | ✓ | – | ✓ | ✓ | ✓ | ✓ |
| Selective attention | ✓ | ✓ | ✓ | ✓ | – | ✓ | ✓ | ✓ | ✓ |
| Sustained attention | ✓ | ✓ | – | ✓ | ✓ | – | ✓ | – | ✓ |
| Divided attention | – | – | – | – | ✓ | – | – | – | ✓ |
| Interference/Flexibility | ✓ | ✓ | – | – | – | – | - | ✓ | ✓ |
| Vigilance | – | ✓– | – | ✓ | – | – | ✓ | – | ✓ |
| Working memory | |||||||||
| Verbal | ✓ | – | ✓ | ✓ | – | – | ✓ | ✓ | ✓ |
| Nonverbal | – | ✓ | – | ✓ | ✓ | – | ✓ | – | – |
| Verbal memory | |||||||||
| Immediate | – | ✓ | ✓ | ✓ | – | ✓ | ✓ | ✓ | – |
| Delayed | – | ✓ | – | ✓ | ✓ | ✓ | – | ✓ | – |
| Nonverbal memory | |||||||||
| Immediate | – | ✓ | ✓ | ✓ | – | ✓ | ✓ | ✓ | – |
| Delayed | – | ✓ | ✓ | ✓ | – | ✓ | – | ✓ | – |
| Language | |||||||||
| Fluency | – | – | – | – | – | – | - | ✓ | – |
| Naming | – | ✓ | – | – | – | ✓ | ✓ | – | – |
| Lexical discrimination | ✓ | – | – | – | – | – | – | – | – |
| Other functions | |||||||||
| Finger tapping | – | – | – | ✓ | – | ✓ | ✓ | – | – |
| Visual/spatial functions | ✓ | ✓ | ✓ | ✓ | – | ✓ | ✓ | – | ✓ |
| Incidental memory | – | – | ✓ | – | – | – | ✓ | – | – |
Table 2Requirements, features and test criteria of the different computerized batteries.
| CALCAP | CANTAB | CCTE | CDR | CNT | CNTB | FePsy | NeuroCog FX | TAP | |
|---|---|---|---|---|---|---|---|---|---|
| Price | 385 € | Variable ≥1250 € | n.a. | Supplied as service | n.a. | ? | 1980 € | Licensed to Eisai | ~1190 € (once, no usage limitation) |
| System requirements | Windows, DOS 80286 + CPU 512K RAM 2MB HD | Windows, touchscreen | Windows, soundcard, touchscreen | Most Windows PCs & available over internet | PC | ? | PC | PC/Laptop, Windows, soundcard | Windows, soundcard, 2 USB-Ports |
| Language support | EN, ESP, FL FRA, NOR, DEN | Interface: EN; feedback: several languages | GER | >65 Languages | EN | ? | 21, incl. EN, FRA, GER, ESP, ITA | GER | AR*, CHIN, DEN, EN, ESP, FIN, FRA, GER, ITA, LIT, NED, POR |
| Number of subtests (and variations) | 7 | Up to 25 | 8 | 12 | 2 | 11 | 11 (24) | 8 | 13 (31) |
| Administration time | <25 min | ~30 min | 18 min | 30–45 min | ~50 min | 120 min | <30 min | Subtest durations from 2 to 30 min | |
| Normative data (N) | >1000 | 250–1450 | 83 | >8000 | ~500 | ? | 250 | 244 | 94–811 |
| Age range (years) | 17–90 | 4–80+ | 20–60 | 5–87 | 6–90 | ? | 7–80 | 17–80 | 6–90, variable |
| Control for education | ✓ | ✓ | – | Not for clinical trial applications | ✓ | ? | – | – | ✓ |
| Test–retest norms | ✓ (N = 175) | – | ✓ (N = 46) | ✓ | ✓ | ✓ | – | ✓ (N = 44) | – |
| RCIs | – | – | ✓ | – | – | ✓ | – | ✓ | – |
| Retest-reliablities (r) | 0.20–0.68 | 0.09–0.86 | 0.40–0.79 | 0.40–0.90 | ~0.80 | ? | 0.70–0.90 (patients) | 0.21–0.69 (healthy controls) 0.70 –0.85 (patients) | −0.34 –0.96 |
| Parallel versions | – | 0–15 | 1 | >50 | ✓ | ? | Some tests | 6 predefined or random material | Some subtests |
| Devised for epilepsy | – | – | ✓ | – | – | – | ✓ | ✓ | – |
| Validated in epilepsy | – | ✓ | ✓ | – | – | – | ✓ | ✓ | – |
| Epilepsy-specific sensitivity | |||||||||
| Localization/lateralization | – | – | ✓ | – | – | – | – | ✓ | – |
| IEDs and/or seizures | – | – | – | – | – | – | ✓ | – | - |
| AED treatment | ✓ | – | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| Special features | Auditory instructions | ✓ | |||||||
| EEG coregistration | – | – | – | ✓ | ✓ | – | ✓ | – | – |
| Summary score | – | – | – | – | ✓ | ✓ | ✓ | – | |
| Graphical cognitive profile | ✓ | – | ✓ | – | ✓ | ? | ✓ | ✓ | |
n.a., Not available; ?, no information available; HD, hard disk; HC, healthy controls; pat., patients; AED, antiepileptic drug.
5.1 California Computerized Assessment Package (CALCAP®)
The 7 subtests of the CALCAP (http://www.calcaprt.com) assess simple and choice reaction times, visual selective attention, response reversal and rapid visual scanning, form discrimination and working memory (1-back and 2-back tasks). Thus the battery primarily focuses on attention and executive functions. A factor analysis in healthy volunteers revealed two primary factors reflecting 1. “Simple reaction time” and 2. “Decision speed”.
21
These two factors were distinct from those derived from traditional paper-pencil-tests. The first comprised the simple reaction time test and the second all other subtests. The battery is designed for longitudinal studies providing normative data for up to six repeated examinations. Practice sessions before each subtest ensure an understanding of the task instructions and minimize practice effects in repeated assessments. An abbreviated version based on four subtests assessing speed of processing and working memory is available and takes 8–10 min to complete.Application in epilepsy: The CALCAP has not been validated in patients with epilepsy, but was applied in a prospective antiepileptic drug withdrawal study in patients with epilepsy demonstrating an increase of completely unimpaired patients from 11% at baseline to 28% at follow-up.
15
- Hessen E.
- Lossius M.I.
- Reinvang I.
- Gjerstad L.
Influence of major antiepileptic drugs on attention, reaction time, and speed of information processing: results from a randomized, double-blind, placebo-controlled withdrawal study of seizure-free epilepsy patients receiving monotherapy.
Epilepsia. 2006; 47: 2038-2045
5.2 Cambridge Neuropsychological Test Automated Battery (CANTAB)
The CANTAB (http://www.cantab.com) consists of a selection of 25 tests that assess a wide range of cognitive domains, including executive function, attention and memory. All tests are supposed to be culturally independent and most are non-linguistic. Factor analysis revealed four factors reflecting (1) general learning and memory ability, (2.)speed of responding, (3) executive processes, and (4) visual perceptual ability. CANTAB tests have been validated
22
, 23
, 24
in over one thousand peer reviewed articles attesting to the sensitivity of these tests in a number of disease areas. The validation study in epilepsy evaluated a heterogeneous group of 15 patients with epilepsy- Sahakian B.J.
- Owen A.M.
- Morant N.J.
- Eagger S.A.
- Boddington S.
- Crayton L.
- Crockford H.A.
- Crooks M.
- Hill K.
- Levy R.
Further analysis of the cognitive effects of tetrahydroaminoacridine (THA) in Alzheimer's disease: assessment of attentional and mnemonic function using CANTAB.
Psychopharmacology. 1993; 110: 395-401
25
and compared three subtests of the CANTAB [memory (Delayed matching to sample (DMS), Paired Associate Learning (PL)) and executive functions (Stockings of Cambridge (SOC)] with traditional paper-pencil tests [including Wechsler Memory Scale revised (WMS-R), the Category Test, the Trail Making Test (TMT) and the Wechsler Adult Intelligence Scale (WAIS-III)]. When classifying the patients as impaired (z-score ≤−2 on two or ≤−1.5 on three of ten tests) vs. unimpaired, the concordance between these two different approaches was 80%. Performance in DMS was correlated with the results of the Visual Paired Associates subtest of the WMS-R. Scores obtained in the PL were related to several test parameters including verbal IQ, TMT-B as well as verbal and visual memory indices. Executive performance assessed via SOC also correlated with several measures including memory indices and intelligence, but not with the TMT-B – a test of executive function.Application in epilepsy: Apart from the above-mentioned small validation study, the sensitivity of the CANTAB to different types or etiologies of epilepsy, antiepileptic drugs, seizures or interictal epileptiform activity has not yet been demonstrated.
5.3 Computerized Cognitive Testing in Epilepsy (CCTE)
The CCTE is a reliable neuropsychological screening instrument that comprises the following eight tasks: digit span forward and backward, focused attention and visual scanning, visuospatial memory, incidental memory, verbal learning and figural memory.
26
, 27
The self-administrable test battery utilizes a touchscreen and it takes about 30 min to complete; the results can be displayed graphically immediately after examination in comparison to age-related normative data. Two parallel versions and RCIs allow for repetitive assessment of a patient's cognitive profile. A factor analysis in a mixed sample of 240 patients with epilepsy and 83 healthy volunteers revealed two factors reflecting (1) attention and (2) memory.- Kurzbuch K.
- Pauli E.
- Gaal L.
- Kerling F.
- Kasper B.S.
- Stefan H.
- Hamer H.
- Graf W.
Computerized cognitive testing in epilepsy (CCTE): a new method for cognitive screening.
Seizure. 2012; https://doi.org/10.1016/j.seizure.2012.08.011
27
Up to now the CCTE is only available in German language, but the development of multilingual versions is in progress.- Kurzbuch K.
- Pauli E.
- Gaal L.
- Kerling F.
- Kasper B.S.
- Stefan H.
- Hamer H.
- Graf W.
Computerized cognitive testing in epilepsy (CCTE): a new method for cognitive screening.
Seizure. 2012; https://doi.org/10.1016/j.seizure.2012.08.011
Application in epilepsy: The authors have demonstrated sensitivity to medical treatment (untreated < monotherapy < polytherapy) and worse verbal memory performance in patients with left versus right mesial temporal lobe epilepsy.
27
Moreover, performance in subtests of the CCTE correlates with results in corresponding paper-pencil tests which have proven validity in the field of epilepsy.- Kurzbuch K.
- Pauli E.
- Gaal L.
- Kerling F.
- Kasper B.S.
- Stefan H.
- Hamer H.
- Graf W.
Computerized cognitive testing in epilepsy (CCTE): a new method for cognitive screening.
Seizure. 2012; https://doi.org/10.1016/j.seizure.2012.08.011
27
- Kurzbuch K.
- Pauli E.
- Gaal L.
- Kerling F.
- Kasper B.S.
- Stefan H.
- Hamer H.
- Graf W.
Computerized cognitive testing in epilepsy (CCTE): a new method for cognitive screening.
Seizure. 2012; https://doi.org/10.1016/j.seizure.2012.08.011
5.4 Cognitive Drug Research (CDR) computerized assessment system
The CDR Computerized Assessment System (http://www.bracketglobal.com/services/cognition) is a sensitive and validated computerized cognitive assessment system specifically designed for clinical trials.
28
The CDR System assesses the core aspects of cognitive function (psychomotor speed, attention, working memory, episodic verbal and nonverbal memory), which underpin the ability to conduct the activities of daily living. Factor analyses in healthy volunteers and in a population with dementia revealed four to five underlying factors reflecting speed and quality of attention and memory processes.29
, 30
, - Wesnes K.A.
- McKeith I.G.
- Ferrara R.
- Emre M.
- Del Ser T.
- Spano P.F.
- Cicin-Sain A.
- Anand R.
- Spiegel R.
Effects of rivastigmine on cognitive function in dementia with lewy bodies: a randomised placebo-controlled international study using the cognitive drug research computerised assessment system.
Dementia and Geriatric Cognitive Disorders. 2002; 13: 183-192
31
Besides the core battery additional tests (e.g. motor control, executive functions, psychophysical flicker fusion thresholds) and automated questionnaires are available. Furthermore it allows for EEG coregistration to obtain evoked potentials for detected and non-detected targets as well as correct and incorrect decisions or recognitions. The CDR System was developed during the late 1970s and early 1980s to provide clinical trials with a tool which could reliably measure change over time in cognitive function associated with the use of novel treatments and/or disease progression. Since 1984 it has been employed in almost 1400 clinical trials worldwide. It has been used extensively in all phases of drug development in over 60 clinical conditions. The core-assessments are brief to complete (18 min) and have numerous parallel forms (>50) and language versions (>65). The CDR System is simple to administer, allowing non-specialists to be rapidly trained in its use. The CDR System has the largest database ever assembled of cognitive data from clinical trials.Application in epilepsy: The CDR System was applied in a single study on the cognitive and psychomotor effects of remacemide and carbamazepine in newly diagnosed epilepsy,
32
but has not yet been explicitly validated in patients with epilepsy.5.5 Cognitive Neurophysiological Test (CNT)
The CNT uniquely combines cognitive task performance measures with simultaneous recording of EEG and evoked potential measures to form a single index of how a drug or illness affects overall neurocognitive function. The advantage of the CNT is that it takes into account not only the speed and accuracy of cognitive task performance but also the amount and type of neural resources and effort required to produce that level of performance, as well as the level of alertness. The practical consequence of this additional knowledge is increased sensitivity and specificity to the effects of a drug.
33
, 34
The CNT is an automated hardware and software system with a fast application EEG hat that allows test administration by personnel with minimal training.35
, 36
The primary cognitive functions measured are working memory (n-back task) and sustained attention,37
with options for verbal episodic memory (word recognition task) and divided attention. In addition to an overall index, sub-indices are provided for each cognitive task, EEG and alertness. The CNT is not commercially available.Application in epilepsy: Apart from its use in studies on the cognitive effects of antiepileptic drugs in healthy subjects,
38
, 39
, 40
, 41
the test has not yet been validated in patients with epilepsy.5.6 Computerized Neuropsychological Test Battery (CNTB)
The CNTB is primarily based on traditional neuropsychological tests that were adapted for computer-aided presentation and registration.
42
, 43
Thus the CNTB is not self-administrable and oral stimuli (verbal memory test) still have to be read aloud by the investigator. In approximately 50 min the 11 subtests of the CNTB assess motor speed (finger tapping), information processing (simple and choice reaction times), attention, verbal and spatial memory (word list learning and recall, paired associate learning, visual memory, visual matching delayed recall), language (confrontation naming), as well as spatial abilities (visual matching).13
In addition to subtest results a summary score reflecting the global performance across all subtests is provided. The CNTB has two alternative forms. However, the battery was originally devised for dementia and used as outcome measure in clinical trials in this domain.Application in epilepsy: To our knowledge, the battery has not yet been applied or validated in patients with epilepsy. The test has been used, however, in a study on the cognitive effects of the antiepileptic drug topiramate in healthy subjects.
44
For their purpose the authors of that study calculated RCIs for the summary score based on the performance of 38 healthy subjects before and 6, 12 and 24 weeks after receiving placebo.44
5.7 FePsy ‘The Iron Psyche’
The FePsy (http://www.fepsy.com) comprises a battery of 11 computerized tests (24 when considering all subtest variations; auditory reaction time, visual reaction time, binary choice reaction time, tapping task, computerized visual searching task, verbal and visual recognition tasks, vigilance task, seashore rhythm task, card sorting task, visual half field tasks, Corsi block tapping, naming test, as well as a memory and a cognitive complaints questionnaire) for cognitive neuropsychological functions and is built around a powerful relational database system which stores all results. These results can easily be converted into statistical or other database programs. FePsy includes a subset of tests for the measurement of side effects of drugs
45
(Epilepsy, Cancer, HIV) and is also used in other fields (Parkinson, Schizophrenia, Diabetes), covering arousal, short-term memory, mental speed, vigilance, naming as well as some questionnaires. A factor analysis in 747 patients with epilepsy revealed six factors reflecting (1) working memory/memory/attention, (2) simple reaction time, (3) motor performance, (4) problem solving, (5) choice reaction time, (6) impulsivity. It runs on every modern PC. Use of a touchscreen is an option. FePsy is available in almost all European languages, 21 in total. FePsy easily connects to the EEG to analyze simultaneous EEG signals and psychological performance. Test-retest norms (e.g. RCIs) are not yet provided.Application in epilepsy: Several multi-center drug trials demonstrated the usefulness of the FePsy for the assessment of drug-related cognitive effects.
45
, 46
, 47
Simultaneous EEG recording and cognitive testing have proven to be a useful procedure to assess the cognitive effects of frequent, short, difficult-to-detect seizures (subtle nonconvulsive epileptic seizures) and of epileptiform EEG discharges in the absence of seizures.48
, 49
, 50
5.8 NeuroCog FX
The NeuroCog FX®
51
is a reliable and valid computerized neuropsychological battery for the repeated assessment of patients with epilepsy and other neurological diseases. In less than 30 min eight subtests assess attention (simple reaction, go/no-go, inverted go/no-go), short-term-memory (digit span), working memory (2-back task), verbal and figural memory (learning and recognition), and language (phonemic fluency).Explorative principal component analysis on raw scores from the total sample of subjects (healthy subjects and patients N = 379) extracted two factors (Eigenvalue > 1, VARIMAX rotation) which could best be described as “processing or operations” on the one hand (digit span, 2-back, verbal and figural memory, and phonemic fluency), and “psychomotor speed” on the other hand (reaction time-based tests: simple reaction, go/no-go, and inverted go/no-go). The model explained 60% of the variance. Based on this analysis, two scores of overall performance are calculated in addition to results on individual subtests and a total score. The two summary scores share about 16% of common variance (r = 0.42, p < 0.001). Random stimulus selection and RCIs allow for longitudinal assessments and determination of statistically significant change in individual patients. Special features include a graphical display of the standardized cognitive profile for profile analysis and export of data for data collection in a data base.
NeuroCog FX has its origins in epilepsy, but the test in its present form had been constructed for use in CNS diseases in general. It has been applied in monitoring in neuro-oncology and neurosurgery.
52
, 53
Moreover, the test has been used in neurological studies on patients with spinocerebellar ataxia54
and myotonic dystrophy.55
Major limitations are that, at present, the test is only available in German and that the patient's reactions have to occur within a fixed time limit so that patients with more severe psychomotor slowing cannot be tested with this tool.The NeuroCog FX has recently been licensed for exclusive distribution and use by Eisai.
Application in epilepsy: Concurrent validity of the NeuroCog FX with a battery of standard neuropsychological tests was established in healthy subjects and, separately, in patients with epilepsy.
56
An unpublished bachelor thesis57
demonstrated sensitivity of the test to several clinical parameters including age at seizure-onset, focus localization (frontal vs. temporal), and pharmacological treatment effects (drug load and specific antiepileptic drugs) in a cohort of 290 patients with epilepsy. This work furthermore confirmed the two-factor structure of the test.5.9 Test of attentional performance
The TAP (German: Testbatterie zur Aufmerksamkeitsprüfung) (http://www.psytest.net) is a collection of computerized tests for the assessment of attentional functions and visual processing among children and adults. It consists of 13 different subtests (31 when considering all subtest variations) covering different aspects of attention such as alertness, selective attention, divided attention, covered shift of attention, flexibility, working memory, sustained attention and vigilance. Furthermore several subtests for the assessment of visual field and neglect are included. Each subtest is preceded by a pretest to practice the exercise before starting the main test. The test is available in 13 different languages. Normative data exists for most of the subtests for adults and for about half of the subtests for children. The norm-values are calculated automatically under consideration of age, sex and educational level of the subject. The TAP has proven to be a reliable instrument in the context of clinical examination, educational psychology, assessment of fitness to drive and scientific research. It has been used in a variety of different research projects, numerous articles have been published during the more than 20 years since the first release of the TAP. Despite the fact that the TAP was devised to assess various aspects of attention, factor analysis indicates that the great number of subtests mainly reflect two constructs: (1) “alertness” and (2)“selective, i.e. divided attention”.
58
Application in epilepsy: In a survey on neuropsychological practice in German-speaking epilepsy centers the TAP was the most frequently used computerized test.
11
, 12
However, only up to five of the 13 TAP subtests (i.e. alertness, divided attention, go/no-go, reaction change and working memory) were used complementary to a comprehensive paper-pencil test battery. This underscores that the TAP, as a test battery primarily focusing on attention, was not devised as a stand-alone computer test, but rather complements other neuropsychological diagnostic procedures. The test has been used for assessment of attention and executive functions in patients with benign partial childhood epilepsy,37
juvenile myoclonus epilepsy,34
or for the determination of the cognitive status after surgery.26
Parts of it have also been used for the assessment of cognitive effects of antiepileptic drugs.59
6. Summary
In contrast to the plethora of coexistent neuropsychological paper-pencil tests which are routinely applied in the field of epilepsy,
10
, 11
only a relatively small number of computer tests has been used in patients with epilepsy or in antiepileptic drug trials. Among the nine computer tests reviewed here, merely three (33%), namely the CCTE, the FePsy and the Neurocog FX were devised for epilepsy. Consequently these three tests have been validated in patients with epilepsy. The CCTE and the Neurocog FX demonstrated concurrent validity with established paper-pencil tests with proven validity in epilepsy.14
, 27
Moreover, the CCTE has shown differential results depending on the lateralization in mesial temporal lobe epilepsy, the Neurocog FX is sensitive to several clinical parameters like age at seizure-onset or focus localization (frontal vs. temporal) and the FePsy can disclose the impact of acute epileptic EEG discharges on cognitive functioning. Some degree of sensitivity to antiepileptic pharmacotherapy (mostly to the total drug load) was demonstrated for most of the reviewed computer batteries (CALCAP, CCTE, CDR, CNT, CNTB, FePsy, and Neurocog FX). However, information on the clinical validity of the tests in epilepsy is mostly limited to single studies and conclusions about differential effects are derived from group statistics. Consequently the reported sensitivities do not imply that the respective computer tests will reliably discriminate between individual patients on the basis of the listed clinical parameters (e.g. side or site of the seizure-focus). The same is true for the sensitivity of the tests to antiepileptic drugs. Thus one cannot presently conclude that any of the tests is appropriate for the individual monitoring of antiepileptic pharmacotherapy - Kurzbuch K.
- Pauli E.
- Gaal L.
- Kerling F.
- Kasper B.S.
- Stefan H.
- Hamer H.
- Graf W.
Computerized cognitive testing in epilepsy (CCTE): a new method for cognitive screening.
Seizure. 2012; https://doi.org/10.1016/j.seizure.2012.08.011
2
. More information on the sensitivity and specificity of the measures in relation to an external criterion would be highly appreciated (e.g. receiver-operator characteristics).Nevertheless, the CCTE, the FePsy and the NeurocogFX appear suitable for screening purposes in the field of epilepsy, although they are not interchangeable since they assess somewhat different cognitive functions (cf. Table 1). Among these three tests delayed verbal (recognition) memory for instance is only assessed within the NeurocogFX. Although the test batteries are designed and proposed to evaluate several cognitive (sub)functions, the independence of the various test parameters is questionable, because corresponding factor analyses revealed comparably few underlying basic concepts.
Test-retest norms and parallel versions are necessary for repeated testing. Both, the CCTE and the Neurocog FX fulfill this criterion and provide reliable change indices. The special advantage of the FePsy is that it allows the coregistration of the EEG in order to detect the impact of epileptiform discharges or non-convulsive seizures on cognition.
7. Outlook
The steadily increasing availability of computer devices and internet access will surely change the current landscape of computerized testing. Web-based assessments could transfer cognitive assessments to the patients’ homes. This could be especially helpful in longitudinal long-term follow up. Futhermore, web-based data collection allows for a de-centralized application of tests in multicenter studies and evaluations of large cohorts of patients. The potential of mobile testing platforms (for instance cell phones or other portable devices like multimedia tablets) may lead to interesting new applications of neuropsychological testing. For example, Frings et al.
8
already demonstrated the benefits of handheld computers with regard to the early detection of cognitive and behavioral side effects of antiepileptic therapy with levetiracetam.How and which computerized tests can be incorporated in standard neuropsychological assessment remains a matter of choice. It is important that tests are used that have been validated and standardized. There is a long-standing history of publications accumulating evidence for the usefulness of paper-pencil tests in epilepsy-related diagnostic areas. To date, there is much less published evidence supporting the usefulness of computerized tests in patients with epilepsy. In particular, there is a lack of studies comparing standard paper-pencil and computerized tests addressing the same cognitive domains. With this in mind, a selection of tests for the domains of interest can be made on basis of Table 1.
For the time being, it is important to emphasize that, in most diagnostic situations, exclusive computerized testing in epilepsy cannot yet substitute a thorough neuropsychological examination of patients with epilepsy, especially not in surgical settings. Irrespective of the testing procedure, validity (clinical and ecological) must be the major selection criterion for neuropsychological instruments. Thus, published evidence is essential to prove the suitability of tests with regard to specific diagnostic questions. Only this will ensure an evidence-based future for neuropsychological testing.
Conflict of interest statement
CH received royalties from Eisai GmbH for the Neurocog FX®. WA reports no disclosures relevant to the manuscript. JAW has no conflicts of interest.
Acknowledgements
Many thanks to those authors, customer service assistants and experienced users of the computer assessment tools who provided us with valuable information and verified the correctness of the information presented in the tables. Since the authors of the CNTB could not be contacted, the information on that test is solely based on cited publications and supplied without liability.
References
- Clinical neuropsychology in epilepsy – theoretical and practical issues.in: Theodore W. Stefan H. Handbook of clinical neurology: epilepsy. Elsevier, 2012: 437-459
- Monitoring the cognitive effects of antiepileptic pharmacotherapy — approaching the individual patient.Epilepsy and Behavior. 2013; 26: 450-456
- Cognitive outcomes of different surgical approaches in temporal lobe epilepsy.in: Helmstaedter C. Hermann B. Kahane P. Arzimanoglou A. Neuropsychology in the care of people with epilepsy. John Libbey Eurotext, 2011
- Developing epilepsy-specific international cognitive assessment batteries: approaches, opportunities and limitations.in: Helmstaedter C. Hermann B. Kahane P. Arzimanoglou A. Neuropsychology in the care of people with epilepsy. John Libbey Eurotext, 2011
- Memory complaints in epilepsy: an accurate reflection of memory impairment or an indicator of poor adjustment? A review of the literature.Clinical Psychology Review. 2009; 29: 354-367
- Long-term memory impairment in patients with focal epilepsy.Epilepsia. 2007; 48: 26-29
- Postictal courses of cognitive deficits in focal epilepsies.Epilepsia. 1994; 35: 1073-1078
- Early detection of behavioral side effects of antiepileptic treatment using handheld computers.Epilepsy and Behavior. 2008; 13: 402-406
- Sensitive and specific neuropsychological assessments of the behavioral effects of epilepsy and its treatment are essential.Epilepsia. 2010; 51: 2365-2366
- Inquiry on assessments across epilepsy centers in different countries.in: Helmstaedter C. Hermann B. Kahane P. Arzimanoglou A. Neuropsychology in the care of people with epilepsy. John Libbey, Eurotext2011
- Neuropsychology in epilepsy part II: towards an establishment of diagnostic guidelines.Fortschritte der Neurologie Psychiatrie. 2009; 77: 691-698
- A survey on neuropsychological practice in German-speaking epilepsy centers.in: Helmstaedter C. Hermann B. Kahane P. Arzimanoglou A. Neuropsychology in the care of people with epilepsy. John Libbey, Eurotext2011
- Status of computerized cognitive testing in aging: a systematic review.Alzheimers Dement. 2008; 4: 428-437
- NeuroCog FX: computerized screening of cognitive functions in patients with epilepsy.Epilepsy and Behavior. 2009; 16: 298-310
- Influence of major antiepileptic drugs on attention, reaction time, and speed of information processing: results from a randomized, double-blind, placebo-controlled withdrawal study of seizure-free epilepsy patients receiving monotherapy.Epilepsia. 2006; 47: 2038-2045
- Cognitive effects of interictal epileptiform discharges in children.European Journal of Paediatric Neurology: EJPN: Official Journal of the European Paediatric Neurology Society. 2012; 16: 697-706
- The cognitive effects of interictal epileptiform EEG discharges and short nonconvulsive epileptic seizures.Epilepsia. 2012; 53: 1051-1059
- Memory alterations during acute high-intensity vagus nerve stimulation.Epilepsy Research. 2001; 47: 37-42
- Ictal neuropsychological findings in focal nonconvulsive status epilepticus.Epilepsy and Behavior. 2008; 12: 269-275
- Cognitive outcome of status epilepticus in adults.Epilepsia. 2007; 48: 85-90
- Computerized screening for HIV-related cognitive decline in gay men: cross-sectional analyses and one-year follow-up.in: Proceedings of the V International Conference on AIDS, Montreal, Canada1989: 465
- Impaired extra-dimensional shift performance in medicated and unmedicated Parkinson's disease: evidence for a specific attentional dysfunction.Neuropsychologia. 1989; 27: 1329-1343
- Cambridge neuropsychological test automated battery (CANTAB): a factor analytic study of a large sample of normal elderly volunteers.Dementia. 1994; 5: 266-281
- Further analysis of the cognitive effects of tetrahydroaminoacridine (THA) in Alzheimer's disease: assessment of attentional and mnemonic function using CANTAB.Psychopharmacology. 1993; 110: 395-401
- Clinical validation of Cambridge neuropsychological test automated battery in a Norwegian epilepsy population.Journal of Behavioral and Brain Science. 2012; 2: 108-116
- Computerized cognitive testing in epilepsy (CCTE).Epileptologia. 2010; 18: 79-86
- Computerized cognitive testing in epilepsy (CCTE): a new method for cognitive screening.Seizure. 2012; https://doi.org/10.1016/j.seizure.2012.08.011
- The value of assessing cognitive function in drug development.Dialogues in Clinical Neuroscience. 2000; 2: 183-202
- The dose-dependent cognitive effects of acute administration of Ginkgo biloba to healthy young volunteers.Psychopharmacology. 2000; 151: 416-423
- Effects of rivastigmine on cognitive function in dementia with lewy bodies: a randomised placebo-controlled international study using the cognitive drug research computerised assessment system.Dementia and Geriatric Cognitive Disorders. 2002; 13: 183-192
- The memory enhancing effects of a Ginkgo biloba/Panax ginseng combination in healthy middle-aged volunteers.Psychopharmacology. 2000; 152: 353-361
- The cognitive and psychomotor effects of remacemide and carbamazepine in newly diagnosed epilepsy.Epilepsy and Behavior. 2009; 14: 522-528
- Combined neuropsychological and neurophysiological assessment of drug effects on groups and individuals.Journal of Psychopharmacology. 2011; 25: 1062-1075
- Tracking the cognitive pharmacodynamics of psychoactive substances with combinations of behavioral and neurophysiological measures.Neuropsychopharmacology. 2002; 26: 27-39
- A method to combine cognitive and neurophysiological assessments of the elderly.Dementia and Geriatric Cognitive Disorders. 2011; 31: 7-19
- A cognitive and neurophysiological test of change from an individual's baseline.Clinical Neurophysiology. 2011; 122: 114-120
- Long-term and within-day variability of working memory performance and EEG in individuals.Clinical Neurophysiology. 2012; 123: 1291-1299
- Task-related EEG and ERP changes without performance impairment following a single dose of phenytoin.Clinical Neurophysiology. 2002; 113: 806-814
- Neurocognitive effects of brivaracetam, levetiracetam, and lorazepam.Epilepsia. 2011; 52: 264-272
- Neuropsychological and neurophysiologic effects of carbamazepine and levetiracetam.Neurology. 2007; 69: 2076-2084
- Distinct cognitive neurophysiologic profiles for lamotrigine and topiramate.Epilepsia. 2006; 47: 695-703
- The use of the computerized neuropsychological test battery (CNTB) in an efficacy and safety trial of BMY 21,502 in Alzheimer's disease.Annals of the New York Academy of Sciences. 1993; 695: 332-336
- A new assessment tool for neuropsychopharmacologic research: the computerized neuropsychological test battery.Journal of Geriatric Psychiatry and Neurology. 1991; 4: 211-217
- Topiramate dose effects on cognition: a randomized double-blind study.Neurology. 2011; 76: 131-137
- Withdrawal of antiepileptic medication in children – effects on cognitive function: the multicenter holmfrid study.Neurology. 1993; 43: 41-50
- The effect of the new antiepileptic drug rufinamide on cognitive functions.Epilepsia. 2006; 47: 1153-1159
- Randomized double-blind parallel-group study comparing cognitive effects of a low-dose lamotrigine with valproate and placebo in healthy volunteers.Epilepsia. 2002; 43: 19-26
- Acute cognitive effects of nonconvulsive difficult-to-detect epileptic seizures and epileptiform electroencephalographic discharges.Journal of Child Neurology. 2001; 16: 119-123
- Acute effects of subclinical epileptiform EEG discharges on cognitive activation.Functional Neurology. 2005; 20: 23-28
- The cognitive impact of epileptiform EEG discharges and short epileptic seizures: relationship to characteristics of the cognitive tasks.Epilepsy and Behavior. 2010; 17: 205-209
- NeuroCogFX – A computer-based neuropsychological assessment battery for the follow-up examination of neurological patients.Fortschritte der Neurologie Psychiatrie. 2006; 74: 643-650
- Computer-based assessment of cognitive functions in brain tumor patients.Journal of Neuro-oncology. 2010; 100: 427-437
- Neurocognitive and sociodemographic functioning of glioblastoma long-term survivors.Journal of Neuro-Oncology. 2012;
- Neuropsychological features of patients with spinocerebellar ataxia (SCA) types 1, 2, 3, and 6.Cerebellum. 2010; 9: 433-442
- The brain in myotonic dystrophy 1 and 2: evidence for a predominant white matter disease.Brain. 2011; 134: 3530-3546
- NeuroCog FX computerized screening of cognitive functions in patients with epilepsy.Epilepsy and Behavior. 2009; 16: 298-310
Lee, S-J. Diagnostic utility of NeuroCog FX in detecting and differentiating cognitive impairments in epilepsy patients. Bachelor Thesis. University of Cologne, 2010.
- Assessment of attention: derivation of a structural model and systematic classification of tests.Zeitschrift für Neuropsychologie. 2008; 19: 59-82
- Abrupt switch from extended-release oxcarbazepine to eslicarbazepine acetate.Der Nervenarzt. 2011; 82: 764-767
Article info
Publication history
Published online: April 29, 2013
Accepted:
April 4,
2013
Received in revised form:
April 3,
2013
Received:
January 18,
2013
Identification
Copyright
© 2013 British Epilepsy Association. Published by Elsevier Inc.
User license
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0) | How you can reuse 
Elsevier's open access license policy
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0)
Permitted
For non-commercial purposes:
- Read, print & download
- Redistribute or republish the final article
- Text & data mine
- Translate the article (private use only, not for distribution)
- Reuse portions or extracts from the article in other works
Not Permitted
- Sell or re-use for commercial purposes
- Distribute translations or adaptations of the article
Elsevier's open access license policy
ScienceDirect
Access this article on ScienceDirectLinked Article
- Computerized Cognitive Testing in Epilepsy (CCTE): A new method for cognitive screeningSeizure - European Journal of EpilepsyVol. 22Issue 6
- PreviewOptimized therapy in epilepsy should include individual care for cognitive functions. Here we introduce a computerized screening instrument, called “Computerized Cognitive Testing in Epilepsy” (CCTE), which allows for time-efficient repetitive assessment of the patient's cognitive profile regarding the domains of memory and attention, which are frequently impaired due to side effects of antiepileptic medication.
- Full-Text
- Preview
