If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Epilepsy Center, Department of Neurology, University of Erlangen-Nürnberg, GermanyInterdisciplinary Epilepsy Center, University of Giessen and Marburg, Germany
Optimized therapy in epilepsy should include individual care for cognitive functions. Here we introduce a computerized screening instrument, called “Computerized Cognitive Testing in Epilepsy” (CCTE), which allows for time-efficient repetitive assessment of the patient's cognitive profile regarding the domains of memory and attention, which are frequently impaired due to side effects of antiepileptic medication.
Methods
The CCTE battery takes 30 min and covers tasks of verbal and figural memory, cognitive speed, attention and working memory. The patient's results are displayed immediately in comparison to age-related normative data. For evaluation of psychometrics and clinical correlations, data from patients of a tertiary referral epilepsy center (n = 240) and healthy subjects (n = 83) were explored.
Results
CCTE subtests show good reliability and concurrent validity compared to standard neuropsychological tests (p < 0.01). Adverse cognitive effects of antiepileptic medication can be detected (p < 0.05), e.g. significant negative effects of increasing drug load. Specific epilepsy subgroups, e.g. focal versus primary generalized epilepsy or right versus left mesial temporal lobe epilepsy, showed different CCTE profiles.
Conclusion
CCTE appears valuable for early detection of individual cognitive alterations related to medication. In addition, it displays interesting differences between epilepsy syndromes. The CCTE battery provides a standardized, time- and personnel-efficient assessment of cognitive functions open to a large number of patients and applicable for clinical and scientific use in epilepsy.
1. Introduction
Cognitive functions and their maintenance play an important role in chronic epilepsies. Various cognitive functions can be influenced by seizures and epilepsy treatments, e.g. cognitive speed, attention, short and long term memory.
Cognitive malfunction may be permanent, corresponding to lesion and/or focus localization, for example poor verbal memory performance in left mesial temporal lobe epilepsy (mTLE).
Differences in memory performance and other clinical characteristics in patients with mesial temporal lobe epilepsy with and without hippocampal atrophy.
Influence of major antiepileptic drugs on attention, reaction time, and speed of information processing: results from a randomized, double-blind, placebo-controlled withdrawal study of seizure-free epilepsy patients receiving monotherapy.
Antiepileptic drugs (AEDs) often cause problems by dampening neuronal excitability and by altering underlying systems, which can lead to impairment of cognitive functioning within various neuronal subsystems.
Cognitive deterioration often seriously compromises the patient's ability to work and reduces quality of life and compliance. Indeed, cognitive impairment – mainly regarding memory and attention performance – is ranked among the most important concerns by patients with epilepsy.
Successful treatment in epilepsy therefore should aim beyond seizure freedom by focusing on an optimal balance between maximal seizure control and minimal adverse effects. As cognitive performance and side effects vary considerably among patients, the individual cognitive assessment should focus on detecting treatment impact on specific functions. Extensive assessment by experienced neuropsychologists, as for example done within presurgical evaluations,
is hardly feasible in everyday practice for large numbers of patients, especially in outpatient clinics and private practices. This implies a risk of delaying or missing relevant cognitive impairments before and during treatment.
A screening tool meeting the requirements of daily clinical practice by providing a standardized, valid and time-efficient measurement of cognitive functioning would be of great value.
Computer-supported neuropsychological testing has been increasingly used in the diagnosis of neurological disorders affecting cognition.
The Computerized Cognitive Testing in Epilepsy (CCTE), as described here, provides an effective and practical instrument for cognitive profile screening.
CCTE evaluates established cognitive performance in verbal and figural memory, cognitive speed, attention and working memory in a patient-controlled manner and allows for individual, age-related interpretation. In the current study, we introduce the CCTE battery, discuss its methodological rationale and present data from epilepsy and control subjects, supporting its psychometric validity as well as its ability in characterizing different and distinct cognitive profiles.
2. Methods
2.1 The CCTE battery
A Microsoft Windows operating system, 1024 × 768 resolution and a sound card are required to run CCTE. Participants perform a single session, seated with headphones in front of a touch screen monitor connected to a notebook in a quiet surrounding. CCTE delivers task instructions and materials verbally via headphones, alongside with the same information in writing via visual display. In a short training phase, each participant receives the opportunity to practice using the touch screen, and volume of the headphones can be adjusted individually.
CCTE consists of six tasks taking approximately 30 min to complete in total. These subtests refer to the following established neuropsychological paradigms. Task descriptions and screenshots can be found in Table 1 and Fig. 1a–h .
1.
The tasks digit span forward and backwards constitute the neuropsychological standard paradigm to assess working memory as the capacity to hold information in the mind and to make it available for further information processing.
In the focused attention task, a paradigm to assess speed of information processing is employed using sequences of simple calculations. Speed of information processing refers to how quickly one can react to incoming information, understand it, formulate and execute a response. This task also involves attention, working memory and decision making.
Performance is affected e.g. by the neurotransmitter balance of the brain, by the organization of neural networks supporting the respective procedure, and by the efficiency of the frontal lobes in organizing and directing information flow. Brain lesions, toxic substances and a variety of medications, including anti-convulsants, can slow information processing which often accompanies cognitive decline.
The visuospatial memory subtest examines the ability of storing and reconstructing the initial spatial arrangement of visually presented information, which is necessary for remembering the location of objects as well as orientation and navigation within one's environment.
Dissociated deficits of visuo-spatial memory in near space and navigational space: evidence from brain-damaged patients and healthy older participants.
Aging, Neuropsychology and Cognition.2011; 18: 362-384
The complex attention task refers to the paradigm of visual scanning as the ability to actively explore the visual surrounding and to find relevant visual information in a fast and efficient way, e.g. a particular object among other objects. This requires gaze control, fixation, controlling one's focus of visual attention and the ability to plan behavior systematically. Visual scanning deficits may result from a variety of brain diseases, neurotoxic substances and medications.
In the task of incidental memory, the paradigm of unintentional learning is represented, which means the random reception of information occurring in one's environment. As this experimental paradigm contains the unexpected demand of recalling previously seen objects, it differs from intentional learning and refers to daily life memory demands.
The verbal learning task refers to the learning of semantically associated and unassociated word pairs, a traditional neuropsychological standard paradigm to assess verbal memory, introduced by Hermann Ebbinghaus (1850–1909) and used in a variety of neuropsychological memory test batteries (i.e. Wechsler Memory Scale). Here, a paired-associate presentation and retrieval of stimulus and response are given.
The figural short term memory task represents a paradigm to assess working memory for figural material, as the capacity to store non-verbal information and to recognize it out of similar stimuli, a paradigm used in different neuropsychological test batteries.
An increasing number of digits (spanning from 3 to 9 digits) is presented acoustically to the participant. Immediately after the presentation, he or she is asked to repeat the ordered series by touching digits displayed on the screen. There are two trials for each series. The task is finished when the participant fails two times on the same number of digits. First part: digits forward; second part: digits reversed
Maximal length of correctly reproduced digit sequences
Focused attention
Out of 80 simple arithmetic expressions, the participant is asked to find and touch on the respective label all 45 expressions that result in even sums. The task is to be completed as quickly as possible and is finished as soon as all even results are found
Time needed to complete the task
Visuospatial memory
Given 13 everyday pictures of objects paired with 13 spatial locations represented by boxes on a screen, the participant is first asked to touch the objects as they are named one by one. Next, the participant is asked to recall and identify the spatial location of each object in an array of blank boxes
Number of correctly recognized picture locations
Complex attention and incidental memory
(a) As quickly as possible, the participant is asked to find and touch the same picture that is presented in the center of the screen. After the correct picture has been touched, another picture appears in the center of screen until all 24 pictures are presented (b) In the second part of this task, the respondent is asked, without previous instruction, to recognize all objects previously seen from an array of 48 written words
(a) Average speed of response (b) Number of correctly memorized objects
Verbal learning
The task consists of AB–AC paired-associate learning. Eight word pairs (four of them moderately related) are presented in three consecutive learning trials. Three additional trials are administered for word pairs with the same cue words but with different response words (e.g. AB: mother–child, AC: mother–father). The presentation is given both acoustically and in written form on the screen. After each trial, the respondent is presented the cue word and is asked to write the response word with a virtual keyboard on the touch screen
Number of correctly reproduced words
Figural short term memory
The respondent is given a series of 12 figures consisting of 6-by-6 squares of equal size. In each figure, six of the 36 squares are colored. Immediately after the 5-s presentation of each single figure, four figures are presented and the respondent is asked to indicate the previously seen figure
The course of subtests is controlled by a script directing the participant through the tasks without any external assistance. After some short instructions, each task is started by the patient himself. Immediately after completion, age-related results are displayed in raw values and z-scores and presented in graphic form. For follow-up examinations, the battery is available in two parallel versions, referred to as versions “A” and “B”, comprising the same task structure but differing slightly in the presented words and pictures.
Original raw scores and z-values for each subtest are separately listed in comparison to respective age-related norm values. Patient scores are displayed graphically: the bold circle represents norm mean value level (z = 0), inner and outer rings illustrate up to 3 standard deviations (z = +3/−3); black lines connecting individual subtest values illustrate the patient's performance.
Fig. 2Cognitive profile as evaluated by CCTE: test results indicate normal cognitive functions in this 50-year-old female patient with newly diagnosed epilepsy before initiating AED treatment.
The patient group comprised unselected male and female patients with active epilepsy from the Epilepsy Center Erlangen (n = 240). To ensure comparability of examined groups, patients with an IQ below 85 (based on the Mehrfachwahl-Wortschatz-Intelligenz test (MWT-B), a German assessment of verbal intelligence) were excluded from the study. All participants gave informed consent prior to their inclusion in the study; approval was given by the local ethics committee. For control data, healthy male and female volunteers were evaluated (n = 83). Exclusion criteria for healthy controls included history of any psychiatric or neurological disorders and IQ below 85.
2.3 Psychometric evaluation
For psychometric evaluation of CCTE, retest reliability and construct validation with standard neuropsychological means as well as a factor analysis were conducted.
2.3.1 Retest reliability
To survey test–retest reliability of CCTE, a group of 46 patients completed the test twice at an interval of three months remaining on stable medication between both investigations. Daytime, location and caring professional were the same in both sessions. Participants were randomized; half of them took version A at the first test and parallel version B at the second test, the other half vice versa.
2.3.2 Construct validation
To determine whether the CCTE subtests give similar results as comparable established methods measuring the same theoretical construct, epilepsy patients undergoing comprehensive assessment with established test instruments also performed CCTE within the next five days. In detail, representative patient groups completed the Trail Making Test (TMT-A, a measure for cognitive processing speed)
Weidlich S, Lamberti G, Hartje W. DCS – Diagnosticum für Cerebralschädigung. Ein visueller Lern- und Gedächtnistest nach F. Hillers. 4. Auflage. Göttingen: Hogrefe; 2001.
Test scores were compared to the representative CCTE subtests using Pearson's product-moment correlation. To work out how much an individual's score must differ from test 1 to test 2 to mark a statistically significant change, reliable change indices (RCI) were calculated for each subtest.
2.3.3 Factor analysis
Principal component factor analysis was applied to uncover comprehensive cognitive parameters underlying the pattern of correlations between the different subtests of CCTE. We used CCTE subtest raw scores from the total sample of subjects. Extraction criterion was an Eigenvalue > 1, as factor rotation (Varimax) with Kaiser normalization was used.
2.4 Clinical correlations
CCTE patient results were compared against healthy controls and across different patients groups and clinical parameters, e.g. number of AED or epilepsy syndrome (focal versus generalized epilepsies; left versus right mesial temporal lobe epilepsy; patients without AED (before treatment) versus AED monotherapy versus AED polytherapy). Scores were compared using independent t-tests.
Statistical analysis was performed with SPSS 18 for Windows (PASW Statistics 18.0, SPSS Inc., 2009). Two-tailed statistics were used throughout defining significance by p < 0.05.
3. Results
3.1 Patient population
Table 2 summarizes patient and control characteristics.
Table 2Demographic data and epilepsy characteristics.
The control group, spanning four decades, was divided in three age groups: 20–30, 31–50 and 51–60 years, addressing possible age-related impact on the examined cognitive functions. Table 3 shows results of the CCTE subtests for the age groups.
Table 3CCTE subtest mean scores (M) and standard deviations (SD) in the three age groups of healthy controls.
All but one subtest showed good retest reliability (p < 0.01; r-values 0.40 ≤ r ≤ 0.79). The figural memory subtest showed improvement from test A to test B and no significant test–retest correlation (r = 0.08; p > 0.05). Table 4 shows reliability scores and reliable change indices for significant changes in individual performance from test 1 to test 2.
Table 4Retest-reliability, mean scores (M), standard deviations (SD), reliable change indices (RCI).
p<0.01. RCI, reliable change index (point score or reaction time difference which marks a significant change from test 1 to test 2); 95%-confidence interval.
p<0.01. RCI, reliable change index (point score or reaction time difference which marks a significant change from test 1 to test 2); 95%-confidence interval.
p<0.01. RCI, reliable change index (point score or reaction time difference which marks a significant change from test 1 to test 2); 95%-confidence interval.
p<0.01. RCI, reliable change index (point score or reaction time difference which marks a significant change from test 1 to test 2); 95%-confidence interval.
p<0.01. RCI, reliable change index (point score or reaction time difference which marks a significant change from test 1 to test 2); 95%-confidence interval.
p<0.01. RCI, reliable change index (point score or reaction time difference which marks a significant change from test 1 to test 2); 95%-confidence interval.
p<0.01. RCI, reliable change index (point score or reaction time difference which marks a significant change from test 1 to test 2); 95%-confidence interval.
** p < 0.01.RCI, reliable change index (point score or reaction time difference which marks a significant change from test 1 to test 2); 95%-confidence interval.
Validity estimates according to Pearson for standard neuropsychological means and respective CCTE subtests showed significant correlations (0.64 ≤ r ≤ 0.82; p < 0.01). Table 5 shows the compared parameters and the respective correlations.
Table 5CCTE subtests, established neuropsychological measures and their correlation coefficients (r).
DCS, DiagnostikumfürcerebraleSchädigung (a visual learning and memory test); VLMT, German version of Rey Auditory Verbal Learning Test; WMS-R, word pair task from Wechsler Memory Scale-R; TMT-A, Trail Making Test A.
Principal component analysis (Eigenvalue > 1, Varimax rotation) of raw scores from the total sample of subjects (n = 323) extracted two factors. The first factor, referred to as “ATTENTION”, comprises the subtests of digit span, focused attention, complex attention and figural short term memory. The second factor, referred to as “MEMORY”, is covered by the subtests of visuospatial memory, incidental memory and verbal memory. Factor loadings are listed in Table 6. The model explained 58.8% of the variance.
Table 6Factor loadings of CCTE subtest scores.
CCTE subtest
Factor
ATTENTION
MEMORY
Digit span forward
0.74
Digit span reversed
0.74
Focused attention
0.71
Complex attention
0.63
Figural short term memory
0.77
Visuospatial memory
0.81
Incidental memory
0.75
Verbal memory
0.63
(n = 323).
Principal component analysis with Varimax rotation.
CCTE subtest results appeared generally lower in epileptic patients versus controls and differed between focal (n = 222) and generalized epilepsies (n = 18) with lower values in partial epilepsy in most subtests (p > 0.05) (Fig. 3a and Table 7a).
Fig. 3(a) CCTE subtest z-scores of healthy controls, patients suffering from focal epilepsies and primary generalized epilepsies. (b) CCTE subtest z-scores of left and right mTLE and healthy controls. *Significance p < 0.05. (c) CCTE subtest z-scores of untreated patients, AED monotherapy, AED polytherapy and healthy controls. *Significance p < 0.05 for comparison of monotherapy/polytherapy. (d) CCTE subtest z-scores of patients on AED monotherapy, therapy with 3 or more AEDs and healthy controls. *Significance p < 0.05. DF, digit span forward; DR, digit span reversed; FA, focused attention; VM, visuospatial memory; CA, complex attention; IM, incidental memory; VEM, verbal memory; FM, figural memory.
Left mTLE patients (n = 31) revealed worse CCTE results in the verbal memory task (p < 0.05), as compared to right mTLE patients (n = 38) (Fig. 3b and Table 7b).
Table 7bCCTE subtest mean scores (M) for the groups of left versus right mTLE and significance of comparison (p).
Patients untreated (n = 42) showed better performance than patients on AED monotherapy (n = 79) or AED polytherapy (n = 119) (Fig. 3c and Table 7c). The two latter groups differed significantly (p < 0.05) with lower scores for patients on polytherapy than on monotherapy in most subtests. Comparisons between untreated patients and patients on monotherapy did not differ significantly.
Table 7cCCTE subtest mean scores (M) for the groups of patients with no AED/monotherapy/polytherapy and significance of comparison (p).
Patients on AED monotherapy (n = 79) did significantly better compared to patients on three or more AEDs (n = 31) in most subtests (Fig. 3d and Table 7d).
Table 7dCCTE subtest mean scores (M) for the groups of patients on 1 AED versus ≥3 AED and significance of comparison (p).
Table 8 lists the frequencies of subaverage scores (z ≤ −1,0) for each subtest in the groups of generalized/focal epilepsies, left/right mesial temporal lobe epilepsies and patients on monotherapy/polytherapy.
Table 8Frequencies of subaverage performance (z ≤ −1,0) in the groups in %.
Significant changes of an individual cognitive profile during treatment adjustments are illustrated in Fig. 4. During her first assessment, the patient suffering from drug resistant focal epilepsy was treated with topiramate (200 mg/daily) and complained of severe cognitive impairment confirmed by underperformance in most subtests of the CCTE. After replacement of topiramate by lamotrigine (150 mg/daily), three months later performance improved in all previously impaired domains of CCTE.
Fig. 4CCTE profiles of a 21 years old female patient with cryptogenic epilepsy. Assessment 1 under medication with topiramate (gray lines), assessment 2 after replacement of topiramate (black lines). A clear improvement was seen in the majority of subtests.
This study presents cognitive profile data from epilepsy patients, assessed by CCTE, a new computerized screening instrument developed to facilitate the assessment of certain cognitive functions in epileptic patients. CCTE tasks represent computer adaptations of established neuropsychological paradigms covering cognitive functions of working memory, verbal and visuospatial/figural memory, cognitive speed and attention.
It could be demonstrated that CCTE provides correlations to clinical syndromes and is able to show different degrees of impaired cognition. It evidently captures the spectrum of dysfunction in the measured cognitive domains in epilepsy patients compared to healthy subjects. Differences between focal and primary generalized epilepsies were shown, with a lower performance of focal epilepsy patients (Fig. 3a). These differences were non-significant, nevertheless they seem to reflect the fact that patients with localization-related epilepsy are often more cognitively impaired due to the impact of the underlying brain lesion.
Patients with left mTLE differed from patients with right mTLE by their significantly lower performance in the verbal memory subtest of CCTE, confirming previous work.
Differences in memory performance and other clinical characteristics in patients with mesial temporal lobe epilepsy with and without hippocampal atrophy.
As words can be typed on the touch screen, the verbal learning task of CCTE is particularly sensitive to memory decline by demanding free recall performance from the participant; while many other computer-based memory tests rely on verbal memory recognition rather than free recall, which is due to the lack of a suitable answering device. Despite the lack of a delayed recall condition, the subtest provides the experimental paradigm of pair-associate learning which serves to be a good measure of unilateral lesions and hippocampal learning
Impairing effects of medication are demonstrated by CCTE performance profiles (Fig. 3c and d), showing a significantly negative impact of administration of three or more AEDs. This underlines the importance of identifying and minimizing deteriorations in the individual cognitive profile over the course of therapy in order to preserve school and job performance. Topiramate's potential negative effects on language, memory and cognitive speed are well described in previous studies
and are reflected in the improvement of the presented patient's CCTE profile after withdrawal (Fig. 4).
Psychometric evaluation was conducted to estimate reliability and validity of the test battery. Test–retest calculations showed good correlations, providing evidence for sufficient reliability. Construct validity was estimated by correlating CCTE subtests with established neuropsychological measures. The results indicated that the test battery is a valid and sensitive assessment tool to address cognitive functions covered by standardized tests, including working, verbal and figural memory as well as attention. By conducting factor analysis, two major components could be extracted, representing the neuropsychological domains of attention and memory. Both domains are especially vulnerable for negative drug effects and are often cause for complaint by patients.
Although computerized assessment has certain important advantages, a relatively short screening cannot provide the amount of data generated by a full neuropsychological examination, as it is required in pre-surgical evaluation.
Therefore, a screening system like CCTE does not make a claim to measure the entire spectrum of cognitive domains. It rather aims to capture those functions which are most likely to be impaired by antiepileptic medication and have crucial effect on the patient's everyday performance – which particularly applies to memory and attention. CCTE enables administration of cognitive screening for certain frequent deficits for many patients in an economic fashion, especially in an outpatient setting. Early detection of deficits facilitates an in time intervention.
Certain advantages arise from the computer-based structure of CCTE.
Without external assistance, the test battery can be accomplished by the examinee in about 30 min. Administration and stimulus presentation are provided in a standardized, multimodal and objective way, independent from an examiner and thus excluding tester bias and variation in the delivery of instructions. Automated comparison with age-related norms is provided as graphical profile immediately after the test. The data of different examinations of the same examinee can be shown in one profile, which enables a comparison in follow-up assessment. The individual results in graphical form can be easily interpreted and explained to the patient. More detailed single case analysis is enabled and provides precise reaction times, response latencies and correctness of response as well as learning curves and interferences or lateral preferences.
Another advantage lies in the high participant acceptance of computerized assessments.
As the test challenge seems playful, no excessive pressure is put on the examinee, mitigating anxiety and stress. Because of the short duration, a steady motivation is facilitated and fatigue effects are prevented. Nevertheless, limitations of computerized assessments may become evident when possible reduced fine motor skills or lack of computer practice of an individual patient interferes with efficiently interacting with the computerized device.
However, because the examinee uses an arbitrary finger to communicate with the touch screen of CCTE and there is no need for other devices such as a keyboard or computer mouse, handling is kept easy for most patients. Nevertheless, some problems were experienced with a minority of cognitively more severely impaired patients who had problems to understand instructions instantly and needed further explanations.
There are two parallel CCTE versions at present, additional parallel forms are currently developed. In this study, tests were administered in German language. Since all of the CCTE text parts are deposited in external text files coded in Unicode, the program can be translated into any other language. Multilingual versions are in progress and could open the way for valuable transcultural comparisons. Because of the script structure, the number of tasks is expandable. As electronic filing is integrated in the program and data can be exported in other file structures, the aspect of data gathering for scientific use is brought into effect. Due to its efficacy, the battery is suitable for implementation of neuropsychological assessment into scientific studies in a standardized and comprehensive way, involving large patient samples. This makes it an interesting tool for e.g. multi-center treatment outcome studies or telemedicine settings, respectively.
5. Conclusion
The present study demonstrates that the computer-based test CCTE can provide effective screening for memory and attention functions in epilepsy. CCTE covers different aspects of these cognitive domains and is able to show different levels of cognitive impairment, especially related to antiepileptic medication. Due to program structure, computerized setting and short duration, the system proved to be time- and personnel-efficient and of high feasibility, usability and acceptance by patients. These factors make CCTE potentially suitable for outpatient units to screen individual cognitive profiles and to monitor unwanted AED related cognitive side effects or changes over the course of therapy. By virtue of its computerized structure, CCTE may also be applicable in scientific multi-center studies and telemedicine.
Conflict of interest statement
None of the authors of this study have any proprietary, financial or other conflicts of interest in relation to this work.
Acknowledgements
This study was supported by ELAN Fond (University Erlangen-Nürnberg), No. 08.05.26.1. We confirm that this study has been approved by the appropriate ethics committee and has therefore been performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki.
Differences in memory performance and other clinical characteristics in patients with mesial temporal lobe epilepsy with and without hippocampal atrophy.
Influence of major antiepileptic drugs on attention, reaction time, and speed of information processing: results from a randomized, double-blind, placebo-controlled withdrawal study of seizure-free epilepsy patients receiving monotherapy.
Dissociated deficits of visuo-spatial memory in near space and navigational space: evidence from brain-damaged patients and healthy older participants.
Aging, Neuropsychology and Cognition.2011; 18: 362-384
Weidlich S, Lamberti G, Hartje W. DCS – Diagnosticum für Cerebralschädigung. Ein visueller Lern- und Gedächtnistest nach F. Hillers. 4. Auflage. Göttingen: Hogrefe; 2001.
00223-3&id=gr1.jpg){kind=link}
00223-3&id=gr2.jpg){kind=link}
00223-3&id=gr3.jpg){kind=link}
00223-3&id=gr4.jpg){kind=link}