Attention test: a short introduction

Usually an attention test requires to detect something in a lot of stimuli. For example, searching 4 dots in lines of groups of 3 or 5 dots, or detecting a 4 between numbers (0-9) sequentially displayed on a computer screen. In such tests you just have to focus on one task or thing and try not to be distracted by another (distractor) task.

These attention tests are set up to investigate concentration (see my page on Attention problems for an explanation of different types).

Unfortunately, attention tests rarely resemble daily life activities and it is therefore questionable how accurately they really measure your concentration skills in daily life. One thing is certain though, most such tests require quite a lot of focus. Probably much more than daily life activities.


A review of some attention tests

As I told you earlier, I can not discuss all available attention tests used in neuropsychology. Therefore, I would like to review tests which in my opinion should be used the most because of their proven ability to measure attention deficits in patients and normal (healthy) persons.

Furthermore, I would like to discuss some well-known and well-used attention tests that - in my opinion - are overrated. I would like to point out serious weaknesses of such tests and urge clinicians to use other attention tests for the sake of patients.


Test of Sustained Selective Attention (TOSSA)

To be honest here, I am a bit biased because this attention test was developed and computerized by myself and it is now published by my own company Pyramid Productions. Nevertheless, it still has several qualities and is therefore used as an example of a fine test.

It is an 8-minute computerized auditory sustained attention test in which you will here 240 groups of 2, 3, or 4 beeps. The 3 beeps are randomly presented throughout the whole test and must be detected. When you hear the 3 beeps you have to press the spacebar. No reaction times are measured but your reactions are recorded. You can either press correctly on 3 beeps, or press incorrectly on 2 or 4 beeps. Pressing incorrectly is interpreted as being insufficiently focused on your task (i.e., pressing only on 3 beeps). You can also fail to react when a 3-beeps group passes by. Then this is also interpreted as not being attentive or focused enough. This attention test is 8-minutes long and that turned out to be sufficiently long to measure quite sensitively concentration problems. And it was not too long to be useless in patient groups. Patients can do the task fully, usually not really liking this test because it can be difficult for them, but the dislike is not too strong.

The TOSSA is an example of a special kind of attention test: so-called continous performance test. You have to perform continously for several minutes. Other fine examples are the TOVA (Test of Variables of Attention), the IVA (Integrated Visual and Auditory CPT) and the Connors CPT. They all last from 4 to 20 minutes. The TOSSA differs in several respects of these other attention tests. It is much easier: most healthy people perform very well on the TOSSA (median score is 93.6%). The TOSSA is one of the shortest continous performance tests. The TOSSA is only auditory, no visual concentration is needed. The TOSSA does not measure any kind of reaction time. The TOSSA is less well normed in healthy people but very well normed in a large group of neurological patients. No other attention test has such a large norm group of different neurological patients (n=1019). The TOSSA only requires a Windows based computer. It can run stand-alone or in a network.

Update 20th July 2012: The TOSSA now includes profiles. It automatically calculates several clinically relevant profiles like Mental Slowness, Enhanced Fatigability, Endogenous Arousal Problems, Structural Impulsivity, Conditional Impulsivity, Auditory Perception problems, Suboptimal Effort. Especially this last profile is interesting because especially with attention tests patients can very easily show suboptimal levels of effort and it is hard to differentiate such levels from extreme - out of the normal (even for brain injuries) - performances. Now the TOSSA version 3.0 gives you the opportunity to do just that.

With these profiles the interpretation of the TOSSA results becomes easier, although any interpretation of a neuropsychological test requires expertise and clinical experience.

Psychometric characteristics of the TOSSA are sufficient and clinically meaningful. We try to publish this in a scientific journal.

The reliability of the main indices range from .84 (test-retest reliability) to .92 (internal consistency) in a sample of 101 neurological patients.

The convergent validity is sufficient, ranging from .42 to .55. The divergent validity ranges from .17 to .43. The problem is that the other attention tests used in the study were not comparable continuous performance tests except for the new attention test TODA (Test of Divided Attention). It would be interesting to see whether the TOSSA yields the same results as the TOVA or IVA.

The TOSSA is fully standardized with clear and short instructions to administer, computerized to ensure perfect recording of responses and reliable calculation of the main indices.

The TOSSA is sufficiently patient friendly meaning that all patients can do the test without becoming too much frustrated or irritated. The TOSSA can be administered to one-handed patients and a large benefit is that it can be used in blind or near-blind patients as well. Most concentration tests are not suited for visually impaired patients.

The costs of the TOSSA are comparable to other concentration tests: 160 euro ex. VAT for 1 year license on 2 computers. License renewals per year will be the same price. When sufficiently popular the prices will go down to ensure worldwide use and distribution.

The TOSSA has shown high sensitivity and specificity in several pilot studies and can detect more easily concentration problems than other tests. It can even differentiate between several patient groups whereas some other tests could not.

The TOSSA is easily available via a download-link from www.pyramidproductions.nl. The test just recently was translated into English with an English manual as well. See the downloadlink below the two graphs for the English manual.

Below you can find two figures: a results screen of the TOSSA in which you can see the blue line which is the focus of the patient; and a red line which tells you how many times there was a hit on a distracter (the lower the red line, the more distracted). The second figure shows a norm groups results graph where the patient (black dot) is compared to the norm groups.



TOSSA manual in English to download here



Improvements to be made to the TOSSA?

The TOSSA seems to be a good test. It is especially very well normed for neurological patients so comparisons with these groups can be made. It is a rather short test but long enough to detect sustained concentration problems. It can also be used in severely visually impaired patients and severe aphasic patients.

But of course, as with other attention tests, improvements can still be made as well. Especially, more healthy people should take the TOSSA. Just to be sure that indeed this test is very easy for most healthy people. Also, it would be interesting to administer the TOSSA in more children, especially in the age range of 7 till 16 years. Furthermore, the data recorded should be saved in an Excel file as well (instead of a CSV-file).

Below you see an Evaluation Table for the TOSSA. Using this Table every neuropsychological test can be evaluated. See for more information about the criteria of evaluation my page on Test-Psychology. See this link: Go to Test-Psychology



Download the Setup file for TOSSA here

Download the Installation instructions for TOSSA here

Go to Pyramid Productions for the TOSSA



Test for Divided Attention (TODA)

The TODA is one of the attention tests developed to measure divided attention: doing more than two things at once. There are not many tests around that assess this aspect of attention. We do have the Paced Auditory Serial Addition Task (PASAT) but there are several problems with this test, although this task is used a lot in scientific research papers. First of all, it is extremely difficult and frustrating for patients but also for a lot of healthy controls. Especially when the shortest inter stimulus intervals of 1.4 or 1.2 seconds are used. In my opinion, the test then does not really measure what it should measure and the differentiation between patients and healthy controls becomes a serious problem. Furthermore, the test is not available in a standardized form. Usually, it is recorded on a tape, voices differ largely across versions and the tape recordings are of dubious quality. Norms for this test are doubtful, often only on a small number of healthy controls or brain injured patients. But one of the most serious problems with this PASAT is that this test is certainly not user friendly. Most patients I have tested get so frustrated they do not want to do the test anymore. So it is rarely administered completely, biasing the results. Last but not least: the PASAT has serious test-retest effects. In some studies doing the test twice can result in 18 to 20% improvement!

The TODA is much more user friendly: it can be done for more brain injured patients than the PASAT. Frustration and irritation can increase in this test as well, but to manageable levels. Furthermore, the TODA isn't that difficult to do for healthy controls, not even for highly aged people. In this way, much less false positives are found and at the same time divided attention deficits can be found. The test is sufficiently sensitive to brain damage and attention deficits.

The TODA is available as a highly standardized computer test (for Windows computers), so every patient has exactly the same version. It is easily available via www.pyramidproductions.nl but currently only in Dutch. An English translation will become available in 2011.

The TODA is 9 minutes long, it will take about 11 minutes to administer and so one of the longer attention tests we have. Usually, it is recommended to do the test after the TOSSA has been done. The TOSSA can find any concentration deficits, the TODA has to be used to find divided attention problems. It happens more than once that the TOSSA does not find any difficulties in concentration, however, the TODA does indeed find that the same patients indeed have problems doing two things at once. Thereby corroborating their complaints in daily life about life being too fast.

The TODA requires to push on one of three buttons (the arrow keys on your keyboard) after being presented a vertical sum (e.g. 2+3=5) and hearing either 2,3, or 4 beeps.

See this Figure for a stimulus of the TODA:


Whenever the sum is correct and 3 beeps can be heard, one has to push the left arrow (with just one hand, using 3 fingers, as if playing the piano). When either the sum is incorrect or not hearing 3 but 4 or 2 beeps, the middle arrow key has to be pushed. The right arrow key is when both the beeps are incorrect (4 or 2) and the sum is incorrect (e.g. 2+3=7). After a short practice trial, the patient usually understands the instructions, and can begin the real test.

The TODA has two speeds: one slow and one quick and this (same) sequence of stimuli is repeated once for each speed. In this way it can be easily calculated how speed, duration and possible learning effects, have an effect on the results.

No reaction times are recorded but only the number of correct responses on each of 4 possible stimuli (both are correct, both are incorrect, only beeps are correct, only the sum is correct).

The TODA measures divided or switching attention. Some might even say that flexibility is required to constant switching of the responses.

It has some learning effects but the TODA can be repeated in a couple of months without having too much test-retest effects.


Trail Making Test

The Trail Making Test is one of the most used neuropsychological tests in the Western world. In fact, it is thé most used attention test. Probably not only because it is very easy to administer (it takes about 5 minutes), but also because there are at least 46 normative studies of the TMT. It has even made it into Nintendo DS Brain Trainer game, thereby seriously reducing its usability as a neuropsychological test due to practice effects a lot of elderly and adults have with this test.

Although very popular in clinical practice (and research studies), I have my reservations with this test due to my experiences with it. When you look at my five criteria for a good neuropsychological test (see my page on Test-Psychology): reliability, validity, standardization, availability and usability it scores reasonably well on reliability, standardization, availability and usability. Most used is the visual form known as TMT-A and TMT-B.

Reliability: The TMT seems to be reasonably reliable when considering test-retest reliability. In several studies in neurological populations the reliability coefficient of TMT-A seems somewhat lower than that of TMT-B, around .55 to .70. It seems to vary across populations and time intervals (as do all neuropsychological tests, as a matter of fact).

Validity: Several studies show a moderate correlation between the TMT and other tests of concentration and/or processing speed, such as the PASAT and the Symbol-Digit Modalities Test. Factor analytic studies show factors like attention, visual search, speed of processing and executive control, that play a role especially in TMT-B. However, it seems to be important what variable is taken as the most important measure of the TMT. Most authors use the TMT-B/TMT-A ratio as a valid measure of problems in executive control and divided attention. It is however not really clear what the cut-off score really should be (also largely depending on age and education). Usually, a ratio of 3 or higher is used.

In my own study, in which I compare several tests like the TMT, Stroop, d2, Digit-span and the new tests TOSSA and TODA, the much used ratio score is NOT very sensitive. That is exactly my clinical experience with more than 3000 neurological patients I have seen over the years. Recent research has suggested to use the difference score TMTB - TMTA instead as a more reliable and valid measure of attentional problems. In my study, this difference score is indeed much more sensitive to attentional problems than the frequently used ratio score. For more details about this study see the TOSSA manual.

Furthermore, the TMT has serious validity issues when you take the difference in time between TMT-A and TMT-B. That should only be allowed when all factors but one are equal. That is not the case: both the spatial arrangements and items sets differ between TMT-A and TMT-B, making a comparison in seconds much more difficult to interpret. It is likely that TMT-B is not just difficult for patients due to attentional problems, but also because of a different demand on motor speed and visual-perceptual processes.

Another serious problem in using the TMT is that it can not be used reliably in patients with visuospatial problems like neglect, hemianopia or visual agnosia.

Practice effects are not as large as in the PASAT, but still there are test-retest effects so retesting should not take place in several months.

All in all, the TMT does tell something about divided attentional problems, information processing speed and visual search. But, it certainly is not a test to use only when trying to detect attentional problems. The probability that you will miss such problems is far too high, leaving a lot of patients without any help for their complaints. My recommendation would be that the TMT can be used as a short screening instrument in case no other tests for concentration are available. However, when the TMT is negative, care should be taken to administer more sensitive tests, especially when patients do have serious complaints indicating attentional problems.

Below you will see a picture of all Trail Making Test forms as can be found on the Halstead-Reitan website www.reitanlabs.com.



Go to Official website for the Trailmaking Test



The Integrated Visual and Auditory Continous Performance Test (IVA plus)

The IVA plus is an improved version of the IVA (www.braintrain.com), a continuous performance test (CPT) that both uses visual and auditory stimuli. It is a computerized attention test and it takes 13 minutes to do. Although a review in Strauss, Sherman and Spreen (2006) was quite critical there are several positive things to say about the IVA. However, I have to admit that I personally do not know this test (but I would like to).

The instructions for the IVA can be found simply on their website http://www.braintrain.com/professionals/adhdtesting/ivaplus_pro.html:

"The subject is required to click the mouse only when he sees or hears a "1" and to inhibit clicking when he sees or hears a "2." During some segments of the IVA+Plus test, the "1"s are more common than the "2"s, creating a response set which "pulls" for errors of commission, or impulsivity. During alternate segments of the IVA+Plus test, the "1"s occur rarely; this invites more errors of omission, or inattention, since the subject must remain vigilant while he waits for a "1" to occur."

First of all, the IVA is easily available via www.braintrain.com. It is completely standardized and that is a big plus. Furthermore, it clearly has a lot of potential to detect attentional problems. It is 13 minutes in duration so sustained attention (and fatigue) can be assessed quite convincingly. It uses a varied Inter Stimulus Interval (ISI) time, putting different demands on attentional capacity. Also a varied target to distracter ratio is used to assess impulsivity and/or concentration more reliably.

However, there are some serious drawbacks. Just as with the TOSSA the large sample of 1700 individuals is put together more conveniently than with stratified sampling (unfortunately, a lot of tests are). The age range is from 6 till 99 but this is not uniformly distributed in cell sizes.

But much more serious is the lack of test-retest reliability of some of the most important indices like the Full-scale Response Control Quotient and the Auditory Response Control Quotient (<.59). The Full-scale Attention Quotient range around .66 and .75. I must say this was only tested in 70(!) normal volunteers and it is usually the case that in normals reliability coefficients can be lower than in patient groups. I wonder whether the test-retest reliability of the IVA would be higher in clinical groups (in the TOSSA it IS higher). Other reliability analyses are not mentioned in the manual.

Also a lot of important research about validity, especially convergent and divergent validity is not mentioned sufficiently in the manual. This lack of for example correlations between the IVA and other well-known attention tests is very serious because now it can not be established how well the IVA can assess attentional problems. According to how the test is developed, it MUST have a high potential in detecting attentional problems. I am pretty sure it has, but of course this has to be corroborated by independent studies in especially clinical (neurological) groups. The TOSSA has a large database of just such neurological groups and it would therefore be very interesting to administer both the IVA and TOSSA for cross-validation studies.

However, a large practical problem can be its length. 13 minutes just the test itself takes; the whole test administration with instructions and practice takes at least 20 minutes (with neurological patients I assume it would take even more, probably up to 28 minutes). However trivial this may seem, it seriously hinders the application of the IVA in clinical practices.

In summary: although there are not enough studies and data to convincingly show that the IVA is a good attention test, I think that there are good reasons to assume that it actually can be a good attention test. Especially its standardization and easy availability, and its well-thought construction, are promising. Only when more studies are done in clinical (neurological) patient groups, perhaps coupled with the TOSSA, data could show that the IVA is one of the attention tests to be used in neuropsychology. (But see below, the TOVA).

Below you will see a picture of an instruction screen for the IVA (free to download from www.braintrain.com) and next you can see the Evaluation Table for the IVA.



Go to Website IVA


The Test of Variables of Attention (TOVA)

Another attention test and Continuous Performance Test on the computer is the T.O.V.A. As you can see in the Evaluation Table it has some nice features so one wonders why it is not used more often in neuropsychological practice.

Well, for one thing, the time to administer this test is quite long: 22 minutes. It comes in 2 versions: either visual or auditory. The stimuli are very simple, the hardware used is a special button box connected to the computer which guarantees 1 ms response accuracy.

Unfortunately, the norms for the TOVA-A (auditory version) are only for children (6 to 19 years, N=2551), whereas the norms for the visual version or both for children and adults (N=1596).

The test construction is set up quite nicely, both eliciting commission (quick and target-high-frequent condition) and omission errors (slow and target-low-frequency condition). Also reaction times are measured with every response.

That the T.O.V.A. takes so long to administer is both an advantage and a disadvantage. The advantage is that it surely can detect sustained attention problems, much better than most very short attention tests (like SDMT or TMT).

The disadvantage is that many patients with brain injury will not like this test or even be able to complete it because it is very boring. If half-way a patient falls asleep or gets really annoyed and does not respond anymore, the test really does not measure what it is supposed to measure. Of course, this is an important observation to note but the test results will not be really valid or reliable.

Nevertheless, it would be interesting to have more studies on the T.O.V.A. in which more brain injured patient groups would do the test. Especially, in the more mild brain injured patients would this test detect more than for example the TOSSA? And what would the false-positive rate be? Since that is another disadvantage of the T.O.V.A.

Although the test is suitable for blind and visually impaired people (the auditory version), there are no norms for adults with this auditory version.

Below you can see a picture of the 2 stimuli the TOVA visual uses (source: clinical manual to be downloaded for free on the website www.tovatest.com). Below this stimuli picture you can see the Evaluation Table for the T.O.V.A. A serious shortcoming of the TOVA is that it is currently unavailable for Windows Vista and Windows7. On the website of the TOVA, WWW.TOVATEST.COM, the company states they are working on a newer version of the TOVA that can be used for Windows7 and MAC users.



Go to TOVA test website



The Symbol Digit Modalities Test (SDMT)

The Symbol Digit Modalities Test is a very much used attention test, just like the Trail Making Test. It has several advantages and these explain why it is still used today almost everywhere. First of all, its administration time is very short, about 2 minutes (there always is a time limit of 90 seconds). Secondly, the SDMT is very sensitive to any kind of brain injury and can therefore be used as a good screening instrument for further elaborate neuropsychological testing. Thirdly, it is very easy to administer, you do not have to be a trained psychologist. A nurse, family member, neurologist, everyone can administer this test. Fourth, there are many norms available for this test.

So, are there any disadvantages for this test? Well, perhaps. It seems to be too short so sustained attention can not be measured properly with this attention test. Furthermore, patients with visual inattention or otherwise visually impaired can not be tested reliably with this test. Practice effects are quite serious when administering the test with a very short time interval. The specificity is questionable: it seems the SDMT largely assesses processing speed and this is one thing which is almost universally damaged after any kind of brain injury. In this sense, it is questionable if the SDMT has more to offer than just the conclusion that someone has brain damage (because thát we already know nowadays with CT- or MRI-scans).

The real question is: has the SDMT some predictive value and can it tell us whether attentional problems in daily life will pop up in a patient group or not? Several studies seem to indicate this. However, since brain injured groups normally do have attentional difficulties ánd the SDMT is usually done worse in these patient groups, such conclusions are not really telling us something new. In my clinical experience the SDMT is not really able to differentiate patient groups from each other. Just like some CPT's. And just this discriminating power of an attention test would be very interesting indeed. Due to its simplicity and rather small scoring range (max. 110 points), it seems that its differentiating power is limited.

Overall, just as with the TMT I would use this SDMT as a very short screening test for attentional problems if there are no other more elaborate attention tests around. It is much better than the Mini Mental State Examination which misses a lot of cognitive problems. A further problem with the SDMT is that it is so much used that nearly everyone can get a copy of it and just practice with the test. Here below you can see the Evaluation Table for the SDMT. Next you will find several links where to get the SDMT.



Go to US publisher for the SDMT

Go to UK Publisher for the SDMT



What do you think of this page? Please give your comments

Please note that all fields followed by an asterisk must be filled in.
First Name*
E-Mail Address*
Country*

Please enter the word that you see below.

  



Return from Attention tests to Psychology tests

Go to Attention problems after brain injury

Return to How-psychology-tests-brain-injury



Read here about all disclaimers relevant to this site:

LegalDisclaimers.html


If you want you can follow me on Twitter. I usually tweet in a serious way: whenever there is any news to share, being either about brain injury, emotional problems, abnormal behavior or other morality issues. Click on the link below:

Als u wilt kunt u me ook op Twitter volgen. Ik tweet eigenlijk alleen serieuze zaken, of dat nu nieuws is over hersenletsel, emotionele problemen, abnormaal gedrag of andere ethische zaken. Klik op onderstaande link:

https://twitter.com/fckovacs