handbook of psychology phần 8 pot

440 Assessment of Intellectual Functioning Keith, T. Z. (1990). Confirmatory and hierarchical confirmatory analysis of the Differential Ability Scales. Journal of Psychoed- ucational Assessment, 8, 391–405. Keith, T. Z., & Kranzler, J. H. (1999). The absence of structural fidelity precludes construct validity: Rejoinder to Naglieri on what the Cognitive Assessment System does and does not measure. School Psychology Review, 28, 303–321. Keith, T. Z., Kranzler, J. H., & Flanagan, D. P. (2001). What does the Cognitive Assessment System (CAS) measure? Joint confirmatory factor analysis of the CAS and the Woodcock-Johnson Tests of Cognitive Ability—3rd Edition. School Psychology Review, 30, 89–119. Keith, T. Z., Quirk, K. J., Schartzer, C., & Elliott, C. D. (1999). Con- struct bias in the Differential Ability Scales? Confirmatory and hierarchical factor structure across three ethnic groups. Journal of Psychoeducational Assessment, 17, 249–268. Keith, T. Z., & Witta, E. L. (1997). Hierarchical and cross-age confirmatory factor analysis of the WISC-III: What does it measure? School Psychology Quarterly, 12, 89–107. Kirby, J. R., & Das, J. P. (1978). Information processing and human abilities. Journal of Experimental Psychology, 70, 58–66. Kirk, S. A., McCarthy, J. J., & Kirk, W. D. (1968). Illinois Test of Psycholinguistic Abilities. Urbana: University of Illinois Press. Kline, R. B., Snyder, J., & Castellanos, M. (1996). Lessons from the Kaufman Assessment Battery for Children (K-ABC): Toward a new cognitive assessment model. Psychological Assessment, 8, 7–17. Konold, T. R., Kush, J. C., & Canivez, G. L. (1997). Factor replication of the WISC-III in three independent samples of children receiving special education. Journal of Psychoeducational Assessment, 15, 123–137. Kranzler, J. H., & Keith, T. Z. (1999). Independent confirmatory factor analysis of the Cognitive Assessment System (CAS): What does the CAS measure? School Psychology Review, 28, 117–144. Kranzler, J. H., Keith, T. Z., & Flanagan, D. P. (2000). Independent examination of the factor structure of the Cognitive Assessment System (CAS): Further evidence challenging the construct validity of the CAS. Journal of Psychoeducational Assessment, 18, 143–159. Kuhn, T. (1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press. Kush, J. C., Watkins, M. W., Ward, T. J., Ward, S. B., Canivez, G. L., & Worrell, F. C. (2001). Construct validity of the WISC-III for White and Black students from the WISC-III standardization sample and for Black students referred for psychological evaluation. School Psychology Review, 30, 70–88. Laurent, J., Swerdlik, M., & Ryburn, M. (1992). Review of validity research on the Stanford-Binet Intelligence Scale: Fourth Edi- tion. Psychological Assessment, 4, 102–112. Lubin, B., Wallis, R. R., & Paine, C. (1971). Patterns of psychological test usage in the United States: 1935–1969. Professional Psychology, 2, 70–74. Luckasson, R., Counter, D. L., Polloway, E. A., Reiss, S., Schalock, R. L., Snell, M. E., et al. (1992). Mental retardation: Definition, classifications and systems of support (9th ed.). Washington, DC: American Association on Mental Retardation. Luria, A. R. (1963). Restoration of function after brain injury (B. Haigh, Trans.). New York: Macmillan. Luria, A. R. (1973). The working brain: An introduction to neuropsychology. New York: Basic Books. Luria, A. R. (1980). Higher cortical functions in man. New York: Basic Books. Luria, A. R., & Tsvetkova, L. S. (1990). The neuropsychological analysis of problem solving (A. Mikheyev & S. Mikheyev, Trans.). Orlando, FL: Paul M. Deutsch Press. Maller, S. J., & Ferron, J. (1997). WISC-III factor invariance across deaf and standardization samples. Educational and Psychologi- cal Measurement, 57, 987–994. Marland, S. P. (1972). Education of the gifted and talented: Vol. 1. Report to the Congress of the United States by the U.S. Commis- sioner of Education. Washington, DC: Government Printing Office. Maruish, M. E. (Ed.). (1999). The use of psychological testing for treatment planning and outcomes assessment (2nd ed.). Mahwah, NJ: Erlbaum. Matarazzo, J. D. (1992). Psychological testing and assessment in the 21st century. American Psychologist, 47, 1007–1018. Mather, N., & Jaffe, L. E. (1992). Woodcock-Johnson Psycho- educational Battery—Revised: Recommendations and reports. Brandon, VT: Clinical Psychology Publishing. Mather, N., & Woodcock, R. W. (2001). Woodcock-Johnson III Tests of Cognitive Abilities examiner’s manual: Standard and ex- tended batteries. Itasca, IL: Riverside. Mayer, R. E. (1992). Cognition and instruction: Their historic meeting within educational psychology. Journal of Educational Psychology, 84, 405–412. McGrew, K. S. (1997). Analysis of the major intelligence batteries according to a proposed comprehensive Gf-Gc framework. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contem- porary intellectual assessment: Theories, tests, and issues (pp. 151–180). New York: Guilford Press. McGrew, K. S., & Flanagan, D. P. (1998). The intelligence test desk reference (ITDR): Gf-Gc cross-battery assessment. Needham Heights, MA: Allyn and Bacon. McGrew, K. S., Werder, J. K., & Woodcock, R. W. (1991). WJ-R technical manual. Itasca, IL: Riverside. McGrew, K. S., & Woodcock, R. W. (2001). Technical manual: Woodcock-Johnson III. Itasca, IL: Riverside. McIntosh, D. E. (1999). Identifying at-risk preschoolers: The dis- criminant validity of the Differential Ability Scales. Psychology in the Schools, 36, 1–10. McNemar, Q. (1942). The revision of the Stanford-Binet scale: An analysis of the standardization data. Boston: Houghton Mifflin. Meikamp, J. (1999). Review of the Das-Naglieri Cognitive Assessment System. In. B. S. Plake & J. C. Impara (Eds.), The References 441 supplement to the thirteenth mental measurements yearbook (pp. 75–77). Lincoln, NE: Buros Institute of Mental Measurements. Millon, T. (1999). Reflections on psychosynergy: A model for inte- grating science, theory, classification, assessment, and therapy. Journal of Personality Assessment, 72, 437–456. Millon, T., & Davis, R. D. (1996). Disorders of personality: DSM-IV and beyond (2nd ed.). New York: Wiley. Naglieri, J. A. (1999). Essentials of CAS assessment. New York: Wiley. Naglieri, J. A., & Das, J. P. (1987). Construct and criterion related validity of planning, simultaneous, and successive cognitive processing tasks. Journal of Psychoeducational Assessment, 5, 353–363. Naglieri, J. A., & Das, J. P. (1988). Planning-Arousal-Simultaneous- Successive (PASS): A model for assessment. Journal of School Psychology, 26, 35–48. Naglieri, J. A., & Das, J. P. (1997a). Cognitive Assessment System. Itasca, IL: Riverside. Naglieri, J. A., & Das, J. P. (1997b). Cognitive Assessment System interpretive handbook. Itasca, IL: Riverside. Naglieri, J. A., & Gottling, S. H. (1995). A cognitive education approach to math instruction for the learning disabled: An individual study. Psychological Reports, 76, 1343–1354. Naglieri, J. A., & Gottling, S. H. (1997). Mathematics instruction and PASS cognitive processes: An intervention study. Journal of Learning Disabilities, 30, 513–520. Naglieri, J. A., & Johnson, D. (2000). Effectiveness of a cognitive strategy intervention in improving arithmetic computation based on the PASS theory. Journal of Learning Disabilities, 33, 591– 597. Naglieri, J. A., & Rojahn, J. (2001). Intellectual classification of Black and White children in special education programs using the WISC-III and the Cognitive Assessment System. American Journal on Mental Retardation, 106, 359–367. Paolitto, A. W. (1999). Clinical validation of the Cognitive Assess- ment System with children with ADHD. ADHD Report, 7, 1–5. Pressley, M. P., & Woloshyn, V. (1995). Cognitive strategy instruction that really improves children’s academic performance (2nd ed.). Cambridge, MA: Brookline Books. Prifitera, A., Weiss, L. G., & Saklofske, D. H. (1998). The WISC-III in context. In A. Prifitera & D. Saklofske (Eds.), WISC-III clinical use and interpretation: Scientist-practitioner perspectives (pp. 1–38). San Diego, CA: Academic Press. The Psychological Corporation. (1997). WAIS-III–WMS-III technical manual. San Antonio, TX: Author. Reinehr, R. C. (1992). Review of the Differential Abilities Scales. In J. J. Kramer & J. C. Conoley (Eds.), The eleventh mental measurements yearbook (pp. 282–283). Lincoln, NE: Buros Institute of Mental Measurements. Roid, G. H., & Worrall, W. (1997). Replication of the Wechsler Intelligence Scale for Children—Third edition four-factor model in the Canadian normative sample. Psychological Assessment, 9, 512–515. Ryan, J. J., & Paolo, A. M. (2001). Exploratory factor analysis of the WAIS-III in a mixed patient sample. Archives of Clinical Neuropsychology, 16, 151–156. Saklofske, D. H., Hildebrand, D. K., & Gorsuch, R. L. (2000). Replication of the factor structure of the Wechsler Adult Intelli- gence Scale—Third Edition with a Canadian sample. Psycholog- ical Assessment, 12, 436–439. Sattler, J. M. (1988). Assessment of children (3rd ed.). San Diego, CA: Author. Snow, R. E. (1998). Abilities at aptitudes and achievements in learning situations. In J. J. McArdle & R. W. Woodcock (Eds.), Human cognitive abilities in theory and practice (pp. 93–112). Mahwah, NJ: Erlbaum. Sokal, M. M. (1981). The origins of the Psychological Corporation. Journal of the History of the Behavioral Sciences, 17, 54–67. Solso, R. L., & Hoffman, C. A. (1991). Influence of Soviet scholars. American Psychologist, 46, 251–253. Stanovich, K. E. (1991a). Conceptual and empirical problems with discrepancy definitions of reading disability. Learning Disability Quarterly, 14, 269–280. Stanovich, K. E. (1991b). Discrepancy definitions of reading disability: Has intelligence led us astray? Reading Research Quarterly, 26, 7–29. Tallal, P. (2000, March 14). The science of literacy: From the labo- ratory to the classroom. Proceedings of the National Academy of Sciences of the United States of America, 97, 2402–2404. Terman, L. M. (1916). The measurement of intelligence. Boston: Houghton Mifflin. Terman, L. M. (1925). Genetic studies of genius: Vol. 1. Mental and physical traits of a thousand gifted children. Stanford, CA: Stanford University Press. Terman, L. M., & Merrill, M. A. (1937). Measuring intelligence: A guide to the administration of the new revised Stanford-Binet tests of intelligence. Boston: Houghton Mifflin. Terman, L. M., & Merrill, M. A. (1960). Stanford-Binet Intelligence Scale: Manual for the third revision. Form L-M. Boston: Houghton Mifflin. Terman, L. M., & Merrill, M. A. (1973). Stanford-Binet Intelligence Scale: 1973 norms edition. Boston: Houghton Mifflin. Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986).The Stanford- Binet intelligence scale: Fourth edition. Itasca, IL: Riverside. Thorndike, R. M. (1990). Would the real factors of the Stanford- Binet Fourth Edition please come forward? Journal of Psycho- educational Assessment, 8, 412–435. Thorndike, R. M., & Lohman, D. F. (1990). A century of ability testing. Itasca, IL: Riverside. Thurstone, L. L. (1938). Primary mental abilities. Chicago: Univer- sity of Chicago Press. Tupa, D. J., Wright, M. O., & Fristad, M. A. (1997). Confirmatory factor analysis of the WISC-III with child psychiatric inpatients. Psychological Assessment, 9, 302–306. 442 Assessment of Intellectual Functioning Undheim, J. O. (1981). On intelligence: II. A neo-Spearman model to replace Cattell’s theory of fluid and crystallized intelligence. Scandinavian Journal of Psychology, 22, 181–187. Wasserman, J. D., & Becker, K. A. (2000, August). Racial and ethnic group mean score differences on intelligence tests. In J. A. Naglieri (Chair), Making assessment more fair—taking verbal and achievement out of ability tests. Symposium conducted at the annual meeting of the American Psychological Association, Washington, DC. Wasserman, J. D., Paolitto,A. M.,& Becker, K.A. (1999, November). Clinical application of the Das-Naglieri Cognitive Assessment System (CAS) with children diagnosed with Attention-Deficit/ Hyperactivity Disorders. Paper presented at the annual meeting of the National Academy of Neuropsychology, San Antonio, TX. Wechsler, D. (1925). On the specificity of emotional reactions. Journal of Psychology, 36, 424–426. Wechsler, D. (1939). The measurement of adult intelligence. Baltimore: Williams and Wilkins. Wechsler, D. (1949). Wechsler Intelligence Scale for Children manual. New York: The Psychological Corporation. Wechsler, D. (1955). Wechsler Adult Intelligence Scale manual. New York: The Psychological Corporation. Wechsler, D. (1958). Intelligence et fonction cérébrale. Revue de Psychologie Appliquee, 8, 143–147. Wechsler, D. (1961). Intelligence, memory, and the aging process. In P. Hoch & J. Zubin (Eds.), Psychopathology of aging (pp. 152– 159). New York: Grune and Stratton. Wechsler, D. (1967). Wechsler Preschool and Primary Scale of Intelligence. New York: The Psychological Corporation. Wechsler, D. (1974). Cognitive, conative, and non-intellective intelligence. In D. Wechsler (Ed.), Selected papers of David Wechsler (pp. 39–48). New York: Academic Press. (Original work pub- lished 1950) Wechsler, D. (Speaker). (1976, January). Unpublished interview with David Wechsler [Transcript]. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1991). Wechsler Intelligence Scale for Children— Third Edition manual. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1997). Wechsler Adult Intelligence Scale—Third Edition: Administration and scoring manual. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1999). Wechsler Abbreviated Scale of Intelligence. San Antonio, TX: The Psychological Corporation. Weider, A. (Speaker). (1995, August). An interview with Arthur Weider. Unpublished manuscript. (Available from John D. Wasserman, George Mason University, 4400 University Drive, MSN 2C6, Fairfax, Virginia 22030-4444) Werner, H. (1948). Comparative psychology of mental development. New York: International Universities Press. Witt, J. C., & Gresham, F. M. (1985). Review of the Wechsler Intelligence Scale for Children—Revised. In J. V. Mitchell (Ed.), Ninth mental measurements yearbook (pp. 1716–1719). Lincoln: University of Nebraska Press. Wolf, T. H. (1973). Alfred Binet. Chicago: University of Chicago Press. Woodcock, R. W. (1998). The WJ-R and Batería-R in neuropsychological assessment (Research Rep. No. 1). Itasca, IL: Riverside. Woodcock, R. W., McGrew, K. S., & Mather, N. (2001a). Woodcock- Johnson III Tests of Cognitive Abilities. Itasca, IL: Riverside. Woodcock, R. W., McGrew, K. S., & Mather, N. (2001b). Woodcock- Johnson III Tests of Achievement. Itasca, IL: Riverside. World Health Organization. (1992). The ICD-10 classification of mental and behavioral disorders: Clinical descriptions and diagnostic guidelines. Geneva, Switzerland: Author. CHAPTER 19 Assessment of Neuropsychological Functioning KENNETH PODELL, PHILIP A. DEFINA, PAUL BARRETT, ANNEMARIE MCCULLEN, AND ELKHONON GOLDBERG 443 BRIEF HISTORY OF NEUROPSYCHOLOGY 444 DEVELOPMENTS IN CLINICAL APPLICATION 445 Sports-Related Concussion Assessment 445 Forensic Neuropsychology 446 ISSUES IN NEUROPSYCHOLOGICAL ASSESSMENT 450 Computerized Assessment 451 Paper-and-Pencil Testing 452 RECENT ADVANCES IN PSYCHOMETRIC APPLICATIONS 452 Reliability of Change Indexes 453 Receiver Operating Curves 455 Positive and Negative Predictive Power 456 NEUROIMAGING 456 Language 457 Executive Control and Memory 458 Schizophrenia 458 Affective Disorders 458 Dementia 459 Transcranial Magnetic Stimulation 459 FUTURE DIRECTIONS 459 REFERENCES 460 In scientific fields, both external and internal forces create, change, and shape that field. Neuropsychology is no different; in fact, this field is in the midst of some of the largest growth, advancements, and changes it has ever undergone. Although it is a relatively young science, neuropsychology has been influenced by many factors that have helped to develop and shape the field, both experimentally and clinically. For example, there was a large amount of clinical information obtained from studying World War II survivors who had pen- etrating missile injuries to the head. Not only did the presence (although not a pleasant event) of war help contribute to our knowledge base, but the use of penicillin in the battlefield also allowed these individuals to survive in the first place in order to be available for study later. Presently, various internal and external forces have shaped researchers and clinicians in the field of neuropsychology. In- ternal forces include cutting-edge neuroimaging technology, such as functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG), the development and application of more sophisticated statistical techniques, and the expansion into new clinical areas (such as sports-related concussion). Similarly, one of the strongest external forces influ- encing and molding the future of neuropsychology (for the better and the worse) is economics. The current situation in health care has had a particular impact on the development of neuropsychology—especially as a clinical discipline. Some of the changes have been good and others have been not so good.As for the latter, the rather dismal prospect of finding an adequate, well-paying job as a neuropsychologist is influenc- ing the career choices of many bright and talented individuals and causing them to seriously consider—and probably choose—other professions. Similarly, numerous graduate and postgraduate training sites have closed due to lack of funding or budget cuts. Paradoxically, there has been a slight increase in the number of students entering graduate psychology programs in general. This situation has led to a glut of (quality) students who cannot find adequate training; moreover, even if they do find such training, many cannot find an acceptable position. However, the shrinking health care dollar is causing neuropsychologists to rethink how they administer neuropsychological services (a much-needed self-check) and is also causing neuropsychologists to be creative and develop or enter new venues for generating revenue. Probably the best example of new revenue opportunities is the explosion of forensic neuropsychology. More and more neuropsychologists have recognized the lucrative area of forensic practice. Although some truly see forensic neuropsychology as a science, others see it as a way of increasing revenue. This situation has caused exponential growth in clinical activity, which has in turn stimulated the critical research required to support this area of neuropsychological practice from a scientific perspective. This research in turn improves 444 Assessment of Neuropsychological Functioning its clinical application and the reputations of neuropsychologists (and probably psychology as a whole) in the forensic arena. The changing face of health care and recent advancements in technology have stimulated the growth of neuropsychology into a more scientific and clinically diverse subspecialty of psychology. However, the field still faces several challenges in the areas of training for and delivery of health care. This chapter focuses on some of these innovative issues in neuropsychology. Our attempt is to introduce these advances and explain some the basic components of each. We focus on how these new developments and progress in neuropsychology advance experimental and clinical neuropsychology, how they contribute to our knowledge of brain-behavior relationships and treatment of patients, and how they are shaping the field of neuropsychology as a whole. Before we discuss the current new developments in neuropsychology, we provide a brief review of the history of neuropsychology as a backdrop and perhaps—at least heuristically—as a context for understanding some of the more recent advancements. BRIEF HISTORY OF NEUROPSYCHOLOGY Neuropsychology is a relatively new field that traces its roots back to at least the late 1800s. It is a hybrid discipline repre- senting the confluence of several fields of study: neurology and psychology, neuroanatomy and neurophysiology, and neurochemistry and neuropharmacology (Benton, 1988). Its early status was dependent upon the status of its contributory disciplines. Modern clinical neuropsychology grew out of— or was at least strongly influenced by—clinical neurology (Bradshaw & Mattingley, 1995). Neuropsychology, although it is closely related to behavioral neurology, distinguishes itself from both neuropsychiatry and behavioral neurology by its ultimate focus on clarifying the mechanisms underlying both abnormal and normal behavior. Neuropsychiatry and behavioral neurology focus on the diagnosis and treatment of abnormal behavior only (Bradshaw & Mattingley, 1995). Modern neuropsychology is based upon data from both brain-injured and healthy individuals. In addition to its clinical neurology parentage, neuropsychology makes use of more than 100 years of research in experimental psychology to help explain the patterns of disordered percep- tual, cognitive, and motor processes seen in patients with neurological damage (Bradshaw & Mattingley, 1995). The term neuropsychology first began to be used in the 1930s and 1940s (Benton, 1987). According to Bruce (1985, cited in Benton, 1988), the term began gaining currency in the 1950s when it displaced older terms, such as psychoneu- rology and brain pathology. The discipline of human neuropsychology was established over a course of about 15 years, roughly between 1950 and 1965 (Benton, 1987). Prior to that time, experimental neuropsychology was largely involved in animal model research. In fact, there was a period, mostly from the 1950s through the early 1970s, during which there was prolific research and understanding in the basic aspects of brain-behavior relationship, mostly through animal model research. Neuropsychology’s status as a discipline was first signaled by the appearance of two international neuropsychological journals between 1963 and 1964—Neuropsychologia founded by Henry Hecaen and Cortex founded by Ennio De Renzi. The first association specifically oriented toward neuropsychology was the International Neuropsychological Society, which was founded in the late 1960s by a group organized by Louis Costa. In the late 1970s, Louis Costa is credited as the individual who gave birth to clinical neuropsychology as a distinct professional specialty and gave it legiti- macy as a subspecialty in psychology (at least in North America). Professor Costa did this by founding (with Byron Rourke) the Journal of Clinical Neuropsychology and by developing Division 40 of the APA, the Division of Clinical Neuropsychology. Modern neuropsychology began by studying the localization of brain function and cognitive and behavioral changes following large lesions to the brain. These advances are perhaps best illustrated by the work of Broca and Wernicke in establishing the major speech areas in the left hemisphere. Some of the seminal researchers of the late 1800s up through the mid-1960s include Broca, Wernicke, Kliest, Goldstein, Henry Hecaen, Denny-Brown, Karl Pribram, Mortimer Mishkin, Hans Lukas-Teuber, Norman Geschwind, Ward Halstead, Ralph Reitan, A. L. Benton, and many others. One individual who requires special attention is A. R. Luria, a Russian psychologist whose contribution to neuropsychology was actual only part of his total contribution to psychology as a whole. Luria, a neurologist trained in psychoanalysis, did extensive research in understanding the cognitive and behavioral alterations following lesions to the brain. Higher Corti- cal Functions in Man (1966/1980), one of several books written by Luria, is considered one of the seminal textbooks on localization neuropsychology. In fact, Luria’s name is virtually synonymous with executive control (i.e., prefrontal functions). Luria was perhaps one of the first to describe in detail the qualitative features of the behavioral and cognitive deficits associated with various lesions of the brain. To give an overview of how neuropsychological research and techniques progressed and evolved over time, one needs to understand the contribution of three general components or phases. The first phase started with efforts to understand brain-behavior relationships by studying the cognitive and Developments in Clinical Application 445 behavioral deficits found following focal lesions. Deficits found in these individuals were used to infer normal functions. For example, a large left inferior frontal lesion (Broca’s area) caused a deficit in speech output. Thus, the area was in- ferred to be important in generating speech output. One of the first and most famous cases used in this manner was Phineas Gage (Damasio, Grabowski, Frank, Galaburda, & Damasio, 1994; Harlow, 1868). Gage sustained a large lesion in his prefrontal cortex (primarily orbito-frontal) when a tamping rod accidentally misfired and entered into his head from below his chin and exited through the top of his skull. Gage went on to develop what is now referred to as an orbito-frontal or pseudo-psychopathic syndrome. Studying focal, localized lesions in humans has been going on for over 150 years and has become particularly refined over the past 30–40 years. Although this line of study has been ex- tensively used in humans, there is a long and storied history of animal research that has contributed immensely to the understanding of neuropsychology. In fact, animal neuropsychology was a major force in the contribution to understanding brain- behavior relationships from the 1940s through the 1970s. The second general phase to contribute to neuropsychology was the study of cytoarchitechtonics and the attempt to better understand brain-behavior relationships as they related to microscopic neuroanatomy (see Barbas & Pandya, 1989). The third and current phase entails in vivo neuroimaging of healthy volunteers. Techniques such as functional magnetic resonance imaging (fMRI), positron emission tomography (PET), regional cerebral blood flow (rCBF), single photon emitting computerized tomography (SPECT), evoked potentials (EP), and MEG have taken neuropsychology to a new level of understanding brain-behavior relationships by allow- ing us to study, in vivo, behavior in healthy individuals rather than inferring it from the deficits demonstrated by brain- injured individuals (see Goldberg, 2001, for a more detailed description of this method). So far, the evidence coming from fMRI research is generally confirming our findings from studies of lesions, but it is also revealing new and exciting (and sometimes counterintuitive) findings. The following sections of this chapter discuss various developments and advances in neuropsychology, both clinical and experimental. We attempt to address some of the more current issues and advances—as well as problems—facing neuropsychology today. DEVELOPMENTS IN CLINICAL APPLICATION Clinical neuropsychology is in the midst of rapid and (in our opinion) historical change. Most of the change is positive, but some may not be so positive. In part, some of these changes are reactions to the shrinking health care dollar and its effect on psychology in general. Neuropsychologists have adapted and developed unique and novel responses to the lack of funding for neuropsychology and the reduction— and even elimination—of health care insurance for tradi- tional mental health services. The two major clinical services to arise from this challenge are the neuropsychologist’s involvement in the new and increasingly popular sports-related concussion assessment and return-to-play decision making and the phenomenal expansion of forensic neuropsychology. Sports-Related Concussion Assessment Although their involvement was virtually nonexistent 10 years ago, neuropsychologists are becoming ever more important in helping sport teams assess and manage sports- related concussions. One of the most exciting aspects is that this development is taking place at every level of competi- tion: international, profession, collegiate, and high school. Neuropsychologists are becoming integral participants in the care of athletes who sustain concussions. The neuropsychologist’s primary role is to diagnose the presence of concussion effect (e.g., cognitive deficits and symptomatology) and to use this information to help the team trainer and physicians determine when an athlete has recovered fully from the concussion and is able to return to play. One reason that this role has become so important is the amount of potential money involved. With the advent of multimillion dollar contracts, it becomes critical that players are cared for properly. See Echemendia and Julian (2001) for an extensive overview of the entire topic. Sports-related concussions are no longer considered trivial injuries. Large epidemiological studies by Powell and others (Barth et al., 1989; Guskiewicz, Weaver, Padua, & Garrett, 2000; Powell, 1999; Powell & Barber-Foss, 1999) have shown that 5–10% of football players are concussed each year and approximately 5% from various others sports (e.g., soccer and field hockey). In American high school football alone, that would indicate approximately 25,500 concussions per season (a base rate of 2,460 concussions per 100,000 high school football players). Maroon et al. (2000) have shown that the rate of concussion in college athletes has decreased from about 10% per season (Barth et al.,1989)toabout4%, probably due to rule changes and new and improved equipment (Powell, 1999). Given these base rates, it is clear that there is a need for better diagnosis, management, and treatment. This area is exactly where clinical neuropsychology has played an integral part and is rapidly developing as the standard for measuring the effects of sports-related concussions and return-to-play issues (Aubry et al., 2002). 446 Assessment of Neuropsychological Functioning Besides opening a new area of clinical services and clien- tele, neuropsychology’s role in sports-related concussion assessment and management has done a tremendous job of ex- posing various areas of clinical care and professional services (e.g., athletic trainers, physicians, and parents) to the expertise of clinical neuropsychology. For example, because of neuropsychology’s role in sports-related concussion assessment, neuropsychologists arenowpresenting to and workingwiththe sports-medicine community, athletic trainers, and other physicians who would not normally have been aware of or utilized this service. Neuropsychologists are publishing in journals that are not typical for them (e.g., Journal of The American Medical Association, Journal of Sports Medicine, Physician and Sports Medicine), broadening and increasing neuropsychology’s ex- posure and prominence to an even greater degree. One neuropsychologist who has led both clinical and experimental neuropsychologists into the area of sports-related concussion is Mark R. Lovell. Through his involvement with concussion committees for both the National Football League (NFL) and National HockeyLeague(NHL),neuropsychological testing is mandatory in the NHL, andapproximately 80% of the NFL teams use neuropsychological testing. Addi- tionally, colleges, high schools, and amateur and professional sports teams worldwide have concussion safety programs in which players undergo baseline testing during the preseason. If concussed during the season, a player is retested, and his or her results are compared to their baseline. This comparison allows for direct intra-individual changes, and the neuropsychologist can use the differences (or lack of differences) in scores to help the team with return-to-play decision making. Lovell and Collins (1998) have demonstrated little change (outside of practice effect) in preseason versus postseason testing in varsity college football players. However, Collins et al. (1999) demonstrated that sports-related concussion in college football players caused significant decrement in memory and attention-concentration (consistent with the initial seminal studies of Barth (see Barth et al., 1989). In addition, Collins et al. (1999) found that after a concussion, those with a prior history of concussion performed more poorly than did those without a prior history of concussion. Moreover, they found that a history of learning disability was a risk factor for greater cognitive impairment following a concussion. The standard protocol for performing neuropsychological evaluations in sports-related concussions is to use a serial assessment approach starting with a baseline (e.g., preseason or prior to any concussions) neuropsychological evaluation. Typically, these computerized neuropsychological batteries are relatively short (approximately 20–25 minutes, focusing on working memory, complex attention-concentration, re- action time, and anterograde memory). Although there are variations across institutions, typically a follow-up neuropsychological evaluation is performed within 24 hours of the concussion and is followed by additional postconcussion evaluations at Day 3, Day 5, and Day 7. After this point, if the athlete is still concussed, additional testing can be done weekly or even every other week. Various programs differ from this pattern, but the general idea is to perform a baseline evaluation, an initial postconcussion assessment, and additional follow-up assessments to document recovery of function and help with return-to-play decision making. The role of neuropsychologists in sports-related concussions has expanded the understanding of concussions and their effects on and recovery of cognition and symptomatology. It has also increased concussion awareness in the general public—particularly parents—and has demystified some of the misconceptions about concussion and placed it alongside other common injuries(e.g.,sprains) in sports.Neuropsychologyhas also improved how athletes’ concussions are diagnosed, managed, and treated (see Collins & Hawn, 2002; Grindel, Lovell, & Collins, 2001). Today, concussions are no longer ignored; rather, they are diagnosed and treated as the injury they are. Be- cause of this enlightened attitude and improved awareness and diagnostic accuracy, athletes—especially younger ones—are more accurately (and frequently) diagnosed and treated. This practice allows for appropriate treatment and decreased risk of greater injury by sustaining a second concussion while still concussed from the first one; this may help to reduce greater long-term, braininjuryand reduce thechanceof second-impact syndrome—a rare but often fatal event (Cantu, 1998). Clearly, neuropsychologists’ leadership role in this area has had and will continue to have a beneficial effect on these athletes. Forensic Neuropsychology Probably the single area within clinical neuropsychology that has seen the greatest growth explosion is forensic neuropsychology; this is due partly to the greater demand by the legal system for expert testimony that can identify neuropsychological deficits (Nies & Sweet, 1994) and also to the potentially lucrative income associated with forensic-related activity. The research related to this area has been explosive in the terms of the quality and wealth of information obtained so far. In just a few short years, the clinical techniques studied and developed have greatly enhanced neuropsychologists’ ability to practice in this area. The one area within forensic neuropsychology that has seen the greatest growth clinically is civil litigation—usually traumatic brain injuries suffered in motor vehicle accidents (Ruff & Richardson, 1999). In fact, motor vehicle accidents account for roughly half of the estimated 2 million traumatic Developments in Clinical Application 447 brain injuries yearly (Krauss & McArthur, 1996). The following is a brief introduction into the area of clinical forensic neuropsychological assessment; we use mild traumatic brain injury (MTBI) as a model. The crux of neuropsychology’s involvement in forensic activity is to find evidence for or refute (through test performance) the presence of central nervous system (CNS) dysfunction. Often, standard neurological testing (such as CT or MRI of the brain and EEG) is insensitive to the subtle deficits of MTBI while neuropsychological deficits are present (Bigler & Snyder, 1995; Gronwall, 1991). Typically, consid- erable monetary compensation is sought in these cases, which augments the importance of the neuropsychological evaluation. Well over half of TBI cases are mild in nature (Ruff & Richardson, 1999). Although most people with MTBI fully recover, a minority of individuals (ranging from estimates of 7–8% to 10–20%) experience more long-term effects (Alexander, 1995; L. M. Binder, 1997). The constella- tion of subjective complaints often reported by individuals with MTBI has been termed the postconcussion syndrome (PCS). The most commonly reported symptoms include irri- tability, fatigue, difficulty concentrating, memory deficits, headache, dizziness, blurred vision, photophobia, ringing of the ears, and disinhibition and loss of temper (Lees-Haley & Brown, 1993). There has been a great deal of debate concerning persistent PCS; many suggest that it is psychologi- cally rather than neurologically based or that patients are exaggerating or malingering symptoms in order to receive compensation (Mittenberg & Strauman, 2000; Youngjohn, Burrows, & Erdal, 1995). Because there is no litmus test to determine the presence of residual MTBI, it can become very difficult to differentiate those who truly have residual deficits from those without deficits who are exploiting their past (recovered) injury for monetary compensation solely based upon self-reported symptomatology. In fact, the base rates of self-reported symptomatology cannot distinguish between groups with verified MTBI from healthy controls or from those seeking compensation for non-TBI-related injuries (Lees-Haley & Brown 1993; Lees-Haley, Fox, & Courtney, 2001). Therefore, when this difficulty is combined with the lack of any neuroimaging evidence, the neuropsychologist becomes the key to determining and proving the presence of residual MTBI. From a forensic perspective, the critical question is Can a neuropsychologist, who applies various neuropsychological and psychological tests, differentiate between those who truly have residual cognitive or emotional deficits from those who are malingering, exaggerating, or even presenting with a somatoform or factitious disorder? The task of detecting suboptimal performance carries a great responsibility because the decision can determine whether services will be provided for a patient or whether the patient will receive large monetary compensation (Davies et al., 1997; Nies & Sweet, 1994). Although the rate of malingering is unknown, estimates range from 7.5–15% (Trueblood & Schmidt, 1993) to 18–33% (L. M. Binder, 1993). However, it is generally believed that the incidence of exaggeration of symptoms is higher than that of actual malingering (Resnick, 1988). There are several ways in which neuropsychological testing can determine whether the test score actually represents a true cognitive deficit—or alternatively, whether it might indicate symptom exaggeration or even malingering. Some of the procedures or tests are more sophisticated and sensitive than others. First, and foremost, the deficits (one of the most common complaints is anterograde memory impairment) must be consistent with the nature of the injury. For example, one cannot have a dense amnesia if the traumatic brain injury wasonly mild. Similarly, the deficit patterns must make neuropsychological sense and conform to known brain-behavior relationships. For example, an individual complaining of worsening memory over time after a MTBI is not consistent with what is known about TBIs (that they are static events from which one can only recover—not worsen over time). Another method that neuropsychologists use to detect true versus malingered or exaggerated deficits is through the use of tests specifically designed to test for suboptimal performance. Test development in the area of the assessment of malingering has flourished over the past several years, and significant strides have been made (see Iverson & Binder, 2000; Sweet, 1999, for comprehensive reviews). The sophistication of the tests developed and refined has improved greatly over the past few years; this is important because lawyers and the clients are becoming more sophisticated and aware of these tests. In fact, plaintiff attorneys have been known to coach their clients about these tests and prepare them for any independent neuropsychological evaluation they may undergo for the defense. Such practices have led some researchers to not publish some of their normative data in journal articles in order to protect the integrity and use of the tests (see Millis, Putnam, Adams, & Ricker, 1995; Sweet et al., 2000). Forced-Choice Recognition Tests There are a number of strategies typically employed to identify malingered performance. The first involves the use of a two- alternative forced-choice (e.g., five-digit numbers) method (Hiscock & Hiscock, 1989). When these tests were first designed and employed in clinical assessments, simple bino- mial distribution theory was applied to interpret performance. In two-choice recognition tests, the probability of responding 448 Assessment of Neuropsychological Functioning correctly on all items by chance alone (i.e., guessing) is 50%. Scores significantly below that predicted by chance are un- likely by chance alone; therefore, such performance is as- sumed to be the result of deliberate selection of incorrect answers, which is suggestive of exaggeration or malingering of deficits. Without any knowledge of the stimulus (as would occur in the case of amnesia) the patient should answer approximately 50% of the items correctly; a score significantly below 50% suggests that the patient knew the correct answer but deliberately chose the incorrect response. More recently, research has shown that patients with more severe head injury and genuine memory loss typically perform well above the chance level on two-alternative forced- choice tests (L. M. Binder & Pankrantz, 1987; L. M. Binder & Willis, 1991; Guilmette, Hart & Giuliano, 1993; Prigatano & Amin 1993). Prigatano and Amin (1993) demonstrated that the performance of postconcussive patients and those with un- equivocal history of cerebral dysfunction averaged over 99% correct compared to a group of suspected malingerers who averaged only 73.8% correct. Guilmette et al. (1993) demonstrated that a group of brain-injured and psychiatric patients obtained almost perfect scores, whereas simulators obtained scores that were significantly lower. However, only 34% of the simulators obtained scores below chance level. These findings suggest that the development of cutoff scores is nec- essary in order to improve the sensitivity of this method. A 90% cutoff score has typically been established based on the large body of evidence, which suggests that those with genuine brain injury typically perform above this level on digit recognition procedures. A number of forced-choice tests have been developed and are briefly reviewed here; they include the Portland Digit Recognition Test (PDRT; L. M. Binder, 1993), the Victoria Symptom Validity Test (VSVT; Slick, Hopp, & Strauss, 1998), the Recognition Memory Test (RMT; Warrington, 1984), the Validity Indicator Profile (VIP; Frederick, 1997), the Computerized Assessment of Response Bias (CARB; Allen, Conder, Green, & Cox, 1998), and the Test of Memory Malingering (TOMM; Tombaugh, 1996). Hiscock and Hiscock (1989) developed a test requiring individuals to choose which of two 5-digit numbers was the same as a number seen prior to a brief delay. The five-digit number is presented on a card for 5 s followed by a delay period, after which another card is presented with the correct choice and a foil. The foil items differed from the target item by two or more digits, including either the first or last digit. A total of 72 items are administered. These 72 items are divided into three blocks with either a 5-s, 10-s, or 15-s delay. The examiner tells the patient that the test is difficult for those with memory deficits and after the first and second blocks, that the test will be more difficult because of the increasing delay period. In an attempt to improve the test’s sensitivity in detecting suboptimal performance, L. M. Binder (1993) refined the Hiscock and Hiscock procedure by developing the PDRT. It is a digit recognition taskwiththreeblocks of items differentiated by the length of delay between target presentation and response. Binder’s version differed from that of Hiscock and Hiscock in a number of ways such as auditory presentation of the target item followed by visual presentation of the target and distractor item and increased delay periods between presentation and response (5 s, 15 s, and 30 s). Research suggests that difficult items (30-s delay) are more sensitive to malingered performance than are easy items (Hiscock & Hiscock, 1989). In addition, it has an intervening activity, which requires that the patient count backwards during the delay period. This activity makes the task appear even more difficult to the patient. L. M. Binder (1992) found that non-compensation-seeking (NCS) patients with well-documented brain injury performed better than did both mild head trauma and compensation- seeking (CS) patients with well-documented brain injury on the PDRT, but that the CS brain-injured group’s performance was superior to that of the mild head injury group on other tests. Binder (1993) administered the PDRT and the Rey Auditory Verbal Learning Test (RAVLT) to two groups of CS patients, including a mild head injury and well documented brain injury group and a group of NCS brain dysfunction patients. His results showed that patients with financial incen- tives were significantly more impaired on the PDRT but performed as well as the NCS groups did on the RAVLT. Binder and Willis (1991) demonstrated that those with affective disorders performed at a level similar to that of a group of NCS brain dysfunction patients, which suggests that the performance of the CS groups in this study was not the result of depression. Binder concluded that poor PDRT performance significant enough to raise concern about malingering is probably not caused by either verbal memory deficits or affective disorders, and the PDRT is therefore a useful tool for the de- tection of exaggerated memory deficits. Vickery, Berry, Hanlon-Inman, Harris, and Orey (2001) performed a meta-analysis ofanumberof malingering procedures. The PDRT had high specificity rates at the level of individual classification (97.3%) but only moderate sensitivity (43.3%) because of a high number of performances that were poor but above chance level (Rose, Hall, & Szalda-Petree, 1995). One suggestion to improve the PDRT has been to measure the response latency (Brandt, 1988). It is expected that to purposely respond incorrectly to the test items require increased information processing time. Brandt usedacomputerizedversionof the Developments in Clinical Application 449 test and found that when response latency and total number correct were used in combination, 32% fewer classification errors were made and overall hit rate increased from 72% to 81%. It was also demonstrated that coaching affected the total number correct in that all subjects scored above the cutoff; however, there was no difference in response latency. Slick (Slick et al., 1998) also modified the Hiscock and Hiscock procedure. First, administration time was decreased by decreasing the number of items from 72 to 48, which are presented in three blocks of 16 items each. The delay period is increased in each block from 5 to 10 to 15 s. Item difficulty was manipulated by making items appear more difficult (i.e., similarity between the correct item and foils). Strauss et al. (1999) administered the VSVT to simulators and controls three times over a 3-week period. Simulators performed less consistently over the three administrations. Results demonstrated that on the hard items, a deviation of 3 points differentiated the control and malingering groups with 95% probability. A deviation of 1 point differentiated the groups with 95% probability on the easy items. Eighty-eight percent of the control group and 89% of the malingering group were correctly classified. On the VSVT, both response latency and number correct are recorded. Slick, Hopp, Strauss, Hunter, and Pinch (1994) found that those who produced invalid profiles had significantly longer response latencies, again suggesting the useful- ness of this measure. In addition, a new third category of classification is added. Performance below chance is still labeled invalid and performance significantly above chance is still labeled valid. The third category, questionable, consists of scores that fall within the remaining 90% confidence inter- val of chance performance. The three-category classification system has shown high specificity and good sensitivity (Slick et al., 1994). The VIP (Frederick, 1997) is a computerized, two- alternative forced-choice procedure that incorporates a fourfold classification system based on two test-taking charac- teristics: motivation (to excel or fail) and effort (high or low). The combination of the concepts of motivation and effort gen- erate four classification schemes; compliant (high effort and motivation), careless (high motivation to perform well but low effort to correctly respond), irrelevant (low effort when motivated to perform poorly), and malingering (high effort and motivation to perform poorly). Only the compliant profile is considered valid. The test contains both verbal (20 min) and nonverbal (30 min) subtests. The nonverbal subtest is a 100-item progressive matrix test modified from the Test of Nonverbal Intelligence (TONI; Brown, Sherbenou, & Johnson, 1982). The verbal subtest contains 78 two-alternative word knowledge items. The VIP uses a performance curve analysis. The performance curve shows the average performance of the test taker across an increasingly difficult range of test items. Compliant responding results in a curve that starts at about 100% and remains at that level until the test taker reaches his or her ceiling of ability (as items increase in difficulty), at which time the curve goes through a period of transition until it results in about 50% correct performance (or random responding). As a result, performance curves for compliant test takers should be similar in shape regardless of ability levels. Standard Clinical Tests Although there have been several tests developed specifically to assess for malingering, several researchers have taken standard clinical tests and studied their ability to distinguish motivated from possibly malingering-exaggerating (or those acting as malingerers) and TBI patients. Some of the more commonly used tests today include the Wechsler Memory Scale–III (Scott Killgore & DellaPietra, 2000) the California Verbal Learning Test (Baker, Donders, & Thompson, 2000; Millis et al., 1995; Sweet et al., 2000), and Wisconsin Card Sorting Test (Suhr & Boyer 1999). Cutoff scores or patterns of performance have been developed that can be used to evaluate those with documented mild TBI. The development of tests used to assess for suboptimal effort has greatly enhanced the neuropsychologist’s ability to accurately detect malingering and thus sincere performance as well. The sophistication of these tests has undergone tremendous and rapid expansion over the past few years. However, a few interesting points should be made re- garding the development of normative data for these tests as well as the appropriate application of these tests. First, it is almost impossible to truly find a known malingering group. By definition, these individuals are trying to fake brain impairment and thus do not admit to malingering. Therefore, the research used in developing these tasks and their normative data has primarily used groups trained to fake brain impairment or has compared groups of TBI patients matched for severity of injury but differing in CS status (e.g., CS vs. NCS). Although these substitutes are adequate and quite frankly the best that can be achieved, it does not allow for the assessment of a group of clearly defined true malingerers. All of the aforementioned tests used to help determine level of motivation depend upon a conscious response by the subject. It is this response that is under the individual’s control. It is up to the neuropsychologist to determine whether the response actually represents the true ability of the individual or whether it was suboptimal (i.e., possibly malingered or [...]... computed tomographic scanning Effects of gender, age, and stage of illness Archives of General Psychiatry, 47, 10 08 1015 Atkinson, L (1991) Three standard errors of measurement and the Wechsler Memory Scale—Revised Psychological Assessment, 3, 136 –1 38 Benton, A (1 988 ) Neuropsychology: Past, present and future In F Boller & J Grafman (Eds.), Handbook of clinical neuropsychology (Vol 1, pp 3–27) Amsterdam:... cutoff chosen is based on the particular use of a test and the user’s assessment of the relative costs of different types of erroneous decisions The sensitivity of a cutoff score refers to the proportion of results considered positive relative to the proportion of the sample that is actually part of the positive distribution In other words, increasing sensitivity results in an increasing number of. .. does so at the expense of also increasing the number of false positives Conversely, the specificity of a cutoff score refers to the proportion of results considered negative relative to the proportion of the sample that is actually part of the negative distribution In other words, increasing specificity reduces the number of false positives at the expense of also increasing the number of false negatives... Johnson, S K (1 982 ) Test of Nonverbal Intelligence Austin, TX: ProEd 462 Assessment of Neuropsychological Functioning Buckner, R L., Logan, J., Donaldson, D I., & Wheeler, M E (2000) Cognitive neuroscience of episodic memory encoding Acta Psychologica, 105, 127–139 of neuropsychological test performance and event-related potentials Journal of Clinical and Experimental Neuropsychology, 21, 86 6 87 9 Cabeza,... a discussion of a definition of interests, offering a working definition of the nature of interests Many of the major interest-assessment measures, some of them among the longest-lived and most psychometrically sophisticated measures in psychology, are then presented and briefly discussed General findings and themes on the reliability and validity of interests are reviewed along with issues of group differences... studies of the stability of interests to date have been those of Strong (1938a, 1938b, 1951, 1952), who persistently found the temporal stability of the interest patterns of men and women to be among the most stable of all psychological variables More recently, Swanson (1999) discussed in detail issues concerning the stability of occupational interests and concludes that, although a small proportion of. .. all of health care, of the potential brain drain that managed care and the shrinking health care dollar have on attracting (or should we say steering away) talented young individuals to more lucrative professions The future of neuropsychology is still blossoming with many more exciting developments waiting to happen However, as in all other health care fields, neuropsychology is also in the midst of. .. neuroimaging Brain, 124, 84 9– 88 1 Chua, S E., & McKenna, P J (1995) Schizophrenia: A brain disease? British Journal of Psychiatry, 66, 563– 582 Chelune, G J., Naugle, R I., Luders, H S J., Sedlak, J., & Awad, I A (1993) Individual change after epilepsy surgery: Practice effects and base-rate information Neuropsychology, 7, 41–52 Christensen, L., & Mendoza, J L (1 986 ) A method of assessing change in a... Journal of Sports Medicine, 28, 643–650 Harlow, J M ( 186 8) Recovery after severe injury to the head Publications of the Massachusetts Medical Society, 2, 327–346 Hinton-Bayre, A D., Geffin, G M., Geffen, L G., McFarland, K A., & Friss, P (1999) Concussion in contact sports: Reliable change indices of impairment and recovery Journal of Clinical and Experimental Neuropsychology, 21, 70 86 Hirsch, J., Ruge, M... 4 68 Specific Measures of Interests 469 Summary 472 RELIABILITY AND VALIDITY OF INTEREST MEASURES 473 Reliability of Interests 473 Validity of Interests 473 GROUP DIFFERENCES IN THE MEASUREMENT OF INTERESTS 475 Age 475 Sex, Gender, and Sexual Orientation 475 Culture 476 COMPREHENSIVE ASSESSMENT OF INTERESTS WITH OTHER DOMAINS 477 Interest-Personality Relationships 477 Interest-Ability Relationships 478 . The exact cutoff chosen is based on the particular use of a test and the user’s assessment of the relative costs of different types of erroneous decisions. The sensitivity of a cutoff score refers. Erlbaum. Sokal, M. M. (1 981 ). The origins of the Psychological Corporation. Journal of the History of the Behavioral Sciences, 17, 54–67. Solso, R. L., & Hoffman, C. A. (1991). Influence of Soviet scholars. American. science of literacy: From the labo- ratory to the classroom. Proceedings of the National Academy of Sciences of the United States of America, 97, 2402–2404. Terman, L. M. (1916). The measurement of

Định dạng
Số trang	66
Dung lượng	604,73 KB