handbook of psychology phần 3 pps

110 Testing and Assessment in Cross-Cultural Psychology techniques. Allalouf et al. (1999) and Budgell et al. (1995) are other fine examples of this methodology in the literature. Exploratory, Replicatory Factor Analysis Many psychological tests, especially personality measures, have been subjected to factor analysis, a technique that has often been used in psychology in an exploratory fashion to identify dimensions or consistencies among the items composing a measure (Anastasi & Urbina, 1997). To estab- lish that the internal relationships of items or test components hold across different language versions of a test, a factor analysis of the translated version is performed. A factor analysis normally begins with the correlation matrix of all the items composing the measure. The factor analysis looks for patterns of consistency or factors among the items. There are many forms of factor analysis (e.g., Gorsuch, 1983) and tech- niques differ in many conceptual ways. Among the important decisions made in any factor analysis are determining the number of factors, deciding whether these factors are permit- ted to be correlated (oblique) or forced to be uncorrelated (orthogonal), and interpreting the resultant factors. A compo- nent of the factor analysis is called rotation, whereby the dimensions are changed mathematically to increase inter- pretability. The exploratory factor analysis that bears upon the construct equivalence of two measures has been called replicatory factor analysis (RFA; Ben-Porath, 1990) and is a form of cross-validation. In this instance, the number of fac- tors and whether the factors are orthogonal or oblique are constrained to yield the same number of factors as in the orig- inal test. In addition, a rotation of the factors is made to attempt to maximally replicate the original solution; this tech- nique is called target rotation. Once these procedures have been performed, the analysts can estimate how similar the factors are across solutions. van de Vijver and Leung (1997) provide indices that may be used for this judgment (e.g., the coefficient of proportionality). Although RFA has probably been the most used technique for estimating congruence (van de Vijver & Leung), it does suffer from a number of prob- lems. One of these is simply that newer techniques, especially confirmatory factor analysis, can now perform a similar analysis while also testing whether the similarity is statisti- cally significant through hypothesis testing. A second prob- lem is that different researchers have not employed standard procedures and do not always rotate their factors to a target solution (van de Vijver & Leung). Finally, many studies do not compute indices of factor similarity across the two solutions and make this discernment only judgmentally (van de Vijver & Leung). Nevertheless, a number of out- standing researchers (e.g., Ben-Porath, 1990; Butcher, 1996) have recommended the use of RFA to establish equivalence and this technique has been widely used, especially in valida- tion efforts for various adaptations of the frequently trans- lated MMPI and the Eysenck Personality Questionnaire. Regression Regression approaches are generally used to establish the relationships between the newly translated measure and measures with which it has traditionally correlated in the original culture. The new test can be correlated statistically with other measures, and the correlation coefficients that re- sult may be compared statistically with similar correlation coefficients found in the original population. There may be one or more such correlated variables. When there is more than one independent variable, the technique is called multi- ple regression. In this case, the adapted test serves as the de- pendent variable, and the other measures as the independent variables. When multiple regression is used, the independent variables are used to predict the adapted test scores. Multiple regression weights the independent variables mathematically to optimally predict the dependent variable. The regression equation for the original test in the original culture may be compared with that for the adapted test; where there are dif- ferences between the two regression lines, whether in the slope or the intercept, or in some other manner, bias in the testing is often presumed. If the scoring of the original- and target-language mea- sures is the same, it is also possible to include cultural group membership in a multiple regression equation. Such a nomi- nal variable is added as what has been called dummy-coded variable. In such an instance, if the dummy-coded variable is assigned a weighting as part of the multiple regression equa- tion, indicating that it predicts test scores, evidence of cultural differences across either the two measures or the two cultures may be presumed (van de Vijver & Leung, 1997). Structural Equation Modeling, Including Confirmatory Factor Analysis Structural equation modeling (SEM; Byrne, 1994; Loehlin, 1992) is a more general and statistically sophisticated proce- dure that encompasses both factor analysis and regression analysis, and does so in a manner that permits elegant hy- pothesis testing. When SEM is used to perform factor analy- sis, it is typically called a confirmatory factor analysis, which is defined by van de Vijver and Leung (1997) as “an exten- sion of classical exploratory factor analysis. Specific to confirmatory factor analysis is the testing of a priori speci- fied hypotheses about the underlying structure, such as the Methods of Evaluating Test Equivalence 111 number of factors, loadings of variables on factors, and factor correlations” (p. 99). Essentially, the results of factor-analytic studies of the measure in the original language are con- strained upon the adapted measure, data from the adapted measure analyzed, and a goodness-of-fit statistical test is performed. Regression approaches to relationships among a number of tests can also be studied with SEM. Elaborate models of relationships among other tests, measuring variables hypoth- esized and found through previous research to be related to the construct measured by the adapted test, also may be tested using SEM. In such an analysis, it is possible for a researcher to approximate the kind of nomological net conceptualized by Cronbach and Meehl (1955), and test whether the struc- ture holds in the target culture as it does in the original culture. Such a test should be the ideal to be sought in estab- lishing the construct equivalence of tests across languages and cultures. Item-Response Theory Item-response theory (IRT) is an alternative to classical psy- chometric true-score theory as a method for analyzing test data. Allen and Walsh (2000) and van de Vijver and Leung (1997) provide descriptions of the way that IRT may be used to compare items across two forms of a measure that differ by language.Althougha detailed description of IRTis beyond the scope of this chapter, the briefest of explanations may provide a conceptual understanding of how the procedure is used, especially for cognitive tests. An item characteristic curve (ICC) is computed for each item. This curve has as the x axis the overall ability level of test takers, and as the y axis, the probability of answering the question correctly. Different IRT models have different numbers of parameters, with one-, two- and three-parameter models most common. These parameters correspond to difficulty, discrimination, and the ability to get the answer correctby chance, respectively. The ICC curvesare plotted as normal ogive curves. When a test is adapted, each translated item may be compared across languages graphi- cally by overlaying the two ICCs as well as by comparing the item parameters mathematically. If there are differences, these may be considered conceptually. This method, too, may be considered as one technique for identifying item bias. Methods to Establish Linkage of Scores Once the conceptual equivalence of an adapted measure has been met, researchers and test developers often wish to pro- vide measurement-unit and metric equivalence, as well. For most measures, this requirement is met through the process of test equating. As noted throughout this chapter, merely trans- lating a test from one language to another, even if cultural biases have been eliminated, does not insure that the two different-language forms of a measure are equivalent. Con- ceptual or construct equivalence needs to be established first. Once such a step has been taken, then one can consider higher levels of equivalence. The mathematics of equating may be found in a variety of sources (e.g., Holland & Rubin, 1982; Kolen & Brennan, 1995), and Cook et al. (1999) provide an excellent integration of research designs and analysis for test adaptation; research designs for such studies are abstracted in the following paragraphs. Sireci (1997) clarified three experimental designs that can be used to equate adapted forms to their original-language scoring systems and, perhaps, norms. He refers to them as (a) the separate-monolingual-groups design, (b) the bilingual- group design, and (c) the matched-monolingual-groups de- sign. A brief description of each follows. Separate-Monolingual-Groups Design In the separate-monolingual-groups design, two different groups of test takers are involved, one from each language or cultural group. Although some items may simply be assumed to be equivalent across both tests, data can be used to support this assumption. These items serve as what is known in equat- ing as anchor items. IRT methods are then generally used to calibrate the two tests to a common scale, most typically the one used by the original-language test (Angoff & Cook, 1988; O’Brien, 1992; Sireci, 1997). Translated items must then be evaluated for invariance across the two different- language test forms; that is, they are assessed to determine whether their difficulty differs across forms. This design does not work effectively if the two groups actually differ, on av- erage, on the characteristic that is assessed (Sireci); in fact, in such a situation, one cannot disentangle differences in the ability measured from differences in the two measures. The method also assumes that the construct measured is based on a single, unidimensional factor. Measures of complex con- structs, then, are not good prospects for this method. Bilingual-Group Design In the bilingual-group design, a single group of bilingual in- dividuals takes both forms of the test in counterbalanced order. An assumption of this method is that the individuals in the group are all equally bilingual, that is, equally proficient in each language. In Maldonado and Geisinger (in press), all participants first were tested in both Spanish and English competence to gain entry into the study. Even under such re- strictive circumstances, however, a ceiling effect made a true 112 Testing and Assessment in Cross-Cultural Psychology assessment of equality impossible. The problem of finding equally bilingual test takers is almost insurmountable.Also, if knowledge of what is on the test in one language affects per- formance on the other test, it is possible to use two randomly assigned groups of bilingual individuals (where their level of language skill is equated via randomization). In such an in- stance, it is possible either to give each group one of the tests or to give each group one-half of the items (counterbalanced) from each test in a nonoverlapping manner (Sireci, 1997). Finally, one must question how representative the equally bilingual individuals are of the target population; thus the external validity of the sample may be questioned. Matched-Monolingual-Groups Design This design is conceptually similar to the separate- monolingual-groups design, except that in this case the study participants are matched on the basis of some variable ex- pected to correlate highly with the construct measured. By being matched in this way, the two groups are made more equal, which reduces error. “There are not many examples of the matched monolingual group linking design, probably due to the obvious problem of finding relevant and available matching criteria” (Sireci, 1997, p. 17). The design is never- theless an extremely powerful one. CONCLUSION Psychology has been critiqued as having a Euro-American ori- entation (Moreland, 1996; Padilla & Medina, 1996). Moreland wrote, Koch (1981) suggests that American psychologists are trained in scientific attitudes that Kimble (1984) has character- ized as emphasizing objectivity, data, elementism, concrete mechanisms, nomothesis, determinism, and scientific values. Dana (1993) holds that multicultural research and practice should emanate from a human science perspective characterized by the opposite of the foregoing terms: intuitive theory, holism, abstract concepts, idiography, indeterminism, and humanistic values. (p. 53) Moreland believed that this dichotomy was a false one. Never- theless, he argued that a balance of the two approaches was needed to understand cultural issues more completely. One of the advantages ofcross-cultural psychology is thatit challenges many of our preconceptions of psychology. It is often said that one learns much about one’s ownlanguage when learning a for- eign tongue. The analogy for psychology is clear. Assessment in cross-cultural psychology emphasizes an understanding of the context in which assessment occurs. The notion that traditional understandings of testing and as- sessment have focused solely on the individual can be tested in this discipline. Cross-cultural and multicultural testing help us focus upon the broader systems of which the individ- ual is but a part. Hambleton (1994) stated, The common error is to be rather casual about the test adaptation process, and then interpret the score differences among the sam- ples or populations as if they were real. This mindless disregard of test translation problems and the need to validate instruments in the cultures where they are used has seriously undermined the results from many cross cultural studies. (p. 242) This chapter has shown that tests that are adapted for use in different languages and cultures need to be studied for equivalence. There are a variety of types of equivalence: lin- guistic equivalence, functional equivalence, conceptual or construct equivalence, and metric equivalence. Linguistic equivalence requires sophisticated translation techniques and an evaluation of the effectiveness of the translation. Func- tional equivalence requires that those translating the test be aware of cultural issues in the original test, in the construct, in the target culture, and in the resultant target test. Conceptual equivalence requires a relentless adherence to a construct- validation perspective and the conduct of research using data from both original and target tests. Metric equivalence, too, involves careful analyses of the test data. The requirements of metric equivalence may not be met in many situations regard- less of how much we would like to use scoring scales from the original test with the target test. If equivalence is one side of the coin, then bias is the other. Construct bias, method bias and item bias can all influence the usefulness of a test adaptation in detrimental ways. The need for construct-validation research on adapted measures is reiterated; there is no more critical point in this chapter. In ad- dition, however, it is important to replicate the construct val- idation that had been found in the original culture with the original test. Factor analysis, multiple regression, and struc- tural equation modeling permit researchers to assess whether conceptual equivalence is achieved. The future holds much promise for cross-cultural psychol- ogy and for testing and assessment within that subdiscipline of psychology. There will be an increase in the use of different forms of tests used in both the research and the practice of psy- chology. In a shrinking world, it is clearer that many psycho- logical constructs are likely to hold for individuals around the world, or atleast throughout muchof it. Knowledgeof research from foreign settings and in foreign languages is much more Appendix 113 accessible than in the recent past. Thus, researchers may take advantage of theoretical understandings, constructs, and their measurement from leaders all over the world. In applied set- tings, companies such as Microsoft are already fostering a world in which tests (suchasfor software literacy) are available in dozens of languages. Costs of test development are so high that adaptation and translation of assessment materials can make the cost of professional assessment cost-effective even in developing nations, where the benefits of psychologicaltest- ing are likely to be highest. Computer translations of language are advancing rapidly. In some future chapter such as this one, the author may direct that the first step is to have a computer perform the first translation of the test materials. As this sen- tence is being written, we are not yet there; human review for cultural and language appropriateness continues to be needed. Yet in the time it will take for these pages to be printed and read, these words may have already become an anachronism. The search for psychological universals will continue, as will the search for cultural and language limitations on these characteristics. Psychological constructs, both of major import and of more minor significance, will continue to be found that do notgeneralize to different cultures.The fact that the world is shrinking because of advances in travel and communications does not mean we should assume it is necessarily becoming more Western—moreAmerican.Todo so is, at best, pejorative. These times are exciting, both historically and psychome- trically. The costs in time and money to develop new tests in each culture are often prohibitive. Determination of those as- pects of a construct that are universal and those that are cul- turally specific is critical. These are new concepts for many psychologists; we have not defined cultural and racial con- cepts carefully and effectively and we have not always incor- porated these concepts into our theories (Betancourt & López, 1993; Helms, 1992). Good procedures for adapting tests are available and the results of these efforts can be evaluated. Testing can help society and there is no reason for any coun- try to hoard good assessment devices. Through the adaptation procedures discussed in this chapter they can be shared. APPENDIX Guidelines of the International Test Commission for Adapting Tests (van de Vijver & Leung, 1997, and Hambleton, 1999) The initial guidelines relate to the testing context, as follows. 1. Effects of cultural differences that are not relevant or im- portant to the main purposes of the study should be min- imized to the extent possible. 2. The amount of overlap in the constructs in the popula- tions of interest should be assessed. The following guidelines relate to test translation or test adaptation. 3. Instrument developers/publishers should ensure that the translation/adaptation process takes full account of lin- guistic and cultural differences among the populations for whom the translated/adapted versions of the instru- ment are intended. 4. Instrument developers/publishers should provide evi- dence that the language used in the directions, rubrics, and items themselves as well as in the handbook [is] appropriate for all cultural and language populations for whom the instruments is intended. 5. Instrument developers/publishers should provide evi- dence that the testing techniques, item formats, test con- ventions, and procedures are familiar to all intended populations. 6. Instrument developers/publishers should provide evi- dence that item content and stimulus materials are famil- iar to all intended populations. 7. Instrument developers/publishers should implement sys- tematic judgmental evidence, both linguistic and psy- chological, to improve the accuracy of the translation/ adaptation process and compile evidence on the equiva- lence of all language versions. 8. Instrument developers/publishers should ensure that the data collection design permits the use of appropriate sta- tistical techniques to establish item equivalence between the different language versions of the instrument. 9. Instrument developers/publishers should apply appropri- ate statistical techniques to (a) establish the equivalence of the different versions of the instrument and (b) iden- tify problematic components or aspects of the instrument which may be inadequate to one or more of the intended populations. 10. Instrument developers/publishers should provide infor- mation on the evaluation of validity in all target pop- ulations for whom the translated/adapted versions are intended. 11. Instrument developers/publishers should provide statisti- cal evidence of the equivalence of questions for all in- tended populations. 12. Nonequivalent questions between versions intended for different populations should not be used in preparing a common scale or in comparing these populations. 114 Testing and Assessment in Cross-Cultural Psychology However, they may be useful in enhancing content valid- ity of scores reported for each population separately. [emphasis in original] The following guidelines relate to test administration. 13. Instrument developers and administrators should try to an- ticipate the types ofproblems that can beexpectedand take appropriate actions to remedy these problems through the preparation of appropriate materials and instructions. 14. Instrument administrators should be sensitive to a num- ber of factors related to the stimulus materials, adminis- tration procedures, and response modes that can moderate the validity of the inferences drawn from the scores. 15. Those aspects of the environment that influence the ad- ministration of an instrument should be made as similar as possible across populations for whom the instrument is intended. 16. Instrument administration instructions should be in the source and target languages to minimize the influence of unwanted sources of variation across populations. 17. The instrument manualshould specify all aspects of the in- strument and its administration that require scrutiny in the application of the instrument in a new cultural context. 18. The administration should be unobtrusive, and the examiner-examinee interaction should be minimized. Explicit rules that are described in the manual for the instrument should be followed. The final grouping of guidelines relate to documentation that is suggested or required of the test publisher or user. 19. When an instrument is translated/adapted for use in an- other population, documentation of the changes should be provided, along with evidence of the equivalence. 20. Score differences among samples of populations admin- istered the instrument should not be taken at face value. The researcher has the responsibility to substantiate the differences with other empirical evidence. [emphasis in original] 21. Comparisons across populations can only be made at the level of invariance that has been established for the scale on which scores are reported. 22. The instrument developer should provide specific infor- mation onthe ways in which the sociocultural and ecolog- ical contexts of the populations might affect performance on the instrument and should suggest procedures to ac- count for these effects in the interpretation of results. REFERENCES Allalouf, A., Hambleton, R. K., & Sireci, S. G. (1999). Identifying the causes of DIF in translated verbal items. Journal of Educa- tional Measurement, 36, 185–198. Allen, J., & Walsh, J. A. (2000). A construct-based approach to equivalence: Methodologies to cross-cultural/multicultural per- sonality assessment research. In R. H. Dana (Ed.), Handbook of cross-cultural and multicultural personality assessment (pp. 63– 85). Mahwah, NJ: Erlbaum. Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall. Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508– 600). Washington, DC: American Council on Education. Angoff, W. H., & Cook, L. L. (1988). Equating the scores of the “Prueba de Aptitud Academica” and the “Scholastic Aptitude Test” (Report No. 88-2). New York: College Entrance Examina- tion Board. Ben-Porath, Y. S. (1990). Cross-cultural assessment of personality: The case of replicatory factor analysis. In J. N. Butcher & C. D. Spielberger (Eds.), Advances in personality assessment (Vol. 8, pp. 27–48). Hillsdale, NJ: Erlbaum. Berry, J. W. (1980). Introduction to methodology. In H. C. Triandis & J. W. Berry (Eds.), Handbook of cross-cultural psychology (Vol. 2, pp. 1–28). Boston: Allyn and Bacon. Betancourt, H., & López, S. R. (1993). The study of culture, ethnic- ity, and race in American psychology. American Psychologist, 48, 629–637. Bracken, B. A., & Barona, A. (1991). State of the art procedures for translating, validating, and using psychoeducational tests in cross-cultural assessment. School Psychology International, 12, 119–132. Bracken, B. A., Naglieri, J., & Bardos, A. (1999, May). Nonverbal assessment of intelligence: An alternative to test translation and adaptation. Paper presented at the International Conference on Test Adaptation, Washington, DC. Brislin, R. W. (1970). Back translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1, 185–216. Brislin, R. W. (1980). Translation and content analysis of oral and written material. In H. C. Triandis & J. W. Berry (Eds.), Hand- book of cross-cultural psychology, Vol. 2: Methodology (pp. 389– 444). Needham Heights, MA: Allyn and Bacon. Brislin, R. W. (1986). The wording and translation of research instru- ments. In W. J. Lonner & J. W. Berry (Eds.), Field methods in cross-cultural research (pp. 137–164). Newberry Park, CA: Sage. Brislin, R. W. (1993). Understanding culture’s influence on behav- ior. New York: Harcourt Brace. Budgell, G. R., Raju, N. S., & Quartetti, D. A. (1995). Analysis of differential item functioning in translated assessment instru- ments. Applied Psychological Measurement, 19, 309–321. References 115 Butcher, J. N. (1996). Translation and adaptation of the MMPI-2 for international use. In J. N. Butcher (Ed.), International adapta- tions of the MMPI-2 (pp. 393–411). Minneapolis: University of Minnesota Press. Butcher, J. N., & Han, K. (1998). Methods of establishing cross- cultural equivalence. In J. N. Butcher (Ed.), International adaptations of the MMPI-2: Research and clinical applications (pp. 44–63). Minneapolis: University of Minnesota Press. Butcher, J. N., & Pancheri, P. (1976). A handbook of cross- cultural MMPI research. Minneapolis: University of Minnesota Press. Byrne, B. M. (1989). A primer of LISREL: Basic applications and programming for confirmatory factor analytic models. New York: Springer. Byrne, B. M. (1994). Structural equation modeling with EQS and EQS/Windows: Basic concepts, applications and programming. Thousand Oaks, CA: Sage. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discrimi- nant validity by the multitrait-multimethod matrix. Psychologi- cal Bulletin, 56, 81–105. Cattell, R. B. (1940). A culture-free intelligence test. Journal of Educational Psychology, 31, 176–199. Cattell, R. B. (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54, 1–22. Cattell, R. B., & Cattell, A. K. S. (1963). A culture fair intelli- gence test. Champaign, IL: Institute for Personality and Ability Testing. Clark, L. A. (1987). Mutual relevance of mainstream and cross- cultural psychology. Journal of Consulting and Clinical Psy- chology, 55, 461–470. Cole, M., & Bruner, J. S. (1971). Cultural differences and inferences about psychological processes. American Psychologist, 26, 867– 876. Cook, L., Schmitt, A., & Brown, C. (1999, May). Adapting achieve- ment and aptitude tests: A review of methodological issues. Paper presented at the International Conference on Test Adapta- tion, Washington, DC. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psy- chological tests. Psychological Bulletin, 52, 281–302. Cuéllar, I. (2000). Acculturation as a moderator of personality and psychological assessment. In R. H. Dana (Ed.), Handbook of cross-cultural and multicultural personality assessment (pp. 113–130). Mahwah, NJ: Erlbaum. Dana, R. H. (1993). Multicultural assessment perspectives for pro- fessional psychology. Boston: Allyn and Bacon. Domino, G. (2000). Psychological testing: An introduction. Upper Saddle River, NJ: Prentice Hall. Durán, R. P. (1988). Validity and language skills assessment: Non- English background students. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 105–127). Hillsdale, NJ: Erlbaum. Embretson, S. E. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179–193. Flaugher, R. J. (1978). The many definitions of test bias. American Psychologist, 33, 671–679. Geisinger, K. F. (1992). Fairness and selected psychometric issues in the psychological assessment of Hispanics. In K. F. Geisinger (Ed.), Psychologicaltestingof Hispanics (pp. 17–42).Washington, DC:American Psychological Association. Geisinger, K. F. (1994). Cross-cultural normative assessment: Translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychological Assess- ment, 6, 304–312. Geisinger, K. F. (2002). Testing the members of an increasingly di- verse society. In J. F. Carlson & B. B. Waterman (Eds.), Social and personal assessment of school-aged children: Developing interventions for educational and clinical use (pp. 349–364). Boston: Allyn and Bacon. Geisinger, K. F., & Carlson, J. F. (1998). Training psychologists to assess members of a diverse society. In J. Sandoval, C. L. Frisby, K. F. Geisinger, J. D. Schueneman, & J. R. Grenier (Eds.), Test interpretation and diversity: Achieving equity in assessment (pp. 375–386). Washington, DC: American Psycho- logical Association. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum. Hambleton, R. K. (1994). Guidelines for adapting educational and psychological tests: A progress report. European Journal of Psy- chological Assessment, 10, 229–244. Hambleton, R. K. (1999). Issues, designs, and technical guidelines for adapting tests in multiple languages and cultures (Labora- tory of Psychometric and Evaluative Research Report No. 353). Amherst: University of Massachusetts, School of Education. Handel, R. W., & Ben-Porath, Y. S. (2000). Multicultural assess- ment with the MMPI-2: Issues for research and practice. In R. H. Dana (Ed.), Handbook of cross-cultural and multicultural per- sonality assessment (pp. 229–245). Mahwah, NJ: Erlbaum. Helms, J. E. (1992). Why is there no study of cultural equivalence in standardized cognitive ability testing? American Psychologist, 47, 1083–1101. Holland, P. W., & Rubin, D. B. (Eds.). (1982). Test equating. New York: Academic Press. Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item func- tioning. Hillsdale, NJ: Erlbaum. Hulin, C. L. (1987). A psychometric theory of evaluations of item and scale translations: Fidelity across languages. Journal of Cross-Cultural Psychology, 18, 721–735. Kamin, L. J. (1974). The science and politics of I.Q. Potomac, MD: Erlbaum. Keitel, M. A., Kopala, M., & Adamson, W. S. (1996). Ethical issues in multicultural assessment. In L. A. Suzuki, P. J. Meller, & J. G. Ponterotto (Eds.), Handbook of multicultural assessment: 116 Testing and Assessment in Cross-Cultural Psychology Clinical, psychological and educational applications (pp. 28– 48). San Francisco: Jossey-Bass. Kolen, M. J., & Brennan, D. B. (1995). Test equating: Methods and practices. New York: Springer. Loehlin, J. C. (1992). Latent variable models: An introduction to factor, path, and structural equations analysis. Hillsdale, NJ: Erlbaum. Lonner, W. J. (1979). Issues in cross-cultural psychology. In A. J. Marsella, R.Tharp, & T. Ciborowski(Eds.),Perspectiveson cross- cultural psychology (pp. 17–45). NewYork: Academic Press. Maldonado, C. Y., & Geisinger, K. F. (in press). Conversion of the WechslerAdult Intelligence Scale into Spanish:An early test adap- tation effort of considerable consequence. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment. Hillsdale, NJ: Erlbaum. Malgady, R. G. (1996). The question of cultural bias in assessment and diagnosis of ethnic minority clients: Let’s reject the null hypothesis. Professional Psychology: Research and Practice, 27, 33–73. Marín, G. (1992). Issues in the measurement of acculturation among Hispanics. In K. F. Geisinger (Ed.), The psychological testing of Hispanics (pp. 235–272). Washington, DC: American Psycho- logical Association. Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35, 1012–1027. Messick, S. (1989) Validity. In R. L. Linn (Ed.), Educational mea- surement (3rd ed., pp. 13–104). New York: American Council on Education/Macmillan. Messick, S. (1995). Validity of psychological assessment: Valida- tion of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749. Moreland, K. L. (1996). Persistent issues in multicultural assess- ment of social and emotional functioning. In L. A. Suzuki, P. J. Meller, & J. G. Ponterotto (Eds.), Handbook of multicultural as- sessment: Clinical, psychological and educational applications (pp. 51–76). San Francisco: Jossey-Bass. Nichols, D. S., Padilla, J., & Gomez-Maqueo, E. L. (2000). Issues in the cross-cultural adaptation and use of the MMPI-2. In R. H. Dana (Ed.), Handbook of cross-cultural and multicultural per- sonality assessment (pp. 247–266). Mahwah, NJ: Erlbaum. O’Brien, M. L. (1992). A Rausch approach to scaling issues in test- ing Hispanics. In K. F. Geisinger (Ed.), Psychological testing of Hispanics (pp. 43–54). Washington, DC: American Psychologi- cal Association. Olmeda, E. L. (1979). Acculturation: A psychometric perspective. American Psychologist, 34, 1061–1070. Padilla, A. M. (1992). Reflections on testing: Emerging trends and new possibilities. In K. F. Geisinger (Ed.), The psychological testing of Hispanics (pp. 271–284). Washington, DC: American Psychological Association. Padilla, A. M., & Medina, A. (1996). Cross-cultural sensitivity in assessment: Using tests in culturally appropriate ways. In L. A. Suzuki, P. J. Meller, & J. G. Ponterotto (Eds.), Handbook of mul- ticultural assessment: Clinical, psychological and educational applications (pp. 3–28). San Francisco: Jossey-Bass. Pennock-Roman, M. (1992). Interpreting test performance in selec- tive admissions for Hispanic students. In K. F. Geisinger (Ed.), The psychological testing of Hispanics (pp. 99–136). Washington, DC: American Psychological Association. Pike, K. L. (1967). Language in relation to a unified theory of the structure of human behavior. The Hague, The Netherlands: Mouton. Ritzler, B. A. (1996). Projective techniques for multicultural person- ality assessment: Rorschach, TEMAS and the Early Memories Procedure. In L. A. Suzuki, P. J. Meller, & J. G. Ponterotto (Eds.), Handbook of multicultural assessment: Clinical, psychological and educational applications (pp. 115–136). San Francisco: Jossey-Bass. Sandoval, J. (1998). Test interpretation in a diverse future. In J. Sandoval, C. L. Frisby, K. F. Geisinger, J. D. Schueneman, & J. R. Grenier (Eds.), Test interpretation and diversity: Achieving equity in assessment (pp. 387–402). Washington, DC: American Psychological Association. Sattler, J. M. (1992). Assessment of children (Revised and updated 3rd ed.). San Diego, CA: Author. Schmitt, A. P. (1988). Language and cultural characteristics that ex- plain differential item functioning for Hispanic examinees on the Scholastic Aptitude Test. Journal of Educational Measurement, 25, 1–13. Sireci, S. G. (1997). Problems and issues in linking assessments across languages. Educational Measurement: Issues and Prac- tice, 16, 12–17. Steele, C. M. (1997). A threat in the air: How stereotypes shape in- tellectual identity and performance. American Psychologist, 52, 613–629. Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intel- lectual test performance of African Americans. Journal of Per- sonality and Social Psychology, 69, 797–811. Sue, S., Keefe, K., Enomoto, K., Durvasula, R., & Chao, R. (1996). Asian American and White college students’ performance on the MMPI-2. In J. N. Butcher (Ed.), International adaptations of the MMPI-2: Research and clinical applications (pp. 206–220). Minneapolis: University of Minnesota Press. Sundberg, N. D., & Gonzales, L. R. (1981). Cross-cultural and cross-ethnic assessment: Overview and issues. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 5, pp. 460– 541). San Francisco: Jossey-Bass. Suzuki, L. A., Vraniak, D. A., & Kugler, J. F. (1996). Intellectual as- sessment across cultures. In L. A. Suzuki, P. J. Meller, & J. G. Ponterotto (Eds.), Handbook of multicultural assessment: Clini- cal, psychological and educational applications (pp. 141–178). San Francisco: Jossey-Bass. References 117 Tanzer, N. K., Gittler, G., & Sim, C. Q. E. (1994). Cross-cultural validation of item complexity in an LLTM-calibrated spatial ability test. European Journal of Psychological Assessment, 11, 170–183. Triandis, H. C., Malpass, R. S., & Davidson, A. (1971). Cross- cultural psychology. In B. J. Siegel (Ed.), Biennial review of anthropology (pp. 1–84). Palo Alto, CA: Annual Reviews. van de Vijver, F. (2000). The nature of bias. In R. H. Dana (Ed.), Handbook of cross-cultural and multicultural personality as- sessment (pp. 87–106). Mahwah, NJ: Erlbaum. van de Vijver, F., & Leung, K. (1997). Methods and data analysis for cross-cultural research. Thousand Oaks, CA: Sage. van de Vijver, F., & Poortinga, Y. H. (1982). Cross-cultural general- ization and universality. Journal of Cross-Cultural Psychology, 13, 387–408. van de Vijver, F., & Poortinga, Y. H. (1991).Testing across cultures. In R. K. Hambleton & J. N. Zall (Eds.), Advances in educational and psychological testing (pp. 277–308). Boston: Kluwer Academic. van de Vijver, F. J. R., & Poortinga, Y. H. (1997). Towards an inte- grated analysis in cross-cultural assessment. European Journal of Psychological Assessment, 13, 29–37. Werner, O., & Campbell, D. T. (1970). Translating, working through interpreters, and the problem of decentering. In R. Narroll & R. Cohen (Eds.), A handbook of cultural anthropology (pp. 398– 419). New York: American Museum of Natural History. Yansen, E. A., & Shulman, E. L. (1996). Language testing: Multi- cultural considerations. In L. A. Suzuki, P. J. Meller, & J. G. Ponterotto (Eds.), Handbook of multicultural assessment: Clini- cal, psychological and educational applications (pp. 353–394). San Francisco: Jossey-Bass. CHAPTER 6 Psychological Assessment in Treatment MARK E. MARUISH 119 PSYCHOLOGICAL ASSESSMENT AS A TREATMENT ADJUNCT: AN OVERVIEW 120 Psychological Assessment for Clinical Decision-Making 120 Psychological Assessment for Outcomes Assessment 121 Psychological Assessment as a Treatment Technique 121 PSYCHOLOGICAL ASSESSMENT AS A TOOL FOR SCREENING AND DIAGNOSIS 121 DIAGNOSIS 122 Diagnosis-Specific Instruments 123 Personality Measures and Symptom Surveys 123 PSYCHOLOGICAL ASSESSMENT AS A TOOL FOR TREATMENT PLANNING 124 The Benefits of Psychological Assessment for Treatment Planning 124 PSYCHOLOGICAL ASSESSMENT AS A TREATMENT INTERVENTION 126 What Is Therapeutic Assessment? 126 The Therapeutic Assessment Process 126 Empirical Support for Therapeutic Assessment 127 TREATMENT MONITORING 127 Monitoring Change 128 Other Uses for Patient Profiling 128 The Effects of Providing Feedback to the Therapist 128 OUTCOMES ASSESSMENT 129 What Are Outcomes? 129 The Use of Outcomes Assessment in Treatment 129 PSYCHOLOGICAL ASSESSMENT IN THE ERA OF MANAGED BEHAVIORAL HEALTH CARE 131 Current Status 132 Opportunities for Psychological Assessment 132 A Concluding Note 133 FUTURE DIRECTIONS 134 What the Field Is Moving Away From 134 Trends in Instrumentation 134 Trends in Technology 135 REFERENCES 136 Society’s need for behavioral health care services provides an opportunity for trained providers of mental health and sub- stance abuse services to become part of the solution to a major health care problem. Each of the behavioral health pro- fessions has the potential to make a particular contribution to this solution. Not the least of these contributions are those that can be made by clinical psychologists. The use of psy- chological tests in the assessment of the human condition is one of the hallmarks of clinical psychology. The training and acquired level of expertise in psychological testing distin- guishes the clinical psychologist from other behavioral health care professionals. Indeed, expertise in test-based psycholog- ical assessment can be said to be the unique contribution that clinical psychologists make to the behavioral health care field. For decades, clinical psychologists and other behavioral health care providers have come to rely on psychological assessment as a standard tool to be used with other sources of information for diagnostic and treatment planning purposes. However, changes that have taken place during the past sev- eral years in the delivery of health care in general, and behav- ioral health care services in particular, have led to changes in the way in which third-party payers and clinical psychologists themselves think about and use psychological assessment in day-to-day clinical practice. Some question the value of psy- chological assessment in the current time-limited, capitated service delivery arena, where the focus has changed from clin- ical priorities to fiscalpriorities (Sederer, Dickey, & Hermann, 1996). Others argue that it is in just such an arena that the ben- efits of psychological assessment can be most fully realized and contribute significantly to the delivery of cost-effective treatment for behavioral health disorders (Maruish, 1999a). Consequently, psychological assessment could assist the health care industry in appropriately controlling or reducing the utilization and cost of health care over the long term. Portions adapted from M. E. Maruish (1999a) with permission from Erlbaum. Portions adapted from M. E. Maruish (1999b) with per- mission from Elsevier Science. Portions adapted from M. E. Maruish (2002) with permission from Erlbaum. [...]... Results of the CAPP practitioner survey Professional Psychology: Research and Practice, 29, 31 36 Piotrowski, C (1999) Assessment practices in the era of managed care: Current status and future directions Journal of Clinical Psychology, 55, 787–796 Piotrowski, C., Belter, R W., & Keller, J W (1998) The impact of “managed care” on the practice of psychological testing: Preliminary findings Journal of Personality... waiting 3 6 months to assess the variables of interest is preferred A longer interval between discharge and postdischarge follow-up should provide a more valid indication of the lasting effects of treatment Comparison of the patient’s status on the variables of interest at the time of follow-up with that found at the time of either treatment initiation or termination will provide an indication of the... level of functioning following the therapeutic episode would either (a) fall outside the range of the dysfunctional population by at least 2 standard deviations from the mean of that population, in the direction of functionality; (b) fall within 2 standard deviations of the mean for the 131 normal or functional population; or (c) be closer to the mean of the functional population than to that of the... MMPI-2 and Rorschach findings Journal of Personality Assessment, 67, 5 43 557 Finn, S E (1996b) Manual for using the MMPI-2 as a therapeutic intervention Minneapolis: University of Minnesota Press Finn, S E., & Butcher, J N (1991) Clinical objective personality assessment In M Hersen, A E Kazdin, & A S Bellack (Eds.), The clinical psychology handbook (2nd ed., pp 36 2 37 3) New York: Pergamon Press Finn,... feelings of mastery and control, and decrease feelings of alienation Therapeutic use of assessment generally involves a presentation of assessment results (including assessment materials such as test protocols, profile forms, and other assessment summary materials) directly to the patient; an elicitation of the patient’s reactions to them; and an in-depth discussion of the meaning of the results in terms of. .. Journal of Clinical Psychiatry, 59(Suppl 20), 22 33 Speer, D C (1992) Clinically significant change: Jacobson and Truax (1991) revisited Journal of Consulting and Clinical Psychology, 60, 402–408 Spitzer, R L., Kroenke, K., Williams, J B., & Patient Health Questionnaire Primary Care Study Group (1999) Validation and utility of a self-report version of PRIME-MD: The PHQ primary care study Journal of the... is lower DIAGNOSIS Key to the development of any effective plan of treatment for mental health and substance abuse patients is the ascertainment of an accurate diagnosis of the problem(s) for which the patient is seeking intervention As in the past, assisting in the differential diagnosis of psychiatric disorders continues to be one of the major functions of psychological assessment (Meyer et al.,... lasting effects of the Psychological Assessment in the Era of Managed Behavioral Health Care intervention Generally, the variables of interest for this type of comparison include symptom presence and intensity, feeling of well-being, frequency of substance use, and social or role functioning Although it provides what is arguably the best and most useful outcomes information, a program of postdischarge... can often assist in the clarification of a known problem Through tests designed for use with populations presenting problems similar to those of the patient, aspects of identified problems can be elucidated Information gained from these tests can both improve the patient’s and clinician’s understanding of the problem and lead to the development of a better treatment plan The three most important types of. .. role of clinical assessment has, for the most part, changed To the dismay of proponents of clinical methods, the future is likely to focus more on specific domain-based rather than comprehensive assessment (p 7 93) Opportunities for Psychological Assessment The foregoing representations of the current state of psychological assessment in behavioral health care delivery could be viewed as an omen of worse . CARE 131 Current Status 132 Opportunities for Psychological Assessment 132 A Concluding Note 133 FUTURE DIRECTIONS 134 What the Field Is Moving Away From 134 Trends in Instrumentation 134 Trends. (1996). The question of cultural bias in assessment and diagnosis of ethnic minority clients: Let’s reject the null hypothesis. Professional Psychology: Research and Practice, 27, 33 – 73. Marín, G. (1992) intelligence test. Journal of Educational Psychology, 31 , 176–199. Cattell, R. B. (19 63) . Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54, 1–22. Cattell,