Statistics salkind 4e test bank ch06

Chapter 6: Just the Truth: An Introduction to Understanding Reliability and Validity Test Bank MULTIPLE CHOICE Reliability and validity should be considered a Before you start analyzing data b After data have been analyzed c When data are being collected d In regard to research on human subjects only ANS: REF: OBJ: COG: A PTS: DIF: Easy What’s Up With This Measurement Stuff? This is a stats class! What’s up with this measurement stuff? Comprehension _ explore(s) the question, “How I know that the test, scale, instrument, etc I use works every time I use it?” a Reliability b Validity c Both reliability and validity d Neither reliability nor validity ANS: REF: OBJ: COG: A PTS: DIF: Easy What’s Up With This Measurement Stuff? What reliability and validity are and why they are important Comprehension _ explore(s) the question, “How I know that the test, scale, instrument, etc I use measures what it is supposed to?” a Reliability b Validity c Both reliability and validity d Neither reliability nor validity ANS: REF: OBJ: COG: B PTS: DIF: Easy What’s Up With This Measurement Stuff? What reliability and validity are and why they are important Comprehension If data are not reliable or not valid, the results of any test or hypothesis a Must be true b Are inconclusive c Are valid in only certain situations d Must be interpreted ANS: REF: OBJ: COG: B PTS: DIF: Easy What’s Up With This Measurement Stuff? This is a stats class! What’s up with this measurement stuff? Knowledge Reliability and validity specifically relate to the a Quality of the data themselves b Appropriateness of the statistical tests chosen c Qualitative methodology d Correlations between variables ANS: REF: OBJ: COG: A PTS: DIF: Easy What’s Up With This Measurement Stuff? This is a stats class! What’s up with this measurement stuff? Comprehension The outcome variable in an analysis is called the _ a Predictor variable b Treatment variable c Dependent variable d Independent variable ANS: REF: OBJ: COG: C PTS: DIF: Easy An Introduction to Reliability and Validity What reliability and validity are and why they are important Comprehension If you wanted to examine the impact of fast-food consumption on weight gain, your measurement of weight gain would be the _ a Dependent variable b Treatment variable c Predictor variable d Independent variable ANS: REF: OBJ: COG: A PTS: DIF: Easy An Introduction to Reliability and Validity What reliability and validity are and why they are important Application _ is the least precise level of measurement a Nominal b Ordinal c Interval d Ratio ANS: A PTS: OBJ: The basic measurement scales DIF: Easy REF: In Sum COG: Comprehension _ is the most precise level of measurement a Nominal b Ordinal c Interval d Ratio ANS: D PTS: OBJ: The basic measurement scales DIF: Easy REF: In Sum COG: Comprehension 10 _ is the level of measurement where outcomes are based on some underlying continuum where it is possible to speak about how much more a higher performance is than a lower one a Nominal b Ordinal c Interval d Ratio ANS: C PTS: DIF: Easy REF: + = 2: The Interval Level of Measurement OBJ: The basic measurement scales COG: Comprehension 11 _ is the level of measurement where outcomes can only be placed into unranked categories a Nominal b Ratio c Interval d Ordinal ANS: A PTS: DIF: Easy REF: A Rose by Any Other Name: The Nominal Level of Measurement OBJ: The basic measurement scales COG: Comprehension 12 _ is the level of measurement where outcomes can be rank ordered a Nominal b Ordinal c Ratio d Interval ANS: B PTS: DIF: Easy REF: Any Order Is Fine With Me: The Ordinal Level of Measurement OBJ: The basic measurement scales COG: Comprehension 13 _ is the level of measurement where outcomes are based on some underlying continuum that also contains a true or absolute zero a Ratio b Interval c Ordinal d Nominal ANS: A PTS: DIF: Easy REF: Can Anyone Have Nothing of Anything? The Ratio Level of Measurement OBJ: The basic measurement scales COG: Comprehension 14 Only the _ level of measurement has a true zero a Nominal b Ordinal c Interval d Ratio ANS: D PTS: DIF: Easy REF: Can Anyone Have Nothing of Anything? The Ratio Level of Measurement OBJ: The basic measurement scales COG: Comprehension 15 Moving from the nominal to the ratio level of measurement, your data a Become less precise and detailed b Become more precise and detailed c Do not change in precision d Change depending on the individual data ANS: B PTS: OBJ: The basic measurement scales DIF: Easy REF: In Sum COG: Comprehension 16 Which of the following is concerned with monitoring estimates of present performance and predictions of future performance? a Content validity b Predictive validity c Concurrent validity d Construct validity ANS: B PTS: DIF: Medium REF: Criterion Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 17 What type of validity is concerned with the adequate representation of test items? a Content validity b Predictive validity c Concurrent validity d Construct validity ANS: A PTS: DIF: Medium REF: Content Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 18 If you want to know that a test measures some underlying psychological construct, what type of validity evidence would you want to collect? a Content validity b Construct validity c Predictive validity d Concurrent validity ANS: B PTS: DIF: Medium REF: Construct Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 19 If you correlate scores from your test with some other valid measure that assesses the same set of abilities, what type of validity evidence are you collecting? a Criterion validity b Content validity c Construct validity d Internal consistency ANS: A PTS: DIF: Medium REF: Criterion Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 20 If I can say that my weekly statistics quiz fairly assesses the material covered, what source of validity evidence should I have collected? a Criterion validity b Content validity c Construct validity d Concurrent validity ANS: B PTS: DIF: Medium REF: Content Validity OBJ: How to compute and interpret various types of validity coefficients COG: Application 21 What is the key to establishing criterion validity? a Number of criteria b Quality of criterion c Type of correlation d Predictive ability ANS: B PTS: DIF: Medium REF: Criterion Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 22 Which of the following is NOT a measure of validity? a Content validity b Parallel forms validity c Criterion validity d Construct validity ANS: B PTS: DIF: Medium REF: Different Types of Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 23 _ is established by consultation with an expert on the topic focused upon by your instrument a Content validity b Criterion validity c Parallel forms validity d Construct validity ANS: A PTS: DIF: Medium REF: Content Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 24 _ is determined by the association between the test scores and some specified present or future criterion a Construct validity b Criterion validity c Content validity d Internal consistency validity ANS: B PTS: DIF: Medium REF: Criterion Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 25 _ is based on the judgment of how well a test reflects an underlying idea a Construct validity b Criterion validity c Internal consistency validity d Content validity ANS: A PTS: DIF: Medium REF: Construct Validity OBJ: How to compute and interpret various types of validity coefficients COG: Knowledge 26 High school class rank is highly correlated with college GPA This is an example of what type of validity? a Concurrent validity b Predictive validity c Content validity d Construct validity ANS: B PTS: DIF: Medium REF: Criterion Validity OBJ: How to compute and interpret various types of validity coefficients COG: Application 27 In terms of the reliability of test scores, there are multiple elements to each person’s score The score that is actually recorded is the _ a Error score b True score c False score d Observed score ANS: D PTS: DIF: Medium REF: Test Scores—Truth or Dare OBJ: How to compute and interpret various types of reliability coefficients COG: Knowledge 28 “A measure of how stable a test is over time” is an example of which of the following types of reliability? a Interrater b Test–retest c Parallel forms d Internal consistency ANS: B PTS: DIF: Medium REF: Test–Retest Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Knowledge 29 The correlation between scores from Time and Time is called _ a Test–retest reliability b Parallel forms reliability c Internal consistency reliability d Similar forms reliability ANS: A PTS: DIF: Medium REF: Test–Retest Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Comprehension 30 The correlation between two forms of the same test is called _ a Parallel forms reliability b Parallel forms validity c Test–retest reliability d Internal consistency reliability ANS: A PTS: DIF: Medium REF: Parallel Forms Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Knowledge 31 Cronbach’s alpha is used to calculate _ a Internal consistency reliability b Internal consistency validity c Parallel forms reliability d Internal validity ANS: REF: OBJ: COG: A PTS: DIF: Medium Internal Consistency Reliability How to compute and interpret various types of reliability coefficients Knowledge 32 The number of agreements between your two raters divided by the total number of possible agreements is the way to calculate _ a Parallel forms reliability b Multiple judges reliability c Interrater reliability d Multiple judges validity ANS: C PTS: DIF: Medium REF: Interrater Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Comprehension 33 _ is the type of validity that examines how well a test outcome is consistent with a criterion that occurs in the present a Content validity b Predictive validity c Concurrent criterion validity d Test–retest validity ANS: C PTS: DIF: Medium REF: Criterion Validity OBJ: How to compute and interpret various types of validity coefficients COG: Comprehension 34 _ is the type of validity that examines how well a test outcome is consistent with a criterion that occurs in the future a Content validity b Test–retest validity c Concurrent criterion validity d Predictive validity ANS: D PTS: DIF: Medium REF: Criterion Validity OBJ: How to compute and interpret various types of validity coefficients COG: Comprehension 35 _ is the score that you would receive if the test contained no error a Observed score b Actual score c True score d Error score ANS: C PTS: DIF: Medium REF: Test Scores—Truth or Dare OBJ: How to compute and interpret various types of reliability coefficients COG: Knowledge 36 The actual or measured score is called the _ a True score b Observed score c Error score d Real score ANS: B PTS: DIF: Easy REF: Test Scores—Truth or Dare OBJ: How to compute and interpret various types of reliability coefficients COG: Knowledge 37 The _ is the difference between the observed and true scores a Score differential b Score difference c Error score d Error of approximation ANS: REF: OBJ: COG: C PTS: DIF: Medium Observed Score = True Score + Error Score How to compute and interpret various types of reliability coefficients Comprehension 38 Which coefficient reflects the occurrence of perfect reliability? a 1.00 b .5 c .100 d 2.00 ANS: REF: OBJ: COG: A PTS: DIF: Medium How Big Is Big? Interpreting Reliability Coefficients How to compute and interpret various types of reliability coefficients Comprehension 39 Two trained professionals observe the behavior of children in a classroom They each rate observed behaviors using the same form, and the number of items that were rated the same is calculated This is an example of which type of reliability? a Parallel forms b Test–retest c Interrater d Internal consistency ANS: C PTS: DIF: Medium REF: Interrater Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Application 40 If the measures associated with a test are said to be consistent, you might conclude that the measure is _ a Standard b Valid c Reliable d Concurrent ANS: REF: OBJ: COG: C PTS: DIF: Medium Reliability—Doing It Again Until You Get It Right What reliability and validity are and why they are important Comprehension 41 Which of the following is NOT a measure of reliability? a Test–retest reliability b Multiple-forms reliability c Internal consistency reliability d Interrater reliability ANS: REF: OBJ: COG: B PTS: DIF: Medium Different Types of Reliability What reliability and validity are and why they are important Comprehension 42 When we calculate reliability, we know the observed score What are the two unknown components of the reliability equation? a Method and error scores b Means and standard deviations c Test–retest and interrater scores d True and error scores ANS: REF: OBJ: COG: D PTS: DIF: Medium Observed Score = True Score + Error Score How to compute and interpret various types of reliability coefficients Comprehension 43 Interrater reliability measures consistency a Over time b From form to form c Across different tests d From rater to rater ANS: D PTS: DIF: Easy REF: Interrater Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Knowledge 44 Which of the following is the Greek letter associated with Cronbach’s alpha, sometimes known as coefficient alpha? a k b  c  d s ANS: REF: OBJ: COG: B PTS: DIF: Medium Internal Consistency Reliability How to compute and interpret various types of reliability coefficients Knowledge 45 Test–retest reliability is an example of which of the following? a Measure of consistency b Measure of reliability over time c Measure of agreement d Measure of equivalence ANS: B PTS: DIF: Easy REF: Test–Retest Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Comprehension 46 Parallel forms reliability is an example of which of the following? a Measure of consistency b Measure of stability c Measure of agreement d Measure of equivalence ANS: D PTS: DIF: Easy REF: Parallel Forms Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Comprehension 47 Interrater reliability is an example of which of the following? a Measure of consistency b Measure of stability c Measure of agreement d Measure of equivalence ANS: C PTS: DIF: Easy REF: Interrater Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Comprehension 48 What would the interrater reliability be for a 50-item measure in which the number of agreements between Rater and Rater was 45? a 0.45 b 80% c 0.90 d 95% ANS: C PTS: DIF: Easy REF: Interrater Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Application 49 What are the four levels of measurement? a Rank, ordinal, nominal, and ratio b Ratio, interval, categorical, and nominal c Nominal, ordinal, interval, and ratio d Nominal, rank, interval, and ordinal ANS: C PTS: REF: All About Measurement Scales COG: Knowledge DIF: Easy OBJ: The basic measurement scales 50 The maximum level of validity possible is equal to a The square of the reliability coefficient b The square root of the reliability coefficient c The cube of the reliability coefficient d The square root of the reliability coefficient, plus one ANS: REF: OBJ: COG: B PTS: DIF: Medium Validity and Reliability: Really Close Cousins How to compute and interpret various types of validity coefficients Knowledge 51 Which level of measurement is concerned only with the characteristics of an outcome that fits into a category? a Ordinal b Interval c Rank d Nominal ANS: D PTS: DIF: Medium REF: A Rose by Any Other Name: The Nominal Level of Measurement OBJ: The basic measurement scales COG: Knowledge 52 If you are interested in describing the order of variables along a continuum, what level of measurement would you use? a Nominal b Interval c Ordinal d Ratio ANS: C PTS: DIF: Medium REF: Any Order Is Fine With Me: The Ordinal Level of Measurement OBJ: The basic measurement scales COG: Knowledge 53 If the underlying continuum we are measuring assumed equal intervals, at what level of measurement is the associated variable being measured? a Ratio b Interval c Ordinal d Nominal ANS: B PTS: DIF: Medium REF: + = 2: The Interval Level of Measurement OBJ: The basic measurement scales COG: Knowledge 54 Which of the following levels of measurement provides the most information about a variable? a Nominal b Ordinal c Interval d Ratio ANS: D PTS: DIF: Medium REF: Can Anyone Have Nothing of Anything? The Ratio Level of Measurement OBJ: The basic measurement scales COG: Knowledge 55 Who is responsible for creating the formula for coefficient alpha? a Lee Cronbach b Jacob Cohen c Karl Pearson d R A Fisher ANS: REF: OBJ: COG: A PTS: DIF: Medium Internal Consistency Reliability How to compute and interpret various types of reliability coefficients Knowledge 56 What is the assignment of values to outcomes defined as? a Measurement b Listing c Assigning d Filing ANS: REF: OBJ: COG: A PTS: DIF: Easy An Introduction to Reliability and Validity This is a stats class! What’s up with this measurement stuff? Comprehension 57 What is the difference between the true score and the observed score? a Critical score b Error score c Expected score d Reliable score ANS: REF: OBJ: COG: B PTS: DIF: Medium Observed Score = True Score + Error Score How to compute and interpret various types of reliability coefficients Knowledge 58 For a total of 18 possible agreements, there are times where Duck and Cow agreed on whether students in college fell asleep at their desks or not What is the interrater reliability coefficient? a 56 b 0.39 c 3.90 d .003 ANS: B PTS: DIF: Medium REF: Interrater Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Application 59 For a total of 10 possible agreements, there are times where Bear and Cat agreed on whether children in ice cream commercials closed their eyes while eating ice cream or not What is the interrater reliability coefficient? a 0.80 b 0.08 c 80.0 d 8.0 ANS: A PTS: DIF: Medium REF: Interrater Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Application TRUE/FALSE The less precise levels of measurement contain all of the qualities of the scales above them ANS: F PTS: OBJ: The basic measurement scales DIF: Easy REF: In Sum COG: Comprehension Observed scores are commonly exactly the same as true scores in the social and behavioral sciences ANS: F PTS: DIF: Easy REF: Test Scores—Truth or Dare OBJ: What reliability and validity are and why they are important COG: Comprehension For high reliability, you want to reliability coefficients to be positive and as close as possible to ANS: REF: OBJ: COG: T PTS: DIF: Easy How Big Is Big? Interpreting Reliability Coefficients How to compute and interpret various types of reliability coefficients Comprehension If your test is not reliable, you must try to lower your error ANS: REF: OBJ: COG: T PTS: DIF: Easy And If You Can’t Establish Reliability Then What? How to compute and interpret various types of reliability coefficients Comprehension When working on a thesis or dissertation, it is recommended to use an instrument that has already been established to be reliable and valid ANS: T PTS: DIF: Easy REF: A Last, Friendly Word OBJ: What reliability and validity are and why they are important COG: Knowledge A test can be valid but not reliable, but it is impossible to have a test that is reliable and not valid ANS: T PTS: DIF: Easy REF: Validity and Reliability: Really Close Cousins OBJ: What reliability and validity are and why they are important COG: Comprehension SHORT ANSWER What are the four levels of measurement? Give an example of each ANS: Nominal—gender (any categorical variable); Ordinal—winners in a race (numerical order important here); Interval—GRE results (continuous variables with no true zero); Ratio—weight (true zero present) PTS: DIF: Medium OBJ: The basic measurement scales REF: All About Measurement Scales COG: Comprehension How does error impact reliability? Write the general equation for reliability that demonstrates this concept ANS: An increase in error results in a decrease in reliability, and a decrease in error results in an increase in reliability Observation = True Score + Error Score PTS: DIF: Hard REF: Observed Score = True Score + Error Score OBJ: How to compute and interpret various types of reliability coefficients COG: Analysis Name four types of reliability and what they measure ANS: Answers should include that test–retest measures stability; parallel forms measure equivalence; interrater measures agreement; and Cronbach’s alpha (or internal consistency) measures internal consistency PTS: DIF: Medium REF: Different Types of Reliability OBJ: What reliability and validity are and why they are important COG: Knowledge Explain the formula for understanding observed scores What is included in any observed score? ANS: Observed Scores = True scores + Error Score PTS: DIF: Medium REF: Observed Score = True Score + Error Score OBJ: How to compute and interpret various types of reliability coefficients COG: Knowledge What are the traditional types of validity? ANS: Content, criterion (both predictive and concurrent), and construct PTS: DIF: Medium REF: Different Types of Validity OBJ: What reliability and validity are and why they are important COG: Knowledge What is the difference between reliability and validity? ANS: Validity wants to know what is being tested, whereas reliability wants to know how consistently it is tested PTS: DIF: Medium REF: Validity and Reliability: Really Close Cousins OBJ: What reliability and validity are and why they are important COG: Comprehension Why can we not know the true score for a test? ANS: We cannot ever know the true score for a test because it is a theoretical reflection of the actual amount of the trait, characteristic, knowledge, or skill possessed by the individual PTS: DIF: Medium REF: Test Scores—Truth or Dare OBJ: How to compute and interpret various types of reliability coefficients COG: Comprehension When should we measure internal consistency reliability? ANS: You should use internal consistency when you want to know if the items on a test assess one—and only one—dimension PTS: DIF: Medium REF: Internal Consistency Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Comprehension When should you use construct validity? ANS: You should use construct validity when you want to know if a test measures some underlying construct or idea behind a measurement tool PTS: DIF: Medium REF: Construct Validity OBJ: How to compute and interpret various types of validity coefficients COG: Comprehension 10 How reliability and validity work together? ANS: You need both to have a good measure First, you establish validity; then, you use other techniques to establish reliability However, a test cannot be valid if it is not reliable PTS: DIF: Medium REF: Validity and Reliability: Really Close Cousins OBJ: What reliability and validity are and why they are important COG: Comprehension 11 To use archery practice as a metaphor for reliability and validity, hitting the bull’s-eye of a target demonstrates _, whereas repeatedly hitting the target in the same small area demonstrates _ ANS: validity; reliability PTS: DIF: Medium REF: Validity and Reliability: Really Close Cousins OBJ: What reliability and validity are and why they are important COG: Application 12 For students’ own research projects, Dr Salkind cautions readers to use measurement tools that are already established as reliable and valid Give at least two reasons for this guidance ANS: Time consuming; instrument development is its own project, in addition to the hypothesis testing; could lead to disaster; by using an established tool, you can get on with hypothesis testing right away PTS: DIF: Medium REF: A Last, Friendly Word OBJ: What reliability and validity are and why they are important COG: Comprehension 13 In order to calculate the different types of reliability, _ and _ use Pearson’s product–moment correlations, introduced in Chapter 5, and _ and _ require the use of their own formulas ANS: parallel forms; test–retest; internal consistency; interrater PTS: DIF: Medium REF: Different Types of Reliability OBJ: How to compute and interpret various types of reliability coefficients COG: Knowledge ... the following types of reliability? a Interrater b Test retest c Parallel forms d Internal consistency ANS: B PTS: DIF: Medium REF: Test Retest Reliability OBJ: How to compute and interpret various... Time is called _ a Test retest reliability b Parallel forms reliability c Internal consistency reliability d Similar forms reliability ANS: A PTS: DIF: Medium REF: Test Retest Reliability OBJ:... Comprehension 30 The correlation between two forms of the same test is called _ a Parallel forms reliability b Parallel forms validity c Test retest reliability d Internal consistency reliability ANS: