What can you say about the dots on the dartboard?

Measures of Validity
Definition of Validity

Degree to which a test or instrument measures what it purports to

measure
- Thomas & Nelson 1996

The soundness or appropriateness of a test or instrument in

measuring what it is designed to measure
- Vincent 1999

Some Helpful Terms

Construct
The trait or characteristic that interests us. We might call it a
target or what we want to get at. We create a test to
cover this attribute.
Validity addresses how well an assessment technique
provides useful information about the construct / target.
Internal validity
Is the experimenter measuring the effect of the independent
variable on the dependent variable?

External validity
Can the results be generalised to the wider population?
3

Important distinction!
The term validity is used in two different ways

1. Validity of an assessment or method of collecting

data
The validity of a test or questionnaire or
interview
Example
Math test
is it covering the right content and concepts?
is it also influenced by reading level or background
knowledge?
Attitude assessment
are the questions appropriate?
does it assess different dimensions of attitudes
(intensity, direction, etc.)

Important distinction!
The term validity is used in two different ways

2. Validity of a research study

Was the entire study of high quality
Did it have high internal and external validity

Validity is also assessed in a particular context

A test may be valid in some contexts and not in
others

A questionnaire may be useful with some populations

and not so useful with other groups

2 Types of validity
1. Criterion-related validity
2. Construct validity

2 Types of validity

1. Criterion-related validity

It is the extent to which peoples scores are correlated with

other variables or criteria that reflect the same construct.

Examples:

An IQ test should correlate positively with school performance.

An occupational aptitude test should correlate positively with

work performance.

2 Types
1a. Concurrent validity
1b. Predictive validity

Types of validity

2 Types of Criterion-related validity

1a. Concurrent validity
forecasting the present
how well does a test predict current similar
outcomes
job samples, alternative tests used to
demonstrate concurrent validity evidence

2 Types of Criterion-related validity (cont)

Concurrent Validity
Infers that the test produces similar results to a previously validated
test
e.g. The VO2max test is the criterion measure of aerobic power in athletes

Incremental Treadmill Protocol

with expired gas analysis

Multi-Stage Fitness (Beep) Test

2 Types of Criterion-related validity (cont)

1b. Predictive validity
When the criterion is something that will happen or be
assessed in the future, this is called predictive validity.
Examples:
how well does a test predict future outcomes
SAT (Scholastic Aptitude Test) predicting 1st yr GPA
(Grade Point Average)
GRE (Graduate Record Examination) scores and graduate
school performance

2 Types of Criterion-related validity (cont)

Example of Predictive Validity
Can performance during test A be used to predict future performance
in test B?

2 Types of validity (cont.)

2. Construct validity
- The extent to which an assessment corresponds to other
variables, as predicted by some rationale or theory.
- Infers not only that the test is measuring what it is supposed to, but
also that it is capable of detecting what should exist, theoretically
- Therefore relates to hypothetical or intangible constructs

2 Types of validity (cont.)

2. Construct Validity
When we ask about a tests construct validity, we are
taking a broad view of the test.
Does the test adequately measure the underlying
unobserved construct?

Construct validity is based on the accumulation of

knowledge about the test and its relationship to other tests
and behaviors.

2 Types of validity (cont.)

2. Construct Validity
Examples
If you can correctly hypothesize that ESOL (English for speakers of
other languages) students will perform differently on a reading test
than English-speaking students (because of theory), the
assessment may have construct validity.

Statistical tools that measure validity

Pearson Product Moment correlation
- to determine the correlation, the measure of the
strength of the relationship between two variables
used to compare individual scores
r-value = correlation coefficient
also called as validity coefficient
- statistical expression of the relationship between the
two variables