Exam 2 STATS 212 : Study Guide

Reliability/Validity

1. Identify and Define Research Designs

1. Descriptive - describe a situation or behavior. It is used to answer questions of who,

what, when, where, and how is associated with a particular research question

2. Correlational - nonexperimental research in which the researcher measures two

variables and assesses the statistical relationship (i.e., the correlation).

3. Quasi-experimental -used to estimate the causal impact of an intervention on target

population without random assignment.

4. Experimental – A scientific approach in which the researcher manipulates one or more

variables, and controls and measures any change in other variable

5. Literature Review – In depth evaluation of previous research. It is a summary and

synopsis of an area of research.

6. Meta-analytic - Study design used to systematically assess previous research studies to

derive conclusions about that body of research

2. Understand the concepts and interpretation of:

1. Reliability - the overall consistency of a measure.

- A measure is said to have a high reliability if it produces similar results

under consistent conditions
- Scores that are highly reliable are accurate and consistent from one
testing occasion to another.
- Values range between 0.00 (much error) and 1.00 (no error)

Test retest: How stable a test is over time; obtained by administering the same test twice to a group of
individuals.

Parallel forms: Obtained by administering different versions of an assessment tool (both versions must
contain items that probe the same construct, skill, knowledge base, etc.) to the same group of
individuals.

Internal Consistency: used to evaluate the degree to which different test items produce similar results.

Interrater reliability: Has different judges or raters agree in their assessment decisions. Inter-rater
reliability is useful because human observers will not necessarily interpret answers the same way.
 2. Validity – Tool measure what it says it measure. Compare to tools that work.

Types of Validity:

 Content
- Most often used
- Established by peer review
 Criterion
- Future – predictive
- Establish by looking at other measures that are already valid
 Construct
- Constructs are theoretical notion (personality)
- Establish by correlating scores to some theorized outcome

3. Chronbach's Alpha – Special measure of internal consistency, the higher the values =
more confidence you cave have a test measure one thing.

 .8 -1.0 = very reliable

 .0 - .2 = very weak

3. Be able to explain how to establish the reliability and validity of measurements.

Reliability: Repeat the trial when you use of the instrument (stability)

Validity: Compare the results of the blood pressure machine to an proven accurate machine

4. Be able to explain the concept of classical test theory

Classical Test Theory: X = T + E

X= Observed Score

T = True Score, the score that you would receive if the test contained no error.

E = Error Score

5. Be able to explain the concept of test interpretation versus use

Test interpretation can be used to measure something, the use is to see if those scores are reliable.
The Normal Distribution

1. Central Limit Theorem, what it means, why is it important

Central Limit Theorem: As the sample size increases, the shape of the sampling distribution will
become more normally distributed.

 A population's parameter can be estimated from a sampling statistics.

2. z-score

1. Concept: Measures the # of standard deviation that an actual value lies from the mean
𝑟𝑎𝑤−𝑚𝑒𝑎𝑛
2. How to calculate: 𝑧 = 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝐷𝑒𝑣.

3. How to interpret: z-score means a location on the normal curve, z-score of 1= 1

standard deviation to the right of the mean, z-score of -2 = 2 standard deviation to the
left of the mean.

3. Area under the curve

1. Understand the difference between the z-score integers:

-1 to +1 = 68%

-2 to +2= 95%

-3 to +3 = 99%

2. Understand how to use a z-table to look up areas under the curve

4. Classical Test Theory

Definition: assumes that every person has a true score on an item or a scale if we can only
measure it directly without error
How it relates to error in measurement

CTT analyses assumes that a person’s test score is comprised of their “true” score plus some
measurement error.

Error:
 Is normally distributed
 Uncorrelated with true score
 Has a mean of Zero

5. Confidence interval

1. Definition: a range of a population values given the sample value, and that there is a
specified probability that the value of a parameter lies within it.

2. How to interpret

 If you took different samples of the mean, 95% of the time the mean would fall
within the range of the interval
 You are accepting a 5% chance that your measurement might not be
representative of the population

3. How to calculate a CI

 What you need to know

- Standard Error
- Reliability Coefficient
- Mean Score

4. How to interpret a CI

5. How to interpret standard error

 the larger the sample size, the smaller the standard error
 Shows how accurate and precise the sample is as an estimate of the population
parameter.
 6. ‘How to differentiate between two different CIs

95%CI = x ± (1.96 * sex )

99%CI = x ± (2.58* sex )

 by 1.96 to obtain an estimate of where 95% of the population sample means are
expected to fall in the theoretical sampling distribution.

7. Understand the concept of scaling and how they are constructed

Statistical Significance

The risk set by the researcher for rejecting a null hypothesis when it is true

1. Understand what it means

 Statistically significant is the likelihood that a relationship between two or

more variables is caused by something other than chance.
 Statistical hypothesis testing is used to determine whether the result of a
data set is statistically significant. This test provides a p-value,
representing the probability that random chance could explain the result;
in general, a p-value of 5% or lower is considered to be statistically
significant.

2. Know the differences between Case I and Case II research

Case 1: Used to determine if the group is from the general population

 Close to 100-The clients belong to the general population

 Not close to 100-The clients don’t belong to the general population

Case II: Answer the question, are two groups different from one another?

3. Hypothesis, definition and concepts

1. Null:
 Shows that any kind of difference or significance you see in a set of data is
due to chance.
 denoted by H0, is usually the hypothesis that sample observations result purely
from chance.
 2. Alternative:

 denoted by H1 or Ha, is the hypothesis that sample observations are influenced

by some non-random cause.

3. Understand what it means to fail to and reject the null hypothesis

1. Reject the null hypothesis (p-value <= alpha) and conclude that the alternative
hypothesis is true at the 95% confidence level (or whatever level you've selected).

2. Fail to reject the null hypothesis (p-value > alpha) and conclude that not enough
evidence is available to suggest the null is false at the 95% confidence level.

4. Error in measurement

1. Type I error

 To Reject a true null hypothesis when it is true

 Sometimes referred to as a false positive
 no difference between groups or relationships between variables

2. Type II error

 Fail to reject the null when it was false

 Sometimes referred to as a false negative

3. Understand the decision matrix and the dynamics between managing error and
correct decisions
 4. Significance levels

The risk set by the researcher for rejecting a null when it is true

1. Understand the concepts

 Also known as the p-value

 Corresponds to the probability that the results were obtained by chance alone.

2. Alpha level

 Also known as the critical region

 The specific level of the p-value that is significant is set by the alpha level
 This is where we get the phrasing “statistically” significant and goes back to the
standard normal curve
 Common levels are 0.10, 0.05, and 0.01
 Based on the type of research and your tolerance for an error

5. Understand the concept of power

 Power is the first indication of whether the study is “good”

 The power of a test is its ability to detect significant differences
 Defined as 1 – β
 A study with low power have a high probability of committing a type II error

Effect size-measure of strength between two variables in the study.

 Small ( < 0.10)

 Medium (0.10 to 0.30)
 Large (> 0.30)

How to increase power

 Increase the sample size

 Use large effect sizes
 Increase the alpha level