Professional Documents
Culture Documents
Sample population
Sampled population
Populasi sampel
Êarget population = domain = population in which the
results of the study will be applied. Usually character-
ized by demographic & clinical characteristics; e.g.
normal infants, teens with epilepsy, post-menopausal
women with osteoporosis.
Accessible population = subset of target population
which can be accessed by the investigator. Frame: time
& place. Example: teens with epilepsy in RSCM, 2000-
2005; women with osteoporosis, 2002 RSGS
Intended sample = subjects who meet eligibility criteria
and selected to be included in the study
Actual study subjects = subjects who actually
completed the participation in the study
!"#
%
$
$%
$%
External validity II:
Does AP represent ÊP?
Accessible
population
[External validity I:
Does IS represent AP?}
% $
t= df = 9998 p = 0.00002
$
Clinical importance vs. statistical significance
Cured Died
Standard Rx 0 10 (100%)
'ew Rx 3 7 (70%)
C 30 (60%) 20 (40%) 50
E 40 (80%) 10 (20%) 50
X2= ; df = 1; p = 0.0432
|
|
'ote:
p value ´ only for analytical studies
CI ´ for descriptive and analytical studies
How to calculate CI
General Formula:
CI = p ÷ Z x SE
þ
þ
'-|
& &
± ' & ± ' &
'
& ± &
â
)
&
'
)
þ ± & ( (
( ( ± (
"
|
Primary sources
² Based on experiments and published
research
Secondary sources
² Systematic reviews
² Clinical guidelines
² Journals of secondary publication e.g.
Evidence Based Medicine
´5Sµ Pyramid of Evidence Resources
Levels of evidence
V
V
V
V
Êypes of Evidence - Question Êypes
6 [þ+
! + - 6+-
treatment, prevention Systematic Review of RCÊs or RCÊ
!
þ 6+- Observational
Study - Cohort or Case Control
& 6+- Observational
Study - Cohort, Case Control
!
6+-
Comparison to Gold Standard
þ
6+-
Cost-effectiveness Study
# 6+-case study,
Key quality parameters
Validity
Reliability
Importance
Validity: internal and external
Evaluate
Efficiently track
performance
down best
available
evidence Implement
changes in clinical
Critically review the practice
validity and usefulness
of the evidence
Check list for medical literature
(completeness)
1. Êitle
2. Authors
3. Abstract: structured? Informative? Abbreviation?
4. Introduction: length? Relevant references? Êarget population?
5. Methods:
Design
Eligibility (inclusion and exclusion) criteria
Sample size, sampling method
Randomization: technique, concealment
Intervention: masking?
Measurement: blinding? - Primary & secondary outcome
Definitions
Analysis
|4
6. Results
Baseline characteristics
Main outcome
Secondary outcome
7. Discussion
General
Strength and weakness
Conclusions
8. References
Vancouver style
Constant
9. Acknowledgments
10. Ethics approval
11. Conflict of interest
What to assess?
(in study of cause-effect relationship)
A. General description
² Êype of design
² Êarget population, source population,
sample
² Sampling method
² Dependent and independent variables
² Main results?
What to assess?
(in study of cause-effect relationship)
Êypes of bias
1. Sample (subject selection) biases, which may result in the
subjects in the sample being unrepresentative of the
population which you are interested in
2. Measurement (detection) biases, which include issues
related to how the outcome of interest was measured
3. Intervention (performance) biases, which involve how the
treatment itself was carried out.
What to assess?
(in study of cause-effect relationship)
C. Internal validity, causal relationship
Êemporality (cause precedes effect)
Strength of association (large difference, RR, OR, etc) or
small p value or narrow confidence interval
Biological gradient (dose dependence)
Consistency among studies (diff. populations/designs)
Specificity (certain factor results in certain effect)
Coherence (does not conflict with current knowledge)
Biological plausibility: can be explained with current
knowledge (at least in part)
What to assess?
(in study of cause-effect relationship)
D. External validity
Applicable to study subjects
Applicable to source population
Applicable to target population
11 items, each with 3 sections
² Is it concerned with the impact of an
intervention, causality or determining the
magnitude of a health problem?
!
² Is it a well stated research
question/hypothesis?
2. What is the study type?
!
² Is the study type appropriate to the research
question?
!
² If not, how useful are the results produced
by this type of study?
3. What are the outcome factors and how
are they measured?
!
² a) are all relevant outcomes assessed
² b) is there measurement error?
!
² a) how important are omitted outcomes
² b) is measurement error an important source
of bias?
4. What are the study factors and how are
the measured?
!
² Is there measurement error?
!
² Is measurement error an important source of
bias?
5. What important potential
confounders are considered?
!
² Are potential confounders examined and
controlled for?
!
² Is confounding an important source of bias?
6. What are the sampling frame and
sampling method?
!
² Is there selection bias?
!
² Does this threaten the + of
the study?
7. Questions of internal validity
!
² Experimental: how were the subjects
assigned to groups?
² Longitudinal study, how many reached
follow-up?
² Case control study, are the controls
appropriate?
'ote: other issues of relevance to internal
validity are considered under the other
headings in this critical appraisal system. You
can add your own questions, and also design
your own questions for other study types such as
cross sectional studies and systematic reviews
!
8. Are statistical tests considered?
!
² Were the tests appropriate for the data?
² Are confidence intervals given?
² Is the power given if a null result?
² In a trial, are results presented as absolute
risk reduction as well as relative risk
reduction?
!
² If not, how useful are the results?
9. Are the results clinically/socially
significant?
!
² Was the sample size adequate to detect a
clinically/socially significant result?
² Are the results presented in a way to help in
health policy decisions?
!
² Is the study useful?
10. Are ethical issues considered?
!
² Does the paper indicate ethics approval?
² Can you identify potential ethical issues?
² Are the results or their application
compromised?
11. What conclusions did the authors reach
about the study question?
!
² Do the results apply to the population in
which you are interested?
² Will you use the results of the study?
Appraisal Êools
Êools from the Critical Appraisal Skills Programme
(CASP)
² Systematic Reviews
² Randomised Controlled Êrials
² Qualitative Research Studies
² Cohort Studies
² Case-Control Studies
² Diagnostic Êest Studies
² Economic Evaluation Studies
Available at:
http://www.phru.nhs.uk/casp/critical_apprai
sal_tools.htm
Study Designs Recap
þ
+
Randomised Controlled Êrial
9[
Qualitative (Interviews,
Observations, etc)
Critical appraisal
- Validity Methods
- Importance Results
- Applicability Discussion
Ê