Ridwan Amiruddin

Assessing Evidence From Intervention Studies

Assesment evidence arising from individual studies
The evidence base for evaluating the option is often

weak Evaluating the quality of evidence originating from research studies.

Cecklist for Apprising randomized controlled trials for use evidence based health care
Are the results of the trial valid Did the trial addres a clearly focussed issue, in terms of population studied Was the assigmnet of patents to treatment concealed Were the group similat antara the start of the trial What were the results How large was the treatment effect How precise was the estimate of the treatment effect How applicable were the research findings How wide were the confidence interval What were the excluison dan inclusion criteria Could the quality of service provided in the trial be reproduced locally

Checklist for appraising review articles for use in evidence based health care
Are the results of the review valid?
Did the review address a focused issue?

Did the authors look for the appropriate sort of papers?

Were the important, relevant studies included? Did the review's authors do enough to assess the quality of the included studies? If the results of the review have been combined, was it reasonable to do so?

What are the results?

What is the overall result of the review? How precise are the results?

Will the results help locally?

Can the results be applied to the local population? Were all important outcomes considered? Are the benefits worth the harms and costs?

Additional criteria for assessing a meta-analysis

Was the searching technique limited to an electronic

search of MEDLINE? (Undesirable) Are the results of the trials all or mostly pointing in the same direction? (Desirable) Are the trials in the meta--analysis all small trials? (Very undesirl able)

Proposed criteria for determining whether an intervention should be included in a health policy
Efficacy: Is the intervention known to work under optimal

conditions? Efectiveness: Is the intervention known to work under normal conditions? Applicability: Is the intervention likely to be effective in the target population for the proposed policy? Efficiency. Could the money be spent more productively on other interventions? Feasibility: Can the intervention be implemented, given the sociopolitical context? Potential "rage: Can the intervention reach the whole target population?

propose the following list for use in development of clinical guidelines - meta-analyses involving analysis of individual data
1.

Supportive evidence from well-conducted RCTs that included 100 patients or more, such as multicenter trial or meta-analysis with quality ratings

2. 3. 4. 5. 6. 7.

Supportive evidence from well-conducted RCTs or metaanalysis with quality ratings, with fewer than 100 patients Supportive evidence from well-conducted cohort studies, such as prospective or retrospective studies, or meta-analysis thereof Supportive evidence from a well-conducted case-control study Supportive evidence from poorly controlled or uncontrolled studies (e.g., significantly flawed RCTs, observational studies with high potential for bias, or case series or reports) Conflicting evidence with weight of evidence supporting the recommendation Expert opinion.

When should a new technology be adopted

1.
2.

3. 4. 5.

Compelling: the new technology is equally or more effective and less costly than the current technology. Strong: the new technology is more effective and costs less than $20,000/QALY gained, or the new technology is less effective but saves more than $100,000/QALY lost through its adoption. Moderate: the new technology is more effective and costs $20,000 $100,000/QALY gained, or the new technology is less effective and saves $20,000-$100,000/QALY lost. Weak: the new technology is more effective but costs more than $ 100,000 /QALY gained, or the new technology is less effective and saves less than $20,000/QALY lost. Compelling evidence for rejection: the new technology is less or equally effective, but is more costly than the technology currently in use.

method for grading health care recommendations

1.
2.

How strong is the evidence? Evaluation is based on: Quality of evidence (RCT vs observational studies)
Presence or absence of important heterogeneity

How big an impact of treatment warrants its use? Evaluation is based on the threshold number needed to treat (TNNT), which is calculated from
1. 2.

Cost of treating the condition and the adverse effects of the therapy Values (economic burden) of the target outcomes and adverse events

3.

How much does the treatment work? The decision is based on

Relationship of observed NNT to threshold TNNT