See the corresponding article in this issue, pp 312.

J Neurosurg 118:12, 2013 AANS, 2013

Editorial
Subarachnoid hemorrhage trials
Ali Zomorodi, M.D.,1 James E. Herndon II, Ph.D.,5,6 and John H. Sampson, M.D., Ph.D., M.H.Sc., M.B.A.15
1 Division of Neurosurgery, Department of Surgery; Departments of 2Pathology, 3Immunology, 4Radiation Oncology, and 6Biostatistics and Bioinformatics; and 5Preston Robert Tisch Brain Tumor Center, Duke University Medical Center, Durham, North Carolina

In this issue of the Journal of Neurosurgery, Ilodigwe et al.1 use advanced statistical techniques to understand the implications of dichotomizing outcomes in clinical studies whose outcomes are ordinal. Ordinal variables rank or order outcomes but do not measure the relative differences between outcomes. A classic example of an outcome that is an ordinal variable is the Glasgow Coma Scale (GCS) score. In many clinical studies, these ordinal variables are collapsed into 2 outcomes. Again, using the GCS as an example, we could dichotomize head injuries into severe or not severe based on a GCS score 8 or > 8, respectively. When such outcomes are dichotomized, information is lost and statistical comparisons lose power. The authors of the following paper attempt to make this point by analyzing several recent randomized clinical trials that were conducted in patients with subarachnoid hemorrhage (SAH). Clinical outcomes in cerebrovascular surgery are measured using well-established ordered scales such as the Glasgow Outcome Scale (GOS) or the modified Rankin Scale (mRS). For the purposes of statistical analysis, most trials collapse these scales into the 2 categories of favorable and unfavorable outcome. While such dichotomization facilitates data analysis, it has several problems, which were well described by Murray et al.3 in 2005. First, this approach creates an artificial threshold for efficacy that may not be clinically relevant. For example, a treatment that enables patients with Hunt and Hess Grade V SAH to consistently achieve an mRS score of 3 would not cross the boundary between favorable and unfavorable, but it would nonetheless be very worthwhile. Second, and more importantly, the dichotomous approach is based on the assumption that every patient within the control group has the same chance of attaining a favorable outcome. This assumption ignores the prognostic heterogeneity that is an essential feature of patients with aneurysmal SAH. The considerable work done to establish admission injury severity and prognostic scales in aneurysmal SAH has demonstrated that very few patients at either extreme of the admission injury severity scale would be expected to cross
J Neurosurg / Volume 118 / January 2013

the outcome boundary established for clinical trials. These patients would contribute very little to the overall power of a study and might even make the study insensitive to some meaningful outcomes. For example, as demonstrated by Machado et al.,2 if prognostic heterogeneity is ignored, a treatment would have to have an OR of 1.75 for a favorable outcome to be observed in a typical study powered to detect a 10% absolute increase in favorable outcomes. This may explain why so many interventions with promising preclinical performance have failed to demonstrate benefit in clinical trials. In their paper, Ilodigwe and colleagues review the individual patient data from 4 tirilazad trials, the CONSCIOUS-1 trial, and the ISAT. All of these trials used the fixed dichotomy approach for analyzing outcomes. Importantly, the tirilazad and clazosentan trials demonstrated significant improvement in vasospasm rates in the treatment arm without significant increases in the number of favorable clinical outcomes. As such, these trials raise the same questions described above and are well suited for reevaluation using alternative methods. The authors evaluated individual patient data from each trial using the traditional fixed dichotomy approach as well as the proportional odds model and the sliding dichotomy approach. The proportional odds model is an extension of logistic regression that allows the ordered categorical outcome to take on more than 2 possible outcomes. Within this model, all fixed dichotomizations are considered within individual logistic models under the assumption that treatment has the same effect within each dichotomization. The latter is a strong assumption that was violated for many of the analyses considered in their paper. The authors also used the sliding dichotomy approach proposed by Murray et al.,3 in which patients are grouped into strata (or tiers) according to baseline prognosis. Within each stratum, stratum-specific definitions of good and bad are used to dichotomize outcome. Treatment groups are then compared with respect to the number of good outcomes pooled across strata. A prognostic score is used to define strata, and the score is estimated from a logistic model that predicts the fixed dichotomized outcome traditionally used in the analysis of these studies as a function of baseline risk factors. Although the goal of using all data without discarding information is commendable, there are challenging issues associated with this approach that must be considered. First, given that the stratification definition is data driven and not obtained from external sources, there is concern about the ability to compare study results with other trials. Murray et al.3 also acknowledged concern from the regulatory perspective that the analysis plan is not completely prespecified and is driven by the trial data. Second, given that the use
1

Editorial
of fixed dichotomy results in a loss of information, it is not clear how the use of that outcome in the development of the prognostic model for stratification will affect the sensitivity of subsequent treatment comparisons using the sliding dichotomy. Murray and associates provide a more comprehensive discussion of the open issues associated with this analytical approach. In conclusion, Ilodigwe and colleagues should be congratulated on their reevaluation of how we statistically analyze randomized controlled trials that are important to our field. In this particular instance, only minor differences were found among fixed dichotomy, sliding dichotomy, and proportional odds models with regard to the magnitude and precision of the odds ratios within these clinical trials. And while these approaches did not help for these particular studies, the principles the authors espoused should be duly considered in analyzing outcomes in future clinical trials important to our field.
(http://thejns.org/doi/abs/10.3171/2012.5.JNS12730) Disclosure The authors report no conflict of interest. References 1. Ilodigwe D, Murray GD, Kassell NF, Torner J, Kerr RSC, Molyneux AJ, et al: Sliding dichotomy compared with fixed dichotomization of ordinal outcome scales in subarachnoid hemorrhage trials. Clinical article. J Neurosurg [epub ahead of print October 5, 2012. DOI: 10.3171/2012.9.JNS111383] 2. Machado SG, Murray GD, Teasdale GM: Evaluation of designs for clinical trials of neuroprotective agents in head injury. European Brain Injury Consortium. J Neurotrauma 16:11311138, 1999 3. Murray GD, Barer D, Choi S, Fernandes H, Gregson B, Lees KR, et al: Design and analysis of phase III trials with ordered outcome scales: the concept of the sliding dichotomy. J Neurotrauma 22:511517, 2005

and an online prognostic prediction tool is available.3 We would point out that the development of prognostic models for SAH has been relatively limited, especially compared to what has been done in head injury. In one of our early attempts, we found numerous factors associated with outcome, but the identified factors explained only about a third of the variation in outcome.5 Another limitation is that none of the SAH prognostic models is externally validated. We believe that our paper raises several interesting questions and points for further study. First, we were disappointed in the performance of some of the alternative statistical approaches. In the ISAT there was a significant benefit in coiling over clipping as measured on the dichotomous mRS.4 We hypothesized that alternative methods of analyzing the mRS would increase the effect size of coiling or the difference in outcome between the groups; however, we were disappointed to see a minimal effect. There are multiple possible reasons for this but a key one may be that the outcome scale itself is not optimal for this disease. Outcome scales that incorporate measures of the deficits present in patients with SAH, such as cognitive and executive dysfunction, must be developed.6,7 Second, as we conclude in our study, additional studies can be done to determine the conditions under which these alternative methods of analyzing outcome may perform better.
References 1.Hernandez AV, Steyerberg EW, Butcher I, Mushkudiani N, Taylor GS, Murray GD, et al: Adjustment for strong predictors of outcome in traumatic brain injury trials: 25% reduction in sample size requirements in the IMPACT study. J Neurotrauma 23:12951303, 2006 2. Machado SG, Murray GD, Teasdale GM: Evaluation of designs for clinical trials of neuroprotective agents in head injury. European Brain Injury Consortium. J Neurotrauma 16: 11311138, 1999 3. McHugh GS, Butcher I, Steyerberg EW, Marmarou A, Lu J, Lingsma HF, et al: A simulation study evaluating approaches to the analysis of ordinal outcome data in randomized controlled trials in traumatic brain injury: results from the IMPACT Project. Clin Trials 7:4457, 2010 4. Molyneux A, Kerr R, Stratton I, Sandercock P, Clarke M, Shrimpton J, et al: International Subarachnoid Aneurysm Trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised trial. Lancet 360:12671274, 2002 5. Rosengart AJ, Schultheiss KE, Tolentino J, Macdonald RL: Prognostic factors for outcome in patients with aneurysmal subarachnoid hemorrhage. Stroke 38:23152321, 2007 6. Schweizer TA, Al-Khindi T, Macdonald RL: Mini-mental state examination versus Montreal cognitive assessment: Rapid assessment tools for cognitive and functional outcome after aneurysmal subarachnoid hemorrhage. J Neurol Sci 316:137140, 2012 7. Schweizer TA, Macdonald RL: Brain hemorrhage: Assessing outcome after subarachnoid hemorrhage. Nat Rev Neurol 6: 427428, 2010 Please include this information when citing this paper: published online October 5, 2012; DOI: 10.3171/2012.5.JNS12730.

Response
R. Loch Macdonald, M.D., Ph.D.
Division of Neurosurgery, St. Michaels Hospital, Labatt Family Centre of Excellence in Brain Injury and Trauma Research, Keenan Research Centre of the Li Ka Shing Knowledge Institute of St. Michaels Hospital, and Department of Surgery, University of Toronto, Ontario, Canada

We thank Zomorodi and colleagues for their comments on our study published in the Journal of Neurosurgery. They mention the value of prognostic models for estimating prognostic heterogeneity and using prognostic models to select patients for clinical trials as well as to modify the method of measuring outcome.2 Excellent prognostic models have been developed in some neurosurgical fields, such as traumatic brain injury.1 The IMPACT (International Mission for Prognosis and Analysis of Clinical Trials in TBI) collaboration has published numerous papers on prognostic modeling for traumatic brain injury,

J Neurosurg / Volume 118 / January 2013