Brian Neelon R Programs

http://people.duke.edu/~neelo003/r/

R Examples
This page contains examples of R programs for running standard Bayesian regression models (linear regression, logistic/probit regression, random effects models, latent class models, etc.). A basic working knowledge of R and Bayesian computation is assumed. Note: Several of the programs below require you to download R packages such as mvtnorm, MCMCpack, etc. General Disclaimer: These programs are meant as heuristic aids for newcomers to Bayesian inference, not as exemplars of great code! Please email me if you find errors or have questions...thanks!

Useful References Bayesian Data Analysis by Andrew Gelman et al. A First Course in Bayesian Statistical Methods by Peter Hoff Bayesian Computation with R by Jim Albert Finite Mixture and Markov Switching Models by Sylvia Fruhwirth-Schnatter

R Programs Normal and Linear Regression Models 1. Normal Mean Gibbs.r: Gibbs sampler for normal model with unknown mean and variance 2. Normal Mean Metropolis.r: Normal mean model with Metropolis update 3. Bivariate Normal.zip: Two approaches to fitting a bivariate normal linear regression model. BivNorm condtional.r updates regression parameters from their univariate full conditionals. BinNorm Joint.r upates Y1,Y2 jointly -- this is easier code to follow, but much slower. 4. Random Intercept Mixed Model.r: Gibbs sampler for balanced-design mixed model with random intercept 5. Random Slope Mixed Model.r: Gibbs sampler for random slope model with unbalanced design. Centers out the random effects to improve mixing of fixed effects parameters, beta. Here is another version which updates beta after marginalizing out the random effects to further improve mixing. See Chib and Carlin (1999) Statistics and Computing for more information. Note: both programs are slow as n increases, due to inversion of large block diagonals. You may want to try a sparse matrix algorithm to improve mixing. Or, more crudely, you can simply update random effects one-at-a-time via a loop, as in this program. This is acutally a bit faster than the matrix-inversion programs for n>200 or so. But it's not the best solution. Random Slope Using MH: Uses MH for updating the random effects and is faster than the previous programs (although still not great). MH Function.r is the Metropolis function for udpating the random slopes, and Random Slope MH.r generates data and runs the MCMC. Logistic, Multinomial and Probit Regression Models 1. Probit Model for Binary Data.r: Gibbs sampler for probit regression using Albert and Chib (1993) data-augmentation method 2. Fixed Effects Logistic Regression.zip: Files for running basic logistic regression using Metropolis step. Contains two files: 1. Update Beta Fixed.r: function for updating beta (run this first) 2. Log1.r: Generates data and runs MCMC (either source in "Update Beta" function or run "Update Beta"

1 of 3

1/11/2013 4:02 AM

Brian Neelon R Programs

http://people.duke.edu/~neelo003/r/

first) 3. Multinomial Logit Model.r: Gibbs sampler for multinomial logit model using Metropolis algorithm. 4. Probit Random Intercept Model.r: Gibbs sampler for probit random intercept model using data-augmentation approach 5. Random Intercept Logistic Regression.zip: Files for running random intercept logistic regression using Gibbs and Metropolis steps. Contains three files: 1. Update Beta Random.r: function for updating beta (run this first) 2. Update b.r: function for updating random intercept b (run this second) 3. Log2.r: Generates data and runs MCMC (either source in "Update Beta" and "Update b" functions or run those first) 6. Bivariate Probit Model.zip: Various MCMC programs for fixed-effects bivariate (correlated) probit regression model. See "Read Me" file for details 7. Bivariate Probit Model with Random Effects.r: MCMC for random-intercept bivariate probit regression model. Count Models 1. Zero Inflated Poisson.r: Fixed effects zero-inflated Poisson (ZIP) regression using MH. 2. Spatial Poisson Hurdle Model.zip: R2WinBUGS and BUGS code for simulating and fitting the spatial Poisson hurdle model described here. See README.txt for file discriptions. Bayesian Two-Part Latent Class Model 1. Bayesian Two-Part.zip: R code for running latent-class two-part model as described in Neelon et al. (2011), with three latent classes and random intercepts within classes. Contains 8 files: 1. Three Class MCMC.r: This is the main program for implementing the MCMC algorithm. It calls the other programs, but, of course, you'll need to change the file path names; 2. Update Gamma.r, Update Alpha.r, etc: These are the five functions used in the Gibbs and MH steps; 3. simdata.txt: Data file 4. Make simdata.r: Creates simdata.txt 2. Growth Mixture Model for Blood Pressure and Birth Outcomes: Contains R code for simulating and fitting the Bayesian growth mixture model (GMM) desribed in Neelon et al. (2011) A Bayesian Growth Mixture Model to Examine Maternal Hypertension and Birth Outcomes.See README.txt for file discriptions. Finite Mixture and Dirichlet Process Models 1. Two-Component Finite Mixture.r: Gibbs sampler for two-component (or "two-class") finite mixture normal model with class-specific means and variances. Dirichlet prior on class-membership probabilities. The program also illustrates the so-called "label-switching" problem and uses order-constraint solution to resolve the issue. See Fruhwirth-Schnatter (2006) for a more general discussion. 2. Two-Class Bivariate Normal Mixture.zip: Zip file contains 5 programs: 1) makedata.r creates two-class bivariate normal data with a class-level covariate ("male"); 2) data.txt is the data file; 3) MCMC.r runs the mcmc using the conditional udpating scheme described above in BivNorm.zip; programs 4 and 5 are Update C.r and Update Gamma.r---these are functions for updating the latent class indicators and the class-specific regression parameters (gamma's) via MH. 3. Three-Component Dirichlet Process.r: Uses stick-breaking, Dirichlet process model to estimate three-component normal mixture density. For a nice review of DPs, and Bayesian nonparametrics in general, see this paper by David Dunson.

2 of 3

1/11/2013 4:02 AM

Brian Neelon R Programs

http://people.duke.edu/~neelo003/r/

Latent Factor Model Bayesian Latent Factor Model.r: Three-factor linear model with M=6 manifest variables. See Lopes and West (Statistica Sinica, 2004) for details. Bayesian Isotonic Regression 1. Bayesian Order-Restricted Inference for Categorical Predictors: Contains files for implemented Bayesian orderrestricted inference on categorical predictors, as described in Dunson and Neelon (2003). The ZIP file contains two R programs: the first is the minmax function used for imposing the order restriction, and the second simulates a linear model with three categorical covariates and applies the minmax monotonicity restriction as part of a Gibbs sampling algorithm. 2. Bayesian Isotonic Regression for Continuous Predictors: The program first generates normal data with a montone-increasing sigmoidal mean function. It then fits a Bayesian, piecewise-linear isotonic regression model with a fixed number of "free" knots (i.e., unknown knot locations). This approach provides a reasonable estimate the mean function, but it assumes that the number of knots is known; to account for uncertainty in the number of knots (K), one can re-fit the model under different choices of K and use model averaging. Alternatively, one can use reversible jump MCMC or a model comparison criterion, such as BIC or DIC, to choose the "optimal" number of knots. The method is outlined in this working paper. See Denison, Mallick and Smith (1998) and Holmes and Heard (2003) for a related approaches. Spatial Analysis for Areal Data 1. Normal ICAR Model.r: Generates and fits data for spatially varying normal data under intrinsic CAR (ICAR) model. To generate data, we set spatial smoothing parameter close to 1 (but not equal to 1) to ensure that the joint covariance of the spatial random effects is non-singular. The full condtionals are given here. 2. Bivariate CAR (biCAR) Model.zip: Generates and fits data for spatially varying bivariate normal data under intrinsic bivariate CAR (biCAR) model. The Gibbs sampling algorithm and data generation are in bicar.r, and the function for updating the spatial effects phi are in update phi.r. The full condtionals are given here. 3. Spatial Bivariate Probit (Biprobit) Model.zip: Generates and fits data for correlated binary data as described in Neelon et al. (2012) A spatial bivariate probit model for correlated binary data with application to adverse birth outcomes. More to come...

Return to Brian's home page

3 of 3

1/11/2013 4:02 AM