Bayesian Methods

Date: t.b.a.

Topics

• Introduction and Motivation
• An example: Linear Regression
• Finding posteriors
  • Analytic
    • Conjugate Priors
  • Metropolis-Hastings
  • Gibbs
  • Checking convergence
• Applications:
  • Robust regression
  • Discrete Choice
  • Instrumental Variables
  • Errors in Variables
  • Interval regression
  • Hierarchical Models
• Nonparametric Methods?
• Identification
• Model Comparison
• Exercises

Handout

If you want to prepare for the lecture (or revise), you can have a look at the Handout (this is a preliminary version only, please expect changes during the next few weeks).

Exam:

Here you will find the exam questions on Saturday, 11:00.

Exercises

... can also be found in the appendix of the handout.

Literature:

• John K. Kruschke , Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. Academic Press, 2nd Edition, 2014.
• Hoff, A First Course in Bayesian Statistical Methods. Springer, 2009.

To learn more about MCMC sampling you can read the following article:

• Christophe Andrieu, Nando de Freitas, Arnaud Doucet, Michael I. Jordan. “An Introduction to MCMC for Machine Learning.” Machine Learning, 2003, 50(1-2), pp 5-43.

  The paper can also be found on the homepage of the Arnaud Doucet.

Software

For our practical examples (during the entire course) we will use the software environment R. I think that it is helpful to coordinate on one environment. R is free, it is very powerful, and it is popular in the field.

• Documentation for R is provided throught the built in help. You also find support on the R Homepage. You might find the following useful:
  • The R Guide, Jason Owen (Easy to read, explains R with the help of examples from basic statistics)
  • Simple R, John Verzani (Explains R with the help of examples from basic statistics)
  • Einführung in R, Günther Sawitzki (In German. Rather compact introduction.)
  • Econometrics in R, Grant V. Farnsworth (The introduction to R is rather compact and pragmatic.)
  • An Introduction to R, W. N. Venables und D. M. Smith (The focus is more on R as a programming language)
  • The R language definition (Concentrates only on R as a programming language.)
  • On the JAGS Homepage you go to the files pages, then to Manuals, to find the JAGS user manual.
• You can download R from the homepage of the R-project.

  Installing R with Microsoft Windows:

  Download and start the Installer. Install R on your local drive. Installing on a network drive or in the cloud (Dropbox, Onedrive,...) is possible but not recommended.

  Installing R with GNU-Linux:

  Follow the advice to install R for your distribution.

  Installing R with MacOS X:

  Here is a guide to install R with MacOS X.

• In the lecture we use RStudio as a front end.
• For the Bayesian parts we will use JAGS. It helps if you have installed R, RStudio, and JAGS on your computer when we start the course.
• We will use the following packages: runjags, coda, boot, Ecdat, boot, bootstrap, survival, lattice, latticeExtra, lme4, lmtest, quantreg, xtable, plyr, reshape2, sampleSelection.

  If, e.g., the command library(Ecdat) generates an error message (Error in library(Ecdat): There is no package called 'Ecdat'), you have to install the package.

  Installing packages with Microsoft Windows:

  With RStudio: Use the tab “Install”. Otherwise: Start Rgui.exe and install packages from the menu Packages / Install Packages).

  Installing packages from GNU-Linux or MacOS X:

  From within R use the command install.packages("Ecdat"), e.g., to install the package Ecdat