Maximum Likelihood Programming in R
Maximum Likelihood Programming in R
Marco R. Steenbergen
Department of Political Science University of North Carolina, Chapel Hill
January 2006
Contents
1 Introduction 2 Syntactic Structure 2.1 Declaring the Log-Likelihood Function . . . . . . . . . . . . . . . . . . . . . . . 2.2 Optimizing the Log-Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Output 4 Obtaining Standard Errors 5 Test Statistics and Output Control 2 2 2 4 5 5 7
1
1
Introduction
The programming language R is rapidly gaining ground among political methodologists. A major reason is that R is a flexible and versatile language, which makes it easy to program new routines. In addition, R algorithms are generally very precise. R is well-suited for programming your own maximum likelihood routines. Indeed, there are several procedures for optimizing likelihood functions. Here I shall focus on the optim command, which implements the BFGS and L-BFGS-B algorithms, among others.1 Optimization through optim is relatively straightforward, since it is usually not necessary to provide analytic first and second derivatives. The command is also flexible, as likelihood functions can be declared in general terms instead of being defined in terms of a specific data set.
2
Syntactic Structure
Estimating likelihood functions entails a two-step process. First, one declares the log-likelihood function, which is done in general terms. Then one optimizes the log-likelihood function, which is done in terms of a particular data set. The log-likelihood function and optimization command may be typed interactively into the R command window or they may be contained in a text file. I would recommend saving log-likelihood functions into a text file, especially if you plan on using them frequently.
2.1
Declaring the Log-Likelihood Function
The log-likelihood function is declared as an R function. In R, functions take at least two arguments. First, they require a vector of
Marco R. Steenbergen
Department of Political Science University of North Carolina, Chapel Hill
January 2006
Contents
1 Introduction 2 Syntactic Structure 2.1 Declaring the Log-Likelihood Function . . . . . . . . . . . . . . . . . . . . . . . 2.2 Optimizing the Log-Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Output 4 Obtaining Standard Errors 5 Test Statistics and Output Control 2 2 2 4 5 5 7
1
1
Introduction
The programming language R is rapidly gaining ground among political methodologists. A major reason is that R is a flexible and versatile language, which makes it easy to program new routines. In addition, R algorithms are generally very precise. R is well-suited for programming your own maximum likelihood routines. Indeed, there are several procedures for optimizing likelihood functions. Here I shall focus on the optim command, which implements the BFGS and L-BFGS-B algorithms, among others.1 Optimization through optim is relatively straightforward, since it is usually not necessary to provide analytic first and second derivatives. The command is also flexible, as likelihood functions can be declared in general terms instead of being defined in terms of a specific data set.
2
Syntactic Structure
Estimating likelihood functions entails a two-step process. First, one declares the log-likelihood function, which is done in general terms. Then one optimizes the log-likelihood function, which is done in terms of a particular data set. The log-likelihood function and optimization command may be typed interactively into the R command window or they may be contained in a text file. I would recommend saving log-likelihood functions into a text file, especially if you plan on using them frequently.
2.1
Declaring the Log-Likelihood Function
The log-likelihood function is declared as an R function. In R, functions take at least two arguments. First, they require a vector of