likelihood estimator
FIN 5309 - Estimation Methods - Part II
Maximum Likelihood Estimation
Guillaume Weisang
Clark University gweisang “at” clarku “dot” edu
Fall 2011
1
Introduction
In this handout, I cover some results of point estimation that are essential for an understanding and an enlightened usage of financial econometrics in our FIN 5309 course. Approachable accounts of point estimation theory are available in many textbooks on statistical inference such as (Casella and Berger, 2002) and (Mittelhammer,
1996) or textbooks on econometrics such as (Stock and Watson, 2011) or (Hamilton,
1994).
The rationale behind point estimation is quite simple. When sampling is from a population described by a probability density function (pdf)1 or a probability mass function (pmf)2 f (x|✓), knowledge of the parameter ✓ yields knowledge of the entire population. Hence, it is natural to seek a method of finding a good estimator of the point ✓, that is, a good point estimator. It may also be the case that some function of ✓, say ⌧ (✓), is of interest. The methods described below can also be used to obtain estimators of ⌧ (✓).
The following definition of a point estimator may seem vague. However, we do not want to eliminate any candidates from consideration.
Definition 1. A point estimator is any function W (X1 , . . . , Xn ) of a sample; that is, any statistic is a point estimator.
1
2
In the continuous case
In the discrete case.
1
2
2.1
Maximum Likelihood Estimation
The Likelihood Function
Definition 2 (The Likelihood Function). Let f (x|✓) denote the joint pdf or pmf of the sample X = (X1 , . . . , Xn ). Then, given that X = x is observed, the function of ✓ defined by
L (✓|x) = f (x|✓)
(1)
is called the likelihood function.
If X is a discrete random variable vector, then L (✓|x) = P✓ (X = x). If we compare the likelihood function at two parameter points and find that
P✓1 (X = x) = L (✓1 |x) > L (x|✓2 ) = P✓2 (X = x) , then the sample we actually
Maximum Likelihood Estimation
Guillaume Weisang
Clark University gweisang “at” clarku “dot” edu
Fall 2011
1
Introduction
In this handout, I cover some results of point estimation that are essential for an understanding and an enlightened usage of financial econometrics in our FIN 5309 course. Approachable accounts of point estimation theory are available in many textbooks on statistical inference such as (Casella and Berger, 2002) and (Mittelhammer,
1996) or textbooks on econometrics such as (Stock and Watson, 2011) or (Hamilton,
1994).
The rationale behind point estimation is quite simple. When sampling is from a population described by a probability density function (pdf)1 or a probability mass function (pmf)2 f (x|✓), knowledge of the parameter ✓ yields knowledge of the entire population. Hence, it is natural to seek a method of finding a good estimator of the point ✓, that is, a good point estimator. It may also be the case that some function of ✓, say ⌧ (✓), is of interest. The methods described below can also be used to obtain estimators of ⌧ (✓).
The following definition of a point estimator may seem vague. However, we do not want to eliminate any candidates from consideration.
Definition 1. A point estimator is any function W (X1 , . . . , Xn ) of a sample; that is, any statistic is a point estimator.
1
2
In the continuous case
In the discrete case.
1
2
2.1
Maximum Likelihood Estimation
The Likelihood Function
Definition 2 (The Likelihood Function). Let f (x|✓) denote the joint pdf or pmf of the sample X = (X1 , . . . , Xn ). Then, given that X = x is observed, the function of ✓ defined by
L (✓|x) = f (x|✓)
(1)
is called the likelihood function.
If X is a discrete random variable vector, then L (✓|x) = P✓ (X = x). If we compare the likelihood function at two parameter points and find that
P✓1 (X = x) = L (✓1 |x) > L (x|✓2 ) = P✓2 (X = x) , then the sample we actually
Bibliography: Wadsworth Group, 2nd edition, 2002. Princeton University Press, Princeton, New Jersey, 1994. Springer, 1996. The Addison-Wesley Series in Economics. Addison-Wesley, 3rd edition, 2011.