STAT 543 Homework

STAT 543 Homework 4 Solution

Feb. 10th, 2005

STAT 543 Homework 4 Solution
1. Problem 2.1.2 Consider n systems with failure times X 1 ,..., X n assumed to be independent and identically distributed with exponential, Σ(λ ) , distributions. (a) Find the method of moments estimate of λ based on the first moment. (b) Find the method of moments estimate of λ based on the second moment. (c) Combine your answers to (a) and (b) to get a method of moment estimate of λ based on the first two moments. (d) Find the method of moments estimate of the probability P( X 1 ≥ 1) that one system will last at least a month. Solution: Since X 1 ,..., X n i.i.d Ε(λ ) , then
∼

f ( x1 ) = λ e − λ x1 , x > 0

, and
E ( X 1 ) = ∫ xλe − λ x dx =
0 ∞

Γ(2)

λ

=

1

λ
2
, and
E ( X 12 ) = ∫ x 2 λ e − λ x dx =

. λ λ2 (a) The method of moments estimate of λ based on the second moment is 1 n λ= = n x ∑ xi
2

Γ(3)

=

(b) The method of moments estimate of λ based on the second moment is 2n λ= n ∑ i=1 xi2

i =1

(c) From λ 2 = 2λ 2 − λ 2 = µ 2 − µ12 , the method of moments estimate of λ based on the first two moments is 1 1 . λ= = 1 n 2 1 n 2 1 n ∑ xi − x 2 n ∑ i=1 xi − ( n ∑ i =1 xi )2 n i =1 (d) Since P( X 1 ≥ 1) = λ ∫ e− λ x dx = e− λ , then the method of moments estimate
1 ∞

of P( X 1 ≥ 1) that one system will last at least a month is ∑ xi − i −x P ( X 1 ≥ 1) = e = e n .

-1-

STAT 543 Homework 4 Solution

Feb. 10th, 2005

Problem 2.1.3 Suppose that i.i.d . X 1 ,..., X n have a beta, β (α1 , α 2 ) distribution. Find the method of moments estimates of α = (α1 , α 2 ) based on the first two moments. Hint: See Problem B.2.5. Solution: Known that for Beta (α1 , α 2 ) distribution, there exist
E( X ) =

α1 α1 + α 2

, and

E ( X 2 ) = (α +α )(α +α +1) 1 2 1 2

α1(α1+1)

(µ2 − µ1 )µ1 ⎧ ⎧ µ = α1 1 ⎪ α1 = µ 2 − µ ⎪ α1 + α 2 2 ⎪ ⎪ 1 ⇒⎨ ⎨ α1(α1 + 1) ⎪ µ2 = ⎪α = (µ1 − µ2 )(µ1 − 1) (α1 + α 2 )(α1 + α 2 + 1) ⎪ ⎪ 2 µ 12 − µ2 ⎩ ⎩ 1 n 1 n 2 Then, plug in ∑ x i , and ∑ x i for µ1 and µ2 , we get the method of moment estimate n 1 n 1 for α = (α1, α 2 ) .as 1 1 1 ⎧ ( ∑ i x i2 − ∑ i x i ) ∑ i x i ⎪ n n n ⎪ α1 = 1 1 2 ( ∑ i x i ) − ∑ i x i2 ⎪ ⎪ n n . ⎨ 1 1 1 ⎪ ( ∑ i x i 1 − ∑ i x i2 )( ∑ i x i − 1) n n ⎪α = n 1 1 ⎪ 2 2 ( ∑ i x i ) − ∑ i x i2 ⎪ n n ⎩

, then by

Problem 2.1.11 In Example 2.1.2 with X ∼ Γ (α , λ ) , find the method of moments estimate based ˆ ˆ on µ1 and µ3 . Hint: See Problem B.2.4. Solution:

-2-

STAT 543 Homework 4 Solution For gamma (α , λ ) distribution, f (x ) =

Feb. 10th, 2005

α ⎧ Ex = µ1 = ⎪ λ ⎪ ⇒⎨ Γ (α + 3 ) α (α + 1)(α + 2) ⎪Ex 3 = µ3 = = ⎪ Γ (α ) λ 3 λ3 ⎩
⇒ −3µ13 + µ16 + 8 µ13 µ3 α = 2(µ13 − µ3 )

λ α α −1 − λx x e ,x > 0 Γ (α )

2 −3µ1 + µ14 + 8 µ1 µ3 2(µ13 − µ3 ) 1 n 1 n 3 By plug in ∑ x i and ∑ x i for µ1 and µ3 , we get the MOM estimates for α and λ as n 1 n 1

, and λ =

⎛1 n ⎞ ⎛1 n ⎞ ⎛1 n ⎞ 1 n −3 ⎜ ∑ x i ⎟ + ⎜ ∑ x i ⎟ + 8 ⎜ ∑ x i ⎟ ( ∑ x i3 ) ⎝n 1 ⎠ ⎝n 1 ⎠ ⎝n 1 ⎠ n 1 ˆ α = 3 1 n ⎛1 n ⎞ 2(⎜ ∑ x i ⎟ − ∑ x i3 ) n 1 ⎝n 1 ⎠
, and

3

6

3

⎛1 n ⎞ ⎛1 n ⎞ ⎛ 1 n ⎞⎛ 1 n 3 ⎞ −3 ⎜ ∑ x i ⎟ + ⎜ ∑ x i ⎟ + 8 ⎜ ∑ x i ⎟ ⎜ ∑ x i ⎟ ⎝n 1 ⎠ ⎝n 1 ⎠ ⎝n 1 ⎠⎝n 1 ⎠ ˆ= λ .
3 ⎛1 n ⎞ ⎛1 n 3⎞ 2(⎜ ∑ x i ⎟ − ⎜ ∑ x i ⎟) ⎝n 1 ⎠ ⎝n 1 ⎠

2

4

Problem 2.2.10 Let X 1 ,..., X n denote a sample from a population with one of the following densities or frequency functions. Find the MLE of θ . (a) f ( x, θ ) = θ e −θ x , x ≥ 0;θ > 0. (Exponential density) (b) f ( x, θ ) = θ cθ x − (θ +1) , x ≥ c; c constant>0; θ > 0. (Pareto density) (c) Solution: (a).

−θ x f (x | θ ) = θ ne ∑ i = L(θ )

(θ ) = n log(θ ) − θ ∑ x i

-3-

STAT 543 Homework 4 Solution

Feb. 10th, 2005

d n ˆ ∑ xi = − ∑ xi = 0 ⇒ θ = dθ θ n n ˆ Check that ′′ = − 2 < 0 , which indicates that θ reaches the maximum of .
'

=

ˆ Hence, θ =

∑x n θ

i

is the MLE of θ . f (x | θ ) = θ nc nθ ∏ x i
−(θ +1)

(b).

(θ ) = n log θ + nθ log c − (θ + 1)∑ log x i
1
'

n

=

d n = + n log c − ∑ log x i = 0 dθ θ

ˆ ⇒θ =
Check that ′′ = −
ˆ Hence, θ = n

∑ log x
1

n

i

− n log c

n

∑ xi n θ

2

ˆ < 0 , which indicates that θ reaches the maximum of .

is the MLE of θ .
−(c +1)

(c) f (x | θ ) = c nθ nc ∏ x i ⋅ 1 [ min(x i ) ≥ θ ] , c > 0; θ >
0

ˆ ⇒ θ = min(x i ) is the MLE of θ , since θ nc is an increasing function of θ . (d)

f (x | θ ) = θ 2 ∏ x i n = log θ + ( θ − 1)∑ log x i 2 n 1 ' = + ∑ log x i = 0 2θ 2 θ θ −1

n

⎛ ∑ log x i ⎞ ˆ ⇒θ =⎜ ⎟ ⎜ ⎟ n ⎝ ⎠ n 1 Check that ′′ = − 2 − 3 ∑ i log xi < 0 . 2θ 4θ 2 ⎛ ∑ log x i ˆ So, θ = ⎜ ⎜ n ⎝ (e) ⎞ ⎟ is the MLE of θ . ⎟ ⎠
−2n

−2

−2

f (x | θ ) = θ

⎛ ∑ x i2 ⎞ ( ∏ x i ) ⋅ exp ⎜ − 2θ 2 ⎟ ⎜ ⎟ ⎝ ⎠

-4-

STAT 543 Homework 4 Solution

Feb. 10th, 2005

∑x = −2n log θ −
'

+ ∑ i log x i 2θ 2 2 2n ∑ x i =− + =0 3

2 i

θ

θ

ˆ ⇒ θ =
Check that ′′ =

∑x
−
2 i 2 i 2n

θ2
2n

2n 3∑ i xi2

θ4

3∑ i xi 4n 2 12n 2 8n 2 ˆ 2n and ′′(θ ) = 2 − = − =− x 2 , since a 2 + b 2 ≤ 2ab ; and the equality holds if a=b. x + x2 . So, the MLE of θ1, θ2 is θ1 = θ2 = 1 2

Problem 2.2.21 (Kiefer-Wolfowitz) Suppose ( X 1 ,..., X n ) is a sample from a population with density
9 ⎛ x−µ ⎞ 1 ϕ⎜ ⎟+ ϕ (x − µ) 10σ ⎝ σ ⎠ 10 , where ϕ is the standard normal density and f ( x, θ ) =

θ = ( µ , σ 2 ) ∈ Θ = ( µ , σ 2 ) : −∞ < µ < ∞, 0 < σ 2 < ∞ .

{

}

Show that maximum likelihood estimates do not exist, but that ˆ supσ p( x, µ , σ 2 ) = sup µ ,σ p( x, µ , σ 2 )
ˆ if, and only if, µ equals one of the numbers x1 ,..., xn . Assume that xi ≠ x j for i ≠ j and

that n ≥ 2 . Solution: ˆ Let µ = x i , 1≤i ≤n (a). Consider σ 2 → 0 across line = {( x1 , σ 2 ) : σ 2 > 0} ⊂ Θ , then
1 ⎛ 9 1 1⎞ + ⎟→∞ ⎜ 2π ⎝ 10 σ 10 ⎠ , as σ 2 → 0 . And also ( x − x )2 ( x − x )2 1 ⎛ 9 1 − 12σ 2i 1 − 1 2 i ⎜ fi ( xi , θ ) = e e 10 2π ⎜ 10 σ ⎝ 2 , as σ → 0 . So, L ( x | θ ) → ∞ , when σ → 0 . f1 ( x1 , θ ) =

( x − x )2 ⎞ 1 1 − 1 2i ⎟→ e ⎟ 2π 10 ⎠

- 12 -

STAT 543 Homework 4 Solution

Feb. 10th, 2005

And thus the likelihood function is unbounded, which means that the MLE ( µ , σ 2 ) in Θ doesn’t exist. (b). From above, we have ˆ supσ p ( x , µ, σ ) = s upσ ,µ p ( x , µ, σ 2 ) = ∞
ˆ If supσ p ( x, µ , σ ) = 0 , then µ must equal to some xi . ˆ If µ ≠ x i, ∀ i.st 1 ≤ i ≤ n , then by

M exp ⎛ − ⎞ → 0 ⎜ ⎟ σ ⎝ σ ⎠ , as σ → 0 , where 2 M = max ( x i − µ ) , 1 i {

}

ˆ ˆ supσ p ( x , µ, σ ) < ∞ , if µ ≠ x i . ˆ So, µ must equal to some xi .

, and hence

- 13 -