ST5215: Advanced Statistical Theory 2014/2015: Semester I Tutorial 7 1. Show that {Pθ : θ ∈ Θ} is an exponential family and find its canonical form and natural parameter space, when (i) Pθ is the Poisson distribution P (θ) : θ ∈ Θ = (0, ∞); (ii) Pθ is the negative binomial distribution N B(θ, r) with a fixed r, θ ∈ Θ = (0, 1); (iii) Pθ is the exponential distribution E(a, θ, ) with a fixed a, θ ∈ Θ = (0, ∞); (iv) Pθ is the gamma distribution Γ(α, γ), θ = (α, γ) ∈ Θ = (0, ∞) ⊗ (0, ∞); (v) Pθ is the beta distribution B(α, β), θ = (α, β) ∈ Θ = (0, 1) ⊗ (0, 1); (vi) Pθ is the Weibull distribution W (α, θ) with a fixed α > 0, θ ∈ Θ = (0, ∞). Solution: (i) The p.d.f. of the Poisson distribution can be expressed as θx e−θ /x! = exp{x ln θ − θ}(x!)−1 I{0,1,2,...,} (x), which has the form of an exponential family. η = ln θ is the natural parameter and the natural parameter space is (−∞, ∞). (iii) The exponential distributions with fixed a has the p.d.f. θ−1 e−(x−a)/θ I(a,∞) (x) = exp{−θ−1 (x−a)−ln θ}I(a,∞) (x), = exp{η(x−a)+ln(−η)}I(a,∞) (x) which is in the form of an exponential family. The natural parameter is η = −θ−1 . The natural parameter space is Ξ = (−∞, 0). (v) The beta distribution has the p.d.f. Γ(α + β) Γ(α + β) α−1 x (1−x)β−1 I(0,1) (x) = exp{α ln x+β ln(1−x)+ln }[x(1−x)]−1 I(0,1) (x), Γ(α)Γ(β) Γ(α)Γ(β) which is in the canonical form of an exponential family. The natural parameter is θ = (α, β). The natural parameter space is Θ = (0, 1) ⊗ (0, 1). 1 2. Show that {Pθ : θ ∈ Θ} is not an exponential family, when (i) Pθ is the exponential distribution E(a, θ) with two unknown parameters a, and θ; (ii) Pθ is the negative binomial distribution N B(θ, r) with two unknown parameters r and θ. Solution: (i) If E(a, θ) is an exponential family, then E(a, θ) has a positive density (1) exp{η(a, θ)τ T (x) − ξ(a, θ)} with respect to a non-zero measure ν. Consider the interval (−∞, t) for any t ∈ R. There is an a ∈ R such that a > t, hence P(a,θ) [(−∞, t)] = 0. This together with (1) implies that ν[(−∞, t)] = 0. Since t is arbitrary, ν must be a zero measure, which is a contradiction. (ii) The proof is the same. The method above is a general method for proving that any family of distributions with domain (the range of nonzero density function) depending on unknown parameters is not an exponential family. 2 3. Show that the family of Cauchy distributions C(µ, σ) is not an exponential family. The Cauchy distribution C(µ, σ) has p.d.f. given by " 2 #−1 x−µ 1 1+ f (x; µ, σ) = , x ∈ R, µ ∈ R, σ > 0. πσ σ Solution: Without loss of generality, assume σ = 1. By a counter-proof, assume that C(µ, 1) is an exponential family, we are going to arrive at a contradiction. If C(µ, 1) is an exponential family, then there exist p-dimensional Borel functions T (X) and η(µ) (p ≥ 1) and one-dimensional Borel functions h(X) and ξ(µ) such that −1 1 τ 1 + (x − µ)2 = eη(µ) T (x)−ξ(µ) h(x) π for any x andPµ. Let X = (X1 , . . . , Xn )Q be a random sample from C(µ, 1), where n > p. Let Tn (X) = ni=1 T (Xi ) and hn (X) = ni=1 h(Xi ). Then the joint Lebesgue density of X is n 1 Y τ 2 −1 1 + (x − µ) = eη(µ) Tn (x)−nξ(µ) hn (x) i n π i=1 for any x = (x1 , . . . , xn ) and µ, which implies that ! n 2 Y 1 1 + xi ˜ ln = η˜(µ)τ Tn (x) − nξ(µ) n 2 π i=1 1 + (xi − µ) ˜ for any x and µ, where η˜(µ) = η(µ) − η(0) and ξ(µ) = ξ(µ) − ξ(0). Define ψµ (x) = n Y 1 + (xi − µ)2 i=1 1 + x2i . √ As a function of µ, ψµ (x) is a polynomial which has 2n complex roots x(i) ± −1, i = 1, . . . , n. If x = (x1 , . . . , xn ) and y = (y1 , . . . , yn ) such that Tn (x) = Tn (y), then ψµ (x) = ψµ (y) for all µ, which implies (x(1) , . . . , x(n) ) = (y(1) , . . . , y(n) ). On the other hand, we may choose real numbers µ1 , . . . , µp such that η˜(µi ), i = 1, ..., p, are linearly independent vectors. Since ˜ i) ψµi (x) = η˜(µi )τ Tn (x) − nξ(µ 3 for any x, Tn (x) is then a function of the p functions ψµi (x), i = 1, ..., p. Since n > p, it can be shown that there exist x and y in Rn such that ψµi (x) = ψµi (y), i = 1, ..., p, (which implies Tn (x) = Tn (y)), but the vector of ordered xi ’s is not the same as the vector of ordered yi ’s. This contradicts the previous conclusion. Hence, P is not an exponential family. 4 4. Consider the multinomial distribution with p.d.f. given by n! xk−1 xk θk , θ1x1 · · · θk−1 x 1 ! . . . xk ! P P where xj ’s are integers satisfying kj=1 xj = n and θj > 0, kj=1 = 1. f (x1 , x2 , . . . , xk ) = (i) Show that the multinomial distribution is an exponential family with θ = (θ1 , . . . , θk ), but it does not have a full rank. (ii) Provide a re-parameterization of the family such that, with the re-parameterized parameter space, the multinomial distribution is an full rank exponential family. Solution: (i) The p.d.f. can be expressed as f (x1 , x2 , . . . , xk ) = exp{ k X ln(θj )xj } j=1 n! , x 1 ! . . . xk ! which is of the form P of an exponential family with parameter space {(θ1 , . . . , θk ) : θj > 0, j = 1, . . . , k, kj=1 θj = 1}. However, the parameter space does not contain a open set in the space Rk . Hence it does not have full rank. (ii) Re-parameterize the family by ϑ = (θ1 , . . . , θk−1 ) where θj > 0, j = 1, . . . , k − Pk−1 1, j=1 θj < 1. With the new parameterization, the p.d.f. becomes f (x1 , x2 , . . . , xk ) = exp{ k−1 X ln(θj )xj + ln(1 − j=1 = exp{ k−1 X j=1 k−1 X θj )(n − j=1 xj ln 1− θj Pk−1 j=1 θj + n ln(1 − k−1 X xj )} j=1 k−1 X j=1 θj )} n! , x1 ! . . . xk ! n! x 1 ! . . . xk ! The new parameter space itself is an open set in Rk−1 . Hence the family is of full rank k − 1. 5 5. Let X1 , . . . , Xn be i.i.d. samples from a population P ∈ P where P is any family of distributions. Show that T (X) = (X(1) , . . . , X(n) ) is sufficient for P, where X(k) is the k-th smallest order statistic. Solution: Given (X(1) , . . . , X(n) ), (X1 , . . . , Xn ) has an equal chance to assume any permutations of (X(1) , . . . , X(n) ). Hence, the conditional probability of (X1 , . . . , Xn ) is P (X1 = x1 , . . . , Xn = xn ) = 1 , n! where (x1 , . . . , xn ) is any permutation of (X(1) , . . . , X(n) ). No matter what family is assumed, the conditional distribution does not depend on any unknowns of the family. Hence T (X) is sufficient by the definition. 6

© Copyright 2018 ExploreDoc