More higher-order efficiency: Concentration probability

Yutaka KANO Osaka University

(Final version, May 1998)

Running Title: MORE HIGHER-ORDER EFFICIENCY. Mailing Address: Faculty of Human Sciences, Osaka University, Suita, Osaka 565-0871,

JAPAN. Email: kano@hus.osaka-u.ac.jp

Abstract Based on concentration probability of estimators about a true parameter, third-order

asymptotic efficiency of the first-order bias-adjusted MLE within the class of first-order bias-adjusted estimators has been well established in a variety of probability models. In this paper we consider the class of second-order bias-adjusted Fisher consistent estimators of a structural parameter vector on the basis of an i.i.d. sample drawn from a curved exponential-type distribution, and study the asymptotic concentration probability, about a true parameter vector, of these estimators up to the fifth-order. In particular, (i) we show that third-order efficient estimators are always fourth-order efficient; (ii) a necessary and sufficient condition for fifth-order efficiency is provided; and finally (iii) the MLE is shown to be fifth-order efficient.

Key Words and Phrases: Bias-adjustment, curved exponential distributions, Edgeworth

expansion, maximum likelihood estimator, Fisher-consistency. AMS 1991 subject classifications: 62F12 1 Introduction In the recently published book by J. K. Ghosh (1994), he mentions, in Section 6.10 located after description of third-order (asymptotic) efficiency, that Where do we go from here: Given that first-order efficiency implies second-order efficiency, it is natural to conjecture that third-order efficiency implies fourth-order efficiency. The proof of that must be very messy. One may also ask if anything like third-order efficiency holds for θ̂ when we go to the fifth-order. Ghosh and Sinha (1982) show with a counterexample that this is not possible.

With fairly simple proofs, this paper gives an affirmative answer to the conjecture above and establishes the fifth-order efficiency of the MLE, under a slightly different biasadjustment.

Let (X , A, P_{θ}) be a complete and separable probability space for each θ ∈ Ω_{θ}, where Ω_{θ}, an open domain of ℝ^{p}, is a parameter space of θ. Let x be a p-dimensional random vector (X → ℝ^{p}, A-measurable) with a continuous exponential-type distribution:

exp(θ′x − ψ(θ))μ(dx),  (1.1)

where μ(·) is a carrier measure on ℝ^{p} (see, e.g., Amari, 1985, section 4.1) and the prime ′ denotes the transpose of a vector or a matrix. The real-valued function, ψ(θ), on Ω_{θ} is defined as

ψ(θ) = log ∫_{ℝ^{p}} exp(θ′x)μ(dx).

Assume that the parameter p-vector θ is a function of a more basic parameter q-vector u (p > q), i.e.,

θ = θ(u)  (1.2)

is a function from Ω, an open domain of ℝ^{q}, to Ω_{θ}(⊂ ℝ^{p}). The family of the distributions (1.1) with the structure θ = θ(u) is said to be of curved exponential-type.

Let x_{1},…, x_{n} be identically and independently distributed (i.i.d.) random p-vectors defined on the probability space (X^{n}, A^{n}, P_{θ}^{n}), each of which is distributed according to (1.1) with the structure (1.2). In this paper, we shall study asymptotic properties of estimators for the unknown parameter vector u on the basis of the random sample x_{1},…, x_{n}.

The likelihood function L(u), defined on Ω, is written as

L(u) = ∏_{j=1}^{n} exp(θ(u)′x_{j} − ψ(θ(u))) = exp{n(θ(u)′x̄ − ψ(θ(u)))},

where x̄ = \frac{1}{n} ∑_{j=1}^{n} x_{j}. Note that x̄ is a sufficient statistic for θ (and hence for u). The maximum likelihood estimator (MLE), û = û(x̄), is defined as a solution to the maximization problem:

max_{u ∈ Ω} L(u).  (1.3)

Some assumptions on the MLE will be stated later.

Let g(·) be a function from ℝ^{p} to ℝ^{q}, smooth in a neighborhood of an arbitrarily fixed η(u)(= E[x]) (see (3.3)). The function g(x̄) is said to be a Fisher consistent estimator if and only if g(η(u)) = u for all possible u ∈ Ω. The property excludes Hodges’ pathological example (see Le Cam, 1953).

Let g_{0}(x̄) and g(x̄) be Fisher consistent estimators of u, and let g_{0}^{∗}(x̄) and g^{∗}(x̄) be corresponding bias-adjusted estimators which have the same asymptotic bias up to a suitable order. Consider the following statement: For a positive integer k,

P_{θ(u)}\{\sqrt{n}(g^{∗}(x̄) − u) ∈ C\} ≤ P_{θ(u)}\{\sqrt{n}(g_{0}^{∗}(x̄) − u) ∈ C\} + o(n^{−(k−1)/2}),  (1.4)

where P_{θ(u)} denotes the probability w.r.t. (1.1) with (1.2) and C is any Borel convex set of ℝ^{q} symmetric about the origin. The statement (1.4) asserts that the estimator g_{0}^{∗}(x̄) is more concentrated about the true parameter u than g^{∗}(x̄) up to the order o(n^{−(k−1)/2}). If g_{0}^{∗}(x̄) satisfies the relation (1.4) for any Fisher consistent estimator g^{∗}(x̄) at k = 1, the estimator g_{0}(x̄) is said to be first-order (asymptotically) efficient. A first-order efficient estimator g_{0}(x̄) is said to be second-order efficient iff (1.4) holds true at k = 2 for any first-order efficient estimator g(x̄); in general, (K − 1)th-order efficient estimator g_{0}(x̄) is said to be Kth-order efficient iff (1.4) holds true at k = K for any (K − 1)th-order efficient estimator g(x̄) (see, e.g., Akahira and Takeuchi, 1981, page 8; Amari, 1985, page 130).

The purpose of this paper is to give an almost complete answer to some natural questions on the fourth- and fifth-order efficiency of the MLE. We show that under a certain second-order bias-adjustment, the fourth-order term of the concentration probability, which is of order O (\frac{1}{n\sqrt{n}}), of third-order efficient estimators does not depend on the estimators, that is, third-order efficiency automatically implies fourth-order efficiency. The consequence corresponds to the impressive result, proved by Pfanzagl (1976), that firstorder efficiency implies second-order efficiency. Next, a necessary and sufficient condition for fifth-order efficiency is given, and we apply the condition to establish the fifth-order efficiency of the MLE.

Technically, the following two things are new: (i) It is seen that an alternative biasadjustment factor must be employed, otherwise Ghosh and Sinha’s (1982) counterexample to the fifth-order efficiency of the MLE is alive; (ii) partial differential equations (PDE) are derived to evaluate characteristic functions. The method of evaluation through the PDE is new and successfully reduces the computation, and even though the method is applied to prove the third-order efficiency, it is new. Rather than emphasizing the novelty of these techniques, we hope that one will develop these technical tools to establish socalled absolute asymptotic efficiency, i.e., the MLE is asymptotically efficient at any order, a conjecture by Rao, Sinha and Subramanyam (1982, Remark 3.1).

An overview of the paper is as follows: Section 2 briefly reviews the asymptotic efficiency, and Section 3 gives notation used throughout the paper. Main results are stated in Section 4. An outline of the proofs of the main results is described in Section 5; some technical details are presented in Appendix.

2 Brief historical review The notion of (asymptotic) efficiency in estimation problem was first discussed by Fisher (1925). The first-order efficiency of the MLE in the sense of (1.4) was proved by Rao(1962), Bahadur (1964) and Kaufman (1966) for the case where a general density function w.r.t. the Lebesgue measure is permitted. Several refinements of the first-order efficiency have been made by Hajek (1970), Inagaki (1970), and Pfanzagl (1973a) among others.

The second-order efficiency of the MLE was proved by Pfanzagl (1973b); and then, as noted in Introduction, he established that first-order efficiency implies second-order efficiency under bias-adjustments (see Pfanzagl, 1976). Extensions to the multi-parameter case or to not i.i.d. cases were made by Akahira (1975), Akahira and Takeuchi (1976), Hosoya (1979) and Taniguchi (1983) among others. Yoshida (1992, 93) recently made an attempt to establish second-order efficiency in diffusion processes.

The third-order efficiency of the MLE was shown by Takeuchi and Akahira (1978), Pfanzagl and Wefelmeyer (1978) and Ghosh, Sinha and Wieand (1980). Extensions to more complicated models were made by Takeuchi and Morimune (1985) and Taniguchi (1986,87). Thorough theory of asymptotic efficiency up to the third-order in estimation problems is described by the well-written text books by Akahira and Takeuchi (1981), Pfanzagl (1985), Taniguchi (1991), and Ghosh (1994). In the discussion of second- or third-order optimality in estimation, the main ideas are bias-adjustments and Edgeworth expansions.

There are more tractable but somewhat indirect criteria of asymptotic efficiency such as the quadratic loss and loss of information. Plenty of research on second- or third-order efficiency based on these criteria have been done by many authors including Rao (1961,63), Ghosh and Subramanyam (1974), Efron (1975), Eguchi (1983), and Amari (1985). Efron (1975) and Hosoya (1990) have pointed out a certain equivalence between these criteria.

There is little work on studying properties of n^{−3} terms of loss functions, see Efron (1975), Rao, Sinha and Subramanyam (1982), and Ghosh and Sinha (1982). Note that the evaluation of n^{−3} terms of loss functions corresponds to that of n^{−2} terms of concentration probability. Kano (1997) recently proved that the MLE minimizes the n^{−3} terms of an expansion of the quadratic loss of Fisher consistent estimators. No work on over thirdorder efficiency on the basis of concentration probability has been yet made; the present work is the first attempt in this direction.

3 Notation Let x = [x_{1},…, x_{p}]′, and let A(x) = [a_{ij}(x)] be an a_{1} ×a_{2}-matrix valued smooth function. Put \frac{∂A(x)}{∂x_{k}} = [\frac{∂a_{ij}(x)}{∂x_{k}}]. Define \frac{dA(x)}{dx′} = \frac{d}{dx′} A(x) = [\frac{∂A(x)}{∂x_{1}}, ⋯, \frac{∂A(x)}{∂x_{p}}] , and \frac{d}{dx}A(x) = (\frac{d}{dx′} A(x)′)′. The higher-order (matrix) derivatives are defined inductively by (\frac{d}{dx′} )^{<k>} = \frac{d}{dx′} (\frac{d}{dx′} )^{<k−1>} with (\frac{d}{dx′} )^{<1>} = \frac{d}{dx′}. For such matrix derivatives, see e.g., Bentler and Lee (1978), Magnus and Neudecker (1985,88), and Kano (1993,97). Let us simply write

A^{<k>} = \overbrace{A ⊗ ⋯ ⊗ A}^{k-fold}, k-fold right Kronecker (tensor) product of the same matrix A. Define A^{<0>} = 1. The symmetric tensor for the Kronecker product of p-vectors a_{1},…, a_{k} is denoted by N_{p^{<k>}}, which operates as

N_{p^{<k>}}(a_{1} ⊗ ⋯ ⊗ a_{k}) = ∑ (a_{σ(1)} ⊗ ⋯ ⊗ a_{σ(k)}) /k!,

where the summation runs over all permutations (σ(1),…, σ(k)) of (1,…, k). See e.g. Sternberg (1964, Section 1.3) for the symmetric tensor. Let g(x) be an ℝ^{q}-valued smooth function defined on a neighborhood of x_{0} ∈ ℝ^{p}. The Taylor series of g(x) about x_{0} is expressible in the form:

g(x) = ∑_{k=0}^{∞} \frac{G_{k}}{k!} (x − x_{0})^{<k>} with G_{k} = (\frac{d}{dx′} )^{<k>} g(x)|_{x=x_{0}} .

The matrix of the derivatives, G_{k}, has an important property:

G_{k}N_{p^{<k>}} = G_{k}.  (3.1)

Recall that x_{1},…, x_{n} are an i.i.d. random sample distributed according to the distribution (1.1) with (1.2). Define

Ψ_{kℓ} = Ψ_{kℓ}(θ(u)) = (\frac{d}{dθ} )^{<k>} (\frac{d}{dθ′} )^{<ℓ>} ψ(θ)|_{θ=θ(u)} .  (3.2)

Since the cumulant generating function of the distribution is written as log E_{θ(u)}[e^{is′x_{j}}] = ψ(θ(u) + is) − ψ(θ(u)), we have

E_{θ(u)}[x_{j}] = Ψ_{10}(θ(u) (= η(u), say),  j = 1,…, n  (3.3)

where E_{θ(u)}[·] denotes the expectation w.r.t. the probability measure in (1.1) with the structure (1.2). Note that

Ψ_{kℓ} = E_{θ(u)}[(x_{j} − η(u))^{<k>}(x_{j} − η(u))^{<ℓ>′}]

for k + ℓ = 2, 3. Put z_{n} = \sqrt{n}(x̄ − η(u)) and Ψ_{kℓ}(n) = z_{n}^{<k>}z_{n}^{<ℓ>′}. Then

E_{θ(u)}[Ψ_{kℓ}(n)] = Ψ_{kℓ} for k + ℓ = 2, E_{θ(u)}[Ψ_{kℓ}(n)] = \frac{1}{\sqrt{n}}Ψ_{kℓ} for k + ℓ = 3;
E_{θ(u)}[Ψ_{40}(n)] = 3N_{p^{<4>}}Ψ_{20}^{<2>} + O (\frac{1}{n}) .

See Holmquist (1988) and Kano (1993, 97) for more general results.

4 Main results First of all, we state some assumptions. Let u_{0} be an arbitrarily fixed (inner) point of Ω, and let U_{0}(⊂ Ω) be a neighborhood of u_{0}. We denote by 𝔹_{k} the k-dimensional Borel σ-field.

(A1) θ(u) is (𝔹_{q} ∩ Ω, 𝔹_{p} ∩ Ω_{θ})-measurable and smooth in u ∈ U_{0}.

Denote Θ_{1} = Θ_{1}(u) = \frac{d}{du′} θ(u) for u ∈ U_{0}. Recall the definitions of Ψ_{kℓ}(θ) in (3.2) and η(u) in (3.3).

(A2) rank(Θ_{1}(u)) = q and rank(Ψ_{11}(θ(u))) = p at u = u_{0}.

Remember that g(x) is an ℝ^{q}-valued function from ℝ^{p}.

(A3) g(x) is (𝔹_{p}, 𝔹_{q})-measurable and smooth in a neighborhood of η(u_{0}), and g(x̄) is Fisher consistent, i.e., g(η(u)) = u for any u ∈ U_{0}.

Assumption (A3) is also said to be locally stable (see Pfanzagl and Wefelmeyer, 1978, page 7).

(A4) The optimization problem (1.3) has a unique solution, û(x̄), say, for any x̄ in a neighborhood of η(u). The MLE û(x̄) is (𝔹_{p}, 𝔹_{q})-measurable and continuous at x̄ = η(u) (u ∈ U_{0}).

The continuity assumption ensures (strong) consistency of û(x̄). Assumptions on Ω and u_{0} are needed to guarantee the continuity; we do not discuss it here. See Rao (1973) and Kano (1986) for details.

It follows from (A4) that for any x̄ in a neighborhood of η(u_{0}), û(x̄) is a solution to the equation:

\frac{d}{du}L(u|x̄) = nΘ_{1}(u)′\{x̄ − η(u)\} = 0.  (4.1)

The function \frac{d}{du}L(u|x̄) is smooth in (x̄, u) in a neighborhood of (η(u_{0}), u_{0}), and

\frac{d}{du′} \frac{d}{du}L(u|x̄)|_{(x̄,u)=(η(u_{0}),u_{0})} = nΘ_{1}(u_{0})Ψ_{11}(θ(u_{0}))Θ_{1}(u_{0}) (= ni_{u_{0}}, say)  (4.2)

is positive definite in view of (A2). The continuity in (A4) ensures that (x̄, û(x̄)) stays in a neighborhood of (η(u_{0}), u_{0}) for any x̄ close to η(u_{0}). Thus, application of the implicit function theorem to (4.1) shows that û(x̄) is smooth in a neighborhood of η(u_{0}). Note that i_{u_{0}} in (4.2) is the Fisher information matrix.

Bias adjustments play an important role in studying higher-order asymptotic optimality of estimators. Recall z_{n} = \sqrt{n}(x̄ − η(u)) and write

G_{k} = G_{k}(η(u)) = (\frac{d}{dx̄′} )^{<k>} g(x̄)|_{x̄=η(u)} .

The function G_{k}(x) is defined merely on a neighborhood of η(u_{0}), but we extend G_{k}(x) to a function defined on ℝ^{p} which is measurable in x and smooth in a neighborhood of η(u_{0}).

The estimator g(x̄) is then expanded formally as

g(x̄) = u + \frac{1}{\sqrt{n}}G_{1}z_{n} + \frac{1}{2n}G_{2}z_{n}^{<2>} + \frac{1}{6n\sqrt{n}}G_{3}z_{n}^{<3>} + ⋯,

and hence the asymptotic bias is expressible in the form:

E_{θ(u)}[g(x̄) − u] = \frac{1}{2n}G_{2}Ψ_{20} + \frac{1}{6n^{2}} G_{3}Ψ_{30} + \frac{1}{8n^{2}} G_{4}Ψ_{20}^{<2>} + O (\frac{1}{n^{3}} ) .

We shall make a first-order bias adjustment (or correction) via

\frac{1}{n}b_{1}(x̄) = \frac{1}{2n}G_{2}(x̄)Ψ_{20}(θ(g(x̄))).  (4.3)

It should be noted that the adjustment factor is slightly but essentially different from the one employed in the literature: \frac{1}{2n} Ĝ_{2}Ψ_{20} = \frac{1}{2n}G_{2}(η(g(x̄)))Ψ_{20}(θ(g(x̄))). Either of the factors can be employed to establish the second- and third-order optimality of the MLE, in other words, the optimality does not depend on the distribution of b_{1}(x̄) but only on its convergent point. In contrast, the distribution does influence upon the terms which contribute to more higher-order optimality. Thus, the choice of the bias-adjustment factor is essential.

The first-order bias-adjusted estimator has the following second-order asymptotic bias:

E_{θ(u)} [g(x̄) − u − \frac{1}{n}b_{1}(x̄)] = − \frac{1}{2n^{2}} \{(\frac{d}{dx̄′} )^{<2>} b_{1}(x̄)|_{x̄=η(u)}\} Ψ_{20}

+ \frac{1}{6n^{2}} G_{3}Ψ_{30} + \frac{1}{8n^{2}} G_{4}Ψ_{20}^{<2>} + O (\frac{1}{n^{3}} )

=\frac{1}{n^{2}} b_{2}(u) + O (\frac{1}{n^{3}} ) ,  say.

We make a second-order bias-adjustment by b_{2}(g(x̄)) (= b_{2}(x̄), simply). The estimator after the second-order bias-adjustment is denoted by

g^{∗}(x̄) = g(x̄) − \frac{1}{n}b_{1}(x̄) − \frac{1}{n^{2}} b_{2}(x̄).  (4.4a)

In the same manner, the MLE û(x̄) is bias-adjusted up to the second-order: Let

Ḡ_{k} = Ḡ_{k}(η(u)) = (\frac{d}{dx̄′} )^{<k>} û(x̄)|_{x̄=η(u)} .

Then,

û^{∗}(x̄) = û(x̄) − \frac{1}{n}b̄_{1}(x̄) − \frac{1}{n^{2}} b̄_{2}(x̄),  (4.4b)

where b̄_{1}(x̄) and b̄_{2}(x̄) are are defined as b_{1}(x̄) and b_{2}(x̄) with Ḡ_{k} for G_{k}. As a result, it holds formally

E_{θ(u)}[g^{∗}(x̄) − u] = O (\frac{1}{n^{3}} ) , E_{θ(u)}[û^{∗}(x̄) − u] = O (\frac{1}{n^{3}} ) .  (4.5)

It is known that g(x̄) is first-order efficient iff G_{1} = Ḡ_{1} (= i_{u}^{−1}Θ′_{1}), see e.g., Takeuchi and Akahira (1978), formula (2.3); under the first-order bias adjustment, the first-order efficient estimator is always second-order efficient (Pfanzagl, 1976), and the estimator is third-order efficient iff G_{2} = Ḡ_{2}. Although there would not be suitable references which explicitly state the last statement, it could be easily deduced from the literature (see e.g. Akahira and Takeuchi, 1981, pages 155-158). As a result, the bias-adjusted MLE is first-, second- and third-order efficient within the class of bias-adjusted Fisher consistent estimators.

The next theorem is the main result of this paper.

Theorem 1. Suppose that x_{1} …, x_{n} are i.i.d. ℝ^{p}-valued random vectors defined on (X^{n}, A^{n}, P_{θ}^{n}), where P_{θ} is of continuous curved exponential-type (1.1) with the structure θ = θ(u) (u ∈ U_{0}). Write x̄ = \frac{1}{n} ∑_{j=1}^{n} x_{j}. Assume (A1)–(A4). Let the MLE û(x̄) be defined by (1.3) and let g(x̄) be third-order efficient; let û^{∗}(x̄) and g^{∗}(x̄) be second-order bias-adjusted in (4.4). Let C ∈ 𝔹_{q} be an arbitrary convex set symmetric about the origin. Then there exists a nonnegative quantity Δ_{û,g,C}^{2} such that for each u ∈ U_{0},

(i) P_{θ(u)}\{\sqrt{n}(g^{∗}(x̄) − u) ∈ C\}

= P_{θ(u)}\{\sqrt{n}(û^{∗}(x̄) − u) ∈ C\} − \frac{1}{n^{2}} Δ_{û,g,C}^{2} + o (\frac{1}{n^{2}} )

uniformly in C and that (ii) Δ_{û,g,C}^{2} = 0 if and only if G_{3} = Ḡ_{3}, provided that C is of positive Lebesgue measure.

The result (i) implies that the term of order n^{−3/2} of the LHS coincides with that of the RHS, that is, under the second-order bias-adjustment, the terms of order n^{−3/2} of expansions of the concentration probability of third-order efficient estimators are independent of the estimators; the terms of all such estimators are the same as the MLE’s. As a result, we have the following.

Corollary 1. Under the second-order bias-adjustment in (4.4), third-order efficiency implies fourth-order efficiency.

It is seen that one can not distinguish third-order efficient estimators via Edgeworth expansions up to the order n^{−3/2}. The statement (ii) of Theorem 1 gives a necessary and sufficient condition for fifth-order efficiency. We apply the condition to get

Corollary 2. Under the second-order bias-adjustment in (4.4), the MLE is fifth-order efficient.

We shall end by stating a consequence on asymptotic completeness. Let

û^{∗∗}(x̄) = û(x̄) − (\frac{1}{n}b̄_{1}(x̄) + \frac{1}{n^{2}} b̄_{2}(x̄)) + (\frac{1}{n}b_{1}(x̄) + \frac{1}{n^{2}} b_{2}(x̄)) ,  (4.6)

where b_{i}(x̄) and b̄_{i}(x̄) are the bias-adjustment factors for g(x̄) and û(x̄), respectively.

Theorem 2. Suppose that x_{1} …, x_{n} are i.i.d. ℝ^{p}-valued random vectors defined on (X^{n}, A^{n}, P_{θ}^{n}), where P_{θ} is of continuous curved exponential-type (1.1) with the structure θ = θ(u) (u ∈ U_{0}). Write x̄ = \frac{1}{n} ∑_{j=1}^{n} x_{j}. Assume (A1)–(A4). Let the MLE û(x̄) be defined by (1.3), and let C ∈ 𝔹_{q} be an arbitrary convex set symmetric about the origin. For any Fisher consistent third-order efficient estimator g(x̄), we define û^{∗∗}(x̄) by (4.6). Then,

P_{θ(u)}\{\sqrt{n}(g(x̄) − u) ∈ C\} = P_{θ(u)}\{\sqrt{n}(û^{∗∗}(x̄) − u) ∈ C\} − \frac{1}{n^{2}} Δ_{û,g,C}^{2} + o (\frac{1}{n^{2}} )

uniformly in C, where Δ_{û,g,C}^{2} is defined in Theorem 1.

Theorem 2 states that for any third-order efficient estimator g(x̄), there exists a modified MLE û^{∗∗}(x̄) which is not inferior to g(x̄) up to the order n^{−2} and that û^{∗∗}(x̄) does have a higher concentration probability than g(x̄) when G_{3} ≠ Ḡ_{3} and C is of positive Lebesgue measure. The result shows a kind of asymptotic completeness to the fourthand fifth-order, cf. Pfanzagl and Wefelmeyer (1978) and Akahira, Hirakawa and Takeuchi (1988).

5 Proofs: Outline In this section we shall describe an outline of the proofs of the theorems given in Section 4.

The formal Edgeworth expansion of the distribution of \sqrt{n}(g^{∗}(x̄) − u) is obtained as follows: For any fixed (u, t) ∈ U_{0} × ℝ^{q}, expand formally the characteristic function of \sqrt{n}(g̃^{∗}(x̄)−u) with g̃^{∗}(x̄) an approximate statistic to g^{∗}(x̄) up to the order n^{−2} and make the inverse Fourier transform of the expanded characteristic function, to get an expansion of the density function of \sqrt{n}(g^{∗}(x̄) − u). We fix (u, t) ∈ U_{0} × ℝ^{q} throughout this section.

Notice that

\frac{d}{dx̄′} b_{1}(x̄)|_{x̄=η(u)} = \frac{d}{dx̄′} \frac{1}{2}G_{2}(x̄)Ψ_{20}(θ(g(x̄)))|_{x̄=η(u)}

= \frac{1}{2}\{G_{3}(I_{p} ⊗ Ψ_{20}) + G_{2}Ψ_{21}Θ_{1}G_{1}\},

and then the approximate statistic g̃^{∗}(x̄) to g^{∗}(x̄) obtained by expanding g^{∗}(x̄) up to the order n^{−5/2} can be expressed in the form:

g̃^{∗}(x̄) = u + \frac{1}{\sqrt{n}}G_{1}z_{n} + \frac{1}{2n}G_{2}(z_{n}^{<2>} − Ψ_{20}) − \frac{1}{2n\sqrt{n}}G_{2}Ψ_{21}Θ_{1}G_{1}z_{n}

+ \frac{1}{6n\sqrt{n}}G_{3}\{z_{n}^{<3>} − 3(I_{p} ⊗ Ψ_{20})z_{n}\} + \frac{1}{n^{2}} P_{n}(z_{n}),

(5.1)

where P_{n}(z_{n}) = P^{(1)}(z_{n}) + \frac{1}{\sqrt{n}}P^{(2)}(z_{n}) = O_{p}(1) is a finite polynomial of z_{n}.

Since x_{j} has a continuous distribution, the distribution meets Cramer’s condition: lim sup_{||s||→∞} E_{θ(u)}[e^{is′(x_{j}−η(u))}] < 1 by the virtue of Bhattacharya and Denker (1990, Lemma 1.5). Therefore, the so-called formal Edgeworth expansion of the distribution of \sqrt{n}(g^{∗}(x̄) − u) is valid up to the order n^{−2} in view of Theorem 2.1 of Bhattacharya and Denker (1990) (see also Bhattacharya and Ghosh, 1978). Accordingly, we evaluate asymptotic concentration probability of the approximate estimator g̃^{∗}(x̄) about the true parameter u up to the order n^{−5/2} or of the standardized form \sqrt{n}(g̃^{∗}(x̄) − u) about the zero vector up to the order n^{−2}, rather than that of g^{∗}(x̄) itself.

Let ũ^{∗}(x̄) be the approximate estimator constituted from the MLE û^{∗}(x̄) in the same manner as g̃^{∗}(x̄). By (5.1) we can express the standardized forms of g̃^{∗}(x̄) and ũ^{∗}(x̄) as

\sqrt{n}(g̃^{∗}(x̄) − u) = G_{1}z_{n} + \frac{1}{2\sqrt{n}}G_{2}(z_{n}^{<2>} − Ψ_{20}) − \frac{1}{2n}G_{2}Ψ_{21}Θ_{1}G_{1}z_{n}

+ \frac{1}{6n}G_{3}\{z_{n}^{<3>} − 3(I_{p} ⊗ Ψ_{20})z_{n}\} + \frac{1}{n\sqrt{n}}P_{n}(z_{n})

and

\sqrt{n}(ũ^{∗}(x̄) − u) = Ḡ_{1}z_{n} + \frac{1}{2\sqrt{n}}Ḡ_{2}(z_{n}^{<2>} − Ψ_{20}) − \frac{1}{2n}Ḡ_{2}Ψ_{21}Θ_{1}Ḡ_{1}z_{n}
+ \frac{1}{6n}Ḡ_{3}\{z_{n}^{<3>} − 3(I_{p} ⊗ Ψ_{20})z_{n}\} + \frac{1}{n\sqrt{n}}P̄_{n}(z_{n}), (5.2)

where P̄_{n}(z_{n}) is defined as P_{n}(z_{n}) with Ḡ_{k} for G_{k}. We have noticed that G_{1} = Ḡ_{1} and G_{2} = Ḡ_{2} for g(x̄) to be third-order efficient. In this case, we have

\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄)) = \sqrt{n}(g̃^{∗}(x̄) − u) − \sqrt{n}(ũ^{∗}(x̄) − u)
= \frac{1}{6n}(G_{3} − Ḡ_{3})\{z_{n}^{<3>} − 3(I_{p} ⊗ Ψ_{20})z_{n}\} + \frac{1}{n\sqrt{n}}Q_{n}(z_{n}), (5.3)

where Q_{n}(z_{n}) = P_{n}(z_{n}) − P̄_{n}(z_{n}) is bounded in probability and a finite polynomial of z_{n} (with coefficients smooth in u ∈ U_{0}) because so are P_{n}(z_{n}) and P̄_{n}(z_{n}). As deduced from (4.5), the second-order bias-adjustment leads to

E_{θ(u)}[\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))] = O (\frac{1}{n^{2}\sqrt{n}}) ,  (5.4)

which is not merely formal but valid.

From (5.3) and (5.4), one can show

E_{θ(u)}[e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)}\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))] = O (\frac{1}{n^{2}\sqrt{n}})  (5.5)

and

E_{θ(u)}[e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)}\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))′]
= \frac{1}{6n^{2}} (G_{3} − Ḡ_{3})Ψ_{11}^{<3>}(G_{3} − Ḡ_{3})′ · e^{−t′i_{u}^{−1}t/2} + O (\frac{1}{n^{2}\sqrt{n}}) .  (5.6)

The Appendix gives the flow of the proof of (5.5). The full proofs of these consequences and (5.9) are given in Kano (1995).

It follows that

E_{θ(u)}[e^{it′\sqrt{n}(_{g}^{∗}(x̄)−u)}]
=E_{θ(u)}[e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)} · e^{it′\sqrt{n}(_{g}^{∗}(x̄)−_{u}^{∗}(x̄))}]
=E_{θ(u)}[e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)}\{1 + \frac{1}{2!}(it′\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄)))^{2} + R_{n}\}]
+ O (\frac{1}{n^{2}\sqrt{n}}) (by (5.5))
=E_{θ(u)}[e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)}]
+ \frac{1}{12n^{2}} (it)′(G_{3} − Ḡ_{3})Ψ_{11}^{<3>}(G_{3} − Ḡ_{3})′(it) · e^{−t′i_{u}^{−1}t/2}  (5.7)
+ O (\frac{1}{n^{2}\sqrt{n}}) (by (5.6)),

where R_{n} is the reminder term of the expansion above, and we have used

|E_{θ(u)}[e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)}R_{n}]| ≤ E_{θ(u)}[|R_{n}|]
≤ E_{θ(u)}[|t′\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))|^{3}/3!]  (5.8)
= O (\frac{1}{n^{3}} )

in view of (5.3). As a result of (5.7), the difference in the concentration probability between \sqrt{n}(g̃^{∗}(x̄) − u) and \sqrt{n}(ũ^{∗}(x̄) − u) based on the Edgeworth expansions appears in the n^{−2} terms, which result from the second term of the RHS of (5.7). Thus, for each u ∈ U_{0}, the probability that \sqrt{n}(g̃^{∗}(x̄) − u) be in the convex region C can be expressed in the form:

P_{θ(u)}\{\sqrt{n}(g^{∗}(x̄) − u) ∈ C\} = P_{θ(u)}\{\sqrt{n}(û^{∗}(x̄) − u) ∈ C\} − \frac{1}{n^{2}} Δ_{û,g,C}^{2} + o (\frac{1}{n^{2}} )

uniformly in C, where

Δ_{û,g,C}^{2} = \frac{1}{12}vec(G_{3} − Ḡ_{3})′
· (Ψ_{11}^{<3>} ⊗ ∫_{C} {i_{u} − i_{u}xx′i_{u}} N_{q}(x|0, i_{u}^{−1})dx) vec(G_{3} − Ḡ_{3}). (5.9)

The vec-operator vec(A) for an a_{1} × a_{2} matrix A denotes an a_{1}a_{2}-vector obtained by stacking a_{2} column vectors of A in order, see Henderson and Searle (1981) and Magnus and Neudecker (1988) for its properties. The expression (5.9) is obtained by making the inverse Fourier transform of the second term in (5.7).

It is known that the matrix of the integrals

∫_{C} {i_{u} − i_{u}xx′i_{u}} N_{q}(x|0, i_{u}^{−1})dx

is positive definite, provided that C is of positive Lebesgue measure and convex, symmetric about the origin. See Pfanzagl (1985, Lemma 13.2.4) for a proof. Thus, Δ_{û,g,C}^{2} is nonnegative, and Δ_{û,g,C}^{2} = 0 if and only if G_{3} = Ḡ_{3}. The proof of Theorem 1 is complete. Proof of Theorem 2. Theorem 2 can be proved in the same manner. The reader should note that

\sqrt{n}(g(x̄) − û^{∗∗}(x̄)) = \sqrt{n}(g^{∗}(x̄) − û^{∗}(x̄))

(so that the corresponding approximate statistics are the same) and that the same derivation in (5.7) holds even though ũ^{∗}(x̄) is replaced with ũ^{∗∗}(x̄) in e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)}. Q.E.D.

Appendix Here we give a condensed proof of (5.5). See Kano (1995) for the full detailed proof.

Kano (1997, Lemma 3.3) showed that for any k ∈ ℕ, there exists a q × qp^{k−1} matrixvalued smooth function B_{k} = B_{k}(u) defined on U_{0} such that Ḡ_{k} = B_{k}(Θ′_{1} ⊗ I_{p^{k−1}})N_{p^{<k>}} for any u ∈ U_{0}. The third-order efficiency, i.e., G_{2} = Ḡ_{2}, imposes a restriction on G_{3}:

(G_{3} − Ḡ_{3})(Ψ_{11}Θ_{1} ⊗ I_{p^{2}}) = (G_{3} − Ḡ_{3})(I_{p} ⊗ Ψ_{11}Θ_{1} ⊗ I_{p})
= (G_{3} − Ḡ_{3})(I_{p^{2}} ⊗ Ψ_{11}Θ_{1})
= 0.

In particular, (G_{3} − Ḡ_{3})Ψ_{11}^{<3>}Ḡ′_{3} = 0.

Let

ϕ_{1} = ϕ_{1}(u, t) =E_{θ(u)}[\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)}]

and

ϕ_{2} = ϕ_{2}(u, t) =E_{θ(u)}[\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))e^{it′\sqrt{n}(_{u}^{∗}(x̄)−u)}z′_{n}],

where (u, t) ∈ U_{0} × ℝ^{q}. Differentiating ϕ_{i}’s by t′, we have the following partial differential equations (PDE):

\frac{dϕ_{i}}{dt′} = − \{(t′i_{u}^{−1}) ⊗ ϕ_{i}\} + C_{i}(u, t)  (i = 1, 2),  (A.1)

where

C_{1}(u, t) =\frac{i}{\sqrt{n}} \frac{dϕ_{1}}{du′} i_{u}^{−1} + \frac{i}{2\sqrt{n}}ϕ_{1}(B_{2}vec(Ψ_{11}Θ_{1}) − Ḡ_{2}Ψ_{20})′
− \frac{1}{2\sqrt{n}}(t′ ⊗ ϕ_{2})B′_{2} + \frac{i}{2n} \frac{dϕ_{2}}{du′} B′_{2} + O (\frac{1}{n^{2}\sqrt{n}})  (A.2a)

and

C_{2}(u, t) =\frac{i}{\sqrt{n}} \frac{dϕ_{2}}{du′} (i_{u}^{−1} ⊗ I_{p}) + iϕ_{1}vec(Ψ_{11}Ḡ_{1})′ + O (\frac{1}{n^{2}} ) .  (A.2b)

If C_{i}(u, t)’s were regarded as unrelated to ϕ_{i}’s, the PDE (A.1) could be solved in a closed form: Let t_{k} = [t_{1},…, t_{k}, 0,…, 0]′, C_{1}(u, t) = [c_{1}^{(1)}(u, t),…, c_{q}^{(1)}(u, t)] and C_{2}(u, t) = [C_{1}^{(2)}(u, t),…, C_{q}^{(2)}(u, t)]. Then, using E_{θ(u)}[\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))] = O (\frac{1}{n^{2}\sqrt{n}}) and E_{θ(u)}[\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))z′_{n}] = O (\frac{1}{n^{2}} ), we have

ϕ_{1} =e^{−t′i_{u}^{−1}t/2} \{E_{θ(u)}[\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))] + ∑_{k=1}^{q} ∫_{0}^{t_{k}} c_{k}^{(1)}(u, t_{k})e^{t′_{k}i_{u}^{−1}t_{k}/2}dt_{k}\}
=e^{−t′i_{u}^{−1}t/2} ∑_{k=1}^{q} ∫_{0}^{t_{k}} c_{k}^{(1)}(u, t_{k})e^{t′_{k}i_{u}^{−1}t_{k}/2}dt_{k} + O (\frac{1}{n^{2}\sqrt{n}})  (A.3a)

and

ϕ_{2} =e^{−t′i_{u}^{−1}t/2} \{E_{θ(u)}[\sqrt{n}(g̃^{∗}(x̄) − ũ^{∗}(x̄))z′_{n}] + ∑_{k=1}^{q} ∫_{0}^{t_{k}} C_{k}^{(2)}(u, t_{k})e^{t′_{k}i_{u}^{−1}t_{k}/2}dt_{k}\}
=e^{−t′i_{u}^{−1}t/2} ∑_{k=1}^{q} ∫_{0}^{t_{k}} C_{k}^{(2)}(u, t_{k})e^{t′_{k}i_{u}^{−1}t_{k}/2}dt_{k} + O (\frac{1}{n^{2}} ) .  (A.3b)

Expand ϕ_{1} and ϕ_{2} as ϕ_{1} = \frac{1}{n}A_{1} + \frac{1}{n\sqrt{n}}A_{2} + \frac{1}{n^{2}} A_{3} + O (\frac{1}{n^{2}\sqrt{n}}) and ϕ_{2} = \frac{1}{n}B_{1} + \frac{1}{n\sqrt{n}}B_{2} + O (\frac{1}{n^{2}} ). It suffices to show A_{1} = A_{2} = A_{3} = 0 for each (u, t). We first note that

∫_{0}^{t_{k}} ϕ_{1}dt_{k} = O (\frac{1}{n}) ,  ∫_{0}^{t_{k}} \frac{dϕ_{1}}{du′} dt_{k} = O (\frac{1}{n});  (A.4a)
∫_{0}^{t_{k}}(t′ ⊗ ϕ_{2})dt_{k} = O (\frac{1}{n}) ,  ∫_{0}^{t_{k}} \frac{dϕ_{2}}{du′} dt_{k} = O (\frac{1}{n})  (A.4b)

(k = 1,…, q) uniformly in (u, t) in any compact set of U_{0} × ℝ^{q}.

It follows from (A.2a) and (A.4) that ∫_{0}^{t_{k}} C_{1}(u, t)dt_{k} = O (\frac{1}{n\sqrt{n}}) (k = 1,…, q) uniformly in (u, t) in any compact set of U_{0}×ℝ^{q} containing (u_{0}, 0), so that ∫_{0}^{t_{k}} c_{k}^{(1)}(u, t_{k})dt_{k} = O (\frac{1}{n\sqrt{n}}) , which leads to ϕ_{1} = O (\frac{1}{n\sqrt{n}}) in view of (A.3a). As a result, A_{1} equals 0 and hence,

∫_{0}^{t_{k}} ϕ_{1}dt_{k} = O (\frac{1}{n\sqrt{n}}) ,  ∫_{0}^{t_{k}} \frac{dϕ_{1}}{du′} dt_{k} = O (\frac{1}{n\sqrt{n}})  (A.5)

uniformly in (u, t) in any compact set of U_{0} × ℝ^{q}. Substitution of (A.4b) and (A.5) into (A.2b) shows ∫_{0}^{t_{k}} C_{2}(u, t)dt_{k} = O (\frac{1}{n\sqrt{n}}) (k = 1,…, q) uniformly in (u, t) in any compact set of U_{0} × ℝ^{q} containing (u_{0}, 0), so that ∫_{0}^{t_{k}} C_{k}^{(2)}(u, t_{k})dt_{k} = O (\frac{1}{n\sqrt{n}}) , which leads to ϕ_{2} = O (\frac{1}{n\sqrt{n}}) in view of (A.3b). As a result, B_{1} equals 0.

Thus, it is seen that starting ϕ_{i} = O (\frac{1}{n}), we get A_{1} = B_{1} = 0 and ϕ_{i} = O (\frac{1}{n\sqrt{n}}). Repeating this process, we have A_{2} = B_{2} = 0 and ϕ_{i} = O (\frac{1}{n^{2}} ). Then, we get A_{3} = 0 and ϕ_{1} = O (\frac{1}{n^{2}\sqrt{n}}). The proof is complete.

Acknowlegments. The author would like to express his thanks to Professor M. Akahira of University of Tsukuba and Dr M. Taniguchi of Osaka University for invaluable discussion on higher-order efficiency.

REFERENCES

[1] Akahira, M. (1975). A note on the second order asymptotic efficiency of estimators in an autoregressive process. Rep. Univ. Electro-Comm., 26, 143-149.

[2] Akahira, M., Hirakawa, F. and Takeuchi, K. (1988). Second and third order asymptotic completeness of the class of estimators. In Probability Theory and Mathematical Statistics, Lecture Notes in Mathematics, Vol.1299 (S. Watanabe and Yu. V. Prokhorov, Eds). pp. 11-27. Springer-Verlag, Berlin.

[3] Akahira, M. and Takeuchi, K. (1976). On the second order asymptotic efficiency of estimators in multiparameter cases. Rep. Univ. Electro-Comm., 26, 261-269.

[4] Akahira, M. and Takeuchi, K. (1981). Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency, Lecture Notes in Statistics, Vol.7. Springer-Verlag, New York.

[5] Amari, S. (1985). Differential-Geometrical Methods in Statistics, Lecture Notes in Statistics, Vol.28. Springer-Verlag, Berlin.

[6] Bahadur, R. R. (1964). On Fisher’s bound for asymptotic variance. Ann. Math. Statist., 35, 1545-1552.

[7] Bhattacharya, R. and Denker, M. (1990). Asymptotic Statistics. Birkhäuser, Basel.

[8] Bhattacharya, R. and Ghosh, J. K. (1978). On the validity of the formal Edgeworth expansion. Ann. Statist., 6, 434-451.

[9] Bentler, P. M. and Lee, Sik-Yum (1978). Matrix derivatives with chain rule and rules for simple, Hadamard, and Kronecker products. J. Math. Psychol., 17, 255-262.

[10] Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Statist., 3, 1189-1242.

[11] Eguchi, S. (1983). Second order efficiency of minimum contrast estimators in a curved exponential family. Ann. Statist., 11, 793-803.

[12] Fisher, R. A. (1925). Theory of statistical estimation. Proc. Cambridge Philos. Soc., 122, 700-725.

[13] Ghosh, J. K. (1994). Higher Order Asymptotics. NSF-CBMS Regional Conference Series in Probability and Statistics, Vol.4. IMS, Hayward, CA.

[14] Ghosh, J. K. and Sinha, B. K. (1982). Third order efficiency of the MLE — A counterexample. Calcutta Statist. Assoc. Bull., 31, 151-158.

[15] Ghosh, J. K., Sinha, B. K. and Wieand, H. S. (1980). Second order efficiency of the MLE with respect to any bounded bowl-shaped loss function. Ann. Statist., 8, 506-521.

[16] Ghosh, J. K. and Subramanyam, K. (1974). Second order efficiency of maximum likelihood estimates. Sankhyā, Ser. A, 36, 325-358.

[17] Hajek, J. (1970). A characterization of limiting distributions of regular estimators. Z. Wahr. verw. Geb., 14, 323-330.

[18] Henderson H. V. and Searle, S. R. (1981). The vec-permutation matrix, the vec operator and Kronecker products: A review. Linear and Multilinear Algebra, 9, 271-288.

[19] Holmquist, B. (1988). Moments and cumulants of the multivariate normal distribution. Stoch. Anal. App., 6, 273-278.

[20] Hosoya, Y. (1979). Higher-order efficiency in the estimation of linear processes. Ann. Statist., 7, 516-530.

[21] Hosoya, Y. (1990). Information amount and higher-order efficiency in estimation. Ann. Inst. Statist. Math., 42, 37-49.

[22] Inagaki, N. (1970). On the limiting distribution of a sequence of estimators with uniformity property. Ann. Inst. Statist. Math., 22, 1-13.

[23] Kano, Y. (1986). Conditions on consistency of estimators in covariance structure model. J. Japan Statist. Soc., 16, 75-80.

[24] Kano, Y. (1997). Beyond third-order efficiency. Sankhyā, Ser. B, 59, 179-197.

[25] Kano, Y. (1993). Beyond third-order efficiency: Supplement. Technical Report, University of Osaka Prefecture.

[26] Kano, Y. (1995). More higher-order efficiency: Concentration Probability. Technical Report of Statistical Group: DATA95-04, Tokyo, Japan.

[27] Kaufman, S. (1966). Asymptotic efficiency of the maximum likelihood estimator. Ann. Inst. Statist. Math., 18, 155-178.

[28] Le Cam, L. (1953). On some asymptotic properties of maximum likelihood estimates and related Bayes’ estimates. Univ. California Publ. Statist., 1, 277-330.

[29] Magnus, J. R. and Neudecker, H. (1985). Matrix differential calculus with applications to simple, Hadamard, and Kronecker products. J. Math. Psychol., 29, 474-492.

[30] Magnus, J. R. and Neudecker, H. (1988). Matrix Differential Calculus with Applications in Statistics and Econometrics. John Wiley, Chichester.

[31] Pfanzagl, J. (1973a). The Accuracy of the normal approximation for estimates of vector parameters. Z. Wahr. verw. Geb., 25, 171-198.

[32] Pfanzagl, J. (1973b). Asymptotic expansions related to minimum contrast estimators. Ann. Statist., 1, 993-1026.

[33] Pfanzagl, J. (1976). First order efficiency implies second order efficiency. In Contribution to Statistics (Hajek Memorial Volume), (J. Jureckova, Ed.), pp. 167-196, Academia, Prague.

[34] Pfanzagl, J. (1985). Asymptotic Expansions for General Statistical Models. Lecture Notes in Statistics, Vol.31. Springer-Verlag, Berlin.

[35] Pfanzagl, J. and Wefelmeyer, W. (1978). A third order optimum property of the maximum likelihood estimator. J. Multivariate Anal., 8, 1-29.

[36] Rao, C. R. (1961). Asymptotic efficiency and limiting information. Proc. Fourth Berkeley Symp., 1, 531-545.

[37] Rao, C. R. (1962). Efficient estimates and optimum inference procedures in large samples. J. Royal Statist. Soc., Ser. B, 24, 46-72.

[38] Rao, C. R. (1963). Criteria of estimation in large samples. Sankhyā, Ser. A, 25, 189-206.

[39] Rao, C. R. (1973). Linear Statistical Inference and its Applications, 2nd ed. Wiley, New York.

[40] Rao, C. R., Sinha, B. K. and Subramanyam, K. (1982). Third order efficiency of the maximum likelihood estimator in the multinomial distribution. Statist. Decisions, 1, 1-16.

[41] Sternberg, S. (1964). Lectures on Differential Geometry. Prentice-Hall Inc., Englewood Cliffs, NJ.

[42] Takeuchi, K. and Akahira, M. (1978). Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case. Rep. Univ. Electro-Comm., 28, 271-293.

[43] Takeuchi, K. and Morimune, K. (1985). Third-order efficiency of the extended maximum likelihood estimators in a simultaneous equation system. Econometrica, 53, 177-200.

[44] Taniguchi, M. (1983). On the second order asymptotic efficiency of estimators of

Gaussian ARMA Processes. Ann. Statist., 11, 157-169.

[45] Taniguchi, M. (1986). Third order asymptotic properties of maximum likelihood estimators for Gaussian ARMA processes. J. Multivariate Anal., 18, 1-31.

[46] Taniguchi, M. (1987). Third order asymptotic properties of BLUE and LSE for a regression model with ARMA residual. J. Time Ser. Anal., 8, 111-114.

[47] Taniguchi, M. (1991). Higher Order Asymptotic Theory for Time Series Analysis, Lecture Notes in Statistics, Vol.68. Springer-Verlag, Berlin.

[48] Yoshida, N. (1992). Asymptotic expansions of maximum likelihood estimators for small diffusions via the theory of Malliavin–Watanabe. Probab. Theory Relat. Fields, 92, 275-311.

[49] Yoshida, N. (1993). Asymptotic expansion of Bayes estimators for small diffusions. Probab. Theory Relat. Fields, 95, 429-450.