OPTIMAL DESIGNS FOR COMPLETE INTERACTION STRUCTURES

By Rainer Schwabe

Freie Universität Berlin

In linear models with complete interaction structures product type designs are shown to be Φ_{q}-optimal for the maximal interactions, 0 ≤ q ≤ ∞, and Doptimal simultaneously for the whole parameter vector and for testing against any complete submodel.

1. Introduction. We consider a general multi-factor linear model

E(X(t_{1}, …, t_{K})) = μ_{H}(t_{1}, …, t_{K}) = ∑_{H∈H} ⨂_{k∈H} f_{k}(t_{k})^{T} β_{H},

t_{k} ∈ T_{k}, k = 1, …, K, with interaction structure H ⊆ P({1, …, K}) := {H; H ⊆ {1, …, K}}. Throughout this paper we assume that the interaction structure H is complete, i. e. each interaction H ∈ H comes with all its lower order interactions:

Definition 1.1. H is complete if P(H) ⊆ H for every H ∈ H.

Models of these type include product models, H = P({1, …, K}), considered by Hoel (1965), additive models, H = {∅, {1}, …, {K}}, (see e. g. Schwabe and Wierich, 1995) and complete M-factor interaction models, H = {H; H ⊆ {1, …, K}, |H| ≤ M}, (see e. g. Schwabe, 1995). Particular examples will be given in section 4.

The aim of the present paper is to establish that, in the setting of approximate designs (see e. g. Kiefer, 1974), product type designs are simultaneously optimal for higher order interactions if their components are optimal in the corresponding marginal models μ_{k}(t_{k}) = β_{0} + f_{k}(t_{k})^{T} β_{k}, t_{k} ∈ T_{k}.

Definition 1.2. H ∈ H is maximal in H if H′ ∉ H for every H′ ⊃ H, H′ ≠ H.

We will show that the product type design δ^{∗} = ⨂_{k=1}^{K} δ_{k}^{∗} is Φ_{q}-optimal for maximal interactions β_{H}, i. e. δ^{∗} minimizes the q-“norm” of the eigenvalues of the covariance matrix of the least squares estimator β̂_{H} of β_{H}, if its components δ_{k}^{∗} are Φ_{q}-optimal for β_{k} in the marginal models, 0 ≤ q ≤ ∞. Note that the Φ_{q}-criteria include the common A-optimality, q = 1, and the D- and E-optimality as limiting cases, q → 0 and q → ∞, respectively. Moreover, for the D-criterion, the product type design δ^{∗} will be shown to be simultaneously optimal for the whole parameter vector β = (β_{H})_{H∈H} as well as for testing every complete model μ_{H} against every complete submodel μ_{H^{∗}}, H^{∗} ⊂ H.

The research was supported by grant Ku719/2-1 of the Deutsche Forschungsgemeinschaft.

AMS 1991 subject classification. 62K05.

Key words and phrases. Product type designs, multi-factor linear models, Φ_{q}-optimality, Doptimality, complete interactions, maximal interaction.

Abbreviated title. Designs for interactions.

2. Notations and auxiliary results. We recall that a subset β_{H} = L_{H}β of parameters is identifiable under a design δ in a linear model μ(t) = a(t)^{T} β if the rows of the selection matrix L_{H} are contained in the space spanned by the regression function a evaluated at the support points t of δ or, equivalently, spanned by the columns of the information matrix I(δ) = ∫aa^{T} dδ. In the present setting the regression functions considered are a_{H}(t_{1}, ..., t_{K}) = (⨂_{k∈H} f_{k}(t_{k}))_{H∈H}.

If β_{H} is identifiable the corresponding covariance matrix is given by C(β_{H}; δ) = L_{H}I(δ)^{−}L_{H}^{T}. For larger subsets β_{H′} = (β_{H})_{H∈H′} we have L_{H′} = diag(L_{H})_{H∈H′} and C(β_{H′}; δ) = L_{H′}I(δ)^{−}L_{H′}^{T} if each β_{H}, H ∈ H′, is identifiable under δ. In case H′ = H we write C(δ) = C(β; δ), for short.

In the corresponding marginal models the parameters are β^{(k)} = (β_{0}, β_{k}^{T})^{T} and the information matrices I_{k}(δ_{k}) = ∫a_{k}a_{k}^{T} dδ_{k} where a_{k} = (1, f_{k}^{T})^{T}. The marginal covariance matrices for β^{(k)} and β_{k} = L_{k}β^{(k)} are denoted by C_{k}(δ_{k}) = I_{k}(δ_{k})^{−1} and C_{k}(β_{k}; δ_{k}) = L_{k}C(δ_{k})L_{k}^{T} respectively. Note that β_{k} is identifiable under δ_{k} iff I_{k}(δ_{k}) is regular.

As usual λ_{max} and tr denote the largest eigenvalue and the trace of a matrix. For every pair of designs δ_{H} on ×_{k∈H}T_{k} and δH̄ on ×_{k∉H}T_{k} the resulting product

design on ×_{k=1}^{K}T_{k} is denoted by π_{H}(δ_{H}, δH̄).

Lemma 2.1. Let μ(t) = f_{0}(t)^{T} β_{0} + f_{1}(t)^{T} β_{1} + f_{2}(t)^{T} β_{2} and let μ_{0}(t) = f_{0}(t)^{T} β_{0} + f_{1}(t)^{T} β_{1} be a submodel of μ.

(i) If β_{1} is identifiable under δ in μ, then C_{0}(β_{1}; δ) ≤ C(β_{1}; δ).

(ii) If β_{1} and β_{2} are identifiable under δ in μ, then det C(β_{1}, β_{2}; δ) = det C_{0}(β_{1}; δ) det C(β_{2}; δ).

Proof. (i) is a common refinement statement (see e. g. Kunert, 1983).

For (ii) we partition the information matrix

I(δ) = ( \begin{matrix}I_{0}(δ) & I_{12} \\ I_{12}^{T} & I_{2} \end{matrix} )

in μ according to the submodel, and the covariance matrix

C(β_{1}, β_{2}; δ) = ( \begin{matrix}C_{1} & C_{12} \\ C_{12}^{T} & C_{2} \end{matrix} )

according to the components β_{1} and β_{2}, C_{j} = C(β_{j}; δ). Further let L be the selection matrix for β_{1} = L(β_{0}^{T} , β_{1}^{T})^{T} in μ_{0}.

By the formula for generalized inverses of partitioned matrices

C_{1} = L(I_{0}(δ)^{−} + I_{0}(δ)^{−}I_{12}C_{2}I_{12}^{T}I_{0}(δ)^{−})L^{T}
C_{12} = LI_{0}(δ)^{−}I_{12}C_{2}

and hence

C_{1} − C_{12}C_{2}^{−1}C_{12}^{T} = LI_{0}(δ)^{−}L^{T} = C_{0}(β_{1}; δ).

The result now follows from det C(β_{1}, β_{2}; δ) = det(C_{1} − C_{12}C_{2}^{−1}C_{12}^{T}) det C_{2}. □

Remark 2.1. If f_{0} ≡ 1 (or f_{0} ≡ 0), then det C(δ) = det C_{0}(δ) det C(β_{2}; δ) from Lemma 2.1.

3. Optimality of product type designs. The first result deals with the optimality of product type designs for the highest interaction in total interactions models

Theorem 3.1. Let H = P and let H = {1, …, K}. If δ_{k}^{∗} is Φ_{q}-optimal for β_{k} in the kth marginal model, k = 1, …, K, then δ^{∗} = ⨂_{k=1}^{K} δ_{k}^{∗} is Φ_{q}-optimal for β_{H}.

Proof. Note that β_{H} = Lβ where L = ⨂_{k=1}^{K} L_{k} and β_{k} = L_{k}β^{(k)}. Hence the covariance matrix C(β_{H}; δ^{∗}) = ⨂_{k=1}^{K} C_{k}(β_{k}; δ_{k}^{∗}) factorizes. The Φ_{q}-optimality can now be obtained by means of the appropriate equivalence theorems. For q < ∞ see Rafajłowicz and Myszka (1988). For the E-citerion (q = ∞) a more implicit equivalence theorem (Pukelsheim, 1993, p. 182) has to be used: δ_{k}^{∗} is E-optimal for β_{k} iff there exists a non-negative definite, symmetric matrix M_{k}, tr M_{k} = 1, such that

a_{k}(t_{k})^{T} C_{k}(δ_{k}^{∗})L_{k}^{T} C_{k}(β_{k}; δ_{k}^{∗})^{−1}M_{k}C_{k}(β_{k}; δ_{k}^{∗})^{−1}L_{k}C_{k}(δ_{k}^{∗})a_{k}(t_{k}) ≤ 1/λ_{max}(C_{k}(β; δ_{k}^{∗}))

for all t_{k} ∈ T_{k}.

Let M = ⨂_{k=1}^{K} M_{k}. Then M is non-negative definite, symmetric, tr M = 1, and

a(t)^{T} C(δ^{∗})L^{T} C(β_{H}; δ^{∗})^{−1}MC(β_{H}; δ^{∗})^{−1}LC(δ^{∗})a(t) ≤ 1/λ_{max}(C(β_{H}; δ^{∗})).

Thus the equivalence theorem proves the E-optimality of δ^{∗} for β_{H}. □

Note that H = {1, …, K} is the only maximal element in H = P. For general, complete interaction structures H a maximal interaction H will be a proper subset of {1, …, K}. Then, conditions are only required on those factors k belonging to H.

Theorem 3.2. Let H be complete, and let H be maximal in H. If δ_{k}^{∗} is Φ_{q}- optimal for β_{k} in the kth marginal model, k ∈ H, then δ^{∗} = π_{H}(⨂_{k∈H} δ_{k}^{∗}, δH̄) is Φ_{q}-optimal for β_{H}, for every design δH̄ on ×_{k∉H}T_{k}.

Proof. Consider the model μ_{P(H)} of total interactions in the factors involved in H. Then μ_{P(H)} can be regarded as a submodel of μ_{H}. Moreover, as H is maximal,

C_{P(H)}(β_{H}; ⨂_{k∈H} δ_{k}^{∗}) = C(β_{H}; δ^{∗}),

and by Theorem 3.1 the design ⨂_{k∈H} δ_{k}^{∗} is Φ_{q}-optimal for β_{H} in μ_{P(H)}. Hence, the result follows by a refinement argument (cf Lemma 2.1 (i)). □

The main result on Φ_{q}−optimality is the following extension to more than one maximal interaction:

Theorem 3.3. Let H be complete, H′ ⊆ {H; H maximal in H}. If δ_{k}^{∗} is Φ_{q}-optimal for β_{k} in the kth marginal model, k ∈ H(H′) := ⋃_{H∈H′} H, then δ^{∗} = π_{H(H′)}(⨂_{k∈H(H′)} δ_{k}^{∗}, δ_{H̄(H′)}) is Φ_{q}-optimal for β_{H′}, for every design δ_{H̄(H′)} on ×_{k∉H(H′)}T_{k}.

Proof. The key observation is that the covariance matrix is block diagonal:

C(β_{H′}; δ^{∗}) = diag(C(β_{H}; δ^{∗}))_{H∈H′}

because each H ∈ H′ is maximal. By Fan’s (1954) result on the eigenvalues of block matrices the Φ_{q}-optimality of δ^{∗} can be deduced for β_{H′} from the Φ_{q}-optimality of δ_{k}^{∗} for every β_{H}, H ∈ H′, established in Theorem 3.2. □

For additive models, H = {∅, {1}, …, {K}} all direct effects {k} are maximal, and we revover a special case of Theorem 3.2 by Schwabe (1994). As indicated there no general results can be expected for Φ_{q}-optimality, q > 0, of product type designs if larger parts of the parameter vector are of interest including interactions which are not maximal. However, for D-optimality (q = 0) a stronger result can be obtained.

Theorem 3.4. Let H be complete, and let H′ ⊆ H, H\H′ complete. If δ_{k}^{∗} is Doptimal in the kth marginal model, k ∈ H(H′), then δ^{∗} = π_{H(H′)}(⨂_{k∈H(H′)} δ_{k}^{∗}, δ_{H̄(H′)}) is D-optimal for β_{H′}, for every design δ_{H̄(H′)} on ×_{k∉H(H′)}T_{k}.

Proof. First note that δ_{k}^{∗} is D-optimal iff δ_{k}^{∗} is D-optimal for β_{k} (see e. g. Schwabe and Wierich, 1995, Lemma 6). Hence δ^{∗} is D-optimal for any set of maximal interactions by Theorem 3.3.

The proof will be completed by a finite induction: Therefore assume that the theorem is valid for H′′. Let H be maximal in H \ H′′. Then it suffices to show that the theorem is also valid for H′′ ∪ {H}, because every set H′ can be generated successively by adding relatively maximal interactions. In fact, by Lemma 2.1 (ii),

det C(β_{H′′∪{H}}; δ) = det C(β_{H′′}; δ) det C_{H\H′′}(β_{H}; δ)
≤ det C(β_{H′′}; δ^{∗}) det C_{H\H′′}(β_{H}; δ^{∗}),

where the inequality follows by Theorem 3.2 and the assumption. □

In particular, for H′ = H, the design ⨂_{k=1}^{K} δ_{k}^{∗} is D-optimal if each of its components δ_{k}^{∗} is D-optimal in the corresponding marginal model. Finally, note that the full product type design ⨂_{k=1}^{K} δ_{k}^{∗} is D-optimal for β_{H′} simultaneously in H and H′ ⊆ H when H and H \ H′ are complete.

4. Examples.

4.1. Multilinear regression on [−1, 1]^{K}. The full 2^{K} factorial design is simultaneously Φ_{q}-optimal for every set of maximal interactions in every complete interactions model

μ_{H}(t_{1}, …, t_{K}) = ∑_{H∈H} β_{H} ∏_{k∈H} t_{k}.

Moreover, the 2^{K} factorial is D-optimal for the whole parameter vector as well as for any set H′ of higher interactions, such that the remaining interaction structure H \ H′ is complete. In particular, in the model of complete M-factor interactions the 2^{K} factorial is the unique D-optimal design (the unique D-optimal design for all interactions of m up to M factors, i. e. H′ = {H; H ⊆ {1, …, K}, m ≤ |H| ≤ M}; the unique Φ_{q}-optimal design for all M-factor interactions) if 2M ≥ K. In case 2M < K also 2^{d} fractions may be optimal, 2M ≤ d < K. For example, in the additive model (M = 1) there exist optimal 2^{d} fractional factorials for d ≤ log_{2}(K + 1), K ≥ 3, constructed from Hadamard matrices.

4.2. K-way layout. Every complete K-way layout

μ_{H}(i(1), …, i(K)) = ∑_{H∈H} β_{H(i(1),...,i(K))}^{(H)},

i(k) = 1, …, I_{k}, k = 1, …, K, where H(i(1), …, i(K)) = (i(k_{1}), …, i(k_{|H|})) is the projection of (i(1), …, i(K)) onto the components k_{1}, …, k_{|H|} of H, can be reparametrized to the form considered above, if suitable identifiability conditions ∑ β_{i(k_{1}),...,i(k_{|H|})} = 0, H ∈ H, are imposed. (Here the summation is to be considered over every set of |H|−1 components). Hence the equireplicated design is D-optimal for testing every model with complete interaction structure H against each submodel with complete interactions structure H \ H′.

4. 3. Qualitative and quantitative factors. Following Kurotschka (1988) those models are of particular interest in applications which contain both kinds of factors: qualitative ones which may vary only over a finite number of levels (see e. g. 4.2) and quantitative ones which vary over a whole continuum of levels (see e. g. 4.1). To be more specific, in a two-way layout with additional straight line regression on [−1, 1] the model may be either additive

μ(i, j, t) = α_{i}^{(1)} + α_{j}^{(2)} + βt

or may have total interactions

μ(i, j, t) = α_{ij} + β_{ij}t

or, as indermediate models, the additive model is supplemented by some or all of the two-factor interactions α_{ij}, β_{i}^{(1)}t and β_{j}^{(2)}t. For all those 9 different models the 6 rainer schwabe

equireplicated design on {1, …, I} × {1, … J} × {−1, 1} is D-optimal for the whole parameter vector (under suitable identifiability conditions) as well as for estimating (testing) the higher interaction terms.

REFERENCES

Fan, K. (1954). Inequalities for eigenvalues of Hermitian matrices. Nat. Bur. Standards,

Appl. Math. Ser. 39 131–139. Hoel, P. G. (1965). Minimax designs in two dimensional regression. Ann. Math. Statist.

36 1097–1106. Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory).

Ann. Statist. 2 849–879. Kunert, J. (1983). Optimal design and refinement of the linear model with applications

to repeated measurement designs. Ann. Statist. 11 247–257. Kurotschka, V. G. (1988). Characterizations and examples of optimal experiments

with qualitative and quantitative factors. In Model-Oriented Data Analysis, Proceedings Eisenach 1987. (V. Fedorov and H. Läuter, eds.) 53–71. Springer, Berlin.

Pukelsheim, F. (1993). Optimal Design of Experiments. Wiley, New York. Rafajłowicz, E. and Myszka, W. (1988). Optimum experimental design for a regres-

sion on a hypercube - generalization of Hoel’s result. Ann. Inst. Statist. Math. 40 821–827.

Schwabe, R. (1994). Optimal designs for additive linear models. Statistics (to appear). Schwabe, R. (1995). Experimental design for linear models with higher order interac-

tion terms. In Symposia Gaussiana. Proceedings of the 2nd Gauss Symposium, Conference B: Statistical Sciences, München 1993. (H. Schneeweiß and V. Mammitzsch, eds.) 281–288. DeGruyter, Berlin.

Schwabe, R. and Wierich, W. (1995). D-optimal designs of experiments with noninteracting factors. J. Statist. Plann. Inference 44 371–384.

Freie Universität Berlin 1. Mathematisches Institut Arnimallee 2–6 14 195 Berlin Germany