THE REDUCTION OF DESIGN PROBLEMS FOR MULTIVARIATE EXPERIMENTS TO UNIVARIATE POSSIBILITIES AND THEIR LIMITATIONS

Viktor G. Kurotschka, Rainer Schwabe^{1}

Freie Universität Berlin, 1. Mathematisches Institut, Arnimallee 2–6, D-14 195 Berlin, Germany

ABSTRACT

Situations are exhibited in which optimum designs for univariate models remain optimum for multiresponse models.

1. Introduction

A multivariate experimental situation is defined by the fact, that for each of the more or less complex experimental conditions t ∈ T the observation X(t) is multivariate, i. e. one observes instead of real valued random variables rdimensional real valued vectors

X(t) = (X^{(1)}(t),…, X^{(r)}(t)), t ∈ T,  (1)

with unknown location, here for simplicity expected responses

μ(t) := E(X(t)) = (E(X^{(1)}(t)),…, E(X^{(r)}(t))) ∈ IR^{r}, t ∈ T,  (2)

and correlation structure cov(X(t)) = Σ ∈ IR_{r×r}^{s,+} (the set of symmetric, positive definite r × r matrices) which is generally assumed to be invariant under change of experimental conditions and replications, because it represents the interior relationship of the components defining the r-dimensional vector of observations.

Despite the different non-linear approaches to experiments under even less complex experimental conditions t ∈ T the most appropriate setting is still apparently a linear parametrization β = (β_{1}, ..., β_{p}) of μ = ∑_{π=1}^{p} a_{π}^{T} β_{π}, for several reasons:

1 research supported by grant Ku719/2-1 of the Deutsche Forschungsgemeinschaft

1) In the case of a small experimental region, mathematically described by a finite set T (which covers the whole area of the classically called analysis of variance models or models with qualitative factors), the practically most useful parametrizations are necessarily linear.

2) In the case of a large set T, which is sufficiently well described by a convex compact subset of IR^{K}, the transparent approach via an appropriate system of adequate regression functions a_{1}, ..., a_{p} on T, approximating the unknown response μ of the experiment, is still the most attractive known, in particular, in view of the easier mathematical treatment which uses continuous mathematics instead of finite mathematics like derivatives and integrals instead of differences and sums.

3) The design problem in the non-linear approaches can be almost always adequately reduced to appropriate design problems arising from linearly parametrized responses of an experiment.

The optimality of designs for multivariate experiments are obtained, here, by a reduction to the associated univariate problems, which themselves may be of a complex structure involving qualitative and quantitative factors with different types of interactions. This means that, throughout the paper, the experimental region T is kept on the most general level including, in particular, experimental regions

T = ×_{k=1}^{K_{1}}{1, ..., I_{k}} × C,  (3)

where C is a convex subset of IR^{K_{2}}. The latter is the most general representation of the most realistic experimental situation, in which K_{1} qualitative factors are operating on levels 1, ..., I_{k}, k = 1, ..., K_{1}, and K_{2} quantitative factors are operating on levels in C. For those situations the whole optimum design theory has been developed by the authors and several other members of the research group in Berlin, and the results have been taken into account in the forthcoming Lecture Notes by the second author (Schwabe, 1995c) including a number of new and general results for the univariate case which, together with the present paper, opens a wide field of new applications.

Also, the main result by Krafft and Schaefer (1992) appears to be a variation of the reduction theme and can be regarded as a particular case of Theorem 5. Note that all the theorems of the present paper may be read as statements on the optimality of an exact design within the class of all exact designs and, alternatively, as statements on the optimality of a general design within the class of all those general designs. More detailed descriptions of these results are omitted because of the limited length of the paper.

2. Multivariate linear models under elementary designs

We consider a model for multivariate observations X(t) = (X^{(1)}(t), ..., X^{(r)}(t)) in which the mean response μ(t) = (μ_{1}(t), ..., μ_{r}(t)) can be linearly parametrized for each component μ_{ϱ}(t) = E(X^{(ϱ)}(t)), ϱ = 1, ..., r, in dependence on the setting t ∈ T for the influential factors. More precisely,

X^{(ϱ)}(t) = a_{ϱ}(t)^{T} β_{ϱ} + Z^{(ϱ)}(t),  (4)

where a_{ϱ} : T → IR^{p_{ϱ}} and β_{ϱ} ∈ IR^{p_{ϱ}} are the known regression functions resp. the unknown parameters for the ϱth component, and Z(t) = (Z^{(1)}(t), ..., Z^{(r)}(t)) is the random error associated with the observation X(t), E(Z(t)) = 0. Hence, a multivariate observation X(t) is described by

X(t) = a(t)^{T} β + Z(t),  (5)

where β ∈ IR^{p}, p = ∑_{ϱ=1}^{r} p_{ϱ}, is the vector of all unknown parameters

β = ( β_{(1)} ; β_{(2)} ; ⋮ ; β_{(r)} )  (6)

and a(t) is block diagonal

a(t) = ( \begin{matrix}a_{1}(t) & 0 & ⋯ & 0 \\ 0 & a_{2}(t) & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & a_{r}(t) \end{matrix} ) .  (7)

We may obtain N different observations

X_{n}(t_{n}) = a(t_{n})^{T} β + Z_{n}(t_{n})  (8)

with components

X_{n}^{(ϱ)}(t_{n}) = a_{ϱ}(t_{n})^{T} β_{ϱ} + Z_{n}^{(ϱ)}(t_{n})  (9)

at N possibly different adjustments t_{n} of the factors of influence, n = 1, ..., N.

An elementary design d of size N is given as an N-dimensional vector (t_{1}, ..., t_{N})^{T} which contains the adjustments for the N different experiments.

With X_{d}^{(ϱ)} = (X_{1}^{(ϱ)}(t_{1}), ..., X_{N}^{(ϱ)}(t_{N}))^{T} and Z_{d}^{(ϱ)} = (Z_{1}^{(ϱ)}(t_{1}), ..., Z_{N}^{(ϱ)}(t_{N}))^{T} being the vectors of observations and errors, respectively, the observations for the ϱth component can be written in the usual matrix notation

X_{d}^{(ϱ)} = A_{d}^{(ϱ)}β_{ϱ} + Z_{d}^{(ϱ)}  (10)

where

A_{d}^{(ϱ)} = ( a_{ϱ}(t_{1})^{T} ; a_{ϱ}(t_{2})^{T} ; ⋮ ; a_{ϱ}(t_{N})^{T} )  (11)

is the design matrix for the ϱth component.

In accordance with the common notations in multivariate analysis we vectorize the model and arrange its components successively

X_{d} = ( X_{d}^{(1)} ; X_{d}^{(2)} ; ⋮ ; X_{d}^{(r)} ) ,  Z_{d} = ( Z_{d}^{(1)} ; Z_{d}^{(2)} ; ⋮ ; Z_{d}^{(r)} ) .  (12)

Hence we obtain

X_{d} = A_{d}β + Z_{d}  (13)

where the design matrix A_{d} is block diagonal

A_{d} = ( \begin{matrix}A_{d}^{(1)} & 0 & ⋯ & 0 \\ 0 & A_{d}^{(2)} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & A_{d}^{(r)} \end{matrix} ) .  (14)

Note that a vectorization X̃_{d} = (X_{1}(t_{1})^{T} , ..., X_{N}(t_{N})^{T})^{T} would be more appealing for design purposes. However, dealing with the rearranged vector X̃_{d} leads to considerably more complicated notations, in particular, in Section 3.

Next we assume that the underlying error structure is homogeneous, i. e. cov(Z_{n}(t_{n})) = Σ, n = 1, ..., N, and that the observations are uncorrelated, i. e. cov(Z_{m}(t_{m}), Z_{n}(t_{n})) = 0 for m ≠ n. This results in a covariance structure cov(Z_{d}) = Σ⊗E_{N} for the vectorized errors Z_{d}, where E_{N} is the N ×N identity matrix and ⊗ denotes the Kronecker product. To avoid trivial cases we assume that Σ > 0 is positive definite.

In the linear model (13) the general (weighted) least squares estimator

β̂_{d} = (A_{d}^{T} (Σ ⊗ E_{N})^{−1}A_{d})^{−1}A_{d}^{T} (Σ ⊗ E_{N})^{−1}X_{d}  (15)

is the best linear unbiased estimator of the unknown parameters β. We will consider only such designs for which β is linearly identifiable (estimable), i. e. for which A_{d} has full column rank. The quality of the estimator β̂_{d} based on the design d and, hence, the quality of the design d itself is measured by the covariance matrix

cov(β̂_{d}) = (A_{d}^{T} (Σ ⊗ E_{N})^{−1}A_{d})^{−1}.  (16)

If we denote the entries of Σ^{−1} by u_{ϱϱ′}, ϱ, ϱ′ = 1, ..., r, then the covariance matrix cov(β̂_{d}) can be calculated more explicitly in terms of the design matrices associated with the components

cov(β̂_{d}) = ( \begin{matrix}u_{11}A_{d}^{(1)^{T}} A_{d}^{(1)} & u_{12}A_{d}^{(1)^{T}} A_{d}^{(2)} & ⋯ & u_{1r}A_{d}^{(1)^{T}} A_{d}^{(r)} \\ u_{21}A_{d}^{(2)^{T}} A_{d}^{(1)} & u_{22}A_{d}^{(2)^{T}} A_{d}^{(2)} & ⋯ & u_{2r}A_{d}^{(2)^{T}} A_{d}^{(r)} \\ ⋮ & ⋮ & ... & ⋮ \\ u_{r1}A_{d}^{(r)^{T}} A_{d}^{(1)} & u_{r2}A_{d}^{(r)^{T}} A_{d}^{(2)} & ⋯ & u_{rr}A_{d}^{(r)^{T}} A_{d}^{(r)} \end{matrix} )^{−1} . (17)

In general, it is not possible to minimize the covariance matrix cov(β̂_{d}) uniformly, i. e. in the positive-definite sense. As a compromise we have to look at real valued functions of cov(β̂_{d}), like the determinant, the trace, or the largest eigenvalue, which are isotonic. In particular, a design d^{∗} will be called D- (Aresp. E-) optimum if it minimizes the determinant (the trace resp. the largest eigenvalue of cov(β̂_{d}).

3. Homogeneous components

Additionally, in the present section, we assume that the mean responses μ_{ϱ} are modeled in the same way for each component, i. e. the regression functions a_{ϱ} for the components coincide: a_{ϱ} = a_{1}, ϱ = 1, ..., r. Note that still, in general, the unknown parameters β_{ϱ} and, hence, the mean responses will be different for the single components. In this situation also the componentwise design matrices A_{d}^{(ϱ)} coincide, and both the regression function a and the design matrix A_{d} factorize according to (cf (7) and (14))

a(t) = E_{r} ⊗ a_{1}(t)  (18)

and

A_{d} = E_{r} ⊗ A_{d}^{(1)}.  (19)

Because of the equi-modelling of the components we can calculate the general least squares estimator (15) by using (19) as

β̂_{d} = (Σ^{−1} ⊗ A_{d}^{(1)^{T}} A_{d}^{(1)})^{−1}(Σ^{−1} ⊗ A_{d}^{(1)^{T}})X_{d}
= (E_{r} ⊗ (A_{d}^{(1)^{T}} A_{d}^{(1)})^{−1}A_{d}^{(1)^{T}})X_{d}  (20)

From (20) we see that the knowledge of Σ is not required for the calculation of β̂_{d} and that the general least squares estimator coincides with the ordinary least squares estimator

β̂_{d}^{ols} = (A_{d}^{T} A_{d})^{−1}A_{d}^{T} X_{d}.  (21)

Moreover, the covariance matrix (16) factorizes

cov(β̂_{d}) = Σ ⊗ (A_{d}^{(1)^{T}} A_{d}^{(1)})^{−1}.  (22)

According to the Kronecker product structure (22) the eigenvalues of the covariance matrix cov(β̂_{d}) factorize, i. e. let λ_{Σ} be the vector of eigenvalues of Σ, λ_{d}^{(1)} the vector of eigenvalues of (A_{d}^{(1)^{T}} A_{d}^{(1)})^{−1}, and λ_{d} the vector of eigenvalues of cov(β̂_{d}), then

λ_{d} = λ_{Σ} ⊗ λ_{d}^{(1)}.  (23)

In particular, we obtain for the determinant

det(cov(β̂_{d})) = det(Σ)^{p_{1}} det(A_{d}^{(1)^{T}} A_{d}^{(1)})^{−r},  (24)

for the trace

tr(cov(β̂_{d})) = tr(Σ) · tr((A_{d}^{(1)^{T}} A_{d}^{(1)})^{−1})  (25)

and for the largest eigenvalue

λ_{max}(cov(β̂_{d})) = λ_{max}(Σ)λ_{max}((A_{d}^{(1)^{T}} A_{d}^{(1)})^{−1}).  (26)

Hence the optimization with respect to the usual D- (A- resp. E-) criterion reduces to the corresponding univariate optimization problem.

Theorem 1. If d^{∗} is D- (A- resp. E-) optimum in the univariate model with mean response μ_{1}(t) = a_{1}(t)β_{1}, then d^{∗} is D- (A- resp. E-) optimum in the multivariate model with homogeneous components μ_{ϱ}(t) = a_{1}(t)β_{ϱ}.

Remark 1. Because of (23) the result of Theorem 1 can be extended to the whole class of Φ_{q}-criteria based on the eigenvalues of the covariance matrix (for a definition see e. g. Pázman, 1986).

Remark 2. Further generalizations are straightforward if we are interested in a linear aspect ψ of the unknown parameters β which is of the form ψ(β) = (L_{0} ⊗ L_{1})β where ψ_{1}(β_{1}) = L_{1}β_{1} is the corresponding linear aspect in the univariate model, L_{1} ∈ IR^{s_{1}×p_{1}} and L_{0} ∈ IR^{s_{0}×r}. Particular cases are covered by L_{0} = E_{r}, if we are interested in the same linear aspect for each component, and by L_{0} with zero row sums, if we are interested in differences between the components.

Finally, we want to mention that in the particular situation of the present section the model can be written in a more appealing way

X_{d} = A_{d}^{(1)}B + Z_{d},  (27)

where X_{d} = (X_{d}^{(1)}, ..., X_{d}^{(r)}), Z_{d} = (Z_{d}^{(1)}, ..., Z_{d}^{(r)}) and B = (β_{1}, ..., β_{r}), such that each row of (27) represents an r-dimensional observation X_{n}(t_{n}). However, the form (27) is not convenient for design considerations because, then, we have to deal with a matrix shape B of parameters rather than the vectorized β.

4. Uncorrelated components

If the components of the multivariate observations do not influence each other then the covariance matrix Σ of a single observation is diagonal

Σ = ( \begin{matrix}σ_{1}^{2} & 0 & ⋯ & 0 \\ 0 & σ_{2}^{2} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & σ_{r}^{2} \end{matrix} ) .  (28)

The induced covariance structure Σ ⊗ E_{N} of the whole observational vector X_{d} is also diagonal

Σ ⊗ E_{N} = ( \begin{matrix}σ_{1}^{2}E_{N} & 0 & ⋯ & 0 \\ 0 & σ_{2}^{2}E_{N} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & σ_{r}^{2}E_{N} \end{matrix} ) .  (29)

Using the block diagonal structure (7) of the design matrix A_{d} we obtain from (17) that the covariance matrix of the general least squares estimator becomes block diagonal

cov(β̂_{d}) = ( \begin{matrix}σ_{1}^{2}(A_{d}^{(1)^{T}} A_{d}^{(1)})^{−1} & 0 & ⋯ & 0 \\ 0 & σ_{2}^{2}(A_{d}^{(2)^{T}} A_{d}^{(2)})^{−1} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & σ_{r}^{2}(A_{d}^{(r)^{T}} A_{d}^{(r)})^{−1} \end{matrix} ) .

(30)

Moreover, the least squares estimator β̂_{d} can be written as

β̂_{d} = ( (A_{d}^{(1)^{T}} A_{d}^{(1)})^{−1}A_{d}^{(1)^{T}} X_{d}^{(1)} ; (A_{d}^{(2)^{T}} A_{d}^{(2)})^{−1}A_{d}^{(2)^{T}} X_{d}^{(2)} ; ⋮ ; (A_{d}^{(r)^{T}} A_{d}^{(r)})^{−1}A_{d}^{(r)^{T}} X_{d}^{(r)} ) .  (31)

Again the calculation of β̂_{d} does not require any further knowledge of the diagonal entries σ_{ϱ}^{2} of Σ and the general least squares estimator coincides with the ordinary β̂_{d}^{ols} = (A_{d}^{T} A_{d})^{−1}A_{d}^{T} X_{d}.

Denote by λ_{d}^{(ϱ)} the vector of eigenvalues of (A_{d}^{(ϱ)^{T}} A_{d}^{(ϱ)})^{−1}. Then the vector λ_{d} of eigenvalues of cov(β̂_{d}) can be partitioned according to

λ_{d} = ( σ_{1}^{2}λ_{d}^{(1)} ; σ_{2}^{2}λ_{d}^{(2)} ; ⋮ ; σ_{r}^{2}λ_{d}^{(r)} ) .  (32)

A design is D-optimum if it minimizes ∏_{ϱ=1}^{r} det(A_{d}^{(ϱ)^{T}} A_{d}^{(ϱ)})^{−1}, irrespectively of the actual values of σ_{1}^{2}, ..., σ_{r}^{2}. Alternatively, an A-optimum design minimizes ∑_{ϱ=1}^{r} σ_{ϱ}^{2} tr((A_{d}^{(ϱ)^{T}} A_{d}^{(ϱ)})^{−1}) and an E-optimum design aims at minimizing

max_{ϱ=1,...,r} σ_{ϱ}^{2} λ_{max}((A_{d}^{(ϱ)^{T}} A_{d}^{(ϱ)})^{−1}), respectively.

In particular, if a design is simultaneously optimum for all components then it solves also the corresponding multivariate optimization problem.

Theorem 2. If d^{∗} is D- (A- resp. E-) optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, then d^{∗} is D- (A- resp. E-) optimum in the multivariate model with uncorrelated components.

5. Components with increasing complexity

In general situations the underlying models may differ for the single components, e. g. regressions of different precisions may be fitted, the components may be affected by different sets of factors of influence, or different interaction structures may occur. In the setting of increasing complexity we assume that the components can be rearranged in such a way that the regression functions a_{ϱ} are included in each other in ascending order

a_{1} = f_{1},  a_{2} = ( \begin{matrix}f_{1} \\ f_{2} \end{matrix} ) ,  ...  , a_{r} = ( f_{1} ; f_{2} ; ⋮ ; f_{r} ) ,  (33)

f_{ϱ} : T → IR^{q_{ϱ}}, q_{ϱ} = p_{ϱ} − p_{ϱ−1}, where some of the f_{ϱ} might be empty (q_{ϱ} = 0). Now, for a fixed design d, we can reparametrize the models for each component by orthogonalization with respect to its predecessor.

Thus we define f̃_{1} = f_{1}, f̃_{ϱ} = f_{ϱ} − A_{d,ϱ}^{T}A_{d}^{(ϱ−1)}(A_{d}^{(ϱ−1)^{T}} A_{d}^{(ϱ−1)})^{−1}a_{ϱ−1}, where A_{d,ϱ}^{T} = (f_{ϱ}(t_{1}), ..., f_{ϱ}(t_{N})), and ã_{ϱ} = (f̃_{1}^{T} , ..., f̃_{ϱ}^{T})^{T}. Then the components of the

corresponding univariate design matrices Ã_{d}^{(ϱ)}, which are defined by (11), are orthogonal, i. e.

Ã_{d,ϱ}^{T}Ã_{d,ϱ′} = 0,  (34)

ϱ ≠ ϱ′, where, again, Ã_{d,ϱ}^{T} = (f̃_{ϱ}(t_{1}), ..., f̃_{ϱ}(t_{N})). Moreover, the design matri-

ces are related by linear transformations Ã_{d}^{(ϱ)^{T}} = L_{ϱ}A_{d}^{(ϱ)^{T}} and Ã_{d}^{T} = LA_{d}^{T}, where L_{ϱ} and L are lower triangular matrices with all their diagonal entries equal to one (see Appendix). Next, we rearrange the regression functions in the multivariate model according to the components f̃_{ϱ} and, hence, permute the parameters. Let

ā = ( \begin{matrix}& & E_{r} ⊗ f̃_{1} \\ 0 & & E_{r−1} ⊗ f̃_{2} \\ ⋮ & ... & ... \\ 0 & ⋯ & 0 & E_{r−ϱ+1} ⊗ f̃_{ϱ} \\ ⋮ & ... & ... \\ 0 & ⋯ & 0 & E_{r} ⊗ f̃_{1} \end{matrix} ) ,  (35)

then ā = Qa for some permutation matrix Q. The design matrices Ā_{d} are defined in analogy to (14)

Ā_{d} = ( ā_{1}(t_{1})^{T} ; ⋮ ; ā_{1}(t_{N})^{T} ; ā_{2}(t_{1})^{T} ; ⋮ ; ā_{r}(t_{N})^{T} ) ,  (36)

where ā_{ϱ} is the ϱth component of ā. As det(L) = | det(Q)| = 1 we obtain

det(cov(β̂_{d})) = det(Ã_{d}^{T} (Σ^{−1} ⊗ E_{N})Ã_{d})^{−1} = det(Ā_{d}^{T} (Σ^{−1} ⊗ E_{N})Ā_{d})^{−1} (37)

by (16). Denote by U_{ϱ} = (u_{ϱ′ϱ′′})_{ϱ′=ϱ,...,r}^{ϱ′′=ϱ,...,r} that part of the inverse covariance matrix Σ^{−1} associated with the components ϱ, ..., r in which the regression functions f_{ϱ} are present, U_{ϱ} = (0, E_{r−ϱ+1})Σ^{−1}(0, E_{r−ϱ+1})^{T}, U_{1} = Σ^{−1}. Then

Ā_{d}^{T} (Σ^{−1} ⊗ E_{N})Ā_{d} = ( \begin{matrix}U_{1} ⊗ Ã_{d,1}^{T}Ã_{d,1} & 0 & ⋯ & 0 \\ 0 & U_{2} ⊗ Ã_{d,2}^{T}Ã_{d,2} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & U_{r} ⊗ Ã_{d,r}^{T}Ã_{d,r} \end{matrix} )

(38)

is block diagonal and, hence,

det(cov(β̂_{d})) = (∏_{ϱ=1}^{r} det(U_{ϱ})^{−q_{ϱ}}) (∏_{ϱ=1}^{r} det(Ã_{d,ϱ}^{T}Ã_{d,ϱ})^{−(r−ϱ+1)})  (39)

in view of (37). Note that, by convention, det(Ã_{d,ϱ}^{T}Ã_{d,ϱ}) = 1 if q_{ϱ} = 0. By

the orthogonality (34) of Ã_{d,ϱ} and as det(L_{ϱ}) = 1 we have det(A_{d}^{(ϱ)^{T}} A_{d}^{(ϱ)}) =

det(Ã_{d}^{(ϱ)^{T}} Ã_{d}^{(ϱ)}) = ∏_{ϱ′=1}^{ϱ} det(Ã_{d,ϱ′}^{T} Ã_{d,ϱ′}) so that, finally,

det(cov(β̂_{d})) = (∏_{ϱ=1}^{r} det(U_{ϱ})^{−q_{ϱ}}) (∏_{ϱ=1}^{r} det(A_{d}^{(ϱ)^{T}} A_{d}^{(ϱ)})^{−1}) .  (40)

Note that formula (40) is the main result obtained by Krafft and Schaefer (1992). Now, the determinant of the covariance matrix is minimized if the determiant of the covariance matrices (A_{d}^{(ϱ)^{T}} A_{d}^{(ϱ)})^{−1} can be minimized simultaneously for each component.

As reparametrizations do not affect the criterion of D-optimality the concept of increasing complexity can be paraphrased as a_{ϱ} = C_{ϱ}a_{ϱ+1} for some p_{ϱ} ×p_{ϱ+1} matrix C_{ϱ}, ϱ = 1, ..., r − 1.

Theorem 3. If d^{∗} is D-optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, a_{ϱ} = C_{ϱ}a_{ϱ+1}, then d^{∗} is D-optimum in the multivariate model with increasing complexity.

For models with increasing complexity, in which a design d^{∗} exists which is simultaneously D-optimum for all components, particular examples are given by (i) polynomial regression of degrees zero and one, (ii) trigonometric regression of different degrees up to degree (N − 1)/2, and (iii) K-factor models with different depth of interactions (see Schwabe, 1995a).

6. Components with mutually exclusive complexity

In the general setting, that the regression functions are not necessarily included in each other, further assumptions have to be made on the optimum design d^{∗}. Essentially, optimality is also required for augmented models which are formed by the regression functions associated with any pair of components. This situation will be exhibited for a bivariate model, first.

In general, a pair of regression functions a_{1} and a_{2} can be reparametrized in such a way that there is a common regression vector f_{0} and there are mutually different components f_{1} and f_{2} such that the set of entries in (f_{0}^{T} , f_{1}^{T} , f_{2}^{T})^{T} is linearly independent on T,

a_{1} = ( \begin{matrix}f_{0} \\ f_{1} \end{matrix} ) ,  a_{2} = ( \begin{matrix}f_{0} \\ f_{2} \end{matrix} ) .  (41)

As in the previous section we orthogonalize f_{1} and f_{2} with respect to f_{0} : T → IR^{p_{0}} and a fixed design d, f̃_{ϱ} = f_{ϱ} − A_{d,ϱ}^{T}A_{d,0}(A_{d,0}^{T}A_{d,0})^{−1}f_{0}, ϱ = 1, 2, where A_{d,ϱ′}^{T} = (f_{ϱ′}(t_{1}), ..., f_{ϱ′}(t_{N})), ϱ′ = 0, 1, 2. Again, the corresponding transformation matrices are lower diagonal with all diagonal elements equal to one. Next we rearrange the regression functions similar to (35)

ā = ( \begin{matrix}E_{2} ⊗ f_{0} \\ f̃_{1} & 0 \\ 0 & f̃_{2} \end{matrix} ) .  (42)

We define Ā_{d} as in (36) and obtain det(cov(β̂_{d})) = det(Ā_{d}^{T} (Σ^{−1} ⊗ E_{N})Ā_{d})^{−1} (cf (39)) and

Ā_{d}^{T} (Σ^{−1} ⊗ E_{N})Ā_{d} = ( \begin{matrix}Σ^{−1} ⊗ A_{d,0}^{T}A_{d,0} & 0 & 0 \\ 0 & u_{11}Ã_{d,1}^{T}Ã_{d,1} & u_{12}Ã_{d,1}^{T}Ã_{d,2} \\ 0 & u_{12}Ã_{d,2}^{T}Ã_{d,1} & u_{22}Ã_{d,2}^{T}Ã_{d,2} \end{matrix} ) (43)

similar to (38). Combining these results we get

det(cov(β̂_{d})) ≥ det(Σ)^{p_{0}}u_{11}^{−(p_{1}−p_{0})}u_{22}^{−(p_{2}−p_{0})} ∏_{ϱ=1}^{2} det(A_{d}^{(ϱ)^{T}} A_{d}^{(ϱ)})^{−1}  (44)

with equality if, and only if, either the components are uncorrelated, u_{12} = 0, or f̃_{1} and f̃_{2} are orthogonal with respect to the design d, Ã_{d,1}^{T}Ã_{d,2} = 0. The latter condition can equivalently be formulated as a factorization for f_{1} and f_{2} adjusted for f_{0}, A_{d,1}^{T}A_{d,2} = A_{d,1}^{T}A_{d,0}(A_{d,0}^{T}A_{d,0})^{−1}A_{d,0}^{T}A_{d,2}.

Theorem 4. If d^{∗} is D-optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, a_{ϱ} = (f_{0}^{T} , f_{ϱ}^{T})^{T}, (f_{0}^{T} , f_{1}^{T} , f_{2}^{T})^{T} linearly independent, and if A_{d^{∗},1}^{T}A_{d^{∗},2} = A_{d^{∗},1}^{T}A_{d^{∗},0}(A_{d^{∗},0}^{T}A_{d^{∗},0})^{−1}A_{d^{∗},0}^{T}A_{d^{∗},2}, then d^{∗} is D-optimum in the bivariate model with mutual different complexity.

If f_{1} is omitted in Theorem 4 the factorization condition vanishes and we recover the bivariate version of Theorem 3.

Additional models which are covered by Theorem 4 include trigonometric regression, in case not all lower order frequencies are present, and K-factor models, in which the components are influenced by different factors or in which interactions between different factors are active.

For the special case that only the constant term is in common, f_{0} = 1, the factorization condition of Theorem 4 simplifies to \frac{1}{N} ∑_{n=1}^{N} f_{1}(t_{n})f_{2}(t_{n})^{T} =

(\frac{1}{N} ∑_{n=1}^{N} f_{1}(t_{n}))(\frac{1}{N} ∑_{n=1}^{N} f_{2}(t_{n}))^{T}. Moreover, in this case, the factorization condition implies that d^{∗} is also D-optimum for the augmented univariate model with mean response μ_{1,2}(t) = β_{0} + f_{1}(t)^{T} β_{1} + f_{2}(t)^{T} β_{2}.

Finally, if f_{0} is omitted in Theorem 4, then the orthogonality of f_{1} and f_{2}, themselves, is required, A_{d^{∗},1}^{T}A_{d^{∗},2} = 0. Also in this case the factorization condition implies the D-optimality of d^{∗} for the corresponding augmented univariate model with mean response μ_{1,2}(t) = f_{1}(t)^{T} β_{1} + f_{2}(t)^{T} β_{2}.

For components with simultaneously common regression part f_{0} the derivation of Theorem 4 can be extended to more than two components in a straightforward manner to obtain a result as follows.

Corollary 1. If d^{∗} is D-optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, a_{ϱ} = (f_{0}^{T} , f_{ϱ}^{T})^{T}, (f_{0}^{T} , f_{1}^{T} , ..., f_{r}^{T})^{T} linearly independent, and if A_{d^{∗},ϱ}^{T}A_{d^{∗},ϱ′} = A_{d^{∗},ϱ}^{T}A_{d^{∗},0}(A_{d^{∗},0}^{T}A_{d^{∗},0})^{−1}A_{d^{∗},0}^{T}A_{d^{∗},ϱ′}, ϱ ≠ ϱ′, then d^{∗} is D-optimum in the multivariate model with mutual different complexity.

We note that the factorization condition can be equivalently rewritten as A_{d^{∗}}^{(ϱ)^{T}} A_{d^{∗}}^{(ϱ′)} = A_{d^{∗}}^{(ϱ)^{T}} A_{d^{∗},0}(A_{d^{∗},0}^{T}A_{d^{∗},0})^{−1}A_{d^{∗},0}^{T}A_{d^{∗}}^{(ϱ′)}. In the general case factorization conditions are requested for the regression functions adjusted to their respective common part f_{ϱ,ϱ′}. To simplify notations we write (a_{ϱ}) for the set {a_{ϱ1}, ..., a_{ϱp_{ϱ}}} of all entries (functions) in a_{ϱ}, and (f_{ϱ,ϱ′}) analogously.

Theorem 5. If d^{∗} is D-optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, and if for every pair ϱ and ϱ′, the sets of functions (f_{ϱ,ϱ′}) = (a_{ϱ}) ∩ (a_{ϱ′}), (a_{ϱ}) \ (f_{ϱ,ϱ′}) and (a_{ϱ′}) \ (f_{ϱ,ϱ′}) are linearly independent and the factorization A_{d^{∗}}^{(ϱ)^{T}} A_{d^{∗}}^{(ϱ′)} = A_{d^{∗}}^{(ϱ)^{T}} A_{d^{∗},ϱ,ϱ′}(A_{d^{∗},ϱ,ϱ′}^{T}A_{d^{∗},ϱ,ϱ′})^{−1}A_{d^{∗},ϱ,ϱ′}^{T}A_{d^{∗}}^{(ϱ′)} holds, then d^{∗} is D-optimum in the multivariate model with mutual different complexity.

The proof can be performed by a more sophisticated rearrangement of the regression functions. We mention that, as in the bivariate case, Theorem 3 follows from Theorem 5 because the factorization condition vanishes for components with increasing complexity.

7. Applications

Within the classical terrain of experimental design the so-called analysis of variance models, i. e. experiments influenced by a number of qualitative factors, play an important role. For such experiments the most general region of experimental conditions is given by T = ×_{k=1}^{K}{1, ..., I_{k}} defining K qualitative factors, with the kth one operating at I_{k} possible levels 1, ..., I_{k}, k = 1, ..., K, and with different types and orders of interactions between subsets of the K factors.

The second classical approach in experimental design is concerned with the so-called regression models, i. e. experiments influenced by a number of quantitative factors. For such experiments the most general region of experimental conditions is given by a convex subset T = C of IR^{K} defining K quantitative factors operating at all possible levels t ∈ T and, again, with different types and orders of interactions between subsets of the K factors.

The specification of these two types of experiments gives rise to a whole class of possible linear parametrizations for univariate linear models in which both kinds of factors, qualitative and quantitative ones, are active and may interact with each other in various kinds.

In the multivariate case each component μ_{ϱ} of the multivariate response μ may have, in general, a different type of parametrization according to the number of factors actually affecting the components and different types of interactions within the classes of factors involved in the individual components. So the main Theorem 5 provides the adequate tool for such realistic cases to solve the design problem for multivariate experiments by solving the associated design problems in the setup of the individual components.

To show only part of the huge variety of parametrizations which fall under the conditions of Theorem 5 and its specialization to hierarchical models (Theorem 3) we present the following general classes of examples:

Example 1. If different factors are affecting different sets of factors this can be generally described by responses μ_{ϱ}(t_{1}, ..., t_{r}) = β_{ϱ,0} + f_{ϱ}(t_{ϱ})^{T} β_{ϱ,ϱ}, ϱ = 1, ..., r, where t = (t_{1}, ..., t_{r}) ∈ T = ×_{ϱ=1}^{r}T_{ϱ}, and T_{ϱ} itself can be a K_{ϱ}-dimensional set defining K_{ϱ} factors of influence.

To be more specific, in an analysis of variance setting f_{ϱ} may describe any analysis of variance model in K_{ϱ} factors operating on T_{ϱ} = ×_{k=1}^{K_{ϱ}}{1, ..., I_{ϱ,k}}. In a regression setting f_{ϱ} may describe polynomial or trigonometric regression in K_{ϱ} factors operating on a convex subset C_{ϱ} of IR^{K_{ϱ}}.

Example 2. The situation that different interaction structures are present for the individual components leads for the particular case of two factors of influence to a hierarchical model. In the analysis of variance setting there are essentially two types of different responses μ_{1}(i, j) = α_{1,i} + β_{1,j} without interactions and μ_{2}(i, j) = α_{2,i} + β_{2,j} + γ_{2,ij} with interactions, respectively. For more than two factors more complicated interaction structures may arise, e. g. in the simplest case responses μ_{1}(i_{1}, i_{2}, i_{3}) = α_{1,i_{1}}^{(1)} + α_{1,i_{2}}^{(2)} + α_{1,i_{3}}^{(3)} + γ_{1,i_{1}i_{2}}^{(1,2)} and μ_{2}(i_{1}, i_{2}, i_{3}) = α_{2,i_{1}}^{(1)} + α_{2,i_{2}}^{(2)} + α_{2,i_{3}}^{(3)} + γ_{2,i_{1},i_{3}}^{(1,3)} with interactions between the first and second resp. the first and third factor only.

Example 3. Another typical example for hierarchical model is obtained in trigonometric regression, where the response is modeled by a Fourier expansion μ_{ϱ}(t) = β_{ϱ,0} + ∑_{m=1}^{M_{ϱ}}(β_{ϱ,2m−1} sin(2πmt) + β_{ϱ,2m} cos(2πmt)) up to order M_{ϱ} on t ∈ T = [0, 1). In more than one dimension also partial interaction structures can be considered like complete and partial first and second order expansions.

Also the still larger class of more complex, realistic multivariate experiments where both kinds of factors, qualitative and quantitative ones, are simultaneously affecting the response can be treated by Theorem 5 using our most recent results for the univariate case, as exhibited in the papers by Kurotschka (1984, 1988) and Schwabe (1995a, 1995b) and the forthcoming lecture notes (Schwabe, 1995c).

8. General designs

The information matrix I(d) of an elementary design d is defined as the inverse of the corresponding covariance matrix cov(β̂_{d})

I(d) = A_{d}^{T} (Σ ⊗ E_{N})^{−1}A_{d}.  (45)

By rearranging terms it can be seen that the information matrix is the sum of the information contained in each single setting

I(d) = ∑_{n=1}^{N} a(t_{n})Σ^{−1}a(t_{n})^{T} .  (46)

If there are only few different settings and, hence, many replications in an elementary design d it is more convenient to write

d = ( \begin{matrix}t_{1} & t_{2} & ⋯ & t_{J} \\ N_{1} & N_{2} & ⋯ & N_{J} \end{matrix} )  (47)

where the settings t_{j} are mutually different and N_{j} is the number of replications at the setting t_{j}, ∑_{j=1}^{J} N_{j} = N. Hence we obtain for the information matrix

I(d) = ∑_{j=1}^{J} N_{j}a(t_{j})Σ^{−1}a(t_{j})^{T} .  (48)

As the sample size N is fixed, an elementary design d is determined by its relative frequencies w_{j} = \frac{N_{j}}{N} at the actual settings. Thus d may be identified with a dicrete measure δ_{d} on a finite support, which assigns weights w_{j} to its support points t_{j}.

To solve the integer optimization problem for N_{j} is often a difficult task. Therefore it is reasonable to embed the optimization problem in a continuous setting and to drop the requirement of w_{j} to be an integer multiple of \frac{1}{N}. Hence, as a general design δ we define any measure on T, normalized to one, which is concentrated on a finite number of supporting points. In accordance with (48) we define the information matrix

I(δ) = ∫ a(t)Σ^{−1}a(t)^{T} δ(dt),  (49)

such that, for any elementary design d of sample size N, I(δ_{d}) = \frac{1}{N} I(d) is a normalized version of the information matrix I(d).

Besides this traditional extension of the design problem to general designs, in order to embed the optimization problem into a richer set which is convex, a more substantial reason for considering general designs is evident. Because the supposition of a prespecified number of units with prespecified variation is sometimes rather artificial a more general idea should be associated with the concept of general designs (see Kurotschka, 1988, or Schwabe, 1995b). Assume that we may observe at certain design points t_{1},…, t_{J} with intensity proportional to w_{j} > 0, ∑_{j=1}^{J} w_{j} = 1, i. e. cov(X_{j}(t_{j})) = w_{j}^{−1}Σ, j = 1,…, J, then in this general multivariate linear model the best linear unbiased estimator is given by the weighted least squares estimator β̂_{δ} = (A_{δ}^{T} (Σ^{−1} ⊗ W_{δ})A_{δ})^{−1}A_{δ}^{T} (Σ^{−1} ⊗ W_{δ})X_{δ} where

X_{δ} = ( X_{δ}^{(1)} ; X_{δ}^{(2)} ; ⋮ ; X_{δ}^{(r)} ) ,  (50)

X_{δ}^{(ϱ)} = (X_{j}^{(ϱ)}(t_{j}))_{j=1,...,J}, are the vectorized observations, cov(X_{δ}) = Σ⊗W_{δ}^{−1},

A_{δ} = ( \begin{matrix}A_{δ}^{(1)} & 0 & ⋯ & 0 \\ 0 & A_{δ}^{(2)} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & A_{δ}^{(r)} \end{matrix} ) ,  (51)

A_{δ}^{(ϱ)} = (a_{ϱ}(t_{1}), ..., a_{ϱ}(t_{J}))^{T}, is the design matrix associated with the supporting points t_{j} of the design δ and the diagonal matrix

W_{δ} = ( \begin{matrix}w_{1} & 0 & ⋯ & 0 \\ 0 & w_{2} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & w_{J} \end{matrix} )  (52)

is the intensity matrix of the observations. We notice that

I(δ) = A_{δ}^{T} (Σ^{−1} ⊗ W_{δ})A_{δ} = ∑_{j=1}^{J} w_{j}a(t_{j})Σ^{−1}a(t_{j})^{T} ,  (53)

where cov(β̂_{δ}) = I(δ)^{−1}. Hence a general design δ which assigns weights w_{j} to the supporting points t_{j}, j = 1,…, J, has to be understood as an instruction which demands that J different observations are to be made at the design points t_{j} with intensity proportional to w_{j}, j = 1,…, J.

A general design δ^{∗} is called D- (A- resp. E-) optimum if it minimizes the determinant (the trace resp. the largest eigenvalue of the inverse I(δ)^{−1} of the information matrix. Note that I(δ_{d})^{−1} coincides with the covariance matrix cov(β̂_{d}) for an elementary design d with sample size N, up to a multiplicative constant \frac{1}{N}, and hence the generalizations of the criteria are straightforward.

All the results obtained for elementary fixed sample size designs d in the preceding sections are also valid for general designs δ^{∗} and are formulated next.

Theorem 1^{∗}. If δ^{∗} is D- (A- resp. E-) optimum in the univariate model with mean response μ_{1}(t) = a_{1}(t)β_{1}, then δ^{∗} is D- (A- resp. E-) optimum in the multivariate model with homogeneous components μ_{ϱ}(t) = a_{1}(t)β_{ϱ}.

Note that this simple theorem, restricted to D-optimality, is the main result of Chang (1994) obtained by using such demanding tools as the equivalence theorem (see Theorem 6 below).

Theorem 2^{∗}. If δ^{∗} is D- (A- resp. E-) optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, then δ^{∗} is D- (A- resp. E-) optimum in the multivariate model with uncorrelated components.

Theorem 3^{∗}. If δ^{∗} is D-optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, a_{ϱ} = C_{ϱ}a_{ϱ+1}, then δ^{∗} is D-optimum in the multivariate model with increasing complexity.

Theorem 4^{∗}. If δ^{∗} is D-optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, a_{ϱ} = (f_{0}^{T} , f_{ϱ}^{T})^{T}, (f_{0}^{T} , f_{1}^{T} , f_{2}^{T})^{T} linearly independent, and if ∫ f_{1}f_{2}^{T} dδ^{∗} = ∫ f_{1}f_{0}^{T} dδ^{∗}(∫ f_{0}f_{0}^{T} dδ^{∗})^{−1} ∫ f_{0}f_{2}^{T} dδ^{∗}, then δ^{∗} is D-optimum in the bivariate model with mutual different complexity.

For the bivariate case we consider the augmented univariate model with mean response μ_{1,2}(t) = f_{0}(t)^{T} β_{0} + f_{1}(t)^{T} β_{1} + f_{2}(t)^{T} β_{2}, such that the corresponding regression function a_{1,2} = (f_{0}^{T} , f_{1}^{T} , f_{2}^{T})^{T} collects all functions occuring in the univariate components. If f_{0} = 1 or if f_{0} is not present in the model then for the information matrix I_{1,2}(δ) = ∫ a_{1,2}a_{1,2}^{T} dδ it is straightforward by a refinement argument that its determinant is dominated by the information contained in the univariate component models

det(I_{1,2}(δ)) ≤ det(∫ f̃_{1}f̃_{1}^{T} dδ)^{−1} det(∫ f̃_{2}f̃_{2}^{T} dδ)^{−1}
≤ det(∫ f̃_{1}f̃_{1}^{T} dδ^{∗})^{−1} det(∫ f̃_{2}f̃_{2}^{T} dδ^{∗})^{−1}.  (54)

Hence, the factorization condition of Theorem 4^{∗} implies the D-optimality of δ^{∗} in the augmented model. In the next section we will establish the result that the additional D-optimality of δ^{∗} in the augmented univariate model is, indeed, necessary for the D-optimality in the bivariate model for general covariance structure Σ^{−1}.

Theorem 5^{∗}. If δ^{∗} is D-optimum in each univariate model with mean response μ_{ϱ}(t) = a_{ϱ}(t)β_{ϱ}, and if for every pair ϱ and ϱ′, the sets of functions (f_{ϱ,ϱ′}) = (a_{ϱ}) ∩ (a_{ϱ′}), (a_{ϱ}) \ (f_{ϱ,ϱ′}) and (a_{ϱ′}) \ (f_{ϱ,ϱ′}) are linearly independent and the factorization ∫ a_{ϱ}a_{ϱ′}^{T} dδ^{∗} = ∫ a_{ϱ}f_{ϱ,ϱ′}^{T} dδ^{∗}(∫ f_{ϱ,ϱ′}f_{ϱ,ϱ′}^{T} dδ^{∗})^{−1} ∫ f_{ϱ,ϱ′}a_{ϱ′}^{T} dδ^{∗} holds, then δ^{∗} is D-optimum in the multivariate model with mutual different complexity.

Finally, we mention that a useful tool for checking a general design δ^{∗} to be D-optimum in a multivariate model is the following equivalence theorem (see Fedorov, 1972, p. 212) of which we will make use in the subsequent section.

Theorem 6.

(i) A design δ^{∗} is D-optimum if, and only if, tr(Σ^{−1}a(t)I(δ^{∗})^{−1}a(t)^{T}) ≤ p, for every t ∈ T.

(ii) If δ^{∗} is D-optimum, then δ^{∗}(tr(Σ^{−1}a(t)I(δ^{∗})^{−1}a(t)^{T}) = p) = 1.

9. A counterexample and its theoretical background

Although the results of the preceding sections seem to include all practically relevant general parametrizations for multivariate linear experiments the following example, which is more of mathematical interest, shows that the generally accepted opinion is misleading that factorization is always possible. This is caused by the fact that the concept of orthogonalization, which is a universally useful tool in the design of experiments, may conflict with the meaning of the parameters if applied across the components of a multivariate experiment:

To keep notations as simple as possible we consider a bivariate model with different one-parameter univariate components for which the same design δ_{1}^{∗} = δ_{2}^{∗} is optimum. In particular, we will deal with a purely linear component,

X^{(1)}(t) = β_{1}t + Z^{(1)},  (55)

and with a purely quadratic component,

X^{(2)}(t) = β_{2}t^{2} + Z^{(2)},  (56)

on the design region T = [τ, 1], for various −1 ≤ τ < 1. For τ > −1 the unique optimum design for both univariate models is given by the one point design δ_{ϱ}^{∗} = ε_{1} concentrated at the location 1 of the largest modulus of both regression functions t and t^{2} on T.

The criterion under consideration for the bivariate model will be Doptimality, which is not affected by scale transformations of the componentwise variances. Hence we may assume without loss of generality

Σ(c) = ( \begin{matrix}1 & c \\ c & 1 \end{matrix} )  (57)

for various correlations c, −1 < c < 1.

As δ_{ϱ}^{∗} is not D-optimum for the augmented univariate model μ_{1,2}(t) = β_{1}t + β_{2}t^{2}, it is to be expected that δ_{ϱ}^{∗} is not D-optimum for the bivariate model as well, at least, for large correlations |c|.

First we compare δ_{ϱ}^{∗} with the best two-point design δ(τ, c) = w^{∗}(τ, c)ε_{τ} + (1−w^{∗}(τ, c))ε_{1} concentrated on the endpoints of the design region [τ, 1], τ ≠ 0. That two-point design dominates δ_{ϱ}^{∗} if

w(τ, c) = \frac{1}{2} \frac{τ^{2}(1 − τ)^{2} − 2(1 − τ^{3})(1 − ϱ^{2})}{τ^{2}(1 − τ)^{2} − (1 − τ^{3})^{2}(1 − ϱ^{2})} > 0,  (58)

in which case w^{∗}(τ, c) = w(τ, c). Note that w(τ, c) = w(τ, −c). Hence, for every τ there is a critical value

c_{krit}(τ) = (1 − \frac{1}{2} \frac{τ^{2}(1 − τ)^{2}}{1 − τ^{3}} )^{1/2}  (59)

for the correlation beyond which (|c| > c_{krit}(τ)) the design δ_{ϱ}^{∗} is dominated by δ(τ, c). Note that δ(τ, c) is not necessarily the D-optimum design for the bivariate model, although the Equivalence Theorem 6 ensures that the optimum design is concentrated on, at most, two supporting points including 1, at least, for τ ≥ 0. Note also that c_{krit}(τ) → 0 for τ → −1.

For the standard design region T = [0, 1] the D-optimum design δ^{∗}(c) is supported by, at most, two points including the right endpoint t_{1} = 1. For |c| ≤ min_{0<τ<1} c_{krit}(τ) = 0.981 the one-point design δ_{ϱ}^{∗} is D-optimum for the bivariate model. However, for stronger correlations |c| → 1, it can be checked that a second supporting point t_{2} is required for the D-optimum design and that t_{2} approaches 0.5 and the corresponding optimal weight w_{2}^{∗} = w^{∗}(t_{2}, c) tends to \frac{1}{2}. Hence δ^{∗}(c) converges to the design δ^{∗}(1) which is D-optimum in the augmented univariate model, as |c| → 1.

In general, we consider

φ_{c}(δ) = det(∫ f_{0}f_{0}^{T} dδ)^{2} det(∫ f̃_{1}f̃_{1}^{T} dδ)
× det (∫ f̃_{2}f̃_{2}^{T} dδ − c^{2} ∫ f̃_{2}f̃_{1}^{T} dδ(∫ f̃_{1}f̃_{1}^{T} dδ)^{−1} ∫ f̃_{1}f̃_{2}^{T} dδ) . (60)

Then, given Σ(c), a design δ^{∗}(c) is D-optimum for the bivariate model if, and only if, it maximizes φ_{c}(δ), |c| < 1. If δ^{∗}(c) → δ(1) as |c| → 1, then φ_{c}(δ^{∗}(c)) → φ_{1}(δ(1)), at least, for continuous regression functions on a compact design region T. Now, for δ^{∗}(1) which maximizes φ_{1}(δ), we have φ_{c}(δ^{∗}(1)) ≤ φ_{c}(δ^{∗}(c)) and φ_{c}(δ^{∗}(1)) → φ_{1}(δ^{∗}(1)), as |c| → 1. Hence φ_{1}(δ^{∗}(1)) ≤ φ_{1}(δ(1)) and δ(1) is also optimum with repsect to φ_{1}. Thus we can formulate the following result

Theorem 7. Let δ^{∗}(c) be D-optimum in the bivariate model with mutual different complexity and underlying covariance structure Σ(c), let a be continuous and T be compact. Then every limiting design δ(1) of (δ^{∗}(c)), as |c| → 1, maximizes φ_{1}(δ).

In the particular case that either f_{0} = 1 or (f_{0}) = ∅ the design δ^{∗}(1) maximizes φ_{1}(δ) if, and only if, δ^{∗}(1) is D-optimum in the augmented univariate model. Moreover, if δ^{∗}(1) is the unique D-optimum design in the augmented univariate model, then δ^{∗}(c) → δ^{∗}(1) as |c| → 1. Hence, in this case, it is necessary for the simultaneous D-optimality of δ^{∗} in the bivariate model for every covariance structure Σ that δ^{∗} is also D-optimum in the augmented univariate model.

For the specific example, above, this situation occurs if τ = 1 and the simultaneously optimum design δ^{∗} assigns equal weights \frac{1}{2} to both endpoints −1 and 1 of the design region T = [−1, 1]. Note that in this case also the factorization conditions are satisfied.

10. Appendix

By ℓ_{ϱ} = A_{d,ϱ}^{T}A_{d}^{(ϱ−1)}(A_{d}^{(ϱ−1)^{T}} A_{d}^{(ϱ−1)})^{−1} we denote the componentwise orthogonalization matrix of f_{ϱ} with respect to a_{ϱ−1} and d. Then the transformation matrix

L_{ϱ} = ( \begin{matrix}E_{q_{1}} & & 0 & 0 & ⋯ & 0 \\ ℓ_{2} & & E_{q_{2}} & 0 & ⋯ & 0 \\ & ℓ_{3} & & E_{q_{3}} & ⋯ & 0 \\ & & ... & & ... & ⋮ \\ & & ℓ_{ϱ} & & & E_{q_{ϱ}} \end{matrix} )  (61)

for the univariate model, ã_{ϱ} = L_{ϱ}a_{ϱ}, is lower triangular with ones on the diagonal (L_{1} = E_{q_{1}}, q_{1} = p_{1}). Finally, the transformation matrix L for the multivariate model, ã = La, is block diagonal

L = ( \begin{matrix}L_{1} & 0 & ⋯ & 0 \\ 0 & L_{2} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & L_{r} \end{matrix} )  (62)

and, hence, by (61), lower triangular with all diagonal entries one.

REFERENCES

Chang, S. I. (1994). Some properties of multiresponse D-optimal designs. J.

Math. Anal. Appl. 184, 256–262. Fedorov, V. V. (1972). Theory of Optimal Experiments. Academic Press, New

York. Krafft, O. and Schaefer, M. (1992). D-optimal designs for a multivariate re-

gression model. J. Multivariate Anal. 42, 130–140. Kurotschka, V. G. (1984). A general approach to optimum design of experi-

ments with qualitative and quantitative factors. In: Statistics: Applications and New Directions: Proceedings of the Indian Statistical Institute Golden Jubilee International Conference Calcutta 1981. (J. K. Ghosh and J. Roy (eds.)). Indian Statistical Institute, Calcutta. 353–368.

Kurotschka, V. G. (1988). Characterizations and examples of optimal experiments with qualitative and quantitative factors. In: Model-Oriented Data Analysis, Proceedings Eisenach 1987. (V. Fedorov and H. Läuter (eds.)). Springer, Berlin. 53–71.

Linde, A. van der (1977). Versuchsplanung für multivariate lineare Modelle.

Diplomarbeit, Freie Universität Berlin, Fachbereich Mathematik. 175 pages. Pázman, A. (1986). Foundations of Optimum Experimental Design. Reidel,

Dordrecht. Schwabe, R. (1995a). Experimental design for linear models with higher order

interaction terms. In: Symposia Gaussiana. Proceedings of the 2nd Gauss Symposium, München 1993. Conference B: Statistical Sciences (V. Mammitzsch and H. Schneeweiß (eds.)). DeGruyter, Berlin. 281–288.

Schwabe, R. (1995b). Optimal designs for additive linear models. Statistics (in press).

Schwabe, R. (1995c). Optimum Designs for Multi-factor Models. Lecture Notes

in Statistics. Springer, New York (to appear). Wegscheider, K. (1977). Optimale Versuchsplanung für multivariate Beobach-

tungsprozesse mit kontinuierlich variierenden Versuchsbedingungen. Diplomarbeit, Freie Universität Berlin, Fachbereich Mathematik. 282 pages.