Orthogonal Comparisons and ANOVA

In the previous lesson, we learned that researchers use comparisons to address follow-up questions that are not answered by a standard, omnibus analysis of variance.

This lesson focuses on a special type of comparison, called an orthogonal comparison.

What is a Comparison?

To be sure everyone is on the same page, let's review what we know about comparisons. A comparison is a weighted sum of mean scores. Mathematically, a comparison can be expressed as:

k L = Σ cj Xj j=1

In addition, all comparisons are subject to the following constraint:

k Σ j=1
  nj cj = 0  

In the equations above, L is the value of the comparison, cj is a coefficient (weight) for treatment j, Xj is the mean score for treatment j, nj is the number of subjects assigned to treatment j , and k is the number of treatment groups.

With balanced designs (i.e., designs in which sample size is constant across treatment groups), the necessary condition for a comparison reduces to:

Σ cj = 0

What is an Orthogonal Comparison?

Two comparisons (L1 and L2) are orthogonal if the following is true:

Σ nj c1j c2j  = 0

where nj is the sample size in treatment group j, c1j is a coefficient for treatment j in comparison L1, and c2j is a coefficient for treatment j in comparison L2.

With balanced designs (i.e., designs in which sample size is constant across treatment groups), the necessary condition for two comparisons to be orthogonal reduces to:

Σ c1j c2j  = 0

For any experiment with three or more treatment groups, it is possible to define different sets of orthogonal comparisons. However, each set of orthogonal comparisons can have a maximum of k - 1 comparisons, where k is the number of treatment groups.

Orthogonal Comparisons and Sums of Squares

The sum of squares for an orthogonal comparison is computed in the same way as the sum of squares for an ordinary comparison. When Σ njcj = 0, the sum of squares for a comparison can be computed from the following formula:

SSi = ( Σ njcij Xj )2 / Σ njc2ij

where SSi is the sum of squares for comparison Li , nj is the sample size in Group j , cij is the coefficient (weight) for level j in the formula for comparison Li, and Xj is the mean score for Group j .

With a balanced design, the sum of squares for a comparison ( Li ) can be computed from a simpler formula:

SSi = n * Li2 / Σ c2ij

where SSi is the sum of squares for comparison Li , Li is the value of the comparison, n is the sample size in each group, and cij is the coefficient (weight) for level j in the formula for comparison Li.

The sums of squares for a complete set of orthogonal comparisons (SSj) are related to the sum of squares for the treatment effect (SST) in a standard analysis of variance, as shown below:

  SST =  
k-1 Σ j=1
  SSj  

SST = SS1 + SS2 + ... + SSk-1

A treatment effect with k levels has a maximum of k - 1 orthogonal comparisons. There is an additive relationship between the sum of square for a treatment effect and the sums of squares for k - 1 orthogonal comparisons. The sum of squares for the treatment effect is equal to the sum of sums of squares for k - 1 orthogonal comparisons.

Orthogonal Comparisons and Uncorrelated Data

When conducting multiple follow-up tests to supplement an omnibus analysis of variance, it is sometimes desirable to formulate hypotheses that can be tested using uncorrelated data from the experiment.

When the data used to test one hypothesis are uncorrelated with the data used to test another hypothesis, the hypothesis tests are independent. It is as if each hypothesis test were conducted using data from different experiments.

So, how do you distinguish hypotheses that can be tested using uncorrelated data from those that cannot? You look for orthogonal comparisons. Hypotheses represented by orthogonal comparisons use uncorrelated data (aka, nonoverlapping data) for significance tests. Hypotheses represented by nonorthogonal comparisons use correlated data (aka, overlapping data).

Test Your Understanding

Problem 1

You are conducting a single-factor experiment with four treatment groups. Here are five comparisons relevant to that experiment:

  • L1 = X1 - X2
  • L2 = X2 - X3
  • L3 = X3 - X4
  • L4 = 0.5X1 + 0.5X2 - 0.5X3 - 0.5X4
  • L5 = 0.5X1 - 0.5X2 + 0.5X3 - 0.5X4

In which of the following sets of comparisons are all three comparisons orthogonal? (Assume a balanced design; i.e., equal sample size in each group.)

(A) L1, L2, and L3
(B) L1, L2, and L4
(C) L1, L2, and L5
(D) L1, L3, and L4
(E) L1, L3, and L5

Solution

The correct answer is (D). With a balanced design, the necessary condition for two comparisons to be orthogonal is:

Σ c1j c2j  = 0

The table below shows weights (cj) for each comparison:

Comparison c1 c2 c3 c4
L1 1 -1 0 0
L2 0 1 -1 0
L3 0 0 1 -1
L4 0.5 0.5 -0.5 -0.5
L5 0.5 -0.5 0.5 -0.5

From the table, it is evident that the necessary condition for orthogonality is satisfied for the set of comparisons composed of L1, L3, and L4. That is,

Σ c1j c3j  = (1)(0) + (-1)(0) + (0)(1) + (0)(-1) = 0

Σ c1j c4j  = (1)(0.5) + (-1)(0.5) + (0)(-0.5) + (0)(-0.5) = 0

Σ c3j c4j  = (0)(0.5) + (0)(0.5) + (1)(-0.5) + (1)(-0.5) = 0

The necessary condition for orthogonality is not satisfied for any of the other sets of comparisons. For example, comparisons L1 and L2 are not orthogonal because:

Σ c1j c2j  = (1)(0) + (-1)(1) + (0)(-1) + (0)(0) = -1

Because Σ c1j c2j is not equal to zero, we know that L1 and L2 are not orthogonal; so options A, B, and C cannot be correct answers to this problem.

Similarly, comparisons L1 and L5 are not orthogonal because:

Σ c1j c5j  = (1)(0.5) + (-1)(-0.5) + (0)(0.5) + (0)(-0.5) = 1

Because Σ c1j c5j is not equal to zero, option E cannot be a correct answer to this problem.


Problem 2

You are conducting a single-factor experiment with three treatment groups, equal sample sizes in each group. And you are interested in three comparisons:

  • L1 = X1 - X2
  • L2 = X2 - X3
  • L3 = X1 - 0.5X2 - 0.5X3

The sum of squares for each comparison is shown below:

SS1 SS2 SS3
10 20 30

What is the sum of squares for the treatment effect in this experiment?

(A) 20
(B) 30
(C) 40
(D) 50
(E) 60

Solution

The correct answer is (D). Here's the logic for solving this problem:

  • There is an additive relationship between the sum of squares for a treatment effect and the sums of squares for k - 1 orthogonal comparisons. That is,

    SST = SS1 + SS2 + ... + SSk-1

    where SST is the sum of squares for the treatment effect, SSi is the sum of squares for comparison i, and k is the number of treatment groups in the experiment.
  • This experiment has three treatment groups; thus, k = 3.
  • If we can identify k - 1 orthogonal comparisons, we can use the formula above to compute the sum of squares for the treatment effect.

With a balanced design, the necessary condition for two comparisons to be orthogonal is:

Σ c1j c2j  = 0

The table below shows weights (cj) for all three comparisons:

Comparison c1 c2 c3
L1 1 -1 0
L2 0 1 -1
L3 1 -0.5 -0.5

Comparisons L1 and L2 are not orthogonal because Σ c1j c2j ≠ 0. Similarly, comparisons L1 and L3 are not orthogonal because Σ c1j c3j ≠ 0.

Σ c1j c2j  = (1)(0) + (-1)(1) + (0)(-1) = -1

Σ c1j c3j  = (1)(1) + (-1)(-0.5) + (0)(-0.5) = 1.5

However, comparisons L2 and L2 are orthogonal because Σ c2j c3j = 0, as shown below:

Σ c2j c3j  = (0)(1) + (1)(-0.5) + (-1)(-0.5) = 0

Because comparisons L2 and L3 represent k - 1 orthogonal comparisons. we can apply the formula below to compute the sum of squares for the treatment effect:

SST = SS2 + SS3

SST = 20 + 30 = 50