One of the simplest ways to learn design of experiments, DOE, is to just give it a try.
The Taguchi DOE approach uses orthogonal arrays. This subset of the possible approaches to DOE simplifies the process to create and analyze experiments.
Let’s plan a simple experiment using the Taguchi DOE approach.
The problem
Web page conversion optimization is a complex mix of layout, copy, and style.
A page inviting a visitor to become a member relies on matching the visitor’s expectations, building trust, and inviting the reader to click on the “join now” button.
The intent is to convert a visitor to a member.
Like most engineering problems there are plenty of variables involved and no one solution that will always work well.
Some variables influence the results more than others, and it is not always clear which variable is most important.
There are general guidelines and engineering principles that apply, yet the optimization for a particular situation is a suitable application of design of experiments.
The factors and levels
Factors are the variables under consideration.
In this example, we have the page title size, the page title wording, and the call to action button wording.
There are other variables (factors) that we could consider, yet here we are keeping the example simple thus limited to 3 factors.
Each factor has levels. A level is a setting for the factor.
For example, for page title size, we are experimenting with small and larger font size, in particular, 24 pixel and 48-pixel font heights. Again, for simplicity, we are using just two levels per factor.
You can explore more levels per factor, especially when the factor may have a non-linearly influence on the result.
For the page title wording, we have two phrases to explore and likewise for the call to action button wording.
In practice the process of designing an experiment including using engineering judgment to select the factors and appropriate levels for each factor. The DOE set of
The DOE set of run (experiments) will use combinations of the factor levels, therefore each combination has to be physically possible to create and operate.
While tempting to explore the extremes of each factor, the intent is to explore the range of viable solutions.
The array
In the Taguchi DOE approach, there are a collection of arrays.
These are the select combinations of factors and levels that form an orthogonal array.
The design includes a balanced representation of each level for all factors, thus permitting the analysis to isolate each factor to determine the effect of that factor’s levels on the results.
For our example problem, we have 3 factors each with 2 levels. This is often represented using 32 notation.
The Taguchi L4 array accommodates 3 factors with 2 levels each. The array looks like:
| L4 (23) | |||
| Run no. | A | B | C |
| 1 | 1 | 1 | 1 |
| 2 | 1 | 2 | 2 |
| 3 | 2 | 1 | 2 |
| 4 | 2 | 2 | 1 |
Where each run is a unique prototype or arrangement of factor levels.
The A, B, and C columns represent the 3 factors. The 1 and 2’s within the factor columns represent the respective factor levels.
Note the array has 4 runs which are half the 8 possible arrangements of factors and levels. This represents the economy of the factional factorial array.
We will learn about all 8 arrangements doing only 4 experiments.
Let’s assign the factors as follows:
| L4 (23) | |||
| Run no. | A – Title Font Size | B Title Phrase | C CTA Phrase |
| 1 | 24 px | Phrase 1 | CTA 1 |
| 2 | 24 px | Phrase 2 | CTA 2 |
| 3 | 48 px | Phrase 1 | CTA 2 |
| 4 | 48 px | Phrase 2 | CTA 1 |
The array now provides a recipe to create four web pages for the experiment.
On the site, there is a bit of code the assigns visitors to one of the four prototype pages and counts the number of visits and conversions (click through to becoming a member).
I set the experiment to run till there are 1,000 visitors per page.
For most physical prototypes, you may only create 1 or a few replicates for each run.
Using multiple copies of each run permits evaluate of the variability of the results for each run in the array.
The experiment will still work with only one prototype per run, yet as with statistics in general, more samples is desirable.
The objective
In the web page design, we want to determine which level of each the factors leads to the most conversions.
We are measuring the conversion rate, which we expect to be relatively small numbers (in the 2% to 8% range).
A higher conversion rate suggests the design of the page is effective at encouraging visitors to become site members.
Depending on your objective and measurements, you may want to optimize the factors to find the smallest value, say with scrap rates.
Or, you may want to focus on a target value or nominal value.
Each is possible and we’ll explore how each of these different objectives lead to slightly different analyses in a future article.
Leave a Reply