When to use DOE (Design of Experiments)?
What is design of experiments, or DOE? What do we use it for and what is it all about? We talk about when we might want to use it during the design cycle, and we do this without getting into all of the how-to and mathematical equations.
DOE is an iterative process, so don’t expect a huge once-and-done event. Practitioners recommend a general rule, that “no more than about 25% of the available resources should be invested in the first experiment.” (Montgomery pp. 17) This allows for subsequent experiments and confirmation testing.
If we need to better understand what factors have an effect on our design output and to what degree, then DOE is a way for us to understand that. Plus, it takes into account the interactions between factors, so it is a more thorough way to experiment. It is also efficient with samples. With the multiple designed experiment recipes available, it’s worthwhile reaching out to a quality or reliability engineer to help plan a DOE.
Citations
NIST/SEMATECH e-Handbook of Statistical Methods, http://www.itl.nist.gov/div898/handbook/, sections 4.3.1 and 5.1.1, accessed October 23, 2021.
Montgomery, Douglas C. Design and Analysis of Experiments, 5th Ed. John Wiley & Sons, Inc. 2001, pp 1-11, 17, 427-430.
For an example, visit ASQ: asq.org/quality-resources/design-of-experiments
Episode Transcript
What is design of experiments, or DOE? What do we use it for and what is it all about? Let’s talk about when you might want to use it during the design cycle. We’ll do this without getting into all of the how-to and mathematical equations, right after this brief introduction.
Hello and welcome to quality during design the place to use quality thinking to create products others love, for less. My name is Dianna. I’m a senior level quality professional and engineer with over 20 years of experience in manufacturing and design. Listen in and then join the conversation at QualityDuringDesign.com.
When we think of DOE, we may think of matrices and hear of factorial designs. But, without getting into all of that, what is DOE? When would we use it during the design cycle, and for what? By the way, the terms “experimental design”, “designed experiments”, “design of experiments”, and DOE are all referring to the same thing, which is what we’re talking about today. Within the umbrella of DOE, there are comparative experiments, ANOVA (or analysis of variance), block or square designs, and factorial designs – these are all designed experiment recipes that are using these methods.
Here’s the big idea:
If we think about our general model of a process or system, we’ll know that there’s an input, an output, and then factors that affect the output. With this model, we want to know what affect the factors have on our output. Do they have an effect at all? What kind of effect does it have: a positive or a negative effect? How much effect does it have?
To figure this out, there are different experimental strategies we could use.
One is a best-guess approach, where we use our technical knowledge and experience. This can work, but the shortfall is that our initial best guess may not do what we want, and then we spend lots of time doing other experiments. Or, we find a solution that works and then stop, never really finding the best solution.
Another way we could experiment is the one-factor-at-a-time, where we set a baseline and then vary the factors one at a time. We can get results with this, too. But then we may be missing any effects that the factors have on the output when they interact with each other. One factor may not affect another factor the same way at its different levels.
Or, we could do a factorial experiment, where we’re varying the factors together instead of one at a time. The factorial model is very efficient with the number of tests, and considers the interaction effects between factors. If we have 4 or more factors, then it’s usually unnecessary to run all possible combinations, and that’s where we can use a fractional factorial experiment.
With experimental problems, there are two aspects: the design of the experiment and the way we plan to analyze the results. Statistical design of experiments is when we plan the experiment so we can use statistical methods to analyze the data.
Here are some typical iterations of tests, using DOE.
When we’re first approaching an experimental problem, we may not fully understand the factors. So, we take a wide-view approach and can conduct a screening design. The screening test design helps us to filter, or screen-out, the factors that have an effect on our output. We can also call this characterizing our process, because we’re better understanding which factors affect our output. After performing a screening test, we’ll have a list of factors that have an effect on our output, and those factors can have a rank of most effect to least effect.
Once we understand the factors that have an effect, we can choose to design a test for modeling. In modeling test design, we’re looking to create a mathematical model of our input and output. We will better understand the level of effect that our factors have on our output. And we would have a way to accurately predict the output.
Then, we may decide we want to optimize our design or process. We want to learn at what level the factor produces the best output. This is where we use a response surface methodology, which uses contour plots. We first perform iterative tests to find the optimum response. With using contour plots, we can think of it as climbing a hill to get to the peak, or optimum. Then we perform additional experiments to create a mathematical model that includes our factors. We use that model to optimize our output.
Once we can draw some conclusions, we follow-up our DOE with confirmation testing, to make sure that the conclusions we made from the experiment is actually true.
We can apply experimental design methods to the product design process. If we have a function that our design needs to perform, then what are the factors that could affect that function? We can use our knowledge to identify the factors we think apply, and then develop a prototype where the factors can be changed over their range.
What are the highlights about DOE? There are multiple factors affecting an output, we use iterative test cycles with a level of rigor in test and scope, and there’s a minimal number of samples to learn the most that we can. Our iterative tests can lead us to better understanding the factors that have an effect, and may allow us to create a mathematical model to our system. A highlight about this method: it’s iterative. We learn from multiple, small, designed tests, each building upon the other until we understand our system at the level we need. Practitioners recommend a general rule, that “no more than about 25% of the available resources should be invested in the first experiment.” (Montgomery pp. 17) This allows for subsequent experiments and confirmation testing.
What’s today’s insight to action? Keep DOE in mind as a tool to help understand factors of a product design. It can be used to better understand what effects our factors have on our outputs. It’s an iterative process, so don’t expect a huge once-and-done event. But, if we need to better understand what factors have an effect on our design output, and to what degree, then DOE is a way for us to understand those answers. Plus, it takes into account the interactions between factors, so it is a more thorough way to experiment without breaking the bank with the number of products to test. With the multiple designed experiment recipes available, it’s worthwhile reaching out to a quality or reliability engineer to help plan a DOE.
Please visit this podcast blog and others at qualityduringdesign.com. Subscribe to the weekly newsletter to keep in touch. If you like this podcast or have a suggestion for an upcoming episode, let me know. You can find me at qualityduringdesign.com, on LinkedIn, or you could leave me a voicemail at 484-341-0238. This has been a production of Denney Enterprises. Thanks for listening!
Leave a Reply