Sample Size Considerations
Abstract
Dianna and Fred discuss sample size considerations for reliability testing.
Key Points
Join Dianna and Fred as they discuss sample size considerations, tackling the frequently asked question: “How many samples do I need?”.
Topics include:
- Understanding the trade-offs between desired reliability/confidence levels and the massive sample sizes often required.
- Balancing statistical significance with practical significance and defining acceptable criteria.
- How tools like FMEA can help justify your required confidence level.
- Overcoming real-world constraints like budget, sample availability, test method limitations, and measurement error.
- What to do when traditional run-to-failure tests aren’t feasible.
Enjoy an episode of Speaking of Reliability. Where you can join friends as they discuss reliability topics. Join us as we discuss topics ranging from design for reliability techniques to field data analysis approaches.
Show Notes
Reliability engineers and quality engineers often face sample size considerations and questions. Fred and Dianna discuss that this question often arises, and the common answer is, “It depends”.
A challenge arises when attempting to prove very high reliability and confidence levels. The required sample sizes can be astronomically large. They discuss the importance of understanding the math behind sample size calculations but also acknowledge that simply running two samples or choosing an arbitrarily low confidence level (like 10% or 50%) makes the results practically meaningless.
Instead of solely focusing on statistical significance, consider practical significance and determine what criteria are considered “good enough”. Using tools like FMEA (Failure Mode and Effects Analysis) or hazard analysis can help justify the required confidence level for a test by linking it to the severity of potential failures. This analysis helps focus testing efforts on what truly matters for the customer and product performance.
Real-world constraints significantly impact sample size decisions, like the cost or availability of samples. It can also involve limitations of the test method itself, such as how long it takes to run a test or measure a sample. Measurement error in the test method can increase the required sample size to detect a meaningful difference. They highlight scenarios where practical constraints, not just statistics, dictate the testing approach.
Alternative strategies can be employed when traditional testing is difficult. Using field data from existing products can sometimes eliminate the need for new demonstration testing if the product has a long history of no failures at high volumes. Degradation testing, which monitors how a property changes over time rather than waiting for outright failure, can provide useful data with fewer samples by modeling the rate of degradation.
Overall, they emphasize that calculating the sample size from a statistics book is just the beginning. It requires a broader discussion considering all these factors. Ultimately, the best test is often the one you don’t need to do if the information is already available or the risk is deemed acceptable.
Leave a Reply