The planning of environmental or reliability testing becomes a question of sample size at some point.
It’s probably the most common question I hear as a reliability engineer – how many samples do we need.
Also, when evaluating supplier run test results, we need to understand the implications of the results, again based on the number of samples in the test. If the supplier runs 22 samples without failure over a test that replicates the shipping set of stresses, then we need a way to interpret those results.
We often use success testing (no expected or actual failures during the testing) to minimize the number of samples required for a test and still show some level of confidence for a specified reliability level.
The basis for success testing is the binomial distribution. The result of the applied stress results in the product either working or not. Binary results.
Recently I received a request to explain where the success testing sample size formula comes from, or it’s derivation. First here’s the formula:
$$ \large\displaystyle n=\frac{\ln \left( 1-C \right)}{\ln \left( R \right)}$$
Where C is confidence and R is the lower limit of the reliability.
Thus if planning test and you wanted to demonstrate the product was at least 90% reliable with 90% confidence, you would need to evaluate 22 units for the equivalent of a lifetime of use. In the shipping example above, the vendor’s testing shows the product would survive the shipping experience with 90% reliability with 90% confidence, for example.
It’s a simple formula. So, back to where it comes from.
Success Testing Sample Size Formula Derivation
Clopper and Pearson in 1934 wrote a paper detailing how to determine a confidence interval for a binomial distribution.
Let’s start with their result. They use the binomial distribution cumulative distribution expression set equal to 1 minus the confidence, or alpha as many would state it.
$$ \large\displaystyle \sum\limits_{i=0}^{r}{\left( \begin{array}{l}n\\i\end{array} \right)}{{\left( 1-R \right)}^{i}}{{R}^{n-i}}=1-C$$
C is the confidence, R is the lower limit of reliability give that confidence, n is the number of samples evaluated, and r is the number of failures experienced in the test.
Setting r = 0 is saying there will be or have been no failures, thus a success test. The first two terms after the summation reduce to 1 leaving
$$ \large\displaystyle{{R}^{n}}=1-C$$
Take the natural log of both sides brings n out of the exponent
$$ \large\displaystyle n\ln \left( R \right)=\ln \left( 1-C \right)$$
And rearrange to get the sample size formula
$$ \large\displaystyle n=\frac{\ln \left( 1-C \right)}{\ln \left( R \right)}$$
Hope that helps explain where this sample size formula comes from.
Related:
Sample Size – success testing (article)
Extended bogey testing (article)
OC Curve with Hypergeometric Method (article)
Please read article again, there are a couple of edits needed, can you spot them?
I like the article, it’s nice of you to share this with us, and I do appreciate it. I would like to add it would be more complete if you mentioned what the variables are, R for example is referred to but never explicitly mentioned in the article. So dare I ask, what is it? How is it measured, does it have units of measure, how does one arrive at it in the real world?
Thank you for taking a few moments to consider my observations.
Learning this stuff is challenging and right now I feel like a color blind artist if you see what I mean.
Hi Pete, R is the reliability, which is the probability of successful operation over a defined duration. As in 98% of units survived 2 years. It’s either the percentage that survive a duration or the probability a unit will survive.
In this formula it is the lower limit to an estimate of the reliability value given a sample from the population. So, in the formula when we say there is a 90% confidence of at least 90% reliability, that is the lower confidence limit of the estimate.
hope that helps.
Cheers,
Fred
Great article Fred!
thanks Micheal, much appreciated. cheers, Fred
Dear Fred,
Thanks for the explanation,
i have a question, i recently stated working Reliability topics for Automotive industry, with product called ECU i.e. Electronic control unit for Engine management (a aluminum housing) electronic product. mostly my work is limited to design validation (B sample or C sample products)
i would be asked to predict reliability either before start of test or after failure (failures from thermal, humidity and vibration like solder joint or complete functional failure)
to built a solid Reliability concepts where should i start from ?
master of Reliability Engineering like you .. which concepts to built up.
thank you
Maddirala
Hi Maddirala, pretty broad question. Knowing the functionality of a system is one step, then adding the effects of stress or aging in another.
Where to start? I would do though things. First, as you are already doing read and learn about reliability engineering on this site. There are many short tutorials and explanations of the tools and techniques that will allow you to estimate the impact of stress of product life.
Second, build you understanding of how failures occur. Some are based on the shear strength of a material, some are based on the aging characteristics of the material, others concern wear, corrosion, or something as simple as latent defects incorporating during manufacturing.
Third, (though of another idea) join Accendo Reliability and review the free course on the 14 ways to learn reliability engineering – each week you will receive a lesson which includes links, references and resources that may prove useful for you.
Cheers,
Fred