What do We Have Confidence In?

Abstract

Chris and Fred discuss what it means to have confidence in something. Confidence can mean all things to all people. How confident are you as a person? How confident are you in someone else? What confidence do you have in a conclusion? Statisticians and reliability engineers need to talk to each other where the term confidence has a clearly defined and fixed meaning. But does it? How confident are you in the results of a test? How confident are you in a test that has 75 data points versus one that has 70? How confident will your boss be in you when you present the outcomes of these tests? Have these tests shaken YOUR confidence? Best you listen to this podcast.

Key Points

Join Chris and Fred as they discuss how we have confidence in our conclusions resulting from data analysis or any other investigation. We often use the term confidence intervals to describe the outcomes of an analysis. But it is more than that.

Topics include:

What is a confidence interval? A confidence interval is typically a range of numbers where you express a level of probability (confidence) that the true value of something lies within that range. There is a whole other discussion to be had on what this actually means – because in some schools of statistics confidence refers to the percentage of times you are right when you make such a claim! But more often than not, a confidence interval is used as first described above.
A test might give a confidence interval. But was the test (or model) right? You can use any model to create a confidence interval. But if you get the wrong model – then you get a different confidence interval. The good news is that if you use a ‘wrong’ model then the confidence interval will be larger – which means you don’t over-estimate your confidence. But by just getting the model right, you can drastically reduce the confidence interval you have on something. Which means you are less uncertain.
But confidence is not just about your statistical activity. It is the comfort a decision-maker has in your conclusions. Fred talked about the time he conducted an analysis on a solder joint. And the decision-maker gained a sense of confidence only when Fred described why and how he knew what the dominant failure mechanisms were. A failure mechanism is a physical, chemical or electrical process that ends up in failure. So you can have the best statistical process in the world, but if your decision-maker doesn’t have in you … then there is no confidence!
… and to have confidence, you need to focus on the right thing. This means knowing what the dominant failure mechanisms are. You might run a test on a component for a particular failure mechanism, but if it is the wrong one then you should have little confidence in the outcomes of your analysis. So we need to know which failure mechanisms matter (the ‘vital few.‘) And a very important deduction might be that if we work out what our dominant failure mechanism is, this is the thing we need to have the most confidence in. So get and get it!
… and to have confidence (when testing), we need to have the right test conditions. So how confident are you in understanding how customers or users will use your product, system or service? Because this needs to be part of the (lack of) confidence you have in your test results.
… and manufacturing can only make things worse. If this is the case, you may question the relevance in fussing so much over confidence in test results during design.

Enjoy an episode of Speaking of Reliability. Where you can join friends as they discuss reliability topics. Join us as we discuss topics ranging from design for reliability techniques to field data analysis approaches.