What is an Outlier?

Abstract

Chris and Fred discuss the concept of an ‘outlier.’ Depending on the definition, an outlier is something that is different, non-representative or somehow separate from the ‘main body’ of something. We often use the term ‘outlier’ when it comes to statistical analysis of something – specifically when we see something that is beyond what we expect to see. So what do we do with ‘outliers’ … particularly when it comes to reliability analysis? It may be tempting to exclude them (particularly if it makes the outcome of your analysis ‘better’!). We have not seen the exclusion of ‘outliers’ have a happy ending when it comes to reliability. Would you like to learn more? Listen to this podcast!

Key Points

Join Chris and Fred as they discuss what it means to have an outlier. An outlier is something that is not representative. But when it comes to testing … an outlier is nothing more than something that you don’t expect or ‘want’ to see. How can we say this? If you are testing, you are trying to learn something about a process. So how do you know enough about the thing you are testing to be able to work out what an outlier is … given that you are so unsure about what it is that you have decided to do the test in the first place!

Topics include:

Outliers need prejudice. If you exclude a test result, it simply means that it doesn’t fit your expectations or preconceived idea of what the test result ought to be. This is OK if you have a fundamental understanding of the thing you are examining. In real estate, studies on human heights and so on, we do know enough about the thing we are studying to be able to work out what is ‘typical’ … which means we can exclude ‘outliers.’ But if you are testing a prototype you simply don’t know enough. Period.
But that ‘thing’ was a mistake – its never going to happen. So why do you even test? What was wrong with your test that allows ‘mistakes’ to occur? In some industries, a 5 per cent failure rate is too high to sustain a profit. Even though 5 per cent is a ‘small’ number that may tempt us to declare failure as an outlier.
If you think it is an ‘outlier’ … then follow through and confirm it. Analyze your outlier. Conduct a physics of failure based root cause analysis. If you conduct a thorough analysis and can work out that there was indeed something wrong with your test – then you can (conditionally) consider that test outcome as an outlier. But if you find that the outlier was a failure caused by a failure mechanism that you expect to see (to any degree) once it is in use … then it is no outlier. Deal with it!
Outliers tend to be full of valuable information. If you have a robust design process, your prototypes should largely ‘work.’ So ‘outliers’ can often be used to work out what your design weak points are. Isn’t this great information for your design engineers!
So let’s be honest … things we label as ‘outliers’ are often things we don’t want to see. So labelling it as an outlier means that we keep the data we want to keep. Not the data we need to keep.

Enjoy an episode of Speaking of Reliability. Where you can join friends as they discuss reliability topics. Join us as we discuss topics ranging from design for reliability techniques to field data analysis approaches.