Regression Metrics

Abstract

Chris and Fred discuss ‘regression metrics’ … or numbers that software packages spit out at us to suggest if a statistical model is ‘good’ or not. But if this is the only thing you (as an engineer) rely on to understand your system … you are already in trouble!

Key Points

Join Chris and Fred as they discuss how software and textbook processes can sometimes give us a single number that tells us how good a fit one model is compared to another. But you can’t just rely on this number … because you still don’t understand what is going on.

Topics include:

Think of sports. Many North American sports have events called ‘combines.’ This is where young, hopeful athletes come together and conduct several physical tests. This can include a 40 yard sprint. A vertical jump. Bench press. And lots of other athletic events where raw performance can be measured. Professional teams can then see measured raw athletic performance of prospective athletes. But teams do not select future players based on these numbers alone. They take into consideration skill, sporting ‘IQ’ and of course review how each actually played in lower leagues and university teams and so on. So if professional sporting teams don’t rely on numbers only … why would you when you are trying to understand how your system fails?
It gets worse. Many software applications can ‘select’ models for you. But this is dangerous! Why? Because it means you stop thinking for yourself. Many software packages will ‘fit’ models that make no sense even though they might have the best numbers. This can include models that suggest that it is possible for systems to fail at times less than zero!
Always understand your process. That might mean you plot your data, understand what parameters mean for models your software is considering or reading more books. If you know your data comes from a failure process involving a component that wears out … and your software suggests you use a model that you know models wear-in … then you should throw it in the bin (and know that you have avoided making lots of bad decisions based on not knowing your process!).
Confidence is a measure of YOU. You can get confidence from understanding your process. Or you can get confidence from statistics. One is much better than the other. Do you know which one it is?

Enjoy an episode of Speaking of Reliability. Where you can join friends as they discuss reliability topics. Join us as we discuss topics ranging from design for reliability techniques to field data analysis approaches.