Accendo Reliability

Your Reliability Engineering Professional Development Site

  • Home
  • About
    • Contributors
  • Reliability.fm
    • Speaking Of Reliability
    • Rooted in Reliability: The Plant Performance Podcast
    • Quality during Design
    • Way of the Quality Warrior
    • Critical Talks
    • Dare to Know
    • Maintenance Disrupted
    • Metal Conversations
    • The Leadership Connection
    • Practical Reliability Podcast
    • Reliability Matters
    • Reliability it Matters
    • Maintenance Mavericks Podcast
    • Women in Maintenance
    • Accendo Reliability Webinar Series
  • Articles
    • CRE Preparation Notes
    • on Leadership & Career
      • Advanced Engineering Culture
      • Engineering Leadership
      • Managing in the 2000s
      • Product Development and Process Improvement
    • on Maintenance Reliability
      • Aasan Asset Management
      • AI & Predictive Maintenance
      • Asset Management in the Mining Industry
      • CMMS and Reliability
      • Conscious Asset
      • EAM & CMMS
      • Everyday RCM
      • History of Maintenance Management
      • Life Cycle Asset Management
      • Maintenance and Reliability
      • Maintenance Management
      • Plant Maintenance
      • Process Plant Reliability Engineering
      • ReliabilityXperience
      • RCM Blitz®
      • Rob’s Reliability Project
      • The Intelligent Transformer Blog
      • The People Side of Maintenance
      • The Reliability Mindset
    • on Product Reliability
      • Accelerated Reliability
      • Achieving the Benefits of Reliability
      • Apex Ridge
      • Metals Engineering and Product Reliability
      • Musings on Reliability and Maintenance Topics
      • Product Validation
      • Reliability Engineering Insights
      • Reliability in Emerging Technology
    • on Risk & Safety
      • CERM® Risk Insights
      • Equipment Risk and Reliability in Downhole Applications
      • Operational Risk Process Safety
    • on Systems Thinking
      • Communicating with FINESSE
      • The RCA
    • on Tools & Techniques
      • Big Data & Analytics
      • Experimental Design for NPD
      • Innovative Thinking in Reliability and Durability
      • Inside and Beyond HALT
      • Inside FMEA
      • Integral Concepts
      • Learning from Failures
      • Progress in Field Reliability?
      • R for Engineering
      • Reliability Engineering Using Python
      • Reliability Reflections
      • Testing 1 2 3
      • The Manufacturing Academy
  • eBooks
  • Resources
    • Accendo Authors
    • FMEA Resources
    • Feed Forward Publications
    • Openings
    • Books
    • Webinars
    • Journals
    • Higher Education
    • Podcasts
  • Courses
    • 14 Ways to Acquire Reliability Engineering Knowledge
    • Reliability Analysis Methods online course
    • Measurement System Assessment
    • SPC-Process Capability Course
    • Design of Experiments
    • Foundations of RCM online course
    • Quality during Design Journey
    • Reliability Engineering Statistics
    • Quality Engineering Statistics
    • An Introduction to Reliability Engineering
    • Reliability Engineering for Heavy Industry
    • An Introduction to Quality Engineering
    • Process Capability Analysis course
    • Root Cause Analysis and the 8D Corrective Action Process course
    • Return on Investment online course
    • CRE Preparation Online Course
    • Quondam Courses
  • Webinars
    • Upcoming Live Events
  • Calendar
    • Call for Papers Listing
    • Upcoming Webinars
    • Webinar Calendar
  • Login
    • Member Home

by Adam Bahret 1 Comment

Is This The Same As That?

Is This The Same As That?

A common tool for comparing if two populations are the same is the “student t-test.”  This is often used in reliability, and science, if we want to investigate if a factor has caused a change in a respnse.

A population was assembled in location “A”.  Another population was assembled in location “B”.  Population “A” has an average defect rate of 4%.  Population “B” has an average defect rate of 5.5%. Does the location of assembly affect defect rate?  That’s just a big argument unless we can project the statistical likelihood that what we have measured is not just an overlap of noise.

What are we really asking?  We are asking if the difference between the two failure rates is more significant than the natural variability if we measured this several times at the same location.  Could the failure rate at location “A” be 5% next month instead of 4%?  This change could flip some people from having said that the populations “were different” to “they are not different.”  But all that happened was that we were able to observe the natural variability month to month at the same location.  In two months we may measure 4% again at location “A”.

We are effectively looking for a signal to noise ratio.  Is the difference between the populations just the noise due to a small population size and the expected variability?

A Descriptive statistic vs an Inferential statistic

A Descriptive Statistic only describes the sample we have.  This would simply be comparing the failure rates. We can say that one is larger than the other by 1.5% of the total population. This doesn’t tell us if our results are likely to happen again

An Inferential Statistics doesn’t just describe our sample.  It tells us what to expect in new samples we do not have.  This allows us to make inferences about the population beyond our data.  This is the t-test! This is a powerful tool!

The t-value is the ratio of variance between groups to variance within groups. (Remember:  Signal to Noise)

A big t-value = different groups                        A small t-value = similar groups

Each t-value has corresponding p-value.  The p-value tells us the likelihood that our calculation is correct. The p-value is the probability that the pattern of data in the sample could be produced by random data. The probability that we got tricked.

  • A p-value of .1 means there is a 10% chance we would get these results with random data. That’s too high for my comfort level.  So we conclude that our t-value is invalid and there is no real difference between the populations.
  • A p-value of .01 means there is only a 1% chance we would get these results with random data.  So it is very likely we have calculated a valid t-value.  We conclude there is a true difference in the populations.
  • A p-value of .05 is a 5% chance we would get these results with random data. This is the common “rule of thumb” threshold in the statistics world for these two conclusions.  So if p=.05 pop some popcorn, sit back, and watch the show as the arguing commences.

The bigger the samples the better the “signal to noise” ratio.  There are diminishing returns on this.  A good rule of thumb is to have 25 samples, but no less than 10.

Three common types of data sets for t-test

  1. Independent samples:  There is no correlation between the two groups.  Product from location “A” and product from location “B.”
  2. Paired samples:  The two groups are related to a common factor.  Measure a subject at two different settings.  measure the group at location “A” at 10 volts and then measure the same group “A” again at 13 volts.
  3. Ons sample test:  Measure a single group and compare to known population.  A group IQ vs the known average IQ of the country.

The t-test is a very helpful tool to be familiar with.  If you practice it you can do the calculations in Excel in two minutes (literally).  That argument that is happening live in a meeting…,  you can stand up, hold your hands out like you are stopping traffic, and say conclusively that “The answer is …” Then just sit back down smugly and wait for that promotion and raise to roll right in.

-Adam

TLDR  (too long didn’t read)

  • The t-value is the likelihood the difference we are observing is just noise.
  • The p-value is the likelihood we got tricked by our data and have a junk t-value calculation  (i.e. “P” stands for “Probably should have stayed home today”)
No Fields Found.

Filed Under: Apex Ridge, Articles, on Product Reliability

About Adam Bahret

I am a Reliability engineer with over 20 years of experience in mechanical and electrical systems in many industries. I founded Apex Ridge Reliability as a firm to assist technology companies with the critical reliability steps in their product development programs and organizational culture.

« Understanding FMEA Controls – Part 1
Top Down or Bottom Up – The Dilemma of Continuous Improvement »

Comments

  1. John Kreucher says

    December 1, 2017 at 5:47 AM

    Very nice article, Adam, on the t-test!! It was very clear and concise. I plan to share it with my team.

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Article by Adam Bahret
in the Apex Ridge series

Join Accendo

Receive information and updates about articles and many other resources offered by Accendo Reliability by becoming a member.

It’s free and only takes a minute.

Join Today

Recent Articles

  • test
  • test
  • test
  • Your Most Important Business Equation
  • Your Suppliers Can Be a Risk to Your Project

© 2025 FMS Reliability · Privacy Policy · Terms of Service · Cookies Policy