How to Explain the Perils of MTBF Use

With a little practice and being aware of the many perils when using MTBF, you can become adept at clear and concise lines of reason to help others at least try a better way.

A trivial objection is ‘our product is not repairable so we’re using MTTF’. The math to estimate MTBF and MTTF from data is the same, total hours divided by total failures, thus both are an estimate of the average. Therefore, most the arguments to switch away from MTBF equally apply to MTTF.

Misunderstanding 1

When someone suggests MTBF is a failure free period, try not to snort or laugh, that doesn’t help. Instead point out the MTBF calculation results in the inverse of the failure rate. So if using hours, it provides the average chance of failure each hour. Then using the exponential distribution reliability function you can quickly show how many are expected to survive (the rest failing) by the end of the so called failure free period – which is about 2/3rds of the items.

Misunderstanding 2

When someone suggests we only use MTBF because ‘we have always used MTBF’, I first ask if the metric and meaning are well understood. There seems enough people with misunderstanding 1 that may be the only reason needed to persuade someone to try another measure. If not, I ask ‘so, how many are expected to survive over the first year?’ This question usually surfaces one more other misunderstandings.

Misunderstanding 3

‘I use MTBF to set the warranty period or our maintenance strategy’. MTBF is the inverse of the average failure rate and devoid of any changing rate of failure information. Are we dealing with a decreasing or increasing failure rate? How do we know the failure rate is actually the same for each hour? The only strategy for maintenance planning, given only an MTBF value and the assumption of constant hazard rate, is to replace or repair upon failure. If the item actually does have an increasing chance of failure with time, then MTBF is not able to describe that increasing rate. Use Weibull or some other model.

Misunderstanding 4

‘All our competitors and customers use MTBF in our industry’. This may be my favorite. Generally it is possible to quickly show that using reliability directly (probability of success over a duration for a function in an environment) provides a clear metric, plus using Weibull or other suitable model to describe the changing rate of failure over time provides a competitive advantage. By using a clear reliability statement and an appropriate model even with your customers, you avoid other misunderstandings, plus help everyone make better decisions. Better decisions concerning reliability mean meeting your customer’s reliability performance expectations.

Misunderstanding 5

Once in a while someone objects to using anything other than MTBF as it is a very easy metric to calculate from time to failure data. It is also easy to conduct test planning, etc. as many guides and books include examples showing the math required. The line of reasoning around ‘why limit yourself to the computing tools of the 50’s’ generally doesn’t work. It maybe the hesitation is actually related to doing the math involved with Weibull Analysis or other approaches. Sure the formulas and algorithms for anything beyond the exponential distribution and chi-square table may seem daunting, yet the benefits far outweigh the need to study and practice just a little. The math will come back quickly (you are talking to college degreed folks most likely in an engineering or science field). Reinforce the need to avoid the other misunderstandings, plus the benefits around accurate models and decisions.

There are other misunderstandings and effective lines of reason to help someone move beyond using MTBF. What have you found useful for particular misunderstandings?

How to Estimate Reliability Early in a Program

In a few discussions about the perils of MTBF, individuals have asked about estimating MTBF (reliability) early in a program. They quickly referred to various parts count prediction methods as the only viable means to estimate MTBF.

One motivation to create reliability estimates is to provide feedback to the team. The reliability goal exists and the early design work is progressing, so estimating the performance of the product’s functions is natural. The mechanical engineers may use finite element analysis to estimate responses of the structure to various loads. Electrical engineers may use SPICE models for circuit analysis.

Customers expect a reliable product. If they are investing in the development of the product (military vehicle, custom production equipment, or solar power plant, for examples) they may also want an early estimate of reliability performance.

Engineers and scientists estimate reliability during the concept phase as they determine the architecture, materials, and major components. The emphasis is often on creating a concept that will deliver the features in the expected environment. The primary method for reliability estimation is engineering judgement.

With the first set of designs, there is more information available on specific material, structures, and components, thus it should be possible to create an improved reliability estimate.

Is testing the true way to estimate MTBF?

Early in a program means there are no prototypes available for testing, just bill of materials and drawings. So, what is a reliability engineer to do?

One could argue that without prototypes or production units available for testing (exercising or aging the system to simulate use conditions) we do not really know how the system will respond to use conditions. While it is true it is difficult to know what we do not know, we often do know quite a bit about the system and the major elements and how they individually will respond to use conditions.

Even with testing, we often use engineering judgement to focus the stresses employed to age a system. We apply prior knowledge of failure mechanism models to design accelerated tests. And, we use FMEA tools to define the areas most likely to fail, thus guiding our test development.

Creating a reliability estimate without a prototype

Engineering judgement is the starting point. Include the information from FMEA and other risk assessment methods to identify the elements of a product that are most likely to fail, thus limit the system reliability. Then there are a few options available to estimate reliability, even without a prototype.
First, it is rare to create a new product using all new materials, assembly methods, and components.

Often a new product is approximately 80% the same as previous or similar products. The new design may be a new form factor, thus mostly a structural change. It may includes new electronic elements – often just one or two components, where the remaining components in the circuit regularly used. Or, it may involve a new material, reusing known structures and circuits.

Use the field history of similar products or subsystems and engineering judgement for the new elements to create an estimate. A simple reliability block diagram may be helpful to organize the information from various sources.

For the new elements of a design, base the engineering judgement on analysis of the potential failure mechanisms, employ any existing reliability models, or use simulations to compare known similar solutions to the new solution.

Second, for the elements without existing similar solutions and without existing failure mechanism models, we would have to rely on engineering judgement or component or test coupon level testing. Rather than wait for the system prototypes, early in a program it is often possible to obtain samples of the materials, structures, or components for evaluation.

The idea is to use our engineering judgement and risk analysis tools to define the most likely failure mechanisms for the elements with unknown reliability performance. Let’s say we are exploring a new surface finish technique. We estimate that exposure to solar radiation may degrade the finish. Therefore, obtain some small swatches of material, apply the surface finish and expose to UV radiation. While not the full product using fully developed production processes, it is a way to evaluate the concept.

Another example, is a new solder joint attachment technique. Again, use your judgement and risk analysis tools to estimate the primary failure mechanisms, say thermal cycling and power cycling, then obtain test packages with same physical structures (the IC or active elements do not have to be functional) and design appropriate tests for the suspected failure mechanisms.

Estimate combine the available knowledge

With a little creativity we can provide a range of estimates for elements of a design that have little or no field history. We do not need to rely on a tabulate list of failure rates for dissimilar product created by a wide range of teams for diverse solutions. We can draw from our team’s prior designs actual field performance for the bulk of the estimate. Then fill in the remaining elements of the estimate with engineering judgement, comparative analysis, published reliability models, or coupon or test structure failure mechanism evaluations.

In general, we will understand the bulk of the reliability performance and have rational estimates for the rest. It’s an estimate and the exercise will help us and the team focus on which areas may require extensive testing.

Variance and MTBF

When the data sheet or only available information is MTBF, how much do you know about the variability of the expected time to failure distribution? Not much really.

Do you need to know when to expect the first one percent of failures, 10 percent? Sure, that information is useful when estimating warranty or service costs, also for estimating readiness to go to market. We often are not interesting when the bulk will fail, rather the early small percentages. [Read more…]

When to use something
other than MTBF

As you may suspect I would say you should never use MTBF.

Given MTBF is prevalent, we may find avoiding MTBF nearly impossible. [Read more…]

Questions to ask a Supplier

Especially if they list MTBF on their data sheets.

My first questions, which I generally keep to myself, is ‘MTBF, yeah, right. Do they know better or not?” This is generally not a good way to start a conversation with a vendor about the reliability information you need to make appropriate decisions. [Read more…]

Any why you shouldn’t care.

With Mark Morelli’s permission here is a copy of his slides for the presentation “Why HALT Won’t Give You an MTBF and why you shouldn’t care.

Give it a look and enjoy.

[slideshare id=38216371&doc=hobbsjune42014-140821090218-phpapp02]

If we only measure Useful Life

Does that mean the early life failures and wear out failures don’t count?

Designing to keep the useful life failure rates low is good design practice. This generally means a design that is robust, operates smoothly, incurs little temperature rise, and is as simple as it needs to be to function. [Read more…]