Accendo Reliability

Your Reliability Engineering Professional Development Site

  • Home
  • About
    • Contributors
  • Reliability.fm
    • Speaking Of Reliability
    • Rooted in Reliability: The Plant Performance Podcast
    • Quality during Design
    • Way of the Quality Warrior
    • Critical Talks
    • Dare to Know
    • Maintenance Disrupted
    • Metal Conversations
    • The Leadership Connection
    • Practical Reliability Podcast
    • Reliability Matters
    • Reliability it Matters
    • Maintenance Mavericks Podcast
    • Women in Maintenance
    • Accendo Reliability Webinar Series
  • Articles
    • CRE Preparation Notes
    • on Leadership & Career
      • Advanced Engineering Culture
      • Engineering Leadership
      • Managing in the 2000s
      • Product Development and Process Improvement
    • on Maintenance Reliability
      • Aasan Asset Management
      • AI & Predictive Maintenance
      • Asset Management in the Mining Industry
      • CMMS and Reliability
      • Conscious Asset
      • EAM & CMMS
      • Everyday RCM
      • History of Maintenance Management
      • Life Cycle Asset Management
      • Maintenance and Reliability
      • Maintenance Management
      • Plant Maintenance
      • Process Plant Reliability Engineering
      • ReliabilityXperience
      • RCM Blitz®
      • Rob’s Reliability Project
      • The Intelligent Transformer Blog
      • The People Side of Maintenance
      • The Reliability Mindset
    • on Product Reliability
      • Accelerated Reliability
      • Achieving the Benefits of Reliability
      • Apex Ridge
      • Metals Engineering and Product Reliability
      • Musings on Reliability and Maintenance Topics
      • Product Validation
      • Reliability Engineering Insights
      • Reliability in Emerging Technology
    • on Risk & Safety
      • CERM® Risk Insights
      • Equipment Risk and Reliability in Downhole Applications
      • Operational Risk Process Safety
    • on Systems Thinking
      • Communicating with FINESSE
      • The RCA
    • on Tools & Techniques
      • Big Data & Analytics
      • Experimental Design for NPD
      • Innovative Thinking in Reliability and Durability
      • Inside and Beyond HALT
      • Inside FMEA
      • Integral Concepts
      • Learning from Failures
      • Progress in Field Reliability?
      • R for Engineering
      • Reliability Engineering Using Python
      • Reliability Reflections
      • Testing 1 2 3
      • The Manufacturing Academy
  • eBooks
  • Resources
    • Accendo Authors
    • FMEA Resources
    • Feed Forward Publications
    • Openings
    • Books
    • Webinars
    • Journals
    • Higher Education
    • Podcasts
  • Courses
    • 14 Ways to Acquire Reliability Engineering Knowledge
    • Reliability Analysis Methods online course
    • Measurement System Assessment
    • SPC-Process Capability Course
    • Design of Experiments
    • Foundations of RCM online course
    • Quality during Design Journey
    • Reliability Engineering Statistics
    • Quality Engineering Statistics
    • An Introduction to Reliability Engineering
    • Reliability Engineering for Heavy Industry
    • An Introduction to Quality Engineering
    • Process Capability Analysis course
    • Root Cause Analysis and the 8D Corrective Action Process course
    • Return on Investment online course
    • CRE Preparation Online Course
    • Quondam Courses
  • Webinars
    • Upcoming Live Events
  • Calendar
    • Call for Papers Listing
    • Upcoming Webinars
    • Webinar Calendar
  • Login
    • Member Home

by Larry George 1 Comment

Covariance of the Kaplan-Meier Estimators?

Covariance of the Kaplan-Meier Estimators?

What are the covariances of Kaplan-Meier reliability estimates at different ages? I need them for the variance of actuarial demand forecasts and for confidence bands on reliability. I thought cohort reliability estimate variances and covariances in the previous article were a good idea. How good? Not as good as bootstrap and jackknife resampling alternatives!

The Kaplan-Meier reliability function estimator uses right-censored and grouped time-to-failure counts in periodic cohorts (rows in table 1). The Nelson-Aalen cumulative failure rate function estimators are theoretically independent [Aalen, Nelson], but not for some examples. The Kaplan-Meier reliability and actuarial failure rate function estimates at different ages are dependent, so their covariances matter to actuarial forecasts and confidence bands on reliability.

Bootstrap, Jackknife, or Empirical Covariances?

Maximum likelihood estimators’ variance-covariance matrix converges asymptotically to the Cramer-Rao lower bound. The Cramer-Rao bound is for unbiased estimators. The Kaplan-Meier estimator is biased, so real data may not be close to asymptotic and the variance and covariance the Kaplan-Meier estimator may not be close to the Cramer-Rao bounds. Greenwood’s variance formula is an estimator of the variance of the Kaplan-Meier reliability estimate, a Cramer-Rao lower bound on variance [Freedman, Greenwood]. 

The previous article compared the Greenwood standard deviations with empirical cohort reliability estimates’ standard deviations for the data in table 1 [George]. Empirical cohort reliability estimates are: R(0)=1; R(t)=R(t-1)*(1-d(t)/(n-∑d(s)); s=1,2,…,t-1, where d(t) is cohort deaths at age t and n is cohort size. Cohorts are independent, although cohorts have different sizes, different numbers of grouped failure counts, and different maximum ages (aka Type 1 censored). Averages of the empirical cohort reliability estimates are unbiased. Multiple cohort reliability estimates provide data for estimation of the reliability variance-covariance matrix. 

Table 1. Ships, grouped failure counts [“Weibull Analysis of Perplexing Field Data,” by James McLinn] and Kaplan-Meier reliability estimates 

WeekShips123456789KMRel
1201010101010.9882
250 101010100.9882
370  10101010.9746
4100   1010100.9746
5100    101010.9578
6100     10100.9578
7120      1010.9278
8120       100.9278
9146.07        10.893

Bootstrap resampling from the Kaplan-Meier estimate (“KMRel”) makes data for estimation of the variance-covariance matrix of reliability function estimates at pairs of different ages [https://en.wikipedia.org/wiki/Bootstrapping_(statistics)/]. (Condition on reliability between 0.893 and 1.0.)

Jackknife sampling computes multiple Kaplan-Meier reliability estimates from all-but-one cohorts (leave one out) [https://en.wikipedia.org/wiki/Jackknife_resampling/]. The jackknife makes data for estimation of the variance-covariance matrix. In this example, the jackknife only generates eight reliability function estimates. The bootstrap generates as many as you want.

Reliability function standard deviation estimates increase with age (figure 1), because there are fewer cohorts with data for older ages. The Greenwood formula under-estimates the standard deviations, because it is an asymptotic bound for unbiased reliability estimators. Empirical cohort reliability standard deviations (“Emp Stdev”) over-estimate standard deviations, because they are for different estimator than the Kaplan-Meier. 

A barchart showing standard deviations for four different methods over age in weeks. It show the increasing emperical standa deviation increat from other estimates starting in week 3.
Figure 1. Greenwood, bootstrap, jackknife, and empirical cohort standard deviation estimates

Tables 2 and 3 show alternative variance-covariance matrix estimates. Notice that the jackknife gives mostly negative covariances? The jackknife computes means and standard deviations from dependent subsets! That makes me suspicious of jackknife covariances. 

Table 2. Bootstrap variance-covariance matrix of the Kaplan-Meier estimator. Covariances are non-negative and not negligible.

Age, Week12345678
18.7E-059.3E-03      
28.7E-058.7E-059.3E-03     
32.6E-052.6E-058.2E-05     
42.6E-052.6E-058.2E-058.2E-05    
54.3E-054.3E-057.3E-057.3E-051.4E-04   
64.3E-054.3E-057.3E-057.3E-051.4E-041.4E-04  
77.2E-057.2E-055.8E-055.8E-051.4E-041.4E-042.6E-04 
87.2E-057.2E-055.8E-055.8E-051.4E-041.4E-042.6E-042.6E-04

Table 3. Jackknife variance-covariance matrix of the Kaplan-Meier estimator. Most jackknife covariances have opposite signs from bootstrap covariances!

Age, Week12345678
11.7E-05       
21.1E-058.4E-06      
33.6E-068.0E-063.6E-05     
4-4.1E-05-2.2E-052.2E-051.6E-04    
5-5.6E-05-3.6E-05-8.1E-071.6E-041.6E-04   
6-6.8E-05-4.5E-05-1.1E-051.8E-041.8E-042.9E-04  
7-4.0E-05-2.2E-051.8E-051.5E-041.5E-041.8E-041.4E-04 
8-5.5E-05-3.5E-05-1.6E-061.6E-041.6E-042.2E-041.4E-041.9E-04

Averages of empirical cohort reliability function estimates differ from Kaplan-Meier reliability function estimates, but they are unbiased estimators. The variance of the average cohort reliability at age 1 is VAR[(d(1,1)/n(1)+d(2,1)/n(2))/2], and the variance of the Kaplan-Meier reliability at age 1 is VAR[(d(1,1)+d(2,1))/(n(1)+n(2))], where d(I,j) counts deaths at age j from cohort I of size n(i). So, bootstrap the Kaplan-Meier estimator for its variance-covariance matrix, if you do not want the variance of the empirical cohort reliability functions.

Recommendations and Questions?

Greenwood variance estimates may be wrong depending on data. Covariance estimates differ depending on method. Check alternatives: empirical cohort, bootstrap, and jackknife reliabilities and their covariances. The Jackknife mean is theoretically unbiased but its covariances are suspicious because of dependence and limited number of jackknife subsamples. Compare variances of actuarial forecasts computed from alternative variance-covariance matrix estimates.

Empirical cohort reliability estimates are computed from independent cohorts, unlike the jackknife. Although cohort durations differ, empirical cohort estimates’ variances, and covariances should be more precise than jackknife estimates.

Should I use weighted variance-covariance estimates [Khan et al.] to compensate for different size cohorts? Should I derive weights that minimize distance of the empirical variance-matrix from the Cramer-Rao bound? Others impute the actual lives of censored observations [Vinzamuri et al.] to avoid using the Kaplan-Meier estimator.

You may ask, “How will you estimate the variance-covariance of nonparametric reliability estimators from ships and returns counts, without grouped failure counts? The sample reliability function covariance may not be close to the Cramer-Rao bound. Send field reliability ships and returns counts data and I will show you: pstlarry@yahoo.com.

References

Aalen, O. O., “Nonparametric Inference for a Family of Counting Processes,” Annals of Statistics, Vol. 6, 701726, 1978

D. A. Freedman, “Greenwood’s Formula,” https://www.stat.berkeley.edu/~freedman/greenwd.pdf

Larry George, “Variance of Kaplan-Meier Reliability?” Weekly Update, https://lucas-accendo-site-speed.sprod01.rmkr.net/variance-of-the-kaplan-meier-estimator/#more-508828/, March 2023

Greenwood, M., “The natural duration of cancer. Reports on Public Health and Medical Subjects,” Vol. 33, pp. 1–26, His Majesty’s Stationery Office, London, 1926

Habib Nawaz Khan, Qamruz Zaman, Fatima Azmi, , Gulap Shahzada, and Mihajlo Jakovljevic, “Methods for Improving the Variance Estimator of the Kaplan–Meier Survival Function, When There Is No, Moderate and Heavy Censoring-Applied in Oncological Datasets,” Frontiers in Public Health, May 2022

James McLinn, “Weibull Analysis of Perplexing Field Data,” ARSymposium, 2010

Wayne Nelson, “Theory and Applications of Hazard Plotting for Censored Failure Data,” Technometrics, Vol. 42, No. 1, February 2000

Bhanukiran Vinzamuri, Yan Li, and Chandan K. Reddy, “Calibrated Survival Analysis using Regularized Inverse Covariance Estimation for Right Censored Data,” IEEE Transactions on Knowledge and Data Engineering, DOI:10.1109/TKDE.2017.2719028, June 2007

Filed Under: Articles, on Tools & Techniques, Progress in Field Reliability?

About Larry George

UCLA engineer and MBA, UC Berkeley Ph.D. in Industrial Engineering and Operations Research with minor in statistics. I taught for 11+ years, worked for Lawrence Livermore Lab for 11 years, and have worked in the real world solving problems ever since for anyone who asks. Employed by or contracted to Apple Computer, Applied Materials, Abbott Diagnostics, EPRI, Triad Systems (now http://www.epicor.com), and many others. Now working on survival analysis, epidemiology, and their applications: epidemics, randomized clinical trials, risk-based inspection, and DoE for risk equity.

« Use Of RFID In Process Safety: Track Hazardous Chemicals And Track Personnel
What is the Difference Between Quality Assurance and Quality Control? »

Comments

  1. Larry George says

    April 9, 2023 at 12:39 PM

    Why not compute the variance of broom chart Kaplan-Meier reliability estimates? Jerry Ackaret told me that’s what he called my graphs Sequent HDD of K-M reliability estimates of successively larger cohorts sizes and grouped failure counts. Shucks, I can’t paste a broom chart here. Anyway, I can post the K-M broom and Greenwood standard deviations. Greenwood underestimates standard deviation! This uses the data from Jim McLinn.
    K-M Greenwood
    0.0056 0.0036
    0.0056 0.0036
    0.0161 0.0059
    0.0076 0.0059
    0.0185 0.0088
    0.0082 0.0088
    0.0189 0.0152
    0.0060 0.0152
    Beware the Greenwood variance! It is a lower bound, for unbiased estimators, asymptotically, which means as age->infinity and reliability=>0.

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Articles by Larry George
in the Progress in Field Reliability? article series

Join Accendo

Receive information and updates about articles and many other resources offered by Accendo Reliability by becoming a member.

It’s free and only takes a minute.

Join Today

Recent Articles

  • test
  • test
  • test
  • Your Most Important Business Equation
  • Your Suppliers Can Be a Risk to Your Project

© 2025 FMS Reliability · Privacy Policy · Terms of Service · Cookies Policy