Your Reliability Engineering Professional Development Site
I tend to write about all things Root Cause Analysis (RCA). I come from a background that engrained a holistic view of Reliability Engineering into me. This involved equipment, process and human Reliability. I write about the following types of RCA topics:
1. The Definition of ‘RCA’ and Why I Think the Current Term is Useless
2. When Should RCA’s be Conducted? Addressing the Application of RCA Proactively versus Reactively
3. What’s the Difference Between Root Cause Analysis and Shallow Cause Analysis? Why Do ‘RCA’ Efforts Fail
4. The Critical Task of Evidence Collection and Preservation to an Effective RCA
5. The Correlation Between Reliability/RCA and Safety, Does Such a Correlation Exist?
7. The Measurement of an RCA System’s Effectiveness
8. The Creation of an RCA Knowledge Base and its Sustained Growth to Store ‘Corporate Memory’ and Leverage Using AI
RCA Self-Assessment Tool. (linked version no longer available) The purpose of this tool is to allow individuals or groups to anonymously self-assess their current RCA system. How they wish to use the results is completely up to them!
All results are private and not retained anywhere. Therefore, if you wanted to keep your results, you would have to print them out and scan them back into your work station (or your past results are gone).
This failure occurred on a turbine driven boiler feedwater pump (TDBFP) at a fossil fuel power plant. The pump in question is an outboard boiler feed pump. Plant is operating normally when unexpectedly pump shuts down and causes a forced outage.
This is a failed shaft that came out of a pump in a paper mill. The pump was only in service for about a month before it failed unexpectedly.
From the top view above, identify the type of failure pattern that you see from the fractured surface(s). If you need more info to make your assessment, just ask.
Where does ‘failure’ come from? Why do some things not go as we planned? When bad things happen, at that time, it can be chaotic and appear very complex. Often, in hindsight (when the urgency has faded), we find that good people made bad decisions at that time. Most of the time such failures were not complicated and we find that most likely, anyone else put in the same position, would have made the same decision. So why do things not always go as planned?
Trick question, it’s neither!
When we hear people use the term ‘error’ it is normally associated with being the cause of some type of bad outcome. We hear it all the time with airline accidents where they conclude ‘pilot error’.
‘Error’ is neither the cause nor the undesirable outcome! Let’s see why.
Is the only type of error a ‘human error’? I struggled to think of an ‘error’ that is not a human error, so I asked if others could think of something else. I would like to thank Dr. Peter Elias who commented that errors can occur in biologic processes, such as RNA and DNA replication or messaging. He also cited non-human animals make errors all the time as well.
If you’ve ever led a Root Cause Analysis (RCA) Team, you will relate to this article. Being an effective RCA analyst is tough work in the healthcare (HC) industry. It is fraught with administrative, technical and political hurdles that continually erode our will to be effective, in spite of these barriers. However, for those ‘driven’ analysts, they just rise to the challenge and find a way to go over or through these often mental barriers, because the end game is the patient’s safety and it is worth it!
As long as humans are involved in running our organizations, there will be holes (vulnerabilities) in our cheese. This is why I state that we will never have American cheese (no holes) representing our safety systems, because this would imply they would be failsafe and not impenetrable.
“There is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success than to take the lead in the introduction of a new order of things, because the innovator has for enemies, all of those who have done well under the old conditions, and luke-warm defenders in those who will do well under the new.”
– Niccole Machiavelli
This quote is appropriate when we seek a paradigm shift towards a proactive culture, after spending our careers trained to become the best ‘react-ors’ or responders, we can be. I encourage you to read this lengthy article to the end for a surprise and to ask you a couple of questions about your experience with reliability. [Read more…]
No matter where we work, we will experience failures or ‘undesirable outcomes’ of some kind. As long as we work with other humans, this will indeed be the case. These failures may surface in the form of production delays, injuries, customer complaints, missed deadlines, lost profits, legal claims and the like. [Read more…]
In an era of rapidly advancing technology, the need for training to keep up is imperative. But training alone is not the panacea to a facility’s problems. Management’s must be aware that the environment in which their people work, will either progress or obstruct any training that is provided to them. We will refer to our need to address the human element, as the “soft side” of technology. It is estimated that over $60B U.S. is spent on industrial training a year and that only 20% of that training investment is ever applied. Are we getting our money’s worth from our training investment? If not, here are some things to consider when training our personnel and using their valuable time from the field. [Read more…]
I guess I am writing this just as a reality/sanity check to see if it’s just me, or do my peers in the Reliability profession have the same problems I do, communicating with non-Reliability professionals?
If anyone has been in the Reliability game for a long period of time (I am in my 32nd year), we know that Reliability is a way of life and not just a job. So our proactive thinking, involuntarily bleeds over into our personal lives. [Read more…]
RCA has an image problem and needs a public relations agent to reshape its reputation in the healthcare industry! RCA is primarily viewed as a reactive tool. This perception is how we have been conditioned by various regulatory agencies that require us to do RCA under very specific circumstances (usually when something very bad has occurred). When such ‘Sentinel Events’ occur, then we pull the microscope out to take a deeper look using our respective RCA tools. Under this use, RCA is viewed as a ‘Money-Taker’ because it appears only to consume people’s time and resources when they already feel they are overloaded. Rarely is the CEO asking for an ROI associated with an RCA. [Read more…]
We hear about ‘poor communication’ so often related to undesirable outcomes, that the term has become somewhat generic in nature. It has become meaningless in terms of implementing corrective action plans to prevent the risk of further miscommunication. How can we act on ‘poor communications’ without understanding what causes such miscommunication? This article will focus on applying key RCA principles to understanding what causes miscommunication. [Read more…]
If we have heard it once, we have heard it a million times – “let’s do an RCA on that failure”. The problem here is that phrase will mean something different to everyone who says it. What is an RCA? That is a question even the notable experts cannot agree on. With all of this RCA “chaos”, how do we make any progress? [Read more…]
Traditionally RCA is only utilized after an event has occurred, so how can one call it proactive? This perception is explored through trying to understand the current paradigms that exist about RCA, what it is and when it is used. Do we really have to wait for an undesirable outcome to occur in order to use RCA? [Read more…]