As reliability professionals, we are in the business of estimating or forecasting the reliability performance of our product, equipment, or system. While we use a range of tools to analytically make these estimates, sometimes we do not have sufficient data or information.
One method is to ask another person that has knowledge of the particular technology, use conditions, or whatever is hampering our work. If you ask two people you most like will get two different answers. If you ask 10, 10 different answers.
One way to work with a group of subject matter experts is to conduct a structured communication technique called the Delphi Method.
The process of the Delphi Method
There are just a few steps to this method:
Identify and recruit participants for the study. The group should not be just anyone, it should be individuals with some knowledge related to the questions of the study. For example, if related to a novel new technology, then participants should be conversant and understand the new tech. If the study would like to examine how a new set of use conditions will impact the longevity of your system, then folks should be knowledgeable about the system plus the current and new set of use conditions. The group may be ten to thousands of individuals, it is up to your resources and ability to find ‘experts’. More is considered better as it provides a broad range of responses along with varied insights, knowledge, and ideas.
One note: the participants are anonymous and may be from within and outside the organization. The intent is to avoid groupthink, following the leader, and avoid a range of other biases that may occur.
Next, frame the questions. One tip is to make the questions specific and best if the expected answer is a number. For example, “How long in years will the new tech survive without repair in our current market?” Then prompt for a comment to support their answer.
The leader of the study gathers the responses, sorts from lowest to highest (hence asking for a number, like years). Then sharing, anonymously, a few of the lowest and highest (the edges of the range of responses) along with their supporting comments with the group. The intent is to share a bit of information and ask for everyone, based on the additional information to answer the questions again.
This second round of sharing responses and permitting answers to change may continue for a set number of rounds or till the results remain stable.
The result may be the mean or median results, as appropriate.
Example uses
Let’s say we are working with a team developing a new portable device with a USB connection for power and information transfer. It’s a new device with a novel new application, and we do not have any fielded units as of yet.
One question that you and the team may need to address is how many times will users plugin/ unplug the device per day or month or year. We need to set product requirements and there isn’t a clear known source for an answer. So, let’s ask a set of ‘experts’.
First identify a group of experts that are familiar with the new product’s expected use.
Second, frame your question. In this case, one question may be: “How many USB plug/unplug cycles do you expect to occur per day by those using product X? Please state your rationale for your answer.”
Gather the answers and sort. Let’s say the lowest response was once per day with the rationale that the battery life is expected to support at least a 24-hour long use without recharging. The highest response of 10 times per day is support with the citation of study of other portable devices, different functionality, that suggests that common user behavior is to plug in anytime one is at their desk, completes a task, plus overnight.
Sure the results and comments and ask for a new response in light of the new shared information.
Continue till you find the answers are remaining stable. Calculate the mean and publish the results.
Experts do not always agree
Each of us, which includes you with your expertise in a range of topics, do not agree with others all the time. That is in part due to our personal and professional experiences, beliefs, biases, etc. That is ok.
The intent of a Delphi method to gather and sort out a consensus or common answers from a range of experts. By sharing rationale for high and lower answers, we help to inform the group with the intent to narrow the range of responses closer to the true, and unknown, answer to our question.
Leave a Reply