What is ‘Production Equivalent’ and Why Does it Matter?
We’re in the middle of our design process and we’re getting some of the first parts in hand, from prototyping or benchtops or our manufacturing cell. Let’s test them and finish our verification testing! Maybe we shouldn’t.
We talk about the concept of production equivalent: what it means and when it’s important to consider.
Do we test prototype parts? Absolutely! Use them for benchtop tests, formative usability studies, and other engineering studies.
But, prototypes that aren’t production or production equivalent shouldn’t be used when we’re doing our final verification and validation tests. Because with those tests we’re making sure everything performs as we want and need, we don’t want to introduce unknown variables and unnecessary risks.
Production equivalent is that middle-ground between prototype and full production (closer to production). But we need to use it strategically and in a smart way. And we need to be able to justify our decision when using it.
Citations
Here is a link to the FDA’s CGMPs:
Sometimes you will catch a news article about good manufacturing practices, like this one:
FDA Forces Johnson & Johnson To Throw Out 60 Million Doses Of Covid-19 Vaccine (forbes.com)
Episode Transcript
We’re in the middle of our design process and we’re getting some of the first parts in hand, either from prototyping or benchtops or our manufacturing cell. Let’s test them and finish our verification testing! Maybe we shouldn’t. I’ll explain why after this brief introduction.
Hello and welcome to Quality during Design the place to use quality thinking to create products, others love for less. My name is Dianna. I’m a senior level quality professional and engineer with over 20 years of experience in manufacturing and design. Listen-in and then join the conversation at qualityduringdesign.com.
Having worked in the quality field for years, I think the principles and philosophes of quality and reliability engineering apply to a broad range of topics. It’s not just for making products or for manufacturing production (although that’s what we talk about lot). There are published papers where quality methods are used for schools, hospitals and other service industries with success. The tones of the papers are that of empowerment. What once seemed difficult or impossible was made easier and possible through Quality.
Quality and reliability methods also cross boundaries between service types, like hospitals and schools or manufacturing facilities. But, also, between production types. And by that I mean: you could be making medical devices, lawn chairs or lawn ornaments, clothing…similar or the same quality, reliability methods and philosophes apply to a range of product types. I mention this today because I’m going to bring up a concept that’s well known and regulated in the medical device area, but the concept applies to all sorts of production. A lot of my career has been in manufacturing and in medical devices.
In the United States, the FDA (Food and Drug Administration) is a regulating body for food, drugs, biologics, and devices. So, if you want to sell any of those things in the US, your business and products need to meet the regulations of the FDA. The FDA has specified their own regulations for quality systems for these industries. It’s also known as the Current Good Manufacturing Practices (or CGMP). The Current Good Manufacturing Practices for medical devices first took effect in 1978 (that’s just to give you an idea about when this all came to be). I’ll include a link on this podcast block to those good manufacturing practices from the FDA.
In the FDA’s current good manufacturing practices, there is a lot to unpack about product design control procedures. Within the design cycle there are two types of tests. There is a design verification to ensure that design outputs meet the design inputs, like requirements and specifications. There’s also design validation, which is product testing done to ensure that the device meets user needs and intended uses. The CGMP touches on something called production equivalence. Here’s what it says: “Design validation shall be performed under defined operating conditions on initial production units, lots or batches, or their equivalents.” Generally, to meet all the CGMP regulations, medical device manufacturers also ensure that their products for design verification are also production or production equivalent.
What is production equivalence and why is it important? Because when you’re in the middle of a project, managing activities and forging ahead toward a deadline. Production equivalence can get a little muddy unless your careful. Manufacturers like to start creating parts in a manufacturing cell or in the final factory location as a limited run and there are lots of benefits to doing this. A couple of the benefits are tweaking your process in an environment where you can make changes quickly, and to really ensure that your product, as designed, is going to pass the test. So then, if we’re making parts and testing them, can’t they just count toward our design testing? Maybe, but maybe not. It depends if it’s production equivalent.
First let’s consider production. Understanding production is sort of easy. It’s the initial product units, lots or batches…it’s formal manufacturing. And the department where it’s made is sometimes called “production”. What would make something production equivalent? Sometimes, and in this case, I think it’s easier to think about what it’s not. To get there, let’s more fully explore what production means from four different areas: process, people, places, and things.
In a production run, products are made using a process that’s defined. The methods of manufacturing are defined within set specifications, and those are inspected and controlled. In production, the people involved are operators who have the skills to make the product or to run that equipment, or they have the right level of training. A production-type place is an environment that’s comparable to where it’s going to be made in the long term. And the things used to make the product are tools and equipment that are available to a typical operator, not a specialized piece of equipment.
Now thinking about what production really means from process, people, places and things, we can think about what production equivalence would need to look like. Let’s talk about some common pitfalls of production equivalence within these four parameters. We may not have a production equivalent process if we’re assembling components that have just been sitting around, and we’re not sure of their ordering history or who manufactured them, and they haven’t gone through an incoming inspection procedure. We may not have a production equivalent process if we are creating a device or component without a drawing or process document. We’re not inspecting our product when it’s done to make sure that it was made right, and ‘did we make the right thing’? We may have trouble claiming production equivalence if our operators were not properly trained, or our operators are over-qualified. For example, an engineer with ten years of experience soldering leads assembles a component, but in production it would really be an operator with two weeks of training experience and that’s not their normal job. We may have trouble with production equivalence if we put together our parts in a clean workbench. But in a factory, it’s going to be assembled in the same room as a powdered metal press, for example. There have also been some issues where plastic injection molding was done in a clean room for prototype, but in the actual manufacturing facility the garage doors were open because the facility got too hot. And finally, the things used to make the product: we may have trouble claiming production equivalence if the part tested was a printed 3D part, but the production part will be injection molded. Or maybe it was assembled gently with a high precision tool on an engineering bench, but in production it will be lined up and snapped into place.
With these type of examples we can envision how production or not production equivalence can affect our product tests. On the one hand, our prototype products might perform with better results than the production parts. That would be a problem because now we don’t truly know how the product is going to perform. Does its output meet the inputs? Can it perform as intended? On the other hand, our prototype products might perform worse, in which case we can’t move forward with the design. And we can have a third hand, I guess: we don’t know what we don’t know. Not using production or production equivalent parts introduces an unknown variable. Since we don’t know it, then we have no way or intention of controlling it and that introduces performance risk.
Now all of this is not to say that we don’t test prototype parts. Yes, test them! Use them in preliminary usability engineering studies, try them on the engineering bench. Test to see if their performance is a factor of safety above how they need to perform. Use them in a fixture-like capacity to test other features. For example, test the electrical continuity of your system while using a 3D printed housing. But prototypes can’t stand-in for production-type parts when we’re doing our final checks to make sure everything performs as we want it to.
In conclusion production equivalence is that middle-ground between prototype and full production. But we need to use it strategically and in a smart way, and we need to be able to justify our decision when using it, which is fair. And I think that goes across any design process in any field. It’s a general good manufacturing practice.
Please visit this podcast blog and others at qualityduringdesign.com. Subscribe to the weekly newsletter to keep in touch. If you like this podcast or have a suggestion for an upcoming episode, let me know. You can find me at qualityduringdesign.com, on Linked-In or you could leave me a voicemail at 484-341-0238. This has been a production of Deeney Enterprises. Thanks for listening!
Leave a Reply