Get our free email newsletter

The Pitfalls of Pass/Fail Testing

You’re at your desk when the phone rings. It’s one of your production lines; they’re having failures with a part or a test that is critical to the safety of the product. They’ve shut down the line. You drop everything, and head to the line to investigate.

As you walk to the line, your mind is filled with both questions and hope.

What are you going to do when you get there? Will you be able to get the line back up and running? Is it a true failure, or did they make a mistake? You hope it’s a mistake, or interpretation error. Or, maybe it’s not a problem at all; you hope they’re being overzealous and over-cautious. And, you hope that whatever it is, they’ve caught them all before they’ve been shipped to the field so that you don’t have to consider a product recall.

- Partner Content -

EMC & eMobility

For a company embarking on EMC testing for either component or vehicle-level testing of their EV products, it is necessary first to have a good understanding of the EMC regulatory situation.

The first thing you want to do is to see the failure for yourself. This will answer all your questions. You hope. If it is a true failure, you hope the cause is obvious and the fix is easy.

Already, you feel the pressure. You’ve been here before; if you can’t fix it in a few minutes, the manufacturing manager will have you and a bunch of others in his office. They’ll be looking to you for instructions as to how to proceed. And, they’ll want those instructions fast!

The pressure is on! You’ve gotta find the root cause, and fast! Once you’ve identified the root cause, the pressure is off you and onto the manufacturing folks who will deal with the problem. So, you’re hoping this will be easy.

 Here’s one scenario:

When you get to the line, they show you damage to the power cord jacket adjacent to the strain-relief mechanism. It’s a strange mark, neither a cut nor a bum, but something of a cross between the two.

- From Our Sponsors -

Is the damage acceptable or not?

You decide a pull-strength test is probably appropriate. So, you apply 35 pounds. At about 30 seconds, the jacket separates.

Clearly not acceptable. You do have a problem.

You look at other units. Some have damage, some don’t. The ones that have damage are not uniform. The damage varies from barely discernible to quite extensive. Now you’re faced with the question: How bad is bad?

This should be easy: Test a number of units at 35 pounds for one minute. Then, relate the degree of damage to breakage.

But, it doesn’t work. Some severely damaged units which should have broken do not break! What is going on here? The problem seems to have shifted from one which should have been easy, to one which seems to have no bounds. How are you going to get control of this situation?

Here’s another scene:

When you get to the line, they explain that about half of the units are failing the hi-pot test.

You check the hi-pot tester and find that it’s both calibrated and working properly. You watch the operator do the test and, again everything is okay. The units are truly failing the hi-pot test: You do have a problem.

It only takes a few minutes more to isolate the particu1ar part in the primary circuit that is the culprit. Let’s say, for discussion, the part is a fan motor. And, it is certified by several certification houses. So, you know that the fan was successfully hi-pot tested as a part of the fan manufacturer’s production process.

Why do some of the fans, all of which passed the manufacturer’s hi-pot test, fail our hi-pot test? What is going on here? The problem should have been easy, but some are okay, and some are not. How are you going to get control of this situation?

Let’s step out of the woods, and look at the forest from afar. What is common to these two scenarios?

In both scenarios, we are dealing with some units failing, and some units passing a requirement specified in a third party test standard. The test process is pass-fail; tested units, by definition, must fit in one category or the other.

Often, our thinking is driven by the standards and by the pass-fail certification submittal process. We tend to think only in terms of pass-fail. So, when we appear at the production line, our concern is for the failed units, and not for the passed units.

Pass-fail thinking and testing is appropriate and acceptable when qualifying a product to a standard. Pass-fail thinking and testing is an appropriate and acceptable process for a certification house. But, pass-fail testing is seldom appropriate and acceptable for the manufacturer. And, it doesn’t work for problem solving.

Your objective is to find what is causing the failures, not to segregate the bad from the good.

The failed units are bad, but we don’t necessarily know how bad. The passed units are good, but we don’t know how good. 

When we perform pass-fail testing, we don’t measure the actual performance of each unit.

When we perform a pull test at 35 pounds, and the unit fails, we don’t know the pull value that it will pass. When we perform a hi-pot test, and the unit fails, we may not note the voltage at which it failed.

More importantly, for a unit that passes, how good is it? If it passes a 35-pound pull test, will it pass a 50-pound pull test? If it passes a 1000-volt hi-pot, will it pass a 1500-volt hi-pot? If it passes 1500, will it pass 2000?

Exactly how good is it? If we test to failure, we have a measure of the performance of the particular unit.

Are the units marginal, or is there a clear distinction between the units that measure above 35 pounds and the units that measure below 35 pounds? Is there a clear distinction between the units that measure above 1000 volts hi-pot and the units that measure below 1000 volts hi-pot?

The answer to this question quickly narrows the scope of the problem. Here’s an example:

If, in investigating the power cord jacket damage, we pull the power cord to failure, we learn that some cords, regardless of the extent of jacket damage, fail at or around the ultimate strain-relief strength of 125 to 150 pounds. Others, depending on the extent of jacket damage, fail between 30 and 50 pounds. We examine the measurement data and note that only one brand of cord fails as a function of jacket damage.

Presto! The scope of the problem is now defined, and the issue can be handed off to the manufacturing folks.

Imagine how many pass-fall tests would be necessary before you finally discovered that the problem occurs in only one brand!

Measuring the performance of a strain-relief mechanism on a number of production units need not sacrifice the unit; upon failure, the unit probably can be repaired at relatively little expense com¬pared to the time to understand and put bounds on the problem.

On the other hand, hi-pot testing to failure may be very expensive to repair. So, you may not want to subject a number of units to a test-to-failure. Let’s look at some other techniques for investigating hi-pot failures. Remember, the objective is to find the root cause for the production-line hi-pot failure.

The hi-pot test tests insulations. It tests, simultaneously, both air and solid insulations — which always exist in parallel, and often exist in series.

While some may argue, I believe it is seldom that solid insulation fails at potentials below about 2000 volts rms.

Since every construction employs air as insulation, when a hi-pot failure occurs, there is a good likelihood the breakdown is in air. (Note that, in the event a breakdown occurs across the surface of an insulator, the “thing” that breaks down is the air, the arc in the air at the surface of the insulator burns the insulator resulting in carbon tracks on the surface.)

The air that breaks down is likely that of a series “circuit” of air and solid insulation. The two insulations in series constitute two capacitors in series. The voltage across each insulation is inversely proportional to the value of the individual capacitances. Where the distance in the air portion of the series is very small (about 0.5 mm or less), the air is a candidate for breakdown during the hi-pot test.

One method of finding the hi-pot failure is to take the unit apart, one piece at a time. Each time you remove a part, you hi-pot that part by itself, and you repeat the hi-pot test on the remaining pans. These two tests will tell you when have removed the pan that caused the failure.

Okay. You’ve found that the hi-pot failure is occurring in the fan. But you don’t stop there. You’ve got to find the particular insulation that is breaking down. You should continue taking things apart.

You’re looking for about 0.5 mm in series with a thin, solid insulation. Maybe the magnet wire to rotor shaft, where the wire can be spaced a fraction of a millimeter from the metal shaft giving you the air-solid series construction.

You may get a low-energy arc through the air, from the shaft to the magnet wire. It may or may not trip your hi-pot tester, depending on how sensitive you’ve set the trip. The arc current is limited by the impedance of the capacitance of the solid insulation portion of the series-connected insulations.

The problem with either corona or the low-energy arc is the very high temperatures in the arc. The temperature is high enough to burn the solid insulation part of the two insulations. (In switches, the arc temperature during the opening process is high enough to melt the metal at the ends of the arc!)

You may not get a complete punch-through of the solid insulation because there isn’t enough energy in the arc to burn all of the series solid insulation. However, with repeated testing, more of the solid insulation is burned away, the hole gets deeper, and successive hi-pot tests trip at lower and lower voltages. When the solid insulation finally has a carbon path all the way through, it is shorted out, and all that is left is the air. This now breaks down consistently at the same relatively low voltage compared to the initial breakdown. But, it doesn’t go to zero because there is always some air between the two conductors.

Yet another technique is to use a high-voltage insulation resistance meter to find the fault as you take the fan apart. Some insulation resistance meters include a switch-selectable voltage source; you want one that goes to at least 1000 volts. The insulation resistance meter is a low-current, high voltage source that will make a small, continuous arc that doesn’t do much damage. The meter tells you what’s happening.

When I evaluate a prototype product, I like to measure the value required to break the unit rather than simply test for pass-fail. In this way, I know how weak or how strong the unit is. I also know what the weakest link is. Then, I take it out and test the remaining parts to failure, and again determine the weakest link.

Pull on the strain-relief until it fails. Run the hi-pot test voltage up until it fails. Increase the 25-amp ground continuity test until it fails. Pull on the handle until it breaks. Increase the impact test until the enclosure breaks.

Later, should a problem arise on the production line, I can guess at what might be the problem, and can quickly test for it. I either know what the problem part is likely to be, or I know what it is not likely to be.

Finding a problem with pass-fail testing requires lots and lots of testing and, consequently, a long time. Finding a problem by measuring the magnitude at which both “passed” and “failed” units fail only requires a few units and, consequently, a short time.

Pass-fail thinking and testing does not tell you how good or how bad, or how strong or how weak. If you don’t know how good or how strong, then you don’t know how close you are to failing. If you don’t know how close you are to failing, then you run the risk of some units failing in production or, worse yet, in the field.

The “passed” ones often can tell you more than the “failed” ones—if you know what breaks, and what it takes to break it.

Measurement is the answer.

Run the unit to failure. Then, measure the magnitude of the force that causes failure. Now, you know how good, how bad, how strong, or how weak.

Pass-fail testing necessarily must be the kind of test in a standard. Pass-fail testing necessarily must be the process of a certification house. But, for you, every pass-fail test should be changed into one of measurement. When you make the measurement and get a value, the value proves whether you pass or fail. If you perform a pull test on a strain-relief and find that it fails at 125 pounds, you have proved that it passed the 35-pound test. If you fail a hi-pot test at 4100 volts, you have proved that it passed the 1500-volt test.

Don’t just know your product passed the test; know how good your product is. It gives you power.

author_nute-richardRichard Nute is a product safety consultant engaged in safety design, safety manufacturing, safety certification, safety standards, and forensic investigations. Mr. Nute holds a B.S. in Physical Science from California State Polytechnic University in San Luis Obispo, California. He studied in the MBA curriculum at University of Oregon. He is a former Certified Fire and Explosions Investigator.

Mr. Nute is a Life Senior Member of the IEEE, a charter member of the Product Safety Engineering Society (PSES), and a Director of the IEEE PSES Board of Directors. He was technical program chairman of the first 5 PSES annual Symposia and has been a technical presenter at every Symposium. Mr. Nute’s goal as an IEEE PSES Director is to change the product safety environment from being standards-driven to being engineering-driven; to enable the engineering community to design and manufacture a safe product without having to use a product safety standard; to establish safety engineering as a required course within the electrical engineering curricula.

Related Articles

Digital Sponsors

Become a Sponsor

Discover new products, review technical whitepapers, read the latest compliance news, trending engineering news, and weekly recall alerts.

Get our email updates

What's New

- From Our Sponsors -

Sign up for the In Compliance Email Newsletter

Discover new products, review technical whitepapers, read the latest compliance news, trending engineering news, and weekly recall alerts.