Implementing an Effective Product Development Process

I have spent most of my engineering career (50 years) at an automotive original equipment manufacturer (OEM), a Tier 1 electronics supplier, and as a consultant who has worked with over 40 different companies. I have observed that the quality of the product development process (PDP) and the experience of the design and test staff vary widely among different OEMs and suppliers.

There are a number of factors that impact the efficiency and effectiveness of the development process. Some examples are short design cycles, increasing complexity, staff reductions (including the most experienced people) and cost-cutting. However, there are also a number of things that are holding progress back including “we’ve always done it this way.”

There is such a large test infrastructure (equipment manufacturers, test labs, large OEM/vendor departments) that it is extremely difficult to change the PDP. OEMs all have similar testing specs which must be contractually met – these are minimum requirements. However, there are things that can be done to improve the process.

Many specs were written when automotive electronics represented an emerging technology. Further, environmental and EMC specs have evolved over many years, but these test methods have many limitations that are not appreciated by contemporary practitioners. To cite just one example, simple pass-fail criteria can result in test results that do not reflect real-world concerns and provide only a false sense of security.

No single designer can be expected to have the scope of experience necessary to consider all aspects of a PDP. Instead, such experience is often spread throughout an organization and not concentrated on any one project. Therefore, a holistic approach is required to achieve the best results in an efficient manner.

Although EMC is a very important part of the PDP, it is important for all involved to be aware of all parts of the process. By being informed of the overall process, more insightful, effective, and efficient designs and test plans can be developed. The main goal should be customer satisfaction and avoidance of very costly warranty repairs and recalls.

This article addresses the overall PDP and condenses the lessons that I’ve learned over many years. The goal is to make the development process more efficient and effective, especially by focusing on the development stage. This includes implementing in‑house EMC development testing capability.

PDP Observations

To start, here are some personal observations on the trajectory of the product development process during the course of my career and where we stand today:

1. Originally, reliability for automotive electronics was poor so a lot of tests were “invented.” There were minimal design practices for automotive EMC.
2. As product complexity and technology evolve, the traditional “cookbook” approach is not effective at finding and addressing many real-world concerns.
3. Testing methods have many limitations and compromises that may not be appreciated by contemporary practitioners.
4. Much time and money are spent on low-value exercises due to limited knowledge of specification history (practitioners don’t know when to “hold or fold”).
5. Much testing addresses old issues with limited added value, especially for modules that follow known basic design rules and use mature technology.
6. Different people looking at the same data can come up with quite different conclusions, depending on their background, insight, and flexibility.
7. Test specs/plans mainly “test for success.” You cannot maximize information by maximizing success rate. Instead, it is testing failures that serve to maximize information in the development stage.
8. Typical testing is often based on repeatability. However, some testing requires randomness to identify real-world issues.
9. Requirements validation (e.g., hardware, software, EMC, etc.) conducted under ideal conditions does not sufficiently address system interactions. Many tests are idealized simulations of the real world.
10. Testing of production-representative modules frequently occurs late in the design cycle. However, simple testing early in the PDP will help identify issues when they can be addressed efficiently and economically.
11. The test process currently employed by many OEMs is so complex, long, and expensive that it diverts from “play” time to identify bugs early.
12. Meeting specifications alone is not sufficient to mitigate field issues. The main goal is to minimize potential field issues, not just to meet the specifications. (See Figure 1.)

SAE J1938, SAE J2628

The recently published 32-page SAE information report, SAE J1938_202211, “Product Development Process and Checklist for Vehicle Electronic Systems” [1] addresses the many aspects of overall design-process issues for automotive electronic modules. The report (which I co-authored with the SAE’s Committee on Automotive Electronic Systems Reliability Standards) serves as a companion document to SAE J2628_201806, “Characterization, Conducted Immunity” [2]. Since it is impossible to be all-inclusive and cover every aspect of the design/validation process, J1938 can be used as a basis for preparation of a more comprehensive and detailed plan that reflects the accumulated “lessons learned” at a particular company.

The following is a condensed Table of Contents for J1938:

1. Scope
2. References
3. Contemporary design-validation perspective
   1. Test-related issues
   2. Test effectiveness example
   3. Cost reduction (CR)
   4. Trouble not indicated (TNI)
   5. Sample sizes
   6. Reliability prediction
4. Robustness validation process (RV)
   1. Preliminary assessment.
   2. Development stage
   3. Design validation (DV) readiness evaluation
   4. Design validation (DV)
   5. Product validation (PV)
   6. Conformity, TNI
   7. Example of RV process results
5. Design checklist for modules
   1. Component selection – 10 sections
   2. Circuit design checklist – 6 sections
   3. Software
   4. Diagnostics
   5. Reparability
   6. Environmental (non-EMC) – 5 sections
   7. Electromagnetic compatibility (EMC)
      • Component (module) level, test requirements
      • PCB layout rules for EMC
   8. Miscellaneous manufacturing process checklist

Typical Design and Test Process

Some of the major parts of such a process are as follows:

1. Design of the product to meet customer requirements
   (e.g., analysis, design reviews, etc.)
2. Development testing, usually based on variations of OEM specs*
3. Design verification (DV) testing following OEM specs*
4. Product validation (PV) testing following OEM specs* (can be modified depending on changes to the device under test (DUT) from DV level)

(* Environmental tests typically are hi-lo temp operation, thermal cycle, thermal shock, humidity, vibration, etc. Most OEM environmental-EMC specs are very similar and based on international standards.)

Design reviews are very important, but in practice there is wide variation in the quality of the event. It is difficult to conduct such reviews in today’s environment where experienced staff are limited. Detailed reviews (e.g., schematics) between the OEM and vendor are not always conducted since vendors may not want to share what they consider confidential information with those outside their company.

Of all the stages in such a process, development testing is one area in which OEMs can differentiate themselves from competitors. Factors include:

• Providing the maximum flexibility to experiment
• Allowing for sufficient reaction time
• Providing opportunity for early staging where failures maximize information
• Ability to push products beyond specification limits to determine design margins

Development testing may not be a large part of the typical test plan. Typical plans usually focus on verifying that a product functions in a known way within a given set of input conditions (i.e., meets requirements). What is often missed are those other unwanted outcomes that result from complex dynamic interactions of hardware/software, timing, throughput, electrical excursions, extreme operation, system interactions, and interfaces.

To be effective, the DUT should be tested in a sub-system configuration using realistic loads and interfaces. This is difficult to do since each module is typically tested independently using simulated loads/interfaces.

Both J1938 and J2628 provide details regarding the type of tests that should be part of the development evaluation, many of which are not included in typical OEM specs. These tests were developed by analyzing actual field issues and devising methods to identify them. There are two levels of such tests:

• • Level 1: Level 1 tests (see Table 1) are not typically called out in OEM specifications but can provide an early indication of DUT robustness. These tests are best done by the design engineer since results are often not pass-fail and must be interpreted by those intimately familiar with details of the design.
  • Level 2: Level 2 tests are more complex and are listed under the section “In-House EMC Testing” (see Table 2). Some Level 2 tests are modified versions of those that are typically called out in the OEM spec.

Table 1: Development tests, Level 1

1. Also useful for detecting changes in DUT response (degradation) after subjected to other environmental stresses.

Table 2: Summary of pre-compliance implementation; many OEMs use variations of these tests

1. Development lab may not be able to cover an entire frequency range.
2. OEM method not practical for development (requires anechoic chamber), use open sided TEM cell, parallel plate or antenna. Useful up to about 1 GHz which covers most issues.

EMC Part of the Process

EMC specification limits are all idealized simulations of the real world. Many specification setups and limits create a situation much worse than what would typically be experienced in a vehicle (could be considered as over-testing to maintain a safety margin). Reference 3 gives the history of an EMC specification for one major OEM. It addresses the quality of the event, the chance of success, and a summary of the many tests (origins, setups, limitations, etc.).

A few examples of EMC testing considerations that may result in undetected issues include:

• Not preconditioning the DUT before majority of EMC testing (i.e., DUT not subjected to other environmental stresses such as thermal cycling/shock, high temp exposure, ESD). These other stresses can “weaken” the DUT (e.g., electrolytic capacitor degradation), and is logistically difficult to assess.
• Not testing subsystem configurations. Each component of a subsystem may be tested separately (separate vendors) and may not represent the interactions that occur when tested as a realistic subsystem. This is difficult to implement since each DUT supplier usually does not want to share details of their design.
• Due to high testing costs and time restraints, it is often not possible to test the DUT in all possible operating modes and supply voltages, and the evaluation must rely on a thorough analysis.

The EMC design and testing process can be time-consuming, inefficient, and costly. EMC issues are often identified during qualification testing in an accredited EMC testing lab late in the design cycle. To develop a cost-effective solution may take a lot of time. However, many accredited EMC labs may be fully booked, have long lead times, and are expensive (typically > $2k per day). To run a full qualification test on a product with multiple operating modes can easily exceed $100k.

In-House EMC Development Testing

Major suppliers to automotive OEMs typically have extensive in-house capability, but smaller suppliers may not. Setting up an in-house pre-compliance test facility can improve first-pass qualification success and can be used early in the design cycle to identify potential issues before formal lab testing. Identifying issues early allows maximum flexibility to experiment and sufficient reaction time before a design is frozen and difficult to change.

This article includes specific details regarding the cost of setting up such a facility at an automotive OEM whose main products are headlight and taillight assemblies. Implementation can be separated into two parts:

1. Primary test characteristics:
2. High frequency
3. Many issues occur in these tests
4. Resolving can be lengthy so need in-house facility

Secondary test characteristics:

1. Low frequency (except ESD)
2. Equipment expensive
3. Minimal test time
4. Does not require a shielded enclosure
5. If an issue is discovered, a fix is often easy to identify (e.g., using an oscilloscope to troubleshoot)
6. Can be performed without resorting to an outside EMC lab

Radiated Emissions and Immunity Testing

Comb Generator

For evaluating radiated emissions (RE), a comb generator is useful to determine test cell response. Figure 3 shows one inexpensive way to implement a comb generator. It consists of a TTL 10 MHz oscillator driving a Tekbox TBCG2. The rise times of the TTL oscillator are not fast enough to produce a broad spectrum, but the TBCG2 contains step recovery diodes which create a much broader spectrum. Figure 4 shows spectrum analyzer results for the TBCG2 output.

Open-Sided TEM Cell

This is one option that can be bought for about $1k (Tekbox TBTC3). However, it has limited test volume, and a plate separation of about 15cm, which may be an issue for larger DUTs. This was the case for some products such as headlight assemblies (see Figure 5).

Parallel Plate

This was implemented by removing the center plate from the aforementioned open-sided TEM cell and modifying the end terminations (see Figures 6 and 7). Reference 4 addresses this issue. Although that paper only covers up to 150 MHz (per EN 55020, EN 61000-4‑20), it is useful up to 1 GHz for development. However, doing so provides a challenge since the impedance theoretically becomes 100 ohms instead of 50 ohms (Reference 5). Although RF high power for radiated immunity (RI) terminations are readily available for 50 ohms, 100 ohm high power ones are not.

For better matching terminations, Figure 8 shows an L-pad and 100 ohm implemented by 100 ohm high power RF resistors. 100 ohm was used as compromise and is more readily available. Even without terminations to better match both ends, it is useful with standard 50 ohm. Figure 9 shows results of RE for the comb generator.

It is important to position the parallel plate (PP) on a non-conductive table (not on a ground plane). Reference 6 shows how the area surrounding a TEM can affect the results. But for development testing purposes, such limitations can be tolerated since we are only looking for differences. The use of a screen room is not critical for RE. The setup consists of a 50‑ohm termination on one end and a direct feed to a spectrum analyzer on the other end.

For RI, field strengths approaching 100v/m can be achieved with a 40-watt amplifier such as a NSG C4070-45 signal generator and power amplifier. Use of a shielded enclosure is required to prevent interference with surrounding communications.

PCB Log Periodic

This is based on Reference 7 by Ken Wyatt. Figure 10 shows the RE comb generator setup and Figure 11 shows the results are good in the antenna specified range of 400-1000 MHz. The antenna can also be used for RI. Reference 8 states that the antenna can easily take 100 watts at 400 MHz. In the antenna data sheet, a table of frequency vs. antenna factor is given, and using online calculators (e.g. A.H. Systems), field strengths exceeding 50 v/m at 0.75 meters require only about 20 watts.

Acknowledgments

Much of the information regarding implementation of a development EMC test lab was done at Flex N Gate in Allen Park, Michigan with the help of Monesh Hazari.

References

1. SAE J1938_202211, “Product Development Process and Checklist for Vehicle Electronic Systems.”
2. SAE J2628_201806, “Characterization, Conducted Immunity.”
3. “Evolution and Analysis of EMC Specification of a Major Automotive OEM,” Arnold Nielsen, In Compliance Magazine, April 2022.
4. “Design and Implementation of a TEM Stripline for EMC Testing,” Sezgin Hilavin and Alp Kusterpeli, 2014.
5. D.M. Pozar, Microwave Engineering, Wiley, 4th edition
6. “Properties of Open Strip Lines for EMC Measurements,” Wolfgang Bittinger, 1993 IEEE International EMC Symposium.
7. “PCB Log Periodic Antenna,” Ken Wyatt, EDN Magazine, December 15, 2012.
8. “Designing PCB Log Periodic Antennas,” Kent Britain.