Reproducibility of CISPR 25 ALSE Test Method

Editor’s Note: The paper on which this article is based was originally presented at the 2017 IEEE EMC Symposium, where it received recognition as the Best Symposium Paper. It is reprinted here with the gracious permission of the IEEE.

Accredited test laboratories, complying with the requirements of ISO/IEC 17025 have to evaluate measurement uncertainty (clause 5.4.6 of [1]). CISPR test methods are accompanied by examples of uncertainty budgets published in the standard CISPR 16-4-2 [2]. The CISPR 25 [3] Absorber Lined Shielded Enclosure (ALSE) test method, the worldwide reference for the assessment of conformity of the radiated and conducted emissions from automotive components, is an exception. No uncertainty budget is indeed available for this electromagnetic compatibility (EMC) test.

A valid (fulfilling the recommendations of the Guide to the Expression of Uncertainty in Measurement, briefly the GUM [4]) uncertainty budget can be issued if and only if a measurement model involving all the quantities (the input quantities) having a non-negligible influence on the value of the measurement and (the output quantity) can be determined. The measurement model must be applicable to any test house, provided that the reproducibility conditions set by the test method (e.g. the requirements on the characteristics of measuring instrumentation, test facility, measurement setup) are met. Unfortunately, some test methods are ill-defined because the specified reproducibility conditions cannot assure adequate measurement reproducibility. For such test methods it is not possible to determine a measurement model and therefore to calculate measurement uncertainty. It is up to standards committees to specify the re- producibility conditions of the test methods. Finally, a measurement method whose reproducibility conditions are not tightly controlled cannot be submitted to proficiency testing [5-7] for quality assurance of test results (clause 5.9.1 of ISO/IEC 17025).

In order to evaluate measurement reproducibility of the ALSE test method an interlaboratory comparison (ILC) started in September 2016 and concluded in February 2017. Nineteen data sets have been received by as many test houses. A stable source of electromagnetic field mimicking a product has been circulating among test houses that have implemented the test method (fulfilling the reproducibility conditions set by the standard CISPR 25). 146 frequencies have been investigated in the frequency range from 1 MHz to 1000 MHz. The measured values of the electromagnetic field have been statistically processed in order to derive, frequency by frequency, a reference value (robust mean) and the spread (robust standard deviation) of the measured values about the reference value. The spread is a measure of non-reproducibility. Information concerning the measuring instrumentation, the ALSE and the specific test setup adopted has been communicated by the participants to the organizers of the ILC.

The frequency range from 10 MHz to 50 MHz resulted to be most critical (worst reproducibility, i.e. larger spread of measured values). This was in part expected since reproducibility issues with the electric field measurement with the 1.04 m rod antenna in the range from 10 kHz to 30 MHz have been identified more than 30 years ago and they are still under consideration [8-18].

Since the publication of MIL-STD-461F [19] the way the counterpoise of the rod antenna is grounded for military tests has been drastically deviating from both MIL-STD-461E [20] and CISPR 25 ed. 3 [21] (these two standards were aligned in this respect). MIL-STD-461G [22] and CISPR 25 ed. 4 [3] confirmed this important difference between the military and automotive test setups. Notably, a new informative annex J on the validation of the performance of the ALSE (from 150 kHz to 1 GHz) was introduced in [3].

Much less attention has been devoted to the frequency range above 30 MHz [23-25]. The results of the ILC in the frequency range between 30 MHz and 1 GHz are therefore particularly interesting since they convey fresh information.

The scope of the ILC is not only to evaluate the non- reproducibility of the ALSE test method but also to identify additional measures for the purpose to improve reproducibility. Finally, a simple and effective solution which can improve the reproducibility by reducing the effects of ALSE resonances from 10 MHz to 30 MHz based on simulation results and verified through experiments is proposed. The solution is not new, see [11] and [26], but it was found before we knew the existence of the referenced documents. Therefore, this study represents a sort of independent confirmation of the validity of the proposal.

This paper is structured as follows. In section “Travelling Sample and Measurement Protocol” the essential information regulating the ILC activity is reported. The statistical analysis of the measured values provided by the participants in the ILC is described in the section “Statistical Analysis of Measured Values.” The results of the ILC are reported in the section “Results of the ICL.” In the “Conclusion,” the solution proposed to improve reproducibility in the frequency range from 10 MHz to 30 MHz is presented.

Travelling Sample and Measurement Protocol

The information to participants in the ILC is in a public document named “scheme” of the ILC and issued by the coordinator of the ILC. The essential information contained in the scheme is here reported.

Scope of the ILC, Participants, Scheduling

The scope of the ILC is to quantitatively evaluate the reproducibility of the Absorber Lined Shielded Enclosure (ALSE) radiated emission measurement method in the frequency range between 150 kHz and 1000 MHz. Measurements are carried out according to the measurement method defined in clause 6.4 of the CISPR 25 ed. 3 [21]. Deviations from the standard measurement method are not allowed.

Accreditation to ISO/IEC 17025 is not required. Participation is free of charge. Each participant has one week available to carry out measurements. Each participant has to select the week during which measurements are carried out at a given (not reported here) Doodle link.

Travelling Sample

The travelling sample is the combination of a comb generator (TESEQ RSG 3000) and a radiating element (see Figure 1). Ancillary equipment is provided by the coordinator for connecting the comb generator to the radiating element and for terminating the radiating element. The radiating element is the long wire antenna described in annex J of [3]. It consists of a round microstrip of 5 cm height and 50 cm length equipped with a female N-connector at the input and at the output. The comb generator is a battery operated, high stability impulse generator switching at the selectable rate of 1 MHz, 5 MHz and 10 MHz equipped with a female N connector (thus an adapter is necessary to connect the comb generator to the radiating element). The radiating element is terminated into a 50 Ω load (adapter and termination provided by the coordinator). The comb generator is equipped with 4 x 1.2 V, NiMH, Mignon, type AA batteries. The working time is approximately 16 h with full charge.

Figure 1: The travelling sample (comb generator plus radiating element). The radiating element is terminated by a 50 Ω load (not visible).

Measurement Procedure

The measurement procedure consists of: 1) preliminary verification (sanity check) of the travelling sample, and 2) measurement of the electric field.

Preliminary Verification

The scope of the preliminary verification is to check the sanity of the travelling sample after transportation and before measurement of the electric field. It consists in connecting the output of the comb generator to a 20dB attenuator (provided by the coordinator) and then measuring the output signal with the EMI receiver in three runs. EMI receiver and comb generator settings are as follows. Run 1 (1 MHz – 30 MHz): 1 MHz comb generator frequency spacing, 9 kHz resolution bandwidth, 5 kHz frequency step, 5 ms dwell time, average detector, at least 10 dB internal attenuation. Run 2 (30 MHz – 200 MHz): 5 MHz comb generator frequency spacing, 120 kHz resolution bandwidth, 50 kHz frequency step, 5 ms dwell time, average detector, at least 10 dB internal attenuation. Run 3 (200 MHz –1000 MHz): 10 MHz comb generator frequency spacing, same settings of the EMI receiver as in run 2. Each participant has to verify that at three harmonic frequencies (10, 50 and 1000 MHz) the deviation between the measured values and three reference voltage values (specified by the coordinator) is within ±3 dB. If the comb generator does not meet the pass criterion the participant shall contact the coordinator. Finally, the sanity of the radiating element and the 50 Ω termination is verified by using a digital multimeter.

Measurement of Electric Field Strength

The travelling sample is placed on the test bench (table) with the radiating element at the center and its edge aligned with the rim of the ground plane (see Figure 1). The center corresponds to the middle of harness position during radiated emission test. The noise floor is recorded using the same receiver settings used for run 1, 2 and 3. Then the radiated emission test according to CISPR 25 standard is performed in five runs. The comb generator and the EMI receiver settings are the same used for the preliminary verification in the corresponding frequency range (except for 0 dB instead of 10 dB minimum internal attenuation of the EMI receiver). Run 4 (1 MHz – 30 MHz) is for rod antenna measurement. Runs 5 and 6 (30 MHz – 200 MHz), run 7 and 8 (200 MHz – 1000 MHz): vertical (runs 5 and 7) and horizontal (runs 6 and 8) polarization of the receiving antenna.

In addition to the measured values information is provided by the participants through a form prepared by the coordinator and consisting of: Pictures of the measurement setups; Size of the ALSE; Type of ALSE, i.e. semi-anechoic or fully-anechoic room; Possible compliance of ALSE with Normalized Site Attenuation (NSA) requirement as specified in CISPR 16-1-4 for semi-anechoic or fully-anechoic rooms; Possible compliance of ALSE with other (different from NSA) requirements; Type of the absorbing material (e.g., only pyramidal absorbers, only ferrite tiles, both pyramidal absorbers and ferrite tiles); Size of the ground plane (e.g. 2.5 m x 1 m); Bonding of the ground plane to the shielded enclosure (picture and short description); If available the maximum deviation obtained from the Balun DM/CM conversion check of the receiving biconical (or bilog) antenna, as specified in CISPR 16-1-4; If the Balun DM/CM conversion check is not available record the maximum deviation obtained rotating the receiving biconical (or bilog) antenna by 180° (in each polarization, in the frequency range between 30 MHz and 200 MHz); Use of antenna cable loaded by ferrites (yes or no); Other information that the participant judges relevant and useful for the interpretation of measurement results.

Statistical Analysis of Measured Values

The coordinator processes the measurement results provided by each participant in the ILC in order to obtain: The amplitude of each harmonic of the voltage measured in runs 1, 2 and 3; The amplitude of each harmonic of the electric field strength measured in run 4; The maximum amplitude, between vertical and horizontal polarization, of each harmonic of the electric field strength measured in runs 5, 6, 7 and 8.

The average and the standard deviation of the measurement values provided by the participants (once processed as described in 2 and 3 above) are obtained by the coordinator through calculation of the robust average x* and the robust standard deviation s* , respectively. Calculation is done according to the algorithm A described in Annex C of ISO 13528 [27]. The robust standard deviation s* quantifies the non-reproducibility of the measurement method. Calculation of the robust average and the robust standard deviation is done frequency by frequency.

If the coordinator of the ILC detects a largely deviating measurement result then informs the test house that produced that measurement result and asks for investigation. If the cause of such large deviation is found and it is due to a mistake, broken or damaged equipment then the measurement result is classified as a blunder and removed from the ensemble of valid measurement results, otherwise it is classified as an outlier. Outliers are not removed and use is made of outlier resistant (robust) statistical analysis.

Results of the ILC

Aggregate data of the voltage measured through runs 1, 2 and 3 are first discussed. The plots showing the robust mean of the voltage measured through runs 1, 2 and 3 is shown in Figure 2, Figure 3 and Figure 4, respectively, as a function of frequency. The robust standard deviation is also shown as an error bar in the same plots. The plots demonstrate the stability of the comb generator and, consequently, of the travelling sample. Indeed, the robust standard deviation is about 0.5 dB. Actually, the robust statistical analysis is not necessary for measured voltages (no outliers). We also observe that in the frequency range from 1 MHz to 30 MHz the voltage increases approximately at 20 dB/dec. This is an undesired behavior since, as it is shown in the subsequent plots, the electric field generated by the travelling sample is relatively low (i.e. close to the noise floor at some test houses) at the first two harmonics. Another issue concerns run 4 and runs 5 and 6 and it is the relatively wide frequency spacing of the harmonics of the comb generator. The peaks and troughs of some resonances may be missed thus underestimating non-reproducibility.

Figure 2: Voltage measured in run 1 (robust mean). Error bars correspond to ±1 standard deviation.

Figure 3: The same as Figure 2 but run 2.

Figure 4: The same as Figure 2 but run 3.

Figure 5, Figure 6 and Figure 7 show the raw results obtained from run 4, the maximum over runs 5 and 6 and the maximum over runs 7 and 8, respectively. The first comment looking at Figure 5 is that the reproducibility in the frequency range of run 4 is simply unacceptable. The situation could appear even worse if a narrower frequency spacing of the comb generator were available. One curve (marked with a bold red cross) has been removed being a blunder (the shield of the antenna cable was partially disconnected).

Figure 5: Electric field measured in run 4. Raw data. The curve identified by the bold red cross is a blunder to be removed.

Figure 6: Electric field measured in run 5 and run 6 (maximum over vertical and horizontal polarizations). Raw data. The curves identified by the bold red cross are blunders to be removed.

Figure 7: The same as Figure 4 but blunders are removed, and distinction is made between use of biconical (red curve) and biconilog (black curve) as receiving antennas.

The noise level (determining the sensitivity) is participant dependent and generally comprised between 0 dB(µV/m) and –5 dB(µV/m). Exceptions are however present with participants exhibiting a noise level of 5 dB(µV/m) or even 10 dB(µV/m). Nine measured values having a margin from the noise floor less than 6 dB have been removed (seven at 1 MHz, two at 2 MHz). Finally note that at 1 MHz, i.e. far from the resonances occurring in the range between 10 MHz and 30 MHz (see [8-17]) and where the influence of the ALSE is expected to be negligible, the maximum-minimum difference is about 11 dB. This indicates the presence of unresolved and, at the moment, unexplained issues with low-frequency electric-field measurements. The reproducibility in the frequency range from 30 MHz to 200 MHz (see Figure 6, where the maximum over horizontal and vertical polarization is shown) appears to be much better than the reproducibility from 1 MHz to 30 MHz.

A group of consistent measurement results can be identified. In particular, four measurement results are removed because they originated from blunders (see Figure 6, the curves identified by the bold red cross). The origin of these blunders is a defective cable (partially disconnected shield), two defective antennas (inversion test of bilog antennas resulting in more than 5 dB and up to 12 dB unbalance) and use of a wrong antenna factor (obtained through 3 m calibration instead of 1 m calibration as required by the standard, causing a lower measurement result by about 5 dB). Figure 7 shows the measurement results of the remaining fifteen test houses once the four blunders were removed. Curves in red color are obtained by using a biconical antenna (seven traces), those in black color (eight traces) are obtained by using a biconilog antenna. The common practice of using hybrid antennas originates from the fact that CISPR 25 ed. 3 [21] lists the biconilog as measurement antenna. CISPR 25 ed. 4 [3] removes the term biconilog but still does not forbid its use. Because of the near-field character of the measurement, laboratories should use identical antennas in order to improve the test reproducibility by controlling the coupling effects between the antenna and the test set-up. This is demonstrated in Figure 7 where it is seen that measurements with the bilog antenna (black) produce an underestimate (by up to 4 dB in the average) of the electric field compared with those obtained by using a biconical antenna (red) between 30 MHz and 50 MHz. Moreover [21] stipulates that the SAE ARP 958 [28], which does not mention the biconilog antenna, calibration procedure shall be used. Several calibration laboratories, where the antennas of many of the participants have been calibrated, modify [28] by using the three-antenna-method for the calibration of the 1 m gain. It is however not clear how much the antenna factors measured with this method diverge from those obtained by following [28] and using two identical antennas.

Also, in the case of the frequency range between 200 MHz and 1000 MHz (see Figure 8, where the maximum over horizontal and vertical polarization is shown) a group of consistent measurement results is identified. The same four blunders as in the range between 30 MHz and 200 MHz were removed (see Figure 4). Use of biconilog compared with log-periodic antenna did not provide evidence of a systematic difference on measurement results.

Figure 8: Electric field measured in run 5 and run 6 (maximum over vertical and horizontal polarizations). Raw data.

The results of the statistical analysis are shown in Figure 9, Figure 10 and Figure 11. The continuous line represents the robust mean, the vertical error bars correspond to ±1 robust standard deviation. The vertical scale of these three figures covers the same 40 dB range to facilitate comparison at glance. Comparing the error bars in Figure 9 to those in Figure 10 and Figure 11 it is immediately confirmed that the reproducibility in the frequency range from 1 MHz to 30 MHz is much worse than from 30 MHz to 1000 MHz. Between 1 MHz and 30 MHz the measurement results are, approximately, within ±15 dB (in terms of two standard deviations) while between 30 MHz and 1000 MHz they are approximately within ±5 dB (again in terms of two standard deviation), which is comparable with the Ucispr values for radiated emission measurements published in [2].

Figure 9: Electric field measured in run 4. Aggregate data (robust mean ±1 std). Values shown by red crosses are obtained from Table J.1 of [3] readapting the voltage level of the source.

Figure 10: Electric field measured in run 5 and run 6 (maximum over horizontal and vertical polarization). Aggregate data (robust mean ±1 std). Values shown by red crosses are obtained from Table J.1 in [3] (readapted for source level).

Figure 11: The same as Figure 10 but in the case of run 6 and run 7.

Red crosses correspond to the values in Table J.1 of [3] (transformed to take into account that the applied voltage is not 1 V attenuated by 10 dB, as required by [3] but the level shown in Figure 2 plus 20 dB). Note that below 2 MHz and above 24 MHz the deviation between the robust mean and the CISPR 25 values is greater than 6 dB (the allowed tolerance for ALSE validation). In Figure 10 and in Figure 11 the red crosses again correspond to the values in Table J.1 of [3] (readapted for source level). The deviation between the robust mean and tabulated values is generally within ±3 dB.

How to Improve the Reproducibility in the Frequency Range From 10 MHz to 30 MHz

The solution proposed is not new since it is already documentd in [11] and [26]. Rod antenna measurements are quite sensitive to the potential of the counterpoise with respect to the shielded enclosure [15].

In order to improve measurement reproducibility above 10 MHz it is necessary to connect the counterpoise to the shielded enclosure keeping the impedance of the connection as low as possible, e.g. by using a 60 cm x 120 cm aluminum foil as shown in Figure 12. The positive effect of this low-impedance connection can be appreciated by comparing Figure 13 (run 4, without aluminum foil, nine laboratories) with Figure 14 (run 4, with aluminum foil, the same nine laboratories). Above 10 MHz the maximum standard deviation passes from 7.7 dB to 4.7 dB and the mean value reduces by 3.3 dB, getting closer to the reference values in Table J.1 of Annex J of [3].

Figure 12: Connection of the counterpoise to the shielded enclosure by using a wide strip of aluminum foil.

Figure 13: Run 4, without aluminum foil.

Figure 14: Run 4, with aluminum foil.

Conclusion

Several practical solutions have been under investigation and confirmed during the ILC: a) In order to improve the reproducibility from 150 kHz to 30 MHz, suppressing the electric field below the counterpoise by keeping its impedance to the shielded enclosure as low as possible is essential; b) Routing the antenna cable perpendicular to the harness under test, to the monopole and elevated over the floor reduces reciprocal mutual couplings; c) Use of the biconilog antenna causes a significant reduction of the measured electric field between 30 MHz and 50 MHz and contributes to worsen reproducibility in the same frequency interval; d) Onsite (i.e. 1 m from the source and ac- cording to the standard setup) sanity verification of the biconical antenna through the balun unbalance test is essential to avoid large measurement errors; e) Use of ferrite-loaded coaxial cables is effective in reducing couplings and balun imbalance. Table J.1 values in Annex J of [3] are a valid reference for improving the ALSE performance. However, such values appear to be too low between 20 MHz and 30 MHz where the measured electric field, for constant voltage source, is expected to increase with frequency in the elevated setup. On the contrary, the reference values slightly decrease as if the measurement was carried out on the floor.

References

1. Conformity Assessment—General Requirements for the Competence of Testing and Calibration Laboratories, ISO/IEC 17025:2005.
2. Specification for Radio Disturbance and Immunity Measuring Apparatus and Methods—Part 4-2: Uncertainties, Statistics and Limit Modeling— Uncertainty in EMC Measurements, CISPR 16-4-2, 2nd ed., 2008.
3. Vehicles, boats and internal combustion engines –
   Radio disturbance characteristics – Limits and methods of measurement for the protection of on-board receivers, CISPR 25, 4th ed., 2016.
4. BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP, and OIML. (2008). Guide to the expression of uncertainty in measurement, JCGM 100:2008, GUM 1995 with minor corrections.
5. C. F. M. Carobbi et alii, “Design, Preparation, Conduct and Result of a Proficiency Test of Radiated Emission Measurements”, IEEE Trans. on Electromagnetic Compatibility, vol. 56, no. 6, 2014, pp. 1251-1261.
6. C. F. M. Carobbi et alii, “Proficiency Testing by Using Traveling Samples With Preassigned Reference Values”, IEEE Trans. on Electromagnetic Compatibility, vol. 58, no. 4, 2016, pp. 1339-1348.
7. C. F. M. Carobbi, Marco Cati, Carlo Panconi, “Generation and Measurement of a Reference Field for Round-Robin Comparison Purposes”,
   in Proc. IEEE Int. Symp. on EMC, Aug. 18-22, 2008.
8. A. C. Marvin, “The use of screened (shielded) rooms for the identification of radiation mechanisms and the measurement of free- space emissions from electrically small sources”, IEEE Trans. Electromag. Compat., vol. 26, no. 4, Nov. 1984, pp. 149-153.
9. A. C. Marvin, L. Steele, “Improved techniques for the measurement of radiated emissions inside a screened room,” IEE Electronics Letters, vol. 22, no. 2, Jan. 1986, pp. 94-96.
10. L. Dawson, A. C. Marvin, “New screened room techniques for the measurement of RFI,” Journal IERE, vol. 58, no. 1, Jan/Feb 1988, pp. 28-32.
11. S. Jensen, “Measurement Anomalies Associated with the “41 Inch Rod” antenna when used in Shielded Enclosures.” [Online]. Available:
    http://www.stevejensenconsultants.com/rod_ant.pdf
12. D. Warkentin, A. Wang, W. Crunkhorn, “Shielded Enclosure Accuracy Improvements for MIL-STD-461E Radiated Emissions Measurements”, IEEE International Symp. on EMC, 2005, pp. 404-409.
13. Turnbull, L., “The Groundplane Resonance – Problems with Radiated Emissions Measurements Below 30 MHz”, Automotive EMC Conference 2007, Newbury, UK, 16th Oct 2007, pp. 1-13.
14. C.W. Fanning, “Improving Monopole Radiated Emission Measurement Accuracy; RF Chamber Influences, Antenna Height and Counterpoise Grounding (CISPR 25 & MIL-STD-461E vs MIL-STD-461F),” IEEE International Symp. on EMC, 2009, pp.103-118.
15. K. Javor, “On the nature and use of the 1.04 m electric field probe,” Interference Technology, 2011, pp. 66-77.
16. P. Kay, “EMC test site validations for 1m automotive and military emissions tests,” 2011 EMC Symposium – Perth (EMCSA).
17. H. W. Gaul, “Electromagnetic Modeling and Measurements of the 104 cm Rod and Biconical Antenna for Radiated Emissions Testing Below 30 MHz,” IEEE International Symp. on EMC, 2013, pp. 434-438.
18. F. Lafon, R. Dupendant, J. Davalan, “Invesigation on dispersions between CISPR25 chambers for radiated emissions below 100 MHz,” EMC Europe, 2014, pp. 29-34.
19. Department of defense interface standard requirements for the control of electromagnetic interference characteristics of subsystems and equipment, MIL STD 461F, US Dept. of Defense, Dec. 2007.
20. Department of defense interface standard requirements for the control of electromagnetic interference characteristics of subsystems and equipment, MIL STD 461E, US Dept. of Defense, Aug. 1999.
21. Vehicles, boats and internal combustion engines –
    Radio disturbance characteristics – Limits and methods of measurement for the protection of on-board receivers, CISPR 25, 3rd ed., 2008.
22. Department of defense interface standard requirements for the control of electromagnetic interference characteristics of subsystems and equipment, MIL STD 461G, US Dept. of Defense, Dec. 2015.
23. A. Kriz, W. Müllner, “An Accurate Validation Procedure for Component Testing Chambers” [Online]. Available: http://itr.arcs.ac.at
24. A. Kriz, W. Müllner, “Analysis of the CISPR 25 Component Test Setup,” IEEE International Symp. on EMC, 2003.
25. F. Lafon, J. Davalan, R. Dupendant, “Inter-laboratory Comparison Between CISPR 25 Chambers, Identification of Influent Parameters and Analysis by 3D Simulation,” APEMC, 2015, pp. 71-74.
26. CISPR/D/WG2 N314, Japanese proposal for the counterpoise bonding 2, 2017-05-13.
27. Statistical Methods for Use in Proficiency Testing by Interlaboratory Comparison, ISO 13528:2016.
28. Electromagnetic Interference Measurement Antennas; Standard Calibration Method, SAE ARP 958, rev. D, reaffirmed 2003.

Carlo Carobbi is an Assistant Professor of Electronic Measurements and EMC at the University of Florence in Firenze, Italy. He can be reached at carlo.carobbi@unifi.it.

Danilo Izzo is an EMC engineer with Valeo in Stuttgart, Germany. He can be reached at danilo.izzo@valeo.com.