The Development of Proficiency Testing Programme for Electrical and Mechanical Safety Tests

Editor’s Note:  The paper on which this article is based was originally presented at the 2018 IEEE International Symposium on Product Safety Engineering in San Jose, CA. It is reprinted here with the gracious permission of the IEEE. Copyright 2020 IEEE.

The ISO/IEC 17025 accreditation is essential to consumer product testing laboratories. According to the requirements of Consumer Product Safety Improvement Act (CPSIA), consumer products shall be sent to third-party testing laboratories for testing before they can be put into the market. The certified third-party testing laboratories must already obtain the ISO/IEC 17025 accreditation in the specified test category [1]. The accreditation bodies require the ISO/IEC 17025 accredited laboratories to regularly join the proficiency testing (PT) scheme in appropriate testing areas.

The Hong Kong Accreditation Service (HKAS) provides ISO/IEC 17043 accreditation services to proficiency testing providers in Hong Kong [2]. Current proficiency testing providers are providing three different proficiency test areas: (1) calibration; (2) medical testing; and (3) chemical testing [3]. As the majority of testing and certification companies in Hong Kong are providing consumer product testing services, they can only join the proficiency testing organized by overseas services providers, such as LGC in UK, IQTC in China, ASTM in USA or IEC. In addition, the HKAS also irregularly organizes proficiency testing programmes to the accredited laboratories (e.g., acoustics testing carried out in 2015) [4]. However; they only invited the laboratories that are already accredited by HKAS to join the programme. Therefore, it is impossible to determine the competitive between HKAS accredited laboratories and other overseas laboratories.

In order to the fulfill the need of the industry, the Open University of Hong Kong (OUHK) is planning to develop a proficiency testing programme that is suitable for electrical and mechanical safety tests in Hong Kong. This article describes the development process of proficiency testing programme, frequency of testing, selection of test specimen and method of data analysis.

Purpose of Proficiency Testing

The ISO/IEC 17025 standard was developed to specify the general quality and technical requirements for the competence of testing and calibration laboratory. It specifies both the management and technical requirements. The management requirements require the laboratories to apply the proficiency testing and use the results to develop appropriate preventive actions. Meanwhile, top management shall review the result of interlaboratory comparisons or proficiency tests in the management review meeting.

In the technical requirements, on the other hand, the quality assurance system shall include: 1)the use of certified reference materials to valid the test; 2) participate the proficiency testing; 3) replicate tests or calibrations using the same or different methods; 4) retesting of retained samples; and 5) correlation results for difference characteristics of a specimen [5].

Some researchers described that proficiency testing is a good platform to supplement a laboratory’s internal quality control procedure and enables the comparability of measurement between laboratories. In other words, it can be considered as an external partial audit of laboratory quality management system [6].

International Standards for Proficiency Testing Programme

Several international standardization bodies have published guidelines and criteria for the development of proficiency testing programmes. ISO/IEC 17043 describes the general requirements for proficiency testing and is adopted by most of the accreditation bodies to offer accreditation services, such UKAS, NATA, A2LA, CNAS and HKAS. Table 1 shows other guidelines published by standardization bodies in describing essential criteria of a proficiency testing programme. Most of them are applicable to a specific product, such as fabric or water.

Table 1: List of standards for developing the proficiency testing programme

The ASTM D6674 and ASTM E2027 are designed for chemical or fabric materials testing. The OUHK, on the other hand, addresses proficiency testing programmes for mechanical and electrical safety testing in accordance with ASTM E1301.

There are other PT programmes, such as medical testing, chemical or fabric testing, available in the market. They summarized the challenges of PT development, including: 1) selection of specimen; 2) homogeneity and stability validation of specimen; and 3) selection of statistical analysis method to test the differences between laboratories [8].

Development Process of Proficiency Testing Programme

The development of a proficiency testing programme includes several major steps: 1) develop a Quality Management System (QMS) and obtain the ISO/IEC 17025 laboratory accreditation; 2) develop the proficiency test scheme; 3) invite laboratories to join the trial run of proficiency testing scheme to valid the scheme; 4) modify the scheme; and 5) apply for the ISO/IEC 17043 proficiency testing provider accreditation.

Step 1: We will appoint a reference laboratory to conduct the preliminary measurement and find the assigned value for determining for interlaboratory differences. We will assign our own laboratory to be the reference laboratory and gain the ISO/IEC 17025 accreditation.

Step 2: All essential requirements of ISO/IEC 17043 must be met by the PT scheme. Table 2 shows the major essential requirements and explains how we proceed to meet the requirements.

Table 2: List of essential requirements of ISO/IEC 17043: 2010 [9]

The criteria of PT specimen homogeneity and stability shall be established and analyzed. ISO Guide 34, ISO Guide 35 and ISO 13528 can be referenced to develop the criteria. According to ISO/IEC 17043, it is acceptable to consider the effect of uncertainty if the specimens are not sufficiently homogeneous or stable. The ISO 13528:2015 Annex B suggested a four-step method to test for homogeneity. Table 3 shows the steps to test for homogeneity.

Step	Descriptions
1	Preparation and packaging of the samples
2	Random select at least ten samples per batch
3	Prepare two subsamples from each sample
4	Samples will be measured randomly under repeatability conditions. The samples will be considered to be sufficiently homogeneous if

Table 3: Method for testing homogeneity [10]

In one of our trial PT programmes, two measurands are selected to be measured, i.e., kinetic energy of projectile toys (Figure 1) and sound pressure level of sound-emitted products (Figure 2). We establish the procedure to test stability.

All samples were brought from the same batch from the supplier such that they are considered to be homogeneous. Therefore any one of the samples is being representative to the batch and is sufficient for this analysis. In order to simulate the actual operations of PT items through the PT program, the reference laboratory will carry out the followings: 1) determine assigned value (KE) of the sample; 2) repeat operation of the sample as required for 20 times; 3) repeat operation of the sample as required for determining the travelling speed for 5 times; 4) determining the kinetic energy (KE) of the sample; and 5) apply the statistical method to determine the homogeneous.

The second trial of PT programme is on electrical safety testing. The measurement of creepage clearance is selected and sent the specimens to the interested laboratories. The creepage distance means the shortest distance along the surface of a solid insulating material between two conductive parts (Figure 3).

IEC 60601, IEC 62115 or IEC 60335 specify the required values and the minimum creepage distance can avoid failure due to tracking. Failure in creepage distance may cause short circuit and fire hazard. It should be noted that larger clearances are required if the product is subjected to mechanical vibration. The degrees of pollution are other factor to affect the required creepage distance [11].

Although the measurement of creepage distance is critical to the safety of electrical products, the standards do not specify the resolution and accuracy of measurement instruments. Some laboratories are using the Vernier caliper and others may use measuring microscopes (Figure 4). ISO Guide to Uncertainty of Measurement (GUM) divides the uncertainty to Type A and B. Type A uncertainty can be evaluated by the statistical analysis of series of observations. Type B uncertainty can only be evaluated by means other than the statistical analysis of series of observations.

Some examples of sources of uncertainty that lead to Type B evaluations are: 1) reference instruments calibrated by laboratory; 2) physical constants used in the calculation of the reported value; 3) environmental effects that cannot be sampled; 4) possible configuration/geometry misalignment in the instrument; and 5) resolution of the instrument [12]. Due to different measurement instruments having different resolutions, the uncertainty of measurements may be different. The Programme aims to evaluate the competence of laboratories and the uncertainty of measurement due to the resolution effect of measurement instruments.

Evaluation of Proficiency Testing Results

The participated laboratories will submit the measurement results to the PT provider. Then the PT provider shall apply the pre-determined appropriate statistical method to analyze the difference between PT participants. The most common methods are z scores and En scores. The assumption of z scores are made with a hypothesized distribution of competent laboratories and not on any assumption about the distribution of the observed result. We can interpret the z scores by:

• A result that gives |z| < 2.0 is considered to be acceptable;
• A result that gives 2.0 < |z| < 3.0 is considered to be questionable;
• A result that gives |z| > 3.0 is considered to be unacceptable.

En scores will be used if the PT participant’s ability having the result close to the assigned value within their claimed expanded uncertainty. The method is commonly used for PT in calibration.

For the PT programme used in this research, the z score method is found to be more suitable to evaluate the difference in the conducting the quantitative measurement, such as kinetic energy and creepage distance.

Conclusion

This article described the development process of a proficiency testing programme that is suitable for electrical and mechanical safety tests. The current HKAS accredited proficiency testing programme only covers chemical and medical tests and calibration. The new proficiency testing programme can fulfill the need of industry in order to meet the ISO/IEC 17025 requirements and help to industry to evaluate the differences between different laboratories or locations.

Acknowledgement

The work described in this paper was partially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (UGC/IDS16/16).

References

1. Consumer Safety Improvement Act (CPSIA) of 2008. Available at https://www.cpsc.gov/s3fs-public/cpsia.pdf. Accessed on January 9, 2018
2. HOKLAS 017: 2016 Technical Criteria for Accrediting Proficiency Testing Providers. Available at http://www.itc.gov.hk/en/quality/hkas/doc/hoklas/HOKLAS017-Abridged.pdf. Accessed on January 9, 2018
3. HOKLAS Index of Accredited Tests & Calibrations, Proficiency Testing Providers. Available at http://www.itc.gov.hk/en/quality/hkas/doc/common/directory/hoklas_p tp_en.pdf. Accessed on January 9, 2018
4. HKAS Proficiency Testing Programme, Report No. 105 “Toys Acoustics Test” (HTC/2016/01), August 2016
5. BS EN ISO/IEC 17025:2005, Incorporating Corrigendum No.1, General requirements for the competence of testing and calibration laboratories
6. Juniper, Ian Robert. “Quality issues in proficiency testing.” Accreditation and quality assurance 4.8 (1999): P. 336-341.
7. CE mark. Available on https://www.gov.uk/guidance/ce-mark
8. Miller, W. Greg, et al. “Proficiency testing/external quality assessment: current challenges and future directions.” Clinical chemistry 57.12 (2011): 1670-1680.
9. BS EN ISO/IEC 17043:2010, Conformity assessment – General requirements for proficiency testing (ISO/CASCP 17043: 2010)
10. BS ISO 13528:2015, Statistical methods for use in proficiency testing by interlaboratory comparison
11. IEC 60601-1-11:2015, Medical electrical equipment – Part 1-11: General requirements for basic safety and essential performance – Collateral standard: Requirements for medical electrical equipment and medical electrical systems used in the home healthcare environment
12. BIPM, IEC, et al. “Evaluation of measurement data – guide for the expression of uncertainty in measurement. JCGM 100: 2008.” Citado en las (2008): 167