Why standards-based conformity assessment is critical and how companies can get involved

Innovative new technologies typically come from individual companies rather than from standards bodies. Even so, standards play a critical role in making those brand-new technologies something that enterprises and other end-users feel comfortable buying. Put simply, standards enable new technologies to go mainstream and allow customers to choose best-of-breed solutions from multiple vendors.

Vendors often start selling a new technology years before relevant standards are published. They know that consumers and businesses want the latest and greatest technologies, so they frequently ship pre-standard versions to meet that demand. If they don’t, they could be at a competitive disadvantage.

This practice raises a host of questions – some philosophical, some practical – about the role that standards play for vendors, their customers, and the challenges they face when trying to get the latest technology implemented to gain competitive advantage. Some of these questions include:

• Should early adopters delay the launch of pre-standard products or accept the associated risk? Analysts often advise enterprises to wait until a standard has been ratified.
• Vendors typically layer on additional, proprietary features to help differentiate their products. However, if these features are not based on an approved standard, there is a possibility that the features may not be interoperable with systems built on the approved standard. 
• Standards can be interpreted in different ways. Vendors need documentation of what requirements must be met to claim conformance so that differences in interpretations are minimized. So how much can enterprises and other end-users rely on standards for enabling multi-vendor environments?

Vendors have a vested interest in helping the industry as a whole address these challenges. For example, some enterprises might be reluctant to buy new gear –
pre-standard or not – because they don’t want the expense of resolving interoperability issues. That means lost or delayed revenue for vendors.

Some enterprises are willing to buy pre-standard solutions, but only on the condition that their vendors provide warranties and other assurances that they will bear responsibility for resolving any interoperability issues. Vendors also have costs associated with helping customers resolve interoperability issues, such as staffing up to field those inquiries, investigating them, and offering solutions. Those support costs may significantly increase. 

Testing for the Common Good

Now for some good news. The same organization behind most major technology standards – the IEEE – also provides multiple opportunities for vendors to overcome these challenges. In the process, vendors also can play a key role in ensuring that standards anticipate their needs and the needs of their customers. That involvement keeps standards ahead of the curve, not behind it.

To understand how to consider the example of synchrophasors. They’re a key part of the power grid and provide utility operators with time-synchronized data about the current, voltage, and frequency at each location.¹ That information helps operators avoid blackouts and brownouts by pinpointing where to upgrade infrastructure to maximize capacity and reliability. In fact, synchrophasors are so important for smart grids that the U.S. American Recovery and Reinvestment Act of 2009 helped fund more than 1000 of them, worth over $328 million.

Synchrophasors also provide a cautionary tale that highlights the importance of independent technology certification—not only for utilities but for just about every other industry, too. In 2014, the National Institute of Standards and Technology (NIST) tested a variety of phasor measurement units (PMUs)² using the IEEE C37.118.1-2011 Standard for Synchrophasors for Power Systems³ and found that 80 percent of the units tested weren’t compliant. Many of the vendors whose PMUs failed subsequently made changes, resubmitted them, and then passed.

The moral of this story is that vendors—in any industry—can avoid the time, expense, and negative PR of re-engineering their products by participating in independent certification before bringing them to market. This also helps them avoid developing a reputation for recalling products due to non-compliance, as well as the expense of reimbursing customers whose operations are disrupted. 

It’s important to note that a standard outlining requirements for a new technology may not address all aspects of implementation for that technology. Some of those aspects aren’t immediately obvious to vendors and end-users. Hence, the importance of testing and certification, which can help identify those shortcomings, misunderstandings, and other gaps, thus contributing to improvements in the standard’s quality and value. In the process, testing/certification can accelerate the adoption of new technologies by helping to validate interoperability. In the case of the synchrophasor testing performed by NIST, for example, the lessons learned from the testing subsequently led IEEE SA to publish a revision in 2014 to address some ambiguities in the standard and to adjust some performance limits.

How IEEE Conformity Assessment Programs Work

Numerous vendors, consultants, and other companies worldwide participate in IEEE Conformity Assessment Programs (ICAP),⁴ which use a steering committee format to holistically develop conformity assessment programs around specific standards. ICAP helps IEEE working groups navigate through the conformity assessment ecosystem, which may encompass conformance testing, commissioning, interoperability, inspection, and laboratory recognition, as well as the development of test suite specifications or plans.

ICAP partners with expert test labs around the world to provide the right level of testing and field evaluation support. One example is the University of New Hampshire InterOperability Lab (UNH-IOL),⁵ whose capabilities include testing to the IEEE 1588 ^TM Precision Time Protocol for Power Applications. Consumers Energy Laboratory Services⁶ became the first lab recognized by ICAP to perform testing of the phasor measurement units based on the IEEE C37.118 standard. ICAP worked collaboratively with NIST and other key players to ensure the right test tools and methodologies were utilized in the testing process.

Figure 1 illustrates how the conformity assessment process works, using synchrophasor PMUs as an example.

ICAP provides the marketplace with several benefits:

• Manufacturers get a proven method of demonstrating compliance with requirements.
• A trustworthy public registry of certified products gives enterprises and other end-users a convenient, credible way to verify which products have successfully completed conformity testing and certification, including addressing software/firmware modifications and product changes.
• As an independent resource, ICAP empowers end-users to reduce risk by making more informed purchasing decisions. For example, the IEEE certification mark gives them peace of mind that their new products will be more likely to interoperate with other certified products.
• New technologies establish themselves in the market with stable and robust products. This also helps the market grow quickly because customers don’t postpone purchases for fear of being de facto beta testers.
• Manufacturers, systems integrators, and the rest of a technology’s ecosystem have a shared resource for testing and certification. That cooperation significantly reduces their individual costs and upkeep compared to doing testing entirely on their own.

ICAP Portfolio Spans Multiple Industries

In addition to synchrophasors, the ICAP portfolio⁷ spans a wide variety of verticals and technologies, including drones/UAVs, time-sensitive networking for industrial automation, cybersecurity, blockchain, federated learning, autonomous vehicles, medical devices, sensors, and the Internet of Things:

Smartphone Cameras

IEEE 1858 ^{TM 8} provides image-quality standards for mobile camera videos and images, including those used in smartphones. Participants such as mobile operators, operating system vendors, handset manufacturers, chipset vendors, software providers, and test labs use these standards to enable “apples-to-apples” comparisons between different products. This overcomes the challenges that occur when everyone uses their own methodology for spatial frequency response, chroma level, color uniformity, texture blur, and other key metrics.

Electric Vehicles

Over 3.7 million electric vehicles (EVs) were sold worldwide over the past two years. That’s an impressive number, considering that consumers and businesses are concerned about running out of power before they can get to a charging station, how long charging takes, and whether a public charging station will be compatible with their EV model. Meanwhile, electric utilities are concerned about the grid’s ability to support the exponential increase in EVs each year.

To help the EV ecosystem address these and other market-limiting concerns, the IEEE 2030.1.1 ^TM DC Quick Charging Test Suite Specification provides a wide variety of test cases, which each participating lab uses to ensure that its methodology is consistent. For example, one test case checks whether a charger meets the standard’s voltage resistance properties. Another assesses whether a charging panel designed for indoor use has a minimum rating of IP 41, which protects against water intrusion.

IEEE is currently revising the IEEE 2030.1.1 standard to include bi-directional charging and ultra-rapid charging up to 400 kW. The existing certification program will also be updated to ensure testability and certification of those chargers. These super-fast chargers are key for addressing EV buyers’ concerns about being able to charge quickly. Fast charging also enables each station to support more EVs because they’re in and out in less time.

Precision Timing for Energy Infrastructure

Electrical utilities and other members of the energy industry rely on the IEEE 1588 Precision Time Protocol (PTP) standard to ensure that their infrastructure is tightly synchronized down to the sub-microsecond range. ICAP worked with NIST and other industry stakeholders to create a test suite specification (TSS) that provides a common approach to verifying a clock’s performance with the requirements of IEEE 1588.

The IEEE 1588 Power Profile Certification program was critical for addressing challenges that many utilities encountered. For example, one said, “We crudely simulated a time spoofing incident and found devices did not follow the time when step changes were introduced.” According to another, “We’ve seen devices update the timestamp/correction-field in the incorrect location.”

Figure 2 summarizes the key benefits that ICAP provides for vendors and end-users.

How to Get Involved

Beyond conformity assessment activities, there is a benefit for vendors, end-users, and others in participating in IEEE working groups so they can be involved in the standards-development process. This is an opportunity to influence new standards and get a valuable insider’s perspective into the future. IEEE working groups also provide a convenient, invaluable opportunity to network with other industry leaders. Even if a vendor or end-user doesn’t have the time or the resources to participate in working groups, another option is to share their expertise about what an ICAP program should assess for new standards. Some vendors, end-users, test laboratories, and others choose to participate in both working groups and ICAP programs.

The IEEE recently initiated the development of a credentialing program for the IEEE 1547 Standard for Interconnecting Distributed Resources with Electric Power Systems, which is key for supporting and enabling the rapidly growing use of renewable energy sources such as wind, solar, and energy storage. In fact, some states have adopted it as a standard in their statewide interconnection regulations.

When launched, the new credentialing program is expected to address the impending need for a trained and knowledgeable workforce, who may be part of engineering and consulting firms, electrical contractors, and other implementers of IEEE 1547. The expectation is that this new program will help expand the use of renewable energy worldwide.

Conclusion

Vendors and everyone else in a particular ecosystem – whether it’s wireless technology, drones, or EVs – benefits when conformity-assessment work is done in parallel with standards development because both can be available simultaneously. Otherwise, implementation of a new technology could be delayed until after the conformity-assessment work is done. That delay doesn’t benefit vendors that are eager to provide customers with cutting-edge technology solutions.

But by working together, vendors can implement their new, standards-based technologies successfully. That’s a future we all can look forward to.

Endnotes

1. P. Hoffman, “How Synchrophasors are Bringing the Grid into the 21st Century,” April 16, 2014, https://www.energy.gov/articles/how-synchrophasors-are-bringing-grid-21st-century
2. A. Goldstein, “2014 NIST Assessment of Phasor Measurement Unit Performance,” February 2016, https://nvlpubs.nist.gov/nistpubs/ir/2016/NIST.IR.8106.pdf
3. IEEE Standard for Synchrophasor Measurements for Power Systems, C37.118.1-2011, December 28, 2011, https://standards.ieee.org/standard/C37_118-2005.html
4. https://standards.ieee.org/products-services/icap/index.html
5. https://www.iol.unh.edu
6. https://www.consumersenergy.com/business/products-and-services/lab-services
7. https://standards.ieee.org/products-services/icap/index.html
8. IEEE Standard for Camera Phone Image Quality, 1858-2016, May 5, 2017, https://standards.ieee.org/standard/1858-2016.html