Ieee Pes 2013 Smart Grid Compendium

Smart Grid:
Challenges & Opportunities
You dont have to be Thomas Edison to be a force for change. Just a PES member.
Joining the IEEE Power & Energy Society can provide a big boost to your career by enabling you to:

Tackle broad-reaching challenges Become recognized as a thought leader by your industry peers Develop contacts that will prove useful throughout all stages of your career Be a part of the very active and engaged global PES Community
We help our members to be successful by providing: Up-to-date information on current trends and the latest technology Industry insight through Power & Energy magazine, technical reports and peer-reviewed publications Compelling programs and networking opportunities at our conferences and events Opportunity to meet, network and collaborate with local members via our vibrant chapters
Over 30,000 members of the IEEE Power & Energy Society recognize that their membership is an exceptional, cost-effective way to acquire the latest information about all aspects of the fast-changing electric power and energy industry. You can too, if you join us now!
To learn more about the IEEE Power & Energy Society, including the many other membership benefits, please visit
www.ieee-pes.org.
IEEE Power & Energy Society
445 Hoes Lane
Piscataway, NJ 08854 USA
magazine
www.ieee.org/power
Smart Grid: Challenges & Opportunities

...a 2013 reprint journal from PES
on the cover
features
Challenges and Opportunities in Power System Security, Resiliency, and Privacy By S. Massoud Amin and Anthony M. Giacomonis
13 A Virtual Smart Grid

Real-Time Simulation for Smart Grid Control and Communications Design By David Anderson, Chuanlin Zhao, Carl H. Hauser, Vaithianathan Venkatasubramanian, David E. Bakken, and Anjan Bose
22 Forward Pass
Policy Changes and Technical Opportunties on the U.S. Electric Grid By Timothy D. Heidel, John G. Kassakian, and Richard Schmalensee
13
30 DC, Come Home

DC Microgrids and the Birth of the Enernet By Brian T. Patterson
41 Enhancing Grid Measurements

Wide Area Measurement Systems, NASPInet, and Security By Rakesh B. Bobba, Jeff Dagle, Erich Heine, Himanshu Khurana, William H. Sanders, Peter Sauer, and Tim Yardley
49 Staying in Control
Cybersecurity and the Modern Electric Grid By Julie Hull, Himanshu Khurana, Tom Markham, and Kevin Staggs
49
ieee power & energy magazine
contents
1
5 Smart Grid Safe, Secure, Self-Healing
magazine
Editor in Chief
Melvin I. Olken 245 East 19th Street #20K New York, NY 10003-2665 USA +1 212 982 8286 (phone fax) m.olken@ieee.org
IEEE Periodicals/Magazines Department

445 Hoes Lane, Piscataway, NJ 08854 USA +1 732 562 3950, fax +1 732 981 1855 www.ieee.org/magazines Geraldine Krolin-Taylor, Senior Managing Editor Janet Dudar, Senior Art Director Gail A. Schnitzer, Assistant Art Director Theresa L. Smith, Production Coordinator Peter M. Tuohy, Production Director Felicia Spagnoli, Advertising Production Manager Dawn Melley, Editorial Director Fran Zappulla, Staff Director, IEEE Publishing Operations
IEEE prohibits discrimination, harassment, and bullying. For more information, visit http://www.ieee.org/web/aboutus/whatis/policies/p9-26.html.
Associate Editors
Gerald B. Shebl, Business Scene Carl L. Sulzberger, History
Editorial Board
S. Massoud Amin, L. Goel, A.P. Hanson, N. Hatziargyriou, M.I. Henderson, S.H. Horowitz, P. Kundur, R. Masiello, K.M. Matsuda, A.P.S. Meliopoulos, M.I. Olken, M. OMalley, A.G. Phadke, R.J. Piwko, C.E. Root, H. Rudnick, P.W. Sauer, M. Shahidehpour, B.R. Shperling, S.S. Venkata, B.F. Wollenberg
IEEE Power & Energy Magazine

IEEE Power & Energy Magazine (ISSN 1540-7977) (IPEMCF) is published bimonthly by the Institute of Electrical and Electronics E ngineers, Inc. Headquarters: 3 Park Avenue, 17th Floor, New York, NY 10016-5997 USA. Responsibility forthe contents rests upon the authors and not upon the IEEE, the Society, or its members. IEEE Operations Center (for orders, subscriptions, address changes): 445 Hoes Lane, Piscataway, NJ 08854 USA. Telephone: +1 732 981 0060, +1 800 678 4333. Individual copies: IEEE members US$20.00 (first copy only), nonmembers US$77.00 per copy. Subscription Rates: Society members included with membership dues. Subscription rates available upon request. Copyright and reprint permissions: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyond the limits of U.S. Copyright law for the private use of patrons 1) those post-1977 articles that carry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 USA; 2) pre-1978 articles without fee. For other copying, reprint, or republication permission, write Copyrights and Permissions Department, IEEE Operations Center, 445 Hoes Lane, Piscataway, NJ 08854 USA. Copyright 2013 by the Institute of Electrical and Electronics Engineers, Inc. All rights reserved. Periodicals postage paid at New York, NY, and at additional mailing offices. Postmaster: Send address changes to IEEE Power & Energy Magazine, IEEE Operations Center, 445 Hoes Lane, Piscataway, NJ 08854 USA. Canadian GST #125634188
Advertising
Barry LeCerf Bullseye International Group, Inc. +1 913 663 1112, fax +1 913 663 1119 blecerf@bullseyeinternationalgroup.com
ieee power & energy society (pes)
Printed in U.S.A.
The IEEE Power & Energy Society is an organization of IEEE members whose principal interest is the advancement of the science and practice of electric power generation, transmission, distribution, and utilization. All members of the IEEE are eligible for membership in the Society. Mission Statement: To be the leading provider of scientific and engineering information on electric power and energy for the betterment of society, and the preferred professional development source for our members.
Officers
N.N. Schulz, President M.M. Begovic, President-Elect M. Selak, Vice President, Chapters J.H. Nelson, Vice President, Technical Activities P.W. Sauer, Vice President, Education S. Rahman, Vice President, Publications W. Rosehart, Vice President, Meetings H. Louie, Vice President, Membership & Image R. Podmore, Vice President, New Initiatives/ Outreach L. Bertling Tjernberg, Treasurer C. Root, Secretary A.C. Rotz, Past-President
Standing Committee Chairs
M. Crow, Awards & Recognition N. Nair, Constitution & Bylaws L. Bertling Tjernberg, Finance & Audit A.C. Rotz, Nominations & Appointments K. Butler-Purry, Power Engineering Education W.K. Reder, Scholarship Plus
Chapter Representatives
IEEE Division VII Director

C. Warren
B. Allaf, J. Ammentorp, A. Bakirtzis, J.G. Calderon, R. Cespiedes, C. Diamond, D. Drumtra, J. Fleeman, B. Gwyn, Z.F. Hussien, I. Kuzle, N. Logic, J.C. Montero, P. Naidoo, T. Rajagopalan, P. Pabst, D.Sharafi, G.N. Taranto, E. Tobin, E. Uzunovic, D. van Hertem
Chapter Committee Chairs
IEEE Division VII Director Elect

W.K. Reder
Region Representatives
M. Chaganti, Y. Chen, T. Hiemer, F. Lambert, M. Nissen, J. Skillman, United States M. Armstrong, Canada C. Vournas, Europe, Middle East, & Africa N. Segoshi, Latin America L. Goel, Asia & Pacific
S. Chakravorti, Chapter Secretary N. Mariun, Chapter/Section Relations M. Armstrong, Electronic Communications E. Carlsen, Awards & Resources Y. Chen, Distinguished Lecturer Program K. Hadzimahovic, Chapters Web site
Technical Committee Chairs M. Sedlak, Electric Machinery R. Groves, Energy Development & Power Generation J. Smith, Insulated Conductors G. Ballassi, Nuclear Power Engineering S. Carneiro, Jr., Power System Analysis, Computing, & Economics D. Nordell, Power System Communications T. Van Cutsem, Power System Dynamic Performance R. Arseneau, Power System Instrumentation & Measurements A. Conejo, Power System Operations M.L. Chan, Power System Planning & Implementation R. Hedding, Power System Relaying L. Varga, Stationary Battery M. Dood, Substations A.J. Surtees, Surge Protective Devices T.W. Olsen, Switchgear W. Chiu, Transformers W.A. Chisholm, Transmission & Distribution Coordinating Committees B. Djokic, Emerging Technologies S. Pullins, Intelligent Grid P. Bishop, Marine Systems R.J. Piwko, Wind & Solar Power Standing Committees J. Randolph, Awards S.S. Venkata, Technical Sessions S.S. Venkata, Meetings & Marketing K. Edwards, Organization & Procedures W. Bartley, Standards Coordination
Membership & Image Committee Chairs
Governing Board Members-at-Large
E. Gunther, M. Jensen, J. Giri, T. Prevost
A. St Leger, GOLD Coordinator A. Bonthron, Membership Development L. Fan, Membership Development Open, Web Site Development S. Bahramirad, PES WIE Liaison W. Bishop, Marketing
PES Executive Director
Patrick Ryan, +1 732 465 6618, fax +1 732 562 3881, e-mail p.ryan@ieee.org
Digital Object Identifier 10.1109/MPE.2013.2239531
Technical Council
J.H. Nelson, Chair S.S. Venkata, Vice-Chair K.S. Edwards, Secretary D. Novosel, Past-Chair M. Maytum, Web Master
Additional Positions
D. Nordell, Editor in Chief of Conference Papers
leaders corner
Noel Schulz, President, IEEE PES
greetings from the IEEE Power & Energy Society (PES)

March 15, 2013 To: Recipients of the 2013 IEEE PES Smart Grid Reprint Journal: Patricia Hoffman, assistant secretary at the U.S. DOE in the office of Electricity Delivery and Energy Reliability, delivered a timely keynote on Grid Modernization and Resiliency, which articulated a theme that underlies much of our work and is reflected by the collection of articles you hold in your hands. No nation is without power-related challenges in supporting its economic vitality and security and our international membership guarantees that we maintain a broad perspective on these issues. Together were working toward solutions and sharing our findings across the world. By solving our shared electricityrelated challenges and sharing the fruits of our work that we can advance grid modernization for every member of the international community. That brings me to the content of this new compendium. Youll glean the details from the Table of Contents, but weve packaged here articles on the array of issues that remain relevant year after year. We bring you up-to-date explorations of: the fundamental drivers, challenges and probable solutions of and for smart, modernized grids, federal and state regulatory issues that bear scrutiny and need carefully considered changes, the rebirth of interest in DC power and how its efficiencies and other characteristics will make it an exciting area for innovative solutions, cost-effective, software-based sim ulation models that save the time and cost of laboratory or real-world trial and error in advancing power systems,
cyber-attack vectors and vulner-
WELCOME TO OUR NEW COMPENdium of articles that appeared in last years IEEE P&E Magazine. If the past is prologue to the future, youll find an array of current issues treated in depth here in this useful digest. In fact, looking over past reprint issues, its clear that a handful of fundamental themes continue to challenge the IEEE PES as well as the power industry at large. Yet as our thinking advances, were seeing these issues more clearly, discovering fresh approaches and our collective work at meeting these challenges continues. I saw many colleagues at the February 2013 IEEE PES Innovative Smart Grid Technologies (ISGT) North America conference in Washington, D.C. and I look forward to seeing many more of you at our upcoming PES meetings across the globe including our ISGT conference suite of meetings in Sao Paulo, April 15-17, and Copenhagen, Oct. 6-9. ISGT conference programs reflect many of the issues presented here and will undoubtedly continue to drive discussion, research and efforts to reach solutions in 2013. Id like to remind all of our members that these conferences offer outstanding networking opportunities to engage with your colleagues and broaden your horizons. The ISGT conference in Washington, DC involved participants from over 30 countries. I had the opportunity to recap PES-specific priorities, including our progress on standards, and our challenge in persuading students to join not only the power field but our Society as well. I also announced a new PES publication, Electrification.

abilities and the value of simulation for developing solutions, the fundamental value of obtaining accurate data on the grids operational state, made possible by wide area measurement systems, or WAMS, and more on cyber security and communication networks. Of course, applying our knowledge, pursuing discoveries and sharing insights is what makes IEEE PES a world-class hub for innovation. So read this digest, share it with colleagues, explain the ideas to family and friends (its a good practice, theyre energy consumers, too). Share your work by writing an article for IEEE P&E Magazine, visit the IEEE Smart Grid Portal (http://smart-grid.ieee.org/), keep up with our IEEE PES and smart grid tweets (@ieee_pes and @ieeesmartgrid) and join IEEE Smart Grid LinkedIn discussion group. The sustainable, efficient provision and wise use of electricity can pave the way for more productive and sustainable societies. I invite you to partake of this volumes thoughtful articles and advance the search for solutions. Sincerely,
Noel N. Schulz IEEE PES President, 2012-2013 n.schulz@ieee.org http://www.ieee-pes.org/

IEEE Transactions on Smart Grid
The IEEE Transactions on Smart Grid is intended to be a cross disciplinary and internationally archival journal aimed at disseminating the results of research on smart grid that relates to energy generation, transmission, distribution and delivery. The journal will publish original research on theories, technologies, design, policies, and implementation of smart grid. The Transactions will welcome manuscripts on design, implementation and evaluation of energy systems that include smart grid technologies and applications. Surveys of existing work on smart grid may also be considered for publication when they propose a challenging perspective on the future of such technologies and systems. Topical issues considered by the Transactions include: > Smart sensing, communication and control in energy systems > Wireless communications and advanced metering infrastructure > Smart grid for energy management in buildings and home automation > Phasor measurement unit applications for smart grid > Smart grid for plug-in vehicles and low-carbon transportation alternatives > Smart grid for cyber and physical security systems > Smart grid for distributed energy resources > Smart grid for energy savings and nancial management > Smart grid in interdependent energy infrastructures > Smart grid for intelligent monitoring and outage management If you are interested in reviewing papers for this journal, please sign up as a reviewer on the Manuscript Central site at: http://mc.manuscriptcentral.com/pes-ieee. The Transactions on the Smart Grid can be accessed via the drop down menu on the PES portal site. If you are interested in reviewing papers for our new Transactions and you are currently a reviewer for PES Transactions , you can access your account in Manuscript Central and add smart grid to your keywords or areas of expertise. If you have an account in Manuscript Central and are not currently a reviewer for PES Transactions and would like to become a reviewer for PES Transactions , access your account and you will automatically be given a reviewer center, then update your areas of expertise. If you do not have an account, create a new user account and complete all the required elds, you will then be given an author center and a reviewer center. About the Editor-in-Chief: If you are interested in participating in the publication activities, please contact the Editor-in-Chief, Dr. Mohammad Shahidehpour at: m.shahidehpour@ieee.org. Prof. Shahidehpour (Fellow 01) has been af liated with IEEE for the last thirty years. His is currently the Carl Bodine Distinguished Professor of Electrical and Computer Engineering at Illinois Institute of Technology. Dr. Shahidehpour is an IEEE Distinguished Lecturer who has lectured in 30 countries on issues related to power system operation and control. He has served as the Vice President of Publications for the IEEE Power & Energy Society and an Editor of the Transactions on Power Systems.
Reprinted from January/February 2012 issue of IEEE Power & Energy magazine
Smart Grid Safe, Secure, Self-Healing

Challenges and Opportunities in Power System Security, Resiliency, and Privacy
THE EXISTING POWER DELIVERY system is vulnerable to both natural disasters and intentional attack. A successful terrorist attempt to disrupt the power delivery system could have adverse effects on national security, the economy, and the lives of every citizen. Secure and reliable operation of the electric system is fundamental to national and international economic systems, security, and quality of life. This is not new: both the importance and the difculty of protecting power systems have long been recognized. In 1990, the U.S. Ofce of Technology Assessment (OTA) issued a detailed report, Physical Vulnerability of the Electric System to Natural Disasters and Sabotage. The report concluded: Terrorists could emulate acts of sabotage in several other countries and destroy critical [power system] components, incapacitating large
By S. Massoud Amin and Anthony M. Giacomoni

Digital Object Identier 10.1109/MPE.2011.943112 Date of publication: 13 December 2011
january/february 2012
1540-7977/11/$31.002012 IEEE 1540-7977/12/$31.002012
IEEE power power & & energy energy magazine magazine ieee
BRAND X PICTURES
33 5
segments of a transmission network for months. Some of these components are vulnerable to saboteurs with explosives or just high-powered ries. The report also documented the potential costs of widespread outages, estimating them to be in the range of US$1 to US$5 per kWh of disrupted service, depending on the length of the outage, the types of customers affected, and a variety of other factors. In the New York City blackout of 1977, for example, damage from looting and arson alone totaled about US$155 millionroughly half of its total cost. During the 20 years since the OTA report, the situation has become even more complex. Accounting for all critical assets includes thousands of transformers, line reactors, series capacitors, and transmission lines. Protecting all these diverse and widely dispersed assets is impractical. Moreover, cyber, communication, and control layers add new benets only if they are designed correctly and securely.
Electricity Infrastructure: Increasing Interdependencies

Energy, telecommunications, transportation, and nancial infrastructures are becoming increasingly interconnected, thus posing new challenges for their secure, reliable, and efcient operation. All of these infrastructures are complex networksgeographically dispersed, nonlinear, and interacting both among themselves and with their human owners, operators, and users (see Figure 1). Virtually every crucial economic and social function depends on the secure and reliable operation of these infrastructures. Indeed, they have provided much of the high standard of living that the more developed countries enjoy. With increased benet, however, has come increased risk. As these infrastructures have grown more complex in order to handle increasing demands, they have become increasingly interdependent. The Internet, computer networks, and our digital economy have all increased the demand for reliable and disturbance-free electricity; banking and nance depend on the robustness of electric power, cable, and wire-
less telecommunications infrastructure. Transportation systems, including military and commercial aircraft and land and sea vessels, depend on communication and energy networks. Links between the power grid and telecommunications systems as well as between electrical power lines and oil, water, and gas pipelines continue to be the lynchpins of energy supply networks. This strong interdependence means that an action in one part of an infrastructure network can rapidly create global effects by cascading throughout the same network and even into other networks. In the aftermath of the tragic events of 11 September 2001 and recent natural disasters and major power outages, there have been increased national and international concerns expressed about the security, resilience, and robustness of critical infrastructures in response to an evolving spectrum of threats. There is reasonable concern that national and international energy and information infrastructures have reached a level of complexity and interconnection that makes them particularly vulnerable to cascading outages, whether initiated by material failure, natural calamities, intentional attack, or human error. The potential ramications of network failures have never been greater, as the transportation, telecommunications, oil and gas, banking and nance, and other infrastructures depend on the continental power grid to energize and control their operations. Despite some similarities, the electric power grid is quite different from gas, oil, and water networks: phase shifters rather than valves are used, and there is no way to store signicant amounts of electricity. Providing the desired ow on one line often results in loop ows on several other lines.
Potential Route Ahead: A Smarter Grid

The key challenge is to enable secure and very high-condence sensing, communications, and control of a heterogeneous, widely dispersed, yet globally interconnected system. It is even more complex and difcult to control it for optimal efciency and maximum benet to the ultimate consumers while still allowing all its business components to compete fairly and freely. To achieve this goal, a new megainfrastructure is emerging from the convergence of energy, telecommunications, transportation, the Internet, and electronic commerce. In the electric power industry and other critical infrastructures, new ways are being sought to improve network efciency by eliminating congestion problems without seriously diminishing reliability and security. Nevertheless, the goal of transforming the current infrastructures into self-healing energy delivery, computer, and communications networks with unprecedented robustness, reliability, efciency, and quality for customers and our society is ambitious. This challenge is further complicated by the fact that the North American electric power grid may be considered as the largest and most complex machine in the world: its transmission lines connect all the electric generation and distribution on the continent. This network represents an
Excellent Power System Reliability A Secure Energy Infrastructure Exceptional Power Quality Integrated Communications Compatible Devices and Appliances
figure 1. A complex set of interconnected webs (source: EPRI, 2002present).

34 6
IEEE ieee power & energy magazine magazine
enormous investment, including more than 15,000 generators in 10,000 power plants and hundreds of thousands of miles of transmission and distribution lines. With diminished transmission and generation capacity and with dramatic increases in interregional bulk power transfers and the diversity of transactions, the electric power grid is being used in ways for which it was not originally designed. Grid congestion and atypical power ows have been increasing during the last 25 years, while customer expectations of reliability and cyber and physical security are rising to meet the needs of a pervasively digital world. Upgrading the control and communication systems for the power grid will present many new security challenges that must be dealt with before extensive deployment and implementation of smart grid technologies can begin. The digitization of such systems may enable remote attacks to grow rapidly, potentially spanning countries or even continents. Moreover, the number of threats against computer systems is rapidly increasing due to the increased availability of highly sophisticated hacker tools on the Internet and the decrease in technical knowledge required to use them to cause damage. While the digitization of such systems will present many new security challenges, it will also provide the grid with increased exibility to prevent and withstand potential threats.
extremely difcult, even for a large, well-organized group of terrorists. Data on terrorist attacks on the worlds electricity sector from 19942004 from the Oklahoma-based Memorial Institute for the Prevention of Terrorism show that transmission systems are by far the most common target in terms of the total number of physical attacks. Figure 2 shows the percentage of terrorist attacks aimed at each of the major grid components. One possible means of increasing the physical security of power lines is to bury them. A 2006 study by the Edison Electric Institute (EEI) calculated that putting power lines underground would cost about US$1 million per mile, compared with US$100,000 per mile for overhead lines, making the idea nancially infeasible.
Cyber Challenges
The number of documented cyberattacks and intrusions worldwide has been rising very rapidly in recent years. The results of a 2007 McAfee survey highlight the pervasiveness of such attacks. For example, Figure 3 shows the percentage of IT and security executives from critical infrastructure enterprises located in 14 countries around the world reporting large-scale distributed denial-of-service (DDoS) attacks and their frequency. DDoS attacks utilize networks of infected computers whose owners often do not even know that they have been infectedto overwhelm target networks with millions of fake requests for information over the Internet. Due to the increasingly sophisticated nature and speed of malicious code, intrusions, and DoS attacks, human responses may be inadequate. Figure 4 shows the evolution of cyberthreats over the last two decades and the types of responses that can be used to combat them effectively. In addition, adversaries often have the potential to initiate attacks from nearly any location in the world. A July 2010 article in The Economist quoted one senior American military source as saying, If any country were found to be planting logic bombs on the grid, it would provoke the equivalent of the Cuban missile crisis. Furthermore, currently
Key Smart Grid Security Challenges

Physical Challenges
The size and complexity of the North American electric power grid makes it impossible both nancially and logistically to physically protect the entire infrastructure. There currently exist more than 450,000 mi of 100-kV or higher transmission lines and many more thousands of miles of lower-voltage lines. As an increasing amount of electricity is generated from distributed renewable sources, the problem will only be exacerbated; the U.S. Department of Energy (DOE) has concluded that generating 20% of all electricity with land-based wind installations will require at least 20,000 square miles. Thus it is probable that a well-organized, determined group of terrorists could take out portions of the grid as they have previously done in the United States, Colombia, and other locations around the globe. Several such incidents in the United States have been publicly reported during the last 30 years, including saboteurs operating in the Pacic Northwest and those using power lines and transformers for target practice on the East Coast. Colombia, for example, has faced up to 200 terrorist attacks per year on its transmission infrastructure over the last 11 years, as reported in a recent IEEE Power & Energy Magazine article by Corredor and Ruiz. Such attacks, although troublesome and costly to the local region, affect only a small portion of the overall grid, however. To cause physical damage equivalent to that from a small to moderate-size tornado would be
13% 14% 11% 62%
Generation Substations
Transmission All Others
figure 2. Electric terrorism: grid component targets, 19942004 (source: Journal of Energy Security).
35 7
100 80 Percentage 60 40 20 Multiple Occurrences Every Day Multiple Occurrences Every Week Multiple Occurrences Every Month Less Than Monthly Occurrences Less Than Annual Occurrences
France
Germany
United States
Brazil
Spain
Australia
China
Japan
United Kingdom
Mexico
Russia
figure 3. Percentage of critical infrastructure enterprise executives reporting large-scale DDoS attacks and their frequency (source: McAfee).
more than 90% of successful cyberattacks take advantage of known vulnerabilities and miscongured operating systems, servers, and network devices. The security of cyber and communication networks is fundamental to the reliable operation of the grid. As power systems rely more heavily on computerized communications and control, system security has become increasingly dependent on protecting the integrity of the associated information systems. Part of the problem is that the existing control systems, which were originally designed for use with proprietary, stand-alone communication networks, were later connected to the Internet (because of its productivity advantages
and lower costs) but without adding the technology needed to make them secure. Moreover, numerous types of communication media and protocols are used in the communication and control of power systems. Within a substation control network, it is common to nd commercial telephone lines as well as wireless, microwave, optical ber, and Internet connections. The diversity and lack of interoperability among the various communication protocols cause problems for anyone who tries to establish secure communication to and from a substation. Electric power utilities also typically own and operate at least certain portions of their own telecommunications
Seconds
Class III Human Response: Impossible Automated Response: Unlikely Proactive Blocking: Possible Class II Human Response: Difficult/Impossible Automated Response: Possible Blended Threats Class I Human Response: Possible Warhol Threats
Flash Threats
Contagion Time Frame
Days Macro Viruses File Viruses
E-Mail Threats
Months
Early 1990s
Mid 1990s
Late 1990s
2000
2003
Saudi Arabia/ Middle East
Time
figure 4. Cyberthreat evolution (source: EPRI).

36 8
Total
India
Italy
systems, which often consist of a backbone of ber optic or microwave links connecting major substations with spurs to smaller sites. Increased use of electronic automation raises signicant issues regarding the adequacy of operational security, if security provisions are not built in. More specically, the operation of a modern power system depends on complex systems of sensors and automated and manual controls, all of which are tied together through communication systems. While the direct physical destruction of generators, substations, or power lines may be the most obvious strategy for causing blackouts, activities that compromise the operation of sensors, communications, and control systems by spoong, jamming, or sending improper commands could also disrupt the system, cause blackouts, and in some cases result in physical damage to key system components. Any telecommunication link that is even partially outside the control of the organization that owns and operates power plants, supervisory control and data acquisition (SCADA) systems, or energy management systems (EMSs) represents a potentially insecure pathway into the business operations of the company as well as a threat to the grid itself. The interdependency analyses done by most companies in the last 1214 years (starting with the preparations for Y2K and continuing after the tragic events of 9/11) have identied these links and the systems vulnerability to their failure. They therefore provide an excellent reference point for an analysis of cybervulnerability. While some of the operations on the system are automatic, human operators in system control centers ultimately make the decisions and take the actions that control the operations of the system. In addition to the physical threats to such centers and the communication links that ow in and out of them, one must be concerned about two other factors: the reliability of the operators within the centers and the possibility that insecure code has been added to a program in a center computer. The threats posed by insiders are real, as is the risk of a Trojan horse embedded in the software of one of more of the control centers. A 2008 survey by the Computer Security Institute and the U.S. Federal Bureau of Investigation of data compiled from 522 computer security practitioners and senior executives of U.S. corporations, government agencies, nancial and medical institutions, and universities reported that within a 12-month period, 59% of the respondents experienced an attack from a virus, 29% reported unauthorized use of computer services, and 44% reported insider abuse. The threat of a Trojan horse embedded in the control center software can only be addressed by means of careful security measures within the commercial rms that develop and supply this software along with careful security screening of the utility and outside service personnel who perform software maintenance within the centers. Today, security patches often are not supplied to end users, or users are not applying the patches, as they fear they will affect system performance. Current practice is to apply an upgrade or
patch only after SCADA vendors thoroughly test and validate it, and this sometimes causes deployment to be delayed by several months. As a result, cybersecurity is just as important as physical security, if not more so. Due to the gravity of these threats, the Federal Energy Regulatory Commission (FERC) policy statement on the smart grid states that cybersecurity is essential to the operation of the smart grid and that the development of cybersecurity standards is a key priority. The DOE has also stated that the ability to resist attack by identifying and responding to disruptions caused by sabotage is one of the smart grids seven crucial functions. Much work remains to be done, however, to create standards that, when implemented, will adequately protect the grid from cyberattacks. Emerging standards fall well short of achieving this ultimate goal.
Smart Grid Security Needs

Layered Security
In order to protect electric infrastructure from the threats outlined above, several layers of security are needed to minimize disruptions to system operations. Layered security (or defense in depth) involves strategically combining multiple security technologies at each layer of a computing system in order to reduce the risk of unauthorized access due to the failure of any single security technology. It exponentially increases the cost and difculty of compromising a system by creating a much stronger defense than the use of any individual component alone, thus reducing the likelihood of an attack. The trend of connecting electrical control systems to the Internet exposes all layers of a system to possible attack. Computing layers that must be considered include personnel networks operating systems applications databases. The security features to be employed at each layer include examination, detection, prevention, and encryption. To protect control systems, well-established information security practices must also be utilized.
Deception
An additional defense mechanism is the use of deception. Deception consists of two possible techniques: dissimulation (hiding the real) and simulation (showing the false). McQueen and Boyer describe several potential dissimulation and simulation techniques that can be used for control systems. Three of the dissimulation techniques described are: masking the real by making a relevant object undetectable or blending it into background irrelevance repackaging, which hides the real by making a relevant object appear to be something it isnt
IEEE magazine ieee power & energy magazine
37 9
Upgrading the control and communication systems for the power grid will present many new security challenges that must be dealt with.
dazzling, which hides the real by making the identi disrupting the load balance of local systems by sud-
cation of a relevant object less certain by confusing the adversary about its true nature. Likewise, three of the simulation techniques described are: inventing the false by creating a perception that a relevant object exists when it doesnt mimicking, which invents the false by presenting characteristics of an actual and relevant object decoying, which displays the false so as to attract attention away from a more relevant object. Deception will need to play a key role in smart grid defense mechanisms. Since existing control system architectures are not random and therefore response characteristics are reproducible, the strength of potential adversaries is amplied. Defense mechanisms using deception can greatly increase the difculty of planning and conducting successful attacks on a system by portraying control system response characteristics as random to attackers. They can also alert operators to possible threats before any systems are harmed. Additional security needs include rapid containment, restoration, and recovery strategies for times when systems are inevitably compromised. Either software patching or the ability to rapidly identify and isolate the exploited systems must be enabled in order to minimize downtime. This is extremely important, since the consequences of an attack are directly proportional to the length of time the service is disrupted.
denly increasing or decreasing the demand for power

gaining control of millions of meters and simultane-
ously shutting them down

sending false control signals disabling grid control center computer systems and
monitors
disabling protective relays.
Advanced Metering Infrastructure

Vulnerabilities
The implementation of advanced metering infrastructure (AMI) is widely seen as one of the rst steps in the digitization of the electric grids control systems. Despite the increase in the utilization of AMI, there has been very little assessment or R&D effort to identify the security needs for such systems. Smart meters, however, are extremely attractive targets for exploitation, since vulnerabilities can be easily monetized through manipulated energy costs and measurement readings. Currently, in the United States alone it is estimated that US$6 billion is lost by electricity providers to consumer fraud in the electric grid. Possible threats to the electrical grid introduced by the use of AMI include: fabricating generated energy meter readings manipulating energy costs
38 10
As more utilities move toward using Internet Protocol (IP)based systems for wide area communications and as the trend of using standardized protocols continues throughout the industry, maintaining the security of such devices will be critical. AMI introduces serious privacy concerns, as immense amounts of energy use information will be stored at the meter. Breaches into this data could expose customer habits and behaviors. Such arguments have led to the recent moratoriums on AMI installations in numerous northern California communities and other areas throughout the country. As a result, several key privacy concerns need to be addressed, including those outlined by the Cyber Security Working Group of the U.S. National Institute of Standards and Technology (NIST). These include: Personal proling: using personal energy data to determine consumer energy behavioral patterns for commercial purposes Real-time remote surveillance: using live energy data to determine whether people are in a specic facility or residence and what they are doing Identity theft and home invasions: protecting personal energy data from criminals who could use the information to harm consumers Activity censorship: preventing the use of energy for certain activities or taxing those activities at a higher rate Decisions based on inaccurate data: shutting off power to life-sustaining electrical devices or providing inaccurate information to government and creditreporting agencies. In addition, AMI systems will need to be defended against more traditional cyberthreats such as mobile and malicious code, DoS attacks, misuse and malicious insider threats, accidental faults introduced by human error, and the problems associated with software and hardware aging.
Security Needs
In order to defend against the vulnerabilities described above, several security features need to be incorporated into
the development of AMI, along with new privacy laws to protect consumers. Current privacy laws in the United States are fragmented and vague and do not specically address consumer energy usage. Data stored at the meter and transmitted over communication networks must also meet standard cybersecurity requirements, including condentiality, integrity, availability, and nonrepudiation. One security feature alone, such as encryption, will not be able to cover all the possible security threats. Since it is imperative that the industry maintain 100% uptime, both the physical security of the AMI system hardware and multiple standard IT security features like encryption and authentication must be provided for. Furthermore, since it will be impossible to protect against all threats, smart meters must be able to detect even the most subtle unauthorized changes and precursors to tampering or intrusion. Additional consideration must also be given to the cost and impact the security features will have on AMI system operations. Smart meters will need to be cost-effective, since millions will need to be purchased and installed to replace antiquated analog devices. And they must also be robust as they will be deployed in very insecure locations.
Current Security Initiatives

Since the terrorist attacks of 11 September 2001, several steps have been taken and initiatives accomplished to enhance the security and reliability of the nations current electricity infrastructure. These include the Complex Interactive Networks/Systems Initiative (CIN/SI), a joint program sponsored by the Electric Power Research Institute (EPRI) and the U.S. Department of Defense (DOD); EPRIs Enterprise Information Security (EIS) program; EPRIs post9/11 Infrastructure Security Initiative (ISI); and various North American Electric Reliability Corporation (NERC) initiatives, such as its information sharing and analysis centers (ISACs), public key infrastructure (PKI), and spare equipment database. Information security frameworks for electric power utilities have also been developed by the International Council on Large Electric Systems (CIGRE). A security framework is considered as the skeleton on which various elements are integrated for the appropriate management of security risk. The various elements considered by CIGRE include security domains, baseline controls, and security processes.
Research and Development Needs

The Smart Infrastructure: A Smarter, More Secure I-35W Bridge
Within less than a year after the August 2007 collapse of the I-35W bridge in Minneapolis, Minnesota, a city of sorts on the south side of the former bridge took shape, complete with a host of heavy-duty equipment pieces, temporary on-site areas for casting and other tasks, and crews constantly at work. The days and months that followed required
extraordinary efforts from many, including alumni of the University of Minnesotas infrastructure systems engineering program. They incorporated a sensor network into the new I-35W bridge (at less than 0.5% of total cost) that provides full situational awareness of stressors, fatigue, material, and chemical changes, so as to measure and understand the precursors to failure and to enable proactive and a priori corrective actions. Analogously, customized and cost-effective advancements are both possible and essential to enable smarter and more secure electric power infrastructures. For example, advanced technology now under development or under consideration holds the promise of meeting the electricity needs of a robust digital economy. The end vision of the smart grid consists of a highly developed electrical platform that engages consumers, enhances efciency, ensures reliability, and enables integration of renewable energy and electric transportation. One key money- and power-saving element of the smart grid is its ability to measure how and when consumers use the most power. This information allows consumers to be charged variable rates for energy, based upon supply and demand. This variable rate will incentivize consumers to shift their heavy use of electricity to times of the day when demand is low. The total cost of a stronger transmission system would be about US$82 billion over the next decade. Additionally, to create a smarter end-to-end power delivery system, we must invest between US$17 and US$24 billion over the next 20 years. Investment in a smart grid would nearly pay for itself by reducing stupendous outage costs, a savings of US$49 billion per year, and improving energy efficiency, a savings of US$20.4 billion per year. Likewise, through smart grid-enhanced energy efficiency, by 2030 carbon dioxide emissions from the electric sector would be reduced by 58%. Americans should not accept or learn to cope with increasing blackouts, nor should we rest on the notion that the technical know-how, political will, or money to bring our power grid up to 21st century standards do not exist. The truth is that, as a nation, we must and absolutely can meet the power needs of a pervasively digital society if the United States wishes to maintain its role as a global economic and political leader. The best of American innovation is yet to come, and the smart grid must be part of our future. The potential exists to create an electricity system that provides the same efciency, precision, and interconnectivity as the billions of microprocessors that it will power. From a strategic viewpoint, long-term developments and research issues relating to the defense of cyber and physical interdependent infrastructure networks must also be considered. The driving scientic motivation is to further our understanding of adaptive self-healing and self-organizing
39 11
mechanisms that can be applied to the development of secure, resilient, and robust overlaid and integrated energy, power, sensing, communication, and control networks. In addition to the above, further research and development needs include the following areas: 1) Enabling technologies for an end-to-end secure system of sensing and measurement, leading to improved analysis and visualization and eventually to automation and self-healing systems: monitoring and analysis, automation and control, materials science, power electronics, and integrated distributed energy resources (DERs) sensing, communication, data management, and mathematical and theoretical foundations to support a better, faster, and higher-condence understanding of what is going on, leading to improved state and topology estimation and fast look-ahead simulation. 2) Enabling a stronger and smarter grid by means of complex dynamical systems, systems science, controls, and applied mathematics: modeling, robust control, dynamic interaction in interdependent layered networks, disturbance propagation in networks, and forecasting and handling uncertainty and risk overall systems science and dynamics (including infrastructure, ecology and environment, markets, and data-driven policy designs). 3) Strategic R&D: digital control of the energy infrastructure integrated energy, information, and communications for the end user transformation of the meter into a secure, two-way energy and information portal robust advanced power generation portfolio. Awareness, education, and pragmatic tool development in this vital area continue to remain challenges. Educating stakeholders and colleagues about the cyber and physical interdependencies has often been difcult, as those who are distinguished members of the community and understand power systems well but are less aware of their cybervulnerabilities routinely minimize the importance of these novel and persistentthreats.
about such reports is mainly one portion of an early article: The response to the alert was mixed. An audit of 30 utility companies that received the alert showed that only seven were in full compliance, although all of the audited companies had taken some precautions. This is the reality that needs to be addressed. Finally, no matter how many layers of security or how much sophistication is used in defense mechanisms, it is essential that the industry hire qualied people. Research ndings suggest that human and organizational factors do affect computer and information security performance in a multilayered fashion. Often vulnerabilities are not the result of a single mistake or conguration error but of numerous latent organizational conditions, such as management support and decisions made by designers that combine to create scenarios in which failures and weaknesses may occur. In many complex networks, the human participants themselves are both the most susceptible to failure and the most adaptable in the management of recovery. Thus, staff members must be well trained to respond to a wide variety of emergencies since no amount of technology can replace well-trained personnel.
For Further Reading

J. Clemente, The security vulnerabilities of smart grid, J. Energy Security, June 2009. P. H. Corredor and M. E. Ruiz, Against all odds, IEEE Power Energy Mag., vol. 9, no. 2, pp. 5966, Mar./ Apr. 2011. G. N. Ericsson, Information security for electric power utilities (EPUs)-CIGRE developments on frameworks, risk assessment, and technology, IEEE Trans. Power Delivery, vol. 24, no. 3, pp. 11741181, July 2009. P. McDaniel and S. McLaughlin, Security and privacy challenges in the smart grid, IEEE Security Privacy, vol. 7, no. 3, pp. 7577, May/June 2009. M. A. McQueen and W. F. Boyer, Deception used for cyber defense of control systems, in Proc. 2nd Conf. Human System Interactions, Catania, Italy, 2009, pp. 624631. NIST, Guidelines for smart grid cyber security, The Smart Grid Interoperability PanelCyber Security Working Group, NISTIR 7628, Gaithersburg, MD, Aug. 2010. S. M. Amin, Securing the electricity grid, Bridge, vol. 40, no. 1, pp. 1320, Spring 2010 S. M. Amin, Energy infrastructure defense systems, Proc. IEEE, vol. 93, no. 5, pp. 861875, May 2005. S. M. Amin, Balancing market priorities with security issues: Interconnected system operations and control under the restructured electricity enterprise, IEEE Power Energy Mag., vol. 2, no. 4, pp. 3038, Jul./Aug. 2004.
Conclusion
Cyberconnectivity has increased the complexity of the control systems and facilities it is intended to safely and reliably control. In order to defend electric infrastructure against the impacts of cyber and physical attacks, signicant challenges must therefore be overcome before extensive deployment and implementation of smart grid technologies can begin. Cybersecurity and interoperability are two of the key challenges of the smart grid transformation. As for security, it must be built in as part of its design, not glued on as afterthought. Regarding recent cyberthreat reports, it is fundamental to separate the hype from the truth. What is most concerning
40 12
IEEE ieee power power & & energy energy magazine magazine
Biographies
S. Massoud Amin is with the University of Minnesota. Anthony M. Giacomoni is with the University of Minnesota.
p&e
A Virtual Smart Grid

Real-Time Simulation for Smart Grid Control and Communications Design
IT IS GENERALLY RECOGNIZED THAT A HIGH-BANDWIDTH and highly available networked communication system should overlay the transmission system topology in order to enable the control and protection envisaged today to make the grid more efcient and more reliable. The specications for such a communication system have been difcult to develop, however, because it needs to support a great variety of applications, many of which have not yet been developed. Organizations such as the North American SynchroPhasor Initiative (NASPI) are trying to build on this vision of a communication system that can utilize phasor measurement data to initiate fast controllers, including exible alternating current transmission system (FACTS) devices. A major hurdle in developing such fast, wide area controls has been the lack of design tools available to do so. In particular, the development
IMAGESTATE
By David Anderson, Chuanlin Zhao, Carl H. Hauser, Vaithianathan Venkatasubramanian, David E. Bakken, and Anjan Bose
1540-7977/12/$31.002012 IEEE
ieee IEEE power & energy magazine magazine
13 49
GridSim can represent a large portion of a grid and runs in real time so that various components running at different sampling rates can be tested together.
of controls that depend on communications to carry the input and output signals and complex software to process these signals requires tools to simulate and analyze such controls. To accurately portray the behavior of such controls, design tools must integrate the dynamic behavior of the power system with the response of the communication and computation system. We describe here a simulatorGridSimthat can simulate in real time the electromechanical dynamic behavior of the power grid, the IT infrastructure that overlays the grid, and the control systems taking advantage of that IT infrastructure. This simulator was devised for designing and testing new wide area control and protection schemes. GridSim is able to represent a large portion of a grid and runs in real time so that various components running at different sampling rates can be tested together. rithms. Using actual power system artifacts is important for two reasons. First, it allows the artifacts to be tested in the simulation environment, which is one way to increase condence in a design. Second, it allows existing artifacts such as the Grid Protection Alliances openPDC product and the GridStat communication framework to be used as building blocks for GridSim, speeding its implementation. From this decision comes another requirement: that GridSim operate in real time so as to properly interface with these artifacts.
The Overall Design of GridSim

GridSim is a real-time, end-to-end power grid simulation package designed using a default sample rate of 30 samples per second (per sensor). The goal of this project is to simulate power grid operation, control, and communications at gridwide scale (e.g., the Western Interconnection) in order to give utilities the ability to explore new equipment and control system deployments. Possibilities include simulating large-scale PMU installations and power applications able to utilize the vast quantities of data generated in such a situation. With the objective of providing tools to simulate real-world equipment usage and the ability to be used in conjunction with readily available utility industry equipment, GridSim uses the IEEE C37.118 data format standard for all streaming measurement data. The GridSim platform consists of a number of components falling into four groups: power system simulation, substation simulation, communication and data delivery, and control center applications (see Figure 1). We rst describe the overall relationship between these groups and then look at each of them in detail. The power system simulation calculates the electromechanical dynamics in real time. Sensor data from the simulated power system are fed in C37.118 format to the substation simulation processes at a rate of 30 samples per second. In the substations, data are optionally processed by substation applications and published, along with the outputs of the substation-level applications, to the data delivery component through simulated substation gateways. Delivery to control center applications and other substations occurs via the data delivery system. Note the design choice here: the wide area data delivery system is not involved in connecting simulated sensors within the simulated substations where they are located. Although the substation-level processing of the data is simulated, the data communication within the substation
Background
The use of time-synchronized, high-data-rate sensor technology is widely viewed as a critical enabler for increasing the reliability of the power grid while allowing the integration of many more stochastically variable renewable energy sources such as solar radiation and wind. For example, the deployment of phasor measurement units (PMUs) is becoming more commonplace. PMUs are capable of sampling frequency, voltage, and current thousands of times per second and outputting accurate, time-stamped measurements 30120 or more times per second. It is difcult, however, for utilities to take full advantage of these devices due to a lack of tools for designing and evaluating the control systems that exploit them. Furthermore, the behavior of such control systems will also depend on the performance of the wide area communications systems that connect the sensors, control logic, and actuatorswide area communications systems whose design and specications are themselves still evolving. Simulation is historically one the principal tools used in the design of power system controls. No existing simulation framework, however, can model at the scale of the power grid the combined behavior of the power system, the communications system that overlays it, and the control system that relies on the latter to monitor and control the former. GridSim is intended to address these issues by providing a very exible simulation framework that incorporates power system simulation, data delivery, exible sensor deployments, and the ability to incorporate actual power system components, protocols, and algo50 14
Powertech TSAT Simulator Measurement Generator C37.118 Generator GridStat FE FE Substation N Substation O Substation Su OM Substation SE SubOM Substation SE Sub OM Substation SE Sub OM Substation SE Substation Gateway SE Substation 1 Control Center Applications OpenPDC Oscillation Monitor State Estimator FE Simulated Power System
Static Data Generator
FE FE FE
Substation Simulation
figure 1. GridSim architecture.
is assumed to be negligible for the current goals of wide area control design. The data delivery component of GridSim is GridStat, a publish-subscribe, wide area data delivery framework designed from the ground up to meet the emerging needs of electric power grids. Once data are published, the exibility provided by the GridStat data delivery middleware allows subscribing applications to be easily integrated into the system without massive reconguration. In the current GridSim implementation, published data are used by the two control center applications included in this project: the hierarchical state estimator and the oscillation and damping monitor.
Power System Simulation

Power system simulation in GridSim is provided by a modied version of TSAT, an industry-proven transient stability simulator produced by Powertech Labs, Inc. Unmodied TSAT accepts power system topologies, initial values, and dynamic simulation variables (such as faults at specic times) as inputs. On execution, the simulator loads the input values, then as quickly as possible computes the state of the system over time; on completion it writes the results to a le.
An off-line transient stability simulation such as TSAT does not perfectly meet the needs of GridSim. To obtain real-time performance, the simulator was modied so that simulation time progresses no faster than wall-clock time. This is accomplished by pausing after computing each set of measurements (30 sets per second) until the correct wallclock time arrives for that set to be published. To extract the measurement sets at the time they are produced by the simulation, certain TSAT functions are used. They directly implement simulated PMUs attached to particular points in the power system topology where they measure frequency, voltage, and current 30 times a second. These sensor data from the simulated PMUs are sent to the measurement generator for postprocessing (see Figure 2).
Substation Simulation
The measurement generator also bridges the gap between the bus-branch power system model supported by TSAT and the more detailed bus-breaker model that represents the substations. To do this, GridSims static data generator creates tables that map the FromBus/ToBus/EquipmentID measurement identication information used in TSAT to the unique CircuitBreaker/BusID/PMUID numbers used throughout the rest of GridSim. Data from the static data generator also
15 51
Data delivery latency and loss rate are important factors in the performance of wide area control and protection applications, but Real-Time Data Generator: the data delivery infrastructure Acquire Data from TSAT Output that will ultimately support those applications is still evolving. GridSims data delivery component, GridStat, is a publish-subscribe middleware framework that has Generator or Load inuenced the NASPInet effort Type of Data? led by NERC and the U.S. Department of Energy (DOE). Its design Branch or Transformer centers on the fact that sensor measurements are digitally repreCalculate Complex Voltage from Calculate Complex Voltage and sented as a periodic stream of data Magnitude and Angle. Active and Current from Magnitude and points. Working from this data Reactive Power Are Given Instead Angle Given in TSAT Output of Current Magnitude and Angle, model, GridStat was designed Thus Calculate Complex Current to allow for efcient, wide area, by Solving the Equation encrypted multicast delivery of of Complex Power Assign Complex Voltage and data. GridStat as a component Current as Measurement to of GridSim is a realistic model Respective Equipment for emerging power system data delivery services and at the same time provides great exibility for No conguring and evaluating potenTSAT Output Ends? tial wide area control and protection applications. Yes GridStat is designed to meet the requirements of emerging Assign Measurements to Circuit Breakers control and protection applications that require data delivery latencies on the order of 1020 ms over hundreds of miles with Real-Time Data extremely high availability. The Generator Ends GridStat architecture consists of two communication planes: figure 2. Measurement generator logic. the data plane and the management plane (see Figure 3). The allow the measurement generator to synthesize additional data plane is a collection of forwarding engines (FEs) measurements, such as breaker currents, from the TSAT designed to quickly route received messages on to the outputs. Noise and other real-world attributes can be added next FE or termination point. The FEs are entirely dediwithin the measurement generator, if desired. Once these cated to delivering messages from publishers to suboperations have been performed, the PMU measurements scribers. Routing configuration information is delivered are sent to a C37.118 encoder and then to the substation sim- to the FEs from the management plane. The forwarding latency through an FE implemented in software is on ulation processes. The substation simulation processes host substation- the order of 100 s, and with network processor hardlevel power applications and substation gateways. Power ware it is less than 10 s. We believe that the perforapplications perform computationsboth the applications mance of a custom hardware implementation of an FE described below have substation-level processingand sub- could match or exceed that of a general-purpose Internet mit results to the substation gateway. Measurement genera- router. Thus, in a typical wide area configuration, Gridtor output for each substation is also published to the data Stat would not add more than 1 ms over the speed of the underlying network while providing quality-of-service delivery component by the substation gateway.
52 16
Static Data Generator Generates Equipment in Each Substation
Communication System and Data Delivery
GridStat allows for virtual substations to be created or reconfigured and additional subscribers and power applications to be added with minimal changes.
(QoS) guarantees tailored to rate-based control and protecQoS Broker tion applications. The management plane is a set of controllers, called QoS broQoS Broker kers, that manage the FEs of the data plane. The QoS brokers are Leaf QoS Broker organized in a hierarchy to reect the natural hierarchy in power grids. When a subscriber wishes to receive data from a publisher, GridStat it communicates with a QoS FE Pub1 broker that designs a route for FE the data and delivers the routing FE information to the relevant FEs, Pub2 FE creating the subscription. Since Sub1 path computations are done out FE of band from data delivery, even heavy loads of new subscription Sub3 creation do not adversely affect Sub2 the performance of the data plane. Beyond this, QoS brokers have a privileged view of routing performance and the router graph figure 3. GridStat architecture. that allows them to create optimal delivery paths. QoS brokers also implement policies for called rate ltering: only forwarding an update on an outgoing link at the highest rate that any subscriber downstream resource usage, cybersecurity, aggregation, and adaptation. Because the entire purpose of GridStat is the efcient via that link requires. Some kinds of data place additional delivery of data, it includes features providing congurable restrictions on the rate ltering. GridStats rate-ltering QoS per subscription while attempting to minimize data algorithms are coordinated across multiple PMU streams in delivery costs. A subscriber can request quality-oriented order to ensure that subscribers receive sets of updates from parameters such as data delivery rate, temporal redundancy different PMUs taken at the same instant. For example, conof data packets, and spatial redundancy of data streams sider PMUs that send updates at a rate of 120 Hz. While such (delivery over multiple independent delivery paths, each of a high rate would be useful for a few application programs, which meets the end-to-end delay requirements). The QoS many applications would not need such frequent updates. brokers ensure that each subscriber gets the resources it needs For an application subscribing to two different PMU streams while preserving the needs of existing subscriptions. To con- at a rate of 20 Hz, ve-sixths of the updates will be dropped serve network resources, the management plane identies before reaching it. But GridStat ensures that the same oneany shared data paths between a publisher and two or more sixth of the updates are delivered from the two PMUs, so subscribers. If there is any overlap in these paths, the manage- they can be used as a global snapshot. This synchronized ment plane ensures that data are only sent once for that leg of rate ltering is set up when subscriptions are being added and is based on time stamps in the updates, so it does not the journey before being duplicated at the split. GridStat supports multicast delivery of a given sensor require any inter-FE coordination when updates are being update stream whereby different subscribers can subscribe delivered. So scalability is not harmed by this strong delivto different rates yet no update message is ever sent over a ery property. When used as the data delivery layer component of Gridnetwork link more than once and it is not forwarded on a link at all if not needed. FEs implement this via a mechanism Sim, GridStat allows for virtual substations to be created or
IEEE power & energy magazine magazine ieee
53 17
recongured and additional subscribers and power applications to be added with minimal changes. This contrasts starkly with the current situation in the power grid, where even minimal changes to the number of sources or consumers of data can require the data delivery system to be completely re-architected. Conversely, GridSim also allows for potential deployments of GridStat to be tested with real-world volumes of data and with different network and power system topologies.
Control Center Applications

Continuing the theme of using existing artifacts as components of the GridSim environment, we now describe two control center applications that have been incorporated into GridSim thus far. One of the main objectives of GridSim is to allow experimentation with and testing of wide area control and protection applications using PMU and other high-rate, timestamped data streams. Thus far, two prototype applications have been included in GridSim: a linear, hierarchical state estimator and an oscillation monitoring system. Both applications were built using components of the Grid Protection Alliances openPDC product. Thus, one benet of incorporating these applications in GridSim is that other openPDC-based applications can easily be brought into the GridSim environment. The openPDC application
Start
Read Data from PDC
Yes Event? No FDD Analysis Damping for Ambient Data Monitor Engine Moving Window Cross-Check Prony Analysis for Postdisturbance Event Data Analysis Engine Moving Window Cross-Check
set is an open-source software system that collects PMU measurements from multiple sources, aligns them according to their time stamps, and processes them with user-de ned functions. The openPDC applications also provide numerous advanced functions, such as cybersecurity and device management, that are necessary for industry use. Thus far, however, GridSim uses only the C37.118 protocol parser and the time-alignment functionality. The openPDC applications contain three kinds of adapters: input adapters, action adapters, and output adapters. GridSims applications, however, use only two of these. Input adapters read data and parse them. Although the openPDC applications provide many built-in input adapters that can read data from les, databases, or the network, none of them supports the publish-subscribe communication pattern used in GridSim. New input adapters were therefore developed supporting the GridStat publish-subscribe system. Action adapters receive time-aligned measurements and process them. In GridSim, all of the power system calculations, including substation-level and control centerlevel state estimation as well as oscillation detection, are implemented using custom action adapters. These new functions embedded in the openPDC applications are not only useful in the simulation environment but can also be run in the real industry environment. Since the openPDC applications were primarily designed and implemented for eld usage, which has different technical requirements from GridSim, work was performed to adapt them for the simulation environment. For example, the openPDC applications provide a user interface for conguring devices, phasors, and measurements. Since GridSim is intended to simulate a variety of systems that may change frequently, manual conguration is too cumbersome and error-prone. A program was therefore created to read the power ow le for TSAT and congure the whole system automatically, saving a lot of effort and simplifying the integration of the openPDC and simulation software.
The Oscillation Monitoring System

The oscillation monitoring system (OMS) application has been developed at Washington State University for realtime monitoring of problematic electromechanical oscillations using wide area PMU measurements. OMS combines advanced signal-processing algorithms with heuristic expert system rules to automatically extract the damping ratio, frequency, and mode shape of poorly damped electromechanical oscillations in a power system from power system measurements. A prototype OMS has been implemented as part of the phasor data concentrator at Tennessee Valley Authority (TVA) since 2007. It is also currently being implemented at Entergy in conjunction with a smart grid investment grant project. In our GridSim project, the OMS is being used as a realtime application example, both serving to illuminate what GridSim must provide in order to incorporate actual applications and demonstrating how executing an application with
Poorly Damped Mode Detected? No
Yes
Alarm Controller Trigger
figure 4. Flowchart of an oscillation monitoring system.

54 18
simulated real-time test data can help validate the application. The OMS engines are integrated into an action adapter module of the openPDC applications. Thus, the OMS receives real-time simulated PMU data streams from TSAT, via the measurement generator and the data delivery system, which are buffered onto the internal signal-processing engines of the OMS. Results from the OMS can be exported to a custom SQL database that can be visualized and set to trigger alerts or alarms whenever damping levels of oscillatory modes fall below prespecied thresholds. The operator can then take manual action to bring the damping back to acceptable levels. Unlike the real power system, where the actual modal characteristics of the system are unknown values, the modal properties of the test system in TSAT can be accurately determined from model-based small-signal stability analysis. Comparing the outputs of the OMS engines with the respective model-based modal values is useful for testing and tuning the OMS engines for target power systems. Since GridSim includes communication models, such studies also reveal the effects of communication delays, the loss of PMU channels, and network congestion on the resulting OMS modal estimates. We plan to use GridSim to test automatic control action by the OMS, although such closed-loop feedback will require further modication of TSAT. The OMS includes two engines, as shown in the ow chart in Figure 4. The event analysis engine, shown on the right side of the ow chart, carries out an expert system based Prony-type ringdown analysis of system responses following disturbances in the system. The objective for this engine is fast detection of sudden changes in the damping of oscillatory modes from large disturbances in a power system, so that mitigating control actions can be initiated before the damping problems degenerate into widespread blackouts. Typical analysis uses 510 s of PMU data at a time, and the calculations are repeated over moving time windows and over different PMU signal groups to ensure the consistency of results. The event monitor engine can typically detect oscillatory problems by using 1015 s of PMU data, starting from the instant the oscillations begin to appear in a power system. The complementary damping monitor engine, shown on the left side of the ow chart, estimates the damping, frequency, and mode shape of poorly damped oscillatory modes from ambient PMU measurements. Unlike the event monitor engine, which only works when the system is subject to disturbances, the damping monitor engine is applicable all the time. By using natural power system responses to routine random uctuations from load variations and generation changes, the damping monitor engine continuously tracks damping levels and mode shapes of poorly damped oscillatory modes. The damping monitor engine uses an extension of a frequency-domain algorithm called frequency domain decomposition (FDD). This engine is aimed at preventive detection of poorly damped oscillations. The damping monitor engine uses about four minutes worth of PMU data in
5.15 5.14 5.13 5.12 5.11 5.1 5.09 820 Am Am Amb mb bient Noise A nalysis Ambient Analysis 1.2 Hz at + 1.8% Damping. Local Mode. E v vent Event Analysis n 1.2 1 .2 2 Hz at + 1.5% Damping. Local Mode. 840 860 880 900 Time (s) 920 940
figure 5. Illustration of analysis results from OMS engines.
every computational run. As with the event monitor engine, the analysis is then repeated over moving time windows and over different signal groups to verify the consistency of modal analysis results. Figure 5 shows the results from the two engines for a recent event near a major generating plant. The system encountered a routine event at about 830 seconds. The event analysis engine of the OMS carried out moving time-window analysis of the PMU measurements using real-time Prony analysis and concluded at 838 seconds (the vertical dotted line in Figure 5) that the oscillation was from a local 1.2-Hz
Control Center Static Maintenance Database CB/ND Connections Equipment Parameters
ND/Equipment Connections
Topology Processor
System Topology
State Estimator
Digital Status
Analog Measurements
Real-Time Database
SCADA
Substation RTU
Substation RTU
Substation RTU
figure 6. The two-level linear state estimator.

IEEE ieee power power & energy energy magazine magazine
55 19
mode (i.e., one involving mainly one PMU or a few nearby PMUs) with a damping ratio of +1.5%. Subsequently, the damping monitor engine analyzed the real-time ambient PMU data and estimated the dominant oscillatory mode to be the same local mode at 1.2 Hz, with a damping ratio of +1.8%. Thus the results of ringdown analysis and ambient noise analysis match well for this example. The two engines serve as complementary techniques for identifying the dominant poorly damped oscillatory modes of a power system whenever such modes exist.
State Estimator
A two-level linear state estimator has been developed at Washington State University that is an excellent candidate
application for testing in the GridSim environment. It is based on PMU data and requires algorithmic processing at the substation level, fast communication of the substation results to the control center, and synchronization of the data at the control center before it nally calculates a state estimate (SE) for the whole system. The power system simulation produces PMU measurements 30 times per second, and the nal SE is also calculated at the same rate. Thus errors in the simulation, communication, synchronization, and SE calculation can all be checked during the testing of this application on GridSim. The processing of this two-level SE is shown in Figure 6 for both the substation level and the control center level. At each substation, the local PMU data are processed using linear estimation algorithms for both current and voltage
TSAT Bus Voltage 1.2 1.0 0.8 0.6 0.4 0.2 0.0 0 1 2 3 4 Value (p.u)
Bus 1 Bus 7 5
Bus 2 Bus 8 6
Bus 3 Bus 9 7
Bus 4 Bus 10 8
Bus 5 Bus 11 9 10
Bus 6
11
Time (s) (a) Generated Bus Voltage 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Value (p.u)
Bus 110 Bus 440 0 1 2 3 4
Bus 150 Bus 500 5 Time (s) (b) 6
Bus 220 Bus 770 7
Bus 260 Bus 880 8
Bus 330 Bus 990 9
Bus 410
10
11
Substation Bus Voltage 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Value (p.u)
Bus 11 Bus 41 0 1 2 3 4 5
Bus 12 Bus 42 6 Time (s) (c)
Bus 21 Bus 71 7
Bus 22 Bus 81 8
Bus 31 Bus 91 9
Bus 32
10
11
Control Center Bus Voltage 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Value (p.u)
Bus 11 Bus 41 0 1 2 3 4 5
Bus 12 Bus 42 6 Time (s) (d)
Bus 21 Bus 71 7
Bus 22 Bus 81 8
Bus 31 Bus 91 9
Bus 32
10
11
figure 7. GridSim results for an 11-substation system using the two-level linear state estimator.
56 20
phasor measurements. This processing has the advantage of estiSE Bus Voltage Curve 2 1.2 mating and eliminating errors from noise, bad analog data, and 1.0 bad circuit breaker status data on 0.8 a small set of measurements. The 0.6 topology, current, and voltage esti0.4 mates from each substation are Bus 11 Bus 12 Bus 71 Bus 22 Bus 81 Bus 91 Bus 42 Bus 41 then sent through the communica0.2 Bus 32 Bus 31 Bus 21 tion network to the control center. 0.0 At the control center, the data are 0 1 2 3 4 5 6 7 8 9 10 11 Time (s) synchronized for the same time stamp, and the whole system states are linearly estimated. figure 8. State estimator results with jitter in the communication system. Figure 7 provides some results for this test as carried out on GridSim for an 11-substation power system. For a small system GridSim, and the signicant changes needed in the power like this, the simulation and communication speeds were system simulator to accomplish this are being developed. not a problem, so the tests purpose was mainly to check the computation processes and data delivery. When the SE Acknowledgments was running perfectly, the gure shows that the bus voltages We gratefully acknowledge the assistance of Powertech Labs (a) calculated by the TSAT simulation, (b) generated by the and the Grid Protection Alliance in adapting their TSAT and PMU data generator, (c) estimated at the substation level, and openPDC products, respectively, for use in GridSim. This (d) estimated at the control center all compare quite well 30 research was supported by a grant from the U.S. Department times a second for about eight seconds after a fault on the sys- of Energy (Award #DE-OE0000032). tem. Many things can go wrong, however, as demonstrated in Figure 8 by introducing some jitter in the data delivery For Further Reading between the substation and the communication level, thus [Online]. Power Tech Labs. TSATTransient Security Asproducing erroneous SE results at the control center. sessment Tool. 2011. Available: http://www.powertechlabs. com/software-modeling/dynamic-security-assessmentsoftware/transient-security-assessment-tool Conclusions D. Bakken, A. Bose, C. Hauser, D. Whitehead, and G. A fast communication and computation system overlaying the power grid is a key enabler for applications taking Zweigle, Smart generation and transmission with coherent, advantage of PMUs and FACTS controllers to achieve the real-time data, Proc. IEEE (Special Issue on Smart Grids) , smart grid of the future. The tools needed to develop and test vol. 99, no. 6, pp. 928951, June 2011. [Online]. Grid Protection Alliance. The Open Source these new applications do not exist today, however. We have described such a toola simulation platform called Grid- Phasor Data Concentrator. 2011. Available: http://openpdc. Simthat can be used to develop and test wide area control codeplex.com G. Liu, V. M. Venkatasubramanian, and J. R. Carroll, and protection schemes. We have developed this platform to simulate the power Oscillation monitoring system using synchrophasors, in grid in real time for electromechanical dynamics and to Proc. IEEE PES General Meeting, Calgary, Canada, July generate and stream PMU data in standard format. It also 2009, pp. 14. T. Yang, H. B. Sun, and A. Bose, Transition to a twoincludes the ability to deliver measurements and processed data over a high-bandwidth networked communication sys- level linear state estimator, part I: Architecture, part II: Altem called GridStat. Finally, we have used GridSim to simu- gorithm, IEEE Trans. Power Syst., vol. 26, no. 1, pp. 4662, late and test two new applicationsoscillation monitoring Feb. 2011. and linear state estimationthat are quite different from each other but both utilize PMU streaming data in real time. Biographies We show that platforms such as GridSim can successfully David Anderson is with Washington State University. and rapidly prototype new smart applications. Chuanlin Zhao is with Washington State University. We should note that closed-loop control is not illustrated Carl H. Hauser is with Washington State University. in this article. Both the OSM and the linear state estimator Vaithianathan Venkatasubramanian is with Washingare real-time but open-loop applications, which means that ton State University. the outputs are used by the operator to initiate manual control David E. Bakken is with Washington State University. p&e if necessary. Closed-loop control will be incorporated into Anjan Bose is with Washington State University.
Voltage (p.u) january/february 2012
57 21
Reprinted from May/June 2012 issue of IEEE Power & Energy magazine
PUBLIC POLICIES AT BOTH THE STATE AND FEDERAL LEVELS in the United States and a variety of technological and economic changes are poised to signicantly alter both the demand for and supply of electricity in the country over the next several decades. These changes will yield a wide range of new challenges and opportunities, including incorporating variable energy sources like wind and solar radiation; adjusting distribution systems to accommodate small-scale, distributed generators; accommodating the charging of electric vehicles and other changes in electricity demand; making the best use of new technologies to ensure reliability and efciency under changing conditions; responding to threats presented by the vast increase of data communications within the grid; and meeting changing workforce needs. A variety of technologies exists today that can help meet the emerging challenges effectively in the United States. In a recently completed two-year study on the future of the U.S. electric grid that we performed with a dozen other economists and engineers, however, we found that the promise of these new technologies will only be fully realized if a number of regulatory policies are changed, if necessary research and development is performed, and if important data are compiled and shared. Maintaining system reliability, keeping electricity rates at acceptable levels, and achieving state and federal policy goals will depend to a large degree on a few key choices madeor not madeat the state and federal levels and within the industry over the next few years. In this article, we rst discuss the performance of the U.S. grid today. Then, we describe several of the most important challenges and opportunities that are likely to face the U.S. grid over the next several decades.
The U.S. Grid Today

Physically, the U.S. electric grid consists of approximately 170,000 mi of high-voltage electric transmission lines (i.e., lines rated at >200 kV) and associated equipment and nearly 6 million mi of lower-voltage distribution lines. In aggregate, the U.S. grid serves about 125 million residential customers, 17.6 million commercial customers, and 775,000 industrial customers that account for 37%, 36%, and 27% of electricity use, respectively. At the highest level, the electric power system of the continental United States consists mainly of three independently synchronized grids: the Eastern Interconnection, the Western Interconnection, and the Electric Reliability Council of Texas (ERCOT). The three grids are linked by only a few low-capacity dc lines. Within these broad areas are 107 balancing authorities, responsible for balancing the supply and demand for power in specied zones.
ARTVILLE, LLC.
Digital Object Identier 10.1109/MPE.2012.2188669 Date of publication: 19 April 2012
30 22
IEEE power power & & energy energy magazine magazine ieee
1540-7977/12/$31.002012 IEEE
may/june 2012
By Timothy D. Heidel, John G. Kassakian, and Richard Schmalensee
Policy Challenges and Technical Opportunities on the U.S. Electric Grid
may/june 2012
23
An important measure of the performance of a transmission and distribution system is the fraction of energy generated that is lost due to heating of transmission and distribution lines and of other components. That fraction has fallen signicantly over time in the United States. As Figure 1 shows, losses in transmission and distribution decreased from more than 16% in the late 1920s to less than 7% today. This reects investments in transmission and distribution systems, the development and deployment of more efcient transformers and other equipment, and transmission at higher voltages. Reliability is also an important dimension of performance. Increases in transmission voltage and many other, less visible, technological advances have contributed to improved reliability over time. Protective relaying enabled the detection and isolation of system faults, for instance, and high-speed reclosing circuit breakers and relaying allowed transmission lines to be reenergized after a fault automatically and in only a few seconds. Lightning arrestors allowed the effects of lightning strikes to be contained automatically. At the bulk power level, data on major disturbances and unusual occurrences have been reported to the U.S. Department of Energy (DOE) since the 1970s and to the North American Electric Reliability Corporation (NERC), which has responsibility for the reliability of the bulk power system, since 1984. These data are not consistent, complete, or necessarily accurate, however, and they cannot reliably be used to assess changes in the reliability of the bulk power system over time. Most outages in the United States occur within distribution systems, but a recent study from Lawrence Berkeley National Lab found that only 35 U.S. states require utilities to report data on the impact 18 of all outages on consumers, and reporting standards and practices 16 differ. It is accordingly impossible 14 to make comprehensive comparisons across space or over time. In 12 particular, the treatment of very short interruptions varies among 10 U.S. states and among different 8 countries, so outage counts cannot be usefully compared. 6 Nonetheless, the data that are 4 available suggest that U.S. reliability is on a par with that of 2 other industrialized countries. 0 According to a recent study from the Electric Power Research Institute (EPRI), U.S. customers Year can expect to experience between one and a half and two power figure 1. U.S. transmission and distribution losses, 19262009. Losses are measured interruptions and between two as the difference between energy generated and energy delivered to customers and and eight hours without power thus in practice include losses due to theft. Theft is not considered to be important in each year. This is on a par with the United States today, but it is significant in some other nations. As a result of the layering of historical policy decisions and the lack of a comprehensive, shared vision of system structure or function, the U.S. electric power system today operates under a fragmented and often inconsistent policy regime. For instance, organized wholesale markets for power play a central role in some areas, but in others the traditional vertically integrated utility model remains dominant. Generation facilities are variously owned by investorowned utilities, rural cooperatives, municipal utilities, federal government entities, and independent power producers. Subsidies of various sorts for public and cooperative entities are important in some regions but not at all in others. Transmission and distribution voltage levels also vary regionally. Several hundred entities currently own parts of the transmission or bulk power system and, at the distribution level, approximately 3,200 organizations provide electricity to retail customers. Assessing the performance of a system as complex as the U.S. electric grid is not a simple task. International comparisons and even comparisons within the United States are difcult because of differing geography, rates of growth, and denitions of performance measures. Systems that have grown more rapidly in recent years, for instance, will on average have newer equipment. Comparisons over time may reveal nothing more than the advance of technology driven by vendor R&D. Moreover, because there are diminishing returns to investing to increase efciency and reliability and because perfection is unattainable at any cost, it is possible not just to underinvest but also to overinvest in these and other dimensions of performance.
T & D Losses (% of Total Generation) 32 24
1926 1929 1932 1935 1938 1941 1944 1947 1950 1953 1956 1959 1962 1965 1968 1971 1974 1977 1980 1983 1986 1989 1992 1995 1998 2001 2004 2007
may/june 2012
most European countries, where customers generally experience from less than one interruption per year to almost three, based on data from the Council of European Energy Regulators. Of course, there is great variation in reliability between urban and rural areas. Such comparisons cannot reveal whether U.S. reliability is too low, too high, or optimal, given the benets of reducing outages and the costs of doing so. A nal dimension of performance involves the use of new technology to increase productivity. The U.S. electric utility industry has historically devoted a very small fraction of its revenues to R&D, instead relying primarily on its suppliers for innovation. U.S. utilities have sometimes collaborated with vendors on R&D activities and have participated in collaborative research through EPRI. In recent years, however, utilities have shifted away from longer-term, collaborative projects and toward shorter-term proprietary efforts. Moreover, investor-owned utilities, which account for almost all nonfederal utility R&D spending, reduced their R&D budgets beginning in the 1990s, spending on average less than 1% of their revenues on R&D. The decrease in utility R&D funding reects, in part, reluctance among utilities to incur (and regulators to approve) R&D expenditures as U.S. federal and state policies pursued more industry competition during those years.
New Challenges and Opportunities

Grid-Scale Variable Generation
Owing to strong federal and state policy support, wind and solar generation are almost certain to become more important in the United States over the next several decades though perhaps not as important in many U.S. regions as they already are in some European Union countries. Efciently increasing the penetration of grid-scale renewable generation while maintaining reliability will require modications to power system planning and operation. At high penetration levels, the variable and imperfectly predictable power output of these variable energy resources, or VERs, causes the demand minus VER generationthat is, the net load that must be met by other generatorsto become noticeably more variable and difcult to predict. To maintain reliability despite this variability, the system and its operation must be modied at some cost. Wind and solar forecasts will have to be fully integrated into system operations and planning. Indeed, utilities and system operators in many U.S. regions are already actively working on improving their forecasting capabilities. Power system exibility will also become more important, and incentives for investments that add generation exibility or for operating generation resources in a exible manner may be needed in regions with organized markets. Full or virtual consolidation of small balancing areas would facilitate VER integration, as would requiring new VER generators to meet performay/june 2012
mance specications, such as low-voltage ride-through and inertial response, appropriate for operation in the high-VER future they are likely to encounter. In the United States, many of the most attractive wind resources are located in the wind belt that stretches north from Texas through the Dakotas to the Canadian border and offshore on both coasts. While the offshore resources are closer to major load centers, the costs of offshore wind installations are generally considerably greater than those for onshore facilities in good locations. Similarly, the prime locations for solar power are in the nearly cloud-free and sparsely populated desert Southwest. Exploiting these resources will require building more transmission than if fossil-fueled or nuclear generating plants built relatively close to load centers were supporting system expansion. The use of very long transmission lines can cause technical issues and compromise system stability; such issues will have to be monitored carefully in the years to come. In addition, adequate planning tools that can deal with complex networks and that take uncertainty rigorously into account do not exist today, and research to develop them is needed. For such research to be most productive, detailed data covering the major interconnectionsdata that are now deemed proprietarymust be made appropriately available to researchers. As VER penetration increases, more of the new transmission lines will cross state borders or the 30% of U.S. land managed by federal agencies. Cost allocation and siting have been particularly contentious for these transmission facilities. When boundary-crossing lines are proposed today, they tend to be evaluated in isolation rather than as part of a wide-area planning process, and allocation of the costs involved is often done via facilities-specic negotiations. Under current law, the siting of all transmission lines is a matter for the states rather than FERC. Lines that cross land managed by federal agencies also need the approval of those agencies. Consequently, the construction of interstate transmission facilities requires the consent of multiple state regulators and, sometimes, of one or more federal agencies. FERC Order No. 1000, issued in July 2011, should signicantly increase wide-area planning of transmission systems, make routine the allocation of the costs of boundary-crossing transmission facilities, and, by explicitly adopting the beneciaries pay principle, rationalize the allocation of those costs. Establishing permanent and collaborative planning processes at the interconnection level and a single cost allocation procedure for boundary-crossing projects in each interconnection would further enhance the ability of the United States to efciently and reliably achieve its renewable energy goals. Expanding FERCs authority for siting boundary-crossing transmission facilities would help facilitate transmission expansion for the integration of VERs. Even with this change, however, siting transmission facilities will remain a difcult challenge to manage.
33 25
Distributed Generation
Policies at both the U.S. state and federal levels favor distributed generation from low-carbon sources, and these policies seem likely to continue. At the federal level, personal and corporate tax incentives encourage distributed generators. Most states have programs that subsidize distributed generation. The DOE has also established the goal that all new commercial and industrial construction should be energyneutral by 2030. That is, such buildings must generate as much energy as they use. Furthermore, net-metering programs in 46 states and the District of Columbia compensate end users for generating their own energy at the retail electricity rate rather than the wholesale cost of energy. Customers who generate electricity on-site in these programs save both the energy charge, or the wholesale cost of energy, and the distribution charge for that electricity. The utility, however, saves only the corresponding energy cost. In this way, recovering network costs through per-kWh charges provides an additional subsidy to distributed generation that can encourage its uneconomic penetration. At low levels of penetration, distributed generation simply reduces the load at individual substations. At high levels of penetration, however, distributed generation can exceed load at the substation level, causing unusual distribution ow patterns. In some cases, power many even ow from the substation into the transmission grid. Many distribution systems are currently not designed to handle such reverse ows, however, and customer power quality can sometimes suffer. High levels of penetration can also add to the stress on electrical equipment, such as circuit breakers, and complicate the operation of the distribution system, particularly during emergencies. Additional monitoring and new systems for the operation, protection, and control of distribution systems will be necessary if U.S. distributed generation penetrations grow signicantly. And since much of this distributed generation will be in the form of VERs, there will be an impact on the control of central generating resources. Enabling such penetration in a cost-effective manner will
require investment by the distribution utility. Current regulatory frameworks may not provide adequate incentives for such investments, however, as growth in distributed generation will often reduce utilities sales and prots. Transitioning away from recovering the largely xed costs associated with transmission and distribution networks through volumetric (US$/kWh) charges would help alleviate this incentive misalignment and could have a signicant impact on the growth of U.S. distributed generation.
Changes in Electricity Demand

Unlike some other regions of the world, electricity demand growth in the United States is not likely to emerge as an important source of disruption in the next few decades. Based on data from the U.S. Energy Information Administration (EIA), between 1949 and 1973 U.S. electricity use grew at an average annual rate of 8.3%. The system was able to meet that demand growth with only sporadic difculty. With rising prices after 1973, electricity use grew at an average annual rate of 2.5% between 1973 and 2006. In contrast, EIAs most recent reference case projection is for growth to average only about 0.9% per year between 2010 and 2030. U.S. electricity demand has changed, however, and is likely to continue to change in ways that pose challenges to the system. Over the past several decades, due in part to the increased penetration of air conditioning and the relative decline of industrial loads, there has been a substantial increase in the ratio of system peak loads to average loads. Because power systems need to be sized to meet peak demand with a margin for reliability, the peakier demand becomes (all else being equal), the lower capacity utilization becomes, and thus the higher rates must be raised to cover all costs. Figure 2 illustrates this change, showing load duration curves for New England and New York expressed as percentages of peak hour demand. The gure shows, for instance, that in the 198084 period in both New York and New England, demand exceeded 80% of its peak for only about 1,000 hoursabout 11.4% of the time. By the 2005 09 time frame, demand in both New York and New England exceeded only 70% of its peak for about 1,000 hours, so that more than 30% of capacity was in use less than 12% of the time. This trend raises average costs because of the need to pay for capital that is idle most of the time and, by increasing capacity requirements,worsens the problem of siting generation plants and transmission lines. Electric vehicles (EVs)including plug-in hybrids and pure electric vehiclescould exacerbate these trends. Although their penetration is generally projected to be slow at the national level, EVs are expected to achieve high levels of penetration quickly in some high-income areas with environmentally conscious consumers. If EVs are charged when commuters return home, as seems most likely under current policies, they could add signicantly to system peak loads, worsening the problem of increasing peakiness of demand.
may/june 2012
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
Normalized Load
New England Average 19801984 New England Average 20052009 NY Average 19801984 NY Average 20052009 1 2,001 4,001 6,001 Hours of Year 8,001
figure 2. Normalized load duration curves for New England and New York.
34 26
Due to a variety of recent policy changes and technical innovations, the U.S. electric grid will encounter significant opportunities and challenges over the next several decades.
On the other hand, measures that encourage overnight charging could increase demand when it would otherwise be low, thus tending to atten load duration curves. Making other loads similarly responsive to system conditions could also shift demand off-peak, helping to slow the trend depicted in Figure 2. Dynamic pricingin which retail prices vary over short time intervals to reect changes in the actual cost of providing electricitycould induce such responses. U.S. demand-response programs have grown substantially in recent years. Most demand-response programs in place today, however, use other approaches and focus on responding to occasional emergencies rather than systematic load leveling. In some regions with organized wholesale markets that include a capacity market, demand response has been allowed to bid in as a proxy for capacity, illustrating its potential value to the economic efciency of the system. A variety of new and emerging technologies, including advanced metering systems, can receive price information based on the real-time cost of providing electricity and can transmit usage information every few minutes. This makes it possible to provide real-time incentives to reduce system peaks caused by central air-conditioning, vehicle charging, and other loads, resulting in more efcient use of grid assets and thus lower rates. Many large commercial and industrial customers already operate under dynamic pricing. Such pricing regimes likely will also be widespread optionsif not the defaultfor residential U.S. consumers by 2030. Existing studies suggest that regulators can achieve substantial load shiftingand perhaps overall demand reductionwhen dynamic pricing is combined with the use of technology to automate responses to price changes. Residential dynamic pricing also requires substantial investment in advanced metering infrastructure (AMI) to measure usage over short time intervals. Substantial AMI investments have recently been funded through the American Recovery and Reinvestment Act of 2009 (ARRA), and some state regulators have mandated universal AMI deployment. But thus far, there has been little if any movement toward the dynamic pricing regimes that AMI enables. As long as the results are shared widely, ARRA-supported and regulator-mandated investments in AMI will provide important learning opportunities to develop efcient paths to universal dynamic pricing. Where wholesale electricity markets exist, effective competition in the retail sales of electricity might stimulate innovation in ways that make dynamic pricing both acceptable to consumers and regulators and effective in modifying demand. Response automation technologies are not yet
may/june 2012
mature, however, and further research into the behavior of U.S. residential consumers faced with dynamic pricing is needed. In this later regard, although there have been many dynamic pricing pilot programs, few have been structured so as to produce reliable data, and results have been highly variable.
Innovative Technologies for Increased Reliability and Efficiency

Innovative technologies that can improve system performance, offering enhanced reliability, increased capacity, and the ability to better accommodate new resources (VERs, EVs, and so on) are poised for signicant growth in the United States. The integration of such technologies into comprehensive networks of sensors, communications infrastructure, control equipment, and intelligent management systems will be a major focus of the U.S. electric power industry over the next several decades. These technology opportunities, often referred to as the smart grid, are likely to provide signicant benets. In the transmission system, phasor measurement units (PMUs) are powerful devices that provide rich streams of frequent, time-stamped data on transmission system conditions. PMUs with appropriate analysis tools that turn the measured data into actionable information could allow system operators to anticipate contingencies, reduce the risk of wide-area blackouts, enhance system efciency, and improve system models. While PMU hardware exists and is currently being installed more widely in the United States as a result of ARRA funding, the software and analysis tools necessary to fully capitalize on this investment are yet to be developed and deployed. More widespread PMU data sharing among utilities, system operators, and researchers will also be essential for the development and effective use of these tools. In addition to PMUs, exible alternating current transmission system (FACTS) devices based on advances in power electronics will provide greater control of voltages and power ows throughout the bulk power system and could allow more power to be transmitted on existing lines without increasing the risk of failure. Historically, deployment of the most versatile FACTS devices has been limited by their relatively high cost. Costs are falling, however, and higher penetration of VERs is likely to increase the value of deploying these technologies within the U.S. transmission system. Furthermore, integrating FACTS devices with PMUs and emerging wide-area measurement systems will allow their
35 27
control capabilities to be leveraged so as to provide even greater benets. Ongoing research into new system control algorithms, software, and communication systems that fully utilize PMUs, FACTS devices, and other new transmission system technologies is likely to create a high payoff and could accelerate the deployment of these technologies. Many technologies are also available to enhance the reliability and efciency of distribution systems. Coping efciently with the integration of distributed generation, electric vehicles, and demand response will require signicant investments in new and emerging technologies, including distribution management software systems, equipment that is capable of more accurately monitoring and controlling voltages, automatic reconguration of distribution circuits, and advanced metering. The benets of deploying these technologies are less well known and may be more difcult to quantify relative to most recent investments in distribution systems; they will aim to provide new capabilities, not just expand capacity. To reduce perceived uncertainties and make possible better system-specic decisions, it is critical that detailed information about the results of technology pilots and early deployments is shared as widely as possible. The DOE has recently funded a variety of smart grid demonstrations and technology pilots. These projects provide an important opportunity for learning. The electric utility industry has traditionally relied primarily on its suppliers for the innovation that has driven its productivity growth. Supplier R&D has naturally focused on equipment that can be sold to utilities. Therefore, although research in the several non-equipment-related research areas mentioned above is likely to bring substantial payoffs, these are unlikely to attract equipment vendors. The electric utility industry itself should be able to support the efforts required, however, even if federal support does not materialize. For this to happen, regulators will need to recognize that technical progress benets consumers broadly and to permit mod-
est increases in utility R&D budgets. It will also likely be necessary for the industry to reverse the downward trend in cooperative R&D spending and make appropriate use of cooperative funding through EPRI, one or more independent system operators, and project-specic coalitions. Finally, utilities and regulators in the United States and elsewhere have historically tended to avoid investments in unfamiliar technologies perceived to have uncertain payoffs. The tendency of traditional regulatory systems to encourage excessively conservative behavior is likely to become more and more expensive over time if the increasingly attractive opportunities to enhance efciency and reduce cost through the deployment of unfamiliar technology are not exploited. Regulatory innovations are necessary in the United States to provide adequate incentives for investments in unfamiliar technologies while also ensuring that the returns on these investments are shared appropriately with ratepayers. This is an important problembut one without an obvious solution.
Data Communications, Cybersecurity, and Information Privacy Challenges

The increasing use of communications systems, sensing and control equipment, AMI, and distribution automation technologies will enhance reliability and efciency but will also give rise to new challenges. As the U.S. grid evolves, increasing amounts of data will be exchanged among meters, other sensors, and various computers and control facilities through complex communications systems. The National Institute of Standards and Technology (NIST), with substantial industry input, is overseeing the critical process of developing the interoperability standards that are needed to ensure these systems are compatible not only with each other but also with future generations of technology. In addition, there are ongoing debates in the United States about the use of spectrum and the roles of public and private networks. Since no communications system can be completely free from errors, the future grid must be designed to mitigate the consequences of data errors. More chilling is the possibility of deliberate sabotage via computers and data communications, the sort of cyberattacks that other industries have experienced. The existence of more communications nodes and channels facilitates the insertion of malicious data into the system; in addition, a greater reliance on automated responses to system conditions that may be misreported can make it more difcult to prevent serious damage. As illustrated in Figure 3, cybersecurity involves more than protecting against attacks. In fact, as communications systems expand into every facet of grid control and operations, their complexity and continuous evolution will preclude perfect protection from cyberattacks. Response and recovery, in addition to preparedness, will therefore be important components of cybersecurity, and it is important for the government agencies involved to work with the private sector and publicly owned utilities in a coordinated fashion, to support the research necessary to develop best
may/june 2012
Assess Vulnerabilities, Threats, Impacts Reduce Vulnerabilities, Threats, Impacts
Recover and Restore
Mitigation Prevent Attacks, Incidents, Other Outages
Respond During Attack
figure 3. The cybersecurity life cycle.

28 36
ieee power & energy magazine magazine IEEE
practices for response to and recovery from cyberattacks on transmission and distribution systems, and to deploy those practices rapidly and widely. NERC is responsible for cybersecurity standards development and compliance for the U.S. bulk power system, but no entity currently has comparable nationwide responsibility for distribution systems. State public utility commissions (PUCs)which generally are responsible only for investor-owned distribution systemsusually lack cybersecurity expertise, and the same is true of municipal utilities, cooperatives, and other public systems. While the consequences of a successful attack on the bulk power system are potentially much greater than an attack at the distribution system level, the boundary between transmission and distribution has become increasingly blurred, and distribution-level cybersecurity risks will require serious attention. Though NIST is facilitating the development of cybersecurity standards broadly, it does not have an operational role, and no single agency currently has responsibility for cybersecurity across all aspects of grid operations, including distribution systems. This is an unsolved problem, but one that the federal government is actively focusing on. The DOE and the U.S. Department of Homeland Security recently announced an initiative to work together with industry to develop a comprehensive approach to cybersecurity. But even if this joint effort proves workable or if a single agency is ultimately given appropriate regulatory authority, cybersecurity preparedness, response, and recovery efforts across the electric power sector, including both bulk power and distribution systems, will be critical. A variety of federal government agencies, NERC, NIST, state PUCs, utilities, public power authorities, and such expert organizations as IEEE and EPRI will need to be involved if these efforts are to be effective. With the collection, transmission, processing, and storage of increasing amounts of information about customer electricity usage comes heightened concern for protecting the privacy of those customers. As advanced metering is implemented, information on personal habits will be available to electric companies at a level never before envisioned by utilities or policy makers. Information about the operation of the electric grid itself will soon be available at a level of detail that will be of interest to those with both commercial and malicious interests. Deciding who has access rights to these data and ensuring consumers privacy will be important considerations in the design and operation of grid communications networks. Many governments have passed laws protecting the privacy of personal information, though this legislation as yet does not specically target electricity usage information. Utilities and related organizations will have to develop systems and procedures to protect the privacy of grid information so as to satisfy the concerns of customers and their governments. The complex issues involved are being actively debated in several U.S. states. Coordination across states will be necesmay/june 2012
sary to mitigate the concerns of companies that operate in multiple jurisdictions and the concerns of their customers, as data on both companies and their customers regularly cross state boundaries.
A Changing Workforce
Even if it faced none of the challenges discussed above, the electric power industry would need to rejuvenate its workforce in order to maintain current levels of performance. The challenge of an aging technical workforce, a problem made more serious by the decline in university power engineering programs, could have a signicant impact on the ability of the grid to meet the new challenges and seize the new opportunities described above. The IEEE U.S. Power and Energy Engineering Workforce Collaborative (PWC) has reported that approximately 45% of U.S. electric utility engineers will be eligible for retirement or could leave engineering for other reasons in the next ve years. While it is difcult to predict exactly how many new engineers will be needed between now and 2030, there appears to be a signicant gap between anticipated industry demands and both the pipeline of students entering power engineering and the faculty in place to train them. Fortunately, U.S. industry workforce challenges have received increasing attention in the past several years. Despite these efforts, this will likely remain an important area of focus in the years to come.
Conclusion
Due to a variety of recent policy changes and technical innovations, the U.S. electric grid will encounter signicant opportunities and challenges over the next several decades. As we have described above, various policy and system-level issues will need to be addressed and new technologies will need to be fully developed and used appropriately for the U.S. grid to evolve along an efcient path with minimal disruption and to ensure electricity rates and levels of reliability remain acceptable. The journey to the electric grid of 2030 has begun, and there will be plenty of surprises along the way. Much can and should be done now to smooth the road ahead.
For Further Reading

Massachusetts Institute of Technology. (2011). The Future of the Electric Grid. Cambridge, MA. [Online]. Available: http://web.mit.edu/mitei/research/studies/the-electricgrid-2011.shtml
Biographies
Timothy D. Heidel is with the Massachusetts Institute of Technology. John G. Kassakian is with the Massachusetts Institute of Technology. Richard Schmalensee is with the Massachusetts Institute of Technology.
p&e
37 29
Reprinted from November/December 2012 issue of IEEE Power & Energy magazine
DC, Come Home

By Brian T. Patterson
M
DC Microgrids and the Birth of the Enernet
60 30
ieee power & energy magazine magazine ieee
MOST DISCUSSIONS ABOUT AC VERSUS DC ELECTRICITY INCLUDE A RETELLING of the famous technical and commercial battle between Edison and Westinghouse/Tesla. Its a story about everything from electrocuting elephants at state fairs to the ambitious work of electrifying both urban and rural America. Its the tale of one of mans greatest engineering feats. It tells of a centralized power generation system based on the dominant use of incandescent light bulbs and ac constant-speed motors. In the end though, it is a retelling of historyand unfortunately, it is a history that doesnt project well into the future. This article is about making history in the power world. Its about the rebirth of the earliest form of electrical powerdc powerand its potential to change the world once again. It is being reborn with the help of modern solid-state power electronics technology. The story is also about the work of EMerge Alliance (EA), a nonprofit open industry association that is creating and promoting new standards based on the contemporary use of dc technology for power generation, storage, distribution, and use. This quickly growing allianceit already includes more than 100 organizations from industry, government, and academiawas conceived by and is populated with thought leaders motivated by the need for a phase change in the way we think about electric power. EA was born into a world searching for ways to move away from its almost exclusive dependence on synchronous fossil-fueled centralized power generation and ac macro grid transmission and distribution toward a system that can adaptively and efficiently include highly distributed, native dc electrical power generation and storage and deliver it an evolved predominance of natively dc loads. In the end, its about a new energy network, or enernet. The future of civilized progress is increasingly underwritten by our use of electrons to do work. So their sourcing, distribution, and efficient use is as fundamental as it is critical to our continued existence on this planet. While seeking better and cleaner ways of collecting and returning energy to and from the environment, it should be fundamentally recognized that electrons play a valuable role in utilizing energy from sustainable sources that can be used to do the vast majority of the work we desire The members of EA propose an expanded use of hybrid ac-dc power systems that are more akin to todays adaptive and information-rich Internet than they are to yesteryears hard-wired partyline telephone system. The application standards they are creating include a family of application areaspecific dc microgrids that, when interconnected with the soon to be smart ac grid, will combine to form the aforementioned enernet. (This term was first used in a presentation made at the Massachusetts Institute of Technology (MIT) by Bob Metcalfe, the well-known inventor of Ethernet, a
Digital Object Identifier 10.1109/MPE.2012.2212610 Date of publication: 18 October 2012
1540-7977/12/$31.002012IEEE
november/december 2012
STOCKBYTE
key enabler of todays Internet.) Such a network should have the means to value and efficiently utilize electrons produced by small private or community-owned renewable generators on an equal footing with those served up by huge private or publicly owned and regulated utilities. This also liberates us from the constrained practice of required behaviors imposed
by the dominant use of highly regulated central power generation and one-way distribution and moves us strongly toward a more democratic, user-centric view that includes distributed local generation and multidirectional networked distribution and use. Such systems are capable of reshaping the prevailing notion that quality of life around the world
61 31
Its about the rebirth of the earliest form of electrical powerdc powerand its potential to change the world once again.
will be constrained by the limits and harmful effects of our current electrical energy systems. It is a view that stimulates innovation and investment in a far more resilient and flexible network with far less impact on the environment in the short termand one that seeks harmony with it in the longer term. To accomplish this, EAs vision includes a system topology that links electrical elements starting at the chip level to electrical elements at the public utilitys generation plants and everything in between in different ways than they are currently configured. In this context, it thus redefines both the physical topology (how things are connected) and the logical topology (how things behave). The essential new physical ingredients of this vision include the concept of semiautonomous microgrids and the recognition that dc power is the technologically preferred form of electricity to be used within these grids. It seeks to minimize the wasteful impact of unnecessary power conversions and recognize that the increasing majority of new sources and uses of electrical power are, for the most part, natively dc or, at the least, are not constant-frequency ac and that they make use of dcbased power electronics. EA acknowledges the technical and social challenges certain to be raised during the pursuit of its vision. When it comes to energyand especially electrical energy and the marvel of our existing 100-year-old ac electrical energy systemmany are tempted to disown the challenge of creating a better future as represented by this more balanced vision of the role dc can play. But what EAs members envision is no more (or less) dramatic, demanding, or risky an undertaking than that associated with the recent transformations of our telephony, information, and computing systems during the creation of todays Internet. In some ways it should be a far less ominous transformational job, as the lessons learned from crafting the Internet are still fresh in our minds. The reward: an electricity network that can enhance business and personal economic growth and ecological well-being in a way that rivals the positive effects of the Internet. Even if one is not swayed by the desirability of the improved economics or ecology related to ZEBsthe U.S. government, for example, has called for all new commercial buildings by 2030 and 50% of existing buildings by 2040 to qualify as ZEBsperhaps with the addition of local power storage, the prospect of making buildings less vulnerable to technical and external threats to our national electric grid system is enough of a motivator. It should be noted that ZEBs are not necessarily islanded from the gridin fact, they are typically connected to the power network with the concept of having the grid provide back-up power supply in the case that a ZEB can not meet net zero energy for some reason. Also, during periods of excess generation to load at a ZEB, a point of interconnection to the grid is provided for the ZEB to sell back to the grid under certain circumstances. And its not just the federal government thats involved. Theres a large movement in the architectural and engineering community, called the 2030 Challenge, that is focusing on building and renovating our way to climate-neutral buildings by 2030. Many leading firms have already joined this effort, and it is supported by the American Institute of Architects. Trying to combine forces, the U.S. Department of Energy (DOE) has funded a Zero-Energy Commercial Buildings Consortium (CBC) to bring industry leaders, building owners, designers, and manufacturers together to identify the challenges and obstacles facing us on this path. The biggest aspect of this challenge is that we are not starting from scratch. We cant just concern ourselves with new buildings. Of all the commercial buildings that will exist in 2030, 85% are already built. So we need ways of taking existing buildings and improving their energy use dramatically. Some of these existing buildings are pretty old. And nationally, more than 95% of our building stock is small: under 50,000 ft2. In New York City, for example, the average age of commercial buildings is 50 years. In the mid-Atlantic region, nearly 50% are that old, and they tend to be small, less than 100,000 ft2. Many havent been renovated significantly, particularly for energy retrofits, in decades. Even in California, a bellwether state for energy efficiency, there are no efficiency standards for existing buildings. This is particularly problematic in office buildings, where 37% of all commercial electrical energy is consumed. These statistics indicate the challenge we face in transforming todays building stock. Fortunately, many individuals and groups are now beginning to focus on the challenge of existing buildings in terms of sustainability. One group is creating strategies for existing buildings in Philadelphia, where DOE has funded an
The Future of Zero-Net-Energy Buildings

The future starts today, so EAs vision is directly connected to the widely discussed contemporary goal of creating zeronet-energy buildings (ZEBs). ZEBs, at least in the context of this article, are buildings that cleanly generate enough energy on-site to equal the energy they use, thus creating a net zero balance at the building level. This further creates the opportunity to lessen the overall impact of energy generation on our economy, climate, and ecology.
62 32
We believe the dc-empowered enernet will be seen as the heart of whats coming: a new electric energy age.
innovation hub for existing buildings, originally called the Greater Philadelphia Innovation Cluster (GPIC) and now known as the Energy Efficient Buildings Hub (EEB Hub). Another effort, on the same campus as the EEB Hub, is the GridSTAR Center, another DOE-assisted program that is coordinated in part by the Penn State Center for Sustainability. Several common approaches to designing for low- or zeronet-energy buildings, whether theyre new or existing, are emerging from these and other similar efforts around the world. Lighting is often a primary target, both in terms of increasing day lighting and making the remaining electric light more energy efficient. And mechanical and heating, ventilation, and air-conditioning (HVAC) systems are seeing a range of new design strategies, including revised ventilation schemes, the use of new technologies like chilled beams and radiant panels, and the expanded use of variable-speed drive motors for pumps and air handlers. So-called smart building approaches add controls and building automation. Another focus is on-site power generation and storage, including using solar, wind, and other clean energy generation and more efficient power distribution throughout a building. In general, design strategies for new building and deep renovation projects are changing, with a growing focus on the 2030 challenge. It is also believed by a growing number of proponents that smart dc microgrids can help us make better use of the energy generated, stored, and used at a local level. Whether they are for new on-site energy generation (e.g., solar installations) or adding smart devices to monitor energy use or intelligently connecting power to electric vehicles and battery storage, such approaches give us added control of energy use at the building level, thus making buildings better partners with the nations smart grid efforts. They also provide a way to buy centrally generated energy at times of the day when it is more abundant, temporarily store it, and then use it during peak demand periods. DC microgrids interconnect a localized grouping of electricity sources and loads that predominately generates, distributes, and uses electrical power in its native dc form at low voltages (up to 1,500 Vdc) and operates either connected to the traditional centralized grid or functions autonomously as physical and/or economic conditions dictate. Such microgrids are typically connected to and operate in conjunction with ac macro grids to form a smart grid. The macro grids are typically utility-operated, centralized generation, wide-area transmission, and local distribution electricity grids that predominately use electrical power in its alternating current (ac) form at high and medium voltages (above 1,500 V) that otherwise require waveform, phase, and voltage synchronization for multiple power source interconnection. A pictogram of typical macro grid-to-microgrid interconnection is shown in Figure 1. Regarding the potential use of such dc microgrids, the DOE-sponsored Zero Energy CBC has reported that dc power may hold the key. The consortium cited dc power and dc microgrids as a next-generation technology and application that could fundamentally change the way we power commercial buildings. They noted that dc power can reduce or eliminate ac-to-dc conversions at the equipment and building level so that we can save more of the energy we need. But how much dc power is being used in commercial buildings? DC power is already used in most of the electronic devices youre familiar with and use in your everyday work environment, from smartphones to computers and printers to your iPad and even the lighting over your head. But it is also used in the racks and racks of equipment in data centers that support your information technology systems. And dc is fundamental to the variable-speed motor drives that help deliver your heating and air conditioning and to
magazine ieee power & energy magazine 63 33
Whats a DC Microgrid?
One of the least publicized but most significant ways a buildings design can change is in the way it is powered. Changing basic infrastructure has never been the glamorous part of any design challenge. But a buildings power infrastructure is one of the key facets linking building design and renovation to the national electrical smart grid effort. A new approach to the way we generate and use power in our buildingsusing an infrastructure called dc microgridsis linked to how we should make and distribute power at the national electrical grid levelthe macro grid. The use of microgrids is partly motivated by the increasing concern for the strain on and vulnerability of our electrical macro grid system. Witness the 2011 blackout in Southern California due to a utility workers mistake in Yuma, Arizona, and the blackout in the northeastern United States in 2003. And these are only the sensationalized events reported by the media; there are thousands of lower-level events, power disturbances, and failures recorded each day. These random disturbances and linear dynamic failures in the power delivery system are putting their own emphasis on creating independent, building-level power self-sufficiency via such microgrids.
The dc microgrid-enabled enernet vision represents a certain level of decentralization of the nations grid and is intended to facilitate the current smart grid overhaul.
system, the native dc power produced by the solar panels is inverted to ac power, Smart Transmission Distribution Generation just so it can be distributed in the building. Meters Then the ac power gets converted back to dc for specific device uses, such as lighting. This double conversion wastes even more energy. After these double conversions, Smart Buildings 15% or more of the solar energy generated Smart Grid is lost. Building The trend toward the use of dc Microgrids devices has been increasing for decades, Why Microgrids? and theres no end in sight. Data center Onsite Renewable Energy Generation growth alone approaches a compound Increase Renewable Energy Availability average annual rate of nearly 30%. The Improve Reliability and Security simple reality is that almost everything Improve Availability in Underserved Markets based on semiconductor electronics is Create Open Environment for Energy Innovation Local Energy Storage also based on the use of dc power, not ac power. In fact, Virginia Techs Center for Power Electronics Systems in Blacksfigure 1. Pictogram of macro grid-to-microgrid interconnection. burg estimates that more than 80% of the electricity used in office buildings the electric vehicles you drive, or are planning on driving, passes through power electronics and experiences one or to and from your buildings in the future. More and more of more conversions between ac and dc electricity. And yet we what uses electricity is utilizing solid-state and semiconduc- dont have comprehensive standards for how best to generate, distribute, and use dc power, the form of electricity tor power electronics based on dc. The challenge is this: for those dc devices to use the ac most of these devices need. Such standards could provide electricity that is delivered to them, they have to convert ac the opportunity to reduce or eliminate unnecessary power conversions. They would also help simplify and improve to dc. Simply put, these conversions waste energy. the reliability of the electronic equipment involved, reduce the waste generated when these chargers and converters The Plague of Wasteful are put into landfills, and help make the user experience Power Conversion A telltale sign of these wasteful conversions from ac to dc simpler by eliminating the many different adapter plugs are the ubiquitous power bricks and chargers cluttering our now necessary. Defining common interfaces and standards work spaces. Every time you plug in your laptop charger, for our dc devices at multiple building levels could help us youre converting the ac available in the building to the dc simplify how we use power while saving energy, offering power that your computer needs to run. The same thing the potential for 515% savings or more, depending on the applies to your smartphone and other personal electronic ac-dc conversions we reduce or eliminate. devices. When you feel these converters get hot, thats the energy lost in the conversion process. The amount of energy The Critical and Clarifying lost differs with various devices, but is generally 1025%. Role of Standards and Codes And whats worse, many of these converters consume nearly Standards and codes play critical roles in moving us toward as much energy when the associated device theyre attached improved energy use. Organizations such as National Fire to is off as when it is on. Protection Association (NFPA), Underwriters LaboraThere are other, less obvious ac-dc conversions going on tory (UL), National Electric Manufacturing Association in buildings. One is in the electrically ballasted fluorescent (NEMA), and newer ones (including EA) are working ceiling lights you see overhead. Another takes place within together and have established task groups to address critisolar installations. For example, in a typical photovoltaic (PV) cal issues for alternative energy, including dc microgrid
64 34 magazine ieee power & energy magazine november/december 2012
distribution systems and electric vehiPower Sources cle charging as well as dc distributed electricity storage, natively dc generation systems, and other new dc elecUtility Battery Fuel Cell trical uses. Model installation codes Meter Solar PV Wind Gen Set Storage Other such as the National Electrical Code (NEC) help assure safety and other Wind 380 Vdc important attributes of energy sysMPPT Contr. Converter tems; they therefore become critically important to energy use improveFacility Power Server and Common Distribution / Collector (380 Vdc Nom) Bus ments. These organizations have committed to addressing these new issues proactively and aggressively. Already, EV HVAC ICT Lighting Plug Electronic new sections have been added to the Data Center Charger Loads Desktop Loads Loads Loads 380 Vdc NEC to cover small wind turbine elec24 Vdc 380 Vdc 380 Vdc 380 Vdc 380/24 Vdc 24 Vdc trical systems and solar PV systems in ways that minimize any associated Electrical Loads safety risk. And for the next code cycle, hundreds of proposals have been submitted and are being configure 2. New microgrid power distribution topologies in buildings. sidered regarding alternative energy systems, new battery technologies for distributed energy standards for dc power distribution that can transport and storage, electric vehicle systems, fuel cells, and low-voltage distribute energy safely and effectively between new energy dc power distribution systems. sources and uses? What are the likely use cases? Product and system standards also play an essential role EA formallyand enthusiasticallytook on this chalis supporting the effective deployment of products for alter- lenge just three short years ago. Based in California, with native energy equipment and systems. Proactive develop- more than 100 member organizations that include national ment of the requirements for appropriate application, design labs, universities, manufacturers, UL, NEMA, and other and test requirements, code compatibility, and the definition industry liaisons, EA has been identifying and creating techof standardized product interoperability, system attributes, nology application standards that promote the safe and effiand usage outcomes are all a part of their clarifying roles. cient use of dc electricity for all types of applications within In the case of dc power distribution systems, UL and EA and around buildings. EA has set out to create open, nonprohave directly teamed up in a number of formal and informal prietary dc application standards in each of four key areas in ways to develop these much-needed standards. Combining buildings as well as dc microgrid standards that interconnect ULs extensive technical, research, and government collabo- all the pieces. Each application area is defined as a potential ration competencies with EAs group of visionary and moti- microgrid that can be implemented by itself, much the way vated leaders in industry has been essential in helping define you can buy a laptop computer and not connect it to a data the preferred alternatives for beginning the fundamentally network but still enjoy improved productivity. In this way, transformative national shift to native dc electricity genera- any or all of the subgrids can be opportunistically created tion, distribution, and use. And together with NEMA and in whatever order makes sense for either new or existing Electric Power Research Institute (EPRI), they have begun buildings. to lay the groundwork for North American and global harFigure 2 shows how the EA member organizations see monization activities. the potential for a larger common bus collecting and distribA good deal work has been done and yet more begun, uting dc power in buildings. It shows a common dc bus that while much work remains for standards organizations. can directly connect a variety of power sources such as solar, But thus far, many of the key standards organizations are wind, fuel cells, and rectified utility ac power, when needed, embracing the challenge before them. Maintaining this early to serve multiple electrical loadsat a number of different momentum and velocity in this regard is vital. dc voltages, high and low, throughout a building. The key application areas (shown in Figure 3) for standardization of dc power use in buildings include: Getting from Here to There The challenge in doing this, of course, lies in the details interiors and occupied spaces where lighting and conof defining whats needed. Both standards and ecosystrol loads dominate the need for dc electricity tem development rely heavily on use cases. What types of data centers and telecom central offices with their dcenergy generation should be used? What loads need to be powered information and communications technology addressed? How do we create a new architecture or new (ICT) equipment
november/december 2012 magazine ieee power & energy magazine 65 35
DC Interiors: The Occupied Space

But buildings are not designed by engineers concerned with energy use alone. They are principally designed by architects, who are also focused on how all aspects of their buildings will perform for the owners and occupants who are their clients. It is important to appreciate that it is not just about energy and energy efficiency but about effective and productive spaces for working, learning, healing, and so on. The sacrifice of good design simply translates into inefficiency of energy in a high order, i.e., poor productivity, under-utilized space, etc. Running a crane motor at a lower horsepower can be more efficient, providing the crane can still safely lift a prescribed load of the correct weight and articulation. The case for properly designed buildings is similar in principle. Electrical system design strategies in ZEBs that implement new standards for power distribution should also help meet a buildings overall goals. A pictogram of EAs dc standards as implemented in the building interior is shown in Figure 4. An example of an implementation of this standard is the headquarters of the U.S. Green Building Council in Washington, D.C. Another is the new Sustainability Resource Center (SRC) at the University of California, San Diego (UCSD), which was looking for innovation in green building strategies. As a leader in promoting new energy approaches and a regular user of the solar power already on campus, the center decided to implement direct dc distribution through a new array created just for this project. Figure 5 shows a solar array put in place for a new commercial interior. The goal was to use this clean energy source directly whenever it was available and not invert it to ac power, avoiding the typical 715% energy loss from the conversion process. The loads for the solar dc power were energy efficient but otherwise ordinary lighting and interior controls. This use of direct dc power led SRC to better-quality power and greater lighting efficiency. In fact, SRC won several awards and a U.S. Green Building Council Leadership in Energy and Environmental Design (LEED) Gold rating under commercial interiors (CI) for the project, which included an innovation credit for its high-efficiency dc microgrid. LEED has also started to recognize the importance of incorporating flexibility into interior design. Proposed 2012 credit areas include a specific credit focused on flexible design. Although the credit is currently envisioned as relating particularly to health care, the importance of design flexibility in many types of buildings is being more generally recognized.
Occupied Space
Data Centers
dc Power Microgrids
Building Services
Outdoor
figure 3. EAs key dc microgrid building application segments. outdoor electrical uses, including electric vehicle
charging and outdoor light-emitting diode (LED) lighting building services, utilities, and HVAC with variable-speed drive (VSD) and electronic dc motorized equipment. The thought leaders and major companies involved in this groundbreaking work of setting new power standards for buildings include power system and information technology networking leaders, lighting and building products innovators, and electromechanical and solar companies. The collective focus of leaders across technology and application areas has jump-started this broad effort, enabling it to quickly reach the kind of critical mass necessary to meet our building efficiency and security challenges. Much of the focus is on using clean, renewable power generation (in its native dc generating form), whether thats biofuel, solar PV, or wind, and on electrical power use, such as green IT and low-energy lighting schemes. Just as weve leveraged hybrid power systems for cars, we can leverage hybrid power systems for buildings. The transformational coexistence of both ac and dc systems will let us focus on existing buildings as well as new buildings. It also seems best to take a modular approach, as the timing of the opportunities to use hybrid power or dc power may differ in various areas and types of buildings. Some areassuch as data centersrepresent a significant potential for dc use when they are new, significantly expanded, or considerably updated. Otherssuch as interior lightingare already recognized as big energy consumers that can be updated area by area to use dc. Still otherssuch as plug loadsmay have to await standards for the conversion of existing branch wiring to reach all your small miscellaneous equipment uses.
66 36
ieee power & energy magazine magazine
DC Data Centers
There are flexible dc power design strategies for other spaces within buildings as well. Data and telecom centers are great candidates. Green data centers and green IT have become hot topics. Data centers are huge and growing energy users in buildings, and there are data centers in nearly every building, not just the huge server farms created for organizations like Facebook and Google. In fact, 99% of
ac Branch Power Interiors Optional Onsite dc Power dc Celling Grid Occupancy and Daylight Sensors 24 Vdc Bus Occupied Space Infrastructure: P1 = Ceiling Lights P2 = Walls AV Devices and Security 208-277 Vac HVAC Actuator
380 Vdc Bus
IT Wireless Access Device
Power Supply
P3 = Furniture Room Controls P4 = Floors P4 P2 P1 P3
P2
2011 EMerge Alliance
figure 4. Pictogram of the EA dc standard as implemented for building interiors.
dc Loads: Lighting and Controls dc Source: Dedicated Solar Array
figure 5. Lighting and controls on a solar-powered dc microgrid at UCSD. (Source: Armstrong World Industries.)
2012 november/december
ieee power power & & energy energy magazine magazine ieee
67 37
Optional Onsite dc Power 100600 Vdc Optional Onsite dc Generator and/or Storage
MPPT Optional Onsite dc Power
ac-dc Converter ac Input
dc ECC
B 380 Vdc Busway (or Cabling) A
ICT Racks Point of Common Connection dc UPS > ac Flow > dc Flow (Native) dc Flow (Converted) Physical Data Center Copyright EMerge Alliance. All rights reserved.
Optional Onsite ac Generator
figure 6. Pictogram of EAs dc standards as implemented in a data center.
all data centers are considered small. But they contain the majority of servers using power, according to EPRI. The challenge is that smaller data centers are operated in organizations that often dont have the internal resources to focus on best practices for power distribution and efficient energy use, as they are busy focusing on making sure the system performs the data management and processing work it is intended to do. But the U.S. Environmental Protection Agency (EPA) has estimated that 6 billion kWh of energy could be saved each year with only a 10% efficiency improvement in these data centers. Again, there are new application standards starting to appear for this dc power application. EAs technical standards group, led by EPRI and including such companies as ABB, Cisco, Delta, Emerson, Intel, Juniper, and others, is nearing completion of a new standard whose key elements are shown in Figure 6. While these standards are being finalized, leading organizations and institutions have started to implement prototype approaches for dc in data centers. These include Duke Energy, Lawrence Berkeley National Laboratory, and, once again, UCSDa national pioneer in new energy research and innovation.
68 38
In particular, the experience of Duke Energy is instructive for those interested in looking at dc data center design. Dukes is a typical small-to-medium-size data center. The owner has years of experience with ac-based data center systems. It worked with EPRI to set up a rigorous comparative study. A review of some highlights of the study follows; a full report is available on the EPRI Web site, along with a video that displays technical details. The big takeaway for Duke was a 15% increase in the electrical efficiency of this data center when running on dc. In its report, EPRI noted that average reductions for other smaller data centers could fall anywhere within a 1030% range.
Barriers: The Challenges of Increased DCUse in Buildings

The use of dc power is not without it challenges. These fall into five major categories: 1) lack of application and equipment standards for dc power distribution 2) lack of common understanding and basic application knowledge of building distribution-level dc
3) differences in safety and power protection device application 4) lack of a robust ecosystem to support the use of dc in building-level electrification 5) an unclear pathway for moving from ac-centric power distribution to dc-inclusive distribution schemes. The first three challenges are being addressed with increasing resources by such standards and trade organizations as EA, the European Telecommunications Standards Institute (ETSI), the International Electrotechnical Commission (IEC), IEEE, NEMA, NFPA, the Power Sources Manufacturers Association (PSMA), the Smart Grid Interoperability Panel (SGIP) of the National Institute of Standards and Technology (NIST), UL, and others. As awareness of and interest in the potential benefits of dc power use increases, so do the resources each of these organizations is willing to dedicate to resolving these challenges. Currently, each of the above-named organizations has a dedicated and clearly identified project or program addressing these needs. The fourth challenge, the lack of an ecosystem, is a classic chicken or egg issue. The power industry, following Darnell Research and Pike Research, has begun formally forecasting and tracking the ecosystem growth opportunity associated with dc microgrids. The numbers they are beginning to report suggest the egg is beginning to hatch. The fifth challenge, the transformational path forward, is perhaps the least clear of all. But EA, via its strategic plan, has plotted a path with a layered approach that allows the transformation to be opportunistic, especially with respect to transforming existing building stock. Dividing building power applications into blocks of subdistribution microgrids, the plan calls for a section-by-section approach over time. Each of the application standards lays out a subsection that can be converted if and when that part of the building is due for a renovation or updating for other reasons, so the cost of the transformation is largely offset by normal capital or leasehold improvement spending. While this means that a complete transformation may take decades, early adopters with fast-churning buildings could be done much sooner. This timing and approach is reminiscent of similar transformations seen with the Internet and with wireless telephony. The dc microgrid-enabled enernet vision represents a certain level of decentralization of the nations grid and is intended to facilitate the current smart grid overhaul. The dc microgrid changes the model from an almost exclusively centralized generation and distribution system of electrical power delivery to one that is significantly more flexible and accommodating of both new alternative sources of on-site electricity generation and storage and the new mix of loads that have increasingly become the norm. It better recognizes that future electrical loads will be even more electronic, more distributed, and more essential to our economy and way of life. By designing electric power systems that focus better on the needs of digital devices, we improve the networks in which they operate (both power and control) so as to benefit
fromor indeed requirethe operational duplicity that comes with efficient electrical storage devices such as batteries and capacitors. But extensive employment of dc microgrids will not happen without human intervention. The impediments to their full deployment, as outlined herein, must be dealt with. The standardization and ecosystem development work EA is doing with the help of others will continue in the areas of dc microgridsupported electric vehicle charging, building services (HVAC, water and waste pumping, compressed air, and so on), and the definition of dc microgrid and smart grid connectivity standards. And although this work can be viewed as disruptive unto itself, it is motivated by the desire to bring new order and logic to the very disruptive technologies it intends to serve and optimize The aggregated and continuously growing use of electronic data and telephony; electric vehicles; solid-state and electronically driven lighting, motors, and controls; and personal electronicscoupled with the increasing use of natively dc distributed clean-tech electricity production has already and hurriedly pushed us past a logical tipping point in the ac-dc electrical energy equation. Its a time for true innovation, not the reiterative extension of our past ways. For as surely as the digitally empowered Internet will be viewed as the heart of the information age, we believe the dc-empowered enernet will be seen as the heart of whats coming: a new electric energy age.
For Further Reading

H. Kakigano, Y. Miura, and T. Ise, Low-Voltage BipolarType DC Microgrid for Super High Quality Distribution, IEEE Trans. Power Electron., vol. 25, no. 12, pp. 3066 3075, Dec. 2010. C. Marnay and S. Vossos. LBNL/DOE Webinar: Direct DC power systems for efficiency and renewable energy integration [Online]. Available: http://efficiency.lbl.gov/news/ lbnl_doe_webinar_direct_dc_power_systems_for_efficiency_and_renewable_energy_integration_0 B. Nordman. What the real world tells us about saving energy in electronics. Lawrence Berkeley National Laboratory Symposium [Online]. Available: http://eetd.lbl.gov/ea/ nordman/docs/e3s_nordman.pdf P. Savage, R. R. Nordhaus, and S. P. Jamieson, DC Microgrids: Benefits and Barriers, Yale School of Forestry & Environmental Studies, 2010. K. Shenai and K. Shah, Smart DC mircro-grid for efficient utilization of distributed renewable energy, in Proc. of IEEE EnergyTech, Cleveland, OH, 2011, pp. 16. M. Ton, B. Fortenbery, and W. Tschudi, DC power for improved data center efficiency, Lawrence Berkeley National Lab, Report, Mar. 2008.
Biography
Brian T. Patterson is with Armstrong World Industries, p&e Lancaster, Pennsylvania.
69 39
Meet Global Modernization Experts at the Worldwide IEEE PES Innovative Smart Grid Technologies Conference Series
The vision of a modernized electrical delivery system The Smart Grid promises to revolutionize the production, delivery and use of electricity worldwide.
Experts around the world gather annually at the IEEE Power & Energy Society's global ISGT Conferences to discuss state-of-the-art innovations in smart grid technologies. Each of the ISGT conferences feature special sessions and tutorials on wide ranging topics related to grid modernization, including: Impact of Smart Grid on Distributed Energy Resources (electric cars, demand response, distributed generation, storage) Smart Sensors and Advanced Metering Infrastructure Cyber Security Systems (intelligent monitoring and outage management) Wide Area Protection, Communication, and Control in Energy Systems Power and Energy System Applications (generation, transmission, distribution, markets, operations, planning) Energy Management Systems (with applications to smart buildings and home automation) Smart Grid Devices and Standards And More...
Networking: Meet and speak directly with utilities, business decision

makers, industry leaders, regulators, and entrepreneurs working in grid modernization
together to explore ways to make smart grid a cost-effective proposition
Research: Noted academics and industry professionals come
Opportunities: International speakers report on real success stories

and pitfalls as well as current business opportunities in their region
Results: Learn about real-practice technology, deployment experience, and customer acceptance related to grid modernization
The ISGT Conferences present the very best of smart grid technology to the global community with events held annually in North America, Europe, Asia and every other year in Latin America. For information on future ISGT events, plus other PES conferences, events publications and membership, please visit www.ieee-pes.org.
40
Enhancing Grid Measurements

Wide Area Measurement Systems, NASPInet, and Security
CREATAS
D
DEREGULATION, MARKET TRANSACTIONS, CONGESTION MANAGEment, and the separation of functions have created increasing complexity that is making it difcult to maintain situational awareness and supervision of power system performance over large areas. Past reliability events (such as blackouts) have highlighted the need for better situational awareness and advanced applications to improve planning, operations, and maintenance. The deployment of a continentwide wide area measurement system (WAMS) is an important part of the solution to these complex problems, but it faces challenges with respect to communications and security.
By Rakesh B. Bobba, Jeff Dagle, Erich Heine, Himanshu Khurana, William H. Sanders, Peter Sauer, and Tim Yardley
1540-7977/12/$31.002012 IEEE
41 67
Wide Area Measurement System (WAMS)

In its recent book A Century of Innovation, the National Academy of Engineering listed widespread electrication rst on its list of the top 20 engineering achievements of the 20th century. Although the highly interconnected North American electrical power grid is rightly hailed as a great engineering feat, managing and operating it in a reliable and safe way remains a challenge that involves many complex technical tasks that must be accomplished at different time and geographic scales. Such tasks include continuous feedback control, protection and control mechanisms that operate every few milliseconds at substations, state estimators and contingency analysis processes that operate every few minutes, and generation dispatch decisions to bring power plants online or take them off-line based on load or expected demand. In earlier years, control areas were vertically integrated in all respects and acted as quasi islands responsible for ow control. The interconnections among control areas enabled emergency ow paths and occasional economic benets. Knowledge beyond control area boundaries was limited and often depended on slow point-to-point communications. Modern operations are far more complex, as reliability constraints require extensive congestion management with signicant economic consequences. Further, given that various parts of the system are owned and operated by many independent entities, reliable operation of the grid depends on those tasks being accomplished at a range of geographic granularities and with a high level of coordination among the various entities that manage and operate the grid. With the passage of the Energy Policy Act of 2005 in the United States, the Federal Energy Regulatory Commission (FERC) and North American Electric Reliability Corporation (NERC) have been given additional authority to regulate electric power entities to ensure reliable operation of the grid. Historically, the grid has been very reliable. While minor outages have been fairly common, large-scale and widespread outages have been rare, and most customer interruptions occur within relatively localized distribution infrastructure. An increasing demand for electricity has not been accompanied by increases in transmission capacity, however, putting growing pressure on the reliability and safety of the grid. Recent large blackouts and outages, such as the 14 August 2003 blackout in the Northeast and the 26 February 2008 outage in Florida, stand as evidence. The nal report by the U.S.-Canada Power System Outage Task Force on the August 2003 blackout pointed out that the job of maintaining the system reliably had become harder because of reduced transmission margins. The report recommended the development and adoption of technologies, such as WAMS, that could improve system reliability by providing better wide area situational awareness. Traditionally, sensor readings from substations in utilities are sent via a communication network to the supervisory control and data acquisition (SCADA) systems in the
42 68
local utility and exchanged regionally with other utilities and reliability coordinators using the Inter-Control Center Communications Protocol (ICCP). Typically, SCADA systems acquire sensor data every 24 s. Since the data are not time-stamped at the point of measurement or acquired synchronously, they do not capture the state of the system at a given moment in time. Rather, the data can provide a good estimate of the system state, assuming that the system is in quasi-steady state. While the grid operates in quasisteady state most of the time, increased stress on the system means that operators views of it must be more negrained and cover a wider area, moving across multiple organizations in order to improve the reliability and stability of the grid. A WAMS can be dened as a system that takes measurements in the power grid at a high granularity, over a wide area, and across traditional control boundaries and then uses those measurements to improve grid stability through wide area situational awareness and advanced analysis. Certain power system measurements cannot be meaningfully combined unless they are captured at the same time. An important requirement of a WAMS, therefore, is that the measurements be synchronized. A high sampling rate typically, 30 or more samples per secondis particularly important for measuring system dynamics and is another important requirement of a WAMS. Certain elements of a WAMS have existed in rudimentary forms in the Western Interconnection since the early 1990s, and the cascading outage of 1996 provided the impetus for further WAMS development. Many advanced applications can take advantage of the measurement capability provided by a WAMS, including: Wide area monitoring: High-speed, real-time measurement data and analysis are essential to achieve wide area visibility across the bulk power system for entire interconnections. Time-synchronized measurements from geographically dispersed locations throughout a large region enable better operational awareness of the real-time condition of the grid and allow operators to make better-informed decisions. Real-time operations: Real-time operations improve operators understanding of how to take advantage of the newfound visibility of grid dynamics, including interarea oscillatory modes and methods for damping and stabilizing frequency oscillations. Improved accuracy of models: Time-synchronized wide area measurements continue to be very valuable for improving the accuracy of planning models by precisely correlating simulation output with observed system behavior under a variety of conditions. Improved planning models enable better assessment of system behavior and will permit a more complete assessment of dynamic performance issues, such as disturbance response, voltage and frequency response, and stability performance.
It is crucial to secure a WAMS to ensure the availability and integrity of the data it carries since monitoring and control applications may rely on those data.
Forensic analysis: Synchronized measurement data
collected at high sampling rates are also helpful for forensic analysis of blackouts and other grid disturbances. Because the data are collected at high speed and are time-synchronized, their analysis can lead to faster and better understanding of precise sequences of events.
NASPInet
Phasor measurement units (PMUs), developed in the early 1990s, were among the rst devices that could monitor the grid in a synchronized way and produce coordinated phasor measurements, also known as synchrophasors. A GPS clock signal is the most commonly used mechanism for providing the time reference needed for synchronizing PMU measurements. Another distinguishing feature of PMUs, in addition to synchronized measurements, is their sampling rate, which ranges from 30 samples per second up to 120 samples per second in current implementations. Even the low end of that spectrum, 30 samples per second, is an order of magnitude higher than the sampling rate of SCADA systems, meaning that PMU devices are capable of measuring system dynamic performance in a manner that is not possible with traditional SCADA systems. Their synchronized monitoring and high sampling rate make PMUs the ideal class of monitoring device for a WAMS. Traditionally, PMUs were stand-alone devices, but today many devices such as relays and digital fault recorders (DFRs) also have the ability to produce synchrophasors at high sampling rates. Realizing the need for wide area measurement, monitoring, and control across the continent and the potential of synchrophasor technology to enable these functions, the U.S. Department of Energy (DOE), the National Electric Reliability Council (NERC), and a range of electric utilities and other organizations formed the North American SynchroPhasor Initiative (NASPI) in 2007. NASPIs vision is to improve power system reliability through wide area measurement, monitoring and control, and its mission is to create a robust, widely available and secure synchronized data measurement infrastructure for the interconnected North American electric power system with associated analysis and monitoring tools for better planning and operation and improved reliability. Realizing a continent-wide WAMS requires not only synchronized measurement but also a high-speed communication infrastructure that enables secure sharing of synchronized monitoring data among control centers. There is
an effort under way at NASPI to develop such an infrastructure, known as the NASPI Network (or NASPInet); it is being designed to be secure, standardized, distributed, and capable of supporting future needs. One of the key requirements for this communication infrastructure is that it must be able to support different classes of applications with varying levels of latency, accuracy, availability, message rate, and time-alignment requirements. For example, one class of applications, such as feedback control, places strict requirements on the latency, availability, and accuracy of data, while another class of applications, such as post-event analysis, values accuracy, availability, and sampling or message rate more than latency. The communication infrastructure should therefore be able to support different quality-of-service (QoS) classes for trafc and should be able to prioritize one class over another. Conceptually, as shown in Figure 1, NASPInet is made up of two components: the phasor gateway (PGW) and the data bus (DB). The PGW is envisioned as a utilitys or control centers sole point of access to the DB. It will let the utility or control center share its synchrophasor data and obtain synchrophasor data from other utilities or control centers. The idea is that the data sharing will follow a publish-subscribe pattern, according to which a gateway that wishes to share data will publish them so that authorized gateways may subscribe to the published stream and receive the data. Each PGW will need to manage QoS and administer cybersecurity and access rights for the data it is sharing. The DB is envisioned as a wide area network that connects all the PGWs and provides the associated services for basic connectivity, QoS management, performance monitoring, and cybersecurity.
NASPInets Cybersecurity Requirements and Challenges

It is crucial to secure a WAMS in order to ensure the availability and integrity of the data it carries, which in turn affect the reliability of the power grid, since monitoring and control applications may rely on those data. The core security goals of a WAMS are to ensure the availability, integrity, and condentiality of the data and the underlying computing and communication infrastructure. Furthermore, the data security should be ensured end to end, that is, from the time of data origination at the sensor to the time of use by a control or monitoring application. Achieving these security objectives is easier within a single organization (that is, from the measurement sensor to the control center owning or managing the sensor) than it is for an infrastructure distributed over
43 69
keys among entities, a certicate authority that issues digital NASPInet certicates that are trusted by members of NASPInet, or just a Data Bus simple secure and authenticated directory service in which entities like gateways can post their PGW PGW PGW public keys or digital certicates. Once an entity is authenticated, Utility A its authorization to access the data needs to be veried. Access Historian APPS control lists (ACLs) associated Historian APPS Other with data are often used to specUtilities and ify the list of entities authorized Monitoring PDC to access the data. In such a case, Centers authorization checking involves Monitoring Center 1 ensuring that the authenticated entity is listed in the ACL assoPMU PMU PMU ciated with the data. In addition PMUs to access control for data, which is enforced by the data owners PGW Phasor Gateway PDC Phasor Data Concentrator APPS Applications gateway, there must be an access control mechanism at the network level to limit access only figure 1. A continent-wide WAMS and NASPInet concept. to authorized entities. In the case of NASPInet, the network-level a wide area like NASPInet, which is envisioned as enabling access control is to be administered and enforced by the data sharing across organizational boundaries and helping DB function. to realize a continent-wide WAMS. Here we highlight the security requirements of NASPInet, the many security func- Integrity and Confidentiality tions and mechanisms needed to meet them, and the chal- of Measurement Data lenges of realizing them. When sending data to an authenticated and authorized entity, it is necessary to protect the datas condentiality and integrity. It is important to protect measurement data condentiAuthentication, Authorization, ality from malicious eavesdroppers because such data may and Access Control Owners of sensor data would not want anyone other than contain information sensitive for the market or reveal senauthorized data-sharing partners to gain access to their data. sitive information about the grid that could be exploited to Toward that end, they need to be able to ensure that an entity disrupt grid operation. Encryption primitives are commonly with which they are communicating is what it claims to be used to protect data condentiality. Similarly, it is important and that it is an authorized data-sharing partner. In other to protect measurement data integrity as inadvertent or maliwords, they need to be able to authenticate the entity with cious modication of measurement data could lead operawhich they are communicating and verify that it is an autho- tors or applications to make catastrophic decisions. A typical rized entity before they share their data. Similarly, a data approach is to use symmetric-key-based cryptographic mesreceiver may want to authenticate the entity from which it sage authentication (or integrity) codes to detect data tamis receiving data to make sure that the incoming data are pering and to ensure that only legitimate data are accepted for use. Another notion, closely related to data integrity legitimate. A naive strategy for authentication is to create an out- protection, is that of data origin or source authentication, of-band security and communication context and use it to which assures a receiver that data indeed originated at the establish communications and perform authentication. A entity from which the receiver was expecting data. As the more dynamic and scalable approach is desirable, how- symmetric key used to compute the cryptographic message ever, and could include leveraging a trusted third-party authentication code is shared only between the sender and service to establish trust and long-term cryptographic receiver, in a two-party setting (one sender to one receiver), keys among the WAMS entities, such as their PGWs. verication of a message authentication code assures the The third-party service could be a Kerberos-like service receiver that the data were not tampered with in transit and that helps establish long-term symmetric cryptographic that the data originated at the expected sender. Thus, in a
44 70
two-party setting symmetric-keybased cryptographic message authentication codes provide both PGW1 PGW1 data integrity protection and data Encrypt{Data, K1} origin authentication. Since a data owner might PGW2 PGW2 share the data with multiple entiPGWS PGWS Encrypt{Data, K2} ties at the same time, for efciency reasons, a WAMS should Encrypt{Data, KG} Encrypt{Data, Kn} support not just unicast or twoparty data sharing (that is, one sender to one receiver), but also PGWn PGWn multicast or multiparty data sharing (that is, one sender to multiple receivers). So multicast integSending Phasor Gateway Receiving Phasor Gateway PGW1 PGWS rity and condentiality issues must also be addressed. Whereas KG: Group Key K1 Through Kn: Pairwise Keys the cryptographic key is shared between two parties in two-party (a) (b) or unicast data sharing, in a multicast setting, the cryptographic keys used for encryption and figure 2. (a) Unicast versus (b) multicast data sharing. message integrity protection are shared among a group of entities. Support for multicast data sharing can make data sharing guarantees is to use digital signatures, which use asymmetric efcient, as shown in Figure 2. With support for multicast keys (public-private key pairs), instead of symmetric-keydata sharing, the sender only needs to encrypt the data and based message authentication codes. The data sender would compute the message authentication code once using the digitally sign the data using a private key. When the signature group keys; the sender then transmits the data only once, is veried as valid using a public key, which corresponds to using the underlying multicast primitive. In contrast, with- the private key and is distributed to all group members, the out support for multicast data sharing, a sender will have to receivers can be sure that the data originated at that sender, as encrypt the data and compute the message authentication only that sender had access to the private key used to genercode separately for each receiving entity, with different ate the signature. Unfortunately, digital signatures are expenkeys for each receiver; then the sender must transmit the sive in terms of both computation and communication, and data as many times as there are receivers, increasing both it is a challenge to meet real-time requirements when every measurement is digitally signed. communication and computation costs. Schemes to amortize the signature cost over multiple While multicast data sharing reduces communication and computation costs, it adds additional complexity for key measurements exist and could reduce the overhead associmanagement and for data origin authentication. Specically, ated with digital signatures. By denition, however, those in a multicast setting, when a symmetric-key-based message schemes provide data source authentication for a group of authentication code is veried as valid by the receiver, the measurements, and the group size must be picked carereceiver is assured that the data havent been tampered with fully to reduce the costs per measurement while providby anyone outside the multicast group. The receiver cannot be ing data source authentication at a meaningful granularsure, however, that the data originated at any particular mem- ity. Furthermore, loss of one or more measurements in the ber of the group, as the symmetric key used to compute the group might mean that the signature cannot be veried. cryptographic message authentication code is shared among To pursue this approach, it would be necessary to design all the multicast group members and any one of the members mechanisms to prevent or deal with loss of measurements. An alternative to schemes that rely on asymmetric-keyis technically capable of generating a valid message authentication code. As a result, the receiver may have to rely on based (or public-key-based) cryptographic primitives would other means of data origin authentication. In a secure, well- be schemes that use symmetric-key-based cryptographic congured, and well-monitored network, the receiver may be primitives but use time synchronization between entities to able to rely on the network layer to provide assurances about create the asymmetry necessary for data origin authentication. But such schemes often introduce a great deal of key the origin of data packets. One straightforward way to achieve data origin authenti- management complexity and, like the amortized signature cation in a multicast setting without relying on network-layer schemes, result in verication delays.
ieee IEEE power power & & energy energy magazine magazine
45 71
Nonrepudiation
Reliability coordinators and other regional entities may need to make decisions based on data from a WAMS that will have economic consequences for their members. They may be held accountable for those decisions, and they might have to defend them. They may therefore need to use an approach that not only protects data integrity but also prevents the data source or sender from denying having sent the data. In other words, they need a nonrepudiation property. Digital signatures are commonly employed to provide nonrepudiation. But as mentioned earlier, digital signatures are expensive in terms of both computation and communication, and it is difcult to meet real-time requirements when every measurement is digitally signed. While signature amortization schemes could perhaps be applied here as well, it is in general harder to provide nonrepudiation via alternative schemes that use symmetric-key-based cryptographic primitives but rely on time synchronization to create asymmetry.
delays in data delivery. That further complicates multicast security solutions, especially for message source authentication and integrity protection, and could suggest a need to deploy sophisticated key management solutions that allow timely source authentication of each data packet, utilizing symmetric-key solutions that are signicantly more efcient than asymmetric-key-based digital signatures.
Data and Infrastructure Availability

To ensure data availability, it is necessary to ensure the integrity and availability of the underlying computing and communication infrastructure. While a carefully thoughtout fault-tolerant design will help, such a design by itself will not be sufcient. As part of a critical infrastructure, NASPInet will be an attractive target and must be resilient against cyberattacks and intrusions by adversaries ranging from novices to nation-states. There should be mechanisms in place to protect against cyberattacks, to monitor for and detect cyberattacks and intrusions, and to respond to and recover from cyberattacks and intrusions in a timely manner. Network access control (NAC) is an example of a network-layer protective mechanism that prevents anyone other than authenticated and authorized devices and entities from accessing the measurement communication infrastructure. Secure logging, along with the associated auditing or monitoring functions, is an example of a mechanism that can help with investigation and recovery from intrusions. While it is clear that data and infrastructure need to be protected, the level and kind of protection depend on each situations relevant threat model and risk assessment. For instance, must data be kept condential from everyone other than the intended recipients, or is it sufcient to keep the data condential from anyone outside the measurement network? The latter scenario might require simpler multicast security solutions than the former. Furthermore, even if data must be kept condential from everyone besides the intended recipients, does the trust model assume that NASPInet organizations are honest, or does it assume that they are potentially malicious? The former scenario might lead to simpler security solutions than the latter. Likewise, depending on the kind of security services available from the underlying network layer or the level of trust in the underlying network layer, security solutions at higher layers such as the application layer may end up being simpler. For example, if the network layer is able to provide data origin authentication, then symmetric-key-based schemes may be sufcient to provide data condentiality and integrity protection, thereby reducing complexity at the application layer. The requirements, threat model, and risk must therefore be carefully analyzed, as they have major implications for the security design of the system, including the policies, components, and tools needed for an appropriate solution. That said, once a security solution has been deployed, it is far easier to relax its security requirements than to make them more rigorous.
Key Management
An important aspect of NASPIs network security solutions will be key management: the ability to generate, distribute, revoke, and update cryptographic keying material among NASPInet entities. The cryptographic keying material might be used to provide various security properties such as entity authentication, data condentiality or integrity protection, and nonrepudiation. Long-term cryptographic keys established between entities, either with the help of a trusted third party or using an out-of-band mechanism, are often used for secure distribution of keys for condentiality and data integrity protection. Key management is more complex in multicast settings than in unicast settings, as the cryptographic keys are shared among a group of entities. When the group composition changes (that is, when a member of the group leaves or a new member joins), group keys need to be updated in a timely manner. Existing group keys need to be revoked and new group keys distributed rapidly, without disrupting the real-time measurement streams. Furthermore, in a multicast setting, a sending gateway may have to maintain a group key for every data stream that it is sharing; that would not be necessary in a unicast or pairwise setting, in which a pairwise key between the sending and receiving gateways might be sufcient to protect all data shared between them. While multicast networks and their associated security challenges are common to several problem domains (such as audio and video conferencing, mobile ad hoc networks, and wireless sensor networks), NASPInet presents more stringent real-time requirements on data delivery. For certain control applications, latency requirements can range from ten to a few hundred milliseconds for continent-scale applications. Such real-time delivery requirements have a signicant impact on security solutions. As discussed above, it is not feasible to digitally sign every data packet for data source authentication and integrity protection, as the computation and communication overhead could lead to unacceptable
46 72
The NASPI community is making steady progress toward achieving the vision of a continent-wide WAMS.
NASPInet: Current Status and Future Directions
The NASPI community is making steady progress toward achieving the vision of a continent-wide WAMS. Through its Smart Grid Investment Grant (SGIG) awards, the DOE is investing signicantly in the deployment of hundreds of PMUs, along with the associated communications infrastructure, across the United States. The realization of the vision of a continent-wide NASPInet will not be trivial, however. It faces many challenges, both technical and business-related. Potential options for creating such a network range from leveraging the public Internet to leased multiprotocol label switching (MPLS) circuits to utility-controlled ber networks to completely isolated highspeed optical networks. Using the public Internet or other shared media poses QoS and security challenges. On the other end of the spectrum, one can provision and manage a completely isolated, private, high-speed network, but doing so could be prohibitively expensive and still retain security and QoS issues such as the identication of a trustworthy entity that would own and/or manage that network. In recognition of the challenges, many of the SGIG awardees with PMU projects are focusing on increasing PMU deployment and utilizing the data from those deployments at a regional level. The idea is to grow these regional systems into a continent-wide WAMS enabled by NASPInet in the future. As part of the ongoing PMU data infrastructure development and deployment efforts, several key cybersecurity requirements will be addressed. In the preceding section there was a progression from basic security and functional requirements to more advanced ones, e.g., from unicast security to multicast security and from data origin authentication provided or supported by the network layer to application-level data origin authentication. Correspondingly, solutions that address the basic requirements are less expensive and better understood than those that meet more advanced requirements. Since the infrastructure is at an early stage of development, there is an opportunity to carefully consider a wide range of threats and security requirements so that security solutions can be built in from the ground up. This will let the WAMS be realized as a resilient critical infrastructure that can withstand sophisticated, targeted cyberattacks. North American SynchroPhasor Initiative. (2009, May). Data bus technical specications for North American Synchrony-Phasor Initiative network . [Online]. Available: https://www.naspi.org/site/Module/Team/dnmtt/naspinet/ naspinet_databus_nal_spec_20090529.pdf North American Synchro Phasor Initiative. (2009, May). Phasor gateway technical specications for North American synchro-phasor initiative network. [Online]. Available: https://www.naspi.org/site/Module/Team/dnmtt/naspinet/ naspinet_phasor_gateway__nal_spec_20090529.pdf D. Novosel, V. Madani, B. Bhargava, K. Vu; and J. Cole. (2008, Jan.Feb.). Dawn of the grid synchronization . IEEE Power Energy Mag. [Online]. 6(1), 4960. Available: http:// ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4412 940&isnumber=4408514 A. G. Phadke and R. M. de Moraes. (2008, Sept.Oct.). The wide world of wide-area measurement. IEEE Power Energy Mag. [Online]. 6(5), 5265. Available: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4610295& isnumber=4610275 R. Bobba, E. Heine, H. Khurana, and T. Yardley. (2010, Jan.). Exploring a tiered architecture for NASPInet. Presented at the Innovative Smart Grid Technologies Conf. (ISGT) [Online]. pp. 18, 1921. Available: http://ieeexplore.ieee.org/stamp/ stamp.jsp?tp=&arnumber=5434730&isnumber=5434721 D. E. Bakken, A. Bose, C. H. Hauser, D. E. Whitehead, and G. C. Zweigle. (2011, June). Smart generation and transmission with coherent, real-time data. Proc. IEEE [Online]. 99(6), 928951. Available: http://ieeexplore.ieee.org/stamp/ stamp.jsp?tp=&arnumber=5768095&isnumber=5768087
Biographies
Rakesh B. Bobba is with the University of Illinois at Urbana-Champaign. Jeff Dagle is with the Pacic Northwest National Laboratory. Erich Heine is with the University of Illinois at UrbanaChampaign. Himanshu Khurana is with Honeywell Automation and Control Systems Labs. William H. Sanders is with the University of Illinois at Urbana-Champaign. Peter Sauer is with the University of Illinois at UrbanaChampaign. Tim Yardley is with the University of Illinois at UrbanaChampaign.
p&e
For Further Reading

G. Constable and B. Somerville, A Century of Innovation: Twenty Engineering Achievements That Transformed Our Lives. Washington, DC: National Academy Press, 2003.
73 47
Life with Power & Energy.
Life without it.
Join the IEEE Power & Energy Society and Dont be Left in the Dark
Your membership in IEEE PES enables you to: We help our members to be successful by providing:
Tackle broad-reaching challenges Become recognized as a thought leader by your industry peers Develop contacts that will prove useful throughout all stages of your career Be a part of the very active and engaged global PES Community
Up-to-date information on current trends and the latest technology Industry insight through Power & Energy magazine, technical reports and peer-reviewed publications Compelling programs and networking opportunities at our conferences and events Opportunity to meet, network and collaborate with local members via our vibrant chapters
Over 30,000 members of the IEEE Power & Energy Society recognize that their membership is an exceptional, cost-effective way to acquire the latest information about all aspects of the fast-changing electric power and energy industry. You can too, if you join us now!
To learn more about the IEEE Power & Energy Society, including the many other membership benefits, please visit www.ieee-pes.org.
48
IEEE Power & Energy Society
445 Hoes Lane
Piscataway, NJ 08854 USA
By Julie Hull, Himanshu Khurana, Tom Markham, and Kevin Staggs
BRAND X PICTURES & LUSHPIX
Staying in Control
Cybersecurity and the Modern Electric Grid
THE USE OF SUPERVISORY CONTROL AND DATA acquisition (SCADA) became popular in the 1960s due to the expense of manual monitoring and control and an increase in the complexity of the systems. The blackout of 1965 in the northeastern United States prompted the U.S. Federal Power Commission to urge passage of the Electric Power Reliability Act of 1967, which would have mandated closer coordination among regional coordination groups. The National Electric Reliability Council was formed in 1968. These events also drove the development of large energy management systems for transmission SCADA. Early SCADA protocols were built on electromechanical telephone switching technology. At that time, the goal of communications security was to ensure that
1540-7977/12/$31.002012 IEEE
49 41
There is an increasing amount of evidence showing that attackers are now focusing on control systems, operating with varying motivations and intentions.
the command got to the mechanism for control (this security was typically implemented through repetition). Subsequently, SCADA moved to digital communications, and the use of parity bits and checksums became prevalent for error checking and is still common today in the eld. Many protocols were in use; typically, each manufacturer created its own, and some end users did the same. The network architecture was typically hierarchical, with the substations isolated. In the 1980s, a number of groups began working toward a common set of standards for protocols. The introduction of master stations and RTUs necessitated local area networks (LANs) and wide area networks (WANs), both of which can utilize more than one linking technology (e.g., satellite, telephone, wireless, power line carrier, ber optics, or microwave) to connect RTUs to master stations. The RTUs typically perform actions requested by the master station and report out-of-bounds conditions; some also perform local control, logging, and reporting. This diversity of communication media and protocols has left its legacy in the eld and has made it difcult to secure the infrastructure. More recently, there has been a merging of the automation and business networks, with a linking of the automation WAN to the corporate network and, in some cases, an extension of these networks into customer sites. The use of intelligent electronic devices (IEDs) has also become common and has caused yet another shift in the communications architecture. Traditionally, the system was serial and hierarchical in nature: users communicated with the substation through an RTU or data concentrator (which then communicated with meters, relays, equipment, and so on), or users communicated directly with feeder devices (reclosers, switch controllers, and other equipment). With the advent of IEDs, there is much more networked information, which then ows up to substations and/or feeder devices using serial, direct-connect, wireless, and packet-switched circuits. The substation communication is often through a router on a LAN, along with the human-machine interface (HMI), data concentrator, equipment, and relays and may offer remote access to feeder-level devices. Figure 1 illustrates a typical architecture for modern SCADA systems. Since many electric grid systems are now built using traditional IT hardware and software, their attack surface is much larger, making them more vulnerable to cyberattack. With that in mind, deployed systems use a layered protection approach, with multiple levels of rewalls and demilitarized zones, as seen in Figure 1.
50 42
Even with this type of layered protection, the system is still vulnerable. The National Electric Sector Cyber Security Organization (NESCO) has published a white paper, DNS as a Covert Channel Within Protected Networks, that demonstrates DNS data exltration techniques that do not require direct connectivity to any external resource from the targeted device. An attacker can get information from the RTU out through the corporate rewall and create a communication path back to that device, highlighting the need to watch outgoing rewall data. There is an increasing amount of evidence showing that attackers are now focusing on control systems. They are operating with varying motivations and intentions, including cybercrime, extortion, and warfare. In the area of cyberextortion, for example, we have been warned for years about the increased cyberextortion being practiced on electric utilities in Africa, Europe, India, and Mexico, where criminals threaten to cut off power if they are not paid. In a recent paper published by McAfee and the Center for Strategic and International Studies (CSIS), In the Dark, Crucial Industries Confront Cyberattacks, 200 industry executives from critical electricity infrastructure enterprises in 14 countries were surveyed. The survey group was composed of IT executives in the energy, oil and gas, and water sectors whose primary responsibilities include IT security, general security, and industrial control systems. According to the paper, One in four survey respondents have been victims of extortion through cyberattacks or threatened cyberattacks. And it follows that once a criminal nds an avenue of attack that works, the attacker tends to use it again and expand the list of victims. Nation-states have also been accused of using cyberattacks on control systems; such intrusions include the Russian cyberattack on Georgias pipelines and the alleged 2007 Russian attack on Estonia. In Kenneth Geerss paper Cyberspace and the Changing Nature of Warfare, the author outlines the strategic reasons why cyberwarfare is on the rise with respect to the electric power sector, including the fact that the Internet is vulnerable to attack. Many may argue that the electric power system is not on the Internet. In many cases it is, however. Even more common is the scenario in which a device without a direct Internet connection is connected to the Internet at some point in its life cycle for software or rmware updates, conguration, or maintenance. Or the device may interface with another device (e.g., a laptop or USB drive) that has been on the Internet and carries an infection or malicious code.
The methods used for a cyberattack vary depending on the realizing secure control systems and some approaches that attacker and the motivation. Some attackers are physically might work. We discuss control system security in general able to access a site through local surveillance, by browsing and use the example of modern SCADA systems to illustrate wireless networks within close physical proximity or even certain ideas. Finally, we review some key ongoing efforts in by accessing the site physically as part of the cyberattack; the control system security area involving the U.S. governsome perform the entire attack from a computer that could ment, industry, and academia. be 10,000 mi away. In any case, typically the rst step is to gather as much information as possible through publicly available sources Legend (say, from the Internet). The Denotes Attack Point Internet can provide names, physical layouts, installed equipment, Note: There Are Many Attack data useful for social engineering, Vectors Not Noted on This Diagram, Including Drivers, Etc. and port scanning for other data. Internet After this reconnaissance, adversaries target specic components and systems using malware that Corporate exploits vulnerabilities to gain Workstation Server Firewall/DMZ access to the system. There are many attack vectors for obtaining Business/ access to a SCADA system, from Corporate Network a brute-force attack through the business network to intercepting Control Firewall Center nonencrypted communications and playing them back, either to mimic control actions or to mask from the operators view the control actions that are really being SCADA Server performed. Attacks can vary from the relatively simplesuch as that of the disgruntled former contractor who used existing privileges Front-End and gained access to the control Processor system of a sewage treatment facility in Australia, then ooded WAN SCADA the surrounding area with milFeeder Network lions of liters of untreated sewDevices ageto the Stuxnet worm, which was purportedly an attack on the Iranian nuclear industry using Network Network Local HMI Local HMI highly sophisticated malware and Interface Interface several zero-day vulnerabilities. Input/ In the rest of this article we Output look at cybersecurity objectives Points and properties and discuss methLAN ods for minimizing cyberattacks RTU RTU Equipment Remote as well as detecting and respondFeeder Monitor Access ing to attacks that do succeed. We Devices Input/ Input/ then describe some cryptographic Output Output Meters Relays protocols commonly used to realPoints Points ize desired security properties Subdivision A Subdivision B such as condentiality and integrity. With this background in mind, we explore the challenges of figure 1. Typical security architecture for SCADA systems.
51 43
What Are the Goals and Objectives of Cybersecurity?

Cybersecurity tools and techniques are aimed at achieving three primary properties, namely, condentiality, integrity, and availability (CIA). Condentiality is the property that ensures that only authorized entities have access to sensitive information. For example, electricity market data and transaction information are considered sensitive and should only be accessible to authorized market agents and not to other entities such as system operators. Integrity is the property that ensures that any unauthorized modications to data and information are detected. For example, an adversary should not be able to modify sensor data without detection. Availability is the property that ensures that critical systems and information must be available when needed. For example, communication networks supporting wide area measurement systems must be available to deliver data and information (e.g., synchrophasor measurements) even in the presence of malicious activity such as an adversary launching a denial-of-service (DoS) attack . For critical infrastructure such as the electric grid, availability and integrity are typically considered to be more important than condentiality. Other security properties of interest to control systems include nonrepudiation and privacy. Nonrepudiation involves assurances that a particular command or message was actually sent, as the receiving entity claims, and is typically realized using digital signatures. Privacy, as a special form of condentiality, refers to adequate protection of personally identiable information and functions so that only authorized entities have access to this data. For example, consumer energy consumption data need to be kept private as AMI systems are realized. Achieving these properties for all computing and communication systems supporting the electricity grid is a major research, development, deployment, and maintenance challenge. A common approach to achieving these properties is to design, develop, and deploy cybersecurity technologies for protection, detection, and response. Protection systems devise security components such as key management, authentication and authorization, and perimeter defense that help ensure the CIA properties against a range of
attacks. For example, encryption tools help provide condentiality, cryptographic message authentication tools help provide integrity, and redundancy helps provide availability. Secure software and hardware development techniques are also an essential form of protection. Given the complexity of todays systems, vulnerabilities are likely to remain after development that can be exploited by adversaries despite the use of advanced protection systems. To deal with this, detection tools observe network and system behavior to identify malicious activities and attacks. For example, intrusion-detection systems may look for malware signatures on the network. Finally, response tools are employed to enable administrators to deal with detected attacks and activities. For example, such tools may allow dynamic changes in rewall policies in order to limit information ow to and from adversaries to contain an attack. Collectively these protection, detection, and response systems create an ecosystem in which secure and trustworthy operations can be executed. Typically, these technical solutions are used in conjunction with appropriate training for people and the use of well-dened processes to form a comprehensive solution.
What Are Some Common Security Components?

Earlier, we discussed the three objectives of security, namely, condentially, integrity, and authentication. Cryptography is used to provide condentiality and integrity. The workhorse of secure communications systems is symmetric cryptography. This is often called secret-key cryptography because the keys, which are the same at both ends of the communications link, must be kept secret. These algorithms are frequently identied by the length of their keys, e.g., the 128-bit Advanced Encryption Standard (AES). They can be thought of as codebooks that take a block of input data and encrypt it in a unique way based on the secret key. Figure 2 illustrates how a symmetric cipher could be used to protect data moving from a control center to a substation. The process unfolds as follows: 1) The secret keys are generated, transported to the ends of the communications link, and loaded into cryptographic devices (often part of a larger computing device) so that they are only known to the authorized sender and receiver. If attackers are able to obtain a copy of this key, they could also decrypt the data, rendering the system insecure. 2) The senders plaintext message is then passed through the codebook algorithm, where it is transformed into ciphertext. The output of the codebook is a function of both the key and the plaintext. 3) The ciphertext is transmitted over the communication link. 4) An eavesdropper listening in on the communications is able to intercept the ciphertext, but without the key the eavesdropper cannot decrypt the data and recover
Plaintext Set 247 On Key Codebook Internet
Plaintext Set 247 On Key Codebook
Ciphertext k3>A+zLcb+ Eavesdropper
Ciphertext k3>A+zLcb+
figure 2. Symmetric key cryptography provides confidentiality.

52 44
ieee power power & & energy energy magazine magazine IEEE
the plaintext. Thus, the symmetric cryptography provides condentiality. 5) The receiver passes the ciphertext through the codebook algorithm in reverse, using the secret key. The output of the codebook is the original plaintext. Securely distributing the keys for symmetric-key cryptography is cumbersome, so asymmetric-key (also called public-key) cryptography, a newer form, is used to transport the secret keys and perform other types of authentication. Three common public-key systems are RSA, El-Gamal, and elliptic curve cryptography (ECC). The underlying mathematics of these algorithms are signicantly different. All three, however, have a private key used to encrypt or sign a message and a related public key used to decrypt or verify messages, as shown in Figure 3. The originator of a message (e.g., a control center) signs the message with its encryption key, which is kept private. It then distributes its public key to everyone, including potential attackers. The legitimate receiver (e.g., a substation) uses the public key to verify that the message indeed came from the claimed source. An attacker could also use the public key to verify the message. But if an attacker attempts to forge a message, the verify operation will fail. Thus, public-key cryptography can be used to provide integrity and nonrepudiation. Nonrepudiation lets a third party verify that a message came from the entity holding the associated private key. Public-key cryptography may also be used to provide condentiality for small messages (e.g., a key for symmetric encryption) by encrypting them with the public key and then having the intended recipient decrypt them with its private key. Hash functions, such as the Secure Hash Algorithm with 256-bit output (SHA-256), are used to produce a mathematical ngerprint of a message or le. The hash function takes in a le of arbitrary size (often quite large) and produces a xed-length output. Hash functions have the following properties: Given a le and its corresponding hash, it is very difcult to nd another le that will produce the same hash output. It is very difcult to produce two les that when hashed will yield the same hash output. The hash output may then be signed using asymmetric cryptography. The resulting signed hash lets a receiver check the integrity of a large le by recalculating the hash and comparing it with a hash signed with the private key of the sender. Certication authorities are organizations that verify the credentials of a user, device, or software and then use asymmetric cryptography together with a hash function to issue the entity a digital certicate (e.g., under the X.509 standard) that may then be used for authentication over a network. Public-key infrastructure, using certication authorities, hash functions, and of course public-key cryptography, is often used to build authentication and key management systems.
Cryptography is helpful in addressing many security issues. But the use of cryptography within the power grid is challenging for the following reasons: Legacy systems often lack the computing power and bandwidth necessary to support strong cryptography. SCADA systems often remain in the eld for years, making it impractical to support the newer, more computationally intensive algorithms required as the attackers computing power increases over the years. Cryptography often relies on random number generators with high entropy. Many embedded devices lack the means to produce good random numbers. The key distribution and revocation process can be labor-intensive and prone to errors. This is especially true when multiple organizations are involved in the process. Mistakes made in the key management process may reduce the ability to communicate, which affects availability. There are many other security functions used to enhance the integrity and availability of systems. Antitamper mechanisms are frequently used to protect hardware accessible to potential attackers (e.g., smart meters). These mechanisms deter the reverse-engineering of devices to recover cryptographic keys or rmware that would disclose how a device operates.
Why Is Cybersecurity for Control Systems Challenging?

There are several contributing factors that make cybersecurity of control systems a challenge. Three of these challenges are: the clash between the operations team and IT team cultures the porting of legacy control software to common offthe-shelf (COTS) platforms the long life cycle of control systems. The rst is a cultural issue. The SCADA system engineers are responsible for the conguration and operation of any process. This includes a requirement to assure that certain control systems, such as SCADA systems, are always available. In many cases, a control system is expected to
Open Breaker #3 Key Sign = Encrypt Internet
Open Breaker #3 Key Verify = Decrypt X Public Key Open Breaker #1
Attacker
figure 3. Asymmetric-key (public-key) cryptography can provide integrity.

ieee power & energy magazine IEEE magazine
53 45
Many control devices will require security devices in the network that act as compensating controls to assist in securing them.
operate a plant over periods of many years with no shutdown or reduction in product manufactured by that control system. This means that availability is one of the most important requirements for any control system. Todays modern control systems are built using open-standard IT technologies such as Microsoft Windowsbased computers and Ethernet networks that include commercial routers, switches, and rewalls. Because the SCADA system engineers are responsible for the operation of the process, they feel responsible for all of the equipment required to run the process. Because IT systems are now part of the equipment required to run the process, the IT department feels it is responsible for the IT equipment running that process. This leads to a clash between the IT department and the process engineering department. Among the factors contributing to this clash are items related to the management and maintenance of those IT assets. One example concerns the installation of security updates in the IT equipment. IT typically pushes out security updates shortly after they are available, and most security updates require a reboot of the computers being updated. These reboots are usually done at a time controlled by the IT department. A reboot of a control system computer can severely affect a process operators ability to operate a process safely, and so the process engineering team wants more control over when the updates are installed. Another example results from migration to Ethernet networks. Many modern control systems integrate the status of Ethernet components such as switches and routers into the overall system status displays. IT wants to manage and monitor the Ethernet equipment, and this can result in a loss of view of that equipment status to the SCADA operators. One way to sum up the clash is that IT is focused on the protection of the intellectual assets of the company while SCADA system engineering focuses on the protection of the physical assets and manufacturing capabilities of the company. The priorities of the two can easily conict, leading to a clash between the two organizations. Standards organizations such as the ISA99 standards development committee have recognized the unique security management needs of SCADA and control systems and are drafting security standards for those systems. The intent of ISA99s proposed standards is to complement the IT standards that already exist while addressing those areas that need special attention for control systems. The North American Electric Reliability Corporation (NERC), the successor to the National Electric Reliability Council, has also realized the need for standards for control systems that con54 46
ieee power & energy magazine IEEE magazine
trol the generation and distribution of power and has created the NERC-CIP standards, which help guide the owners and operators of critical SCADA power systems. The migration from proprietary control systems to open systemsbased control has also contributed to some of the challenges. The IT industry and the control industry have evolved at different rates. While the IT industry was moving to PCs and servers, the control industry was still producing proprietary systems on proprietary networks. The control industrys shift to open systems followed that of the IT industry by approximately seven years, and the control system industry is approximately that far behind in understanding how to develop and deploy secure systems. Many security issues that existed in IT systems six or seven years ago are now just starting to appear in control systems. One reason for this is that the way the migration of control systems to open systems occurred was to port as much of the proprietary software to open systembased platforms as possible. Because the proprietary control systems had an implicit trust in the communications among devices in those systems, very few checks were performed in the code. Once ported to an open system, an application or device may become compromised by invalid input. Control device protocols were also developed with implicit trust, meaning that as they were moved to Ethernet, there was no attempt to add such things as authenticated and authorized communications. Users of control systems expect them to last for a long time. It is not unusual for a control system to operate a plant for a period of 20 years or more. Most operators dont expect to have to change the control system during that period. This period far exceeds the life cycle of any modern piece of open-systems hardware or software. The IT industry has a turnover rate of new systems every three to ve years, while the turnover rate for control systems has traditionally exceeded 20 years. As the control industry evolves further, the turnover rate will have to decrease. This will be a signicant challenge for the industry as we move forward.
How Does One Design Secure Systems?

There are several steps that can be taken to design secure control systems. First, consider procuring components that were designed with security in mind. Designing with security in mind means, for example, that the vendor of those components can demonstrate that it has integrated a security development life cycle (SDL) into its development process. The SDL will include security steps at all phases of development. This means there are security requirements for the
table 1. Representative efforts in the area of best practices for control systems security. Type Organization Description DHS Title and URL Industrial Control Systems Joint Working Group (ICS JWG); Cross Sector Cyber Security Working Group (CSCSWG); IT Sector Coordinating Council (IT SCC); Communications Sector Coordinating Council (CommSCC) Cyber Attack Task Force (CATF) and several related task forces http://www.nerc.com/filez/catf.html Security guidelines: NERC 1300, CIP-002-1 through CIP-009-1 http://www.nerc.com/page.php?cid=2%7C20 http://www.nerc.com/docs/standards/sar/Draft_Version_1_Cyber_Security_ Standard_1300_091504.pdf NIST Special Publication 800-53, Revision 3 http://csrc.nist.gov/publications/PubsSPs.html
Organization with enforced standards
NERC
Publication
National Institute of Standards and Technology (NIST) SP800-53R3 NISTIR 7628 DOE-supported and industry-led roadmap DOE
Publication Publication
NIST publication on guidelines for smart grid cybersecurity Roadmap to Secure Control Systems in the Energy Sector http://www.oe.energy.gov/DocumentsandMedia/roadmap.pdf Office of Electricity Delivery and Energy Reliability; Control Systems Security; Cyber Security for Energy Delivery Systems (CEDS) http://www.oe.energy.gov/controlsecurity.htm National SCADA Test Bed (NSTB) http://www.oe.energy.gov/nstb.htm http://www.sandia.gov/ccss/home.htm Draft road map: http://energy.gov/oe/downloads/roadmap-achieve-energy-delivery-systemscybersecurity-2011 NIST Smart Grid Interoperability Panel, the Cyber Security Working Group (CSWG) http://www.nist.gov/smartgrid/ Open SG Security Working Groups Advanced Security Acceleration Project (ASAP-SG) Data and Communications Security; focused on security for protocols 608705, 60870-6, 61850, 61970, and 61968
Working Groups/ Research
Working Group
NIST Smart Grid Interoperability Panel (SGIP) UCA International Users Group OpenSG International Electrotechnical Commission (IEC) Technical Committee 57 Working Group 15 AGA 12
Working Group Standards/ Working Group
Standards
Cryptographic Protection of SCADA Communications Part 1: http://www.aga.org/our-issues/security/Documents/0603REPORT12.PDF Part 2, Performance Test Plan: http://cipbook.infracritical.com/book3/chapter8/ch8ref4.pdf Pipeline SCADA Security http://engineers.ihs.com/document/abstract/BPZBGBAAAAAAAAAA Security Requirements for Cryptographic Modules Power System Control and Associated CommunicationsData and Communication Security http://webstore.iec.ch/preview/info_iec62210%7Bed1.0%7Den.pdf Power Systems Management and Associated Information ExchangeData and Communications Security, Part 1 (there are seven parts, all of which can be found on the IEC Web site): http://webstore.iec.ch/preview/info_iec62351-1%7Bed1.0%7Den.pdf Standard for Intelligent Electronic Devices (IEDs) Cyber Security Capabilities IEEE Guide for Electric Power Substation Physical and Electronic Security Manufacturing and Control Systems Security http://www.isa.org/MSTemplate.cfm?MicrositeID=988&CommitteeID=6821 Trustworthy Cyber Infrastructure for the Power Grid http://tcipg.org/
Standards Standards Standards
API 1164 FIPS 140-2 IEC 62210
Standards
IEC 62351
Standards Standards Academic Research
IEEE 1686, IEEE 1402 ISA-99 Trustworthy Cyber Infrastructure for the Power Grid
55 47
This article provides an introduction to relevant cybersecurity concepts and issues pertaining to emerging modern electric grid systems.
product. Roles are dened for the conguration, operation, and administration of control systems. These roles should include privileges for each role and identifying how the device responds when a user attempts to perform an operation on the device that the user does not have privileges to perform. Providing a role with only those privileges necessary to perform the associated functions is commonly called least privilege. The device should be deployed with least privilege already congured, so that the end user or integrator does not have to perform any additional steps for the device to be secure. Many control devices will require security devices in the network that act as compensating controls to assist in securing them. When this is the case, the device specification should define the compensating control, how to configure it, and an explanation of why it is required. The device vendor should be following secure coding practices. Finally, the device vendor should have processes in place to respond to a security vulnerability disclosure if one ever occurs for its product. These are just some of the steps required. There are many good examples of SDLs available, including Microsofts Security Development Lifecycle, the Open-Web Application Security Project, and the Common Lightweight Application Security Process. Once components are procured, system integrators also need to have methodologies for developing and conguring control systems for end users. The system integrator is responsible for integrating all of the pieces that together form a control system. As a control system is integrated, it will consist of multiple devices connected to multiple areas of a process with multiple functions. A model for how a control system is to be congured and information is to ow within it exists within the international ISA-95 standard. This model provides a topology to be applied while designing and conguring a control system. This topology provides a natural defense-in-depth approach to help protect the more vulnerable components of a control system. In addition to ISA-95, the International Society of Automation (ISA) standards committees have formed the previously mentioned ISA99 standards development committee, which is developing the security requirements for industrial automation and control systems. The ISA-99 standards build on the reference models in ISA-95 and create security reference models for a typical SCADA system and a typical digital control system, the two classic types of control systems.
What Is Being Done to Secure Control Systems Today?

It is important to note that if the attackers and attack vectors are studied, a common set of high-ranking vulnerabilities can be created that will signicantly affect the success of the attack. There are many good studies that can be found on common vulnerabilities and recommendations. Here are several: Common Cyber Security Vulnerabilities Observed in Control System Assessments by the INL NSTB Program (November 2008, U.S. Department of Energy) Catalog of Control Systems Security: Recommendations for Standards Developers (June 2010, U.S. Department of Homeland Security; www.us-cert.gov/ control_systems) Common Cyber Security Vulnerabilities Observed in DHS Industrial Control System Assessments (July 2009, U.S. Department of Homeland Security). Many organizations and governments have spent millions of dollars and years worth of effort in studying and recommending good practices for control systems security. In addition, most vendors today are actively including security in the design of their products. Table 1 provides examples of representative work in this area, rather than an exhaustive list of the many activities currently taking place.
Conclusions
This article provides an introduction to relevant cybersecurity concepts and issues pertaining to emerging modern electric grid systems. We looked at the history of these systems, the objectives of cybersecurity, challenges in addressing security for control systems, common security tools and components, processes for designing secure grid systems, and some key efforts under way today.
Biographies
Julie Hull is with Honeywell ACS Research Labs. Himanshu Khurana is with Honeywell ACS Research Labs. Tom Markham is with Honeywell ACS Research Labs. Kevin Staggs is with Honeywell ACS Research Labs.
p&e
56 48
EW
IEEE Electrification Magazine

Join PES by July 3, 2013 and receive the first issue FREE
Limited Time Offer. First Issue: $0.00
Launching in 2013!
www.ieee-pes.org/electrification
IEEE Electrification Magazine is the only publication dedicated to disseminating information on all matters related to microgrids onboard electric vehicles, ships, trains, planes and off-grid applications.
Published quarterly starting in mid-2013, each issue will provide:
News, analysis and insights on electric vehicles, electric ships,

electric trains and electric planes
Feature articles that allow you to stay current and connected to the
in the field of advanced electrification
challenges and opportunities for electrification in remote parts of the world
Access to comprehensive, in-depth technical analysis from engineers Industry insights, public sector programs and case studies on electric transportation
This is a limited time offer. To ensure you are among the first to sample this exciting new publication, be sure to join the IEEE Power & Energy Society by July 3, 2013 and we will send you the first issue absolutely FREE.
There are a lot of exciting things to come from IEEE Electrification Magazine, so be sure to join PES now and reserve your issue TODAY!
www.ieee-pes.org/electrification

Ieee Pes 2013 Smart Grid Compendium

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ieee Pes 2013 Smart Grid Compendium

Uploaded by

Copyright:

Available Formats

Smart Grid:

Challenges & Opportunities

IEEE Power & Energy Society

445 Hoes Lane

Piscataway, NJ 08854 USA

Smart Grid: Challenges & Opportunities

13 A Virtual Smart Grid

30 DC, Come Home

41 Enhancing Grid Measurements

ieee power & energy magazine

5 Smart Grid Safe, Secure, Self-Healing

IEEE Periodicals/Magazines Department

IEEE Power & Energy Magazine

ieee power & energy society (pes)

Standing Committee Chairs

IEEE Division VII Director

Chapter Committee Chairs

IEEE Division VII Director Elect

Membership & Image Committee Chairs

Governing Board Members-at-Large

E. Gunther, M. Jensen, J. Giri, T. Prevost

PES Executive Director

D. Nordell, Editor in Chief of Conference Papers

ieee power & energy magazine

Noel Schulz, President, IEEE PES

greetings from the IEEE Power & Energy Society (PES)

Noel N. Schulz IEEE PES President, 2012-2013 n.schulz@ieee.org http://www.ieee-pes.org/

IEEE Transactions on Smart Grid

Smart Grid Safe, Secure, Self-Healing

By S. Massoud Amin and Anthony M. Giacomoni

1540-7977/11/$31.002012 IEEE 1540-7977/12/$31.002012

Electricity Infrastructure: Increasing Interdependencies

Potential Route Ahead: A Smarter Grid

figure 1. A complex set of interconnected webs (source: EPRI, 2002present).

Key Smart Grid Security Challenges

13% 14% 11% 62%

Transmission All Others

Contagion Time Frame

Days Macro Viruses File Viruses

Saudi Arabia/ Middle East

figure 4. Cyberthreat evolution (source: EPRI).

Smart Grid Security Needs

denly increasing or decreasing the demand for power

ously shutting them down

Advanced Metering Infrastructure

Current Security Initiatives

Research and Development Needs

For Further Reading

A Virtual Smart Grid

ieee IEEE power & energy magazine magazine

The Overall Design of GridSim

Static Data Generator

figure 1. GridSim architecture.

Power System Simulation

Static Data Generator Generates Equipment in Each Substation

Communication System and Data Delivery

Control Center Applications

Read Data from PDC

The Oscillation Monitoring System

Poorly Damped Mode Detected? No

Alarm Controller Trigger

figure 4. Flowchart of an oscillation monitoring system.

figure 5. Illustration of analysis results from OMS engines.