Emergency Restoration Training: When You Exercise, Go Big

We recently administered an emergency restoration exercise for a utility client and thought it might be good to share some of the situations and results with our readers. The client is a utility that has been blessed with generally benign operating conditions and few major disruptions. But to the organization’s credit, they have become increasingly concerned about the potential impact of unprecedented weather conditions and other challenges.

We based this exercise on the nature, intensity and effects that a very severe storm had on another utility. The storm was relatively recent, so unlike many disaster drills, the impact was still fresh in most of the participant’s memories, even though the actual event had little impact on the role players themselves.

Finding #1: Newbies Need Exercise

During debrief, the Incident Commander noted that a surprisingly large number of the participants in the Emergency Operations Center had not taken part in any of the previous annual exercises the company had run. Time and again, we read comments in the post-drill evaluations about how valuable the experience was for personnel who were new to the company, or new to their particular roles or responsibilities. Promotions, transfers, new hires or the basic need to fill gaps left by retirees create a need for something beyond training. Just as you would test a transmission line or system, people need to be pressure-tested before they can fully understand how they are to function in an emergency.

Finding #2: Open the Book

As we have noted in previous articles, the first impulse in an emergency is to dive in and “Git ‘er done.” Some teams came equipped with their emergency restoration plans in one form or another, and some did not. Even experienced players who had access to the plans failed to open them up to find the checklists, work guides, agendas, org charts or guidance they contained. Some teams started re-inventing wheels that had been well-rounded from previous experience and documentation. Our advice to team leaders everywhere, regardless of drill or event: Job One is Safety. Job Two is to remind all on your team to Open The Book.

Finding #3: Spread Them Out

By design, by accident or by necessity, the 90+ drill participants were housed in three fairly large, yet overcrowded rooms. While this provided an atmosphere of creative tension to the day, few appreciated it. Many debrief comments complained about lack of work space or difficulty in communicating within their own teams due to the noise and activity.

But sometimes, difficulty gives birth to inspiration. Many suggestions were made about how productive a full-functional exercise would be – one that would have participants based in their actual field locations. Would communications in such environs be improved, or would “out of sight, out of mind” locations create more or different challenges to teamwork if the unprecedented were to happen? There is only one way to find out before an outage forces you into it.

Finding #4: It Pays To Be Shifty

One of the contributing factors to the overcrowding and noise may have been the admirable desire to get second shift personnel the training and experience they need without making the exercise overlong. But shift changes carry with them a number of unique challenges, like situation analysis, handoff of issues that are hanging fire or in progress, unresolved community or media issues, and so on. In debrief, many participants recommended simulation of actual shift changes in future drills.

Finding #5: Think Ahead

The drill scenario concerned a storm that had been developing for several days before it hit the utility’s territory. Therefore, ahead of the drill itself, we provided our client with conditions as they would have occurred two days before the storm hit. Even with this foreknowledge, a few teams appear to have been less prepared than they should have been. Granted, this was “only a drill.” But the hints about what was coming were communicated to the teams days in advance. When the drill started, some teams were already behind the proverbial eight ball because they did not have all the resources they would need to do their jobs more easily.

Finding #6: Use What You’ve Got

In debrief comments, some participants asked for tools to be kept up to date with simulated events and issues as they were happening outside of their own teams. In real life, it does help to have some overall perspective of what’s going on elsewhere in the territory and the outside world as a whole. There are software providers who will fill this need, but sometimes simple file sharing or intranet email blasts will suffice. In this utility, the emergency operations center sported two large flat screen monitors that were noticeably dark throughout the drill – a potential tool that went unused.

Finding #7: Get Digital

Exercise controllers saw lots of pens, pencils and paper at work, busy copying down notes, conditions, data, all sorts of things. While it would be unwise to banish analog transcription from the emergency restoration system (even astronauts prepare for digital resource crashes by practicing with analog backups), the use of digital media has major advantages in speed, revision, duplication and retrieval. Try it; you might like it.

Finding #8: The Power of the Unprecedented

Perhaps the most revealing finding from this exercise is that some tools, resources and processes had to be invented during the drill. That’s because the scale of the scenario was unprecedented in the client utility’s experience. A decision was made early on in the drill to try to capture all the “new” stuff for inclusion in the emergency restoration plan as the basis for “Level 5” response. Smart move.