Published on May 20th, 2019 📆 | 5434 Views ⚑
0What Colorado learned from treating a cyberattack like a disaster
The Colorado Department of Transportation joined the ranks of dozens of other U.S. government entities affected by the SamSam ransomware virus when it was infected with the malware in February 2018. While the incident was costly â nearly 2,000 computers, servers and network devices were encrypted, while the state spent about $1.5 million to undo the damage after refusing to pay the ransom â Colorado also created a new model for state and local governments dealing with cyberattacks in handling it like it would a natural disaster.
The decision by then-Gov. John Hickenlooper to declare a statewide emergency on March 1, ten days after the initial infection was detected, allowed officials to bring in resources from the National Guard and other states, create a unified command structure and perhaps most crucially, spare the stateâs IT workers from having to work any more 20-hour shifts fueled by junk food, said Kevin Klein, Coloradoâs director of homeland security and emergency management.
âWe switched from Doritos and Mountain Dew to actual food,â Klein said Tuesday at the National Governors Associationâs cybersecurity summit in Shreveport, Louisiana.
Klein also recounted for the audience of state IT and security officials how the SamSam malware infested CDOTâs network. In mid-February 2018, the department activated a new virtual server for testing, but the serverâs security software was still on its default settings, making it an appealing target when it started broadcasting its IP address to the rest of the internet.
âIt started broadcasting âIâm here, Iâm here, come attack me,â which of course happened within 48 hours,â Klein said.
Within a day, Klein said, the server was subjected to 40,000 brute-force attacks. A day after that, SamSam malware had found an entrance and used the serverâs administrative privileges to penetrate the rest of the CDOT network.
In total, the ransomware infected 1,274 laptops, 427 desktops, 339 servers, 158 databases, 154 software applications and all voice-over-IP phones used by CDOT at 200 locations across the entire state, Klein said. While the stateâs traffic operations were not impacted, the departmentâs internal business systems â including finance and payroll operations â had been knocked offline.
The first days after the attack were messy, as Colorado Chief Information Security Officer Deborah Blyth recounted to StateScoop last month, with teams from the state Office of Information Technology working around-the-clock and subsisting on pizza runs carried out by Blyth herself. Ten days in, with the malware starting to spread again, Hickenlooper signed his disaster declaration â the first time any state used one for a cyberattack.
The declaration reshuffled the response to the ransomware attack by bringing in Kleinâs office to coordinate emergency operations â including better catering and shift scheduling â and allowing Colorado to call on other states for assistance, which is common practice following a hurricane or wildfire.
Klein said the first task after Hickenlooperâs order was to establish ârecovery priorities,â starting with CDOTâs financial operations so the agency could make its next payday. Other priorities included protecting traffic operations by keeping those systems separated from the infected portions of CDOTâs network, and finally getting the department back to its regular operations. Now with several agencies responding to the incident â CDOT, OIT and the state emergency management office â they formed a unified command group and brought in more support from the National Guard, FBI and Department of Homeland Security. Workers who responded from other states helped re-image the large number of devices that had been taken out.
âSomebodyâs got to be in charge, and thatâs where the incident command structure comes into place,â Klein said. âPlanning priorities were based on consensus.â
Still, there were missteps as the state took this new approach, he said. Organizing communications among the unified command group proved more difficult than expected because of the addition of vendors, federal help and spokespeople from multiple state agencies talking to the media. Klein also said IT workers struggled to get a complete picture of the affected systems after discovering the state did not maintain an offline version of its network map.
And one provision in CDOTâs continuity-of-operations plan couldâve inadvertently made the crisis worse, Klein said, as it instructed workers to take their laptops to the Department of Public Healthâs headquarters, which could have exposed another agencyâs network to an infected device. Klein said one CDOT official told him the agencyâs continuity plan was more appropriate for a meteor strike than a cyberattack.
âWe had two people who did that and fortunately we stopped them before they could get there,â he said.
Despite the hiccups, the disaster approach proved effective. About 80 percent of CDOTâs systems were recovered within a month of the initial SamSam attack. Other governments hit by ransomware, including Alaskaâs Matanuska-Susitna Borough, have since issued their own disaster declarations, and many states are starting to incorporate simulated cyberattacks into their natural disaster drills.
âWe put a structure around it, just like any other incident,â Klein said.
Gloss