Down a brightly lit hallway in an Idaho Falls office building, Eric Whiting opens the door to a room humming with dozens of large, black computer racks.
Each rack resembles a high-tech refrigerator, some with neon green and black grating and others with glowing blue lights. The floor vibrates and cool air wafts up from grates in the floor.
The bulk of these racks make up Falcon — a supercomputer with 35,000 computer cores that can multiply 1,000 16-digit numbers a trillion times each second.
On a large touchscreen, Whiting, INL’s high performance computing and analytics manager, scrolls through a list of users. Some Falcon users commandeer up to 3,000 computer cores at a time to run advanced models and simulations that help design an array of complex and detailed energy systems and components.
Next, Whiting brings up a graph of the number of users who are waiting to use the computer. In the last year, the wait list has grown dramatically.
“It’s as if there was a room at Disneyland for 35,000 people and a queue of half a million others in line,” he says. “There are months when Falcon never catches up with demand.”
This long waiting list is just one reason why Falcon is fast becoming obsolete.
In the fall, Sawtooth, with more than approximately 100,000 computer cores, is scheduled to arrive. When Sawtooth is installed, it’ll be one of the top 100 fastest supercomputers in the world — five times faster than Falcon.
Sawtooth will be housed in the brand new 64,000-square-foot Collaborative Computing Center (C3), one of two new buildings at INL’s Research and Education Campus. Both C3 and the Cybercore Integration Center are under construction after the Idaho Legislature authorized funding to build them. The State Board of Education is overseeing the construction project and will be a close partner in the facilities once they are completed. INL will lease both from the state on long-term agreements.
The need for C3 is two-fold: First, there isn’t enough physical space for the anticipated 20 to 40 computer racks that will accompany Sawtooth’s installation, or the dozens of researchers and technicians who will maintain and operate the machine. Second, the current facility doesn’t have nearly enough infrastructure to handle the power required to run Sawtooth.
“Sawtooth will require at least one megawatt of power by itself,” Whiting said. “Without C3, I can’t even turn it on.”
In the fall, C3 will become the hub for modeling and simulation at INL, with connections to Idaho’s three research universities and access for collaborators around the world.
About a football field away from C3 is another new facility also under construction by the state of Idaho. The 80,000-square-foot Cybercore Integration Center will focus on bringing together the laboratory’s diverse portfolio of research that is increasing the resiliency of the computer control systems operating the nation’s critical infrastructures. Control systems are unique operational technology found in nearly every automated process from the electric power grid to water treatment plants and even transportation systems. As the internet has expanded and evolved, control systems have become vulnerable to exploits putting essential services like electricity and water delivery at risk to disruption.
“Security, particularly cybersecurity, can no longer be treated as an afterthought in critical infrastructure and our lives,” said Wayne Austad, Cybercore’s technical director. “As we start to deploy things like artificial intelligence, fifth generation (5G) wireless, cloud computing, and smart devices to our worldwide communication fabric, the impact of disruption becomes more significant. Cybersecurity is now fundamental across all our engineering, agriculture, business, and operational environments.”
Cybercore was designed to provide the space necessary for collaboration among government, industry and academia. It represents the future of control systems cybersecurity, which has evolved over the last 20 years from reacting to threats, to responding to them, to engineering new systems for resiliency to cope and adapt to modern cyberattacks.
Both C3 and Cybercore represent more than just space for computers and people. They are a forward-thinking partnership with the state of Idaho that will pay dividends for Idaho residents, students, and the laboratory far into the future.
Both facilities will connect to Boise State University, the University of Idaho and Idaho State University via the Idaho Regional Optical Network (IRON), an ultra-high-speed fiber optic network. The network will allow Idaho students the chance to develop modeling, simulation and cybersecurity skills and work on research projects either on site, or from their individual campuses. It will allow INL researchers and university faculty to collaborate on research proposals or joint classroom instruction. Collaborative spaces, offices and laboratories in C3 and Cybercore will increase opportunities for interns, graduate students and visiting faculty.
As the research grows, the connection to other universities and community colleges is likely to expand, too. In turn, the state, laboratory, and local communities will benefit from a highly talented pool of job applicants produced at the universities who graduate with relevant and immediately employable skills.
When INL and the state’s universities proposed construction of the two new facilities in 2017, they only needed to look next door for inspiration. Located to the north of C3 is the headquarters for Center for Advanced Energy Studies, a state-supported collaboration that opened its doors in 2009 to focus on regional energy research initiatives. Based on the growing need for skilled workers supporting nationally significant energy and national security work, the new construction projects received overwhelming support from the state’s elected officials who approved funding for construction.
“This project is good for the state, for its universities, for the lab, and for the people of Idaho, but mainly for a future generation of Idahoans,” said John Revier, INL’s director of state and regional government affairs. “These facilities are just one of the ways INL is reaching out to every corner of Idaho to develop the next generation of students who will be able to have long-term careers here, working on some of the nation’s most critical and gratifying research.”
