Image courtesy of Oak Ridge National Laboratory and NVIDIA Corp.
The new Titan supercomputer at Oak Ridge National Laboratory is now the world’s fastest, according to a Top 500 list released this morning.
A Cray XK7 system, Titan achieved 17.59 petaflops on a benchmark test, equal to more than 17,000 trillion calculations each second, a press release said.
Titan knocked Lawrence Livermore National Laboratory’s Sequoia from the No. 1 spot and into second place. Sequoia is an IBM BlueGene/Q system, and it was No. 1 in June with 16.32 petaflops.
Other systems in the Top 5 are Fujitsu’s K computer at the RIKEN Advanced Institute for Computational Science in Kobe, Japan (No. 3); a BlueGene/Q system named Mira at Argonne National Laboratory (No. 4); and a BlueGene/Q system named JUQUEEN at the Forschungszentrum Juelich in Germany (No. 5), which was upgraded and is now the most powerful system in Europe.
Titan is 10 times more powerful than Jaguar, ORNL’s last top supercomputer. It will be used for research in energy, climate change, engine efficiency, materials, and other disciplines.
Titan has 560,640 processors, including 261,632 NVIDIA K20x accelerator cores.
Sequoia has more than 1.5 million cores, and it is the first system with one million or more cores, the press release said.
The other new system in the Top 10 is Stampede, a Dell PowerEdge C8220 system installed at the Texas Advanced Computing Center at the University of Texas in Austin. It uses brand new Intel Xeon Phi processors to achieve its 2.6 petaflops, the press release said.
Today’s list was the 40th edition of the twice-yearly Top 500 list, and it was embargoed until 9 a.m. this morning. The results were scheduled to be announced during a supercomputing conference in Salt Lake City.
The press release said there are now 23 systems with petaflop performance on the latest list, just four-and-a-half years after the debut of Roadrunner, the world’s first petaflop supercomputer.
The TOP500 list is compiled by Hans Meuer of the University of Mannheim, Germany; Erich Strohmaier and Horst Simon of Lawrence Berkeley National Laboratory; and Jack Dongarra of the University of Tennessee, Knoxville.
More information can be found at www.top500.org.
Billythrowtheballatmyhead says
Yes…but can it play Crysis?
TJ Garland says
Partially financed by NSA as a decryption center for US internet traffic.
Big Brother needs to read your communications. I dare anyone in building 5300 to deny this.
Matilda Ramos Espinoza says
The Titan supercomputer is not located in building 5300; it’s in building 5700.
Titan will be used for research on grand challenges in energy, materials, climate change, and so on.
TJ Garland says
The plan was launched in 2004 as a modern-day Manhattan Project. Dubbed the High Productivity Computing Systems program, its goal was to advance computer speed a thousandfold, creating a machine that could execute a quadrillion (1015) operations a second, known as a petaflop—the computer equivalent of breaking the land speed record. And as with the Manhattan Project, the venue chosen for the supercomputing program was the town of Oak Ridge in eastern Tennessee, a rural area where sharp ridges give way to low, scattered hills, and the southwestward-flowing Clinch River bends sharply to the southeast. About 25 miles from Knoxville, it is the “secret city†where uranium- 235 was extracted for the first atomic bomb. A sign near the exit read: what you see here, what you do here, what you hear here, when you leave here, let it stay here. Today, not far from where that sign stood, Oak Ridge is home to the Department of Energy’s Oak Ridge National Laboratory, and it’s engaged in a new secret war. But this time, instead of a bomb of almost unimaginable power, the weapon is a computer of almost unimaginable speed.
In 2004, as part of the supercomputing program, the Department of Energy established its Oak Ridge Leadership Computing Facility for multiple agencies to join forces on the project. But in reality there would be two tracks, one unclassified, in which all of the scientific work would be public, and another top-secret, in which the NSA could pursue its own computer covertly. “For our purposes, they had to create a separate facility,†says a former senior NSA computer expert who worked on the project and is still associated with the agency. (He is one of three sources who described the program.) It was an expensive undertaking, but one the NSA was desperate to launch.
Known as the Multiprogram Research Facility, or Building 5300, the $41 million, five-story, 214,000-square-foot structure was built on a plot of land on the lab’s East Campus and completed in 2006. Behind the brick walls and green-tinted windows, 318 scientists, computer engineers, and other staff work in secret on the cryptanalytic applications of high-speed computing and other classified projects. The supercomputer center was named in honor of George R. Cotter, the NSA’s now-retired chief scientist and head of its information technology program. Not that you’d know it. “There’s no sign on the door,†says the ex-NSA computer expert.
At the DOE’s unclassified center at Oak Ridge, work progressed at a furious pace, although it was a one-way street when it came to cooperation with the closemouthed people in Building 5300. Nevertheless, the unclassified team had its Cray XT4 supercomputer upgraded to a warehouse-sized XT5. Named Jaguar for its speed, it clocked in at 1.75 petaflops, officially becoming the world’s fastest computer in 2009.
Meanwhile, over in Building 5300, the NSA succeeded in building an even faster supercomputer. “They made a big breakthrough,†says another former senior intelligence official, who helped oversee the program. The NSA’s machine was likely similar to the unclassified Jaguar, but it was much faster out of the gate, modified specifically for cryptanalysis and targeted against one or more specific algorithms, like the AES. In other words, they were moving from the research and development phase to actually attacking extremely difficult encryption systems. The code-breaking effort was up and running.
The breakthrough was enormous, says the former official, and soon afterward the agency pulled the shade down tight on the project, even within the intelligence community and Congress. “Only the chairman and vice chairman and the two staff directors of each intelligence committee were told about it,†he says. The reason? “They were thinking that this computing breakthrough was going to give them the ability to crack current public encryption.â€
In addition to giving the NSA access to a tremendous amount of Americans’ personal data, such an advance would also open a window on a trove of foreign secrets. While today most sensitive communications use the strongest encryption, much of the older data stored by the NSA, including a great deal of what will be transferred to Bluffdale once the center is complete, is encrypted with more vulnerable ciphers. “Remember,†says the former intelligence official, “a lot of foreign government stuff we’ve never been able to break is 128 or less. Break all that and you’ll find out a lot more of what you didn’t know—stuff we’ve already stored—so there’s an enormous amount of information still in there.â€
The NSA believes it’s on the verge of breaking a key encryption algorithm—opening up hoards of data.
That, he notes, is where the value of Bluffdale, and its mountains of long-stored data, will come in. What can’t be broken today may be broken tomorrow. “Then you can see what they were saying in the past,†he says. “By extrapolating the way they did business, it gives us an indication of how they may do things now.†The danger, the former official says, is that it’s not only foreign government information that is locked in weaker algorithms, it’s also a great deal of personal domestic communications, such as Americans’ email intercepted by the NSA in the past decade.
But first the supercomputer must break the encryption, and to do that, speed is everything. The faster the computer, the faster it can break codes. The Data Encryption Standard, the 56-bit predecessor to the AES, debuted in 1976 and lasted about 25 years. The AES made its first appearance in 2001 and is expected to remain strong and durable for at least a decade. But if the NSA has secretly built a computer that is considerably faster than machines in the unclassified arena, then the agency has a chance of breaking the AES in a much shorter time. And with Bluffdale in operation, the NSA will have the luxury of storing an ever-expanding archive of intercepts until that breakthrough comes along.
But despite its progress, the agency has not finished building at Oak Ridge, nor is it satisfied with breaking the petaflop barrier. Its next goal is to reach exaflop speed, one quintillion (1018) operations a second, and eventually zettaflop (1021) and yottaflop.
These goals have considerable support in Congress. Last November a bipartisan group of 24 senators sent a letter to President Obama urging him to approve continued funding through 2013 for the Department of Energy’s exascale computing initiative (the NSA’s budget requests are classified). They cited the necessity to keep up with and surpass China and Japan. “The race is on to develop exascale computing capabilities,†the senators noted. The reason was clear: By late 2011 the Jaguar (now with a peak speed of 2.33 petaflops) ranked third behind Japan’s “K Computer,†with an impressive 10.51 petaflops, and the Chinese Tianhe-1A system, with 2.57 petaflops.
But the real competition will take place in the classified realm. To secretly develop the new exaflop (or higher) machine by 2018, the NSA has proposed constructing two connecting buildings, totaling 260,000 square feet, near its current facility on the East Campus of Oak Ridge. Called the Multiprogram Computational Data Center, the buildings will be low and wide like giant warehouses, a design necessary for the dozens of computer cabinets that will compose an exaflop-scale machine, possibly arranged in a cluster to minimize the distance between circuits. According to a presentation delivered to DOE employees in 2009, it will be an “unassuming facility with limited view from roads,†in keeping with the NSA’s desire for secrecy. And it will have an extraordinary appetite for electricity, eventually using about 200 megawatts, enough to power 200,000 homes. The computer will also produce a gargantuan amount of heat, requiring 60,000 tons of cooling equipment, the same amount that was needed to serve both of the World Trade Center towers.
In the meantime Cray is working on the next step for the NSA, funded in part by a $250 million contract with the Defense Advanced Research Projects Agency. It’s a massively parallel supercomputer called Cascade, a prototype of which is due at the end of 2012. Its development will run largely in parallel with the unclassified effort for the DOE and other partner agencies. That project, due in 2013, will upgrade the Jaguar XT5 into an XK6, codenamed Titan, upping its speed to 10 to 20 petaflops.
Yottabytes and exaflops, septillions and undecillions—the race for computing speed and data storage goes on. In his 1941 story “The Library of Babel,†Jorge Luis Borges imagined a collection of information where the entire world’s knowledge is stored but barely a single word is understood. In Bluffdale the NSA is constructing a library on a scale that even Borges might not have contemplated. And to hear the masters of the agency tell it, it’s only a matter of time until every word is illuminated.
Terry Pfeiffer says
Nonetheless, it IS in building 5700; I’ve seen it.
John Huotari says
TJ,
Can you post a link to a story rather than a long excerpt? That will help keep the comments shorter and provide credit to the original author and publication. I assume you got this excerpt from the James Bamford story in the March 15 issue of Wired. Is that correct?
Regarding Titan, I’ve been told it’s an unclassified computer.
Thank you,
John
TJ Garland says
Will do.
And The Lord spoke unto Moses, go unto Pharaoh, and say unto him, thus saith The Lord, “Let my people go…” Exodus 8:1
TJ Garland says
Bldg 5300 construction
http://cryptome.org/2012-info/nsa-mrf/nsa-mrf.htm
Daniel Powers says
impressive. However, it is only a matter of time before it is number 2.