On the hardware front, one of the biggest developments from the company is the announcement of a new version of the GH200 Grace Hopper platform, powered with next-generation HBM3e memory technology. The GH200 announced today is an update to the
existing GH200 chip
announced at the Computex show in Taiwan in May.
“We announced Grace Hopper recently several months ago, and today we’re announcing that we’re going to give it a boost,” Nvidia founder and CEO Jensen Huang said during his keynote at SIGGRAPH.
What’s inside the new GH200
The Grace Hopper Superchip has been a big topic for Nvidia’s CEO since at least 2021 when the
company revealed
initial details.
Nvidia also claims that HBM3e memory will allow the next-generation GH200 to run AI models 3.5 times faster than the current model.
“We’re very excited about this new GH200. It’ll feature 141 gigabytes of HBM3e memory,” Ian Buck, VP and general manager, hyperscale and HPC at Nvidia, said during a meeting with press and analysts. “HBM3e not only increases the capacity and amount of memory attached to our GPUs, but also is much faster.”
Faster silicon means faster, larger AI application inference and training
Nvidia isn’t just making faster silicon, it’s also scaling it in a new server design.
Buck said that Nvidia is developing a new dual-GH200-based Nvidia MGX server system that will integrate two of the next-generation Grace Hopper Superchips. He explained that the new GH200 will be connected with NVLink, Nvidia’s interconnect technology.
With NVLink in the new dual-GH200 server, both CPUs and GPUs in the system will be connected with a fully coherent memory interconnect.
“CPUs can see other CPUs’ memory, GPUs can see other GPU memory, and of course the GPU can see CPU memory,” Buck said. “As a result, the combined supersized super-GPU can operate as one, providing a combined 144 Grace CPU cores over 8 petaflops of compute performance with 282 gigabytes of HBM3e memory.”
While the new Nvidia Grace Hopper Superchip is fast, it will take a bit of time until it’s actually available for production use cases. The next generation GH200 is expected to be available in the second quarter of 2024.
VentureBeat's mission
is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact.
Discover our Briefings.
