Supermicro launches three NVIDIA based, full-stack, ready to deploy generative AI SuperClusters
Supermicro has announced its latest portfolio to accelerate the deployment of generative AI. The Supermicro SuperCluster solutions provide foundational building blocks for the present and the future of large language model (LLM) infrastructure.
The three powerful Supermicro SuperCluster solutions are now available for generative AI workloads. The 4U liquid-cooled systems or 8U air-cooled systems are purpose-built and designed for powerful LLM training performance, as well as large batch size and high-volume LLM inference. A third SuperCluster, with 1U air-cooled Supermicro NVIDIA MGXTM systems, is for cloud-scale inference.
Supermicro 4U NVIDIA HGX H100/H200 8-GPU systems double the density of the 8U air-cooled system by using liquid-cooling, reducing energy consumption and lowering data centre TCO. These systems are designed to support the next-generation NVIDIA Blackwell architecture-based GPUs. The Supermicro cooling distribution unit (CDU) and manifold (CDM) are the main arteries for distributing cooled liquid to Supermicro’s custom direct-to-chip (D2C) cold plates, keeping GPUs and CPUs at optimal temperature, resulting in maximum performance. This cooling technology enables up to a 40% reduction in electricity costs for the entire data centre and saves data centre real estate space.
The NVIDIA HGX H100/H200 8-GPU equipped systems are ideal for training Generative Al. The high-speed interconnected GPUs through NVIDIA NVLink, high GPU memory bandwidth, and capacity are key for running LLM models, cost effectively. The Supermicro SuperCluster creates a massive pool of GPU resources acting as a single AI supercomputer.
Whether fitting an enormous foundation model trained on a dataset with trillions of tokens from scratch or building a cloud-scale LLM inference infrastructure, the spine and leaf network topology with non-blocking 400Gb/s fabrics allows it to scale from 32 nodes to thousands of nodes seamlessly. With fully integrated liquid cooling, Supermicro’s proven testing processes thoroughly validate the operational effectiveness and efficiency before shipping.
Supermicro’s NVIDIA MGX system designs featuring the NVIDIA GH200 Grace Hopper Superchips will create a blueprint for future AI clusters that address a crucial bottleneck in Generative Al: the GPU memory bandwidth and capacity to run large language (LLM) models with high inference batch sizes to lower operational costs. The 256-node cluster enables a cloud-scale high-volume inference powerhouse, easily deployable and scalable.
SuperCluster with 4U Liquid-cooled System in 5 Racks or 8U Air-cooled System in 9 Racks
• 256 NVIDIA H100/H200 Tensor Core GPUs in one scalable unit
• Liquid cooling enabling 512 GPUs, 64-nodes, in the same footprint as the air-cooled 256 GPUs, 32-node solution
• 20TB of HBM3 with NVIDIA H100 or 36TB of HBM3e with NVIDIA H200 in one scalable unit
• 1:1 networking delivers up to 400 Gbps to each GPU to enable GPUDirect RDMA and Storage for training large language models with up to trillions of parameters
• 400G InfiniBand or 400GbE Ethernet switch fabrics with highly scalable spine-leaf network topology, including NVIDIA Quantum-2 InfiniBand and NVIDIA Spectrum-X Ethernet Platform.
• Customisable AI data pipeline storage fabric with industry-leading parallel file system options
• NVIDIA AI Enterprise 5.0 software, which brings support for new NVIDIA NIM inference microservices that accelerate the deployment of AI models at scale
SuperCluster with 1U Air-cooled NVIDIA MGX System in 9 Racks
• 256 GH200 Grace Hopper Superchips in one scalable unit
• Up to 144GB of HBM3e + 480GB of LPDDR5X unified memory suitable for cloud-scale, high-volume, low-latency, and high batch size inference, able to fit a 70B+ parameter model in one node.
• 400G InfiniBand or 400GbE Ethernet switch fabrics with highly scalable spine-leaf network topology
• Up to 8 built-in E1.S NVMe storage devices per node
• Customisable AI data pipeline storage fabric with NVIDIA BlueField-3 DPUs and industry leading parallel file system options to deliver high-throughput and low-latency storage access to each GPU
• NVIDIA AI Enterprise 5.0 software
Supermicro’s SuperCluster solutions are optimised for LLM training, deep learning, and high volume and high batch size inference. Supermicro’s L11 and L12 validation testing combined with its on-site deployment service provides customers with a seamless experience. Customers receive plug-and-play scalable units for easy deployment in a data centre and faster time to results.
