At Extreme Investor Network, we are thrilled to share the latest advancements in NVIDIA’s Blackwell architecture, which has set unprecedented benchmarks in the MLPerf Inference v4.1. This cutting-edge platform, unveiled at NVIDIA GTC 2024, features a superchip with 208 billion transistors and utilizes the TSMC 4NP process, making it the largest GPU ever constructed.
The performance of the NVIDIA Blackwell architecture truly shines in its debut on the Llama 2 70B LLM benchmark, achieving up to 4 times higher tokens per second per GPU compared to the previous H100 GPU. This remarkable improvement is attributed to the new second-generation Transformer Engine, which harnesses Blackwell Tensor Core technology and TensorRT-LLM innovations.
In the MLPerf results, Blackwell’s FP4 Transformer Engine executed approximately 50% of the workload in FP4, delivering a math throughput of 5.2 petaflops. The submissions based on Blackwell were in the closed division, showcasing unmodified models that met high accuracy standards.
Furthermore, the NVIDIA H200 Tensor Core GPU, an upgrade to the Hopper architecture, demonstrated outstanding performance across all benchmarks. With enhancements in memory capacity and bandwidth using HBM3e memory, the H200 showcased a 14% improvement in the Llama 2 70B benchmark solely through software enhancements in TensorRT-LLM.
Additionally, the Jetson AGX Orin platform exhibited significant advancements in edge AI, achieving up to 6.2 times more throughput and 2.4 times better latency on the GPT-J 6B parameter LLM benchmark. This platform is well-suited to run complex models like GPT-J and vision transformers at the edge, providing real-time insights from sensor data.
In conclusion, NVIDIA’s continuous innovation across its technology stack ensures it remains a leader in AI inference performance, from large-scale data centers to low-power edge devices. Stay tuned to Extreme Investor Network for more updates on the latest advancements in the world of cryptocurrency, blockchain, and technology.