This is one outlet's own report from CNBC — the article as it was filed.
AIPROPX ReportCNBC · 2h ago
Nvidia says Groq racks will be online this year following $20 billion purchase
Nvidia announced on Monday that its Groq 3 LPX rack is in full production, marking the commercialization of technology from the company's largest acquisition on record.
The Groq rack will be deployed alongside Vera central processors and Rubin graphics processors at neocloud Nebius , and will be online later this year, Nvidia senior director Dion Harris told reporters.
Nvidia's race to manufacture Groq's chip and make it available to customers highlights the growing importance of low-latency inference that's needed to make AI agents feel responsive without long lags for users, especially for coding. Cloud companies can charge more for these kind of tokens, Nvidia says.
"For folks who are serving tokens, it unlocks the ability to offer premium tiers of service for those users and those customers who actually demand the most latency-sensitive" service agreements, Harris said on the call.
In December, Nvidia bought assets from chip startup Groq for $20 billion, the company's largest purchase..
The Groq architecture includes 500 megabytes of speedy SRAM on the chip's die itself to reduce memory-related bottlenecks. Groq chips are manufactured by Samsung, while Taiwan Semiconductor Manufacturing makes Nvidia's GPUs.
Nvidia packages 256 individual Groq 3 chips into its LPX racks. Nvidia said that its Groq 3 LPX rack can deliver 3,400 tokens per second, citing a benchmark from Artificial Analysis.
It's a competitive space. Smaller GPU maker Advanced Micro Devices announced earlier this year it would integrate its rack-scale systems with chips from Cerebras , which recently went public, focusing on low-latency inference. OpenAI's newly announced Ultrafast mode currently promises 750 tokens per second, and is "powered by Cerebras."
Low-latency chips don't replace the GPU, the workhorse of AI chips, which can do training as well as inference and are flexible enough to adapt to new technologies and models. Low-latency chips like Groq mainly focus on a part of serving models called the "decode" phase.
"This isn't about replacing GPUs," Harris said. "It's about using the right price, right processor for the right part of the workload,"
Nvidia is currently ramping up shipments of its Vera Rubin systems, which started production earlier this year. At the Vera Rubin and Groq 3 LPX unveiling in March , Nvidia CEO Jensen Huang projected $1 trillion in cumulative sales between the current-generation Blackwell chips and the new Vera Rubin systems, through 2027.
Huang said at the time he would allocate a quarter of data center space intended for coding applications to Groq chips.
"The rest of my data center is all 100% Vera Rubin," Huang said.
WATCH: Inside Nvidia's Vera Rubin AI system
Got a confidential news tip? We want to hear from you.
Get this delivered to your inbox, and more info about our products and services.
Data is a real-time snapshot *Data is delayed at least 15 minutes. Global Business and Financial News, Stock Quotes, and Market Data and Analysis.
AIPROPX is an independent multi-source news index — we track, compare, and connect coverage from across the web into one place you won't find anywhere else.
