Groq is an AI inference company: its chip, the LPU (Language Processing Unit), is built to run trained models and generate their output quickly. The LPU keeps model weights in on-chip SRAM instead of external memory, and Groq sells access to it through GroqCloud, which it calls “the premier neocloud for fast inference.” Since a December 2025 licensing agreement, Nvidia also builds on Groq’s technology: the Nvidia Groq 3 LPX rack entered full production on August 24, 2026.

On this page

What the LPU does differently

Training a model and serving it put different loads on hardware. When a language model generates text, it produces one token at a time, and for each token the chip reads the model’s weights again. How fast those weights reach the compute units sets a ceiling on how quickly an answer appears. GPUs keep weights in high-bandwidth memory (HBM) stacked beside the processor: 80 GB on an Nvidia H100 SXM and 288 GB on a Rubin GPU, according to Nvidia. Groq’s design keeps them on the chip itself.

Groq describes three features of the LPU on its architecture page:

  • SRAM as main memory. “The LPU integrates hundreds of MB of SRAM as primary weight storage (not cache), cutting latency and feeding compute units at full speed.”
  • Compiler-scheduled execution. “Groq’s purpose-built compiler enables static scheduling and deterministic execution, ensuring predictable performance at every scale.” Static scheduling means the compiler fixes ahead of time when each operation runs and when data moves, instead of the hardware deciding while the model runs.
  • Chips that act as one. “LPUs connect directly via a plesiosynchronous protocol, aligning hundreds of chips to act as a single core.”

The first and third points go together. Hundreds of megabytes of SRAM per chip is much less than the tens or hundreds of gigabytes of HBM next to a data-center GPU, so a large model’s weights have to be spread across many LPUs working in step. Nvidia’s LPX rack, described below, links 256 of them.

GroqCloud

GroqCloud is Groq’s own inference service. Groq says it is “now operating 13 data centers across four continents,” with sites in the USA, Canada, Finland, the UK, Australia and Saudi Arabia, and that “millions of developers run trillions of tokens on Groq every week.” For enterprise customers it offers GroqAssured, which it describes as “enterprise-grade governance, auditability, and control,” and GroqRack infrastructure. AI data-center build-outs are listed in our AI data center tracker.

The Nvidia licensing agreement

On December 24, 2025, Groq announced that it had entered “a non-exclusive licensing agreement” with Nvidia for its inference technology. Under the announcement, “Jonathan Ross, Groq’s Founder, Sunny Madra, Groq’s President, and other members of the Groq team will join Nvidia” to advance the licensed technology. Groq said it would continue as an independent company with Simon Edwards as chief executive, and that “GroqCloud will continue to operate without interruption.” The companies said their shared aim is “expanding access to high-performance, low cost inference.”

Because the license is non-exclusive, Groq keeps the right to use its own technology. As of October 2026 the LPU reaches customers by two routes: Groq’s own GroqCloud, and Nvidia’s LPX racks.

Nvidia Groq 3 LPX

Nvidia announced the Groq 3 LPX on March 16, 2026 as part of its Vera Rubin platform: an inference accelerator rack with 256 LPU processors and 128 GB of on-chip SRAM. On August 24, 2026, Nvidia announced that Groq 3 LPX “is now in full production.”

Nvidia’s product page gives these figures:

Groq 3 LPX Per LPU Per rack
LPU accelerators 1 256
On-chip SRAM 500 MB 128 GB
SRAM bandwidth 150 TB/s 40 PB/s
Scale-up bandwidth 2.5 TB/s 640 TB/s

Nvidia positions the LPX alongside its GPU racks. It says Groq 3 LPX “extends the inference performance of NVIDIA Vera Rubin NVL72 systems by dramatically increasing token generation rates,” and that “Vera Rubin and LPX unite the extreme performance of NVIDIA Rubin GPUs and LPUs through a co-designed architecture.” In its August announcement, Nvidia said the system “delivered a record 3,400 output tokens per second” running Gemma 4 31B with a 100,000-token context, and that it enables agentic tasks “such as coding in minutes versus hours, providing 4x faster responsiveness for agents.” These are Nvidia’s own measurements and claims.

Nvidia named Nebius as “the first AI cloud to adopt NVIDIA Groq 3 LPX,” and said Groq “plans to be among the platform’s earliest adopters.” Nvidia’s Rubin-generation hardware and other accelerators are listed in our AI chip catalog.

Key dates

Date Event Source
December 24, 2025 Groq and Nvidia announce non-exclusive licensing agreement; Simon Edwards becomes Groq CEO Groq newsroom
March 16, 2026 Nvidia announces Vera Rubin platform including Groq 3 LPX Nvidia newsroom
August 24, 2026 Nvidia says Groq 3 LPX is in full production; Nebius first AI cloud to adopt it Nvidia

Profiles of Groq, Nvidia and other chipmakers are in our AI companies directory.

Sources