Groq Raises $350 Million to Fund AI Inference Goals

Artificial intelligence (AI) inference company Groq has raised $350 million in new funding.

The Series A round will help Groq support customers “seeking usage of medium and larger sized clusters of Nvidia accelerated computing for training and inference,” the company said in a news release Monday (Aug. 17).

The round was led by tech investment firm Disruptive, with planned participation from Nvidia, which signed a $20 billion licensing deal with Groq last year.

“We look forward to continuing our partnership with Nvidia at such an important juncture for the ecosystem,” Alex Davis, Groq executive chairman and CEO of Disruptive, said in the release.

“Inference will without a doubt become the largest and most critical layer of AI infrastructure,” Davis added. “Our team has unmatched experience operating LPUs at scale and delivering the performance, efficiency and reliability that the next generation of AI demands.”

According to the release, Groq operates 13 data centers around the world, serving more than six million developers, Fortune 500 enterprises and thousands of “AI-native companies.” The company raised $650 million in June.

Nvidia last year acquired tech from Groq and hired several members of the company’s team, though Groq said it would continue to operate as an independent business.

As covered here last year, inference refers to a stage in which a trained AI model processes new data and generates results.

“When a customer service chatbot answers a query or an AI system analyzes a financial document, that is inference at work,” that report said. “While training creates the model by processing vast datasets to learn patterns, inference applies that learned knowledge to perform specific tasks at scale.”

As companies deploy AI systems that handle thousands or millions of requests daily, inference becomes the key operational challenge and cost driver.

Another PYMNTS report last fall examined inference and why, for most enterprises, it now matters more than training. Training a large language model happens just once or only occasionally. Inference takes place each time a user interacts with an AI system. A single model might manage millions of inference requests every month, each needing computational resources, adding both latency and costs.

“For companies running artificial intelligence in customer-facing applications, inference performance directly affects user experience, system reliability and operational expenses,” PYMNTS wrote.

The post Groq Raises $350 Million to Fund AI Inference Goals appeared first on PYMNTS.com.