Groq has raised $640 million in a Series D funding round led by BlackRock, Samsung Catalyst Fund, and Cisco Investments, boosting its valuation to $2.8 billion—more than double its 2021 figure.
The funding comes on the heels of a significant chip supply deal with a Saudi Arabian firm and is a strategic move to challenge Nvidia’s dominance in the AI chip market.
Groq CEO Jonathan Ross revealed plans to supply 108,000 language processing units (LPUs) by Q1 next year, with the potential to increase production to 1.5 million units based on demand.
The company is partnering with Samsung Foundry to manufacture its AI inference chips, which are touted as faster and more energy-efficient alternatives to Nvidia’s processors. These chips will be produced at Samsung’s new factory in Taylor, Texas, using the advanced 4-nanometer process.
Oil money funds Nvidia rivals
After Google’s AlphaGo spotlighted the potential of AI in 2016, a wave of startups specializing in AI hardware emerged. Among them, Jonathan Ross, who played a key role in developing Google’s tensor processing units (TPUs) designed to accelerate machine learning, founded Groq. This period also saw the rise of other unicorns like Graphcore, Cerebras, and SambaNova.
However, having cutting-edge technology doesn’t guarantee market success. Despite their innovations, these startups struggled to secure substantial contracts in a market largely dominated by Nvidia. For instance, the British government chose Nvidia over the domestic startup Graphcore for the UK’s largest supercomputer project in Cambridge. Graphcore was eventually acquired by SoftBank last month after many of its engineers moved to Meta Platforms.
A significant opportunity arose in the Middle East, where oil-rich nations like Saudi Arabia and the UAE are investing in AI infrastructure to prepare for a post-petroleum future. In March, Groq secured a contract with Saudi Arabia’s state-owned Aramco, the world’s fifth most valuable firm with a market value of $1.7 trillion. Aramco is investing heavily in digital infrastructure as part of its transition strategy. Similarly, Cerebras signed a deal with UAE’s G42, a company linked to the UAE’s ruling family, to build up to nine supercomputers, each costing around $100 million. G42 plays a central role in integrating AI at a national level in Abu Dhabi.
The companies are actively addressing U.S. government concerns about the potential for AI-related technology and semiconductors to reach China via the Middle East. To mitigate these concerns, G42 has secured a $1.5 billion strategic investment from Microsoft, with Microsoft’s president sitting on G42’s board of directors. Additionally, both Cerebras and Groq manufacture their chips in U.S. facilities. In an interview with JoongAng Ilbo last year, Groq CEO Jonathan Ross highlighted the company’s decision to partner with Samsung Foundry over Taiwan’s TSMC, noting that Samsung’s U.S. factory was equipped to mass-produce 4-nanometer chips, aligning with their production needs and geopolitical considerations.
Samsung betting on HBM-free AI chips
Groq’s language processing units (LPUs) are specifically designed for AI inference rather than training large language models. These LPUs enable real-time interaction with AI models by using SRAM integrated into the chip, which offers speed but has limited data capacity compared to the high bandwidth memory (HBM) interfaces used in GPUs. SRAM’s capacity can be expanded by connecting multiple LPU chips, making it a more efficient and cost-effective choice for AI inference compared to training, according to Groq.
Nvidia’s dominance in the GPU market and the limited supply of HBM have led companies like Groq, Tenstorrent, and SambaNova to either eliminate HBM from their designs or develop specialized AI chips that minimize its need. Samsung Catalyst Fund has also invested in Tenstorrent and SambaNova. Despite these efforts, no startup has yet managed to significantly challenge Nvidia’s stronghold in the GPU sector.