The tech mogul's new AI training system claims the title of world's most powerful computing cluster with 100,000 Nvidia processors
In an email to colleagues, a sales lead at the chipmaker said the billionaire’s demand for its chips was straining its supply chain, the Wall Street Journal (NWS) reported, citing the email it viewed.
“We’ve worked hard to meet the needs of all customers and have greatly expanded the available supply today,” an Nvidia spokesperson said in a statement shared with Quartz.
In September, Musk announced that his AI startup, xAI, had brought its Colossus training cluster online after only 122 days. Colossus, which he called “the most powerful AI training system in the world,” is powered by 100,000 Nvidia H100 training chips.
Musk said the cluster “will double in size” “in a few months” to 200,000 chips, including 50,000 of Nvidia’s more powerful H200 units. While AI rivals including OpenAI and Meta $META (META) also have hundreds of thousands of Nvidia’s chips, Colossus has the most processors of an individual AI computing cluster in the world.
During an Oracle $ORCL (ORCL) investor event in September, Oracle co-founder Larry Ellison said he and Musk spent a dinner “begging” Nvidia chief executive Jensen Huang for more of Nvidia’s AI chips.
“Please take our money. No, no take more of it. We need you to take more of our money. Please,” Ellison said about the conversation. “It went okay. It worked.”
xAI’s supercomputer will be used to train the next generation of xAI’s large language model (LLM), Grok, which is meant to rival OpenAI’s GPT-4. In May, the AI startup raised $6 billion from heavyweight investors including Andreessen Horowitz and Sequoia Capital. The Series B funding round pushed xAI to a $24 billion valuation. Earlier this month, it was reported that xAI is set to raise another $6 billion at a $50 billion valuation.
Join 500,000+ readers who start their day with Quartz.
By subscribing, you agree to our Terms of Service and Privacy Policy.
