LinkedIn Won’t Be Expanding Its Data Centers in the Next Year

The spending calculations apply to LinkedIn’s fiscal year that began last month and ends next June. The company says it was able to avoid spending big on AI hardware because it found ways to use its existing GPUs twice as efficiently over the past six months. LinkedIn’s plan could still unravel because the hardware demands of AI are shifting rapidly, but executives say the company has already taken into account surging prices for memory chips.

“One of the goals we've set is to try to basically keep our compute footprint flat or as close to flat as possible while shipping more compute-hungry things to production,” says Erran Berger, LinkedIn’s chief technology officer for engineering. “That’s a pretty bold statement to make in today's world.”

Berger and Raghu Hiremagalur, LinkedIn’s chief technology officer for infrastructure, say they want to be prudent about spending and that the new constraints will motivate engineering teams to get more creative when developing the many new generative AI features LinkedIn is planning to launch. Berger says he believes the efficiency gains could compound over time, enabling LinkedIn to get more out of data center expansions when it eventually increases its budgets again.

“I really want to double underscore that for a company of our scale, to say a full year we're going to do this with no incremental storage and compute is no small feat, but it's taken a ton of work to get there,” Hiremagalur says.

Companies such as OpenAI, Meta, and Google are scrounging up all the money they can find and coupling up in unexpected partnerships to construct, furnish, and operate massive data centers filled with the newest computer chips. Labor and parts shortages have held up many projects, and many businesses have had to limit customer usage of some AI tools. But there are also growing questions about whether the relentless investment in AI is sustainable. LinkedIn, with more than 1.3 billion users, is perhaps the largest business yet to publicly address spending concerns by bucking the building boom.

“It is encouraging for the industry,” says Songyee Yoon, managing partner of Principal Venture Partners and a board member at the server maker HP. “It suggests AI is beginning to move from experimentation into production discipline. The companies that win will not simply be the ones that spend the most on infrastructure.”

A few years after Microsoft acquired LinkedIn in 2016, the company tried moving to its parent company’s Azure cloud service, but it didn’t make economic sense to squeeze the giant social network into general-purpose data centers. “Microsoft Azure was growing like crazy, the level of customer demand was through the roof, and at the same time we saw skyrocketing growth on the LinkedIn side,” Hiremagalur says.

In 2022, LinkedIn went all-in on its own data centers in Oregon, Texas, and Virginia. The ownership gave LinkedIn significant control over every detail of its technology, setting itself up well to meet the realities of a new era. Around the same time, LinkedIn began developing AI-based assistants that could help users write messages, find jobs, and recruit candidates. The endeavor wasn’t cheap. “Every query that's coming to our site has increased in cost over time,” Hiremagalur says, adding that the amount of data LinkedIn stored was doubling annually. “That is not a sustainable place to be.”

LinkedIn moved to optimize its data center usage at every stage of the AI pipeline, from training models to serving them in response to user queries. Hiremagalur’s team developed measurement tools to understand how much compute and storage individual teams were using. It then set up a system to allocate projects to the computers in its data centers more efficiently so that they sat idle less often. “Our allocation efficiency and utilization of GPUs on the training side is the best that I have seen,” Hiremagalur says, describing usage at north of 95 percent.

LinkedIn also used techniques such as distillation to train smaller AI models from larger ones. For its job recommendation tools, a single model learned from two larger models to both identify relevant openings and predict which users were most likely to click on them. While the smaller model is more affordable to operate, Berger says it doesn't sacrifice on quality. “People are finding and discovering jobs that they were not successfully finding before, because the model is doing a really good job of understanding” their desires, he says.

The model used to select which posts to show users on their newsfeeds also was expensive to operate initially, but LinkedIn found a way to get it to run in “a reasonably cost-effective way,” Berger says. He says that the company made dozens of improvements, such as streamlining model training, reusing information from earlier recommendations, and better balancing workloads between CPUs and GPUs.

LinkedIn even reworked some of the foundational software on Nvidia processors to make them capable of handling tasks larger than they were designed for. It also rejiggered other software to run tasks on CPUs instead of Nvidia GPUs, which are pricier, more difficult to procure, and consume greater amounts of electricity. Altogether, LinkedIn estimates its efficiency work has saved about $24 million over the past 12 months, or the equivalent of roughly 1,100 GPUs running around the clock for a year.

The LinkedIn executives acknowledge that the savings don’t amount to much for a company that has $18 billion in annual sales. But Hiremagalur says “craft” matters too, and so does “the agility” created by freeing up resources. Engineers can take on their next project sooner and integrate more AI capabilities without having to increase LinkedIn’s computing footprint.

Berger contends that LinkedIn has been able to deliver better job and candidate results while still keeping a lid on costs and generating a financial return for Microsoft. “We should be able to deliver better qua...

Indexed and credited by AIPROPX. Originating outlet: Wired. Open at source →