Inside LinkedIn’s Generative AI Cookbook for People Search

3 weeks ago7 min read6 comments

LinkedIn is finally launching its AI-powered people search this week, a development that arrives a full three years after ChatGPT's debut and six months following the platform's AI job search feature. For technical leaders observing this timeline, it illustrates a crucial enterprise lesson: deploying generative AI at scale—particularly for 1.3 billion users—is a brutal exercise in pragmatic optimization rather than a swift implementation of cutting-edge technology. The system represents a fundamental shift from keyword-based searches to semantic understanding.Where LinkedIn's legacy search would have stumbled over a query like 'Who is knowledgeable about curing cancer?' by merely scanning for the term 'cancer,' the new LLM-powered engine grasps conceptual relationships, recognizing that 'cancer' connects to 'oncology' and even 'genomics research,' thereby surfacing relevant professionals whose profiles might not contain the exact search term. More importantly, it balances relevance with practical usefulness, prioritizing accessible first-degree connections alongside world-leading experts, effectively building bridges rather than just listing names.The technical architecture behind this capability reveals LinkedIn's replicable 'cookbook'—a multi-stage pipeline of distillation, co-design, and relentless optimization. According to exclusive interviews with LinkedIn's product and engineering leadership, the breakthrough came after six to nine months of struggling with a single model approach.The team ultimately developed a 'multi-teacher' ensemble where a 7-billion-parameter 'Product Policy' model, trained on both real and synthetic data, was distilled into specialized smaller models—a 1. 7B teacher focused solely on relevance paired with separate models predicting user actions like connecting or following.This ensemble produces probability scores that a final student model learns to mimic through KL divergence loss, creating a two-stage pipeline where an 8B parameter model handles broad retrieval before a highly compressed 220M parameter student model performs fine-grained ranking. The scaling challenges were monumental—moving from job search across 'tens of millions of jobs' to people search across 'north of a billion members' required fundamental architectural shifts, including migrating retrieval from CPU to GPU-based infrastructure to maintain snappy response times.Further optimizations included training another LLM with reinforcement learning specifically to summarize input context, reducing input size twenty-fold and achieving a 10x increase in ranking throughput. Throughout this process, LinkedIn's philosophy emphasized perfecting recommender systems over chasing 'agentic hype,' with VP of Product Engineering Erran Berger noting that 'agentic products are only as good as the tools that they use.

#featured

#LinkedIn

#generative AI

#people search

#large language models

#enterprise scaling

#model distillation

#AI optimization

#AI cookbook