Essays and notes on recommendation systems, search, relevance, and building AI products by Tullie Murrell.https://tullie.ai/en-ushttps://tullie.ai/blog/hnsw-explained-the-algorithm-powering-fast-vector-search/https://tullie.ai/blog/hnsw-explained-the-algorithm-powering-fast-vector-search/Hierarchical Navigable Small World (HNSW) is the approximate nearest neighbor algorithm that powers fast similarity search in most production vector databases. This post explains how HNSW works, what each parameter controls, where the algorithm breaks down, and how it fitsThu, 30 Apr 2026 00:00:00 GMTsearchretrievalmodelshttps://tullie.ai/blog/grep-vs-vector-db-retrieval/https://tullie.ai/blog/grep-vs-vector-db-retrieval/Keyword retrieval keeps winning in production for reasons that have little to do with benchmark leaderboard scores. This post explains when grep beats vectors, and why.Thu, 23 Apr 2026 00:00:00 GMTsearchretrievalagentshttps://tullie.ai/blog/ranking-architecture-part-5-feedback/https://tullie.ai/blog/ranking-architecture-part-5-feedback/Welcome to the final post in our series on the anatomy of modern recommender systems. Over the last four parts, we've deconstructed the online request path, following a user's request from the initial billions of items all the way to a final, ranked page.Mon, 02 Mar 2026 00:00:00 GMTrecommendationsarchitectureevaluationhttps://tullie.ai/blog/beyond-the-hashing-trick-the-math-of-scaling-to-100m-ids-in-production/https://tullie.ai/blog/beyond-the-hashing-trick-the-math-of-scaling-to-100m-ids-in-production/If you follow machine learning today, you’ve been told that tokenization is a solved problem. In the world of Natural Language Processing (NLP), we have Byte Pair Encoding (BPE) or WordPiece. These algorithms compress the infinite complexity of human language into a neat,Thu, 05 Feb 2026 00:00:00 GMTrecommendationsmodelsinfrastructurehttps://tullie.ai/blog/building-the-relevance-layer/https://tullie.ai/blog/building-the-relevance-layer/Why retrieval and relevance are becoming the most important infrastructure in modern AI products, and what we're building at Shaped to solve it.Thu, 15 Jan 2026 00:00:00 GMTrecommendationssearchrelevancehttps://tullie.ai/blog/why-i-built-a-database-for-relevance/https://tullie.ai/blog/why-i-built-a-database-for-relevance/After five years at Meta and three years building Shaped, I think relevance infrastructure should work like a database: declarative, composable, and fast enough for humans and agents.Thu, 11 Dec 2025 00:00:00 GMTrelevancesearchrecommendationsagentsinfrastructurehttps://tullie.ai/blog/modeling-behavior-as-language-the-next-era-of-recommendations/https://tullie.ai/blog/modeling-behavior-as-language-the-next-era-of-recommendations/A major shift is underway in recommender systems, moving from traditional Two-Tower and DLRM models to a new paradigm that treats user behavior as a language. This approach models a user's sequence of interactions, such as clicks and purchases, allowing Transformer-basedThu, 13 Nov 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/scaling-laws-beyond-llms-the-future-of-search-and-recommendations/https://tullie.ai/blog/scaling-laws-beyond-llms-the-future-of-search-and-recommendations/When people talk about scaling laws in AI, they usually mean one thing: language models. The empirical laws first quantified in Kaplan et al. (2020) showed that loss scales predictably as a power law with model size, dataset size, and compute budget. Train a biggerThu, 13 Nov 2025 00:00:00 GMTrecommendationssearchmodelshttps://tullie.ai/blog/ranking-infrastructure-part-1-serving/https://tullie.ai/blog/ranking-infrastructure-part-1-serving/Welcome to a new, hands-on series for builders. In our previous series, \"Anatomy of a Modern Ranking Architectures,\" we deconstructed the conceptual blueprint of the multi-stage ranking architecture. We followed the logic of a request from retrieval to scoring to the finalMon, 03 Nov 2025 00:00:00 GMTrecommendationsinfrastructurehttps://tullie.ai/blog/ranking-infrastructure-part-2-data-layer/https://tullie.ai/blog/ranking-infrastructure-part-2-data-layer/Welcome back to our series on the infrastructure of modern ranking systems. In Part 1, we designed the online serving layer: a set of decoupled, scalable microservices orchestrated by Kubernetes to handle real-time requests. We built the engine of our ranking system.Mon, 03 Nov 2025 00:00:00 GMTrecommendationsinfrastructurehttps://tullie.ai/blog/ranking-infrastructure-part-3-mlops/https://tullie.ai/blog/ranking-infrastructure-part-3-mlops/Welcome to the final post in our series on the infrastructure of modern ranking systems. So far, we've designed our high-performance online services and fueled them with a specialized data layer:Mon, 03 Nov 2025 00:00:00 GMTrecommendationsinfrastructureproductionhttps://tullie.ai/blog/ranking-architecture-part-1/https://tullie.ai/blog/ranking-architecture-part-1/If you look under the hood of recommendation systems at Netflix, YouTube, or Amazon, you won't find identical models, but you will find a remarkably similar architectural blueprint. This multi-stage ranking system is the industry's shared solution to a fundamentalMon, 13 Oct 2025 00:00:00 GMTrecommendationsarchitecturehttps://tullie.ai/blog/ranking-architecture-part-2-retrieval/https://tullie.ai/blog/ranking-architecture-part-2-retrieval/Welcome back to our series on the anatomy of modern recommender systems. In our first post, we established the multi-stage architecture as the industry-standard blueprint for balancing relevance, latency, and cost. We framed it as a system of cascading approximations,Mon, 13 Oct 2025 00:00:00 GMTrecommendationsarchitectureretrievalhttps://tullie.ai/blog/ranking-architecture-part-3-scoring/https://tullie.ai/blog/ranking-architecture-part-3-scoring/Welcome back to our series on the anatomy of modern recommender systems. In Part 1, we introduced the multi-stage architecture as a blueprint for balancing relevance, latency, and cost. In Part 2, we explored the Retrieval Stage, where we used an ensemble of strategies toMon, 13 Oct 2025 00:00:00 GMTrecommendationsarchitecturemodelshttps://tullie.ai/blog/ranking-architecture-part-4-ordering/https://tullie.ai/blog/ranking-architecture-part-4-ordering/Welcome back to our series on the anatomy of modern recommender systems. So far, we've deconstructed the core machine learning pipeline.Mon, 13 Oct 2025 00:00:00 GMTrecommendationsarchitectureevaluationhttps://tullie.ai/blog/youtube-semantic-ids-ctr-lift/https://tullie.ai/blog/youtube-semantic-ids-ctr-lift/TL;DR: The shift from massive embedding tables to generative retrieval with Semantic IDs is accelerating. YouTube's new PLUM framework represents the next evolution, using an adapted LLM and enhanced 'SID-v2' to achieve a +4.96% Panel CTR lift for Shorts in live A/B tests.Fri, 10 Oct 2025 00:00:00 GMTrecommendationsindustrymodelshttps://tullie.ai/blog/building-a-hackernews-for-you-feed/https://tullie.ai/blog/building-a-hackernews-for-you-feed/TL;DR: The HackerNews top feed felt stale, so I built a personalized For You feed in a weekend using Lovable and Shaped. See it at hn.shaped.ai.Tue, 23 Sep 2025 00:00:00 GMTrecommendationspersonalizationproductionhttps://tullie.ai/blog/vector-bottleneck-embedding-retrieval/https://tullie.ai/blog/vector-bottleneck-embedding-retrieval/DeepMind’s latest paper formalizes a long-suspected limitation of embedding-based retrieval: single-vector models cannot scale to combinatorial query complexity, no matter how large the dimension. The result reframes hybrid and multi-vector approaches, not as patches, but asMon, 08 Sep 2025 00:00:00 GMTsearchretrievalrelevancehttps://tullie.ai/blog/dual-flow-generative-ranking/https://tullie.ai/blog/dual-flow-generative-ranking/TL;DR: Meta's generative recommender (MetaGR) is powerful but slow. Researchers from Meituan and top universities just dropped DFGR, a dual-stream architecture that's 2x faster at training and 4x faster at inference, while also beating MetaGR and heavily-engineered industrialThu, 28 Aug 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/research-to-production-recommendations/https://tullie.ai/blog/research-to-production-recommendations/If you’ve ever tried to take a promising machine learning experiment from an offline notebook to a live A/B test, you know the pain. Weeks, sometimes months, pass between proving an idea works and actually seeing it in front of users. Internal handoffs, infrastructure gaps,Wed, 13 Aug 2025 00:00:00 GMTrecommendationsproductionhttps://tullie.ai/blog/renttherunway-dataset-deep-dive-into-fashion-fit-context-and-recommendation-challenges/https://tullie.ai/blog/renttherunway-dataset-deep-dive-into-fashion-fit-context-and-recommendation-challenges/Online fashion retail faces unique challenges, moving beyond simple preference prediction. Accurately recommending clothing requires understanding complex factors like fit, body type, and the context of use. The RentTheRunway (RTR) dataset emerges as a crucial and fascinatingTue, 05 Aug 2025 00:00:00 GMTrecommendationsdatasetshttps://tullie.ai/blog/where-matters-location-feature-engineering-for-search-recs/https://tullie.ai/blog/where-matters-location-feature-engineering-for-search-recs/Location is more than just coordinates, it’s a powerful signal for making search and recommendation systems more relevant. This post explores how proximity, regional preferences, delivery constraints, and geo-targeting can all be encoded into machine learning models throughMon, 04 Aug 2025 00:00:00 GMTrecommendationssearchfeature-engineeringhttps://tullie.ai/blog/lambdamart-explained-the-workhorse-of-learning-to-rank/https://tullie.ai/blog/lambdamart-explained-the-workhorse-of-learning-to-rank/LambdaMART is one of the most widely used algorithms in Learning-to-Rank, powering the ranking logic behind search engines, recommendation systems, and e-commerce platforms. By combining gradient boosting trees (MART) with metric-aware optimization from LambdaRank, itWed, 30 Jul 2025 00:00:00 GMTrecommendationssearchmodelshttps://tullie.ai/blog/average-popularity-are-your-recommendations-just-chasing-trends/https://tullie.ai/blog/average-popularity-are-your-recommendations-just-chasing-trends/Relevance metrics like NDCG and Precision@K are crucial for evaluating recommendation systems, but they don’t tell the full story. Two systems can perform similarly on these scores while exhibiting drastically different behaviors, one favoring only popular hits, the otherThu, 24 Jul 2025 00:00:00 GMTrecommendationsevaluationhttps://tullie.ai/blog/decoding-timestamps-time-based-feature-engineering-for-search-recs/https://tullie.ai/blog/decoding-timestamps-time-based-feature-engineering-for-search-recs/Timestamps hold far more value than just marking when an event occurred, they encode powerful signals like recency, seasonality, user lifecycle, and content freshness that can significantly boost the performance of recommendation and search systems. But unlocking theirThu, 24 Jul 2025 00:00:00 GMTrecommendationssearchfeature-engineeringhttps://tullie.ai/blog/goodreads-datasets-powering-book-recommendations-and-research/https://tullie.ai/blog/goodreads-datasets-powering-book-recommendations-and-research/The GoodReads datasets are a foundational resource for building and evaluating book recommendation systems. They combine explicit ratings, implicit feedback (like user shelves), rich textual reviews, and detailed metadata, making them ideal for hybrid models that mixTue, 22 Jul 2025 00:00:00 GMTrecommendationsdatasetshttps://tullie.ai/blog/peering-inside-the-black-box-leveraging-user-item-embeddings/https://tullie.ai/blog/peering-inside-the-black-box-leveraging-user-item-embeddings/Learn how user and item embeddings power personalized recommendations, similarity search, analytics, churn prediction, and custom ML models.Fri, 18 Jul 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/dlrm-feature-interactions-ranking/https://tullie.ai/blog/dlrm-feature-interactions-ranking/Deep Learning Recommendation Models (DLRMs) like Wide & Deep, DeepFM, DCN, and MaskNet have become essential tools for pointwise ranking in recommendation systems, where the goal is to predict the likelihood of user-item interactions such as clicks or conversions. TheseThu, 17 Jul 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/categorical-features-the-backbone-of-search-recs-engineering/https://tullie.ai/blog/categorical-features-the-backbone-of-search-recs-engineering/Categorical features like category, brand, and user ID are essential to search and recommendation systems, but transforming them into meaningful signals for machine learning is often more complex than it appears. This post explains how to handle categorical data effectively,Tue, 15 Jul 2025 00:00:00 GMTrecommendationssearchfeature-engineeringhttps://tullie.ai/blog/gowalla-dataset-understanding-location-check-ins-social-ties-and-mobility-patterns/https://tullie.ai/blog/gowalla-dataset-understanding-location-check-ins-social-ties-and-mobility-patterns/The Gowalla dataset, a historical benchmark from the now-defunct location-based social network, offers rich check-in and social graph data that has powered foundational research in Point-of-Interest (POI) recommendations, human mobility modeling, and social influence onMon, 14 Jul 2025 00:00:00 GMTrecommendationsdatasetshttps://tullie.ai/blog/catalog-coverage-are-your-recommendations-exploring-your-whole-inventory/https://tullie.ai/blog/catalog-coverage-are-your-recommendations-exploring-your-whole-inventory/While traditional recommendation metrics focus on individual user experience, Catalog Coverage measures the breadth of a system’s recommendations across its entire inventory i.e how much of the catalog gets shown to anyone at all. It’s a valuable diagnostic for spottingFri, 11 Jul 2025 00:00:00 GMTrecommendationsevaluationhttps://tullie.ai/blog/content-based-filtering-explained-recommending-based-on-what-you-like/https://tullie.ai/blog/content-based-filtering-explained-recommending-based-on-what-you-like/Content-Based Filtering (CBF) is one of the fundamental approaches to building recommendation systems. Rather than relying on the preferences of similar users, CBF focuses on the characteristics of the items a user has engaged with to suggest others with similar attributes,Tue, 08 Jul 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/movielens-dataset-the-essential-benchmark-for-recommender-systems/https://tullie.ai/blog/movielens-dataset-the-essential-benchmark-for-recommender-systems/The MovieLens dataset is one of the most widely used benchmarks in recommender systems, offering real-world, explicit feedback data for evaluating collaborative filtering, content-based, and hybrid recommendation models. This article explores why MovieLens remains a goldWed, 02 Jul 2025 00:00:00 GMTrecommendationsdatasetshttps://tullie.ai/blog/from-zero-to-relevant-solving-the-cold-start-user-problem/https://tullie.ai/blog/from-zero-to-relevant-solving-the-cold-start-user-problem/New or anonymous users often face irrelevant, generic content, hurting engagement from the very first visit. This article explores the cold start user problem in personalization and search systems, outlining common strategies like global popularity lists, rule-based segments,Fri, 27 Jun 2025 00:00:00 GMTrecommendationspersonalizationhttps://tullie.ai/blog/last-fm-datasets-unlocking-music-recommendations-through-listening-history-and-social-connections/https://tullie.ai/blog/last-fm-datasets-unlocking-music-recommendations-through-listening-history-and-social-connections/The article explores the significance of Last.fm datasets in developing music recommendation systems, highlighting their value as benchmarks for modeling implicit feedback, sequential listening behavior, and social influence. It breaks down what’s included in these datasetsFri, 27 Jun 2025 00:00:00 GMTrecommendationsdatasetshttps://tullie.ai/blog/privacy-first-personalization/https://tullie.ai/blog/privacy-first-personalization/This post introduces a step-by-step framework for building privacy-first personalization systems that earn user trust and support sustainable growth. It covers key strategies like data minimization, user control, edge processing, and satisfaction-based metrics—along with howTue, 24 Jun 2025 00:00:00 GMTrecommendationspersonalizationhttps://tullie.ai/blog/see-the-bigger-picture-image-feature-engineering-for-search-recs/https://tullie.ai/blog/see-the-bigger-picture-image-feature-engineering-for-search-recs/In visually rich digital environments, text and tags alone often fall short in powering relevant search and recommendations. This article explores how visual feature engineering, extracting embeddings from images using models like CLIP or ViT, unlocks deeper relevance byMon, 23 Jun 2025 00:00:00 GMTrecommendationssearchfeature-engineeringhttps://tullie.ai/blog/how-youtubes-algorithm-works/https://tullie.ai/blog/how-youtubes-algorithm-works/YouTube’s recommendation engine combines large-scale data processing, real-time feedback loops, and multi-objective optimization to deliver highly personalized video suggestions that prioritize both engagement and satisfaction. This post breaks down how the system works, fromThu, 19 Jun 2025 00:00:00 GMTrecommendationsindustryhttps://tullie.ai/blog/mrr-how-quickly-do-users-find-the-first-relevant-item/https://tullie.ai/blog/mrr-how-quickly-do-users-find-the-first-relevant-item/Mean Reciprocal Rank (MRR) is a metric that captures how quickly a user finds the first relevant item in a ranked list, making it especially valuable for tasks like known-item search or question answering where just one good result matters. This article introduces the conceptThu, 19 Jun 2025 00:00:00 GMTrecommendationssearchevaluationhttps://tullie.ai/blog/mastering-cold-start-challenges/https://tullie.ai/blog/mastering-cold-start-challenges/Cold start challenges can derail personalization efforts by making it difficult to deliver relevant experiences for new users, items, or markets. This post explores proven strategies and modern system architectures — including modular, AI-native platforms like Shaped — thatWed, 18 Jun 2025 00:00:00 GMTrecommendationspersonalizationhttps://tullie.ai/blog/explainable-personalization/https://tullie.ai/blog/explainable-personalization/This post examines how to develop explainable personalization systems that enhance user trust, enhance internal visibility, and foster long-term engagement. It covers the key components of explainability, including transparent logic, user feedback, and internal observability,Tue, 17 Jun 2025 00:00:00 GMTrecommendationspersonalizationhttps://tullie.ai/blog/matrix-factorization-the-bedrock-of-collaborative-filtering-recommendations/https://tullie.ai/blog/matrix-factorization-the-bedrock-of-collaborative-filtering-recommendations/Matrix Factorization (MF) has long been a foundational technique in collaborative filtering for recommendation systems. It works by learning latent factors that represent hidden preferences of users and characteristics of items, allowing it to predict unknown interactions.Tue, 17 Jun 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/modular-ai/https://tullie.ai/blog/modular-ai/This post explores how modular AI infrastructure enables faster, more flexible, and more scalable personalization systems. It outlines the key components of a composable stack, like data ingestion, candidate generation, ranking, and feedback, and offers design principles toMon, 16 Jun 2025 00:00:00 GMTrecommendationsinfrastructurehttps://tullie.ai/blog/ndcg-graded-relevance/https://tullie.ai/blog/ndcg-graded-relevance/How do you know if your ranking model is getting the order right, not just retrieving the right items? This post introduces NDCG, a powerful metric that accounts for both how relevant each item is and where it appears in the ranked list, enabling a more nuanced evaluation ofThu, 12 Jun 2025 00:00:00 GMTrecommendationssearchevaluationhttps://tullie.ai/blog/10-best-practices-in-data-ingestion/https://tullie.ai/blog/10-best-practices-in-data-ingestion/This post outlines 10 best practices for designing robust, scalable data ingestion pipelines that support real-time analytics, personalization, and machine learning. It covers essential topics like choosing the right ingestion pattern, enforcing data contracts, handlingWed, 11 Jun 2025 00:00:00 GMTrecommendationsinfrastructurehttps://tullie.ai/blog/unlock-text-data-nlp-feature-engineering-for-search-recs/https://tullie.ai/blog/unlock-text-data-nlp-feature-engineering-for-search-recs/Keyword matching and interaction history aren’t enough for modern relevance. Language data, like product descriptions, search queries, and user reviews, holds rich signals that drive deeper personalization. But turning text into model-ready features requires complex NLPWed, 11 Jun 2025 00:00:00 GMTrecommendationssearchfeature-engineeringhttps://tullie.ai/blog/monolithic-vs-modular-ai-architecture/https://tullie.ai/blog/monolithic-vs-modular-ai-architecture/This blog post explores the differences between monolithic and modular AI-native architectures, helping businesses choose the best approach for their AI personalization systems. It explains the fundamental distinction: monolithic architectures integrate all AI components intoMon, 09 Jun 2025 00:00:00 GMTrecommendationsinfrastructurehttps://tullie.ai/blog/how-to-unify-data-ecosystems/https://tullie.ai/blog/how-to-unify-data-ecosystems/This blog post addresses the challenge of fragmented data ecosystems, which hinders companies' ability to provide effective personalization. It presents a 6-step framework for unifying data across systems, enabling seamless, AI-driven customer experiences. The steps includeSat, 07 Jun 2025 00:00:00 GMTrecommendationsinfrastructurehttps://tullie.ai/blog/ai-powered-recommendation-engines/https://tullie.ai/blog/ai-powered-recommendation-engines/This article explores how AI-powered recommendation systems are transforming digital experiences across e-commerce, music, and marketplaces.Wed, 04 Jun 2025 00:00:00 GMTrecommendationspersonalizationhttps://tullie.ai/blog/h-m-dataset-powering-personalized-fashion-recommendations-at-scale/https://tullie.ai/blog/h-m-dataset-powering-personalized-fashion-recommendations-at-scale/The H&M Personalized Fashion Recommendations dataset is a favorite in the ML community for testing large-scale, real-world recommendation systems. With millions of transactions and rich metadata, it offers a challenging benchmark for building personalized fashion experiences.Wed, 04 Jun 2025 00:00:00 GMTrecommendationsdatasetshttps://tullie.ai/blog/customer-data-platform/https://tullie.ai/blog/customer-data-platform/This article explores how effective personalization relies on collecting, unifying, and analyzing first-party data through tools like Customer Data Platforms (CDPs). It highlights the role of data mining, real-time ingestion, and machine learning in transforming raw data—fromTue, 03 Jun 2025 00:00:00 GMTrecommendationsinfrastructurehttps://tullie.ai/blog/feed-ranking-systems/https://tullie.ai/blog/feed-ranking-systems/This article explores how LinkedIn’s large-scale ranking framework, LiRank, integrates deep learning and large language models (LLMs) to power personalized content across feeds, job recommendations, and ads. It details core innovations such as Residual DCN, isotonicTue, 03 Jun 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/multi-armed-bandits/https://tullie.ai/blog/multi-armed-bandits/This article unpacks how multi-armed bandits offer a smarter alternative to A/B testing for real-time personalization. By dynamically balancing exploration and exploitation, bandit algorithms adapt to user behavior on the fly—delivering more relevant content, faster.Tue, 03 Jun 2025 00:00:00 GMTrecommendationsmodelsevaluationhttps://tullie.ai/blog/approximate-nearest-neighbors-algorithms/https://tullie.ai/blog/approximate-nearest-neighbors-algorithms/This article explores the role of approximate nearest neighbor (ANN) search in scaling personalization and similarity search across large, high-dimensional datasets. It contrasts ANN with exact search, highlighting its speed-accuracy trade-offs and practical relevance forMon, 02 Jun 2025 00:00:00 GMTrecommendationsretrievalmodelshttps://tullie.ai/blog/how-does-temu-work/https://tullie.ai/blog/how-does-temu-work/This article examines how Temu became one of the fastest-growing e-commerce platforms by using AI to fuel engagement across the user journey. It explores how real-time deep learning models, gamification, and multi-objective optimization drive personalization, session depth,Mon, 02 Jun 2025 00:00:00 GMTrecommendationsindustryhttps://tullie.ai/blog/retrieval-augmented-generation-rag/https://tullie.ai/blog/retrieval-augmented-generation-rag/This blog post explores the challenges of real-time personalization in AI, such as high computational costs, slow experimentation, and the cold start problem. It introduces retrieval-augmented generation (RAG) as a solution, highlighting how it combines generative AI withMon, 02 Jun 2025 00:00:00 GMTsearchretrievalagentshttps://tullie.ai/blog/how-amazon-masterminds-real-time-product-discovery/https://tullie.ai/blog/how-amazon-masterminds-real-time-product-discovery/This article examines how Amazon leads in real-time product discovery by guiding users beyond search through personalized, AI-driven experiences. Using a blend of collaborative filtering, content-based filtering, and reinforcement learning, Amazon tailors recommendationsFri, 30 May 2025 00:00:00 GMTrecommendationsindustryhttps://tullie.ai/blog/relevancy-precision-and-recall/https://tullie.ai/blog/relevancy-precision-and-recall/This article explains how precision, recall, and relevancy serve as core metrics for evaluating and optimizing recommendation systems. Precision measures how many recommended items are truly relevant, while recall captures how many relevant items are successfullyFri, 30 May 2025 00:00:00 GMTrecommendationssearchevaluationhttps://tullie.ai/blog/ai-powered-cross-selling-recommendations/https://tullie.ai/blog/ai-powered-cross-selling-recommendations/This article explores how AI is transforming cross-selling from a static, rules-based tactic into a dynamic personalization engine that adapts in real time. It contrasts traditional methods with AI-driven systems that detect subtle product relationships, adjust suggestionsThu, 29 May 2025 00:00:00 GMTrecommendationspersonalizationhttps://tullie.ai/blog/ab-testing-ranking-metrics/https://tullie.ai/blog/ab-testing-ranking-metrics/You’ve trained a model, optimized offline metrics, and picked a winner, but how do you know it’ll perform with real users? In this post, we explore why A/B testing is essential for validating personalization and ranking models in production. We cover key online metrics likeWed, 28 May 2025 00:00:00 GMTrecommendationsevaluationhttps://tullie.ai/blog/deep-learning-for-hyper-personalized-recommendations/https://tullie.ai/blog/deep-learning-for-hyper-personalized-recommendations/This blog explores how deep learning is revolutionizing personalized recommendations by enabling real-time, context-aware experiences for users. Traditional recommendation systems, such as collaborative filtering and content-based models, struggle with static data, cold startTue, 27 May 2025 00:00:00 GMTrecommendationspersonalizationmodelshttps://tullie.ai/blog/golden-tests-in-ai/https://tullie.ai/blog/golden-tests-in-ai/This article introduces golden tests as a practical method for detecting regressions in AI systems—especially real-time recommendation engines—by comparing current model outputs against a saved “golden” baseline. Unlike traditional tests, golden tests capture subtle changesMon, 26 May 2025 00:00:00 GMTrecommendationsproductionevaluationhttps://tullie.ai/blog/bridging-worlds-training-language-models-on-user-behavior-for-smarter-recommendations/https://tullie.ai/blog/bridging-worlds-training-language-models-on-user-behavior-for-smarter-recommendations/Traditional recommendation models face a tradeoff: language models excel at understanding item semantics, while collaborative filtering shines at capturing behavioral patterns. But what if you could combine both? In this post, we explore a new generation of hybrid techniques,Fri, 23 May 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/evaluation-metrics-for-search-and-recommendation-systems/https://tullie.ai/blog/evaluation-metrics-for-search-and-recommendation-systems/This article explores key metrics used to evaluate search and recommendation systems, from precision and recall to NDCG and diversity. It explains how offline and online evaluations work together to assess performance, and highlights challenges like data sparsity and feedbackThu, 22 May 2025 00:00:00 GMTrecommendationssearchevaluationhttps://tullie.ai/blog/modern-ranking-models/https://tullie.ai/blog/modern-ranking-models/This article offers a comprehensive guide to ranking models — algorithms that power personalized search, product recommendations, and content discovery. It breaks down the components of modern ranking systems, including retrieval, scoring, and ordering, and explains keyMon, 19 May 2025 00:00:00 GMTrecommendationssearchmodelshttps://tullie.ai/blog/collaborative-filtering/https://tullie.ai/blog/collaborative-filtering/This article explores collaborative filtering, a foundational technique behind personalized recommendations on platforms like Netflix and Amazon. It explains how user-based and item-based filtering work, compares memory-based and model-based approaches, and highlightsSun, 18 May 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/vector-search-explained/https://tullie.ai/blog/vector-search-explained/This blog post explains how vector search is transforming search and recommendation systems by focusing on the meaning behind data, not just matching keywords.Thu, 15 May 2025 00:00:00 GMTsearchretrievalhttps://tullie.ai/blog/tweedie-regression-video-recommendations-tubi/https://tullie.ai/blog/tweedie-regression-video-recommendations-tubi/TL;DR: Tubi boosted VOD revenue (+0.4%) and watch time (+0.15%) by ditching weighted LogLoss for CTR and instead using Tweedie Regression to directly predict user watch time. Their paper shows Tweedie loss better models the zero-inflated, skewed nature of watch time data,Wed, 14 May 2025 00:00:00 GMTrecommendationsindustrymodelshttps://tullie.ai/blog/wayfair-pinterest/https://tullie.ai/blog/wayfair-pinterest/This blog post explores how leading companies like Wayfair and Pinterest use visual data and user behavior to create personalized discovery experiences. It highlights the growing role of visual data in enhancing personalization, moving beyond traditional text-based methods.Tue, 13 May 2025 00:00:00 GMTrecommendationsindustryhttps://tullie.ai/blog/netflix-personalization-workshop-2025/https://tullie.ai/blog/netflix-personalization-workshop-2025/The Shaped team was thrilled to be at the 2025 Netflix Personalization, Recommendations & Search workshop last week! This event, first held by Netflix in 2016, is one of our highlights on the AI recommendation & search calendar. The day was packed with insightful talks fromTue, 13 May 2025 00:00:00 GMTrecommendationsindustryhttps://tullie.ai/blog/two-tower-recommendation-models/https://tullie.ai/blog/two-tower-recommendation-models/The Two-Tower model is a foundational architecture for large-scale recommendation systems, built to efficiently retrieve relevant items from massive catalogs. By learning separate embeddings for users and items, it enables fast candidate generation via approximate nearestFri, 09 May 2025 00:00:00 GMTrecommendationsretrievalmodelshttps://tullie.ai/blog/criteo-dataset-tackling-large-scale-click-through-rate-prediction/https://tullie.ai/blog/criteo-dataset-tackling-large-scale-click-through-rate-prediction/Click-through rate (CTR) prediction is central to modern advertising and recommendation systems, and the Criteo dataset has become the de facto benchmark for advancing this task at industrial scale. With hundreds of millions to billions of rows and a blend of dense numericalThu, 08 May 2025 00:00:00 GMTrecommendationsdatasetshttps://tullie.ai/blog/sequential-models-for-recommendations/https://tullie.ai/blog/sequential-models-for-recommendations/In a world where user behavior changes by the minute, traditional recommendation systems fall short. Sequential recommendation models offer a powerful upgrade, capturing evolving intent by analyzing the order of interactions. This article breaks down the evolution of theseTue, 06 May 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/how-to-build-a-killer-for-you-feed/https://tullie.ai/blog/how-to-build-a-killer-for-you-feed/The “For You” feed has become the gold standard of personalized digital experiences—but behind the magic lies serious technical complexity. From wrangling massive datasets to training cutting-edge ML models and serving results in real time, building a high-quality feed fromFri, 02 May 2025 00:00:00 GMTrecommendationspersonalizationinfrastructurehttps://tullie.ai/blog/beyond-retrieval-optimizing-relevance-with-reranking/https://tullie.ai/blog/beyond-retrieval-optimizing-relevance-with-reranking/Retrieving a strong list of candidate items is just the first step—the real challenge is ranking them in the most relevant, personalized order for each user and goal. This post explores how reranking transforms basic search results or recommendations into truly optimizedMon, 28 Apr 2025 00:00:00 GMTrecommendationssearchretrievalhttps://tullie.ai/blog/precision-at-k-ranking/https://tullie.ai/blog/precision-at-k-ranking/Is your recommender system truly hitting the mark? Imagine a user binging blockbusters like Avengers and Top Gun—will they click on Love Actually or John Wick next? This article breaks down Precision@K, the go-to metric for judging how many of your top K recommendations areFri, 25 Apr 2025 00:00:00 GMTrecommendationsevaluationhttps://tullie.ai/blog/cross-encoder-rediscovers-a-semantic-variant-of-bm25/https://tullie.ai/blog/cross-encoder-rediscovers-a-semantic-variant-of-bm25/This article explores how cross-encoders, long praised for their performance in neural ranking, may in fact be reimplementing classic information retrieval logic, specifically, a semantic variant of BM25. Through mechanistic interpretability techniques, the authors uncoverThu, 24 Apr 2025 00:00:00 GMTsearchretrievalmodelshttps://tullie.ai/blog/one-embedding-to-rule-them-all/https://tullie.ai/blog/one-embedding-to-rule-them-all/Pinterest’s OmniSearchSage represents a major step forward in unified semantic search. By extending the two-tower model into a multi-task, multi-entity framework, it enables a single query embedding to power retrieval across pins, products, and related queries. The systemTue, 22 Apr 2025 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/beyond-relevance-optimizing-for-multiple-objectives-in-search-and-recommendations/https://tullie.ai/blog/beyond-relevance-optimizing-for-multiple-objectives-in-search-and-recommendations/Building effective recommendation and search systems means going beyond simply predicting relevance. Modern users expect personalized experiences that cater to a wide range of needs and preferences, and businesses need systems that align with their overarching goals. ThisWed, 05 Mar 2025 00:00:00 GMTrecommendationssearchevaluationhttps://tullie.ai/blog/beyond-dot-products-retrieval-with-learned-similarities/https://tullie.ai/blog/beyond-dot-products-retrieval-with-learned-similarities/The world of vector databases is exploding. Driven by the rise of large language models and the increasing need for semantic search, efficient retrieval of information from massive datasets has become paramount. Approximate Nearest Neighbor (ANN) search, often using dotThu, 27 Feb 2025 00:00:00 GMTrecommendationsretrievalmodelshttps://tullie.ai/blog/chatgpt-moment-for-recommendations/https://tullie.ai/blog/chatgpt-moment-for-recommendations/Researchers at Meta recently published a ground-breaking paper that combines the technology behind ChatGPT with Recommender Systems. They show they can scale these models up to 1.5 trillion parameters and demonstrate a 12.4% increase in topline metrics in production A/BWed, 05 Jun 2024 00:00:00 GMTrecommendationsmodelshttps://tullie.ai/blog/evaluating-recommendations-map-mmr-ndcg/https://tullie.ai/blog/evaluating-recommendations-map-mmr-ndcg/Imagine you’re shown two ordered feeds of product recommendations from separate algorithms. In the first one (A) you’re shown: Nike sneakers, Adidas shorts, and an Apple Watch. In the second one (B) you’re shown the order: Apple Watch, Adidas shorts, and Nike Sneakers. --Wed, 01 Mar 2023 00:00:00 GMTrecommendationsevaluationhttps://tullie.ai/blog/evaluating-recommendations-precision-recall/https://tullie.ai/blog/evaluating-recommendations-precision-recall/Imagine you’re given three movie recommendations from separate algorithms. In the first one (A) you’re given: The Terminator, James Bond, and Star Wars. In the second (B) you’re given: Cars, Toy Story, and Iron Man --Tue, 07 Feb 2023 00:00:00 GMTrecommendationsevaluationhttps://tullie.ai/blog/day-2-of-recsys2022-our-favorite-5-papers-and-talks/https://tullie.ai/blog/day-2-of-recsys2022-our-favorite-5-papers-and-talks/It’s been another fantastic day at RecSys 2022. Following the Women in RecSys Breakfast, the day started with a keynote from Catherine D’Ignazio and then throughout the day had the following sessions: Fairness & Privacy, Diversity & Novely, and Models and Learning I. Here areTue, 20 Sep 2022 00:00:00 GMTrecommendationshttps://tullie.ai/blog/data-centric-ai-for-ranking/https://tullie.ai/blog/data-centric-ai-for-ranking/Data quality and volume is what makes rankings algorithms at big-tech so seamless. How can you create the same experiences with the data you have? Data-centric AI may be the answer!Tue, 12 Jul 2022 00:00:00 GMTrecommendationsproduction