I didn't switch to multi-vector embeddings because it was trendy.
I did it because a single pooled vector kept lying to my search.
Collapse a candidate into one embedding and you're asking a single point in space to stand for "career arc" and "licenses" and "skills" and "general profile vibe" all at once. In a recruitment dataset, where one designation like CFP/CFA can be the gating factor, that pooling turns into a weird kind of blur. The vector ends up kind of about everything, which is another way of saying it's not sharply about the thing you're filtering on.
So I built an Embedding Agent that generates four parallel embeddings for multi-vector search:
profile_vector: overall candidate profile embeddingexperience_vector: work experience and career historyskills_vector: skills, designations, and certificationsgeneral_vector: general-purpose embedding for broad matching
One design choice, and suddenly a pile of engineering decisions people don't talk about enough:
- cache key schema (you can't cache "the embedding" anymore, you cache a typed embedding under a model)
- index layout (Azure AI Search needs to know which vector fields exist)
- query-time composition (your search stack decides how to use one vector or many)
- reindex/backfill strategy (you need a way to reconcile inconsistent embeddings without redoing everything)
The rest of this post is the under-the-hood view of how I wired those pieces together in the embedding agent module and the generator job that produces embeddings at scale.
Key insight: typed vectors turn "one expensive truth" into "four cheap, composable facts"
The real payoff here is a control surface, not sharper math.
With a single pooled vector, every change is global:
- new model? regenerate everything
- new field? regenerate everything
- bad content? it poisons the one representation you have
With typed vectors, I can treat each embedding like a separate instrument channel in a mix. If the "skills" channel is wrong, I don't have to remix the whole song.
That's why the Embedding Agent is explicit about its outputs:
- it generates 4 parallel embeddings
- it uses OpenAI
text-embedding-3-large - it fixes the dimensionality at 3072
- it implements Redis caching with a 24hr TTL
Those four lines are not trivia. They're the reason the cache keys, storage blobs, and reindex passes look the way they do.
(If you want a concrete precedent for decomposing profiles into section-specific vectors, LinkedIn has publicly described encoding structured profile sections separately, Summary, Experience, Education, to enable more granular matching in profile search.) [https://www.linkedin.com/blog/engineering/search/reimagining-linkedins-search-stack]
How it works: producer → cache → persistent blob → index rebuild
Here's the shape of the pipeline I run:
The generator job is the producer. It feeds records to the agent.
The agent is responsible for generating and caching multi-vector embeddings, and for returning consistent shapes even when things go wrong.
The persistent store holds the multi-vector blob, so reindexing doesn't need to call the model again.
The index rebuild process reads those blobs and pushes them into the search index's vector fields.
That dotted "hash-first delta sync" arrow is how I keep backfills from turning into full regenerations. Compare hashes first, then regenerate only what's actually different.
The Embedding Agent: four vectors, one contract
The most useful thing I did in the embedding agent module was make the contract dull and predictable.
It always returns the same typed structure, and it's explicit about the model and dimensionality:
"""
Embedding Agent - Specialized agent for generating and caching multi-vector embeddings.
Generates 4 parallel embeddings for multi-vector search:
- profile_vector: Overall candidate profile embedding
- experience_vector: Work experience and career history
- skills_vector: Skills, designations, and certifications
- general_vector: General-purpose embedding for broad matching
Uses OpenAI text-embedding-3-large (3072 dimensions) for high-quality embeddings.
Implements Redis caching with 24hr TTL to minimize API costs.
"""
import asyncio
import hashlib
import json
import logging
import os
import time
from dataclasses import dataclass
from typing import Any, Dict, List, Optional
from openai import AsyncOpenAI
from .base import BaseAgent, AgentConfig, AgentResponse, AgentType
logger = logging.getLogger(__name__)
# Constants
EMBEDDING_MODEL = "text-embedding-3-large"
EMBEDDING_DIMENSIONS = 3072 # text-embedding-3-large native dimensions
# EMBEDDING_TTL_ ... (defined in the source file)
What surprised me here is how much reliability comes out of naming. Once skills_vector was a stable name, every downstream system stopped guessing.
I am not reproducing the internals here. The contract is clear enough to design everything else around.
Why the naive approach fails
The naive approach is "generate one embedding for the whole candidate record and call it a day."
That fails in two ways:
- Semantic dilution: credentials and licenses become a small part of a large text, so similarity isn't sharp where it matters.
- Operational coupling: any change forces full regeneration.
Typed vectors fix both. I can query on "skills" when the user cares about designations, and I can regenerate only the vectors whose inputs changed.
The tradeoff
Multi-vector increases storage and index complexity. Four vectors to store, four to index, and decisions to make at query time.
I took that trade anyway, because the system already has multiple search modes (atlas candidates vs jobs vs notes vs transcripts), and the cost of "wrong matches" is recruiter time.
Cache key design: hash field subsets + model id
Once "the embedding" stops being a single thing, you need a cache key schema that makes collisions hard and invalidation obvious.
The Embedding Agent uses:
hashlibjson- Redis caching with a 24hr TTL
- a model identifier:
text-embedding-3-large
So the key design I anchored on is:
- include the model id in the key
- include the vector type (
profile_vector,experience_vector,skills_vector,general_vector) - hash the canonical JSON of the input subset that feeds that vector
The engineering property that matters: the hash has to be computed from only the fields that matter for that vector.
That's what makes typed vectors worth it. The skills_vector key doesn't churn when you edit a candidate's biography, and the experience_vector key doesn't churn when you add a new designation.
(Operationally, this matches how other teams have versioned embedding artifacts by putting the embedding model id into keys so cached vectors and indexed vectors don't collide across model rollouts.) [https://www.uber.com/blog/evolution-and-scale-of-ubers-delivery-search-platform/]
Eviction: TTL
The Embedding Agent states it plainly: Redis caching with 24hr TTL.
It's a TTL-based cache, which means repeat inputs inside that 24-hour window reuse an existing embedding instead of paying for a new one. Fewer API calls, lower cost.
The generator job: retries, validation, and DLQ routing
The embedding pipeline is more than four calls to OpenAI. The production question is what happens when the input is malformed, too long, or comes back with the wrong dimensionality.
That's why the generator job exists.
The commit summary for the embedding generator module is explicit:
- "Generate embeddings with retry logic and DLQ routing."
- "Handles content length validation and dimension mismatch detection."
Those are the failure modes that actually hurt you during backfills.
Why naive batch generation fails
Bulk-generate embeddings without guardrails and:
- one poison record can crash the batch
- transient API failures stall the pipeline
- a dimension mismatch can silently corrupt your index
So the generator job is designed to be stubborn. Validate early, retry transient failures, route poison records to a dead-letter path.
(For practical guidance on designing retries, exponential backoff, idempotency, and DLQ handling in producer jobs, see this engineering writeup that walks through patterns used to keep large-scale pipelines moving while isolating bad records.) [https://softbuilds.medium.com/how-to-design-a-reliable-retry-system-with-backoff-dlqs-idempotency-82c8c001cfde]
Code shape
Here's the interface, sketched from the generator's documented behavior:
"""app/jobs/embedding_generator.py
Generates embeddings with retry logic and DLQ routing.
Handles content length validation and dimension mismatch detection.
"""
from dataclasses import dataclass
from typing import Any, Dict, Optional
@dataclass
class EmbeddingJobResult:
ok: bool
error: Optional[str] = None
payload: Optional[Dict[str, Any]] = None
def embedding_generator_job(record: Dict[str, Any]) -> EmbeddingJobResult:
"""Run embedding generation for a single record.
Real job behavior:
- validates content length
- detects dimension mismatch
- retries with exponential backoff + jitter
- routes poison records to a DLQ
This stub returns a structured result.
"""
return EmbeddingJobResult(ok=False, error="stub — see full implementation in source")
if __name__ == "__main__":
# Runnable placeholder demonstrating the contract.
example = {"id": "record_123", "text": "..."}
print(embedding_generator_job(example))
What I like about this job design is that it treats "dimension mismatch" as a first-class failure. That's the sort of bug that doesn't crash loudly. It just makes search feel haunted.
Idempotent storage: multi-vector blobs, not scattered fields
Four vectors in hand, you have a choice:
- store them as separate rows/keys
- store them as a single blob keyed by record id + model
The system has to support:
- "idempotent storage of multi-vector blobs" (my own direction for the system)
- "dimension mismatch detection" (the generator summary)
So the claim here is architectural: I treat the multi-vector output as an atomic unit for persistence, because that's what prevents partial updates (three vectors fresh, one stale) from slipping into index rebuilds unannounced.
The limitation is obvious. Blobs are less queryable. But embeddings are rarely queried directly; they're read for indexing and search.
Query-time composition: why four vectors don't mean four searches
The platform includes a SearchAgent for semantic search with vector embeddings on Azure AI Search, and an Advanced Matching Engine that uses embeddings as one component of a multi-modal scoring algorithm.
Multi-vector embeddings affect query-time in one practical way: the search layer can choose which vector field to use depending on intent. A designation search hits skills_vector. A "find someone like this person" query hits profile_vector. A recruiter scanning for career trajectory hits experience_vector. Same indexed record, three questions, no re-embedding.
Backfill and reindex: hash-first delta sync
The direction I set for this system calls out "hash-first delta sync" for reconciling inconsistent embeddings.
The embedding code imports hashlib and json, which are exactly the tools you reach for when you want stable content hashes.
The operational trick is simple:
- compute a hash of the canonical input subset for each vector type
- compare it to what you last stored
- only regenerate the vectors whose hashes changed
That's how you get smaller reindex windows without regenerating everything.
Those two imports, plus the multi-vector contract, confirm this is how the system keeps backfills incremental.
Model variants: swapping model IDs without wrecking caches
The Embedding Agent defines the model as a constant:
EMBEDDING_MODEL = "text-embedding-3-large"
That one string carries more weight than it looks like it should.
If your cache keys include the model id, then trying a model variant becomes operationally safe:
- new model id → new keys
- old cache remains valid for the old model
- you can A/B by selecting which model id to use
The mechanism is nothing more than model id in the key, and it means model rollouts don't invalidate existing caches or require stop-the-world reindexing.
Practical wins
A few outcomes are real and immediate from the design itself:
- Lower API cost due to caching: the Embedding Agent explicitly implements Redis caching with a 24hr TTL.
- Smaller reindex windows: multi-vector plus hash-first delta sync means you can regenerate only what changed.
- Safer rollouts of model changes: model id is explicit (
text-embedding-3-large), so it can be incorporated into keying and storage.
Nuances: where this design bites back
Typed vectors are not free.
- You now have four ways to be wrong instead of one.
- If you don't keep the "input subset per vector" disciplined, you'll churn caches and lose the whole point.
- If your generator doesn't treat dimension mismatch as fatal, you can corrupt your index without anyone noticing.
Which is why I like the generator job's explicit focus on validation and poison routing. It admits that production data is messy, and it builds a system that keeps moving anyway.
Closing
Once I stopped treating embeddings as a single magical artifact and started treating them as typed, versioned, cacheable facts, the rest of the pipeline stopped fighting me. Cache invalidation became scoped. Reindexing became incremental. Model rollouts became safe. The embedding stopped being the hard part. It's four facts about a person now, each one independently verifiable, each one independently replaceable, and that's the design insight that made everything downstream simpler.
RESEARCH SOURCES:
- Reimagining LinkedIn’s Search Tech Stack: LinkedIn details its move toward embedding-based search where profiles are decomposed into structured sections (Summary, Experience, Education) and encoded into separate vectors. This architecture allows for more granular semantic matching than single-vector blobs, directly supporting the 'typed vector' approach for recruitment data.
- Evolution and Scale of Uber’s Delivery Search Platform: Uber's engineering team explains their system for versioning embedding artifacts using unique model IDs (query_model_id, doc_model_id). This ensures that cache keys and indexed vectors remain consistent across model deployments, preventing collisions and enabling safe rollbacks during backfills.
- How to Design a Reliable Retry System, With Backoff, DLQs, and Idempotency: This technical deep dive outlines the implementation of idempotent operations and retry mechanisms using exponential backoff and Dead Letter Queues (DLQs). It provides the blueprint for building resilient data pipelines that can isolate bad records to prevent stalling large-scale reindexing or backfill tasks.
Research summary: The research identifies key authoritative sources for building resilient, multi-vector embedding pipelines in the recruitment domain. LinkedIn Engineering's blog validates the 'typed vector' approach by decomposing profiles into sections (experience, education) for granular embedding. Uber Engineering provides the gold standard for embedding versioning using model IDs in cache keys to ensure consistency during deployment. General reliability patterns like idempotency, exponential backoff, and DLQs are covered by system design experts to handle pipeline failures without stalling reindexes.