Agent Memory Errors Handbook: Cases 36–40
This installment is part of the Agent Memory Errors: 100 Cases Handbook series.
Case 36: Vector Database & Indexing Errors — Sharding Boundary Query Ambiguity
Scenario
A vector database is sharded by time: shard_1 holds 2023 documents, shard_2 holds 2024 documents. The user queries “market trends in early 2024.” Because “early” is ambiguous, relevant documents span both shards. Each shard returns its local top-5, but the globally best results may be suppressed by local ordering within shards.
Error Manifestation
The Agent’s answer is biased toward January 2024 and misses key December 2023 groundwork, making the analysis incoherent.
Solution
- Perform global reranking: merge results from all shards and rerank them jointly instead of taking each shard’s top-K
- Use overlapping shards: keep boundary documents in both adjacent shards
- Use intelligent query routing: send likely cross-shard queries to multiple shards and merge results
Principle
Sharding turns global ranking into local ranking, but local optimum is not global optimum. Cross-shard queries need special handling to avoid systematic bias.
Case 37: Vector Database & Indexing Errors — Embedding Cold Start Sparse Data
Scenario
Shortly after launch, the knowledge base contains only ten documents. A user query partially overlaps one document, but because the embedding space is sparse, the query-document similarity is 0.85 while an almost unrelated document scores 0.65, compressing relative distances.
Error Manifestation
The Agent is overconfident about low-confidence results and uses an irrelevant document as evidence, producing poor answers. Only after the collection grows to 1,000 documents does the similarity distribution become reasonable.
Solution
- During cold start, use sparse retrieval (BM25) instead of dense retrieval to avoid embedding-space sparsity
- Set a minimum document-count threshold: below it, switch to rule-based or template-based responses
- Calibrate similarity thresholds dynamically based on knowledge-base size
Principle
Absolute similarity values need sufficient context. In sparse spaces, pairwise similarities concentrate, making threshold judgments unreliable.
Case 38: Vector Database & Indexing Errors — Ivf Partial Search Cluster Miss
Scenario
Using an IVF index, the vector space is divided into 100 clusters. The query is assigned to cluster #23 and searches only that cluster. A relevant document near the boundary is assigned to the adjacent cluster #24.
Error Manifestation
The Agent misses the boundary document although it is highly relevant. Increasing nprobe solves the issue but adds latency.
Solution
- Increase the IVF nprobe parameter (e.g., from 1 to 5–10) to search neighboring clusters
- Use IVF-PQ to keep memory low while increasing nprobe
- Handle boundary vectors specially during index construction by replicating them into neighboring clusters
Principle
IVF clusters vectors for fast retrieval, but boundary vectors can be misclassified into adjacent clusters. The nprobe parameter controls the precision-latency trade-off.
Case 39: Vector Database & Indexing Errors — Ann Search Recall Precision Tradeoff
Scenario
To meet a <50 ms latency target, HNSW’s ef parameter is reduced from 200 to 50. Benchmark recall@10 drops from 95% to 72%, but on real business queries the distribution differs and actual recall is only 45%.
Error Manifestation
The Agent frequently misses key documents. When users ask where a policy is, the Agent says it cannot find it even though the policy exists in the knowledge base.
Solution
- Tune ANN parameters on real business queries, not only standard benchmarks
- Use adaptive precision-latency: low ef for simple queries, high ef for complex or important ones
- Monitor recall continuously and alert when it falls below threshold
Principle
ANN search is approximate and trades recall for speed. The ef parameter controls search thoroughness. Business query distributions often differ from academic benchmarks.
Case 40: Vector Database & Indexing Errors — Vector Index Version Skew Read After Write
Scenario
The Agent cluster runs multi-replica vector databases. A document write is routed to replica A and its index updates successfully. The next query is load-balanced to replica B, whose index has not yet synchronized (replication lag 500 ms).
Error Manifestation
The user immediately asks, “What does the document I just uploaded say?” The Agent returns “No relevant document found.” The user thinks the upload failed and re-uploads, creating duplicates.
Solution
- For read-after-write, route related queries to the written replica for N seconds after the write
- Implement write-confirmation waiting: poll all replicas to confirm index synchronization before returning success
- Set query consistency level to quorum or all
Principle
Distributed replicas have replication lag. Without consistency guarantees, write-then-read scenarios create an illusion of data loss.
Source: Agent Search