Agent Memory Errors Handbook: Cases 31–35

This installment is part of the Agent Memory Errors: 100 Cases Handbook series.

Case 31: Vector Database & Indexing Errors — Hnsw Graph Connectivity Island

Scenario

In a vector library using an HNSW index, a batch of documents has poor embeddings (very small norms). With a small M parameter, they fail to establish enough connections to the main graph during construction. These vectors become islands: they exist in the database but cannot be reached by search traversal.

Error Manifestation

When users query topics covered by these documents, the documents are missing from results even when highly relevant. Admins can see the records but cannot retrieve them via search.

Solution

1. Check graph connectivity during HNSW construction: force-connect isolated components or rebuild their embeddings
2. Regularly inspect vector-norm distributions and normalize or re-encode outliers
3. Increase HNSW’s M and efConstruction parameters to improve graph density

Principle

HNSW is a graph-based approximate nearest-neighbor algorithm whose search reachability depends on graph connectivity. Poor embeddings or a small M create search blind spots.

Case 32: Vector Database & Indexing Errors — Embedding Quantization Precision Loss

Scenario

To save storage, float32 embeddings are quantized to int8, cutting space by 75%. Quality is usually unaffected, but a query-document pair with original similarity 0.71 (just above the 0.7 threshold) drops to 0.69 after quantization and is filtered out.

Error Manifestation

The same query-document pair retrieves in environment A (unquantized) but returns “I don’t know” in environment B (quantized), producing unpredictable behavior.

Solution

1. Analyze the similarity distribution before quantization; ensure boundary-case misclassification is acceptable
2. Use adaptive quantization: lower precision in dense regions and higher precision in sparse regions
3. Run retrieval-consistency tests after quantization and set acceptable recall-difference thresholds

Principle

Quantization trades precision for efficiency and introduces quantization error. Vector pairs near the similarity threshold are most vulnerable.

Case 33: Vector Database & Indexing Errors — Vector Dimensionality Mismatch Silent Failure

Scenario

During a system upgrade the embedding model changes from 768 to 1,024 dimensions, but old data in the vector database is not re-indexed. New queries produce 1,024-dimensional vectors while the database expects 768. Because client validation is loose, vectors are silently truncated or padded.

Error Manifestation

All queries return empty or random results without errors. Operations staff spend hours discovering the dimension mismatch.

Solution

1. Store dimension metadata in the vector database and enforce strict dimension checks on insert/query, raising clear errors on mismatch
2. Make re-indexing mandatory during model upgrades and bind index version to model version
3. Add application-layer health checks using known queries to detect retrieval-quality anomalies

Principle

Vector dimension is the identity of the embedding space. Dimension mismatch means vectors are not comparable, and inner-product or cosine-similarity scores are meaningless. Silent truncation/padding hides this fundamental error.

Case 34: Vector Database & Indexing Errors — Cosine Similarity Magnitude Bias

Scenario

The Agent uses cosine similarity to score query-document relevance. One document embedding has a large norm (10) because it contains many generic words; a high-quality specialized document has norm 1. The user’s specialized query is more relevant to the second, but cosine similarity ignores magnitude and gives both similar scores.

Error Manifestation

The generic, longer document ranks higher, and the Agent cites generic content rather than precise technical detail.

Solution

1. Use dot product or Euclidean distance so magnitude affects the score
2. L2-normalize embeddings before using cosine similarity to eliminate magnitude differences
3. At the reranking stage add an independent document-quality score (information density, source authority) combined with similarity

Principle

Cosine similarity normalizes away vector magnitude. Long documents tend to have larger magnitudes because more token vectors are summed, but that does not imply higher quality. Cosine similarity is limited when information density matters.

Case 35: Vector Database & Indexing Errors — Index Stale Deleted Documents Ghost

Scenario

A user deletes an outdated policy document from the knowledge base, but vector-index deletion is asynchronous via a message queue. During the five minutes before the queue is consumed, the user makes a related query.

Error Manifestation

The Agent retrieves the deleted document and answers based on it. The user clicks the document link and gets a 404, questioning reliability. Five minutes later the same query no longer returns the document, creating inconsistent behavior.

Solution

1. Use soft deletion: mark the primary record deleted first and filter soft-deleted records during retrieval while the index lags
2. Use vector-database transactions so vector-index deletion commits with primary-database deletion
3. Add a grace period after deletion during which related queries show a “content may be outdated” notice

Principle

Asynchronous index updates improve performance but create temporary inconsistency. For deletions, this inconsistency manifests as ghost data.

Source: Agent Search