My Agent Got Dementia: A GraphRAG Memory System Post-Mortem

This article is based on my systematic study of LLM-generated knowledge graphs. All data come from published academic papers. A full academic analysis is referenced at the end.

1. From Vector Retrieval to Graph Memory: The Pitfalls I Hit

Last year I started building a multi-turn conversational Agent. At first I used a standard RAG pipeline: chunk text → embed → dump into Pinecone → retrieve Top-K. Simple, effective—until one day a user asked:

“Last week I told you I don’t eat cilantro. Do you remember?”

The Agent paused for three seconds, then solemnly recommended a cilantro-and-beef salad.

The problem lies in the nature of vector retrieval: it can find semantically similar text chunks, but it cannot understand that “these are two sentences spoken by the same person at different times and are logically related.” The vector database has no entity called “User A” and no fact node “User A dislikes cilantro”—only a bunch of floating text fragments.

So I turned to GraphRAG. I wrote user profiles, preferences, and conversation history as nodes and edges, giving the Agent “real memory.”

It worked well at first. But after a month of operation, I noticed several bizarre phenomena:

Phenomenon 1: Cloning

The same user appeared in the graph as more than a dozen distinct nodes. The reason is that LLM entity extraction is extremely unstable: today it writes “Zhang San,” tomorrow “Mr. Zhang,” the day after “User 13800138000”—three completely different nodes, and the system never realizes they are the same person.

Phenomenon 2: Relation Inflation

When I inspected the graph, I found that the LLM had invented a relation type called “liked it a bit the day before yesterday but then lost interest.” Only one edge in the entire graph used it. Similar relations included “might know,” “indirectly related,” and “heard about from a friend”—more than 200 relation types in total, 80% of which appeared only once.

Phenomenon 3: Zombie Memories

A casual “hello” the user uttered three months ago was still lying in the graph as a high-weight memory. No expiration mechanism, no decay strategy; all information lived forever. The result: truly important memories were drowned out by thousands of casual chat fragments.

Phenomenon 4: Query Timeouts

After the number of nodes exceeded ten thousand, a 3-hop path query jumped from 50 ms to 3 seconds and then timed out. The user waited five seconds with no response and simply closed the chat box.

These four phenomena map to four systemic defects in knowledge-graph construction: unconstrained entity generation, missing relation schemas, missing temporal decay, and full-graph traversal bottlenecks.

2. Diagnosis: Four Quantitative Metrics for Agent Memory Health

After two months of trial and error, I distilled a self-check method. You don’t need to rewrite the whole system; just look at these four metrics first:

Metric 1: Entity Ambiguity

Open your graph, randomly sample 100 entities, and count how many are different surface forms of the same real-world object. If the number of unmerged synonymous entities exceeds 3, your entity resolution is already broken.

Why is this number critical? Because GraphRAG retrieval works by locating an entity first and then spreading along relations. If “LLM” and “Large Language Models” are two independent nodes, when the user asks about “latest LLM progress,” the system will only retrieve the sub-graph connected to the former and completely miss the latter—the same thing treated as two separate topics.

Experiments from Deg-Rag show that in LightRAG this kind of redundancy can inflate graph size by roughly 40%. Note that this is not a 40% increase in data volume; it is the same entity being created 1.4 times.

Metric 2: Relation Redundancy

Count the average out-degree per node. If it exceeds 15, be alert.

The reason is simple: if a query needs to traverse 3 hops with an average of 10 edges per hop, the number of paths to evaluate is 10³ = 1,000. If the average out-degree rises to 50, that’s 50³ = 125,000 paths—and query time explodes.

Metric 3: Data Pollution Rate

The proportion of non-knowledge nodes among all nodes. Casual chat, temporary statements, and garbage data from automated bulk imports—if this share exceeds 40%, half of your RAG context is noise. The LLM generates answers based on noise, so hallucination rates naturally soar.

A reference: the OntoKG paper’s cleaning of Wikidata found that among roughly 100 million entries, only 34.6 million (34.6%) were considered valid core entities; the rest were low-quality bulk imports. If Wikidata is like this, LLM-auto-generated graphs will only be worse.

Metric 4: Compute Overload

Look directly at the P95 query latency. Above 500 ms, users perceive lag. The target should be under 100 ms.

3. The Fix: A “Memory Whitewash” Pipeline

After diagnosing the problems, I started fixing them. Not by rewriting the whole system, but by adding four layers between write and query.

Layer 1: Cleaning and Normalization (Deduplication)

The core idea is simple: before the LLM writes to the graph, ask, “Does this entity already exist?”

The concrete approach is vector clustering plus entity linking. Embed all entities; those whose similarity exceeds a threshold enter the same candidate pool. Then use a finer matching algorithm (classic KG embeddings such as TransE or DistMult are sufficient; you don’t need to call GPT-4 every time) to judge whether they are the same entity.

Deg-Rag’s experiments validated this method: on four multi-hop QA datasets, removing about 40% of redundant entities and relations actually improved QA performance. It shows that a graph is not better just because it is bigger—cleanliness beats size.

Layer 2: Structured Filtering (Pruning)

Add a “house rule” to LLM memory writes:

• Predefine allowed relation types; forbid the LLM from inventing new ones.
• Relations occurring fewer than 3 times are treated as noise and deleted.
• Conversation memories older than 7 days with no subsequent interaction automatically expire.
• Every triple gets a quality score; low-scoring ones don’t enter the main graph.

The OntoKG paper proposes a more elegant idea called “intrinsic-relational routing”: divide attributes into “intrinsic attributes” (node properties, e.g., birth date) and “relational attributes” (traversable edges, e.g., “works at”), and use the schema to explicitly control what can be written into the graph. This is more efficient than filtering after the fact.

Layer 3: Tiered Storage

Not all memories deserve permanent storage. I designed a three-tier architecture:

Hot data goes in Redis, storing only the current session’s context with a TTL of 5–30 minutes; dropped when the session ends.

Warm data goes in a vector database (Pinecone/Milvus), storing valid memories from the last 7–30 days with a temporal decay coefficient—newer memories have higher weight; those older than 30 days are automatically down-weighted.

Cold data goes in a graph database (Neo4j), storing only validated long-term knowledge and user profiles. Read-mostly, updated periodically in batches.

Writes are automatically routed to the appropriate tier based on memory type; reads follow hot → warm → cold order, returning on first hit and avoiding unnecessary full-graph scans.

Layer 4: Graph Optimization (Dimensionality Reduction)

The most fundamental optimization idea: don’t traverse the whole graph every time.

Microsoft’s GraphRAG approach is worth borrowing: first cut the large graph into communities with the Louvain algorithm, then generate a summary for each community. At query time, match community summaries first to locate the most relevant community, then inspect it in detail. This shrinks the search space from the whole graph to a small community, reducing latency by an order of magnitude.

In addition, replacing exact traversal with HNSW approximate nearest-neighbor search can maintain recall above 95% while dropping query complexity from O(n) to O(log n).

4. Problems I Still Haven’t Solved

After this pipeline went live, my Agent became much more stable. But a few problems still lack satisfying solutions:

Dynamic Knowledge Evolution

When a knowledge graph continuously receives incremental updates, the cost-benefit ratio of denoising and schema maintenance changes. Temporal decay models (such as the Hawkes process in DynTKG) perform well academically, but the engineering complexity of integrating them into production systems is high.

Beneficial Noise

Can denoising go too far? Deg-Rag removed 40% of content, but if that 40% contained some low-frequency yet critical memory (e.g., the user rarely mentions a peanut allergy), then filtering becomes harmful. There is currently no theoretical framework for distinguishing “harmful noise” from “beneficial low-frequency information.”

Cross-Lingual Scenarios

All validation data come from English settings. Entity ambiguity rates may differ significantly for Chinese, Japanese, Arabic, and other languages—Chinese entity disambiguation, for example, must handle homophones, simplified/traditional variants, and nickname variants, which are far more complex than English.

5. Three Trends for 2026

Incremental Updates

LightRAG proposes an incremental update mechanism: only extract and deduplicate new documents, then merge them into the existing graph. This means an Agent can learn while chatting, instead of rebuilding the entire graph every night at 2 a.m.

Background Async Governance

Cutting-edge frameworks are introducing background threads that automatically run when the Agent is inactive: entity deduplication, low-frequency edge cleanup, community structure recomputation, and expired-memory archiving. It’s like giving the Agent a “night cleaning crew.”

Write-Time Constraints

Instead of letting the LLM write freely and then govern, impose constraints at write time. A likely future direction: the LLM only “proposes candidate memories,” while the actual write decision is made by an independent “memory gatekeeper” module—one that knows the schema, frequency statistics, and quality scores.

References and Data Sources

The quantitative data and core conclusions in this article come from the following studies:

• Zheng, Y., et al. (2025). Less is More: Denoising Knowledge Graphs For Retrieval Augmented Generation. arXiv:2510.14271. — Denoising can remove ~40% redundancy and improve RAG performance.
• Liu, Z., et al. (2026). OntoKG: Ontology-Oriented Knowledge Graph Construction with Intrinsic-Relational Routing. arXiv:2604.02618. — 34.6% of 100M Wikidata entries are core entities.
• Edge, D., et al. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. Microsoft Research. — Community detection enables large-scale graph retrieval.
• Guo, X., et al. (2024). LightRAG: Simple and Fast Retrieval-Augmented Generation. arXiv:2410.05779. — Incremental update mechanism.
• Blondel, V. D., et al. (2008). Fast Unfolding of Communities in Large Networks. Journal of Statistical Mechanics. — Louvain algorithm.
• Malkov, Y. A., & Yashunin, D. A. (2018). Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE TPAMI. — HNSW algorithm.