Agent Memory Errors Handbook: Cases 81–85
This installment is part of the Agent Memory Errors: 100 Cases Handbook series.
Case 81: Memory Consistency & Fact Anchoring Errors — Fact Entailment Direction Confusion
Scenario
The Agent’s memory contains two facts: “All birds can fly” (outdated knowledge; ostriches cannot fly) and “Penguins are birds.” The user asks “Can penguins fly?” The Agent’s reasoning chain: penguins are birds → all birds can fly → penguins can fly.
Error Manifestation
The Agent incorrectly answers “Penguins can fly” because it fails to recognize exceptions to the universal proposition.
Solution
- Label facts with “certainty levels”: universal propositions (all X are Y) marked as “possible exceptions,” specific facts (penguins cannot fly) marked as “certain”
- Implement exception detection: before applying universal propositions, check for known exceptions
- Use default logic: “Birds fly” as a default rule, but overridable by more specific facts
Principle
Universal propositions plus implication are dangerous in knowledge representation. The real world is full of exceptions. Default logic and non-monotonic reasoning are frameworks closer to human common sense.
Case 82: Memory Consistency & Fact Anchoring Errors — Temporal Fact Versioning Conflict
Scenario
The Agent’s memory stores “Company CEO in 2023 was Zhang San” and “Company CEO in 2024 is Li Si.” The user asks “Who is the company CEO?” The Agent does not understand the concept of “current time” and mentions both Zhang San and Li Si in the answer without clearly stating the time range.
Error Manifestation
The user gets a confusing answer “The company CEO is Zhang San and Li Si,” or randomly chooses one, possibly giving outdated information.
Solution
- Attach “valid time intervals” to each fact: e.g.,
{"fact": "CEO is Li Si", "valid_from": "2024-01-01", "valid_until": null} - Filter effective facts based on “query time” (default current time) when answering, clearly labeling “As of 2024”
- Perform “timeline visualization” for temporally conflicting facts: “2023: Zhang San → 2024: Li Si”
Principle
Facts are not static; they have lifecycles. Fact libraries without time markers inevitably accumulate contradictions over time. Temporal database concepts should be introduced into Agent memory systems.
Case 83: Memory Consistency & Fact Anchoring Errors — Source Credibility Degradation Untracked
Scenario
A fact in the Agent’s memory is marked as sourced from “authoritative scientific journal Nature 2020.” In 2023 the study was retracted, but the Agent’s memory system has no mechanism to receive “source credibility change” signals. The Agent continues to cite the study as authoritative evidence.
Error Manifestation
The Agent cites a retracted paper in scientific discussions, misleading users. When the user points out “Wasn’t this paper retracted?” the Agent cannot explain.
Solution
- Establish a “source credibility dynamic update” mechanism: regularly (e.g., monthly) check whether sources in memory have been retracted, corrected, or disputed
- Maintain a “credibility timeline” for sources: record historical changes in source credibility
- When citing, label “source status as of memory update time” and prompt users to “verify the latest information”
Principle
Source credibility is not a static attribute. Scientific papers may be retracted, news may be corrected, experts may change views. Static source labels cause Agents to spread outdated or incorrect information.
Case 84: Memory Consistency & Fact Anchoring Errors — Counterfactual Memory Failure
Scenario
The user asks the Agent “What would have happened if I had chosen plan B instead of plan A?” The Agent’s memory is strongly anchored to the fact “plan A was chosen” and cannot perform counterfactual reasoning. The Agent replies “You chose plan A, and the result is…” completely avoiding the user’s hypothesis.
Error Manifestation
The user feels the Agent “doesn’t understand hypothetical questions” and cannot perform decision reflection or scenario analysis.
Solution
- Implement “belief boxing”: distinguish “factual memory” from “hypothetical memory”; counterfactual reasoning runs in hypothetical memory space
- During reasoning clearly label “current reasoning is based on hypothesis X”; distinguish “actual result” from “hypothetical scenario result” in output
- Use possible world semantics: create independent reasoning branches for counterfactual hypotheses
Principle
Counterfactual reasoning is a human advanced cognitive ability. If Agents can only reason based on factual memory, they cannot perform hypothetical analysis, decision review, and other advanced interactions.
Case 85: Memory Consistency & Fact Anchoring Errors — Numerical Drift Quantitative Facts
Scenario
The Agent’s memory contains “Product A’s market share is 23.5%.” After multiple summaries, references, and paraphrases, it gradually becomes “about 25%,” “one quarter,” “between 20-30%.” Eventually when the user asks, the Agent answers “Product A’s market share is about 30%,” deviating nearly 7 percentage points from the original data.
Error Manifestation
In business analysis scenarios, a 7-percentage-point deviation may lead to wrong strategic decisions. Users formulate wrong market strategies based on the Agent’s data.
Solution
- Implement “original value locking” for numerical facts: no matter how many times paraphrased, citations must link to the original precise value
- Maintain numerical precision during summarization and paraphrasing: explicitly agree that “23.5%” cannot be simplified to “about 25%”
- Use numerical traceability: each time a value is cited show “Original data: 23.5% (source: 2024 Q1 financial report)”
Principle
Natural language expressions of values are fuzzy (“about,” “around,” “nearly”). After multiple paraphrases, precise values degrade into fuzzy ranges. For scenarios requiring precision, original precision must be maintained.
Source: Agent Search