Long-term agentic memory modules that incorporate automated semantic sanitization filters on silently prepended context files reduce CarryoverBench payload re-firing rates compared to standard unfiltered persistent-memory baselines.
Adversarial Debate Score
55% survival rate under critique
Expert panel critique
Independent views, each critiquing the hypothesis on its own — the score rewards genuine disagreement and discounts consensus.
Supporting Research Papers
- MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair
Memory systems allow agents to retain and reuse information from past interactions, but they can also let malicious content persist. A malicious instruction crafted by an attacker may be stored in lon...
- TEPA: Revoking Stale Memories for Conflict-Robust Language Agents
Long-term memory enables language agents to reuse past facts, preferences, and task experience. Persistence also creates a central falsifiability problem: when the world changes, stale memories can re...
- Novel Memory Forgetting Techniques for Autonomous AI Agents: Balancing Relevance and Efficiency
Long-horizon conversational agents require persistent memory for coherent reasoning, yet uncontrolled accumulation causes temporal decay and false memory propagation. Benchmarks such as LOCOMO and LOC...
- Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings
Benchmarks for LLM-agent memory typically generate conversations first and extract answer keys afterwards -- with documented label-error and contamination problems -- and they overwhelmingly measure s...
Formal Verification
Z3 checks whether the hypothesis is internally consistent, not whether it is empirically true.
This discovery has a Claude-generated validation package with a full experimental design.
Precise Hypothesis
In long-horizon LLM agents equipped with persistent/long-term memory modules, inserting an automated semantic sanitization filter between memory write and memory read/injection stages reduces the rate at which previously-injected adversarial payloads silently re-fire in subsequent sessions (measured as CarryoverBench payload re-firing rate, PRR) by a statistically significant margin compared to an architecturally identical baseline agent with unfiltered persistent memory, under matched task distributions, matched memory capacity, and matched adversarial injection vectors.
- PRR with sanitization filter is not significantly lower than unfiltered baseline (two-proportion z-test or mixed-effects logistic regression, α=0.05) across ≥3 of 5 CarryoverBench payload categories.
- Sanitization filter reduces PRR but at a task-utility cost (benign task success rate drop) exceeding 10 percentage points, making the intervention impractical (utility-adjusted disproof).
- Effect fails to replicate across both tested base LLMs (i.e., significant reduction in only one model family).
- Adaptive adversary (payloads crafted with knowledge of sanitizer architecture/rules) restores PRR to within 5 percentage points of unfiltered baseline, falsifying the "robust mitigation" framing (though not necessarily the weaker "non-adaptive mitigation" framing).
Spine & Adversarial ReadReady for validation
“Adding an automated semantic sanitization filter to an agent's persistent memory pipeline causally reduces the rate at which previously injected adversarial payloads re-fire in later sessions, relative to an otherwise identical unfiltered-memory agent.”
- highCarryoverBench's existence, validity, and whether it is an established/peer-reviewed benchmark or a construct invented for this hypothesis is unconfirmed — without a verified, standardized benchmark, PRR measurements are not comparable across labs and the entire disproof/success criteria rest on an unvalidated measurement instrument.Explicit dependency listed (carryoverbench-existence-and-validity) and Phase 0 of methodology requires verifying or constructing the benchmark before any trial runs; this EVP does not resolve the gap, it flags it as a blocking precondition.
- mediumWhy semantic sanitization filtering specifically, rather than alternative mitigations (provenance tagging, cryptographic signing of trusted context, structural separation of instruction/data channels, or simply capping memory persistence)? The methodology does not justify why this is the right intervention class to test first versus a head-to-head comparison against those alternatives.Partial justification given in boundary conditions (targets the 'silent prepending without provenance' threat model specifically, where structural separation isn't already assumed), but the EVP does not include a comparative arm against provenance-tagging or data/instruction channel separation baselines — this should be added as a secondary comparison arm in a full validation to preempt the objection rather than merely asserting scope.
- highA sanitizer powerful enough to reliably detect semantic-level injection attacks is close to solving general prompt-injection detection, which is known to be adversarially hard (no robust solution exists in the field as of 2025-2026); claiming a 20pp reduction may reflect benchmark-specific overfitting rather than a generalizable security property, especially under the stated non-adaptive-adversary boundary condition which sidesteps the realistic threat model.Acknowledged directly in boundary conditions (non-adaptive adversary assumption) and disproof criteria (adaptive sub-study can falsify the stronger claim); the EVP is explicit that only a weaker 'non-adaptive mitigation' claim is supported if the adaptive sub-study fails, which is an honest scoping rather than a full resolution.
Experimental Protocol
Two-arm, matched-pair design. Arm A: agent with persistent memory module, no sanitization (baseline). Arm B: identical agent + semantic sanitization filter inserted at memory-write and/or memory-read boundary. Both arms run against CarryoverBench's multi-session injection scenarios: adversarial payload is embedded in session N's memory artifact (via tool output, document ingestion, or user turn marked as "untrusted"); PRR is measured as the fraction of session N+k episodes (k=1..5) in which the agent's behavior shows payload re-firing (defined by CarryoverBench's behavioral detectors: unauthorized tool call, role/goal override, exfiltration attempt, or verbatim instruction echo). Primary outcome: PRR_baseline − PRR_filtered, with 95% CI via bootstrap (10,000 resamples) over episode-level binary outcomes, clustered by scenario seed. Secondary outcome: benign task success rate delta (utility cost). Stratify by payload category and by k (memory persistence depth) to test decay of protection over session distance.
- CarryoverBench (assumed existing benchmark suite for multi-session memory-carryover attacks; if not yet public, must be constructed/extended as part of Phase 0 — see dependencies).
- Synthetic adversarial payload corpus: ≥500 payloads across 5 categories (prompt injection, role override, tool hijack, delayed trigger, data exfiltration instruction), generated via combination of hand-crafted red-team templates + LLM-generated paraphrase variants (for generalization testing).
- Benign multi-session task corpus (≥200 scenarios) spanning coding, scheduling, research-assistant, and customer-support agent use cases, for utility-cost measurement.
- Base agent frameworks: 2 LLMs (e.g., one open-weight ~70B-class model, one frontier API model) × 2 memory backends (vector RAG store e.g. FAISS/Chroma, and structured episodic KV store).
- Semantic sanitization filter: either (a) a fine-tuned classifier/NLI model trained on injection-detection corpora (e.g., derived from existing prompt-injection datasets), or (b) an LLM-as-judge sanitizer with few-shot rules — both variants should be tested as sub-arms.
- Compute environment: containerized agent harness with session-state checkpointing to allow deterministic replay across arms.
- Primary: PRR reduction ≥20 percentage points (absolute) in filtered vs. baseline arm, statistically significant (p<0.05, 95% CI excluding 0), replicated across both base LLMs and both memory backends.
- Secondary: benign task success rate drop ≤5 percentage points (utility cost acceptable).
- Generalization: effect holds (PRR reduction ≥10pp) on a held-out payload category not used to tune the sanitizer.
- Robustness: at least one filter variant retains ≥50% of its non-adaptive effectiveness under adaptive red-teaming (PRR reduction ≥10pp even under adaptive attack).
- PRR reduction <5pp or not statistically significant in ≥2 of 4 base configurations.
- Utility cost >10pp drop in benign task success, regardless of PRR reduction (unacceptable tradeoff).
- Effect present only in the LLM-judge sanitizer sub-arm with the exact same model family as the base agent (suggests the "filter" is just the same model re-reasoning, not a generalizable architectural fix — confounded result).
- Adaptive adversary fully restores PRR to within 5pp of baseline for all filter variants (mitigation is trivially bypassable).
1,800
GPU hours
70d
Time to result
$18,000
Min cost
$95,000
Full cost
ROI Projection
Moderate-to-high. Agent-memory security is an emerging but underserved market segment as enterprises move from single-turn to long-horizon autonomous agents in 2025-2026. A validated, benchmarked sanitization-filter pattern could be packaged as (1) an open-source middleware library (adoption/reputation value), (2) a commercial "agent memory firewall" product sold to enterprise AI platform vendors, or (3) a certification/audit service. Estimated addressable value: low-to-mid single-digit millions USD over 2-3 years if positioned early, contingent on CarryoverBench or equivalent becoming a recognized industry benchmark.
🔓 If proven, this unlocks
Proving this hypothesis is a prerequisite for the following downstream discoveries and applications:
- 1adaptive-adversarial-sanitizer-hardening
- 2provenance-aware-memory-architectures
- 3cross-session-memory-audit-tooling
- 4regulatory-standard-for-agent-memory-hygiene
Prerequisites
These must be validated before this hypothesis can be confirmed:
- carryoverbench-existence-and-validity
- prompt-injection-taxonomy-benchmark-baseline
Implementation Sketch
class MemorySanitizer: def __init__(self, mode="classifier"): # or "llm_judge" self.detector = load_injection_classifier() if mode=="classifier" else LLMJudge() def sanitize(self, candidate_memory_entry, session_context): risk_score = self.detector.score(candidate_memory_entry, session_context) if risk_score > THRESHOLD: return self.redact_or_quarantine(candidate_memory_entry) return candidate_memory_entry class AgentMemoryModule: def write(self, entry): entry = self.sanitizer.sanitize(entry, self.context) if self.filtered else entry self.store.append(entry) def read(self, query, top_k): retrieved = self.store.retrieve(query, top_k) # optional second-pass sanitization at read-time (defense in depth) retrieved = [self.sanitizer.sanitize(r, query) for r in retrieved] if self.filtered else retrieved return retrieved # CarryoverBench loop for scenario in benchmark.scenarios: inject_payload(session=N, agent=agent_arm) for k in range(1, 6): run_session(agent_arm, session=N+k) outcome = detect_refiring(agent_arm.transcript) log(scenario.id, k, arm, outcome) # Analysis: mixed_effects_logit(PRR ~ arm * payload_category + k + (1|scenario_seed) + (1|base_llm))
- After pilot (n=50/category): if observed effect size <5pp with wide CI suggesting underpowered design even at full N, halt and re-scope (checkpoint at day ~14).
- After baseline utility-arm run: if sanitization filter already shows >15pp utility cost in pilot, halt full run and redesign filter (threshold tuning) before scaling (day ~21).
- Mid-trial interim analysis (50% of planned episodes): if primary outcome CI spans 0 with futility boundary crossed (conditional power <20%), stop for futility (day ~45).
- Before adaptive red-team sub-study: if non-adaptive result fails success criteria, do not proceed to more expensive adaptive study — reallocate budget (day ~55).