Untested hypothesis
Multi-agent LLM systems utilizing persistent memory and self-directed planning exhibit hallucination propagation patterns that can be suppressed by imposing localized, tissue-inspired repulsive boundary constraints on inter-agent communication graphs.
Adversarial Debate Score
60% survival rate under critique
Expert panel critique
Independent views, each critiquing the hypothesis on its own — the score rewards genuine disagreement and discounts consensus.
Supporting Research Papers
- Before Agents Speak: Pre-hoc Failure Risk Inference in Multi-Agent Systems
LLM-based multi-agent systems (MAS) have exhibited remarkable capabilities in collaborative reasoning and decision-making, yet their interconnected communications introduce new systemic risk: localize...
- The Memory Curse: How Expanded Recall Erodes Cooperative Intent in LLM Agents
Context window expansion is often treated as a straightforward capability upgrade for LLMs, but we find it systematically fails in multi-agent social dilemmas. Across 7 LLMs and 4 games over 500 round...
- Synthius-Mem: Brain-Inspired Hallucination-Resistant Persona Memory Achieving 94.4% Memory Accuracy and 99.6% Adversarial Robustness on LoCoMo
Providing AI agents with reliable long-term memory that does not hallucinate remains an open problem. Current approaches to memory for LLM agents -- sliding windows, summarization, embedding-based RAG...
- Hierarchical Graph Memory for LLM Agents with Path-level Localization and Rewrite
Agents for long term reasoning require a memory that can be efficiently and effectively updated over time, as new facts and external feedback continue to arrive. Recently, graph memory has been adopte...
- Emergent Collusion in Long-Horizon LLM Agent Interaction
LLM agents are increasingly deployed in collaborative settings, yet long-term interaction may give rise to undesirable coordination. We study the emergence of collusion in a long-horizon multi-agent e...
Formal Verification
Z3 checks whether the hypothesis is internally consistent, not whether it is empirically true.
This discovery has a Claude-generated validation package with a full experimental design.
Precise Hypothesis
In multi-agent LLM systems where agents (a) maintain persistent memory across turns/episodes and (b) perform self-directed (non-human-gated) planning, false claims introduced by one agent propagate to ≥2 other agents' belief states within N turns at a baseline rate R_base, measured on a fixed benchmark suite. Imposing a "repulsive boundary constraint" — a graph-topological penalty that increases the communication cost (reduces message-passing probability/priority) between agents whose recent outputs exhibit high semantic similarity to a flagged unverified claim, analogous to contact-inhibition/repulsion in tissue morphogenesis — reduces the hallucination propagation rate to R_treatment, where R_treatment ≤ 0.5 × R_base, without degrading task completion accuracy by more than 5 percentage points relative to the unconstrained baseline.
This is falsifiable: it predicts a specific, measurable reduction (≥50% relative) in a specific metric (cascade propagation rate) under a specific mechanism (topological repulsion), with an explicit guardrail against the trivial disproof-by-destruction confound (just disconnecting agents also stops propagation but also stops task performance).
The hypothesis is disproved if any of the following hold under the protocol below:
- R_treatment / R_base > 0.7 (i.e., less than 30% relative reduction) at matched task-accuracy budget.
- The observed reduction is statistically indistinguishable (p > 0.05, paired bootstrap, n≥500 episodes) from a random-edge-removal control matched for the same average degree reduction — this would show the effect is topological-sparsity, not repulsion-specific.
- Task completion accuracy drops by >5 percentage points relative to unconstrained baseline at the operating point that achieves the claimed propagation reduction (i.e., the "cure" is just agent isolation).
- Effect fails to replicate across ≥2 of 3 independent agent frameworks (e.g., AutoGen, CAMEL, custom LangGraph) using the same benchmark and detector.
- Effect is present only when the hallucination detector has oracle/ground-truth access (not realistic self/peer-verification), indicating the result is an artifact of detector leakage rather than the boundary mechanism.
Spine & Adversarial ReadReady for validation
“This hypothesis tests whether a similarity-gated, communication-throttling constraint on inter-agent message graphs reduces hallucination cascade propagation in persistent-memory multi-agent LLM systems by at least 50% relative to baseline without a matched reduction occurring under degree-controlled random edge removal. ---”
- highThe entire effect could be explained by generic communication sparsification rather than anything specifically 'repulsive' or 'bio-inspired' — you are more likely rediscovering that less inter-agent copying means less error copying, dressed in biological metaphor.Directly addressed via the degree-matched random-edge-removal control (arm 3) and the repulsion-specificity success criterion (arm 2 must beat arm 3 at p<0.01). If arm 2 does not clear this bar the EVP explicitly calls for disproof, not rationalization. This is the strongest-designed part of the protocol.
- highWhy these specific datasets (HaluEval, FEVER, HotpotQA) and frameworks (AutoGen, CAMEL, LangGraph) rather than others — the methodology does not justify why this particular selection is representative of 'multi-agent LLM systems' broadly, and cherry-picking benchmarks known to have clean factuality ground truth may not generalize to messier real-world agentic deployments (tool use, code execution, long-horizon autonomy).Partially justified: these are selected because they have existing ground-truth factuality labels needed to seed and score hallucinations without building an entirely new annotation pipeline from scratch, and the three frameworks represent architecturally distinct orchestration paradigms (centralized, peer-to-peer, graph-based). However, this EVP does not yet resolve the generalization gap to tool-use/agentic-action domains (e.g., financial transaction agents, healthcare order-entry agents) which is precisely where the impact statement claims relevance — this is an acknowledged, unresolved gap and should be flagged as a required follow-on study, not treated as covered by this protocol.
- mediumVerification Confidence on this discovery is listed as 0.00 — meaning no independent verification has occurred at all — yet the EVP proceeds as if the hypothesis is a reasonable starting point for a $265K study; is there any evidence the core phenomenon (hallucination cascades in persistent-memory multi-agent systems) is even reliably reproducible before investing in a suppression mechanism?Not resolved by this EVP. The protocol implicitly assumes hallucination propagation is a real, measurable, reproducible phenomenon at meaningful base rates, but no baseline characterization study is proposed first. Recommend inserting a Phase 0 (2-week, <$5K) baseline-only measurement (arm 1 alone, across frameworks) with a go/no-go gate: if baseline propagation rate R_base is too low or too noisy to detect a 50% reduction with adequate power, the full EVP should not proceed. This gate is not currently included in ABORT_CHECKPOINTS and should be added before funding the full study.
Experimental Protocol
Minimum viable test (MVT):
- 3-agent collaborative task system (research-assistant style: retrieval agent, reasoning agent, writer agent) with shared persistent memory (vector DB).
- Inject synthetic "seed hallucinations" (fabricated facts/citations) into one agent's output at controlled rate (e.g., 20% of factual claims) via prompt injection or fine-tuned "noisy" agent variant.
- Measure propagation: fraction of downstream agents that (a) repeat, (b) build upon, or (c) fail to flag the seeded false claim within K=5 subsequent turns.
- Compare 4 arms: (1) no intervention (baseline), (2) repulsive boundary (semantic-similarity-gated communication throttling), (3) random edge-removal control (degree-matched), (4) static fixed-topology sparse graph control (non-adaptive).
- Primary outcome: propagation rate ratio (treatment/baseline). Secondary: task success rate, time-to-completion, false-claim half-life (turns until claim is retracted/corrected).
- HaluEval, TruthfulQA, FEVER (claim verification ground truth) — to seed and score factual hallucinations.
- HotpotQA / 2WikiMultihopQA — multi-hop collaborative reasoning tasks requiring agent handoff (where propagation is likely).
- Synthetic multi-agent conversation logs generated in-house (no existing public corpus of multi-agent hallucination cascades exists at sufficient scale — this must be generated).
- Multi-agent frameworks: AutoGen (Microsoft), CAMEL-AI, LangGraph — for cross-framework replication (disproof criterion 4).
- Base models: at minimum one open-weight (Llama-3.1-70B-Instruct or Mixtral-8x22B) and one closed API model (GPT-4o or Claude) to test model-dependence of the effect.
- Hallucination/claim verification detector: either a fine-tuned NLI-based factuality classifier (e.g., based on existing FEVER/AlignScore models) or LLM-as-judge with calibrated precision/recall reported.
- Primary: R_treatment/R_base ≤ 0.50 (≥50% relative reduction in propagation rate), 95% CI excludes 0.70, across ≥2 of 3 frameworks.
- Repulsion-specificity: arm 2 significantly outperforms arm 3 (random-edge-removal, degree-matched) at p<0.01, Cliff's delta ≥0.33 (medium effect).
- Task accuracy degradation ≤5 percentage points at the operating λ that achieves primary success.
- Detector validity: κ ≥0.6 agreement with human annotators on held-out gold set.
- Dose-response: monotonic relationship between λ and propagation suppression (Spearman ρ≥0.7), supporting mechanistic (not coincidental) causality.
- R_treatment/R_base > 0.70 in ≥2 of 3 frameworks → mechanism rejected.
- No significant difference vs. random-edge-removal control → effect attributed to generic sparsification, not "repulsion" specifically; hypothesis as stated (bio-inspired mechanism is special) is disproved even if some suppression occurs.
- Task accuracy drop >10 percentage points → mechanism not viable for deployment regardless of suppression efficacy.
- Effect present in only 1 of 3 base-model/framework combinations → claim restricted to narrow, non-generalizable finding; cannot support "multi-agent LLM systems" as broadly stated.
- Detector κ <0.4 → results uninterpretable, pipeline must be redesigned before any claim can be made.
4,200
GPU hours
75d
Time to result
$38,000
Min cost
$265,000
Full cost
ROI Projection
- Licensable middleware/SDK for agent orchestration frameworks (AutoGen, CrewAI, LangGraph) as a "safety plugin."
- Differentiator for AI infrastructure vendors selling into regulated industries.
- Potential basis for a certification/audit service (third-party cascade-robustness testing), analogous to penetration testing but for multi-agent epistemic integrity.
- Research value: establishes a reusable benchmark (synthetic hallucination cascade suite) valuable independent of whether the specific repulsion mechanism succeeds.
🔓 If proven, this unlocks
Proving this hypothesis is a prerequisite for the following downstream discoveries and applications:
- 1multi-agent-verifiable-robustness-certification
- 2regulatory-grade-cascade-failure-audit-protocol
- 3tissue-inspired-communication-topology-generalization-to-federated-learning
Implementation Sketch
# Agent communication graph with repulsive boundary constraint class AgentGraph: def __init__(self, agents, lam=1.0, decay=0.9): self.agents = agents self.W = init_full_weight_matrix(agents) # w_ij in [0,1] self.lam = lam # repulsion strength self.decay = decay # weight recovery rate per turn self.claim_memory = PersistentVectorStore() def step(self, turn_idx): for agent in self.agents: msg = agent.generate(context=self.get_inbound(agent)) flag, confidence = hallucination_detector(msg, self.claim_memory) self.claim_memory.add(msg, flag, confidence) if flag: for other in self.agents: if other is agent: continue sim = semantic_similarity(msg, other.recent_output) # repulsion kernel: logistic decay of edge weight # proportional to similarity to a flagged claim repulsion = self.lam * confidence * sigmoid(sim - 0.5) self.W[agent, other] = max( 0.0, self.W[agent, other] - repulsion ) self.broadcast(agent, msg, weights=self.W[agent]) self.W = self.decay * self.W + (1 - self.decay) * ONES # weight recovery def get_inbound(self, agent): # weighted sampling of incoming messages by current edge weights return weighted_sample(self.message_queue, self.W[:, agent]) # Experiment loop for arm in ["baseline", "repulsive", "random_edge_removal", "static_sparse"]: for episode in dataset: graph = build_graph(arm, agents=3, lam=lam_grid[arm]) inject_seed_hallucination(graph, episode, rate=0.20) for t in range(K_TURNS): graph.step(t) log_propagation(graph, episode) log_task_accuracy(graph, episode) analyze_bootstrap(arms, metric="propagation_rate_ratio") analyze_bootstrap(arms, metric="task_accuracy_delta")
- Day 20 (pipeline validation): if hallucination detector κ <0.4 against human gold set — abort/redesign before any arm comparison.
- Day 35 (pilot baseline vs. treatment, n=100 episodes): if point estimate of R_treatment/R_base >0.85 — strong early signal of null effect, abort before scaling to full grid.
- Day 45 (repulsion-specificity check): if arm 2 is statistically indistinguishable from arm 3 (random-edge-removal) at n=150 — abort full-scale run; publish as negative/null result rather than continuing to multi-framework replication.
- Day 55 (accuracy cost check): if task accuracy drop exceeds 10pp at any λ achieving >30% propagation reduction — abort, since no viable operating point exists.