solver.press

Patient-selection-style analogue for catalysis: site-selection biomarker for when Hamiltonian-RL is beneficial: A computable 'heterogeneity biomarker' (e.g., variance of Bader charge across surface atoms, distributional entropy of local coordination environments, or KL-divergence between site-descriptor clusters) predicts whether Hamiltonian entropy-weighted RL will outperform GP-UCB by ≥20% in oracle-call reduction. Hypothesis: selecting only surfaces with biomarker above threshold yields a statistically significant improvement vs applying HRL indiscriminately. Test by training a classifier/regressor on simulation datasets to predict HRL-vs-GP-UCB advantage and validating prospectively on held-out alloy systems.

Computer ScienceAug 15, 2026Evaluation Score: 70%

Adversarial Debate Score

53% survival rate under critique

Expert panel critique

Independent views, each critiquing the hypothesis on its own — the score rewards genuine disagreement and discounts consensus.

Mistral: The hypothesis is well-motivated, falsifiable, and grounded in validated computational catalysis and RL literature, but its predictive power depends on untested generalization across alloy systems and lacks direct empirical support from the owner’s experiments (which focus on precision/RL in drug...
ChatGPT: 5 The hypothesis is clearly falsifiable and prospectively testable, with a sensible held-out-alloy design. However, the cited evidence is largely indirect, the validated UCB result establishes a strong comparator rather than supporting HRL, and the proposed biomarkers, ≥20% effect size, and thre...
Claude: The hypothesis is conceptually coherent and falsifiable — a classifier predicting HRL-vs-GP-UCB advantage from surface heterogeneity descriptors is a testable, operationalizable idea with precedent in patient-stratification and adaptive algorithm-selection literature — but it rests on an unva...
Grok: Falsifiable and conceptually coherent (alloy-site heterogeneity as a selection biomarker is a reasonable analogue to AEDs/patient stratification), but owner-validated experiments supply no supporting evidence in catalysis or HRL—only unrelated precision and docking BO results—so the ≥20% claim an...

Supporting Research Papers

Computational Result

📖 Literature-assessed (LLM)· literature_meta

An LLM's reading of the literature — not computational verification.

Heterogeneity biomarkers may enhance RL performance in catalysis.

Method: literature_meta · Result: inconclusive · Confidence: 60%

Formal Verification

Z3 logical consistency:✅ Consistent

Z3 checks whether the hypothesis is internally consistent, not whether it is empirically true.

Experimental Validation Package

This discovery has a Claude-generated validation package with a full experimental design.

Precise Hypothesis

There exists a computable "surface heterogeneity biomarker" H(S) — defined as a scalar or low-dimensional function of (a) variance of Bader charge across symmetry-inequivalent surface atoms, (b) Shannon entropy of local coordination-number/generalized-coordination-number distributions, and/or (c) KL-divergence between clusters of site-level descriptor vectors (e.g. d-band center, coordination, strain) — such that: for catalyst surfaces S with H(S) > τ (a fixed, pre-registered threshold), Hamiltonian entropy-weighted reinforcement learning (HRL) reduces the number of oracle calls (DFT single-points) required to reach a fixed regret/accuracy target by ≥20% relative to GP-UCB, with statistical significance (paired one-sided test, α=0.05, effect size Cohen's d≥0.5) across a held-out set of alloy surfaces; and for S with H(S) ≤ τ, HRL shows no significant advantage (mean improvement <10%, or GP-UCB non-inferior). The claim is specifically that H(S) is predictive (AUC ≥ 0.75 for the binary "HRL beats GP-UCB by ≥20%" classification task), not merely correlated in-sample.

Disproof criteria:
  1. Classifier/regressor trained on H(S) → (HRL advantage) achieves held-out AUC < 0.65 (chance-adjacent) or R² < 0.15 across ≥3 independent alloy families.
  2. Selecting surfaces with H(S) > τ (chosen via cross-validation, not post-hoc) does NOT yield significant advantage separation vs. random surface selection (permutation test p > 0.05).
  3. HRL fails to beat GP-UCB by ≥20% even on the highest-heterogeneity decile of held-out surfaces (i.e., biomarker points to the wrong regime entirely).
  4. The relationship inverts (high heterogeneity predicts GP-UCB superiority) on ≥2 of 4 held-out alloy test systems, indicating the biomarker captures noise/non-causal structure rather than a real optimization-landscape property.
  5. Effect vanishes or reverses when oracle-call budget is doubled/halved, indicating the "advantage" was an artifact of a specific budget regime rather than a heterogeneity-linked property.

Spine & Adversarial Read

  • highThe EVP has no citation to an actual, validated implementation of 'Hamiltonian entropy-weighted RL' — without a concrete, peer-reviewed or at least code-verified baseline, the entire comparison (HRL vs GP-UCB) risks being built on an under-specified or under-tuned method, making any 'advantage' finding uninterpretable.
    Unresolved gap: this EVP requires, as a Phase -1 prerequisite (not currently budgeted separately), an independent validation that the HRL implementation used matches or exceeds published/claimed performance on a standard benchmark before it is used as the comparator in the heterogeneity-biomarker study. Recommend adding a dedicated HRL-implementation-verification milestone with its own abort checkpoint.
  • highWhy these specific descriptors (Bader charge variance, coordination entropy, KL-divergence) and not the extensive existing landscape-feature toolkit from Exploratory Landscape Analysis (ELA) used in optimization algorithm-selection research? The methodology choice is not justified against known alternatives, risking a 'reinvented wheel with worse features' critique.
    Partially addressed: physically-grounded descriptors (Bader charge, coordination) have domain interpretability advantages over generic ELA features for catalysis practitioners, but no ablation against standard ELA feature sets (dispersion, ruggedness, information-content metrics) is currently in the protocol. Should add a baseline comparison arm using generic ELA features vs. the proposed physical biomarkers to justify the specific feature choice quantitatively, not just qualitatively.
  • mediumUsing an OC20-pretrained GNN as an oracle proxy for DFT introduces a confound: the GNN's own error surface may itself be a function of surface heterogeneity (poor training coverage on unusual/heterogeneous alloys), meaning any 'HRL advantage on heterogeneous surfaces' could reflect the GNN oracle's noise structure rather than a true optimization-algorithm property.
    Partially resolved in protocol via the Phase 0 requirement that surrogate MAE be validated against real DFT on a ≥5% subsample with an abort checkpoint at MAE>0.1eV; however, this checks *average* error, not whether error is *heteroscedastic with respect to H(S)* specifically — a stratified error analysis by heterogeneity decile should be added before trusting surrogate-based conclusions.

Experimental Protocol

Design: Simulation-based supervised learning + prospective held-out validation, structured as a train/validate/test split over alloy systems (not just data points), to avoid leakage across chemically similar surfaces.

  • Training corpus: N₁ = 40–60 alloy surface systems (binary alloys × 3–4 facets × 2–3 adsorbates = ~300–500 optimization "runs"), each run producing (H(S) features, HRL-vs-GP-UCB oracle-call advantage %).
  • Validation corpus: N₂ = 10–15 held-out alloy systems, same protocol, used for threshold calibration and early stopping.
  • Prospective test corpus: N₃ = 8–12 completely held-out alloy systems (different metal pairs, ideally including at least 2 ternary or unusual-facet systems to test extrapolation) — biomarker + classifier applied blind, HRL vs GP-UCB run to completion, actual advantage measured and compared to prediction.
  • Baselines per run: GP-UCB (standard exploration-exploitation, Matérn kernel), HRL (Hamiltonian entropy-weighted variant as defined in the parent paper), and a naive random-sampling baseline as a sanity floor.
  • Repeats: Each (surface, method) pair run with ≥10 random seeds to estimate variance in oracle-call-to-target.
Required datasets:
  • Open Catalyst Project (OC20 / OC22) — pretrained GNN surrogates (e.g., GemNet-OC, EquiformerV2) to serve as a fast, validated oracle proxy for DFT adsorption energies, enabling large N without full ab initio cost.
  • Catalysis-Hub.org and NOMAD — DFT-computed adsorption/reaction datasets for cross-validation of surrogate oracle accuracy against real DFT on a subset.
  • Materials Project — bulk alloy structures and Bader charge / DOS data for slab generation and d-band center computation.
  • In-house or newly generated slab library: ASE + pymatgen-generated surface slabs for the chosen binary alloys (Pt-Ni, Pt-Co, Cu-Ni, Ag-Au, Pd-Cu, Au-Pd, Ni-Fe, etc.), 3 low-index facets each, spin-polarized DFT relaxation for a validation subset (~5% of total) using VASP or Quantum ESPRESSO.
  • Bader charge analysis toolkit (Henkelman group bader code) for heterogeneity feature computation.
  • Existing HRL and GP-UCB codebases — must be obtained/re-implemented; none provided in the discovery materials (this is the single largest methodological gap — see ADVERSARIAL_OBJECTIONS).
Success:
  • Classifier AUC ≥ 0.75 (stretch: ≥0.85) on prospective held-out test systems (N₃≥8) for the binary "≥20% advantage" label.
  • Regressor R² ≥ 0.40 for continuous advantage% prediction on held-out systems.
  • Selecting only H(S)>τ surfaces yields mean measured advantage ≥20% with p<0.05 (paired, one-sided) vs. GP-UCB, replicated in ≥6 of 8-12 test systems.
  • Below-threshold surfaces show mean advantage <10% (confirming discriminative, not just additive, value of the biomarker).
  • Ablation shows ≥1 individual feature component achieves AUC ≥0.65 alone (interpretability / mechanism signal, not just ensemble black-box).
Failure:
  • AUC <0.65 or R² <0.15 on prospective test set → biomarker not predictive; hypothesis disproven as stated.
  • Advantage does not exceed 20% even in top-heterogeneity decile → HRL's benefit is not heterogeneity-gated at all (may still have unconditional value, but this specific hypothesis fails).
  • Effect size collapses (Cohen's d <0.2) when oracle surrogate is swapped for real DFT on the validation subsample → result is a surrogate-model artifact, not a physical/algorithmic property.
  • Threshold τ is unstable (>50% relative change) across bootstrap resamples of the training set → biomarker lacks robustness for practical deployment.

ROI Projection

Commercial:

Directly monetizable as a pre-screening module/plugin for computational chemistry SaaS platforms (e.g., integrated into Citrine, Kebotix, or internal pharma/materials discovery pipelines) sold as a "compute governor" for active-learning campaigns. Estimated addressable market: mid-size materials/catalysis computational teams spending $500K–$5M/yr on HPC/DFT cycles; a 15-30% efficiency gain is a defensible ROI pitch for a $50K-$250K/yr licensing/consulting engagement per enterprise customer. Secondary value as an academic open-source contribution (citation/reputation value) and as a template methodology generalizable to any RL-vs-classical-optimizer selection problem (drug screening, robotic experimentation, battery materials).

TIME_TO_RESULT_DAYS: 120

Implementation Sketch

# Phase 0: Feature + label generation
for alloy in training_systems:
    slab = generate_slabs(alloy, facets=[111,100,211])
    relaxed = surrogate_relax(slab)  # OC20 GNN
    bader_var = compute_bader_charge_variance(relaxed)
    coord_entropy = shannon_entropy(coordination_numbers(relaxed))
    kl_div = kl_divergence(cluster(site_descriptors(relaxed)))
    H = feature_vector(bader_var, coord_entropy, kl_div)

    for seed in range(10):
        calls_gpucb = run_GPUCB(surrogate_oracle, slab, target_eps=0.02, seed)
        calls_hrl   = run_HamiltonianRL(surrogate_oracle, slab, target_eps=0.02, seed)
    advantage_pct = mean_reduction(calls_gpucb, calls_hrl)
    dataset.append((H, advantage_pct))

# Phase 1: Model fitting
X, y = dataset.features, dataset.advantage
clf = GradientBoostedTrees()  # binary: advantage>=20%
reg = GradientBoostedTrees()  # continuous advantage%
nested_cv(clf, reg, X, y, outer_k=5, inner_k=3)
tau = youden_threshold(clf, validation_set)

# Phase 2: Prospective test (frozen model)
for alloy in held_out_test_systems:
    H_test = compute_biomarker(alloy)
    predicted_advantage = reg.predict(H_test)
    predicted_class = clf.predict(H_test) > tau

    actual_advantage = run_full_comparison(alloy)  # real HRL vs GP-UCB
    log(predicted_advantage, actual_advantage, predicted_class)

evaluate(AUC, R2, calibration_curve, paired_significance_test)
Abort checkpoints:
  1. After Phase 0 data generation (training set complete): if surrogate-oracle MAE vs. real DFT validation subsample >0.1 eV, abort and re-select oracle before any further spend.
  2. After initial classifier fit (nested CV on training set only): if in-sample cross-validated AUC <0.60, abort — no signal exists even before touching held-out systems.
  3. After validation-set threshold calibration: if τ estimate has bootstrap 95% CI spanning >60% of the H(S) range, abort/redesign biomarker before prospective test (avoids wasting compute on an unstable threshold).
  4. Mid-prospective-test (after first 4 of N₃ test systems): if predicted vs actual advantage direction is wrong on ≥3 of 4, abort remaining test systems and report negative result rather than completing full N₃ at full cost.

NAMED_EXPERTS: []

CLOSEST_EXISTING_WORK: []

NOVELTY_NARROWING_REQUIRED: true

SPINE_STATEMENT: This hypothesis tests whether a pre-computable surface heterogeneity biomarker can reliably predict, ahead of time, on which catalyst surfaces Hamiltonian entropy-weighted reinforcement learning will reduce oracle calls by at least 20% relative to GP-UCB.

Source

AegisMind Research
Need AI to work rigorously on your problems? AegisMind uses the same multi-model engine for personal and professional use. Get started