solver.press

Untested hypothesis

Segmenting existing lesions (SWIMSS) and forecasting new ones (FORESEE-MS) connect directly: a segmentation model's voxel-level output on a patient's baseline scan could supply the lesion-burden and lesion-proximity features that the forecasting task lacks, if forecasting is not already saturated by the population prior.

MedicineNov 21, 2026Evaluation Score: 74%

Adversarial Debate Score

66% survival rate under critique

Expert panel critique

Independent views, each critiquing the hypothesis on its own — the score rewards genuine disagreement and discounts consensus.

Gemini: Assessment: The hypothesis is logical, falsifiable, and directly bridges the functional gap identified between segmentation (SWIMSS) and spatial forecasting (FORESEE-MS) without relying on any refuted internal findings. However, its success hinges on whether voxel-level features provide n...
Claude: The hypothesis is falsifiable and plausible. Segmentation-derived lesion burden and proximity features are a natural, testable addition to a baseline-FLAIR forecasting pipeline, and the conditional "if forecasting is not already saturated by the population prior" is a clear ablation (populati...
ChatGPT: The hypothesis is plausible and falsifiable: baseline segmentation can supply lesion-burden and proximity features, whose incremental value should be tested against population-prior and baseline-image-only models on held-out patients. However, the excerpts establish task compatibility rather than...
Grok: Plausible and falsifiable link between complementary MS tasks, but owner's validated experiments are wholly unrelated (precision/drug-discovery dead-ends) so add no support, and the saturation caveat plus lack of direct evidence leave it speculative.

Supporting Research Papers

Formal Verification

Z3 logical consistency:✅ Consistent

Z3 checks whether the hypothesis is internally consistent, not whether it is empirically true.

Experimental Validation Package

This discovery has a Claude-generated validation package with a full experimental design.

Precise Hypothesis

In relapsing or progressive MS patients undergoing serial brain MRI, voxel-level outputs from a trained lesion segmentation model (SWIMSS-class architecture: per-voxel lesion probability maps, lesion instance masks, lesion-boundary distance transforms) computed on a baseline scan, when added as input features to a lesion-forecasting model (FORESEE-MS-class architecture predicting 12-month new-lesion locations), will improve forecasting performance — measured as voxel-wise AUPRC for new T2/FLAIR lesion appearance and per-lesion F1 at ≥3mm tolerance — by a statistically significant margin (≥5 percentage points AUPRC, two-sided paired bootstrap p<0.05, n≥150 held-out patients) over a forecasting model using only (a) raw baseline imaging and (b) population-level priors (lesion-probability atlas, demographic/EDSS covariates), UNLESS the population prior already explains ≥90% of achievable predictive variance (defined via an oracle upper-bound using ground-truth baseline lesion masks), in which case the hypothesis is falsified for that cohort/model regime.


Disproof criteria:
  • Segmentation-derived features yield <2 percentage points AUPRC improvement (or any degradation) over the population-prior-only baseline, with 95% CI excluding the pre-registered 5-point threshold, across ≥2 independent cohorts.
  • An ablation showing the population prior alone achieves ≥90% of the oracle (ground-truth mask) upper bound — i.e., forecasting is already saturated and segmentation features are redundant.
  • Improvement is observed on the training-distribution cohort but fails to replicate (effect size shrinks by >50%) on an external cohort with different scanner/protocol, indicating the effect is a harmonization artifact rather than a genuine biological/predictive signal.
  • Feature importance analysis (SHAP/permutation) shows segmentation-derived features contribute <5% relative importance versus baseline covariates (lesion count, EDSS, age, prior relapse rate).

Spine & Adversarial Read

“Adding voxel-level lesion segmentation outputs as input features to a 12-month MS lesion-forecasting model improves new-lesion prediction (AUPRC) by at least 5 percentage points over a population-prior-only baseline, unless the population prior already captures nearly all forecastable signal. ---”

  • highThe claimed improvement may simply be re-deriving lesion count/volume, a feature any reasonable forecasting baseline would already include — the hypothesis risks being trivially true (segmentation outputs correlate with lesion burden, lesion burden predicts new lesions) and is being dressed up as a novel architectural insight.
    Addressed via the SHAP/feature-importance step (Methodology #9) and the explicit requirement that Arm A already includes lesion-count/volume covariates, so Arm C's marginal gain must come from spatial/distance/shape features specifically. This is a real test, but the EVP does not yet guarantee Arm A's covariate set is maximally strong — a skeptic could argue the baseline was under-specified. Gap: pre-register the exact Arm A feature list with domain experts before unblinding to preempt this critique.
  • mediumWhy nnU-Net and this specific MSSEG-2/forecasting-cohort combination rather than other published segmentation architectures (e.g., transformer-based lesion segmentation) or other longitudinal cohorts (e.g., ADNI-MS equivalents, OFSEP's full registry)? The methodology choice is not justified against alternatives, which is a common rejection cause.
    Partial justification given: nnU-Net is the de facto reproducible benchmark for medical image segmentation with self-configuring architecture search, minimizing researcher-degrees-of-freedom in the segmentation arm so that the forecasting comparison (the actual hypothesis under test) isn't confounded by segmentation-architecture tuning choices. MSSEG-2/OFSEP are chosen for expert-consensus mask availability. However, the EVP does not run a sensitivity analysis across alternative segmentation backbones (e.g., SynthSeg, transformer U-Net) to show the result is architecture-agnostic — this is an acknowledged gap, addressable as an optional Phase 2 extension at +15% cost.
  • highNo real forecasting cohort (≥150 patients, 2 timepoints, DUA-accessible) is currently identified or secured — the entire protocol is contingent on a data-access step that is unresolved and could independently sink the timeline/budget regardless of the hypothesis's truth.
    This is an explicit, unresolved gap. REQUIRED_DATASETS flags the longitudinal forecasting cohort as 'the critical gap... not currently in-repo.' No mitigation beyond recommending OFSEP/clinical-trial-placebo-arm data-use-agreement pursuit is offered here; TIME_TO_RESULT_DAYS and COST estimates assume this access is secured within the first 20-30 days, which is optimistic and should be treated as the single largest schedule/budget risk in this EVP.

Experimental Protocol

Design: Multi-arm ablation study, paired comparison, nested cross-validation with held-out external cohort.

Arms:

  • Arm A (control): Forecasting model with population-prior features only (lesion-probability atlas registered to patient space, demographics, EDSS, treatment).
  • Arm B (raw-image): Forecasting model with baseline raw FLAIR/T1 as CNN input, no explicit segmentation.
  • Arm C (test): Forecasting model with Arm A + B features PLUS voxel-level segmentation outputs (probability map, binary mask, distance-to-nearest-lesion-boundary map, lesion-size/shape features per connected component).
  • Arm D (oracle ceiling): Same as C but using ground-truth expert-annotated baseline masks instead of model-predicted segmentation (defines upper bound / saturation test).

Primary endpoint: Voxel-wise AUPRC for new-lesion appearance at 12 months, paired bootstrap (10,000 resamples) comparing Arm C vs Arm A. Secondary endpoints: Per-lesion F1 (3mm tolerance), lesion-count MAE, NBV (new T2 lesion volume) Spearman correlation, calibration (Brier score).


Required datasets:
  1. ISBI 2015 Longitudinal MS Lesion Segmentation Challenge (5 patients, 4–5 timepoints) — pilot/sanity-check only, too small for primary analysis.
  2. MSSEG-2 / MSSEG Challenge datasets (OFSEP consortium, ~100 patients, expert consensus masks) — segmentation model training/validation.
  3. A longitudinal forecasting cohort with ≥150 patients, 2 timepoints 6–24 months apart, e.g., an internal/clinical-trial placebo-arm dataset (candidate: OFSEP, UBC MS clinic, or a pharma trial placebo arm such as a historical interferon/fingolimod RCT with imaging substudy) — this is the critical gap; must be sourced via DUA, not currently in-repo.
  4. Shift/external validation cohort: different scanner vendor/site (for generalization test) — e.g., a second OFSEP site or ADNI-style multi-site MS cohort if available.
  5. Pretrained segmentation backbones: nnU-Net, SynthSeg-lesion variant, or published SWIMSS checkpoint if obtainable; otherwise train nnU-Net from scratch on MSSEG-2.
  6. Compute environment: PyTorch ≥2.1, MONAI ≥1.3, nnU-Net v2, scikit-learn, SimpleITK/ANTs for registration, FSL/ComBat-GAM for harmonization.
  7. No genomics/transcriptomics data required for this imaging hypothesis — the GEO/CELLxGENE/GTEx resources listed in the context belong to a separate (molecular) discovery track and are not inputs here.

Success:
  • Arm C beats Arm A by ≥5 percentage points AUPRC, bootstrap p<0.05, replicated in external cohort (effect retains ≥50% of internal-cohort magnitude).
  • Per-lesion F1 improvement ≥0.08 absolute.
  • SHAP/Grad-CAM confirms segmentation-distance and probability-map features rank in top-5 feature importances (not dominated by trivial lesion-count proxy already in Arm A).
  • Oracle ceiling (Arm D) shows population prior explains <90% of achievable AUPRC (i.e., headroom exists) — necessary precondition for the hypothesis to even be testable/true.
  • Calibration (Brier score) for Arm C is non-inferior to Arm A (±0.01).

Failure:
  • AUPRC improvement <2 points or not statistically significant (p≥0.05) in primary cohort.
  • Improvement present internally but vanishes (>50% shrinkage) externally — indicates overfitting/harmonization artifact, not genuine transfer.
  • Arm A already achieves ≥90% of Arm D oracle ceiling — forecasting saturated by population prior, confirming the hypothesis's own stated disproof condition.
  • Segmentation Dice <0.70 on the forecasting cohort's baseline scans (pipeline not viable in this population regardless of hypothesis truth).

2,400

GPU hours

150d

Time to result

$45,000

Min cost

$185,000

Full cost

ROI Projection

Commercial:

Moderate-to-high if validated: a validated segmentation-to-forecasting pipeline is licensable as a clinical decision-support / trial-enrichment tool to pharma sponsors running MS imaging-endpoint trials (estimated addressable market: 15-20 active MS DMT trials/year globally with imaging substudies) and to imaging CROs. Standalone commercial value is capped without regulatory clearance (FDA SaMD pathway would require a separate, substantially larger validation program, est. $3-8M, 2-3 years). Near-term value is primarily as a research tool and trial-design accelerator, not a standalone diagnostic product.


Implementation Sketch

# Stage 1: Segmentation
seg_model = nnUNetv2.train(MSSEG2_data, folds=5)
assert seg_model.dice_score(val_set) >= 0.70

for patient in forecasting_cohort:
    baseline_scan = load_flair(patient.baseline)
    prob_map, mask = seg_model.predict(baseline_scan)
    dist_map = euclidean_distance_transform(mask)
    lesion_shapes = extract_connected_component_features(mask)
    patient.seg_features = {prob_map, mask, dist_map, lesion_shapes}

# Stage 2: Population prior (leakage-safe, built on separate split)
atlas = build_lesion_incidence_atlas(train_split_only)
patient.prior_features = {atlas_value_at_voxel, age, sex, EDSS, tx, relapse_rate}

# Stage 3: Forecasting arms
model_A = Forecaster(features=prior_features)
model_B = Forecaster(features=raw_image_CNN_embedding)
model_C = Forecaster(features=prior_features + seg_features)
model_D = Forecaster(features=prior_features + ground_truth_mask_features)  # oracle

for model in [A, B, C, D]:
    model.fit(train_split)
    preds = model.predict(test_split)  # voxel-wise P(new lesion at 12mo)

auprc_C, ci_C = paired_bootstrap_auprc(model_C, model_A, test_split, n_boot=10000)
shap_values = compute_shap(model_C_as_GBM, test_split)
external_auprc = model_C.evaluate(external_cohort, frozen=True)

assert (auprc_C - auprc_A) >= 0.05 and ci_C.p_value < 0.05
assert external_auprc_delta >= 0.5 * internal_auprc_delta

Abort checkpoints:
  1. Day 10: Literature search (currently unavailable in this run) must be completed before full build — if a closely matching published result already exists with a similar effect size, narrow scope/novelty claim before proceeding to full cost.
  2. Day 25: Segmentation model fails to reach Dice ≥0.70 on MSSEG-2 validation — abort/redesign segmentation stage before any forecasting work.
  3. Day 55: Oracle ceiling (Arm D) shows <2-point headroom over Arm A — hypothesis is likely false in this regime; abort before external validation spend (saves ~60% of full budget).
  4. Day 90: Internal-cohort primary endpoint not significant (p≥0.05) — do not proceed to external validation; declare failure per pre-registered criteria.
  5. Day 120: External validation shows >50% effect shrinkage — flag as harmonization-artifact failure mode, halt before commercial-value claims are made.

Source

AegisMind Research
Need AI to work rigorously on your problems? AegisMind uses the same multi-model engine for personal and professional use. Get started