Intermediate-temperature sampling during training of protein language models can be interpreted as entropic regularization of the parameter-space optimal transport problem between initialization and converged distributions.
Adversarial Debate Score
20% survival rate under critique
Model Critiques
Supporting Research Papers
- Performative Scenario Optimization
This paper introduces a performative scenario optimization framework for decision-dependent chance-constrained problems. Unlike classical stochastic optimization, we account for the feedback loop wher...
- ParetoEnsembles.jl: A Julia Package for Multiobjective Parameter Estimation Using Pareto Optimal Ensemble Techniques
Mathematical models of natural and man-made systems often have many adjustable parameters that must be estimated from multiple, potentially conflicting datasets. Rather than reporting a single best-fi...
- On Lipschitzian properties of multifunctions defined implicitly by"split"feasibility problems
In the present paper, a systematic study is made of quantitative semicontinuity (a.k.a. Lipschitzian) properties of certain multifunctions, which are defined as a solution map associated to a family o...
- Sampling at intermediate temperatures is optimal for training large language models in protein structure prediction
We investigate the parameter space of transformer models trained on protein sequence data using a statistical mechanics framework, sampling the loss landscape at varying temperatures by Langevin dynam...
Formal Verification
Z3 checks whether the hypothesis is internally consistent, not whether it is empirically true.