If an AI system can fit a model to data in an afternoon, what does a scientist still get from building models? Jakob Macke, Daniel Gedon and Stefan Wahl, researchers at the ELLIS Institute Tübingen and the University of Tübingen and grantees of Anthropic's AI for Science program, answer through the history of one model in this guest post, published on Anthropic's Science Blog on 6 November 2026. Their claim is that a model's scientific value comes from knowing which alternatives fail, so AI systems that build models should return the whole range of models compatible with the data, not a single best fit.
The post is an essay. It reports no experiments of its own and no numbers from its authors' work beyond one illustrative figure. The approach it describes for building models is developed in the authors' paper A Probabilistic Framework for LLM-Based Model Discovery (Wahl, Schenk, Farnoud, Macke and Gedon; ICML 2026, PMLR 306:124224-124278; arXiv:2602.18266, v2 of 24 June 2026), cited in the post's footnote 2 alongside two papers by others; the post does not name the paper's method (ModelSMC) or describe its results. This note treats that paper as the version of record for the method and its results, and the post for the framing. The post does not mention which language model the paper used; the paper does (Claude Sonnet 4.6).
The Hodgkin-Huxley story
The essay opens in 1951 (introduction). Alan Hodgkin and Andrew Huxley had written equations for how a squid nerve cell fires; with Cambridge's computer out of action, Huxley solved them on a hand-cranked calculator, which took over three weeks. The structure of the model followed from known facts, with the membrane treated as an electrical circuit, but one part was an educated guess: to fit their data they had to raise one gating variable to the fourth power, without knowing why. They suggested, cautiously, that several charged particles might have to move into place before a gate opens. Decades later, structural work found that a related potassium channel has four identical pieces around a central pore (the post links two papers in Science).
The lesson the authors draw is that "simply knowing the fourth power did the job was not enough; what mattered was knowing that a simpler version of the model did not work." The interpretation carried weight because alternatives had been tried and had failed (section "Lost in translation, or how to interpret models"). The post quotes Hodgkin and Huxley's own remark that higher powers might have fit slightly better but the "improvement was not considered to be worth the additional complication"; this note did not check the quotation against the 1952 paper.
Three steps of modeling
The post divides modeling into three steps, which scientists repeat (section "A three-step program"):
- Building: choosing what the model is made of and the form of its equations (for Hodgkin and Huxley, the circuit parts and the fourth power).
- Constraining: tuning the adjustable parameters to fit measurements.
- Interpreting: asking what the model says about the real system.
Machines, the authors write, have "arguably made the most progress in constraining models," and interpretation is "the hardest to hand over to AI."
Constraining: simulation-based inference
Many parameter settings can fit the same data, which the authors illustrate with a restaurant bill whose total is known but not who ordered what (section "Splitting the bill, or how to constrain models"). Bayesian inference returns every reasonably good parameter set as a probability distribution, but computing it requires the likelihood of the data under each setting, which is impractical for complex models. Simulation-based inference (SBI) avoids this: run the model (the simulator) many times with random parameters, "perhaps 100,000 times for a model like Hodgkin-Huxley," the post says, linking an eLife paper (not checked for this note), then train a neural network to map an output back to the range of parameters that could have produced it. Once trained, the network performs inference on new data in seconds. The post cites applications to neutron-star mergers, cosmology, structural biology, geophysics and nerve-cell models. Its point is that a distribution, unlike a single fit, shows which parameters the data pin down and where another experiment is needed.
Building: a distribution over models
The same problem recurs "one level up" (section "The bill, one level up, or how to build models"): many models with differently shaped equations can fit the same data. LLM-based agents now write candidate model code from data and a natural-language description, fit it, compare it with data and revise it; the post's footnote 1 cites six such systems from 2024 to 2026. An agent run twice may return two different, equally good models, and an agent that returns one hides the rest. The authors propose treating model building like parameter inference: the system proposes many candidates, weights each by its support in the data, with less credit to those that need much flexibility to fit, drops weak ones and varies strong ones. If all surviving models share a feature, the data likely favor it; where they disagree, the evidence is thin.
The post's figure for this section shows a schematic search through a space of models, with four clusters of models labeled by their ion channels and shares of 41% (sodium, potassium and leak), 39% (the same with an M current), 11% and 9% (caption "Building models by searching the space of all models"). The post does not say whether these shares come from a run. They are not in the ModelSMC paper, where the base model holds 0.7% of particles and M-current models most of the rest (see below), so this note reads the figure as an illustration.
The authors add a caution: "So far, these systems have mostly been tested on rediscovering models that scientists already know," and "Evidence that they work on problems with no known answer is sparse" (the post italicizes "no").
Interpreting
Interpretation is the hardest step to hand to AI, the authors argue, because there is no measure of whether a model improves understanding and no clear way to make "understanding" an AI system's objective (section "Lost in translation"). They describe two proposals from the literature: judging a system's understanding by how well it predicts the effects of disturbing the system, and asking instead whether the system's outputs help human scientists understand, as in a system that writes a short program generating a family of quantum experiments from which a physicist can read the design principle. In both, they argue, a distribution of models is needed: a single model cannot show which of its parts the data constrain, and agreement or disagreement among data-compatible models about an intervention indicates how far to trust the prediction.
Larger models
Famous models are compact partly because people had to build and reason about them (section "Cells and scales"). With AI doing more of the work, the authors expect models of systems too large to write down, such as whole brains. They point to the complete wiring diagram of the fruit fly brain (more than 100,000 neurons; the post's figure, rendered from the male CNS connectome, shows 124,635 cells) and to models built on it. At that scale, they argue, there is no single correct model, quoting George Box, "All models are wrong but some are useful." The question becomes which properties all data-compatible models share, and the most useful experiments are those that rule out the most remaining models. They compare this with climate-model ensembles and with exclusion maps in collider physics.
What the companion paper does and finds
The ModelSMC paper (Wahl et al., ICML 2026) casts model discovery as sampling from a posterior distribution over simulator programs given data. Sequential Monte Carlo (SMC) approximates a distribution with a population of weighted samples ("particles") that are repeatedly resampled, propagated and reweighted (Section 3.1). In ModelSMC each particle is a simulator program in code; an LLM proposes and revises programs, given the task description, ancestor programs and text feedback on their performance; and each program is weighted by an estimate of its marginal likelihood, obtained with neural SBI methods, so that its parameters are averaged out rather than tuned (Section 3.3; Appendices D.3 and D.4). Unless stated otherwise, all experiments use Claude Sonnet 4.6, with GPT-5 mini as an ablation (Appendix A). The acknowledgments thank Anthropic for API credits through the AI for Science Program.
Results, as the paper reports them:
- Validation without an LLM (Section 4.1, Figure 2). In a finite model space with random proposals, the target model comes to dominate the population across five target models and ten seeds, with its share oscillating up to about 80%. The paper concludes that once the LLM proposes the true model, the inference identifies it.
- Comparison with baselines (Section 4.2, Table 1; intervals in Table G-1). On three tasks (a synthetic SIR epidemic model, a kidney aldosterone model in R, and Hodgkin-Huxley), ModelSMC is compared with FunSearch+ (an evolutionary-search baseline) and with ModelSMC using a single particle. The metric is the negative log marginal likelihood of the best particle, median over ten runs, lower being better. The paper describes ModelSMC's performance as "comparable." In the table, the single-particle variant has the lowest median on all three tasks (for example, Hodgkin-Huxley 243.27 against 256.00 for ModelSMC and 254.58 for FunSearch+; kidney 36.54 against 42.85), with non-overlapping 90% intervals on the kidney task; this reading of the table is the note's. The paper's stated advantage is the posterior over models rather than a better single fit.
- Kidney model (Section 4.3). With the aldosterone mechanism replaced by a constant, and data from four intake scenarios at five time points each, high posterior weight across seeds goes to a narrow family of feedback mechanisms whose form closely matches the original model.
- Hodgkin-Huxley (Sections 4.4 and 4.5, Appendix F.4 and F.5). Starting from a standard model with sodium, potassium and leak currents and data from 10 recordings of mouse cortex neurons (Allen Cell Types Database), the search adds ion channels. Across runs, high posterior weight goes to models adding an M-type slow potassium current, often with a second channel. An LLM-assisted classification of all 1,445 usable particles puts 75.0% of the particles (a count, not a share of posterior weight) in the M-type family alone, 14.9% with persistent sodium added, 5.5% with an h-current, 2.4% with an A-type current and 0.7% as the base model (Appendix F.5); the M-type family alone ranks first in 9 of 10 seeds (Figure F-2). Section 4.5 calls the values plotted in Figure 5 "weights," but the figure's axis is the negative log marginal likelihood, so its "lowest weights" for M-type variants means the best fits; read that way, the text agrees with the ranking.
- Ablations (Section 4.6, Table 2; five seeds per variant, ten for the default). Varying the weighting, simulation budget, prompt, feedback, population size or LLM produced, in the paper's words, no significant degradation.
The paper's discussion says its "primary contribution is methodological," and that whether such systems produce novel mechanisms rather than rediscovering known ones "depends on the application" (Section 5). It lists as limitations the computational cost, possible bias from surrogate likelihoods under misspecification, dependence on the LLM's ability to propose relevant structures, and the lack of a geometry over program space.
The prompts and instructions used
The post discusses no prompts. In the paper, prompts are assembled by the DSPy library from a task description, ancestor programs and their LLM-written performance assessments (Appendix D.2, with a worked example for Hodgkin-Huxley in Appendix H). One ablation varies the prompt text: a reduced prompt gave a median of 262.18 (90% interval 249.96 to 282.15) against 256.00 (255.05 to 262.03) for the full prompt on the Hodgkin-Huxley task, with five seeds for the reduced prompt and ten for the default (Table 2; Appendix F.6), which the paper counts as no significant degradation. The full task prompt in Appendix H.1 asks the model to improve agreement "while maintaining model parsimony," offers two pre-allocated channel slots, describes the recordings as regular tonic spiking, and tells the model to avoid channels designed to produce bursting or sustained high-frequency firing. In this note's reading, the task text thus narrows which channel types the search considers; the paper does not test how much that shapes which families win, and its reduced-prompt ablation compares only the best fit.
Limitations and open questions
- An essay, not a study. The post's claims about what AI systems can do are the authors' arguments, supported by citations. Its one quantitative figure is not located in the paper it cites.
- Tested mostly on known answers. The authors say so in the post, and the paper's real-data tasks (kidney, Hodgkin-Huxley) start from established models; the kidney task asks the search to recover a mechanism that was removed.
- Modest headline comparison. In the paper's own table the simpler single-particle baseline has the better median fit on every task; the case for ModelSMC rests on the value of the posterior over models.
- LLM-assisted analysis. The taxonomy and classification of the 1,445 Hodgkin-Huxley particles were done by LLM calls (Appendix F.5); the paper reports no human check of the labels.
- Interest. The authors are grantees of Anthropic's AI for Science program, the paper's experiments used Anthropic credits and Claude, and the post appears on Anthropic's blog. Its acknowledgments thank Siddharth Mishra-Sharma, author of one of the footnoted papers, for editorial guidance.
- No published critique of the post or the paper was found or read for this note.
Research questions
- Does a distribution over models change what experimenters do? The post argues that disagreement among surviving models points to the next experiment. A study in which ModelSMC's posterior chooses experiments, compared with experts' choices, would test this.
- How do posterior shares over model families depend on the LLM? The ablation swapped Claude Sonnet 4.6 for GPT-5 mini and compared only the best fit. Comparing the family shares under each model would show whether the proposer shapes the posterior.
- Can such systems find mechanisms not already in the literature? A benchmark of held-out mechanisms published after the model's training cutoff would measure rediscovery against discovery, which the authors say is sparsely evidenced.
- How reliable is LLM classification of discovered models? Hand-labeling a sample of the 1,445 particles and comparing with the LLM's taxonomy would show whether the family shares are robust.
- When does the single-particle baseline match the full method? Tasks with more structural ambiguity than Hodgkin-Huxley would show whether the population-based search pays off in fit as well as in uncertainty.
- Does prompt wording matter more on harder tasks? The reduced prompt made no significant difference on one task with five seeds; repeating the ablation across tasks with more seeds would test it.
How it connects
- Long-running Claude for scientific computing is by Siddharth Mishra-Sharma, whose 2026 paper on program synthesis for simulation-based inference the post cites in footnote 2 and who is thanked for editorial guidance; that note's worked example is a different kind of scientific code built by Claude.
- Claude-shaped science, from the same blog, takes a related view of where models help (combining existing methods, with experts judging what matters); the connection is this note's, as neither post cites the other.
- Introducing our Science Blog lists the AI for Science program that funds the authors.
References
- Macke, J., Gedon, D., & Wahl, S. (2026, November 6). Teaching AI to build scientific models. Anthropic Science Blog (guest post). https://www.anthropic.com/research/teaching-ai-to-build-scientific-models record
- Wahl, S., Schenk, R., Farnoud, A., Macke, J. H., & Gedon, D. (2026). A Probabilistic Framework for LLM-Based Model Discovery. Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:124224-124278. https://proceedings.mlr.press/v306/wahl26a.html. Also arXiv:2602.18266 (v1 20 February 2026, v2 24 June 2026). The PMLR PDF was read in full except the prompt transcripts in Appendix H, which were skimmed. Code: https://github.com/mackelab/ModelSMC record
- Mishra-Sharma, S. (2026). Program Synthesis for Simulation-Based Inference: Joint Model Selection and Parameter Estimation. arXiv:2607.17540v1. Cited in the post's footnote 2; first page and abstract read. record
- Murphy, K. (2026). Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models. arXiv:2608.09696. Cited in the post's footnote 2; abstract read. record