When AI fails, and why

1. Hallucination and verification

   Why do models get facts wrong when the evidence is in front of them, and how can we catch it?

   Our ICML 2026 paper treats hallucination on evidence-grounded yes-or-no questions as a measurable shortfall of information. When the evidence doesn’t carry enough to support an answer, the model fills the gap with something plausible. That gives a rule for when to answer and when to abstain.

   In a pre-specified audit of 528 held-out questions, the rule kept hallucinations to 0.0–0.7% while abstaining on 20.6–27.9% of them (95% confidence intervals).

   Where it stops working. The gate trades coverage for reliability. It abstained on a fifth to a quarter of questions, and in later tests it also dropped 10–20% of good claims.

   Berry, our open-source verifier, puts the same idea to work. It checks each claim in an answer against the evidence cited for it and keeps a tamper-evident record of every decision, inside Claude Code, Cursor, Codex or Gemini CLI.

   Read the paperBerry on GitHubMore on this area

2. In-context learning

   What is a model actually computing when it learns from examples in a prompt?

   Across 92,160 positional edits on held-out prompts, our exact formula for attention predicted the direction of the change 95.4–96.5% of the time (Huang et al., 2026).

3. Changing models without retraining

   Can a running model be adapted without retraining it?

   On Qwen2.5-7B, a 50,000-parameter controller came within a point of LoRA on GSM8K (71% against 72%), and lost 4.7 points on held-out code where LoRA lost 16.3 (ntkmirror).

4. Faster, cheaper AI

   How much of a large model’s memory can be dropped within a set error budget?

   Our add-on makes Triton, a widely used tool for writing fast GPU programs, work on NVIDIA’s GB10 (Blackwell) desktop hardware until official support arrives (triton-blackwell).

5. World models and science

   When do models trained on data learn the world’s symmetries, and when do they break them?

   Write the same robot trajectory as absolute joint targets instead of changes, and a world model’s retrieval degrades 2.6 to 13.4 times across three robot datasets (Karim and Chlon, 2026).

All research areas and open problems

1. Robot World Models Are Not Invariant to How the Actions Are Written

   Ahmed Karim, Leon Chlon. arXiv preprint, September 2026. arXiv:2609.23252

2. Exact Finite Attention Responses From RoPE Derivatives

   Julie Huang, Maggie Chlon, Gregory Gutin, Leon Chlon. arXiv preprint, September 2026. arXiv:2609.14127

3. Predictable Compression Failures: Order Sensitivity and Information Budgeting for Evidence-Grounded Binary Adjudication

   Leon Chlon, Ahmed Karim, Maggie Chlon, MarcAntonio Awada. ICML 2026. arXiv:2509.11208

4. LLMs are Bayesian in Expectation, Not Realization

   Leon Chlon, Zein Khamis, Fatima Sheaib, Maggie Chlon, Mahdi El Zein, MarcAntonio M. Awada. arXiv preprint, 2025, revised 2026. arXiv:2507.11768

All papers. Our code is on the open-source page: Berry, Mezzanine, ntkmirror, triton-blackwell.

• Why a CNN beats a vision transformer on small data

  Every symmetry you build into an architecture is data you don’t have to collect. 1 October 2026

• What masked reconstruction teaches a model about shape

  Hide part of a shape, ask a model to rebuild it, and symmetry becomes a shortcut worth learning. 28 September 2026

• Noether’s theorem on a spring

  Emmy Noether proved that every symmetry comes with a quantity that never changes, and a mass on a spring makes the idea easy to see. 26 September 2026

• The 6-or-9 problem

  Why world models trained to compress what they see can learn the wrong thing about the world. 22 September 2026

All 7 explainers