DEV Community: Papers Mache

AI/ML Research Digest — Oct 31, 2026

Papers Mache — Mon, 02 Nov 2026 06:00:00 +0000

Accelerated inference & algorithmic efficiency

Gradient‑recovery speculative decoding isolates the KL self‑distillation signal, cuts perplexity and delivers up to six‑fold speedups without adding parameters [1]. A follow‑up analysis shows that sparse autoencoders let this decoder keep >99 % task accuracy while slashing FLOPs by two‑thirds [2], [1].

Any‑order autoregressive generation lets image models edit content flexibly and with only modest compute overhead compared with standard raster scans [3].

Fiber‑bundle solvers compute game‑theoretic equilibria far faster than traditional methods; the technique is not tied to language or diffusion models but illustrates how cross‑domain solvers can accelerate inference in other settings [4].

Agentic benchmarking & traceability

Reference‑free benchmarks such as Co‑Design, SATE and blockchain fund‑tracing produce auditable evidence of task completion, exposing systematic gaps in early stopping and overconfidence for autonomous agents [5] [6] [7].

Memory‑augmented agents that log their internal state further improve traceability, enabling post‑hoc verification of decisions.

Embodied multimodal language‑action models

Youtu‑Parsing‑Omni combines a unified encoder with on‑policy distillation to handle documents, vision and audio within a single model [8].

OpenFlyV2 equips an aerial platform with an onboard visual‑language agent that navigates without external computation [9].

HuGe scales humanoid visual‑language agents using egocentric experience, pushing embodied AI toward longer horizons [10].

AskStream introduces a proactive interaction benchmark for streaming video understanding, shifting focus from static encoders to real‑time decision making [11].

Standout contributions

Trinity recursive self‑improvement reorganizes tasks into a capability tree, generates verified simulated worlds and routes specialized harnesses, achieving autonomous skill acquisition in LLM agents [12].
Quantcheck scans low‑bit GGUF models for code‑generation failures, providing the first systematic safety check for 4‑bit and 8‑bit releases [13].
L3G‑World stores VAE features in compact latent 3D Gaussian memories; this enables real‑time rendering with state‑of‑the‑art visual quality while using orders of magnitude fewer resources [14].
Trope‑memory narrative generator uses a multi‑agent pipeline and evolutionary tree search to produce long, logically consistent stories that respect genre tropes [15].

Notable details

Tiny per‑option encoders as small as 17 M parameters outperform larger joint models on decision‑making benchmarks, offering higher accuracy with lower latency [16].

Amortized sequential Bayesian experimental design rewards LLMs for information gain; the approach lowers compute and improves performance on interactive tasks such as Twenty Questions and clinical dialogue [17].

A calibration‑free top‑k mixture‑of‑experts selector removes offline thresholds while preserving decoding accuracy and speed, simplifying deployment of MoE models [18].

The Span Coverage F1 metric evaluates hallucinations across provenance, typology and granularity, delivering a finer‑grained assessment of factual errors than token‑level scores alone [19].

References

Synthetic high‑value questions boost distillation efficiency

Papers Mache — Sun, 01 Nov 2026 06:00:00 +0000

Generating high‑value synthetic questions injects dense teacher supervision into on‑policy distillation, substantially reducing the number of required training questions. The twist is that these queries need no reference answers; their value comes from how much they expose the teacher’s knowledge when a strong model answers them.

Before this work, practitioners relied on real user questions or unoptimized student‑generated prompts, often filtering them with KL divergence to approximate relevance. Vanilla on‑policy distillation (OPD) and KL‑filtered real queries have been the default baselines for math reasoning, code generation, and agentic tasks.

A2D lifts average test scores to the mid‑40s and 60s, outpacing vanilla OPD by several points. “As shown in Table 1, A2D achieves average scores of 46.2, 51.6, and 64.5 for Qwen3‑1.7B, R1‑Distill‑1.5B, and R1‑Distill‑7B, respectively, compared with 43.0, 50.5, and 62.9 for Vanilla OPD.” [1] This gain translates into a 3.2‑point lift on the smallest model and a 1.6‑point lift on the largest, confirming that synthetic high‑value questions can replace real‑bank data without sacrificing performance.

The improvement stems from denser supervision per token rather than sheer quantity of examples. “Signal analyses further show that optimized questions can elicit denser supervision and more total signal from shorter student responses, supporting elicited distributional supervision as an objective for synthetic distillation data.” [1] Shorter answers mean the student consumes fewer tokens while receiving richer teacher feedback, a property that directly speeds learning.

The method assumes a frozen student–teacher pair to compute the surrogate value, so any drift between them during training could degrade the quality of generated questions. Moreover, the evaluation covers only three domains; it remains unclear whether the same gains hold for open‑ended dialogue or multimodal tasks. This suggests an open question: can distributional supervision be reliably estimated without a static teacher in settings where continual adaptation is required?

If synthetic high‑value queries truly provide denser supervision, benchmark pipelines should swap real query banks for the released A2D prompt collections whenever labeled data are scarce. Running standard OPD on the gasolsun/<pair> datasets will give a faster convergence curve and higher final scores without extra annotation effort.

References

Ask to Distill: Learning to Synthesize Data for On-Policy Distillation

On‑policy distillation works with limited additional data

Papers Mache — Sat, 31 Oct 2026 06:00:00 +0000

AutoDAgger recovers teacher trajectories online, letting compact policies inherit large‑model skill while requiring only modest extra data. By embedding the teacher’s corrective behavior directly into the student’s rollout loop, the method sidesteps the costly separate data‑collection stage that has dominated prior distillation pipelines [1].

Before this work, practitioners relied on pure behavior cloning from static human demonstrations or offline distillation that frozen the teacher’s expertise before any interaction with the environment. Human‑in‑the‑loop DAgger variants such as HG‑DAgger still needed a person to monitor progress and intervene whenever the learner deviated, inflating both labor and dataset size. These approaches made scaling down high‑capacity policies to edge devices an engineering bottleneck.

AutoDAgger collects recovery data more than 2 × faster than HG‑DAgger, eliminating the human monitoring loop entirely [1]. The autonomous reward model watches task progress and triggers teacher takeovers only when necessary, turning what used to be a manual handoff into a rapid self‑service pipeline.

With just 30 AutoDAgger episodes the student jumps from a baseline score of 27.8 to 63.9, and after 90 episodes it reaches 77.8, showing that a handful of on‑policy recovery rollouts suffice for dramatic improvement [1]. This efficiency suggests that a substantially smaller dataset can achieve performance levels comparable to those obtained with larger offline collections, as demonstrated by the modest additional data used in the experiments.

TILDE achieves scores of 80.6 on the Pick Pen task (and 63.9 and 69.2 on Fold Dishcloth and Pick Beverage), outperforming all evaluated baselines on each benchmark [1]. Those numbers sit squarely in the range of the original embodied foundation models, confirming that on‑policy distillation can transfer near‑teacher capability without exhaustive pretraining.

The study evaluates only three manipulation tasks and assumes a reliable reward model to signal failure, leaving open whether the same speedup and performance gains hold for more complex or less well‑instrumented domains. One unanswered question is how AutoDAgger scales when teacher interventions become sparse or when the reward signal is noisy; future work will need to probe robustness in those regimes.

If autonomous recovery truly replaces human‑in‑the‑loop data collection, benchmark pipelines that currently amass tens of thousands of trajectories should be re‑run with AutoDAgger, expecting comparable scores at a fraction of the data cost. This shift could make policy distillation practical for on‑device deployment across a wide array of robotics applications.

References

TILDE: Teacher-in-the-Loop Distillation from Embodied Foundation Models

Multimodal benchmark exposes 37% evidence gap

Papers Mache — Fri, 30 Oct 2026 06:00:00 +0000

OmniDR‑Bench forces multimodal agents to retrieve, locate, and justify audiovisual evidence, exposing a 37 % shortfall in grounded reasoning despite strong answer accuracy[1]. The benchmark’s three‑step rubric—retrieve the right media, pinpoint the supporting segment, then produce a justification—turns “I think so” into “here’s where it happens”.

Before this work, multimodal evaluation largely measured whether a model could name the correct object or answer a question about a video, using metrics such as VQA accuracy or recall‑based retrieval scores. Those benchmarks never required agents to point to a concrete timestamp or cite a supporting clip, leaving an invisible gap between answering and evidencing.

The best of 18 agent configurations reaches only 63.3 out of 100 on OmniDR‑Bench’s evidence metric[1]. Even the highest‑scoring systems excel at presentation—averaging 81.8 points—but fall to roughly mid‑40s on content (45.4) and citation quality (47.8), showing that polished reports mask weak grounding.

Presentation averages 81.8 points, while content and citation average only 45.4 and 47.8 points respectively[1]. This systematic disparity indicates that current pipelines prioritize surface‑level formatting over the substantive linkage of claims to video moments, a pattern evident in the overall benchmark results.

The benchmark does not yet measure long‑term consistency or cross‑modal causal reasoning beyond the annotated criteria, and the authors note agents still need broader content coverage, stronger causal and logical analysis, and more accurate citation support[1]. This suggests that improving raw retrieval or language modeling alone will not close the gap; dedicated grounding architectures and multimodal reasoning components are required.

Future model releases should treat the evidence score as a primary KPI, prompting developers to integrate segment‑level grounding modules before publishing answer‑only APIs. The clear 37 % shortfall sets an immediate target: raise content and citation performance into the 70s, or risk lagging behind real‑world research assistants that must back every claim with concrete audiovisual proof.

References

Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports

Memory‑native model shrinks token history size ~66

Papers Mache — Thu, 29 Oct 2026 06:00:00 +0000

MemT‑1.0 shatters the long‑standing belief that extending context length inevitably inflates memory consumption, delivering a two‑order‑of‑magnitude compression (~66×) with only negligible accuracy loss. The paper’s Mixture‑of‑Storage architecture packs token histories into compact slots, turning what used to be a linear KV cache into a sublinear memory footprint.

Before MemT‑1.0, transformer models relied on a full key‑value cache that grew linearly with each additional token, forcing practitioners either to truncate context or to augment the model with external retrieval systems that still maintained a complete cache for attention. Those workarounds preserved raw capacity but did not address the fundamental scaling bottleneck.

Mixture‑of‑Storage cuts persistent storage at a 128K‑token history by roughly 66× compared with an uncompressed KV cache, making the entire context fit into a fraction of the original memory budget. This reduction is achieved without altering the model’s core attention mechanism, simply by routing tokens through learned compressed slots [1].

The same compression yields about a 1.86× end‑to‑end inference speedup for MemT‑1.0‑Nano at the same 128K length, because fewer memory accesses translate directly into faster kernel execution while the model still attends over the full effective context [1].

The paper evaluates primarily on benchmark suites and agentic tasks and does not report fine‑grained token‑level recall on extremely long documents, leaving open whether subtle factual details might degrade under aggressive compression. This suggests a need for targeted probing on dense‑information retrieval scenarios to confirm that the speed gains do not hide hidden accuracy cliffs.

If the reported compression holds across downstream workloads, developers can now deploy multi‑turn assistants and long‑document processors on modest GPUs without resorting to external memory stores, effectively democratizing ultra‑long‑context capabilities.

References

MemT-1.0: Towards Memory-Native Evolution

Adaptive compute reduces latency by up to ~45%

Papers Mache — Wed, 28 Oct 2026 06:00:00 +0000

Adaptive compute primitives now shave 30–50% off inference latency across video diffusion, large‑scale language flow models, and long‑context transformers. While ALLOT operates without any additional training, the other primitives rely on models that have been pre‑trained (Lafa’s flow model and MemT‑1.0’s memory‑native architecture) and achieve speedups through inference‑time techniques.

Historically, dense attention has been the dominant bottleneck in generative video pipelines; “On a single GPU, Wan2.1-14B takes roughly one hour to generate a 720p, 81‑frame video, and attention accounts for more than half of the end-to-end time” [1]. Researchers therefore relied on static sparsity patterns or brute‑force hardware scaling, neither of which adapts to the varying sensitivity of layers and timesteps.

ALLOT raises the end‑to‑end speedup to 1.90× while delivering a 3.38 dB PSNR lift, “Notably, on HunyuanVideo (13B) with SVOO, ALLOT improves PSNR by 3.38 dB while increasing end-to-end speedup from 1.75 to 1.90” [1]. The method probes each layer‑step cell and reallocates block‑sparse density under a latency budget, preserving fidelity across all nine evaluated video pipelines.

The approach still hinges on sparsity staying below a critical density; “A sparse kernel is faster than dense only below a crossover density” [1]. Consequently, aggressive pruning can backfire on layers that are intrinsically sensitive, limiting the universal applicability of a single schedule.

Lafa’s blockwise flow design cuts inference time roughly in half while staying on par with autoregressive baselines; “With block size four, it is about twice as fast while also more accurate on average” [2]. By caching prefixes and verifying flow‑generated drafts inside the same model, it attains 2.54× AR throughput without sacrificing instruction‑following or coding performance.

Speedup depends heavily on the chosen block granularity; “With block size sixteen, Lafa 7B achieves the throughput of 7B AR SFT at comparable average accuracy” [2]. Larger blocks dilute parallelism gains, so practitioners must tune block size to their hardware and latency budget.

MemT‑1.0‑Nano delivers roughly 1.86× end‑to‑end inference speedup and a ≈66× reduction in persistent storage for 128K‑token histories, “At a history length of 128K tokens, MemT-1.0-Nano achieves approximately 1.86× end-to-end inference speedup and a 66× reduction in persistent storage relative to full context KV cache” [3]. Its gated DeltaNet updates keep memory native, avoiding the quadratic blow‑up of traditional KV caches.

The advantage appears only for sufficiently long contexts; “MemT-1.0-Nano achieves an end-to-end advantage over full context at long histories” [3]. Shorter sequences still fall back to dense caching, so the technique is not a universal drop‑in for all workloads.

If these three orthogonal primitives hold up in production, developers can retrofit existing diffusion, flow‑based LLMs, and long‑context transformers with block‑sparse schedules, speculative draft verification, or memory‑native updates and expect roughly a 45% latency reduction without re‑training. The cost savings alone make the engineering effort worthwhile.

References

Iterating layers at inference lifts reasoning scores

Papers Mache — Tue, 27 Oct 2026 06:00:00 +0000

Looped Transformers achieve logical‑reasoning gains without any extra generated tokens. By reusing the same Transformer block multiple times at inference, they deepen the computation on‑the‑fly; although the model size remains unchanged, deeper loops naturally increase inference time compared to a single‑pass forward pass.

Before this work, most reasoning systems either relied on chain‑of‑thought prompting—adding token‑by‑token steps that linearly increase runtime—or fixed‑depth decision heads that could not allocate more compute to hard questions. Neither approach let a model dynamically deepen its thinking without paying the cost of additional output tokens.

Repeating the same layers at inference lifts JevBench accuracy from 63.2 % at depth 1 to 76.2 % at depth 4 for a 2.6 B checkpoint, a full 13‑point jump on a standard logical reasoning benchmark [1].

Most of that boost comes from the hard tier: LEV gains ten correct answers out of 72 standard items and nineteen out of 111 hard items when moving from depth 1 to depth 4, while easy questions are already solved at the shallowest level [1].

The papers also warn that deeper loops do not automatically solve all inference shortcomings. Early reasoning steps may produce richer pragmatic readings that later collapse to literal answers; “Answering from that early reading, an exploratory inference‑time remedy, recovers these implicatures without losing literal accuracy” [2]. This suggests a missed opportunity: decision models could expose intermediate loop states as optional outputs, but the current implementations treat them as internal bookkeeping only.

If looping layers can be toggled at runtime, developers can upgrade the reasoning performance of existing checkpoints with no extra memory or model‑size overhead. Benchmarks that measure logical and scalar inference should therefore be re‑run with a configurable loop depth, and production services ought to expose a “depth” knob rather than committing to a single forward pass.

References

AI/ML Research Digest — Oct 24, 2026

Papers Mache — Mon, 26 Oct 2026 06:00:00 +0000

Adaptive computation for efficiency

Block‑sparse attention scheduling cuts inference latency by up to 60 % without hurting quality; the ALLOT scheduler achieves this by allocating sparse blocks across layer‑step pairs [1]. MambaAct modules apply a similar adaptive principle to vision‑language‑action pipelines, letting agents skip unnecessary computation steps [2]. In diffusion language modeling, SpecKCast generates speculative drafts that let the model bypass many denoising iterations, delivering up to 2.5× higher throughput while keeping output fidelity [3]. Four‑bit state quantization compresses attention memory and linear‑attention kernels, halving the memory bandwidth required for long contexts [4]. The Mixture‑of‑Storage architecture further reduces token storage by two orders of magnitude, enabling fast inference for LLM agents that must remember thousands of tokens [5].

On‑policy and distribution‑matching distillation

AutoDAgger recovers teacher trajectories on the fly and aggregates them autonomously, letting embodied agents inherit large‑model behavior with virtually no extra data collection [6]. Self‑teaching via the tilde operator injects privileged traces from a stronger model during training, sharpening policy predictions without expanding the dataset [7]. High‑value question synthesis creates dense supervision signals by generating challenging queries; this dramatically improves distillation efficiency for multi‑turn generation [8]. The FRED framework matches teacher and student intermediate embeddings across dialogue turns, aligning distributions rather than individual outputs and reducing policy drift in conversational agents [9].

Unified multimodal models with evidence grounding

A token‑level prediction backbone now spans object detection, pixel synthesis, and language generation, providing a single route from perception to output [10]. Benchmarks such as OmniDR‑Bench require retrieval, spatio‑temporal localization, and textual justification across video, audio, and documents; current omnidirectional LLMs score only 63.3/100 despite decent answer accuracy, exposing a clear evidence gap [11]. EvaOmni extends this evaluation to embodied settings where visual grounding must be paired with action cues [12]. MMEnterprise pushes further by demanding cross‑modal reasoning over enterprise documents and multimodal tables, highlighting the need for integrated retrieval and generation pipelines [13].

Recursive self‑improvement via autonomous evaluation

EvalResearchBench lets agents generate their own test suites, removing the bottleneck of manually curated benchmarks [14]. HarnessPro builds label‑free harnesses that expose failure modes without human annotation, feeding these signals back into model training loops [15]. RSIFlow orchestrates meta‑agents that select improvement workflows based on observed gains, closing the loop between performance and evaluation cost [16]. Autonomous code discovery pipelines use self‑generated test cases to evolve programming assistants, demonstrating a full cycle of creation, testing, and refinement without external supervision [17].

Memory and long‑context management for LLM agents

Four‑bit linear‑attention quantization compresses key–value caches while preserving attention fidelity, allowing contexts up to sixteen times longer on the same hardware budget [4]. MemT‑1.0’s Mixture‑of‑Storage design stores token histories in a hierarchical fashion, cutting memory use by two orders of magnitude and speeding up inference for agents that must reason over extensive dialogues or documents [5].

Standout papers

AutoDAgger enables on‑policy distillation without extra data – The method recovers teacher trajectories online, aggregates them autonomously, and transfers large‑model capabilities to embodied agents with minimal data overhead [6].

Large Flow Language Models (Lafa) achieve significant speedups in diffusion language generation – By combining flow‑based parallel decoding with self‑speculative drafting, Lafa reaches up to 2.54× higher throughput than standard autoregressive decoders while keeping quality intact [3].

Neurosymbolic policy compilation yields high accuracy and stable predictions – Compiling policies into executable neural predicates and training against adversarial counterexamples produces state‑of‑the‑art performance on low‑capacity models and markedly improves prediction stability [18].

OmniDR‑Bench exposes multimodal evidence gaps – The benchmark forces models to retrieve, localize, and justify audiovisual evidence; current omnidirectional LLMs achieve only 63.3/100 despite reasonable answer scores, highlighting a critical weakness in evidence grounding [11].

MemT‑1.0 shows order‑of‑magnitude token storage reduction – Its Mixture‑of‑Storage architecture compresses token histories dramatically and accelerates inference for long‑context LLM agents, making large‑scale reasoning feasible on modest hardware [5].

Additional details worth noting

High‑value question synthesis supplies dense supervision without reference answers, boosting the efficiency of on‑policy distillation pipelines [8].

VLA‑Rolling reuses discarded denoising tails to accelerate vision‑language‑action inference by 1.48× and raise success rates across several backbones [19].

Looped Transformers iterate over the same layers at inference time, improving logical and scalar reasoning accuracy without generating extra tokens; gains appear on hard question sets [20][21].

These advances collectively shrink latency, lower memory demands, and tighten the feedback loop between models and their evaluations—trends that directly affect deployability, research productivity, and the reliability of AI systems across domains.

References

Box regression head boosts visual grounding speed, accuracy

Papers Mache — Sun, 25 Oct 2026 06:00:00 +0000

Current visual‑grounding pipelines waste cycles by decoding box coordinates token‑by‑token, turning a simple localization problem into an autoregressive sequence generation task. The community has accepted this overhead as the price of expressive multimodal reasoning. Replacing the classification head with a direct regression module slashes latency and boosts precision at once.

Before Jag, most systems combined a pretrained language encoder with a visual merger and then emitted box corners through a sequential decoder such as an LLM or a transformer‑based pointer network. Works like ReferItGAN and GLIP demonstrated strong accuracy but required multiple forward passes to parse the generated coordinate string, inflating end‑to‑end runtime. Benchmarks such as RefCOCO measured success with Acc@0.5 while ignoring the hidden cost of text‑based box synthesis.

Jag achieves the lowest measured mean end‑to‑end latency in single‑request inference, 65.97 ms (Figure 2) [1]. The regression head reads the final token state of the fused image–language stream and emits all four corners simultaneously, eliminating the need for a generation loop. Because the visual encoder and language backbone are reused unchanged, the whole pipeline runs in one forward pass on a single GPU.

Jag improves grounding accuracy over Base on every test split, with gains ranging from 9.63 to 17.51 accuracy points (Table 2) [1]. The joint supervision forces the multimodal representations to align with geometric targets, so even small referring expressions benefit from tighter box predictions. Across RefCOCO, RefCOCO+, and RefCOCOg the model consistently outperforms its autoregressive predecessor despite a leaner architecture.

Jointly updating the language backbone, visual merger, and regression head improves Acc@0.5 by more than 60 points on each dataset relative to head‑only training [1]. This requirement means that Jag’s speedup presumes access to the full pretrained checkpoint and additional fine‑tuning compute; freezing any component collapses the accuracy advantage. Moreover, the reported gains stem from a compact encoder, leaving open whether the same head redesign scales to massive multimodal LLMs without prohibitive memory overhead.

Benchmarks that currently rely on autoregressive grounding models should replace their output heads with Jag‑style direct regressors, expecting substantially lower latency and a double‑digit accuracy lift on standard GUI datasets.

References

Jag: Direct Box Prediction for Efficient Visual Grounding

Stateless agents cut long‑horizon token usage

Papers Mache — Sat, 24 Oct 2026 05:00:00 +0000

Current long‑horizon LLM agents burn tokens at a rate that makes billion‑step research infeasible. Their prompts grow as every intermediate result is replayed, leading to duplicated effort and runaway costs. Decoupling durable state from the model’s active context, as proposed by Stateless Language Agents (SLAs), flips this trend on its head.

Historically, agent designs have embedded the entire research history inside each model invocation, so every call re‑feeds an ever‑larger transcript. Benchmarks were consequently limited to short budgets where token blow‑up never surfaced. This coupling forced researchers to accept a trade‑off between horizon length and compute expense.

Stateless Language Agents reach the strongest kernel baseline’s final performance while using over 84 % fewer tokens. By storing candidate solutions and outcomes in an external harness, each agent receives only the concise evidence it needs for its next step. The paper reports that “SLA achieves the best final result on every task and reaches the strongest kernel baseline’s final performance with over 84% fewer tokens” [1]. This reduction holds across software engineering, kernel optimization, and algorithm design experiments at budgets up to one billion tokens.

The coordination component – the Advisor – consumes less than 0.6 % of tokens, keeping overhead negligible. The harness builds a role‑specific context for each stateless worker, while the Advisor merely reads summarized evidence and issues concrete experiment assignments. As quoted, “while the Advisor consumes less than 0.6% of tokens” [1], meaning almost all budget is spent on substantive reasoning rather than bookkeeping.

Stateless agents still depend on a harness that must summarize evidence accurately, and the paper’s evaluation is limited to three engineered domains. If the summarizer drops critical nuance, workers may pursue suboptimal directions despite the token savings. Moreover, tasks lacking clear candidate‑outcome structures could resist this externalization, leaving open whether the same gains translate to more unstructured research settings.

Future research pipelines should externalize durable state and re‑evaluate long‑horizon benchmarks with token budgets in the billions. By treating the harness as the sole keeper of history, practitioners can scale experiments on fixed‑size models without incurring prohibitive prompt costs. This shift will likely render many current “short‑budget” results obsolete and set a new baseline for cost‑effective automated research.

References

Stateless Language Agents: Scaling Long-Horizon Automated Research

Plug‑and‑play harness evolution boosts benchmarks without weight changes

Papers Mache — Fri, 23 Oct 2026 05:00:00 +0000

Iteratively refining execution pipelines delivers measurable gains even when model parameters stay frozen. Recent work shows that a few plug‑and‑play edits to an agent’s harness can lift benchmark scores by double digits, challenging the assumption that performance upgrades must come from costly weight updates.

Until now most performance jumps were achieved through fine‑tuning, prompt‑engineering sweeps, or adding external tools that required re‑training or heavy hyper‑parameter searches. The prevailing view treated the LLM’s internal weights as the primary lever for improvement, while the surrounding execution harness was regarded as a static wrapper.

A plug‑and‑play harness revision lifts GPT‑5.4‑nano’s SpreadsheetBench accuracy to 77.58 %, exceeding GPT‑5.5 with SkillOpt’s 76.20 % and improving all four tested models by 5.69–11.39 percentage points [1]. The authors describe this as “Harness self‑evolution revises an agent’s software scaffolding, including prompts, memory, workflows, and executable components, while keeping model weights fixed,” and each retained revision feeds back into the next update cycle.

A single recursive harness edit adds up to 11.5 points on WorkBuddy‑Bench for GLM‑5, with similar gains of 4.5 points on JobBench and an 11.8 % lift in pass rate on LAB, all without touching model weights [2]. The paper frames the process as “Let be a model with frozen parameters… A harness consists of editable components… It composes with to form an agent without changing its weights,” demonstrating that orchestration alone can rival traditional fine‑tuning.

The results leave open how well these methods scale beyond curated professional suites and whether the high‑probability admission rule based on fresh, independent validation data will hold in truly open‑world deployments [1]. Moreover, the authors note that Harness² uses a contrastive Propose‑Probe‑Compose step designed to guide edits without ground‑truth verifiers, though its performance in scenarios lacking any reliable verification signal remains an open question.

If these findings generalize, benchmark pipelines should incorporate systematic harness optimisation as a first‑order hyperparameter, rerunning evaluations after each approved revision. Treating the execution scaffold as an editable artifact unlocks low‑cost performance boosts and reduces reliance on expensive retraining cycles.

References

Video diffusion compute drops from thousands to one hour

Papers Mache — Thu, 22 Oct 2026 05:00:00 +0000

Video diffusion has long been tethered to massive compute budgets, with PF‑ODE trajectory collection alone consuming “tens of thousands of GPU‑hours” [1]. Glance Forcing shatters that paradigm by partitioning the denoising process into slow and fast experts, trimming the entire trajectory‑collection stage down to roughly a single A100‑GPU hour.

Prior pipelines distilled bidirectional diffusion models into causal students but relied on exhaustive offline PF‑ODE sampling, a step that dominated the overall training cost and required substantial compute resources [1]. The high‑noise regime governs global layout and motion, while the low‑noise regime refines texture; collapsing both into one monolithic student caused severe interference and quality collapse.

Using only one PF‑ODE trajectory, Glance Forcing already surpasses the fully‑trained state‑of‑the‑art baseline, raising the VBench Total score from 84.04 to 84.22 while cutting trajectory‑collection cost from thousands of GPU‑hours to about one A100‑GPU‑hour [1]. The slow‑LoRA and fast‑LoRA checkpoints are trained with stage‑specific objectives, preserving the frozen teacher’s knowledge yet allowing each expert to specialize in its curvature regime.

The method still hinges on a pretrained bidirectional teacher that must remain frozen, and the decoupling introduces two LoRA modules that need to be stored and swapped during inference [1]. This suggests that while compute for trajectory generation is negligible, the overall training pipeline retains a modest overhead for expert‑specific fine‑tuning, an aspect future work could streamline further.

If high‑quality video generators can now be trained on a single workstation, the community should retire benchmarks that implicitly assume “GPU‑weeks” as a baseline requirement and re‑evaluate past results under this new compute budget.

References

Glance Forcing: Making Bidirectional Diffusion Autoregressive at a Glance