AI Paper Digest · October 21, 2026
AI Paper Digest · October 21, 2026
This digest covers three papers from the October 19 arXiv wave: OpenVLHarness, a real-world visual agentic tool-use harness with persistent multimodal memory; WeAct, which turns capability-level weight switching into agent actions; and TokenGate, learned token pruning for fast, memory-efficient prefill.
1. OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
- Title: OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
- Authors: Yu Zhou, Cheng-Fu Yang, Rui Sun, Di Wu, Sibo Peng, Mingyang Zhang, Zhen Yang, Ruiyi Zhang, Nanyun Peng, Zhe Gan, Kai-Wei Chang
- Problem: Multimodal LLMs have made rapid progress on visual reasoning, yet many tasks require evidence that internal reasoning alone cannot supply. The authors’ controlled study across 22 benchmarks shows that simply scaling up the base model’s inference budget yields only modest gains. These tasks demand grounded perception, external knowledge, and numerical measurements obtained through heterogeneous tools — outputs that must be rendered interpretable to the model and retained across reasoning steps.
- Method: OpenVLHarness, a multimodal reasoning harness organized around a capability layer plus persistent multimodal memory. The capability layer combines task-level tool interfaces with evidence rendering, giving consistent access to heterogeneous backends and converting their outputs into model-readable observations. Persistent multimodal memory retains the corresponding images, structured data, and web references as addressable artifacts for the whole query. A coding interface executes generated programs over stored data, enabling computations that combine results from multiple tools. Together these support an iterative acquire-inspect-compose loop over multimodal evidence.
- Results: Across 22 benchmarks and 8 open/proprietary backbones, OpenVLHarness improves average scores by 9.7–12.6 points over matched no-tool baselines (+11.2 with Qwen3-VL-8B, +9.7 with Kimi K3, +9.9 with GPT-6 Astra). Efficiency result worth highlighting: GPT-6 Luna at medium reasoning effort with OpenVLHarness outperforms the same model — as well as Codex and Visual Sketchpad — at maximum effort by 7.7–8.5 points, while incurring at most 37% of their model-token cost.
- arXiv: https://arxiv.org/abs/2610.26860
2. WeAct: Dynamic Weight Switching as Agent Actions
- Title: WeAct: Dynamic Weight Switching as Agent Actions
- Authors: Rui Shi, Xunjian Yin, Jiacheng Sang, Zichuan Liu, Shuyan Zhou
- Problem: Long-horizon agents must learn heterogeneous capabilities from interleaved trajectories, but updating shared parameters to learn new capabilities degrades previously acquired ones — catastrophic forgetting. Different stages of a trajectory may need different capabilities, yet conventional joint training and static adapter merging hold one fixed parameter configuration for the entire execution.
- Method: WeAct (Weight-level Internal Actions) trains independent capability-specific LoRA adapters over a frozen backbone and adds adapter selection to the agent’s action space. These internal actions reuse the model’s native tool-calling interface for on-demand capability composition in a shared context, so the model dynamically switches its “capability configuration” within a single trajectory instead of committing to one set of parameters.
- Results: Across eight deep-research benchmarks, Qwen3-32B with WeAct improves average accuracy by 6.2% relative to full fine-tuning counterparts despite using less compute, and by 27.5% relative to the strongest static LoRA-merging method. WeAct also supports modular continual adaptation: adding a new code-computation adapter expands the agent’s capabilities and updating the existing writing adapter improves report quality — both leave the backbone and other adapters untouched, offering a non-forgetting path to ongoing agent learning.
- arXiv: https://arxiv.org/abs/2610.26847
3. TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
- Title: TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
- Authors: Liana Patel, Rishabh Ranjan, Ion Stoica, Matei Zaharia, Carlos Guestrin
- Problem: Expert human readers process large inputs (code repositories, long documents) by skimming and reading only what matters. Language models, by contrast, push every input token through every layer — making prefill over long contexts computationally and memory-expensive. Can a model learn to skim first and read closely only where it counts?
- Method: TokenGate, a small drop-in component that extends existing trained models with end-to-end learnable token pruning over long-context inputs for fast, memory-efficient prefill. To make pruning learnable despite the non-differentiable top-K operator, the authors introduce token-gated attention — a mechanism inspired by Mixture-of-Experts that re-weights attention logits with token importance scores, enabling true end-to-end training without a two-stage pipeline.
- Results: Across diverse tasks, TokenGate achieves 1.4–11.3× faster time-to-first-token and 1.5–3.4× lower peak memory for prefill with no accuracy degradation. Against the best prior token-pruning methods it delivers up to 6.4× faster prefill and 2× lower peak memory. It natively extends both standard transformers (e.g., Qwen3) and hybrid-attention models (e.g., Qwen3.5) with lightweight adaptation comparable to standard parameter-efficient fine-tuning. Code is open-sourced (github.com/tokengate-models/TokenGate).
- arXiv: https://arxiv.org/abs/2610.26754
All articles on this blog are licensed under CC BY-NC-SA 4.0 unless otherwise stated.









