arXiv 论文速递

Post-training with Reinforcement Learning (RL) has substantially improved reasoning in Large Language Models (LLMs) via test-time scaling. However, extending this paradigm to Multimodal LLMs (MLLMs) through verbose rationales yields limited gains for perception and can even degrade performance. We propose Reinforced Attention Learning (RAL), a policy-gradient framework that directly optimizes internal attention distributions rather than output token sequences. By shifting optimization from what to generate to where to attend, RAL promotes effective information allocation and improved grounding in complex multimodal inputs. Experiments across diverse image and video benchmarks show consistent gains over GRPO and other baselines. We further introduce On-Policy Attention Distillation, demonstrating that transferring latent attention behaviors yields stronger cross-modal alignment than standard knowledge distillation. Our results position attention policies as a principled and general alternative for multimodal post-training.

标题：强化注意学习

通过测试时缩放，后训练的强化学习（RL）在大型语言模型（LLMs）的推理方面取得了显著改进。然而，将这一范式扩展到多模态LLMs（MLLMs）并通过冗长的推理理由，仅在感知方面获得有限的收益，甚至可能降低性能。 我们提出了一种强化注意学习（RAL），这是一种策略梯度框架，直接优化内部注意分布，而不是输出的标记序列。通过将优化从生成什么转移到注意哪里，RAL促进了有效信息分配并提高了对复杂多模态输入的语义关联。跨多种图像和视频基准的实验显示，RAL在GRPO和其他基线方法上均表现出一致的改进。我们进一步引入了策略梯度注意蒸馏，表明转移潜在的注意行为比标准的知识蒸馏能获得更强的跨模态对齐。我们的结果将注意策略定位为多模态后训练的一种原理上和通用的替代方案。

We present protein autoregressive modeling (PAR), the first multi-scale autoregressive framework for protein backbone generation via coarse-to-fine next-scale prediction. Using the hierarchical nature of proteins, PAR generates structures that mimic sculpting a statue, forming a coarse topology and refining structural details over scales. To achieve this, PAR consists of three key components: (i) multi-scale downsampling operations that represent protein structures across multiple scales during training; (ii) an autoregressive transformer that encodes multi-scale information and produces conditional embeddings to guide structure generation; (iii) a flow-based backbone decoder that generates backbone atoms conditioned on these embeddings. Moreover, autoregressive models suffer from exposure bias, caused by the training and the generation procedure mismatch, and substantially degrades structure generation quality. We effectively alleviate this issue by adopting noisy context learning and scheduled sampling, enabling robust backbone generation. Notably, PAR exhibits strong zero-shot generalization, supporting flexible human-prompted conditional generation and motif scaffolding without requiring fine-tuning. On the unconditional generation benchmark, PAR effectively learns protein distributions and produces backbones of high design quality, and exhibits favorable scaling behavior. Together, these properties establish PAR as a promising framework for protein structure generation.

标题：基于多尺度结构生成的蛋白质自回归建模

我们提出了蛋白质自回归建模（PAR），这是第一个通过粗到细逐级预测的多尺度自回归框架，用于蛋白质主链生成。利用蛋白质的分层特性，PAR 生成结构，模仿雕塑雕像的过程，先形成粗略的拓扑结构，再逐步细化结构细节。为此，PAR 包含三个关键组件：（i）多尺度下采样操作，在训练过程中表示不同尺度的蛋白质结构；（ii）自回归变压器，编码多尺度信息并生成条件嵌入以指导结构生成；（iii）基于流的主链解码器，根据这些嵌入生成主链原子。此外，自回归模型由于训练和生成过程的不匹配而遭受曝光偏差，这显著降低了结构生成的质量。我们通过采用噪声上下文学习和计划采样有效缓解了这一问题，使主链生成更加稳健。值得注意的是，PAR 具有强大的零样本泛化能力，支持灵活的人类提示条件生成和模式支架，无需微调。在无条件生成基准测试中，PAR 有效地学习了蛋白质分布并生成了高质量的主链，表现出良好的扩展行为。这些特性使 PAR 成为蛋白质结构生成的一个有前途的框架。

The research introduces protein autoregressive modeling (PAR), a multi-scale autoregressive framework for protein backbone generation. PAR uses coarse-to-fine next-scale prediction and consists of multi-scale downsampling, an autoregressive transformer, and a flow-based backbone decoder. It addresses exposure bias through noisy context learning and scheduled sampling, enhancing structure generation quality. PAR demonstrates strong zero-shot generalization and favorable scaling behavior, producing high-quality protein backbones without fine-tuning, making it a promising framework for protein structure generation.

研究引入了蛋白质自回归建模（PAR），这是一种多尺度自回归框架，用于蛋白质主链生成。PAR采用自上而下的逐尺度预测和蛋白质结构的层次表示。它包括多尺度下采样、自回归变压器和基于流的主链解码器。为解决暴露偏差问题，PAR采用了噪声上下文学习和计划采样。PAR展示了强大的零样本泛化能力和高质量的主链生成，支持灵活的条件生成和基序构建，无需微调。在基准测试中，PAR有效地学习了蛋白质分布并生成了高质量的主链，显示出有利的扩展行为。

Internet of Things (IoT) deployments operate in nonstationary, dynamic environments where factors such as sensor drift, evolving user behavior, and heterogeneous user privacy requirements can affect application utility. Continual learning (CL) addresses this by adapting models over time without catastrophic forgetting. Meanwhile, contrastive learning has emerged as a powerful representation-learning paradigm that improves robustness and sample efficiency in a self-supervised manner. This paper reviews the usage of \emph{contrastive continual learning} (CCL) for IoT, connecting algorithmic design (replay, regularization, distillation, prompts) with IoT system realities (TinyML constraints, intermittent connectivity, privacy). We present a unifying problem formulation, derive common objectives that blend contrastive and distillation losses, propose an IoT-oriented reference architecture for on-device, edge, and cloud-based CCL, and provide guidance on evaluation protocols and metrics. Finally, we highlight open unique challenges with respect to the IoT domain, such as spanning tabular and streaming IoT data, concept drift, federated settings, and energy-aware training.

标题：对比连续学习在物联网中的模型适应性

物联网(IoT)部署在非平稳、动态环境中运行，其中传感器漂移、用户行为演变和异质用户隐私要求等因素会影响应用效用。连续学习(CL)通过在不发生灾难性遗忘的情况下随着时间调整模型来解决这一问题。同时，对比学习作为一种强大的自监督表示学习范式，提高了鲁棒性和样本效率。本文回顾了对比连续学习(CCL)在物联网中的应用，将算法设计（重放、正则化、蒸馏、提示）与物联网系统现实（TinyML约束、间歇性连接、隐私）联系起来。我们提出了一种统一的问题表述，推导出结合对比和蒸馏损失的共同目标，提出了面向设备、边缘和云的CCL参考架构，并提供了评估协议和指标的指导。最后，我们强调了与物联网领域相关的开放独特挑战，如跨越表格式和流式IoT数据、概念漂移、联邦设置和能量感知训练。

This paper explores contrastive continual learning (CCL) for adapting models in IoT environments, which are characterized by nonstationary and dynamic conditions. The authors integrate contrastive learning with continual learning to enhance robustness and sample efficiency. Key findings include a unifying problem formulation, a reference architecture for on-device, edge, and cloud-based CCL, and guidance on evaluation protocols. The study addresses unique challenges in the IoT domain, such as concept drift and federated settings.

本文探讨了在非稳定动态的物联网环境中应用对比连续学习（CCL）以适应模型的方法。作者将对比学习与连续学习相结合，以提高鲁棒性和样本效率。主要发现包括统一的问题表述、针对设备端、边缘和云端的CCL参考架构，以及评估协议的指导。研究还针对物联网领域的独特挑战，如概念漂移和联邦设置等进行了讨论。

Reinforcement learning (RL) has become a cornerstone for fine-tuning Large Language Models (LLMs), with Proximal Policy Optimization (PPO) serving as the de facto standard algorithm. Despite its ubiquity, we argue that the core ratio clipping mechanism in PPO is structurally ill-suited for the large vocabularies inherent to LLMs. PPO constrains policy updates based on the probability ratio of sampled tokens, which serves as a noisy single-sample Monte Carlo estimate of the true policy divergence. This creates a sub-optimal learning dynamic: updates to low-probability tokens are aggressively over-penalized, while potentially catastrophic shifts in high-probability tokens are under-constrained, leading to training inefficiency and instability. To address this, we propose Divergence Proximal Policy Optimization (DPPO), which substitutes heuristic clipping with a more principled constraint based on a direct estimate of policy divergence (e.g., Total Variation or KL). To avoid huge memory footprint, we introduce the efficient Binary and Top-K approximations to capture the essential divergence with negligible overhead. Extensive empirical evaluations demonstrate that DPPO achieves superior training stability and efficiency compared to existing methods, offering a more robust foundation for RL-based LLM fine-tuning.

标题：重新思考LLM强化学习中的信任区域

强化学习（RL）已成为精细调整大型语言模型（LLMs）的基石，其中近端策略优化（PPO）已成为事实上的标准算法。尽管PPO无处不在，但我们认为PPO中的核心概率剪辑机制在本质上不适合LLMs固有的大词汇量。PPO基于采样标记的概率比来限制策略更新，这实际上是一个噪声单样本蒙特卡洛估计的真实策略偏差。这导致了次优的学习动态：低概率标记的更新被过度惩罚，而高概率标记的重大变化则被过度约束，导致训练效率低下和不稳定。为了解决这个问题，我们提出了差异近端策略优化（DPPO），它用基于直接策略偏差估计（例如，总变差或KL）的更原则性的约束替代了启发式剪辑。为了避免巨大的内存占用，我们引入了高效的二进制和Top-K近似来捕捉关键的偏差，几乎没有额外开销。广泛的实证评估表明，DPPO在训练稳定性和效率方面优于现有方法，为基于RL的LLM精细调整提供了更稳健的基础。

The paper addresses the limitations of Proximal Policy Optimization (PPO) in fine-tuning Large Language Models (LLMs) due to its ratio clipping mechanism, which is ill-suited for LLMs' large vocabularies. It proposes Divergence Proximal Policy Optimization (DPPO), which uses a direct estimate of policy divergence to constrain updates, and introduces Binary and Top-K approximations to reduce memory usage. Experimental results show that DPPO improves training stability and efficiency, outperforming existing methods.

论文针对使用Proximal Policy Optimization (PPO)对大型语言模型（LLM）进行微调时存在的效率低下和不稳定性问题，这些问题源于其比率剪辑机制。提出了Divergence Proximal Policy Optimization (DPPO)，该方法使用直接估计的策略发散度代替比率剪辑。DPPO 包括高效的二进制和Top-K近似，以管理内存使用并保持性能。实验结果表明，DPPO 在提高训练稳定性和效率方面优于PPO和其他方法。

Dense point tracking is a fundamental problem in computer vision, with applications ranging from video analysis to robotic manipulation. State-of-the-art trackers typically rely on cost volumes to match features across frames, but this approach incurs quadratic complexity in spatial resolution, limiting scalability and efficiency. In this paper, we propose \method, a novel dense point tracker that eschews cost volumes in favor of warping. Inspired by recent advances in optical flow, our approach iteratively refines track estimates by warping features from the target frame to the query frame based on the current estimate. Combined with a transformer architecture that performs joint spatiotemporal reasoning across all tracks, our design establishes long-range correspondences without computing feature correlations. Our model is simple and achieves state-of-the-art performance on standard dense point tracking benchmarks, including TAP-Vid-DAVIS, TAP-Vid-Kinetics, and Robo-TAP. Remarkably, the model also excels at optical flow, sometimes outperforming specialized methods on the Sintel, KITTI, and Spring benchmarks. These results suggest that warping-based architectures can unify dense point tracking and optical flow estimation.

标题：CoWTracker：通过扭曲而非相关性进行跟踪

密集点跟踪是计算机视觉中的一个基本问题，应用于从视频分析到机器人操作等多个领域。最先进的跟踪器通常依赖于代价体来在帧间匹配特征，但这种方法在空间分辨率上导致了二次复杂度，限制了其可扩展性和效率。在本文中，我们提出了一种名为\method的新颖密集点跟踪器，它放弃了代价体，转而使用扭曲。受最近光流进展的启发，我们的方法通过根据当前估计将目标帧中的特征扭曲到查询帧上来迭代地细化跟踪估计。结合一个在所有跟踪上执行联合时空推理的变压器架构，我们的设计能够在不计算特征相关性的情况下建立长距离对应关系。我们的模型简单且在标准密集点跟踪基准测试（包括TAP-Vid-DAVIS、TAP-Vid-Kinetics和Robo-TAP）上达到了最先进的性能。令人惊讶的是，该模型在光流估计上也表现出色，有时在Sintel、KITTI和Spring基准测试上甚至超过了专门的方法。这些结果表明，基于扭曲的架构可以统一密集点跟踪和光流估计。

CoWTracker is a novel dense point tracker that uses warping instead of cost volumes, addressing the quadratic complexity issue of existing methods. By iteratively refining track estimates through feature warping and leveraging a transformer for joint spatiotemporal reasoning, CoWTracker achieves state-of-the-art performance on dense point tracking benchmarks and also outperforms specialized methods in optical flow estimation on several benchmarks.

CoWTracker 是一种新颖的密集点跟踪器，采用变形而非代价体的方式，解决了传统方法的二次复杂度问题。它通过迭代地将目标帧中的特征变形到查询帧来细化跟踪估计，并使用变压器进行联合时空推理。该模型在 TAP-Vid-DAVIS、TAP-Vid-Kinetics 和 Robo-TAP 基准测试中优于最先进的跟踪器，并且在 Sintel、KITTI 和 Spring 基准测试中的光学流任务上也表现出色，有时甚至超过了专门的方法。

Recent progress has rapidly advanced our understanding of the mechanisms underlying in-context learning in modern attention-based neural networks. However, existing results focus exclusively on unimodal data; in contrast, the theoretical underpinnings of in-context learning for multi-modal data remain poorly understood. We introduce a mathematically tractable framework for studying multi-modal learning and explore when transformer-like architectures can recover Bayes-optimal performance in-context. To model multi-modal problems, we assume the observed data arises from a latent factor model. Our first result comprises a negative take on expressibility: we prove that single-layer, linear self-attention fails to recover the Bayes-optimal predictor uniformly over the task distribution. To address this limitation, we introduce a novel, linearized cross-attention mechanism, which we study in the regime where both the number of cross-attention layers and the context length are large. We show that this cross-attention mechanism is provably Bayes optimal when optimized using gradient flow. Our results underscore the benefits of depth for in-context learning and establish the provable utility of cross-attention for multi-modal distributions.

The paper addresses the theoretical understanding of multi-modal in-context learning, focusing on the limitations of single-layer self-attention and introducing a novel cross-attention mechanism. It proves that single-layer self-attention cannot achieve Bayes-optimal performance uniformly, while cross-attention, when optimized using gradient flow, can achieve this in the large context length and layer number regime.

论文探讨了基于注意力的神经网络在多模态上下文学习中的理论机制。证明了一层线性自注意力无法在任务分布上统一达到贝叶斯最优性能。为此，引入了一种多层交叉注意力机制，该机制在使用梯度流优化时被证明是贝叶斯最优的，强调了深度和交叉注意力对于多模态数据的重要性。

Large language models have transformed many applications but remain expensive to train. Sparse Mixture of Experts (MoE) addresses this through conditional computation, with Expert Parallel (EP) as the standard distributed training method. However, EP has three limitations: communication cost grows linearly with the number of activated experts $k$, load imbalance affects latency and memory usage, and data-dependent communication requires metadata exchange. We propose Multi-Head LatentMoE and Head Parallel (HP), a new architecture and parallelism achieving $O(1)$ communication cost regardless of $k$, completely balanced traffic, and deterministic communication, all while remaining compatible with EP. To accelerate Multi-Head LatentMoE, we propose IO-aware routing and expert computation. Compared to MoE with EP, Multi-Head LatentMoE with HP trains up to $1.61\times$ faster while having identical performance. With doubled granularity, it achieves higher overall performance while still being $1.11\times$ faster. Our method makes multi-billion-parameter foundation model research more accessible.

标题：多头潜在MoE和头并行：通信高效且确定性的MoE并行

大型语言模型已改变了许多应用，但训练成本仍然很高。稀疏混合专家（MoE）通过条件计算解决了这一问题，专家并行（EP）是标准的分布式训练方法。然而，EP有三个局限性：通信成本随激活专家数量k线性增长，负载不平衡影响延迟和内存使用，数据依赖的通信需要元数据交换。我们提出了一种新的架构和并行方式——多头潜在MoE和头并行（HP），它实现了与k无关的O(1)通信成本，完全平衡的流量，以及确定性的通信，同时仍与EP兼容。为了加速多头潜在MoE，我们提出了I/O感知路由和专家计算。与EP的MoE相比，HP的多头潜在MoE训练速度快1.61倍，性能相同。在加倍粒度的情况下，它实现了更高的整体性能，同时仍快1.11倍。我们的方法使多亿参数的基础模型研究更具可访问性。

The paper addresses the high cost of training large language models by proposing Multi-Head LatentMoE and Head Parallel (HP), which reduces communication cost to $O(1)$, ensures balanced traffic, and achieves deterministic communication. This method overcomes the limitations of Expert Parallel (EP) such as increasing communication cost, load imbalance, and data-dependent communication. Experiments show that Multi-Head LatentMoE with HP trains up to $1.61\times$ faster with the same performance, and with doubled granularity, it achieves higher overall performance while still being $1.11\times$ faster than EP.

论文通过提出Multi-Head LatentMoE和Head Parallel (HP)来降低大型语言模型的训练成本，该方法将通信成本降低到$O(1)$，确保了流量平衡，并实现了确定性的通信。这种方法克服了Expert Parallel (EP)的局限性，如通信成本增加、负载不平衡和数据依赖性通信。实验表明，Multi-Head LatentMoE与HP相比可以快$1.61\times$的速度训练，性能相同，而在粒度翻倍的情况下，它仍然可以实现更高的整体性能，同时比EP快$1.11\times$。

Continual reinforcement learning (CRL) requires agents to learn from a sequence of tasks without forgetting previously acquired policies. In this work, we introduce a novel benchmark suite for CRL based on realistically simulated robots in the Gazebo simulator. Our Continual Robotic Simulation Suite (CRoSS) benchmarks rely on two robotic platforms: a two-wheeled differential-drive robot with lidar, camera and bumper sensor, and a robotic arm with seven joints. The former represent an agent in line-following and object-pushing scenarios, where variation of visual and structural parameters yields a large number of distinct tasks, whereas the latter is used in two goal-reaching scenarios with high-level cartesian hand position control (modeled after the Continual World benchmark), and low-level control based on joint angles. For the robotic arm benchmarks, we provide additional kinematics-only variants that bypass the need for physical simulation (as long as no sensor readings are required), and which can be run two orders of magnitude faster. CRoSS is designed to be easily extensible and enables controlled studies of continual reinforcement learning in robotic settings with high physical realism, and in particular allow the use of almost arbitrary simulated sensors. To ensure reproducibility and ease of use, we provide a containerized setup (Apptainer) that runs out-of-the-box, and report performances of standard RL algorithms, including Deep Q-Networks (DQN) and policy gradient methods. This highlights the suitability as a scalable and reproducible benchmark for CRL research.

标题：CRoSS：一种用于具有高任务多样性和真实物理模拟的可扩展强化学习的持续机器人模拟套件

持续强化学习（CRL）要求智能体从一系列任务中学习，而不忘记之前学到的策略。在本工作中，我们基于Gazebo模拟器中真实模拟的机器人引入了一个新的基准套件。我们的持续机器人模拟套件（CRoSS）基准依赖于两个机器人平台：一个具有激光雷达、摄像头和碰撞传感器的两轮差速驱动机器人，以及一个具有七个关节的机械臂。前者代表了在路径跟随和物体推移场景中的智能体，其中视觉和结构参数的变化产生了大量不同的任务，而后者则用于两个目标到达场景，分别采用高级笛卡尔手位置控制（参考Continual World基准）和基于关节角度的低级控制。对于机械臂基准，我们还提供了额外的仅包含运动学的变体，这些变体可以绕过物理模拟（只要不需要传感器读数），并且可以快两个数量级的速度运行。CRoSS旨在易于扩展，并允许在具有高度物理真实性的机器人环境中进行持续强化学习的研究，特别是允许使用几乎任意的模拟传感器。为了确保可再现性和易用性，我们提供了一个容器化设置（Apptainer），可以开箱即用，并报告了标准RL算法（包括深度Q网络（DQN）和策略梯度方法）的性能。这突显了其作为CRL研究的可扩展和可再现基准的适用性。

The research introduces CRoSS, a benchmark suite for continual reinforcement learning (CRL) using realistically simulated robots in Gazebo. It includes two robotic platforms: a two-wheeled differential-drive robot for line-following and object-pushing tasks, and a robotic arm for goal-reaching scenarios. CRoSS supports high physical realism and allows for controlled studies of CRL in robotic settings. The suite provides kinematics-only variants for the robotic arm to run faster, and includes a containerized setup for easy reproducibility. Standard RL algorithms like DQN and policy gradient methods were tested, showing CRoSS's suitability for scalable CRL research.

研究引入了CRoSS，这是一个基于Gazebo的连续强化学习（CRL）基准套件，使用真实模拟的机器人。它包括两个机器人平台：一个两轮差速驱动机器人用于路径跟随和物体推移任务，以及一个机械臂用于目标抓取场景。CRoSS支持高精度和低精度控制，并提供了仅包含运动学的变体以加快执行速度。实验展示了标准的RL算法如DQN和策略梯度方法的有效性，突显了CRoSS作为可扩展和可重复使用的CRL研究基准的适用性。

Current autoregressive Vision Language Models (VLMs) usually rely on a large number of visual tokens to represent images, resulting in a need for more compute especially at inference time. To address this problem, we propose Mask-LLaVA, a framework that leverages different levels of visual features to create a compact yet information-rich visual representation for autoregressive VLMs. Namely, we combine mask-based object representations together with global tokens and local patch tokens. While all tokens are used during training, it shows that the resulting model can flexibly drop especially the number of mask-based object-tokens at test time, allowing to adapt the number of tokens during inference without the need to retrain the model and without a significant drop in performance. We evaluate the proposed approach on a suite of standard benchmarks showing results competitive to current token efficient methods and comparable to the original LLaVA baseline using only a fraction of visual tokens. Our analysis demonstrates that combining multi-level features enables efficient learning with fewer tokens while allowing dynamic token selection at test time for good performance.

标题：当LLaVA遇到物体：视觉语言模型的标记组成

当前自回归视觉语言模型（VLMs）通常依赖大量的视觉标记来表示图像，这在推理时需要更多的计算资源。为了解决这个问题，我们提出了Mask-LLaVA框架，该框架利用不同级别的视觉特征来创建一种紧凑但信息丰富的视觉表示，适用于自回归VLMs。具体来说，我们结合了基于掩码的对象表示、全局标记和局部补丁标记。虽然所有标记都在训练中使用，但在测试时，结果显示模型可以灵活地减少尤其是基于掩码的对象标记的数量，允许在推理时动态调整标记数量而无需重新训练模型且性能无显著下降。我们在一系列标准基准上评估了该方法，结果显示其结果与当前的标记高效方法相当，并且仅使用少量视觉标记即可达到与原始LLaVA基线相当的结果。我们的分析表明，结合多级特征可以在较少标记的情况下实现高效的训练，并允许在测试时动态选择标记以获得良好的性能。

The paper aims to address the high computational demand of current autoregressive Vision Language Models (VLMs) by proposing Mask-LLaVA, which uses a combination of mask-based object representations, global tokens, and local patch tokens to create a compact visual representation. The model can flexibly reduce the number of mask-based object tokens at inference time without significant performance degradation, making it more efficient. Experiments show that Mask-LLaVA performs competitively on standard benchmarks using fewer visual tokens compared to other token-efficient methods and the original LLaVA baseline.

该论文旨在通过提出Mask-LLaVA框架来解决当前自回归视觉语言模型（VLMs）的高计算需求问题，该框架结合了基于掩码的对象表示、全局令牌和局部补丁令牌，以创建一个紧凑的视觉表示。模型可以在推理时灵活减少基于掩码的对象令牌的数量，而不显著影响性能。实验表明，Mask-LLaVA在标准基准测试中使用更少的视觉令牌与其它令牌高效方法和原始LLaVA基线相比表现相当。

Training modern large language models (LLMs) has become a veritable smorgasbord of algorithms and datasets designed to elicit particular behaviors, making it critical to develop techniques to understand the effects of datasets on the model's properties. This is exacerbated by recent experiments that show datasets can transmit signals that are not directly observable from individual datapoints, posing a conceptual challenge for dataset-centric understandings of LLM training and suggesting a missing fundamental account of such phenomena. Towards understanding such effects, inspired by recent work on the linear structure of LLMs, we uncover a general mechanism through which hidden subtexts can arise in generic datasets. We introduce Logit-Linear-Selection (LLS), a method that prescribes how to select subsets of a generic preference dataset to elicit a wide range of hidden effects. We apply LLS to discover subsets of real-world datasets so that models trained on them exhibit behaviors ranging from having specific preferences, to responding to prompts in a different language not present in the dataset, to taking on a different persona. Crucially, the effect persists for the selected subset, across models with varying architectures, supporting its generality and universality.

标题：数据中的潜意识效应：通过对数线性机制的一般机制

训练现代大型语言模型（LLMs）已经成为一种算法和数据集的盛宴，旨在引发特定行为，因此开发技术以理解数据集对模型属性的影响变得至关重要。最近的实验表明，数据集可以传递无法从单个数据点直接观察到的信号，这为数据集中心的LLM训练理解提出了概念性挑战，并暗示了对这些现象缺乏基本的解释。为了理解这些效应，受最近关于LLM线性结构工作的启发，我们揭示了一种通过隐藏次文本在通用数据集中产生的一般机制。 我们引入了对数概率线性选择（LLS）方法，该方法规定了如何选择通用偏好数据集的子集以引发广泛隐藏效应。我们应用LLS发现现实世界数据集的子集，使得在这些数据集上训练的模型表现出从具有特定偏好，到以数据集中不存在的语言响应提示，再到采取不同人格等行为。关键的是，该效应在所选子集上持续存在，支持其普遍性和普适性。

This study explores the subliminal effects in large language models (LLMs) by developing a method called Logit-Linear-Selection (LLS) to uncover hidden subtexts in datasets. The research demonstrates that by selecting specific subsets of a dataset, models can exhibit a variety of behaviors, such as having specific preferences or responding to prompts in a different language. These effects persist across different model architectures, indicating the generality of the LLS method.

该研究通过开发一种名为Logit-Linear-Selection (LLS)的方法来探索大型语言模型（LLMs）中的潜意识影响。动机是理解数据集如何传递隐藏信号以影响模型行为。关键发现是，通过使用LLS选择特定的数据子集，模型可以表现出各种隐藏效果，如特定偏好、不同语言的响应和改变的人格，并且这些效果在不同模型架构中保持一致。

From generating headlines to fabricating news, the Large Language Models (LLMs) are typically assessed by their final outputs, under the safety assumption that a refusal response signifies safe reasoning throughout the entire process. Challenging this assumption, our study reveals that during fake news generation, even when a model rejects a harmful request, its Chain-of-Thought (CoT) reasoning may still internally contain and propagate unsafe narratives. To analyze this phenomenon, we introduce a unified safety-analysis framework that systematically deconstructs CoT generation across model layers and evaluates the role of individual attention heads through Jacobian-based spectral metrics. Within this framework, we introduce three interpretable measures: stability, geometry, and energy to quantify how specific attention heads respond or embed deceptive reasoning patterns. Extensive experiments on multiple reasoning-oriented LLMs show that the generation risk rise significantly when the thinking mode is activated, where the critical routing decisions concentrated in only a few contiguous mid-depth layers. By precisely identifying the attention heads responsible for this divergence, our work challenges the assumption that refusal implies safety and provides a new understanding perspective for mitigating latent reasoning risks.

标题：CoT 不是真理的链条：生成假新闻的大型语言模型内部推理的实证分析

从生成标题到编造新闻，大型语言模型（LLMs）通常通过其最终输出来评估，假设拒绝回答意味着整个过程中都是安全的推理。我们的研究挑战了这一假设，揭示了在生成假新闻时，即使模型拒绝了有害请求，其链式推理（CoT）中仍可能包含和传播不安全的叙述。为了分析这一现象，我们引入了一个统一的安全分析框架，系统地分解了模型各层的CoT生成，并通过基于雅可比谱度量评估各个注意力头的作用。在该框架下，我们引入了三个可解释的度量标准：稳定性、几何结构和能量，以量化特定注意力头如何响应或嵌入欺骗性推理模式。在多个推理导向的LLMs上的大量实验表明，当思考模式被激活时，生成风险显著上升，其中关键的路由决策集中在少数连续的中层。通过精确识别负责这种分歧的注意力头，我们的工作挑战了拒绝回答意味着安全的假设，并为缓解潜在推理风险提供了新的理解视角。

This study challenges the safety assumption that a model's refusal response guarantees safe reasoning throughout the entire process during fake news generation. By analyzing the Chain-of-Thought (CoT) reasoning of Large Language Models (LLMs), the researchers introduce a safety-analysis framework that evaluates individual attention heads. Key findings show that even when a model rejects a harmful request, unsafe narratives can still be internally propagated. The experiments reveal that the generation risk increases significantly when the thinking mode is activated, with critical routing decisions concentrated in a few mid-depth layers. The work introduces three interpretable measures to quantify deceptive reasoning patterns and provides a new perspective for mitigating latent reasoning risks.

研究挑战了模型拒绝响应意味着整个过程中推理都是安全的这一假设。通过分析大型语言模型（LLM）的链式推理（CoT），研究发现即使模型拒绝了有害请求，其CoT中仍可能包含不安全的叙述。研究引入了一种安全分析框架，通过Jacobian基谱度量系统地分解CoT生成，并评估各个注意力头的作用。研究提出了稳定性、几何和能量三个可解释的度量标准，以量化欺骗性推理模式。实验表明，当思考模式被激活时，生成风险显著增加，关键路由决策集中在少数几个中间深度层中。

Neural audio codecs are at the core of modern conversational speech technologies, converting continuous speech into sequences of discrete tokens that can be processed by LLMs. However, existing codecs typically operate at fixed frame rates, allocating tokens uniformly in time and producing unnecessarily long sequences. In this work, we introduce DyCAST, a Dynamic Character-Aligned Speech Tokenizer that enables variable-frame-rate tokenization through soft character-level alignment and explicit duration modeling. DyCAST learns to associate tokens with character-level linguistic units during training and supports alignment-free inference with direct control over token durations at decoding time. To improve speech resynthesis quality at low frame rates, we further introduce a retrieval-augmented decoding mechanism that enhances reconstruction fidelity without increasing bitrate. Experiments show that DyCAST achieves competitive speech resynthesis quality and downstream performance while using significantly fewer tokens than fixed-frame-rate codecs. Code and checkpoints will be released publicly at https://github.com/lucadellalib/dycast.

标题：超越固定帧：动态字符对齐语音分词

神经音频编解码器是现代对话式语音技术的核心，将连续语音转换为可以被LLMs处理的离散令牌序列。然而，现有的编解码器通常以固定帧率运行，时间上均匀分配令牌，产生不必要的长序列。在本工作中，我们引入了DyCAST，这是一种动态字符对齐语音分词器，通过软字符级对齐和显式时长建模，实现可变帧率分词。DyCAST在训练过程中学习将令牌与字符级语言单元关联，并支持在解码时直接控制令牌时长的无对齐推理。为了在低帧率下提高语音重合成质量，我们进一步引入了一种检索增强解码机制，无需增加比特率即可增强重建保真度。实验表明，DyCAST在使用显著较少的令牌数量的同时，实现了与固定帧率编解码器相当的语音重合成质量和下游性能。代码和检查点将在https://github.com/lucadellalib/dycast公开。

This work addresses the inefficiency of fixed-frame-rate tokenization in neural audio codecs by introducing DyCAST, a dynamic character-aligned speech tokenizer. DyCAST models speech with variable frame rates and explicit duration modeling, improving speech resynthesis quality and downstream performance while using fewer tokens than fixed-frame-rate codecs. It supports alignment-free inference and includes a retrieval-augmented decoding mechanism to enhance reconstruction fidelity at low frame rates.

该研究通过引入DyCAST动态字符对齐语音分词器，解决了固定帧率分词在神经音频编解码器中的低效问题。DyCAST支持可变帧率和显式时长建模，通过减少分词数量，在保持语音重建质量和下游性能的同时，提高了性能。它在训练中学习将分词与字符级语言单元对齐，并在解码时支持对分词时长的灵活控制。此外，引入的检索增强解码机制在低帧率下提高了重建保真度，而不增加比特率。实验结果表明，DyCAST具有竞争力的表现和显著的分词节省。

Large language models often struggle to recognize their knowledge limits in closed-book question answering, leading to confident hallucinations. While decomposed prompting is typically used to improve accuracy, we investigate its impact on reliability. We evaluate three task-equivalent prompting regimes: Direct, Assistive, and Incremental, across different model scales and multi-hop QA benchmarks. We find that although accuracy gains from decomposition diminish in frontier models, disagreements between prompting regimes remain highly indicative of potential errors. Because factual knowledge is stable while hallucinations are stochastic, cross-regime agreement provides a precise signal of internal uncertainty. We leverage this signal to implement a training-free abstention policy that requires no retrieval or fine-tuning. Our results show that disagreement-based abstention outperforms standard uncertainty baselines as an error detector, improving both F1 and AUROC across settings. This demonstrates that decomposition-based prompting can serve as a practical diagnostic probe for model reliability in closed-book QA.

标题：分解提示并不能填补知识空白，但有助于模型说“我不知道”

大型语言模型在闭卷问答中往往难以识别其知识界限，导致自信的虚构。虽然分解提示通常用于提高准确性，但我们研究了其对可靠性的影响。我们评估了三种等效任务的提示制度：直接、辅助和增量，跨越不同模型规模和多跳问答基准。我们发现，尽管分解带来的准确性收益在前沿模型中逐渐减少，但不同提示制度之间的分歧仍然高度表明潜在错误。由于事实性知识是稳定的而虚构是随机的，跨制度的一致性提供了内部不确定性的确切信号。我们利用这一信号实现了一种无需检索或微调的训练免费弃权策略。我们的结果显示，基于分歧的弃权在错误检测方面优于标准不确定性基线，提高了各种设置下的F1和AUROC。这表明基于分解的提示可以作为闭卷问答中模型可靠性的实用诊断探针。

Quantum chemistry is a foundational enabling tool for the fields of chemistry, materials science, computational biology and others. Despite of its power, the practical application of quantum chemistry simulations remains in the hands of qualified experts due to methodological complexity, software heterogeneity, and the need for informed interpretation of results. To bridge the accessibility gap for these tools and expand their reach to chemists with broader backgrounds, we introduce El Agente Quntur, a hierarchical, multi-agent AI system designed to operate not merely as an automation tool but as a research collaborator for computational quantum chemistry. Quntur was designed following three main strategies: i) elimination of hard-coded procedural policies in favour of reasoning-driven decisions, ii) construction of general and composable actions that facilitate generalization and efficiency, and iii) implementation of guided deep research to integrate abstract quantum-chemical reasoning across subdisciplines and a detailed understanding of the software's internal logic and syntax. Although instantiated in ORCA, these design principles are applicable to research agents more generally and easily expandable to additional quantum chemistry packages and beyond. Quntur supports the full range of calculations available in ORCA 6.0 and reasons over software documentation and scientific literature to plan, execute, adapt, and analyze in silico chemistry experiments following best practices. We discuss the advances and current bottlenecks in agentic systems operating at the research level in computational chemistry, and outline a roadmap toward a fully autonomous end-to-end computational chemistry research agent.

标题：El Agente Quntur：量子化学研究合作者代理

量子化学是化学、材料科学、计算生物学等领域的重要基础工具。尽管其功能强大，但由于方法复杂性、软件异构性和结果解释需要专业知识，量子化学模拟的实际应用仍主要掌握在合格专家手中。为缩小这些工具的可访问性差距并将其扩展到背景更广泛的化学家，我们引入了El Agente Quntur，这是一种分层的多代理AI系统，旨在不仅作为自动化工具，而且作为计算量子化学研究的合作者。Quntur的设计遵循了三个主要策略：i) 用基于推理的决策取代硬编码的程序政策，ii) 构建通用且可组合的操作，以促进泛化和效率，iii) 实施引导式深度研究，以在跨子学科整合抽象的量子化学推理，并详细理解软件的内部逻辑和语法。尽管在ORCA中实例化，但这些设计原则适用于更广泛的科研代理，并且易于扩展到其他量子化学软件包。Quntur支持ORCA 6.0中可用的全部计算，并通过软件文档和科学文献进行推理，以计划、执行、适应和分析遵循最佳实践的计算化学实验。我们讨论了在计算化学研究级别操作的代理系统中的进展和当前瓶颈，并概述了通向完全自主的端到端计算化学研究代理的道路。

El Agente Quntur is a hierarchical multi-agent AI system designed to assist in computational quantum chemistry by reasoning-driven decision-making, generalizable actions, and integration of quantum-chemical reasoning. It supports a wide range of calculations in ORCA 6.0 and uses software documentation and scientific literature to plan and analyze experiments. Quntur aims to bridge the accessibility gap for quantum chemistry tools and expand their use among chemists with diverse backgrounds.

El Agente Quntur 是一个分层的多智能体 AI 系统，旨在通过基于推理的决策和可泛化的操作来协助研究人员进行计算量子化学研究，从而扩大对量子化学模拟的访问。关键发现包括 Quntur 能够支持 ORCA 6.0 的全部计算，并利用软件文档和科学文献来规划、执行、适应和分析基于最佳实践的虚拟化学实验。当前的挑战包括解决计算化学研究中代理系统存在的瓶颈，并勾画出完全自主的端到端计算化学研究代理的路线图。

We present El Agente Estructural, a multimodal, natural-language-driven geometry-generation and manipulation agent for autonomous chemistry and molecular modelling. Unlike molecular generation or editing via generative models, Estructural mimics how human experts directly manipulate molecular systems in three dimensions by integrating a comprehensive set of domain-informed tools and vision-language models. This design enables precise control over atomic or functional group replacements, atomic connectivity, and stereochemistry without the need to rebuild extensive core molecular frameworks. Through a series of representative case studies, we demonstrate that Estructural enables chemically meaningful geometry manipulation across a wide range of real-world scenarios. These include site-selective functionalization, ligand binding, ligand exchange, stereochemically controlled structure construction, isomer interconversion, fragment-level structural analysis, image-guided generation of structures from schematic reaction mechanisms, and mechanism-driven geometry generation and modification. These examples illustrate how multimodal reasoning, when combined with specialized geometry-aware tools, supports interactive and context-aware molecular modelling beyond structure generation. Looking forward, the integration of Estructural into El Agente Quntur, an autonomous multi-agent quantum chemistry platform, enhances its capabilities by adding sophisticated tools for the generation and editing of three-dimensional structures.

标题：结构特工：一种人工智能分子编辑器

我们介绍了结构特工，这是一种多模态、自然语言驱动的几何生成和操作代理，用于自主化学和分子建模。与通过生成模型进行的分子生成或编辑不同，结构工模仿了人类专家如何直接在三维空间中操作分子系统的方式，通过整合一系列全面的领域指导工具和视觉语言模型。这种设计使得在无需重建大量核心分子框架的情况下，能够精确控制原子或功能团替换、原子连接性和立体化学。通过一系列代表性案例研究，我们展示了结构工如何在广泛的实际场景中实现有意义的几何操作。这些场景包括选择性功能化、配体结合、配体交换、立体化学控制的结构构建、异构体互变、片段级结构分析、基于示意图反应机制的结构生成以及机制驱动的几何生成和修改。这些示例说明了当多模态推理与专门的几何感知工具结合时，如何支持超越结构生成的交互式和上下文感知的分子建模。展望未来，将结构工整合到量子化学多代理平台El Agente Quntur中，通过添加复杂的三维结构生成和编辑工具，增强了其功能。

El Agente Estructural is a multimodal agent that uses natural language to manipulate molecular structures, enabling precise control over atomic and functional group replacements, atomic connectivity, and stereochemistry. Through various case studies, it demonstrates effective geometry manipulation in diverse scenarios such as site-selective functionalization and isomer interconversion. This approach supports interactive and context-aware molecular modeling beyond simple structure generation.

El Agente Estructural 是一种多模态代理，通过自然语言操作分子结构，实现对原子和功能团替换、原子连接性和立体化学的精确控制。通过多种案例研究，它展示了在多种场景中有效进行几何操作的能力，如选择性功能化和配体交换。这种方法结合了视觉-语言模型和专门的几何感知工具，支持超出简单结构生成的交互式和上下文感知的分子建模。

Reasoning language models, which generate long chains of thought, dramatically outperform non-reasoning language models on abstract problems. However, the internal model mechanisms that allow this superior performance remain poorly understood. We present a mechanistic analysis of how QwQ-32B - a model specifically trained to produce extensive reasoning traces - process abstract structural information. On Mystery Blocksworld - a semantically obfuscated planning domain - we find that QwQ-32B gradually improves its internal representation of actions and concepts during reasoning. The model develops abstract encodings that focus on structure rather than specific action names. Through steering experiments, we establish causal evidence that these adaptations improve problem solving: injecting refined representations from successful traces boosts accuracy, while symbolic representations can replace many obfuscated encodings with minimal performance loss. We find that one of the factors driving reasoning model performance is in-context refinement of token representations, which we dub Fluid Reasoning Representations.

标题：推理模型中的流体表示

生成长链思维的推理语言模型在抽象问题上显著优于非推理语言模型。然而，这些模型内部机制如何实现这种优越性能仍不甚明了。我们对QwQ-32B进行了机械分析，该模型专门训练以产生广泛的推理痕迹。在Mystery Blocksworld（一个语义模糊的规划领域）中，我们发现QwQ-32B在推理过程中逐渐改进其对动作和概念的内部表示。模型发展出专注于结构而非具体动作名称的抽象编码。通过引导实验，我们建立了因果证据，表明这些适应性改进提高了问题解决能力：注入来自成功痕迹的精炼表示可以提高准确性，而符号表示可以最小化性能损失地替代许多模糊编码。我们发现，推动推理模型性能的一个因素是在上下文中对标记表示的精炼，我们称之为流体推理表示。

The research aims to understand the internal mechanisms that enable reasoning language models to outperform non-reasoning models on abstract problems. QwQ-32B, a model trained to produce detailed reasoning traces, was analyzed on the Mystery Blocksworld domain. The study found that QwQ-32B develops abstract encodings focused on structural information rather than specific action names. Steering experiments showed that these adaptations enhance problem-solving accuracy, and refined representations from successful traces can boost performance. The key finding is that in-context refinement of token representations, termed Fluid Reasoning Representations, is a critical factor for reasoning model performance.

研究探讨了QwQ-32B推理语言模型内部机制，以理解其如何处理抽象结构信息并在Mystery Blocksworld任务中提高性能。QwQ-32B开发了专注于结构而非具体动作名称的抽象编码，这些适应性改进提升了问题解决的准确性。模型在上下文中精炼了词表示，这一过程被称为流体推理表示，这对其实现抽象问题上的优异性能起到了重要作用。

Imagine Alice has a specific image $x^\ast$ in her mind, say, the view of the street in which she grew up during her childhood. To generate that exact image, she guides a generative model with multiple rounds of prompting and arrives at an image $x^{p*}$. Although $x^{p*}$ is reasonably close to $x^\ast$, Alice finds it difficult to close that gap using language prompts. This paper aims to narrow this gap by observing that even after language has reached its limits, humans can still tell when a new image $x^+$ is closer to $x^\ast$ than $x^{p*}$. Leveraging this observation, we develop MultiBO (Multi-Choice Preferential Bayesian Optimization) that carefully generates $K$ new images as a function of $x^{p*}$, gets preferential feedback from the user, uses the feedback to guide the diffusion model, and ultimately generates a new set of $K$ images. We show that within $B$ rounds of user feedback, it is possible to arrive much closer to $x^\ast$, even though the generative model has no information about $x^\ast$. Qualitative scores from $30$ users, combined with quantitative metrics compared across $5$ baselines, show promising results, suggesting that multi-choice feedback from humans can be effectively harnessed for personalized image generation.

标题：基于人类在环的贝叶斯优化的个性化图像生成

想象一下，爱丽丝脑海中有一张特定的图像$x^*$，比如她童年时期居住街道的景象。为了生成这张精确的图像，她引导生成模型经过多轮提示，最终得到图像$x^{p*}$。尽管$x^{p*}$与$x^*$相当接近，但爱丽丝发现仅通过语言提示难以进一步缩小差距。本文旨在通过观察即使语言达到极限后，人类仍能判断新图像$x^+$是否比$x^{p*}$更接近$x^*$来缩小这一差距。利用这一观察，我们开发了MultiBO（多选偏好贝叶斯优化），该方法根据$x^{p*}$精心生成K张新图像，从用户那里获取偏好反馈，利用反馈指导扩散模型，并最终生成新的K张图像。我们展示了在B轮用户反馈后，即使生成模型对$x^*$一无所知，也有可能接近$x^*$。30名用户的定性评分与5个基线的定量指标对比显示了有希望的结果，表明人类的多选反馈可以有效用于个性化图像生成。

This paper addresses the challenge of generating a specific image that a user has in mind by developing MultiBO, a method that generates multiple images and uses user feedback to iteratively refine the output. Within 5 rounds of user feedback, the method significantly narrows the gap between the generated image and the user's desired image, demonstrating the effectiveness of multi-choice preferential feedback in personalized image generation.

本文旨在通过利用人类反馈来生成与用户心理图像完全匹配的图像，提出了一种名为MultiBO的方法，该方法生成多个图像并根据用户的偏好进行迭代优化。在5轮反馈后，该方法显著缩小了生成图像与用户期望图像之间的差距，展示了人类反馈在个性化图像生成中的有效性。

Most flagship language models generate explicit reasoning chains, enabling inference-time scaling. However, producing these reasoning chains increases token usage (i.e., reasoning tokens), which in turn increases latency and costs. Our OverThink attack increases overhead for applications that rely on reasoning language models (RLMs) and external context by forcing them to spend substantially more reasoning tokens while still producing contextually correct answers. An adversary mounts an attack by injecting decoy reasoning problems into public content that is consumed by RLM at inference time. Because our decoys (e.g., Markov decision processes, Sudokus, etc.) are benign, they evade safety filters. We evaluate OverThink on both closed-source and open-source reasoning models across the FreshQA, SQuAD, and MuSR datasets. We also explore the attack in multi-modal settings by creating images that cause excessive reasoning. We show that the resulting slowdown transfers across models. Finally, we explore both LLM-based and systems-level defenses, and discuss the societal, financial, and energy implications of the OverThink attacks.

标题：OverThink：对推理大语言模型的减速攻击

大多数旗舰语言模型生成显式的推理链，使推理时能够扩展。然而，生成这些推理链会增加标记使用量（即推理标记），进而增加延迟和成本。我们的OverThink攻击通过迫使依赖推理语言模型（RLM）和外部上下文的应用程序花费大量更多的推理标记，同时仍然生成上下文相关正确的答案，增加了这些应用的开销。攻击者通过向RLM在推理时消费的公共内容中注入伪装的推理问题来发动攻击。由于我们的伪装（例如马尔可夫决策过程、数独等）是无害的，它们能够绕过安全过滤器。我们在FreshQA、SQuAD和MuSR数据集上对OverThink进行了评估，同时还在多模态环境中通过创建导致过度推理的图像来探索攻击。我们展示了这种减速效应在不同模型之间具有转移性。最后，我们探讨了基于大语言模型和系统层面的防御措施，并讨论了OverThink攻击的社会、经济和能源影响。

Open-ended self-improving agents can autonomously modify their own structural designs to advance their capabilities and overcome the limits of pre-defined architectures, thus reducing reliance on human intervention. We introduce Group-Evolving Agents (GEA), a new paradigm for open-ended self-improvements, which treats a group of agents as the fundamental evolutionary unit, enabling explicit experience sharing and reuse within the group throughout evolution. Unlike existing open-ended self-evolving paradigms that adopt tree-structured evolution, GEA overcomes the limitation of inefficient utilization of exploratory diversity caused by isolated evolutionary branches. We evaluate GEA on challenging coding benchmarks, where it significantly outperforms state-of-the-art self-evolving methods (71.0% vs. 56.7% on SWE-bench Verified, 88.3% vs. 68.3% on Polyglot) and matches or exceeds top human-designed agent frameworks (71.8% and 52.0% on two benchmarks, respectively). Analysis reveals that GEA more effectively converts early-stage exploratory diversity into sustained, long-term progress, achieving stronger performance under the same number of evolved agents. Furthermore, GEA exhibits consistent transferability across different coding models and greater robustness, fixing framework-level bugs in 1.4 iterations on average, versus 5 for self-evolving methods.

标题：群体演化智能体：通过经验共享实现开放式的自我提升

开放式自我提升智能体能够自主修改自身的结构设计，以提升能力并克服预定义架构的限制，从而减少对人类干预的依赖。我们提出了群体演化智能体（GEA），这是一种新的开放式自我提升范式，将一群智能体视为基本的演化单元，使群体内的经验共享和重用在整个演化过程中得以实现。与现有的采用树状演化结构的开放式自我演化范式不同，GEA克服了孤立演化分支导致的探索多样性利用效率低下的局限性。我们在具有挑战性的编码基准测试上评估了GEA，结果显示它显著优于最先进的自我演化方法（SWE-bench Verified上71.0% vs. 56.7%，Polyglot上88.3% vs. 68.3%），并且在两个基准测试上达到了或超过了顶级人工设计的智能体框架（71.8%和52.0%）。分析表明，GEA更有效地将早期阶段的探索多样性转化为持续的长期进步，在相同数量的演化智能体下实现了更强的性能。此外，GEA在不同编码模型之间表现出一致的可迁移性，并具有更大的鲁棒性，平均只需1.4次迭代即可修复框架级别的错误，而自我演化方法则需要5次。

The research aims to develop open-ended self-improving agents that can autonomously modify their designs to enhance their capabilities. Group-Evolving Agents (GEA) are introduced as a new paradigm, treating groups of agents as evolutionary units for explicit experience sharing. GEA outperforms existing methods on coding benchmarks, showing significant improvements in performance and robustness, and achieving stronger long-term progress with fewer evolved agents.

研究旨在开发能够自主修改设计以提升能力的开放性自我改进代理。提出了组演化代理（GEA）的新范式，将一组代理视为基本的进化单元，实现显式的经验共享。GEA在编码基准测试中表现出色，显著优于现有方法，并且在相同数量的进化代理下实现了更持久的进步和更强的鲁棒性。

Grammatical error correction (GEC) aims to improve text quality and readability. Previous work on the task focused primarily on high-resource languages, while low-resource languages lack robust tools. To address this shortcoming, we present a study on GEC for Zarma, a language spoken by over five million people in West Africa. We compare three approaches: rule-based methods, machine translation (MT) models, and large language models (LLMs). We evaluated GEC models using a dataset of more than 250,000 examples, including synthetic and human-annotated data. Our results showed that the MT-based approach using M2M100 outperforms others, with a detection rate of 95.82% and a suggestion accuracy of 78.90% in automatic evaluations (AE) and an average score of 3.0 out of 5.0 in manual evaluation (ME) from native speakers for grammar and logical corrections. The rule-based method was effective for spelling errors but failed on complex context-level errors. LLMs -- Gemma 2b and MT5-small -- showed moderate performance. Our work supports use of MT models to enhance GEC in low-resource settings, and we validated these results with Bambara, another West African language.

标题：低资源语言中的语法错误修正：扎马拉语案例

语法错误修正（GEC）旨在提高文本质量和可读性。此前的工作主要集中在高资源语言上，而低资源语言缺乏稳健的工具。为解决这一不足，我们对扎马拉语进行了GEC研究，这是一种在西非有超过五百万使用者的语言。我们比较了三种方法：基于规则的方法、机器翻译（MT）模型和大型语言模型（LLMs）。我们使用包含超过250,000个示例的数据集进行评估，其中包括合成数据和人工标注数据。我们的结果显示，使用M2M100的MT方法在自动评估（AE）中表现最佳，检测率为95.82%，建议准确率为78.90%；在母语使用者的手动评估（ME）中，语法和逻辑修正的平均得分为3.0/5.0。基于规则的方法对于拼写错误有效，但在复杂上下文错误方面表现不佳。LLMs——Gemma 2b和MT5-small——表现出中等水平的性能。我们的工作支持在低资源环境中使用MT模型来增强GEC，并通过另一种西非语言巴马拉语验证了这些结果。

This study addresses the lack of grammatical error correction (GEC) tools for low-resource languages by focusing on Zarma, spoken by over five million people in West Africa. Three approaches—rule-based methods, machine translation (MT) models, and large language models (LLMs)—were compared using a dataset of over 250,000 examples. The MT-based approach using M2M100 outperformed others, achieving a detection rate of 95.82% and a suggestion accuracy of 78.90% in automatic evaluations, and an average score of 3.0 out of 5.0 in manual evaluations from native speakers for grammar and logical corrections. Rule-based methods were effective for spelling errors but struggled with complex context-level errors, while LLMs showed moderate performance.

本研究针对低资源语言缺乏语法纠错（GEC）工具的问题，聚焦于西非地区超过五百万人口使用的扎尔马语。三种方法——基于规则的方法、机器翻译（MT）模型和大型语言模型（LLMs）——被用于超过250,000个样本的数据集进行比较。使用M2M100的基于MT的方法表现最佳，自动评估的检测率为95.82%，建议准确率为78.90%，并且来自母语者的手动评估（ME）平均得分为3.0/5.0，用于语法和逻辑纠错。基于规则的方法在拼写错误方面有效，但在处理复杂上下文错误时表现不佳，而LLMs表现出中等水平的性能。

Online fake news profoundly distorts public judgment and erodes trust in social platforms. While existing detectors achieve competitive performance on benchmark datasets, they remain notably vulnerable to malicious comments designed specifically to induce misclassification. This evolving threat landscape necessitates detection systems that simultaneously prioritize predictive accuracy and structural robustness. However, current detectors often fail to generalize across diverse and novel comment attack patterns. To bridge this gap, we propose AdComment, an adaptive adversarial training framework for robustness enhancement against diverse malicious comments. Based on cognitive psychology, we categorize adversarial comments into Fact Distortion, Logical Confusion, and Emotional Manipulation, and leverage LLMs to synthesize diverse, category-specific perturbations. Central to our framework is an InfoDirichlet Resampling (IDR) mechanism that dynamically adjusts malicious comment proportions during training, thereby steering optimization toward the model's most susceptible regions. Experimental results demonstrate that our approach achieves state-of-the-art performance on three benchmark datasets, improving the F1 scores by 17.9%, 14.5% and 9.0%, respectively.

标题：针对恶意评论的分组自适应对抗学习以增强虚假新闻检测的鲁棒性

在线虚假新闻严重扭曲公众判断并侵蚀社交平台的信任。尽管现有的检测器在基准数据集上取得了竞争力的表现，但它们仍然明显容易受到专门设计以诱导分类错误的恶意评论的影响。这种不断演变的威胁环境需要同时注重预测准确性和结构鲁棒性的检测系统。然而，当前的检测器往往无法在多样且新颖的评论攻击模式中泛化。为弥补这一差距，我们提出了一种AdComment，这是一种针对多种恶意评论的自适应对抗训练框架，以增强鲁棒性。基于认知心理学，我们将对抗性评论分为事实扭曲、逻辑混淆和情感操控三类，并利用大语言模型（LLM）生成多样化的、类别特定的扰动。我们框架的核心是InfoDirichlet重采样（IDR）机制，该机制在训练过程中动态调整恶意评论的比例，从而引导优化向模型最脆弱的区域。实验结果表明，我们的方法在三个基准数据集上取得了最先进的性能，分别提高了F1分数17.9%、14.5%和9.0%。

This paper addresses the vulnerability of existing fake news detectors to malicious comments by proposing AdComment, an adaptive adversarial training framework. It categorizes adversarial comments into three types: Fact Distortion, Logical Confusion, and Emotional Manipulation, and uses LLMs to generate diverse perturbations. The InfoDirichlet Resampling mechanism dynamically adjusts the proportion of malicious comments during training to enhance the model's robustness. The approach achieves state-of-the-art performance, improving F1 scores by 17.9%, 14.5%, and 9.0% on three benchmark datasets.

论文针对现有假新闻检测器对特定诱导错误分类的恶意评论的脆弱性，提出了AdComment，一种适应性对抗训练框架。该框架利用LLMs生成针对事实扭曲、逻辑混淆和情感操控的多样化对抗评论。框架中的InfoDirichlet重采样机制在训练过程中动态调整恶意评论的比例，以聚焦模型最脆弱的部分。实验结果表明，AdComment在三个基准数据集上的F1分数分别提高了17.9%、14.5%和9.0%。

Rapidly increasing AI capabilities have substantial real-world consequences, ranging from AI safety concerns to labor market consequences. The Model Evaluation & Threat Research (METR) report argues that AI capabilities have exhibited exponential growth since 2019. In this note, we argue that the data does not support exponential growth, even in shorter-term horizons. Whereas the METR study claims that fitting sigmoid/logistic curves results in inflection points far in the future, we fit a sigmoid curve to their current data and find that the inflection point has already passed. In addition, we propose a more complex model that decomposes AI capabilities into base and reasoning capabilities, exhibiting individual rates of improvement. We prove that this model supports our hypothesis that AI capabilities will exhibit an inflection point in the near future. Our goal is not to establish a rigorous forecast of our own, but to highlight the fragility of existing forecasts of exponential growth.

标题：人工智能能力是否呈指数增长？一种竞争性假设

迅速增长的人工智能能力在现实世界中产生了重大影响，从人工智能安全问题到劳动力市场影响。《模型评估与威胁研究》(METR) 报告认为，自2019年以来，人工智能能力已经表现出指数增长。本文认为，即使在较短的时间范围内，数据也不支持这种指数增长。虽然METR研究声称拟合S型/逻辑曲线会导致拐点出现在遥远的未来，但我们拟合当前数据的S型曲线发现，拐点已经过去。此外，我们提出了一种更复杂的模型，将人工智能能力分解为基础能力和推理能力，各自具有不同的改进率。我们证明，该模型支持我们的假设，即人工智能能力将在不久的将来出现拐点。我们的目标不是建立我们自己的严谨预测，而是强调现有指数增长预测的脆弱性。

This paper challenges the claim that AI capabilities are growing exponentially, as suggested by the METR report. Instead, the authors fit a sigmoid curve to the data and find that the inflection point has already passed. They also propose a more complex model that decomposes AI capabilities into base and reasoning components, showing that these components improve at different rates. The key finding is that AI capabilities will likely exhibit an inflection point in the near future, contrary to the exponential growth hypothesis.

该论文质疑METR报告中自2019年以来AI能力呈指数增长的说法。作者通过拟合S型曲线发现拐点已经过去。他们还提出一个更复杂的模型，将AI能力分解为基础能力和推理能力，显示这些能力的改进速度不同。研究指出，AI能力将在不久的将来出现拐点，突显了现有指数增长预测的脆弱性。

Our theoretical understanding of neural networks is lagging behind their empirical success. One of the important unexplained phenomena is why and how, during the process of training with gradient descent, the theoretical capacity of neural networks is reduced to an effective capacity that fits the task. We here investigate the mechanism by which gradient descent achieves this through analyzing the learning dynamics at the level of individual neurons in single hidden layer ReLU networks. We identify three dynamical principles -- mutual alignment, unlocking and racing -- that together explain why we can often successfully reduce capacity after training through the merging of equivalent neurons or the pruning of low norm weights. We specifically explain the mechanism behind the lottery ticket conjecture, or why the specific, beneficial initial conditions of some neurons lead them to obtain higher weight norms.

标题：这不是彩票，而是比赛：理解梯度下降如何调整网络容量以适应任务

我们对神经网络的理论理解落后于其实际应用。其中一个重要的未解现象是在使用梯度下降训练过程中，神经网络的理论容量是如何被减少到一个适合任务的有效容量的。我们通过分析单隐藏层ReLU网络中单个神经元的学习动态，研究了梯度下降如何实现这一点。我们确定了三种动态原则——相互对齐、解锁和比赛——这些原则共同解释了为什么我们可以通过等价神经元的合并或低范数权重的剪枝，在训练后通常能够成功地减少容量。我们具体解释了彩票票猜想背后的机制，即为什么某些神经元的特定、有益的初始条件使它们获得更高的权重范数。

The research aims to understand how gradient descent reduces the theoretical capacity of neural networks to an effective capacity that fits the task during training. By analyzing the learning dynamics at the level of individual neurons in single hidden layer ReLU networks, the study identifies three principles: mutual alignment, unlocking, and racing. These principles explain why equivalent neurons can merge or low norm weights can be pruned after training, effectively reducing the network's capacity. The findings specifically address the lottery ticket hypothesis, elucidating why certain neurons with beneficial initial conditions achieve higher weight norms.

研究旨在理解梯度下降如何在训练过程中将神经网络的理论容量减少到适合任务的有效容量。通过分析单隐藏层ReLU网络中单个神经元的学习动力学，研究确定了三个原则：相互对齐、解锁和竞争。这些原则解释了为什么在训练后可以通过合并等效神经元或修剪低范数权重来有效减少网络的容量。研究还具体解释了彩票票猜想背后的机制，阐明了为什么某些具有有利初始条件的神经元会获得更高的权重值。

Early-exit neural networks enable adaptive inference by allowing predictions at intermediate layers, reducing computational cost. However, early exits often lack interpretability and may focus on different features than deeper layers, limiting trust and explainability. This paper presents Explanation-Guided Training (EGT), a multi-objective framework that improves interpretability and consistency in early-exit networks through attention-based regularization. EGT introduces an attention consistency loss that aligns early-exit attention maps with the final exit. The framework jointly optimizes classification accuracy and attention consistency through a weighted combination of losses. Experiments on a real-world image classification dataset demonstrate that EGT achieves up to 98.97% overall accuracy (matching baseline performance) with a 1.97x inference speedup through early exits, while improving attention consistency by up to 18.5% compared to baseline models. The proposed method provides more interpretable and consistent explanations across all exit points, making early-exit networks more suitable for explainable AI applications in resource-constrained environments.

标题：注意力一致性正则化以提高可解释的早期退出神经网络

早期退出神经网络通过在中间层允许预测来实现自适应推理，从而降低计算成本。然而，早期退出通常缺乏可解释性，并且可能关注与深层层不同的特征，限制了信任度和解释性。本文提出了一种多目标框架——解释引导训练(EGT)，通过基于注意力的正则化来提高早期退出网络的可解释性和一致性。EGT 引入了一种注意力一致性损失，将早期退出的注意力图与最终退出对齐。该框架通过损失加权组合联合优化分类准确性和注意力一致性。在真实世界的图像分类数据集上的实验表明，EGT 在保持 1.97 倍的推理加速的同时，实现了高达 98.97% 的总体准确率（与基线性能相当），并且与基线模型相比，注意力一致性提高了 18.5%。所提出的方法在整个退出点提供了更可解释和一致的解释，使早期退出网络更适合资源受限环境中的可解释人工智能应用。

This paper addresses the lack of interpretability in early-exit neural networks by proposing Explanation-Guided Training (EGT), which uses attention consistency regularization to align early-exit attention maps with the final exit. The method jointly optimizes classification accuracy and attention consistency, achieving up to 98.97% accuracy with a 1.97x speedup and up to 18.5% improvement in attention consistency compared to baseline models. The framework enhances the interpretability and explainability of early-exit networks, making them more suitable for resource-constrained environments.

本文提出了一种名为Explanation-Guided Training (EGT)的方法，通过注意力一致性正则化使早期退出的注意力图与最终退出对齐，以解决早期退出神经网络缺乏可解释性的问题。EGT同时优化分类准确性和注意力一致性，实现了高达98.97%的整体准确率，并通过早期退出提高了1.97倍的推理速度，同时将注意力一致性提高了最多18.5%。该方法增强了早期退出网络的可解释性和一致性，使其更适合资源受限的环境。

In an era of AI-generated misinformation flooding the web, existing tools struggle to empower users with nuanced, transparent assessments of content credibility. They often default to binary (true/false) classifications without contextual justifications, leaving users vulnerable to disinformation. We address this gap by introducing TRACE: Transparent Reliability Assessment with Contextual Explanations, a unified framework that performs two key tasks: (1) it assigns a fine-grained, continuous reliability score (from 0.1 to 1.0) to web content, and (2) it generates a contextual explanation for its assessment. The core of TRACE is the TrueGL-1B model, fine-tuned on a novel, large-scale dataset of over 140,000 articles. This dataset's primary contribution is its annotation with 35 distinct continuous reliability scores, created using a Human-LLM co-creation and data poisoning paradigm. This method overcomes the limitations of binary-labeled datasets by populating the mid-ranges of reliability. In our evaluation, TrueGL-1B consistently outperforms other small-scale LLM baselines and rule-based approaches on key regression metrics, including MAE, RMSE, and R2. The model's high accuracy and interpretable justifications make trustworthy information more accessible. To foster future research, our code and model are made publicly available here: github.com/zade90/TrueGL.

标题：TRACE: 透明网络可靠性评估与上下文解释

在AI生成的虚假信息泛滥的时代，现有工具难以赋予用户关于内容可信度的细致、透明的评估。它们通常默认使用二元（真/假）分类，而缺乏上下文解释，使用户容易受到虚假信息的影响。我们通过引入TRACE：透明可靠性评估与上下文解释来填补这一空白：一种统一框架，执行两项关键任务：（1）为网络内容分配一个细粒度的连续可靠性评分（从0.1到1.0），（2）生成评估的上下文解释。TRACE的核心是TrueGL-1B模型，该模型在包含超过140,000篇文章的新型大规模数据集上进行了微调。该数据集的主要贡献是使用人类-大语言模型共同创造和数据污染方法标注了35种不同的连续可靠性评分。这种方法通过填充可靠性范围的中间值，克服了二元标签数据集的局限性。在我们的评估中，TrueGL-1B在关键回归指标（包括MAE、RMSE和R2）上始终优于其他小型语言模型基线和基于规则的方法。该模型的高准确性和可解释的依据使可信信息更加易于获取。为了促进未来的研究，我们的代码和模型在此公开：github.com/zade90/TrueGL。

TRACE is a framework that assesses the reliability of web content with a continuous score and provides contextual explanations. It uses the TrueGL-1B model, fine-tuned on a large dataset with 35 continuous reliability scores, which improves upon binary classification by covering the mid-ranges of reliability. TRACE outperforms other small-scale LLM baselines and rule-based approaches in regression metrics, demonstrating high accuracy and interpretable justifications.

TRACE 是一个框架，通过连续评分和上下文解释来评估网络内容的可靠性。它使用了经过超过140,000篇文章训练的 TrueGL-1B 模型，这些文章被标注了35个连续的可靠性评分。TRACE 在 MAE、RMSE 和 R2 等回归指标上优于其他小型语言模型基线和基于规则的方法，提高了内容可信度的评估。模型的解释具有可解释性，增强了信息的信任度。代码和模型已公开，供进一步研究使用。

Urban traffic management demands systems that simultaneously predict future conditions, detect anomalies, and take safe corrective actions -- all while providing reliability guarantees. We present STREAM-RL, a unified framework that introduces three novel algorithmic contributions: (1) PU-GAT+, an Uncertainty-Guided Adaptive Conformal Forecaster that uses prediction uncertainty to dynamically reweight graph attention via confidence-monotonic attention, achieving distribution-free coverage guarantees; (2) CRFN-BY, a Conformal Residual Flow Network that models uncertainty-normalized residuals via normalizing flows with Benjamini-Yekutieli FDR control under arbitrary dependence; and (3) LyCon-WRL+, an Uncertainty-Guided Safe World-Model RL agent with Lyapunov stability certificates, certified Lipschitz bounds, and uncertainty-propagated imagination rollouts. To our knowledge, this is the first framework to propagate calibrated uncertainty from forecasting through anomaly detection to safe policy learning with end-to-end theoretical guarantees. Experiments on multiple real-world traffic trajectory data demonstrate that STREAM-RL achieves 91.4\% coverage efficiency, controls FDR at 4.1\% under verified dependence, and improves safety rate to 95.2\% compared to 69\% for standard PPO while achieving higher reward, with 23ms end-to-end inference latency.

标题：基于不确定性感知同变预测和世界模型强化学习的城市交通安全控制

城市交通管理需要能够同时预测未来状况、检测异常并采取安全纠正措施的系统——同时提供可靠性保证。我们提出了STREAM-RL，一个统一框架，引入了三个新颖的算法贡献：(1) PU-GAT+，一种不确定性引导自适应同变预测器，利用预测不确定性动态重新加权图注意力，通过置信单调注意力实现无分布覆盖保证；(2) CRFN-BY，一种同变残差流网络，通过贝叶斯-耶库埃利随机发现控制下的归一化流模型不确定性归一化的残差；(3) LyCon-WRL+，一种不确定性引导的安全世界模型强化学习代理，具有李亚普诺夫稳定性证书、认证的利普希茨界和不确定性传播的想象滚动。据我们所知，这是第一个从预测到异常检测再到安全策略学习传播校准不确定性的一体化框架，具有端到端的理论保证。在多个真实世界交通轨迹数据上的实验表明，STREAM-RL 的覆盖效率为 91.4%，在验证相关性下 FDR 控制在 4.1%，安全率提高到 95.2%，而标准 PPO 为 69%，同时获得更高的奖励，端到端推理延迟为 23ms。

The research addresses the need for reliable urban traffic management systems that predict future traffic conditions, detect anomalies, and take safe actions. It introduces STREAM-RL, a unified framework with three novel components: PU-GAT+, which uses prediction uncertainty to improve coverage guarantees; CRFN-BY, which models uncertainty-normalized residuals; and LyCon-WRL+, which ensures safe policy learning with theoretical guarantees. Experiments show that STREAM-RL achieves high coverage efficiency, controlled FDR, improved safety rates, and higher rewards with minimal latency.

研究旨在解决城市交通管理系统需要同时预测未来交通状况、检测异常并采取安全纠正措施的需求。STREAM-RL 提出了三种新颖算法：PU-GAT+ 使用预测不确定性动态重新加权图注意力；CRFN-BY 通过归一化流模型不确定性归一化的残差；LyCon-WRL+ 确保安全策略学习具有理论保证。实验表明，STREAM-RL 实现了 91.4% 的覆盖效率，将 FDR 控制在 4.1%，并将安全性提高到 95.2%，同时具有更高的奖励和 23ms 的端到端推理延迟，优于标准 PPO。

The rapid growth of large language models (LLMs) has outpaced the evolution of single-GPU hardware, making model scale increasingly constrained by memory capacity rather than computation. While modern training systems extend GPU memory through distributed parallelism and offloading across CPU and storage tiers, they fundamentally retain a GPU-centric execution paradigm in which GPUs host persistent model replicas and full autograd graphs. As a result, scaling large models remains tightly coupled to multi-GPU clusters, complex distributed runtimes, and unpredictable host memory consumption, creating substantial barriers for node-scale post-training workloads such as instruction tuning, alignment, and domain adaptation. We present Horizon-LM, a memory-centric training system that redefines the roles of CPU and GPU for large-model optimization. Horizon-LM treats host memory as the authoritative parameter store and uses GPUs solely as transient compute engines through a CPU-master, GPU-template execution model. By eliminating persistent GPU-resident modules and autograd graphs, employing explicit recomputation with manual gradient propagation, and introducing a pipelined double-buffered execution engine, Horizon-LM decouples model scale from GPU count and bounds memory usage to the theoretical parameter footprint. On a single H200 GPU with 1.5\,TB host RAM, Horizon-LM reliably trains models up to 120B parameters. On a standard single A100 machine, Horizon-LM achieves up to 12.2$\times$ higher training throughput than DeepSpeed ZeRO-3 with CPU offloading while preserving numerical correctness. Across platforms and scales, Horizon-LM sustains high device utilization and predictable memory growth, demonstrating that host memory, not GPU memory, defines the true feasibility boundary for node-scale large-model training.

标题：地平线-LM：一种以内存为中心的大模型训练架构

大型语言模型（LLMs）的快速增长已经超过了单GPU硬件的进化速度，使得模型规模越来越受到内存容量的限制，而不是计算能力。尽管现代训练系统通过分布式并行性和跨CPU和存储层卸载来扩展GPU内存，但它们仍然保留了以GPU为中心的执行范式，在这种范式中，GPU托管持久化的模型副本和完整的自动微分图。因此，扩展大型模型仍然紧密依赖于多GPU集群、复杂的分布式运行时以及不可预测的主机内存消耗，这为节点规模的后训练工作负载，如指令调优、对齐和领域适应，设定了巨大的障碍。我们提出了地平线-LM，这是一种以内存为中心的训练系统，重新定义了CPU和GPU在大模型优化中的角色。地平线-LM将主机内存视为权威的参数存储，并通过CPU为主、GPU为模板的执行模型，仅将GPU用作临时计算引擎。通过消除持久化的GPU驻留模块和自动微分图，采用显式重计算和手动梯度传播，并引入流水线双缓冲执行引擎，地平线-LM将模型规模与GPU数量解耦，并将内存使用量限制在理论参数足迹内。在单个H200 GPU和1.5 TB主机RAM的情况下，地平线-LM可靠地训练了多达1200亿参数的模型。在标准单A100机器上，地平线-LM的训练吞吐量比DeepSpeed ZeRO-3高12.2倍，同时保留了数值正确性。在不同平台和规模上，地平线-LM保持了高设备利用率和可预测的内存增长，证明了主机内存而非GPU内存定义了节点规模大模型训练的实际可行性边界。

Horizon-LM is a memory-centric training system designed to address the memory constraints of large language models (LLMs) by redefining the roles of CPU and GPU. It treats host memory as the authoritative parameter store and uses GPUs solely as transient compute engines. This approach decouples model scale from GPU count and bounds memory usage to the theoretical parameter footprint, enabling reliable training of models up to 120B parameters on a single H200 GPU with 1.5 TB host RAM. Compared to DeepSpeed ZeRO-3 with CPU offloading, Horizon-LM achieves up to 12.2 times higher training throughput while maintaining numerical correctness and high device utilization across platforms and scales.

Horizon-LM 是一种面向内存的训练系统，旨在解决大规模语言模型训练中的内存瓶颈问题。该系统重新定义了 CPU 和 GPU 的角色，将主机内存视为权威的参数存储，并仅使用 GPU 作为临时计算引擎。这种方法将模型规模与 GPU 数量解耦，并将内存使用量限制在理论参数足迹内。实验表明，Horizon-LM 可以在单个配备 1.5 TB 主机 RAM 的 H200 GPU 上训练至多 120 亿参数的模型，并在标准单个 A100 机器上实现比 DeepSpeed ZeRO-3 与 CPU 卸载高出 12.2 倍的训练吞吐量，同时保持数值正确性。

The rapid progress of generative models such as GANs and diffusion models has led to the widespread proliferation of AI-generated images, raising concerns about misinformation, privacy violations, and trust erosion in digital media. Although large-scale multimodal models like CLIP offer strong transferable representations for detecting synthetic content, fine-tuning them often induces catastrophic forgetting, which degrades pre-trained priors and limits cross-domain generalization. To address this issue, we propose the Distillation-guided Gradient Surgery Network (DGS-Net), a novel framework that preserves transferable pre-trained priors while suppressing task-irrelevant components. Specifically, we introduce a gradient-space decomposition that separates harmful and beneficial descent directions during optimization. By projecting task gradients onto the orthogonal complement of harmful directions and aligning with beneficial ones distilled from a frozen CLIP encoder, DGS-Net achieves unified optimization of prior preservation and irrelevant suppression. Extensive experiments on 50 generative models demonstrate that our method outperforms state-of-the-art approaches by an average margin of 6.6, achieving superior detection performance and generalization across diverse generation techniques.

标题：DGS-Net: 生成模型引导的梯度手术网络用于CLIP微调以检测AI生成的图像

生成模型如GANs和扩散模型的快速发展导致了AI生成图像的广泛传播，这引发了对数字媒体中信息误导、隐私侵犯和信任侵蚀的担忧。尽管大规模多模态模型如CLIP提供了检测合成内容的强大可迁移表示，但对其进行微调往往会引起灾难性遗忘，这会削弱预训练先验并限制跨域泛化。为了解决这一问题，我们提出了生成模型引导的梯度手术网络（DGS-Net），这是一种新颖的框架，可以保留可迁移的预训练先验，同时抑制与任务无关的组件。具体而言，我们引入了一种梯度空间分解，将优化过程中的有害和有益下降方向分离。通过将任务梯度投影到有害方向的正交补空间，并与来自冻结CLIP编码器的有益方向对齐，DGS-Net实现了先验保留和无关抑制的统一优化。在50个生成模型上的广泛实验表明，我们的方法在平均意义上比最先进的方法高出6.6的性能差距，实现了在多种生成技术下的优越检测性能和泛化能力。

The paper addresses the challenge of detecting AI-generated images using CLIP, which can suffer from catastrophic forgetting during fine-tuning. To tackle this, the authors propose DGS-Net, a novel framework that decomposes gradients into harmful and beneficial components. By aligning task gradients with beneficial ones distilled from a frozen CLIP encoder and orthogonalizing them from harmful ones, DGS-Net preserves pre-trained priors while suppressing irrelevant components. Experiments show that DGS-Net outperforms existing methods by an average margin of 6.6 across 50 generative models, achieving better detection performance and generalization.

论文解决了使用CLIP模型检测AI生成图像时遇到的灾难性遗忘问题。为了解决这个问题，作者提出了DGS-Net，该方法通过梯度手术来保留预训练的先验知识，同时抑制无关组件。实验结果显示，DGS-Net 在50个生成模型上的表现优于现有方法，平均领先6.6，提高了检测性能和不同生成技术下的泛化能力。

High-dynamic-range (HDR) formats and displays are becoming increasingly prevalent, yet state-of-the-art image generators (e.g., Stable Diffusion and FLUX) typically remain limited to low-dynamic-range (LDR) output due to the lack of large-scale HDR training data. In this work, we show that existing pretrained diffusion models can be easily adapted to HDR generation without retraining from scratch. A key challenge is that HDR images are natively represented in linear RGB, whose intensity and color statistics differ substantially from those of sRGB-encoded LDR images. This gap, however, can be effectively bridged by converting HDR inputs into perceptually uniform encodings (e.g., using PU21 or PQ). Empirically, we find that LDR-pretrained variational autoencoders (VAEs) reconstruct PU21-encoded HDR inputs with fidelity comparable to LDR data, whereas linear RGB inputs cause severe degradations. Motivated by this finding, we describe an efficient adaptation strategy that freezes the VAE and finetunes only the denoiser via low-rank adaptation in a perceptually uniform space. This results in a unified computational method that supports both text-to-HDR synthesis and single-image RAW-to-HDR reconstruction. Experiments demonstrate that our perceptually encoded adaptation consistently improves perceptual fidelity, text-image alignment, and effective dynamic range, relative to previous techniques.

标题：X2HDR：在感知均匀空间中生成高动态范围图像

高动态范围（HDR）格式和显示器正在变得越来越普遍，然而最先进的图像生成器（例如Stable Diffusion和FLUX）通常仍然受限于低动态范围（LDR）输出，因为缺乏大规模HDR训练数据。在本文中，我们展示了现有的预训练扩散模型可以轻松适应HDR生成，而无需从头开始重新训练。一个关键挑战是HDR图像以线性RGB形式原生表示，其强度和颜色统计与sRGB编码的LDR图像有显著差异。然而，通过将HDR输入转换为感知均匀编码（例如使用PU21或PQ），这一差距可以得到有效解决。实验证明，LDR预训练的变分自编码器（VAEs）可以以与LDR数据相当的保真度重建PU21编码的HDR输入，而线性RGB输入会导致严重的降解。受此发现的启发，我们描述了一种高效的适应策略，即冻结VAE并仅通过低秩适应在感知均匀空间中微调去噪器。这产生了一种统一的计算方法，支持从文本到HDR的合成和单张RAW图像到HDR的重建。实验表明，与以前的技术相比，我们的感知编码适应始终能提高感知保真度、文本-图像对齐和有效动态范围。

The research addresses the challenge of generating HDR images using pretrained LDR models by converting HDR inputs into perceptually uniform encodings. The key finding is that LDR-pretrained variational autoencoders (VAEs) can reconstruct PU21-encoded HDR inputs with high fidelity, while linear RGB inputs lead to severe degradations. The proposed adaptation strategy freezes the VAE and fine-tunes only the denoiser in a perceptually uniform space, enabling both text-to-HDR synthesis and single-image RAW-to-HDR reconstruction with improved perceptual fidelity and dynamic range.

该研究通过将HDR输入转换为感知均匀编码来解决使用预训练的LDR模型生成HDR图像的挑战。关键发现是，使用感知均匀编码（如PU21）的LDR预训练变分自编码器可以高保真地重建HDR输入，而直接使用线性RGB输入会导致严重降解。所提出的X2HDR方法通过感知均匀空间中的低秩适应来微调去噪器，支持文本到HDR合成和单图像RAW到HDR重建，并在感知保真度、文本图像对齐和动态范围方面优于先前的技术。

Large Language Model (LLM)-based agents that plan, use tools and act has begun to shape healthcare and medicine. Reported studies demonstrate competence on various tasks ranging from EHR analysis and differential diagnosis to treatment planning and research workflows. Yet the literature largely consists of overviews which are either broad surveys or narrow dives into a single capability (e.g., memory, planning, reasoning), leaving healthcare work without a common frame. We address this by reviewing 49 studies using a seven-dimensional taxonomy: Cognitive Capabilities, Knowledge Management, Interaction Patterns, Adaptation & Learning, Safety & Ethics, Framework Typology and Core Tasks & Subtasks with 29 operational sub-dimensions. Using explicit inclusion and exclusion criteria and a labeling rubric (Fully Implemented, Partially Implemented, Not Implemented), we map each study to the taxonomy and report quantitative summaries of capability prevalence and co-occurrence patterns. Our empirical analysis surfaces clear asymmetries. For instance, the External Knowledge Integration sub-dimension under Knowledge Management is commonly realized (~76% Fully Implemented) whereas Event-Triggered Activation sub-dimenison under Interaction Patterns is largely absent (~92% Not Implemented) and Drift Detection & Mitigation sub-dimension under Adaptation & Learning is rare (~98% Not Implemented). Architecturally, Multi-Agent Design sub-dimension under Framework Typology is the dominant pattern (~82% Fully Implemented) while orchestration layers remain mostly partial. Across Core Tasks & Subtasks, information centric capabilities lead e.g., Medical Question Answering & Decision Support and Benchmarking & Simulation, while action and discovery oriented areas such as Treatment Planning & Prescription still show substantial gaps (~59% Not Implemented).

标题：医疗健康领域中的代理型AI：基于大型语言模型代理的七维度分类学评估

基于大型语言模型（LLM）的代理能够规划、使用工具和行动，已经开始塑造医疗健康领域。已有研究展示了这些代理在从电子病历分析和鉴别诊断到治疗规划和研究工作流等各项任务上的能力。然而，文献主要由综述性文章构成，要么是广泛的综述，要么是单一能力（如记忆、规划、推理）的深入探讨，这使得医疗工作缺乏一个共同的框架。我们通过使用七维度分类学审查了49项研究，该分类学包括认知能力、知识管理、交互模式、适应与学习、安全与伦理、框架类型学和核心任务与子任务，共29个操作维度。我们使用明确的纳入和排除标准以及标签标准（完全实现、部分实现、未实现），将每项研究映射到分类学，并报告了能力的出现频率和共现模式的定量总结。我们的实证分析揭示了明显的不对称性。例如，在知识管理下的外部知识集成子维度通常被实现（约76%完全实现），而在交互模式下的事件触发激活子维度则很少见（约92%未实现），而在适应与学习下的漂移检测与缓解子维度也很罕见（约98%未实现）。从架构上看，框架类型学下的多代理设计子维度是最常见的模式（约82%完全实现），而协调层则主要部分实现。在核心任务与子任务中，以信息为中心的能力占主导地位，例如医学问答与决策支持和基准测试与模拟，而以行动和发现为导向的领域，如治疗规划与处方，仍然存在显著差距（约59%未实现）。

This study addresses the lack of a common framework for evaluating LLM-based agents in healthcare by proposing a seven-dimensional taxonomy. The taxonomy includes Cognitive Capabilities, Knowledge Management, Interaction Patterns, Adaptation & Learning, Safety & Ethics, Framework Typology, and Core Tasks & Subtasks. The empirical analysis of 49 studies reveals that while some dimensions like External Knowledge Integration are commonly realized, others like Event-Triggered Activation and Drift Detection & Mitigation are largely absent or rarely implemented. The multi-agent design is the dominant architectural pattern, but orchestration layers remain partial. Information-centric capabilities are more prevalent than action and discovery-oriented areas.

该研究通过提出一个七维度分类法，解决了评估医疗领域基于LLM的代理缺乏共同框架的问题。分类法包括认知能力、知识管理、交互模式、适应与学习、安全与伦理、架构类型和核心任务与子任务。对49项研究的实证分析显示，虽然外部知识集成等维度被广泛实现，但事件触发激活和漂移检测与缓解等维度则很少实现或未实现。多代理设计是主要的架构模式，但协调层仍处于部分实现状态。信息中心型能力比行动和发现导向型领域更为普遍。