Artificial Intelligence · Cross-listed from cs.CL
Do New Attention Mechanisms Actually Fix Attention Sinks at Million-Token Context?
Sara Rizwan, Samaanah Abdus Salam and Mohammed Mudassir Uddin
Long context
language models now advertise windows of one million
tokens, but two habits limit how much of that window is used.
Attention heads with nothing useful to read still spend their budget on the first
token, which is called the
attention sink, and where a fact sits in the context changes whether the model finds it. Gated
attention cut first
token attention from 46.7 percent to 4.8 percent at NeurIPS 2025, and Kimi K3 pairs that idea with Kimi Delta
Attention and
Attention Residuals behind a one million
token window, eight times past the range where these diagnostics have been reported. This paper asks whether the fix survives that jump. We build SinkProbe, a suite that measures sink mass, massive activation, position resolved recall and the recency gap, and apply it to four small models that differ only in how they mix
tokens and depth. Three results follow. The training objective produces the sink, not the architecture. Gating did not reproduce its published effect at our scale. Sink mass, activations and position bias moved independently. Code, data and the measurement protocol are released at https://github.com/sararizwan7/
Attention-Mechanisms-in-1M-Context-Window
Read the full abstract ▾
Artificial Intelligence
AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems
Jaewon Chu, Jinwoo Seo, Jaewon Cho et al. (+4 more)
Large
language model (LLM)-based multi-
agent systems (MAS) achieve strong performance by employing specialized multiple
agents, yet their performance depends on the
prompt design of each
agent. For MAS
prompt optimization, textual gradient methods that guide
prompt updates using natural-
language feedback have emerged as a leading paradigm. In this paper, we identify limitations in two stages of existing textual gradient approaches: gradient extraction and gradient aggregation. In gradient extraction, previous works select a target
prompt without verifying whether modifying it resolves the failure, and derive gradients without
agent-level supervision over the corresponding
agent's intermediate output. In gradient aggregation, individual gradients are randomly grouped and concatenated, often mixing unrelated failure modes and producing
prompts that fail to generalize. To address these limitations, we propose AgentGrad, a
prompt optimization framework for multi-
agent systems based on sequential intervention and semantic textual gradient abstraction. For each failure, sequential intervention modifies the behavior of one
agent at a time to identify the target
agent whose modification resolves the failure. The modified output of the target
agent then serves as
agent-level supervision for extracting a fine-grained gradient. Semantic textual gradient abstraction clusters semantically similar gradients to prevent mixing unrelated failure modes, and abstracts each cluster into a generalized gradient that captures the shared corrective pattern. Experimental results show that AgentGrad achieves state-of-the-art performance across five MAS
benchmarks while reducing wall-clock optimization time by $2.5\times$ and optimization cost by 21.8% on average compared to the next-best baselines.
Read the full abstract ▾
All authors Jaewon Chu, Jinwoo Seo, Jaewon Cho, Jeehye Na, Yunyang Xiong, Youngdae Kim, Hyunwoo J. Kim
Artificial Intelligence
BIO-MEMART: Biometric-Aware KV Cache Memory for Multi-User LLM Agents
Yanhong Qian, Xuanying He, Qingguo Meng et al. (+3 more)
KV cache is evolving from a serving optimization into an external
memory substrate for long-term LLM
agents. In a shared multi-user deployment, however, reusable KV blocks introduce a missing access-control question: semantic relevance alone cannot determine whether a
memory block is authorized for the current physical user. We propose Bio-MemArt, a biometric-aware KV-cache
memory framework for multi-user LLM
agents. Bio-MemArt attaches a normalized biometric template to each stored KV
memory block, filters the shared
memory pool with the current user's biometric probe, and then runs the original MemArt
retrieval and KV reuse pipeline only inside the authorized candidate pool. This design preserves latent-space
retrieval, direct cache reuse, and decoupled position encoding while adding physical-user access control to shared KV
memory. We evaluate Bio-MemArt under Owner and Non-owner query conditions on long-term dialogue QA with face and palmprint
benchmarks. Across face
benchmarks, the average owner and non-owner biometric success rates are 95.71% and 0.86%; across palmprint
benchmarks, they are 97.60% and 2.00%. In the efficiency study, average prefill
tokens drop from 18,781.96 under full-context prompting to 28.57 with Bio-MemArt, showing that biometric gating preserves the low-
token operating regime of KV-cache
memory.
Read the full abstract ▾
All authors Yanhong Qian, Xuanying He, Qingguo Meng, Shihao Ding, Xingbo Dong, Zhe Jin
Artificial Intelligence
Personalizing LLM Agent Memory Using Biometrics
Yanhong Qian, Qingguo Meng, Shihao Ding et al. (+4 more)
Personalized
memory helps LLM
agents deliver stable, tailored assistance by storing and reusing user-specific data across interactions. In multi-user scenarios, however,
retrieval must consider not only semantic similarity but also whether the current requester matches the identity associated with the stored
memory. We propose Bio-
Memory, a biometric-aware
memory architecture that conditions
memory retrieval on both semantic similarity and biometric matching. Built on top of A-Mem, Bio-
Memory augments each atomic
memory note with a biometric
embedding and uses biometric matching to form the
retrieval candidate pool before semantic ranking. We evaluate Bio-
Memory on LoCoMo in a 10-user shared-
agent setting over 7 face
benchmarks and 10 palmprint protocols. Across
datasets, Bio-
Memory consistently separates owner and non-owner queries. Under face-based personalization, the largest average gap reaches 27.29% / 21.15% in F1 / BLEU-1 on CALFW; under palmprint-based personalization, the corresponding gap is 25.75% / 19.22% on MS_Blue. These results support biometrics as a practical control signal for personalized
memory retrieval in shared environments.
Read the full abstract ▾
All authors Yanhong Qian, Qingguo Meng, Shihao Ding, Xingbo Dong, Zhe Jin, Hanrui Wang, Isao Echizen
Artificial Intelligence · Cross-listed from cs.CL
Same Values, Different Languages? From Multilingual Probing to Steering LLMs Toward Chinese Social Values
Yuemei Xu, Kexin Xu, Jian Zhou et al. (+3 more)
As Large
Language Models (LLMs) are increasingly integrated into human society, aligning them with pluralistic social values has become a critical priority. However, whether LLMs exhibit consistent value preferences across
languages remains underexplored, particularly for culturally grounded values, which are more abstract and difficult to evaluate and align than safety-centric principles. We investigate this issue through Chinese Social Values (CSV), a value system rooted in Chinese culture and comprising $12$ dimensions across national, societal, and personal levels. We construct C-Voices, the first comprehensive
multilingual contrastive probe
dataset for CSV, with 86,400 dilemma-based instances in six
languages, each pairing a CSV-aligned action with a value-conflicting alternative. Building on the contrastive probes of C-Voices, we then propose a fine-tuning-free value vector steering method that derives value directions from hidden-state discrepancies and selectively intervenes on value-sensitive layers during inference. Experiments on six
languages show that CSV-oriented preferences are model-dependent and
language-sensitive, with the same dilemma eliciting divergent responses across
languages. Our method achieves effective CSV steering, supports cross-lingual transfer of value vectors, and generalizes to existing FLAMES and ValuePrism.
Read the full abstract ▾
All authors Yuemei Xu, Kexin Xu, Jian Zhou, Haoyu Lu, Yequan Wang, Aishan Liu
Artificial Intelligence
SRPO: Setwise Relative Policy Optimization for Multi-Agent Systems
Shengtian Yang, Ziyu Xiong, Yu Li et al. (+3 more)
Multi-
agent systems enable complex
reasoning and tool use by coordinating
agents that divide roles and refine candidate solutions. Existing methods typically update individual
agent responses or treat a complete trajectory as one training example. However, these methods may produce misleading policy updates because they assign the same final outcome to responses or trajectory segments that may play different roles in different team decisions. This is because treating each response as an independent update may separate outputs that jointly determine the next action, while treating an entire trajectory as one update may combine decisions made after different observations. These limitations call for a policy update defined at the level of a team decision, outputs that lead to the same state transition are optimized under a shared objective. In this paper, we propose Setwise Relative Policy Optimization (SRPO) for multi-
agent systems. We represent the outputs used together to produce one state transition as an active set. A singleton set covers division of labor, while a larger set covers joint co-
evolution, and the set composition can change across decisions. SRPO assigns a shared advantage to each active set, clips the combined policy change, and normalizes its scale according to the set size. This ties each update to the decision that produced the next state. Experiments on mathematical
reasoning and multi-turn search demonstrate the effectiveness of SRPO across both tasks. Additional ablation studies analyze the normalization choice and training behavior under changing active-set sizes.
Read the full abstract ▾
All authors Shengtian Yang, Ziyu Xiong, Yu Li, Yewen Li, Qingpeng Cai, Lei Feng
Artificial Intelligence · Cross-listed from cs.IR
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Lin Guan, Jia-Qi Yang, Zhishan Zhao et al. (+15 more)
Modeling long-term user behavior is central to sequential recommendation and billion-scale industrial recommender systems, yet production ranking models operate under strict latency,
memory, communication, and training-throughput constraints. At the 100K scale, the challenge extends beyond
attention complexity: raw sequence features must be stored, transferred, and repeatedly processed during training and online serving. Existing approaches based on history truncation, multi-stage behavior
retrieval, compressed lifelong histories, or train-short/infer-long extrapolation either weaken end-to-end optimization or retain substantial length-dependent cost. We present SequenceO1, an end-to-end framework for ultra-long user behavior sequence modeling, deployed at full traffic on Douyin with histories of up to 100K interactions. SequenceO1 follows a compress-then-reason design. Its Sketch
Attention (SA) uses learnable prototypes and prototype-wise normalization to compress the raw history into a fixed-size, target-agnostic user representation. Target-conditioned Stacked Target-to-History Cross
Attention (STCA) then models complementary time scales: a recent 10K suffix for short-term interests and the compact sketch for long-term preferences. To make training and inference practical, SequenceO1 combines low-rank user representation caching, multi-request user-level batching, pipeline lift, and a fused FlashSA kernel to amortize feature storage, communication, and computation across targets, training instances, and consecutive requests. Production experiments show consistent offline and online gains, while the compact cached sketch retains most of the benefit of directly scaling end-to-end sequence ranking to 100K. These results provide a practical model-system approach to efficient
attention, sequence compression, and scalable long-sequence and long-context recommendation systems.
Read the full abstract ▾
All authors Lin Guan, Jia-Qi Yang, Zhishan Zhao, Jiaqi Huang, Hangyu Wang, Longbin Li, Beichuan Zhang, Haonan Jiang, Jinan Ni, Xiangyu Fan, Xiaowen Li, Ziyao Ren, Yuhang Qi, Xiaolong Zhu, Xuanyuan Luo, Qiwei Chen, Yi Cheng, Lele Yu
Artificial Intelligence · Cross-listed from cs.CV
AirAnchor: Bridging Local and Global Spatial Information for Zero-Shot Aerial Vision-and-Language Navigation
Shanwei Fan, Bin Zhang, Zhiwei Xu et al. (+4 more)
Aerial
Vision-and-
Language Navigation requires drones to follow natural-
language instructions and navigate through complex urban environments. Accurate
navigation relies on both local and global spatial information, which support immediate action grounding and long-horizon path planning, respectively. However, existing zero-shot methods typically operate at a single spatial scale, relying either on local representations constructed online from current observations or on global
memories built offline from historical experience. To address this limitation, we propose AirAnchor, a new paradigm that bridges local and global spatial information through spatial anchors and integrates both into a shared
navigation framework, enabling comprehensive spatial grounding for decision-making. AirAnchor consists of three core components: (1) Query-Driven Spatial Anchor Grounding, which identifies decision-relevant anchors from visual observations and organizes them into local spatial representations; (2) Persistent Object Spatial
Memory, which incrementally maintains an object knowledge base as persistent global spatial
memory and retrieves landmark-related spatial priors; and (3) a Spatially-Informed
Navigation Agent, which explicitly integrates both local and global spatial information into an agentic framework for decision-making. Extensive experiments on AerialVLN demonstrate that AirAnchor substantially outperforms existing zero-shot baselines, validating the effectiveness and efficiency of the proposed paradigm.
Read the full abstract ▾
All authors Shanwei Fan, Bin Zhang, Zhiwei Xu, Yingxuan Teng, Siqi Dai, Lin Cheng, Guoliang Fan
Artificial Intelligence
EvolveScaler: Synthesizing Information-Evolution Contexts via Executable State Machines and Natural-Language Rendering
Ziliang Zhao, Zenan Xu, Shuting Wang et al. (+6 more)
In persistent interactions, long contexts may encode an evolving process rather than a fixed record: later events can revise or revoke earlier information, changing what remains valid and what conclusions follow. We call this setting information
evolution (IE). Solving IE requires identifying valid records, applying updates in order, and reconstructing the query-relevant state from the event history. Existing text-first synthesis pipelines make such data difficult to verify because state transitions and answer logic remain implicit. We introduce EvolveScaler, a code-driven framework that defines information
evolution before rendering it as natural
language. Human-authored operational specifications define state transitions, record validity, difficulty controls, and executable answer logic; a strong LLM then synthesizes a self-contained simulator from each specification. Executing validated simulators produces natural-
language multi-turn event histories, while deterministic replay computes reference answers and atomic checklists. We instantiate EvolveScaler with 117 task prototypes and 159 final-question operators across five difficulty levels spanning approximately 7 to 1,200 events per instance, yielding about 35,100 training examples and 585 validated evaluation instances. On the very_long tier, the strongest model reaches 59.3% avg@5, while six models score below 10%. Training an internal A3B model on 6,000 EvolveScaler examples improves performance over its base checkpoint on all eight independently constructed out-of-distribution
benchmarks, with a 5.25-point average gain. These results show that code-driven IE synthesis provides both challenging evaluation and transferable training supervision.
Read the full abstract ▾
All authors Ziliang Zhao, Zenan Xu, Shuting Wang, Zhao Wang, Bowen Cao, Minda Hu, Lincheng Li, Pluto Zhou, Zhicheng Dou
Artificial Intelligence
Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models
Zongjie Li, Alan Z. W, John Nicolas J et al. (+4 more)
Training capable cyber
agents is often treated primarily as a problem of model scale, yet open-weight post-training is constrained more directly by the cost of executable environments, reliable multi-turn supervision, and access to strong teachers. We present a data-centric framework that addresses these bottlenecks through five complementary systems: Choulea analyzes hidden
reasoning signatures, SkyReal reduces teacher-sampling cost, Hongzwang bypasses API restrictions on teacher execution, PSBreakup restores capabilities weakened by model merging, and Kreator converts expert interventions into trainable
reasoning. Our data engine constructs resettable coding, vulnerability, CTF, kernel-history, full-exploit, firmware, and device-backed environments. Candidate trajectories are retained only after execution verification and evidence
auditing, yielding 164,269 trajectories for long-context supervised fine-tuning. The three checkpoints improve over their starting models by an average of 23.76% on the full CyberGym suite and 10.49% across the pooled CTF suites. As of September 1, 2026, Feyospace-s1 achieves a verified success rate of 63.24% and ranks 10th on the official CyberGym leaderboard, while all three checkpoints rank 1st among models at comparable parameter scales. To our knowledge, this is the first end-to-end demonstration that a seven-person independent team can train open-weight models with leading agentic cyber capability.
Read the full abstract ▾
All authors Zongjie Li, Alan Z. W, John Nicolas J, Walter H. F, Scott Donald L, Gordon Y. P, Deke X Jr
Artificial Intelligence
FastE: Readout-Triggered Token Compression for LLM Embedding Inference
Jinsong Shu, Jinyong Wen, Baokun Wang et al. (+4 more)
In this study, we identify depth-dependent prefix redundancy in final-readout LLM
embedding models, notably across representative backbones including Qwen3-
Embedding and Qwen3-VL-
Embedding. We find that removing prefix states is substantially more damaging in shallow layers than at greater depth, showing that prefix states become increasingly compressible as the prefix and readout states propagate through the network. To this end, we introduce FastE, a training-free, plug-and-play method. FastE uses a shared fixed threshold on batch-mean readout-prefix
alignment as a lightweight online heuristic for selecting when compression occurs, and ranks prefix states by the
attention scores they receive from the readout position to determine which states are retained in subsequent layers. Our evaluations demonstrate FastE's ability to substantially reduce computational costs: on NarrativeQA with Qwen3-
Embedding-0.6B, it reduces decoder-backbone FLOPs by 40.11% while retaining 99.53% of Full Forward nDCG@10. Across five text
embedding benchmarks, two backbone scales, and three cross-modal
retrieval tasks, the quality-efficiency trade-off is directly customizable through the maximum removal ratio without retraining. We believe FastE offers practical value for scalable
embedding generation in
retrieval, indexing, clustering, and
multimodal representation systems.
Read the full abstract ▾
All authors Jinsong Shu, Jinyong Wen, Baokun Wang, Zhongle Xie, Lidan Shou, Weiqiang Wang, Gang Chen
Artificial Intelligence · Cross-listed from cs.LG
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
Hongbang Yuan, Zhuoran Jin and Yixin Cao
Large
Language Models demonstrate remarkable proficiency in static
reasoning, yet training them as autonomous
agents through Reinforcement Learning (RL) for long-horizon tasks is often hindered by severe reward sparsity. While conventional
agent-side warming up via supervised fine-tuning (SFT) can alleviate this, it is frequently limited by data scarcity and constrained exploration. To address this, we propose a paradigm shift to environment-side adaptation by constructing Feedback-Enriched Environments (FEEs). Through a pilot study, we establish a feedback design strategy that reformulates environments by transitioning from action guidance to observation enrichment during the later stages of both intra-episode exploration and inter-episode
evolution. Large-scale experiments on SciWorld and BFCL
benchmarks using various Qwen3 model scales and RL
algorithms such as GRPO, GSPO, and DAPO demonstrate that FEEs consistently yield performance improvements over standard settings. Furthermore, our analysis reveals that training with FEEs (1) stabilizes training dynamics by reducing
entropy volatility, (2) facilitates proactive state-space exploration in difficult tasks, (3) ensures the internalization of environmental guidance into policy weights rather than acting as a mere inference-time prior, and (4) identifies intra-group feedback consistency as a critical boundary for stable optimization.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Towards Embodied Air-Ground Cooperative Object Search: Benchmark, Dataset and Agentic Method
Boao Yu, Zimo Chen, Junreng Rao et al. (+4 more)
Air-Ground Object Search (AGOS) in urban environments is a challenging
embodied task, which requires an Unmanned Aerial Vehicle (
UAV) and an Unmanned Ground Vehicle (UGV) to jointly search for and verify a specified target vehicle from multi-view visual references. To study this underexplored problem, we introduce AGOS-Bench, the first dedicated
benchmark for evaluating whether general-purpose
Vision-
Language Models (VLMs) can integrate aerial discoveries and ground-level verification through
UAV-UGV cooperation. We further provide AGOS-
Dataset as the companion resource of exemplary trajectories constructed by an automatic pipeline. It consists of 7.7k episodes for searching objects of diverse categories and attributes, spanning three difficulty levels. To address the AGOS task, we propose AGOS-
Agent, a training-free and tool-augmented approach. The agentic method relieves VLMs from complex and dynamic coordination via a deliberate search-handoff-verify cooperation protocol, only demanding VLMs for scene understanding and decision-making. Extensive experiments on nine VLMs show that AGOS-
Agent improves overall success rate for eight of the nine evaluated backbones while reducing decision steps for all nine. On the hard split, the SR and SPL of Gemini-3.6-Flash increase from 8.6% to 55.7% and from 7.6% to 44.0%, respectively.
Read the full abstract ▾
All authors Boao Yu, Zimo Chen, Junreng Rao, Yue Hu, Zhengqiu Zhu, Yong Zhao, Rusheng Ju
Artificial Intelligence · Cross-listed from cs.SD
Noise Adaptive Streaming Audio-Visual Speech Token Enhancement for Robust Full-Duplex Spoken Dialogue Models
Bella Godiva, Yeonju Kim and Yong Man Ro
Full-duplex spoken dialogue systems enable simultaneous listening and speaking, but their audio-only perception often fails under background noise and overlapping
speech, leading to incoherent responses. Recent audio-visual dialogue approaches show that incorporating visual cues such as lip movements improve robustness under audio corruption. However, existing approaches often adapt the large
speech dialogue model itself to process visual input, requiring costly
multimodal training. We propose AV-STE, a modular streaming audio-visual front-end that restores corrupted semantic
speech tokens from noisy audio and lip
video before they reach the
speech LLM. The downstream dialogue model remains entirely frozen, preserving its pretrained conversational capabilities. When integrated with frozen Moshi, AV-STE improves average GPT-4o-judged response coherence from 1.42 to 1.91 under same-
dataset speaker interference while largely preserving turn-taking behavior. Gains also transfer to out-of-domain Seamless Interaction.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Equivariance Breaks the Learning Rate
Andrei Manolache and Mathias Niepert
Equivariant networks are commonly trained with Adam, yet recent work reports that
matrix structured optimizers such as
Muon can perform better, with the reasons for these gains only partly understood. We identify one source of this difference inside equivariant layers. An equivariant layer learns one channel mixing
matrix $W_l$ per degree $l$, which we call an irrep block, and shares it across the $2l+1$ components, giving the expanded map $W_l \otimes I_{2l+1}$. This sharing sums gradient contributions across components and can produce different update scales under SGD. Adam's entrywise normalization reduces sensitivity to gradient scale, but neither optimizer directly controls the effective step size of each block. A single learning rate can therefore produce different effective step sizes across blocks.
Muon instead controls the effective step size by approximately equalizing the singular values of each momentum
matrix. We normalize each irrep block update by a single scalar, preserving its singular value ratios while letting the learning rate control its size. We implement this with spectral normalization or a simpler root-mean-square normalization. We evaluate spectral normalization in a controlled $\mathrm{SO}(3)$-equivariant model with a matched non-equivariant model. In this setting, the step size mismatch grows with width in the equivariant model but not in the non-equivariant model. We evaluate both variants across molecular force prediction on the rMD17 and MD22
datasets, QM9 molecular property prediction, and charged particle dynamics. Across these applications, block normalization generally improves Adam and closes part of its gap to
Muon. These results highlight an overlooked interaction between equivariant architectures and their optimizers. Studying and designing the two together may help explain and address training difficulties often attributed to equivariance itself.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
IPM-FM: A Foundation Model with Consensus Feature Selection for Industrial Process Monitoring
Liang Cao, Weide Liu, Yan Qin et al. (+3 more)
Industrial process monitoring is fundamental to the safety and economic performance of modern process plants. Current practice remains a one-task-one-model paradigm that is label-inefficient and prone to degradation under operating drift. Foundation models have reshaped
language,
vision, and generic time-series
forecasting, but it has not been adapted to industrial process monitoring. This setting poses domain-specific challenges, including safety-critical decisions and asymmetric sampling between process variables and laboratory measurements. We propose the industrial process monitoring foundation model (IPM-FM). It first learns general-purpose representations from unlabeled industrial process data through self-supervised pretraining, then adapts to specific monitoring tasks using a small amount of task-labeled data, and finally produces calibrated predictions through an uncertainty-aware prediction head. IPM-FM integrates a self-supervised Informer backbone with a multi-criteria consensus feature selector, a recursive lag-feature regression head, and a calibrated Monte Carlo dropout uncertainty module. On a seven-year hydrotreater
dataset for diesel flash-point soft sensing, IPM-FM attains an RMSE of 2.99, $R^2$ of 0.50, and 97% coverage of its 95% predictive interval, outperforming the strongest classical and from-scratch sequence baselines by 8.3% and 14.6% in RMSE respectively, supporting the viability of a unified pretraining–adaptation framework for industrial process monitoring.
Read the full abstract ▾
All authors Liang Cao, Weide Liu, Yan Qin, Jun Cheng, Weisi Lin, Bhushan Gopaluni
Artificial Intelligence · Cross-listed from cs.CV
Segment Any Motion with Radar: Robust Multimodal Moving-Object Segmentation and Tracking
Jue Wang, Xuan Wang, Hao Zhou et al. (+6 more)
Moving-object perception must decide which image regions correspond to real motion and keep every instance identified over time. Methods that read motion from appearance, optical flow, or estimated trajectories lose that evidence under poor illumination, adverse weather, reflections, and occlusion. Radar is a natural remedy because it measures radial velocity directly instead of inferring it from photometric correspondence. However, existing
benchmarks do not jointly provide radar measurements, dense moving-instance masks, and temporally consistent identities for surveillance. We therefore introduce RGBTR-Motion, a synchronized and calibrated fixed-camera
benchmark that pairs RGB, thermal, and radar streams with dense instance masks and temporally consistent identities across diverse surveillance scenes. We also develop SAM-Radar, an RGB, thermal, and radar-based
segmentation and tracking framework built on SAM 3. SAM-Radar's radar-aware detector fuses calibrated RGBT features with radar returns that are grounded at their projected image locations, and motion supervision, implemented as foreground classification of those projected returns, teaches the detector to reject clutter without any text
prompt. The tracker associates accepted radar returns with individual trajectories and uses them as physical evidence that a visually degraded target remains present. This allows it to bridge short periods of low visibility or occlusion and reconnect a reappearing target to its existing identity instead of starting a new track. SAM-Radar attains 0.7027 IoU and 0.8090 F1-50, and raises MOTA, HOTA, and IDF1 by 0.2977, 0.1603, and 0.2857 over the strongest competing values.
Read the full abstract ▾
All authors Jue Wang, Xuan Wang, Hao Zhou, Ruixiang Zhou, Yixuan Zhou, Tianshuo Yuan, Jieming Ma, Jie Zhang, Fei Luo
Artificial Intelligence · Cross-listed from cs.RO
RoboCousin: Build Your Own Simulation Playground for Robust Bimanual Robotic Manipulation
Jingxuan Zhu, Jingyi Li, LiangLiang Chen et al. (+3 more)
Bimanual manipulation policies require large and diverse training
datasets, yet collecting demonstrations on physical
robots is expensive and difficult to scale. Simulation can generate data efficiently, but existing pipelines typically operate within closed asset libraries and predefined scenes: adding a newly observed object or environment still requires substantial effort to reconstruct geometry, specify physical and semantic properties, annotate interactions, and integrate the result into executable tasks. We present RoboCousin, an extensible simulation-based data-generation platform that turns user-provided observations into reusable assets, scenes, and expert trajectories for bimanual manipulation. Built on RoboTwin~2.0, RoboCousin converts object images into simulation-ready assets with visual and collision geometry, semantic and physical metadata, and automatically generated grasp-contact candidates. It further constructs digital cousins that vary compatible objects, backgrounds, layouts, and
language instructions while preserving task-relevant affordances and spatial relations. The same asset system supports tabletop and room-level scene construction, with collision-aware base control for interaction beyond a fixed workspace. We release RoboCousin-OBD, containing more than 3,000 annotated object instances and 50 background environments, and use RoboCousin to generate over one million expert trajectories across 50 tasks. Simulation and real-
robot experiments show that the automatically generated interaction annotations are comparable to curated annotations, generated assets provide effective sim-to-real supervision, and tabletop cousins can improve transfer beyond training on a single reconstructed scene. RoboCousin therefore provides a practical path for expanding both the scale and coverage of synthetic bimanual manipulation data.
Read the full abstract ▾
All authors Jingxuan Zhu, Jingyi Li, LiangLiang Chen, Zhiyuan Jing, Jidong Zhang, Hongming Li
Artificial Intelligence · Cross-listed from cs.RO
A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing
Davide Malvezzi, Michele Pestarino, Vittoria Cavicchioli et al. (+9 more)
Object detection and tracking are fundamental components of perception systems for autonomous driving. Achieving robust performance under adverse conditions such as limited visibility, sensor noise, and failures remains an open challenge, particularly in autonomous racing, where vehicles operate at very high speeds, experience strong vibrations, and interact under small safety margins. This paper presents a multi-modal late-fusion perception pipeline for object detection and tracking in the autonomous racing domain. The proposed system extends previous work by exploiting all onboard sensors through a late-fusion approach and a dedicated multi-object tracking framework. Independent detections from cameras,
LiDARs, and RADARs are combined to provide timely and robust state estimates of surrounding vehicles. The tracking method explicitly compensates for detection delays and embeds in its model prior knowledge of vehicle dynamics and track layout. Experimental evaluation on real-world data across diverse critical scenarios, representative of challenging edge cases also in urban driving, confirms the effectiveness of the proposed pipeline and its suitability to support safe and adaptive planning decisions.
Read the full abstract ▾
All authors Davide Malvezzi, Michele Pestarino, Vittoria Cavicchioli, Valentina La Gamba, Silvia Severi, Fabio Bagni, Luca Bartoli, Massimiliano Bosi, Francesco Gatti, Micaela Verucchi, Ayoub Raji, Marko Bertogna
Artificial Intelligence · Cross-listed from cs.CL
Tracing Stereotypes from Representation to Output in Multilingual LLMs
Ariun-Erdene Tumurchuluun, Yusser Al Ghussin, Pinzhen Chen et al. (+2 more)
Multilingual LLMs show stereotype-related behavior that varies across
languages, but behavioral scores do not show where the relevant information is represented or how it affects the output. To investigate these internal mechanisms, we compare linear probing, attribution patching, sparse autoencoders (SAEs) and feature ablation in Llama-3.1-8B, Qwen3-8B, and Gemma-2-9B. Probe performance peaks substantially earlier than attribution in all three models, with a separation of 36-53% of model depth. Retained Llama-Scope features often match the social category on which they were selected and form recurring semantic families, but their lexical
alignment and ablation effects vary across SAE suites. Only 6-18% of evaluated residual-stream features have
language-agnostic effects under our criterion, and none are category-agnostic.
Language-agnostic features have larger mean ablation effects in Llama-Scope, but this pattern does not repeat in the other SAE suites. Decodability, output influence, and cross-lingual ablation effects therefore need to be measured separately.
Read the full abstract ▾
All authors Ariun-Erdene Tumurchuluun, Yusser Al Ghussin, Pinzhen Chen, Josef van Genabith, Koel Dutta Chowdhury
Artificial Intelligence · Cross-listed from cs.SE
AttnCompress: Dynamic Attention-Guided Trajectory Compression for Software Engineering Agents
Zhengran Zeng, Yixin Li, Rui Xie et al. (+2 more)
The transition from human-centric assistance to Autonomous Software Engineering (ASE)
agents has enabled the resolution of complex real-world SE tasks. However, the trial-and-error nature of these
agents generates lengthy interaction trajectories, creating severe bottlenecks in terms of context window limits and cost. While context compression offers a potential remedy, prior approaches suffer from static
pruning strategies and granularity mismatches, often failing to preserve the semantic dependencies and syntactic details crucial for SE tasks. To strictly preserve critical task evidence while reducing context length, we introduce AttnCompress, a dynamic
attention-guided trajectory compression framework. Unlike existing approaches, AttnCompress bridges the gap between semantic integrity and dynamic adaptability through three key mechanisms: (1) structure-aware
segmentation via perplexity (PPL) spikes to preserve the syntactic structure of code and logs; (2) relevance estimation using proxy
attention weights to quantify the precise relevance of historical blocks to the
agent's current
reasoning; and (3) a dynamic rolling window to re-evaluate and recall historical context as the task evolves. Extensive evaluation on SWE-Bench-Verified and Multi-SWE-Bench demonstrates that AttnCompress achieves a pass rate of 53.17%, outperforming prior state-of-the-art baselines while reducing
token consumption by 21.6% and total costs by 33.6%. The framework proves to be model-agnostic and generalizes effectively across diverse programming
languages.
Read the full abstract ▾
All authors Zhengran Zeng, Yixin Li, Rui Xie, Wei Ye, Shikun Zhang
Artificial Intelligence · Cross-listed from cs.IR
Exploring Bottom-Up Clustering for Creating Semantic IDs
Leah Woldemariam, Sudhanshu Garg, Taha Belkhouja et al. (+2 more)
The success of generative
retrieval has largely been attributed to the use of Semantic IDs, which improve over arbitrary item-level identifiers such as hashes by capturing the semantics of items. The main challenges faced when constructing Semantic IDs, however, is in mapping each identifier to a unique product and capturing information valuable to downstream tasks. Past works have appended additional codewords to de-duplicate item identifiers and utilized residual quantization to create hierarchical clusters. In this work, we present an
algorithm for generating Semantic IDs that ensure the identifiers are both unique and preserve the structure of the original
embedding. Key to our work is the use of bottom-up clustering to preserve local structure in the
embedding space, improving the clustering quality of the resulting Semantic IDs and their utility for downstream generative
retrieval.
Read the full abstract ▾
All authors Leah Woldemariam, Sudhanshu Garg, Taha Belkhouja, Charles Kim-Yip, Ali Sahami
Artificial Intelligence · Cross-listed from cs.DC
A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware
Maysam Khatib, Moysis Symeonides, Demetris Trihinas et al. (+2 more)
Large
language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires balancing quality, latency, model footprint, and energy. This paper presents a controlled measurement study of self-hosted LLM inference across edge and near-edge deployment nodes: an NVIDIA Jetson AGX Orin and a near-edge server with CPU-only and GPU-enabled inference modes. We evaluate multiple open-weight LLMs and quantization variants using a fixed question-answering workload, and compare them against GPT-4o as a cloud-hosted accuracy and latency reference. Our benchmarking pipeline reports accuracy, model footprint, per-
token decoding latency, prefill latency, and overall execution energy. The results show that GPU-enabled server execution provides the lowest compute-side latency, while Jetson Orin shows lower measured energy, consistent with its lower platform power under our setup. CPU-only execution is consistently dominated in latency for our workload and shows higher measured energy. We also show that parameter count and downloaded weight-file size alone do not reliably predict observed accuracy or latency. Finally, using Pareto-frontier analysis, we study how deployment decisions may change under possible streamed-
token delivery overheads, highlighting that compute-side inference metrics alone can lead to suboptimal placement for latency-sensitive interactive web services.
Read the full abstract ▾
All authors Maysam Khatib, Moysis Symeonides, Demetris Trihinas, George Pallis, Marios D. Dikaiakos
Artificial Intelligence · Cross-listed from cs.CV
FPicker: Topology-Guided Evolution for Filament Tracing in Low-SNR Microscopy
Tingyin Zhao, Mingtao Huang and Yuan Shen
Automating filament tracing in Cryo-Electron Microscopy (Cryo-EM) is essential for 3D helical reconstruction but challenged by intersecting
topologies and extremely low Signal-to-Noise Ratios ($\text{SNR} = \sigma_s^2/\sigma_n^2$ < 0.1 or -10 dB). Existing paradigms fail: pixel-wise segmenters suffer from severe topological fracturing, box-based detectors face ghost center drift, sequential trackers derail due to error accumulation, and traditional active contours collapse under artificial closed-curve constraints. To resolve these bottlenecks, we present FPicker, the first
topology-guided framework reconciling these incompatibilities. It unifies perception via a center-endpoint representation and an open-curve
evolution module to explicitly model non-cyclic connectivity. On simulated
benchmarks, FPicker outperforms top baselines by over $40\%$ relative gain in mean spatio-angular precision (mSAP) and reduces topological gap rates by over $60\%$ under extreme noise ($-20\text{ dB}$). By learning intrinsic physical geometry rather than local texture, FPicker demonstrates strong potential as a resilient
geometric backbone. Its zero-shot performance on the real-world EMPIAR
dataset exhibits robust topological resistance, achieving a state-of-the-art 82.9% mSAP upon fine-tuning. Our results also suggest modeling physical priors is a highly robust path toward bridging the sim-to-real gap in signal-starved scientific imaging. The code is publicly available at: https://github.com/tomzhaosky/FPicker.
Read the full abstract ▾
Artificial Intelligence
LEBGen: An LLM-Enhanced Bayesian Network Framework for Few-Shot Travel Survey Data Generation
Zijian Shen, Bin Zhou, Jiguang Wang et al. (+2 more)
Travel survey data are essential for transportation planning and travel behavior analysis, yet collecting large-scale representative samples is costly and time-consuming. A practical alternative is to generate synthetic survey records from a few-shot sample. However, such samples provide incomplete coverage of heterogeneous traveler groups and insufficient evidence for recovering the complex dependencies between demographic characteristics and travel behavior. Existing approaches have complementary limitations. Probabilistic generative models such as
Bayesian networks (BNs) offer explicit distributional control, but structures learned from few-shot samples may omit meaningful dependencies or retain spurious ones. Large
language models (LLMs) can help address these difficulties in BN structure learning by providing behavioral knowledge that complements the limited statistical evidence. We therefore propose LEBGen, an LLM-enhanced BN framework that uses this knowledge to refine network structure for few-shot travel survey data generation. Specifically, the LLM first identifies traveler personas from demographic attribute and travel behavior statistics, then recovers dependencies missed by the persona-augmented BN structure and prune spurious ones. The refined BN is parameterized exclusively from the observed data to generate synthetic records. Under a 2% few-shot setting on the 2022 Hong Kong Travel Characteristics Survey, LEBGen reduces the mean marginal Jensen-Shannon divergence from 0.0671 to 0.0091 and the mean absolute Cramer's V error by 14.3% over the best-performing baseline, substantially improving both distributional and dependency fidelity.
Read the full abstract ▾
All authors Zijian Shen, Bin Zhou, Jiguang Wang, Ya Zhao, Jintao Ke
Artificial Intelligence · Cross-listed from cs.CL
What Eviction Destroys: A Restore-Counterfactual Audit of Forgetting in Agent Memory
Chen Shen
Agent memory systems must discard stored information when their history exceeds a fixed
token budget. Existing budget-accuracy frontiers quantify the resulting loss in accuracy, but do not distinguish irreversible losses caused by eviction from recoverable
retrieval failures. We introduce the restore counterfactual, a per-question paired intervention that reinstates the question's gold evidence in the read-time context and reruns the same reader. Combining the change in correctness with whether the evidence was retained after eviction classifies each oracle-answerable error as recoverable, irreversible, or residual; in the residual case, the answer remains incorrect after restoration. We evaluate FIFO, random, redundancy-aware, and LLM-importance eviction on LongMemEval-S at three budgets and under two
retrieval regimes, using GPT-4o-mini as the primary reader and judge and GPT-5.4-mini as a robustness reader. Under top-k
retrieval at an 80k-
token budget, the irreversible share among errors corrected by restoration is 0.67-0.73 for FIFO, random, and redundancy-aware eviction, compared with 0.60 for LLM-importance. At 8k
tokens, it reaches 1.00 for all four policies. Recoverable errors occur under top-k
retrieval at 80k
tokens but are absent under forced-gold injection by construction, so budget-accuracy results are not directly comparable unless the
retrieval regime is reported. An exploratory matched-accuracy analysis detects no difference in irreversible rate among accuracy-matched policy pairs at a resolution of 1.2-6 percentage points. The same analysis detects the deliberately destructive control. To our knowledge, this is the first per-item, per-question restore-counterfactual audit of eviction for external
agent-
memory stores on a standard conversational
benchmark.
Read the full abstract ▾
Artificial Intelligence
Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation
Tianyi Zeng, Junchao Liao, Yujie Wei et al. (+9 more)
Recent multi-shot audio-
video generators can produce increasingly coherent and cinematic outputs, but coherence does not imply the ability to execute editing techniques. Professional editing depends on shot structure, transition grammar, audio-
video cut relations, and montage, yet existing
benchmarks largely rely on proxies such as content quality, synchronization, or physical plausibility, systematically missing whether such editing instructions are actually executed. We introduce CutCraft, the first
benchmark for editing-technique execution in multi-shot audio-
video generation. CutCraft extends structured multi-shot
prompts with explicit editing specifications and is paired with a hierarchical hybrid evaluation framework that combines shot-structure
alignment, expert-model metrics, tool-grounded
multimodal judgment, and rubric-based question answering. Beyond evaluation, we design an agentic editing baseline that decomposes generation into planning, shot-level synthesis, and post-hoc composition, explicitly realizing editing semantics such as J-cuts, L-cuts, and transition timing. Across 13 state-of-the-art closed- and open-source models, CutCraft reveals a consistent gap between coherence and editing-technique execution: current systems often produce plausible multi-shot
videos yet fail to execute editorial instructions reliably. We find unstable shot structures, weak control of transition execution, and sharp degradation on higher-order montage, while aesthetic quality is only weakly correlated with editing-technique compliance. The
benchmark and metrics, and the editing
agent baseline are available at https://github.com/AlibabaResearch/cut-craft-bench.
Read the full abstract ▾
All authors Tianyi Zeng, Junchao Liao, Yujie Wei, Ziying Zhang, Litao Li, Tianyi Wang, Zhichao Wei, Shuyao Xu, Wenwen Qiang, Siyu Zhu, Zhenghao Zhang, Long Qin
Artificial Intelligence
MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging
Junxi Wang, Te Sun, Jiayi Zhu et al. (+9 more)
Agent memory systems have demonstrated significant potential in long-term dialogue, personalized assistants, and
video understanding. However, continuously accumulated
memory introduces substantial storage and
retrieval costs during inference. To address this issue, we propose MemForest, a general
memory compression framework adaptable to various
agent memory systems. Specifically, MemForest partitions historical
memory into event-centric units by leveraging global semantic similarity and local temporal continuity. For each unit, it constructs a maximum spanning tree, termed an EventTree, and progressively merges redundant
memory nodes by selecting high-weight edges, reducing storage overhead. Furthermore, we introduce an anchor-guided propagation
retrieval mechanism that retrieves relevant
memory nodes from the temporal neighborhoods of key nodes, improving
retrieval accuracy. Extensive experiments demonstrate the effectiveness of MemForest. Under the unimodal Mem0 framework, MemForest retains 97.1% of the original performance while compressing 50% of historical
memory across three
benchmarks (LoCoMo, LongMemEval, and PersonaMem), achieving a 1.89x
retrieval speedup. Under the
multimodal M3-
Agent framework, it preserves 99.7% of the original performance with a 50% compression ratio across two
benchmarks (M3-Bench-
robot and M3-Bench-web), achieving a 2.24x
retrieval speedup. \textcolor{RoyalBlue}{Our code is available at [https://github.com/Celina-love-sweet/MemForest.}](https://github.com/Celina-love-sweet/MemForest.}})
Read the full abstract ▾
All authors Junxi Wang, Te Sun, Jiayi Zhu, Chen Zhang, Siyuan Li, Xuyang Liu, Zichen Wen, Xiaobing Tu, Jinkui Ren, Xiantao Zhang, Ziqi Yuan, Linfeng Zhang
Artificial Intelligence
Three Types of Negation of Triple and its Elements and an Extension of Triple
Zhenghua Pan
In various data models, the classical triple is a typical semantic data model. However, due to the design of the triple as a simple structure for representing positive assertions, it cannot sufficiently express different forms of negation present in the triple and its elements. This paper conceptually proposes that there are three distinct forms of negation within triples and their elements: contradictory negation, opposite negation and intermediary negation. Based on the the set SCOI and the logic LCOI+PLCOI with three kinds of negation, we propose an extension of triple that can distinguish and express these three different negations in the triple and its elements, called the TCOI triple with contradictory negation, opposite negation and intermediary negation. The TCOI triple is a semantic and structural extension of the classical triple. While retaining the ability to express positive assertions, it systematically introduces the three semantic dimensions of three negations, allowing these negations to independently act on the elements of the triple and on the whole triple. This significantly enhances the triple model capability to represent and
reasoning about complex negative information. This paper also explores the expressive power and
reasoning of the TCOI triple, as well as the application of TCOI triple implication
reasoning in counterfactuals and counterfactual
reasoning. We propose a truth-value (continuous value)
algorithm for TCOI triple implication
reasoning and perform its calculation through an example of the counterfactuals and counterfactual
reasoning.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Synergistic Fusion of Topological Structure and Temporal Semantics of Mobility for Urban Region Embedding
Namwoo Kim, Jeeyun Chang, Kanghoon Lee et al. (+1 more)
Urban region
embeddings have shown promising results in diverse urban sensing tasks such as crime, income, and service-call prediction. Recent methods improve representation quality by integrating mobility data with auxiliary modalities, using cross-view
attention or contrastive objectives to align heterogeneous features into a unified region representation. However, leveraging the temporal dynamics of human mobility remains under-explored. Regional inflow and
outflow fluctuate throughout the day, and inter-region connections emerge, persist, and dissolve over time. Moreover, prevailing fusion strategies combine views additively and miss the joint signal that emerges only when views co-occur. To address these gaps, we propose Mobility Stream-Structure Synergy (MoSS), which derives complementary views from mobility data: a Sequence view that preserves each region's hourly inflow/
outflow profile, and a Structure view based on zigzag persistence diagrams that capture how regional connectivity emerges, persists, and dissolves over time. A synergy module then extracts emergent representations from the co-occurrence of these views through multi-degree interactions, explicitly capturing higher-order signal across views. Extensive experiments on New York City and Chicago show that MoSS achieves state-of-the-art performance across three downstream tasks using mobility data alone, outperforming baselines that rely on auxiliary modalities.
Read the full abstract ▾
All authors Namwoo Kim, Jeeyun Chang, Kanghoon Lee, Yoonjin Yoon