Computer Vision
Multi-Scale Semantic Mapping in Urban Environments via Observation Calibration and Policy Dependence Regularization
Runling Long, Junhao Feng and Jia Wan
Semantic mapping is fundamental to
embodied navigation, yet existing methods are developed for indoor environments, where objects exhibit relatively limited scale variation and are observed from a restricted range of viewpoints. Urban environments pose substantially greater challenges:
agents must map objects ranging from pedestrians to buildings while navigating large spaces with highly diverse viewing distances. These conditions introduce two key difficulties that existing
datasets and methods fail to cover. First, object scale and observation distance can be severely mismatched. For example, small objects may be viewed from far away, whereas large objects may be observed at extremely close range, resulting in unreliable observation likelihoods. Second, objects with substantially different sizes and geometries require distinct mapping behaviors, which are difficult to capture with a single shared value
estimator. To investigate these challenges, we introduce a large-scale urban semantic mapping
dataset featuring realistic city layouts, high-fidelity rendering, and instance-level annotations spanning multiple object scales. We then propose a category-aware likelihood calibration policy that identifies and alleviates unreliable observations according to object category and viewing distance. Because the calibration and motion policies are optimized toward the same mapping objective, they may learn redundant shortcuts and become excessively coupled. We therefore introduce a mutual-information (MI) regularizer that penalizes their estimated representation dependence and encourages complementary behaviors. To better model heterogeneous mapping strategies across object scales, we further employ category-wise value
estimators. We formulate their joint optimization as a Pareto optimization problem to mitigate conflicting gradients across categories.
Read the full abstract ▾
Artificial Intelligence
BV Loss: Block Verification-Aware Loss for Block Diffusion Speculative Decoding
Suyoung Kim, Jahyun Koo, Hyeonjin Kim et al. (+5 more)
Diffusion drafters accelerate speculative decoding by proposing multiple
tokens in parallel. Despite recent advances in speculative decoding through sequence-level drafting and verification, existing training objectives remain largely designed around
token-level verification. To address this mismatch, we introduce Block Verification-aware loss (BV loss), a training objective designed to maximize the expected acceptance length of a drafted sequence. BV loss is directly derived from the block verification acceptance rule, providing a principled connection between the drafter training objective and the inference-time verification mechanism at the sequence level. Across math, code, and chat
benchmarks, BV loss increases the mean number of
tokens accepted per verification call under block verification by 13.0–21.0% over cross-
entropy loss training for DFlash and DSpark with Qwen3-4B and Qwen3-8B without changing the inference procedure. BV loss also outperforms tokenwise acceptance objectives such as TV loss and LK loss, and its gains extend to
token verification and greedy decoding. These results demonstrate the benefit of training block
diffusion drafters with an objective aligned with sequence-level verification, rather than optimizing each
token independently.
Read the full abstract ▾
All authors Suyoung Kim, Jahyun Koo, Hyeonjin Kim, Inhyeok Bang, Seunghyun Lee, Hyunjae Oh, Baeseong Park, Dongsoo Lee
Machine Learning
Structured Neural SDEs for Functional Calibration
Francesco Piatti, Andrea Iannucci and Thomas Cass
Neural
Stochastic Differential Equations (Neural SDEs) provide flexible continuous-time generative models, but generic neural drift and
diffusion networks are costly to simulate on long horizons and can give unstable gradients when the training signal is a path functional rather than a pointwise observation. We introduce SLiSDE, a family of Neural SDE models built from structured linear
stochastic layers. Parallel-in-time simulation is obtained at the layer level, while expressivity is recovered by gated in-flow stacking: previous-layer paths modulate the next layer's latent flow through learned gates. For functional calibration tasks in which rare paths dominate the loss, we add an optional Girsanov tilt that acts as a learned importance sampler with an exact likelihood-ratio correction. We prove well-posedness, a discretisation error bound, validity of the change of measure, and a universality result: the terminal laws of the gated stack are dense in the space of square-integrable laws. Experiments on functional calibration
benchmarks show that the structured model outperforms fully neural SDE baselines while retaining parallel-time simulation and stable importance weights.
Read the full abstract ▾
Computation and Language
From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Construction
Zixiao Dong, Wei Yang, Zihao Liu et al. (+4 more)
Most methods that optimize LLM
prompts and
agent workflows assume that task-specific output schemas, extraction instructions, and evaluation criteria are predefined. For scientific extraction
agents, however, a short task goal may not fully determine these components, while specifying them manually is costly. We study the upstream problem of constructing the task-specific configuration from a weak specification containing only a short goal and unannotated reference documents. Rather than treating automatic construction as a fixed preprocessing step, our framework constructs a task-specific schema, extraction instructions, and base training rubrics, then keeps schema construction and extraction instructions editable during optimization. Failure-focused updates concentrate textual-gradient feedback on lower-scoring documents, while training-time evaluation criteria adapt to recurring failures. On a heterogeneous-catalysis literature corpus, automatic construction remains improvable, and optimizing both schema construction and extraction instructions performs best across all four judge-rubric settings, with ablations and blinded human evaluation supporting the proposed formulation.
Read the full abstract ▾
All authors Zixiao Dong, Wei Yang, Zihao Liu, Chenshu Li, Longzhang Liu, Tao Tan, Hong Xie
Artificial Intelligence
Simulating Respondents, Not Single Questions: Coherent Survey Generation with Large Language Models
Ji Huang, Mengfei Li and Shuai Shao
Large
language models are increasingly used to simulate response distributions in social surveys. Prior work has achieved accurate population-level simulation for individual questions. Real questionnaires, however, ask each respondent a sequence of related questions. A simulated respondent should show coherent preferences across the whole questionnaire, not merely accurate distributions for isolated items. Existing single-item methods cannot accurately reproduce how the same person answers a complete survey. We propose FullRespondent-LLM (FR-LLM), which fine-tunes two specialized LLMs: a marginal model for each item's response distribution and a respondent-level autoregressive model for dependencies across answers. Marginal-Constrained Joint Projection (MCJP) then projects the autoregressive joint distribution onto the set satisfying the item-level marginals learned by the first model. This yields complete questionnaires with realistic cross-item relationships while retaining strong item-level accuracy. On two real-world social survey
datasets, FR-LLM more accurately reproduces multi-question response patterns, maintains competitive single-item accuracy, and generalizes better to unseen populations and questions. In a small commercial-survey
dataset, we use simulated responses to make pricing and stocking decisions; FR-LLM achieves the highest realized profit.
Read the full abstract ▾
Computer Vision
WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
Yuheng Zha, Yilei Wang, Qiyue Gao et al. (+5 more)
Humans often solve spatial problems by mentally simulating visual transformations. In contrast, conventional
vision-
language models (VLMs) reason primarily through
language. We investigate whether VLMs can solve spatial problems by
reasoning with both text and generated visual states. To this end, we introduce WM-VLM, which equips a pretrained VLM with a lightweight world model branch for generating intermediate visual states. Our two-stage training first teaches the model to generate the next visual state and then to use that state for
reasoning. We programmatically construct spatial
reasoning tasks with verifiable intermediate visual states. These tasks allow us to evaluate how well the model generates visual states and how much it relies on them to answer the question. On 2D and 3D mental rotation tasks, WM-VLM consistently outperforms the supervised fine-tuned backbone, with gains of up to 39.25 percentage points. Ablations suggest that these gains depend on the generated visual states, as removing or corrupting them sharply reduces performance. Together, these results suggest that internal world models offer a promising path toward VLMs that reason in both
language and visual space. Code is publicly available at: https://github.com/yuh-zha/WM-VLM
Read the full abstract ▾
All authors Yuheng Zha, Yilei Wang, Qiyue Gao, Junrong Chen, Yujia Wu, Zhengfeng Lai, Zhengzhong Liu, Eric P. Xing
Machine Learning
Gaussian Neural Networks
Peter Kuhn and Victoria Heusinger-Heß
Gaussian neural networks (GaNNs) are proposed as a novel regularization mechanism for neural networks. From a
Bayesian perspective standard regularization techniques can be viewed as imposing priors over weight-space. Assuming priors over activation-space remains a largely unexplored possibility. GaNNs assume such priors. They do this by treating activities from earlier layers like signals with Gaussian noise and predicting the properties of the noise distribution using an additional unsupervised loss. While training, the unsupervised loss acts as a penalty on unexpected activities, allowing greater weight updates in less surprising directions. The paper demonstrates the superiority of Gaussian neural networks over standard neural networks on a variety of classification and regression tasks. We also investigate the ability of GaNNs to quantify uncertainty.
Read the full abstract ▾
Computer Vision
Generative Residual Factorization
Letian Gong and Yuzhou Hong
Under a shared-factor model, the conditional law of the next image patch factors into a posterior over the shared scene factor and a residual kernel given that factor. A sufficient statistic of the past replaces the raw past in the posterior and does not replace the kernel. The conditional
entropy splits into residual
entropy, which no observation of the factor can remove, and a posterior term, which a better representation of the past can remove. Next-
embedding prediction is a directional likelihood on a shallow map, so the fiber of that map is unidentified and a constant
embedding remains a minimizer. The same split is an equality in a scalar Gaussian model, evaluated in closed form.
Read the full abstract ▾
Robotics
AGRO-SUVIDE: Agentic Robotics for Surgical Viscoelastic Debridement
Shutong Jin, Ziyang Chen, Preethi Satish et al. (+4 more)
Augmented dexterity has the potential to reduce the fatigue experienced by surgeons during repetitive
surgical tasks. In this paper, we propose the first AGentic RObotics framework for
SUrgical VIscoelastic DEbridement (AGRO-SUVIDE), the repeated removal of small fragments attached to a viscoelastic substrate. Leveraging the self-improving and coding capability of
agents, AGRO-SUVIDE adopts a modular framework. Specifically, the demonstration analysis module automatically identifies recurring skills from a single expert demonstration, using both visual and kinematic information. The construction module then builds each skill, either as a procedural model-based skill the
agent codes against a scaffolded library or as a model-free policy-based skill. At runtime, the monitoring module composes the skills into a loop-style graph sized to the number of fragments it observes, then verifies pre- and post-conditions of each skill to decide whether to advance or retry. We evaluate AGRO-SUVIDE through 340 physical trials on the da Vinci Research Kit (dVRK). AGRO-SUVIDE achieves an average single-fragment removal success rate of 85%, completing consecutive three-fragment removal at 60% and at 95% with one human intervention. It further generalizes to unseen five-fragment scenarios with an average success rate of 80% for single-fragment removal. Project page: https://
surgical-robotics.github.io/AGRO-SUVIDE/
Read the full abstract ▾
All authors Shutong Jin, Ziyang Chen, Preethi Satish, Meadow Shen, Gary Guthart, Florian T. Pokorny, Ken Goldberg
Multiagent Systems
CEO Arena: Evaluating Long-Horizon Multi-Agent Decision-Making in Competitive Markets
An Yan, Yu Huo, Zhiwei Shang et al. (+2 more)
Long-horizon competition tests
agents' ability to coordinate business decisions under uncertainty and adapt to changing rival strategies. We introduce CEO Arena, a
benchmark that uses matched replacement evaluation to assess operating returns alongside an
agent's effects on rivals and the
market. Each CEO
agent is compared with a reference policy in the same company under the same economic seed, holding other
agents' identities and assignments fixed while all
agents adapt. In a shared eight-company
market spanning 500 simulated days, CEOs make sequential decisions on pricing, procurement, marketing, research and development, and service using private company information and noisy
market signals, under resource constraints and delayed feedback. We evaluate eight LLM-based CEO
agents in 27 main runs and 26 robustness runs. In the main evaluation, most
agents have negative mean returns, and private gains can accompany
market losses. Robustness analyses suggest that aggregate patterns extend beyond the original rule-based baseline; four of the 56 directed pairs show relatively stable effects.
Memory, action, and accounting traces suggest demand capture and rivals' pricing and spending responses as possible explanations. CEO Arena provides a controlled testbed for studying long-horizon
agent competition, strategic interaction, and
market externalities.
Read the full abstract ▾
All authors An Yan, Yu Huo, Zhiwei Shang, Yiran Peng, Chenglin Wu
Distributed Computing
Accelerator Choice Is Not Enough: AlphaFold2 Inference on Cloud TPUs
Lorenzo Pazienza and Ihab El Bani
AlphaFold2 is written in JAX, so the same inference code compiles and runs unchanged on CPUs, GPUs and Google Cloud TPUs. That portability makes the accelerator look like the main decision a user has to make. We show that it is not. Running one AlphaFold2 inference workload across a Colab CPU runtime, an NVIDIA T4 GPU and a dedicated eight-chip Cloud TPU v5e slice, we find a large hardware advantage for the TPU, 0.47 s per call in steady state on a single chip against 13.1 s on the T4 in the same measurement campaign, and three ways in which the software layer decides how much of it a user actually gets. The default execution path uses one chip of the eight, and at list prices the idle capacity makes the slice cost about as much per prediction as the GPU. Batching with JAX's vmap never exceeds single-query throughput, while mapping queries across chips with JAX's pmap gives eight chips 6.5-7.9x the throughput of one on a matched grid; automatic sharding leaves the per-chip footprint unchanged, consistent with replication, most plausibly because AlphaFold2 carries no sharding annotations. Our retained trace analysis of a first call at a new input shape reports about three quarters of the traced span in JAX tracing and compilation rather than execution. Reruns five weeks later reproduced neither cloud baseline, the GPU one off by roughly a factor of two, so the hardware ratio above is specific to one campaign.
Read the full abstract ▾
Computer Vision
ESTHER: Egocentric Stereo Hand Estimation and Reconstruction in the Wild
Hongyu Ma, Hairong Qu, Shiqi Zhao et al. (+2 more)
Human dexterity is guided by two eyes watching two hands: binocular
vision supplies the metric 3D structure that fine-grained manipulation consumes.
Egocentric stereo is therefore the natural perceptual interface for
robots, AR, and VR-yet metric 3D hand reconstruction from this very signal still has neither an end-to-end model nor an in-the-wild
benchmark. We propose ESTHER, a model whose stereo geometry, temporal
reasoning, and output representation are designed for wearable
egocentric stereo. It is trained on pseudo-labels from a calibrated labeling pipeline and in turn assembles our
benchmark ESTHER3D, an
egocentric stereo hand
dataset pairing a large in-the-wild training set of model-generated labels with a motion capture test set of true metric ground truth. Experiments show state-of-the-art accu?racy, superior external generalization, and robustness to the missing views, dropped frames, and lighting and motion blur extremes of real
egocentric capture that break existing meth?ods. This robustness runs deeper than graceful degradation: stereo guidance teaches the model to bind apparent hand scale to metric depth, so it not only adapts to different stereo rigs and modalities with minimal fine-tuning, but more strikingly preserves true metric scale even after collapsing to a single
monocular view.
Read the full abstract ▾
All authors Hongyu Ma, Hairong Qu, Shiqi Zhao, Yongsong Yang, Peng Yin
Distributed Computing
WaveAlign: Cache-Aware Query-Row Scheduling for Sparse Attention in Long-Video Generation
Zijian Dai, Sen Han, Youhui Bai et al. (+6 more)
Long-
video generation with
diffusion transformers (DiTs) produces extremely long
token sequences, making
attention a dominant inference bottleneck. Dynamic sparse
attention reduces computation, but its realized speedup remains limited because irregular query-row execution degrades L2 cache locality and increases HBM traffic. We present WaveAlign, a lightweight, cache-aware query-row reordering framework for dynamic sparse
attention. WaveAlign formulates row ordering as an optimization problem and approximates it with two stages. The first stage derives a low-rank SVD representation of sparse-mask rows and groups query rows with similar K/V access patterns, increasing K/V overlap among concurrently scheduled rows. The second stage exploits streaming GPU scheduling by sorting rows within each wave in descending order of their K/V-block counts, so that short rows from the current wave are followed by long rows from the next. This aligns K/V accesses across wave boundaries and enables shared blocks to be reused before eviction. An adaptive skip module avoids unprofitable reordering. By only permuting query and mask rows, WaveAlign preserves sparse-
attention semantics and requires no changes to existing methods or backend kernels. Across two GPU architectures, two
video DiTs, and four sparse-
attention methods, WaveAlign raises the L2 cache hit ratio from 28.48%–36.35% to 79.38%–89.06%, reduces HBM read traffic by up to 92.11%, and achieves up to 1.25x kernel and 1.17x end-to-end generation speedup without quality loss.
Read the full abstract ▾
All authors Zijian Dai, Sen Han, Youhui Bai, Shannon Wang, Kan Wu, Jingkai Huang, Yuhang Wang, Jing Li, Cheng Li
Machine Learning
Polylogarithmic Nash Regret in Matrix Games with Bandit Feedback
Yuheng Zhang
We study Nash regret minimization in unknown finite
matrix games with bandit payoff feedback and observed opponent actions. We develop Optimistic Payoff Balancing (OPB), which achieves instance-dependent $\mathcal{O}(\log^2 T)$ Nash regret against arbitrary adaptive opponents, including games with nonunique
equilibria. This resolves the open problem posed by Maiti et al. (2025), extending their polylogarithmic guarantee under bandit feedback from $2\times2$ games to arbitrary finite dimensions. To handle nonunique
equilibria, we construct a reference strategy that leaves room for local adjustments. We order independent payoff differences by estimation accuracy and scale these adjustments by uncertainty, allowing the learner to exploit the opponent's imbalance to offset estimation costs. Our result thus shows that observing opponent actions suffices for polylogarithmic Nash regret in general finite
matrix games.
Read the full abstract ▾
Artificial Intelligence
UniOPSD: Unifying Outcome and Hindsight Feedback for Agentic Reinforcement Learning
Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai et al. (+8 more)
Reinforcement learning has become an effective approach to training
language model
agents, but sparse and delayed outcome rewards provide limited guidance for credit assignment across long interaction sequences. Recent work on on-policy self-
distillation (OPSD) offers complementary supervision by evaluating a policy's sampled responses under privileged training-time context. However, our diagnostics show that positive average agreement between outcome and hindsight feedback coexists with substantial local disagreement, raising the question of how to allocate influence between them at each decision. We introduce UniOPSD (Unified On-Policy Self-
Distillation), which unifies these feedback sources through adaptive local credit arbitration. UniOPSD constructs comparable credit estimates from environmental returns and successful-peer hindsight at shared interaction anchors. Historical agreement determines the global mixing level, while current signal availability and relative precision adjust each source's influence at individual decisions. The episode-level outcome contribution is retained, and bounded
token modulation refines the fused step credit for policy optimization. With Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct, UniOPSD achieves ALFWorld success rates of $82.8\%$ and $83.6\%$, WebShop success rates of $75.0\%$ and $82.0\%$, and Search-QA aggregate accuracies of $45.3\%$ and $49.8\%$, respectively. On 3B WebShop, UniOPSD improves over SDAR by $7.0$ percentage points. Our code is available at https://github.com/Zenghuang-Fu/Uniopsd
Read the full abstract ▾
All authors Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai, Xiaofeng Han, Zelong Zheng, Haoyu Wu, Tianyu Fu, Chenxu Zhao, Minghui Wu, Guannan He, Changwei Wang
Computer Vision
From Perception to Integration: Revisiting the Internal Dynamics of Reasoning in Vision-Language Models
Rong Yu Xu, Prayag Tiwari and Shaolei Zhang
Vision-
language models (VLMs) can answer simple visual questions, but often struggle when one question requires several visual judgments. We study this gap with controlled tasks for feature binding, numerosity, spatial relations, and amodal completion, together with a Composite task that combines them. Matched counterfactual image pairs isolate changes in the visual evidence needed to answer. Across four models, direct answers, hidden-state readouts, and state interventions show that the individual judgments can be made without explicit
reasoning and that intervening on the corresponding states can affect the answer. During
reasoning, the Composite answer becomes decodable from hidden states and usable from shortened traces, often before the model stops on its own. We train a small detector to predict this readiness and stop
reasoning at that point. On MMStar and RealWorldQA, this reduces mean
reasoning tokens by 79.1% and 74.5%, while average accuracy rises by 3.13 and 3.30 percentage points, respectively. These findings connect the internal development of answer readiness to a practical rule for allocating
reasoning computation.
Read the full abstract ▾
Computer Vision
ControlTrace: Recovering Control Fields for Hidden-Content Recognition
Zijian Liu, Yaoguang Chen, Liwei Liu et al. (+4 more)
Spatially conditioned
diffusion models can embed words and contours in natural-looking images, but
vision-
language models (VLMs) may fail to recognize the hidden content. Transformation-based recovery depends on parameter and view selection. To evaluate hidden-content recovery and recognition, we construct FreqBlind, a 6,000-image
benchmark spanning contours, real words and non-words across three conditioning strengths. The evaluated transformation-based methods show limited recognition of contour patterns and weakly conditioned hidden content. To address this limitation, we propose ControlTrace to recover the grayscale control field used during generation. An 8.4M-parameter U-Net predicts this field from the carrier image, and a VLM then identifies its content. With Qwen2.5-VL-7B-Instruct, ControlTrace achieves 60.2% open-ended contour recognition accuracy across the three conditioning strengths, exceeding the best of the three evaluated prior methods by 26.9 percentage points. On an A100 GPU, the complete pipeline adds only 7.4 ms (5.3%) to direct VLM inference. Recovered fields have lower pixel errors and higher structural similarity than the evaluated transformation views. Across four evaluated VLMs, ControlTrace retains its overall contour recognition advantage. Recognition remains stable under the tested JPEG compression, Gaussian noise and downsampling. These results support control-field recovery for hidden-content recognition in the evaluated setting.
Read the full abstract ▾
All authors Zijian Liu, Yaoguang Chen, Liwei Liu, Weixi Wu, Hanming Zhang, Jiashui Wang, Na Ruan
Sound
On Temporal Binding in Large Audio Language Models
Paul Primus and Gerhard Widmer
Reasoning about temporal structure of audio recordings requires Large Audio
Language Models (LALMs) to associate sound events with their temporal position. Understanding the underlying mechanisms is a first step toward diagnosing failures and identifying model components that may need improvement. Using mechanistic interpretability, we investigate how temporal information is represented and bound to sound events in three open-source LALMs. We find that across all three, event-specific location becomes concentrated in event name representations at intermediate modality integration layers. These representations encode coarse event position along a low-dimensional, curved relative time trajectory. Steering event name representations along this trajectory systematically shifts before/after beliefs, providing evidence that these representations contribute to coarse temporal
reasoning. In contrast, the same interventions do not reliably shift predicted onset timestamps, suggesting that coarse temporal
reasoning and precise metric event localization rely on distinct mechanisms.
Read the full abstract ▾
Artificial Intelligence
SIPO: Selective-Inference Policy Optimization for Tree-Structured Agentic RL
Zenghuang Fu, Ningqi Chen, Mingda Jia et al. (+10 more)
Tree-structured reinforcement learning trains search
agents by comparing alternative continuations and propagating terminal rewards to intermediate decisions. Adaptive expansion, however, creates a statistical asymmetry: an incumbent is selected using its own generation statistic, whereas fresh siblings are sampled after selection. When that statistic is associated with return, branch values can reflect selection history as well as continuation quality, even for a shared parent. We propose Selective-Inference Policy Optimization (\SIPO{}), which incorporates this distinction into tree-based credit estimation. Its scale-free branch criterion keeps generation scores and sibling penalties on a consistent relative scale; exchangeable branching supplies multiple fresh continuations from each selected parent; and order-statistic correction adjusts retained incumbent values using selection rank and the estimated score–outcome association. These mechanisms preserve the leaf budget and the host policy optimisation objective. Across seven QA
benchmarks using Qwen3-4B, Qwen3-8B, and Qwen2.5-7B, \SIPO{} achieves the highest reported multi-hop and single-hop averages among the compared methods. On Qwen3-8B, it improves these averages over AT\textsuperscript{2}PO by $1.31$ and $1.07$ percentage points, respectively, and ranks first on six of seven
benchmarks. Component ablations evaluate the individual and combined changes, while early-training paired diagnostics show a selected–fresh value gap alongside a near-zero fresh–fresh reference. Together, these results support accounting for selection history when constructing and evaluating search-
agent rollouts. Our code is available at https://github.com/Zenghuang-Fu/SIPO
Read the full abstract ▾
All authors Zenghuang Fu, Ningqi Chen, Mingda Jia, Xiaofeng Han, Zhaoyang Li, Qiuyuan Ai, Zelong Zheng, Haoyu Wu, Tianyu Fu, Chenxu Zhao, Minghui Wu, Guannan He, Changwei Wang
Cryptography and Security
Physics-Attested Federated Learning: Securing Collaborative Anomaly Detection in Critical Water Infrastructure
Jeff Nijsse, Shu Su, Benjamin Oholeguy et al. (+1 more)
Federated learning enables industrial operators to train shared intrusion detection models without disclosing proprietary operational telemetry. However, existing defenses operate strictly in update space, leaving aggregators blind to data poisoning; model updates derived from fabricated telemetry remain indistinguishable from honest contributions. We repurpose cyber-physical process invariants, such as conservation laws and actuator couplings, from runtime detection heuristics into a verifiable admission requirement for
federated updates, mined automatically from clean operational data. We evaluate this admission gate across two physical water testbeds (SWaT, WADI) and a distribution
benchmark (BATADAL), testing seven aggregation rules against telemetry fabrication, exposure-only replay poisoning, and an invariant-aware adaptive adversary. Across three testbeds the mined invariants reject none of 100 honest shards and all naively fabricated ones, including optimised perturbations that FoolsGold admits in full. On real telemetry, five mined invariants detect 12 of SWaT's 35 attacks, while nine invariants detect 20, with no honest shard rejected. With nine rules, the physics gate recovers 69–100% of the targeted-attack recall lost to replay poisoning, and 54–100% of that lost to fabricated telemetry, across five standard aggregators. To reconcile physical admission control with
federated data
privacy, we show invariant compliance using zero-knowledge proofs (zk-SNARKs) to allow clients to prove batch adherence without revealing operational telemetry.
Read the full abstract ▾
All authors Jeff Nijsse, Shu Su, Benjamin Oholeguy, Sreenivas Sremath Tirumala
Information Retrieval
No Attention, No Problem: Rethinking Session-based Recommendation with Pure Convolution
Tao Huang and Wei Zhou
Session-based recommendation (SBR) predicts the next choice in a session by analyzing recent interactions.
Transformer-based models are widely used because of their ability to capture long-range dependencies through self-
attention mechanisms. In contrast, traditional convolutional models, although more efficient, are often limited by their weak global modeling capabilities and are losing ground in SBR tasks. In this work, we propose a Next-generation Pure Convolutional Framework (NextConvRec) for SBR tasks, aiming to balance efficiency and performance. NextConvRec uses a Structural and Positional Convolutional Encoder (SPCE) for preprocessing, combining learnable convolutional positional biases with session-level structural signals extracted through GCN layers. Its backbone convolutional module effectively expands the effective receptive field through depthwise convolutions and pointwise convolutions, enabling robust long-range preference modeling without
attention mechanisms. Extensive experiments on 4
benchmark datasets show that NextConvRec outperforms several state-of-the-art baselines by around 1.73% on average, and reduces the average inference time per session by 16.7%. The convolutional architectures remain a promising direction for efficient and accurate session-based recommendations.
Read the full abstract ▾
Machine Learning
Do Personal Adapters Carry Over Across Sessions? Separating Personal From Population Gains When Calibrating EEG Foundation Models for Motor Imagery
Xilin Tao and Kani Chen
Every new user of a motor imagery
brain-computer interface must be calibrated, and users typically return on different days. A common approach with electroencephalography (
EEG) foundation models fits a small personal adapter on a frozen backbone, but evaluations are usually confined to one session and rarely separate the user's own contribution from that of the population model. We quantified personal adapters with two paired contrasts, the net gain of a user's own adapter over the population model and its advantage over adapters fitted to other users, for three frozen foundation models (CBraMod, REVE and LaBraM). In 235 held-out subjects from three
datasets, same-session calibration produced a positive net gain in all nine model and
dataset combinations (means of 1.5 to 5.4 percentage points), and the gain remained positive when population training was quadrupled. In 54 two-session subjects, adapters fitted on the first day still improved on the population model on the second day by 1.31 points for REVE and 1.64 points for LaBraM (Holm-adjusted p<0.01), retaining 61% and 83% of the gain from same-day recalibration, whereas the gain for CBraMod was 0.04 points and indistinguishable from zero. Same-day recalibration gave the highest mean accuracy for all three models, and adding 5 to 20 second-day labels to the first-day adapter mostly fell short of it. Whether personal adapters carry over across sessions therefore depends on the model and must be tested per model, and personalization should be reported against both a population reference and exchanged parameters, including across sessions.
Read the full abstract ▾
Computation and Language
Pass or Fail? Evaluating LLMs on Two Greek Examination Benchmarks
Panagiota Kyriazi, Eleni Kasoura and Prokopis Prokopidis
The rapid advancement of Large
Language Models (LLMs) imposes a thorough evaluation of their linguistic and analytical capabilities as well as constraints, particularly for a
language with limited
benchmark coverage such as Greek. To address the limited availability of comprehensive
benchmarks in this domain, we introduce Prot-Ex and Pan-Ex, two
benchmarks consisting of questions from entrance exams for Greek Model and Experimental schools as well as the Panhellenic exams (the Greek national university entrance examinations). These
benchmarks are employed to assess the performance of text-only LLMs-including the Greek-adapted KriKri-8B-Instruct, Llama-3.1-8B, Gemma-4-26B, and Qwen-3-32B-across diverse academic disciplines (Modern Greek, Mathematics, Physics, etc.) and task formats (closed, structured, and open-ended), including textualized visual context (i.e., image descriptions). Our findings indicate the localized KriKri-8B significantly outperforms its base model, successfully rivalling much larger LLMs in linguistically demanding humanities tasks. By leveraging an LLM-as-a-Judge methodology, we expose the inadequacy of traditional lexical metrics for evaluating complex
reasoning. Crucially, we uncover a few-shot prompting paradox: while synthetic examples improve accuracy in closed-ended questions, they severely overload the context window of 8B models in structured tasks, causing significant performance degradation. Ultimately, this study suggests targeted linguistic adaptation offsets lower parameter counts in specialized domains, despite the fragility of smaller models to
prompt verbosity.
Read the full abstract ▾
Artificial Intelligence
STRIDE: Automated Evaluation of Text-to-Trajectory Alignment across Diverse Contexts
Wanchun Ni, Tao Qi, Leonel Aguilar et al. (+4 more)
Language-conditioned trajectory generation is here, but its evaluation has not kept pace. Existing pedestrian trajectory metrics compare trajectories with real-world human data. This does not scale to text-to-trajectory generation across diverse contexts, as collecting human trajectories for every scenario is costly and infeasible. Moreover, pedestrian behavior is heterogeneous and context-dependent, with no single metric as the correct answer, and current evaluation frameworks are not transferable to this domain. These challenges make scalable, reliable evaluation difficult. We introduce STRIDE, the first framework for evaluating context
alignment between scenario descriptions and pedestrian trajectories. STRIDE addresses these challenges through three design choices. First, we derive our VRDST evaluation protocol from sociological theories to define a complete evaluation space. Second, it decomposes high-level context into scenario-adaptive behavioral questions. Third, every question is resolved against a deterministic measurement tool library that yields reproducible answers. Together, STRIDE enables complete, verifiable, automated, and scalable evaluation across diverse contexts without requiring human trajectory data. We instantiate STRIDE in the crowd domain as STRIDE-Bench, comprising 1K scenarios, 6K behavioral questions, and 11K measurements with calibrated expected answers across 30 real-world maps. Comprehensive human validations show that STRIDE-Bench is consistent with human behavior and judgment, achieving 80% human agreement. We further evaluate several text-to-trajectory models, finding limited context-
alignment capability and persistent challenges in fine-grained context conditioning. We believe that the STRIDE framework provides a first step toward principled evaluation of context-aligned pedestrian trajectory generation.
Read the full abstract ▾
All authors Wanchun Ni, Tao Qi, Leonel Aguilar, Jiugeng Sun, Marlene Wagner, Verena Zimmermann, Mennatallah El-Assady
Computation and Language
InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision
Guanghao Zhu, Zeyu Liu, Zhitian Hou et al. (+9 more)
Recent medical
multimodal models have benefited from larger corpora, broader modality coverage, and stronger
reasoning-oriented training, yet effective data design across continued pretraining (CPT) and post-training remains challenging. Medical sources vary substantially in structure, granularity, and information density, and their utility shifts as training progresses from broad knowledge acquisition to late-stage consolidation. Meanwhile, post-training is often dominated by short-form visual question answering, providing limited supervision for informative and answer-consistent explanations. We introduce InfiMed2, a family of 4B and 27B generalist medical
multimodal foundation models built around stage-aware data design. We curate a 55.68B-
token corpus that combines broad clinical knowledge with context-rich biomedical visual evidence through source-specific processing. Our CPT pipeline first adapts the
vision encoder, then builds broad medical knowledge, and finally transitions to an evidence-focused data mixture during learning-rate decay. For supervised fine-tuning (SFT), we regenerate visual question-answering responses using answer stability, answer-masked reconstruction, and correctness-constrained selection to produce more informative and answer-consistent supervision. The 4B model is further optimized with reinforcement learning with verifiable rewards (RLVR). Across five medical
multimodal benchmarks, InfiMed2-4B achieves 66.73% mean accuracy after RLVR, surpassing the larger Qwen3.5-9B, while InfiMed2-27B reaches 73.72%, the highest among the evaluated open-weight models.
Read the full abstract ▾
All authors Guanghao Zhu, Zeyu Liu, Zhitian Hou, Pengkai Wang, Zhijie Sang, Shuo Cai, Yang Yu, Yuanyi Wang, Yanggan Gu, Congkai Xie, Jianmin Wu, Hongxia Yang
Computer Vision
Physics-Guided Spectral Distillation for Underwater Image Enhancement on Resource-Constrained Devices
Yifan Chen, Kai He, Ye Zheng et al. (+3 more)
Underwater image enhancement is crucial for improving visual perception in marine applications. Existing
underwater image enhancement studies mainly focus on enhancement quality and visual fidelity, while rarely considering real-time deployment capability, which is essential for resource-constrained
underwater robots. To this end, we introduce a physics-guided spectral
distillation (PSD) method, which reduces model capacity for real-time applications while maintaining the high performance of
underwater image enhancement models. To decompose the outputs of teacher and student models, PSD adopts a multilevel Haar discrete wavelet transform. It transfers low-frequency color and illumination information as well as high-frequency structural details through band-specific objectives. Moreover, the
distillation process of PSD is degradation-aware. We estimate degradation-aware weights through a physical head and combine them with ground-truth-guided reliability masks to selectively retain valuable teacher guidance. Experiments on the UIEB, LSUI, and EUVP
datasets validate the effectiveness of the proposed method. Furthermore, we demonstrate the benefits of enhanced images for downstream perception tasks, including object detection. Deployment on a self-developed ROV further demonstrates its practical applicability in real-world
underwater scenarios.
Read the full abstract ▾
All authors Yifan Chen, Kai He, Ye Zheng, Jijun Lu, Zhe Sun, Tao Chen
Computational Engineering
Where Do Embodied Decisions Come From? Rethinking Latent and Explicit Reasoning
Yuan Lin, Ziyue Zhou, JinLong Zhao et al. (+4 more)
Chain-of-Thought (CoT)
reasoning is increasingly incorporated into
Vision-
Language-Action (VLA) models, yet it can degrade the performance of stronger
embodied agents. We investigate this capability-dependent effect by distinguishing explicit
reasoning from latent decision computation, i.e., perception-grounded computation that directly supports action prediction. Under the standard think-then-act (TTA) paradigm, intervening on the generated CoT while fixing the visual input and model parameters causes a substantial performance collapse, with
Navigation F1 dropping from 72.14% to 11.84%, demonstrating the strong influence of explicit
reasoning on action generation. We then propose decide-then-explain (DTE), which predicts actions before generating explanations, and introduce Visual Conditional Contribution (VCC) and
Reasoning Conditional Contribution (RCC) to characterize the resulting decision process. Across autonomous driving and robotic manipulation, DTE consistently outperforms TTA and conventional no-CoT baselines, while exhibiting greater reliance on perception-grounded computation. Further TTA-trained, DTE-inference experiments show that this benefit is not solely attributable to retraining under the new factorization. Our results suggest that for capable
embodied agents, explicit CoT may be better used to shape decision computation during training rather than mediate action generation at inference time. Code: https://github.com/ocean-luna/openvla-decide-then-explain.
Read the full abstract ▾
All authors Yuan Lin, Ziyue Zhou, JinLong Zhao, Pei Liu, Haipeng Liu, Pan Zhou, Kun Zhan
Computer Vision
D$^2$-VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation
Zijian Ye, Chengqi Wei, Wei Huang et al. (+9 more)
Long-horizon manipulation requires
robots to remember cues that are no longer in view while responding to moving objects. Yet
vision-
language-action (VLA) policies often rely on the latest observation, and refreshing their visual context typically requires another costly
vision-
language model (VLM) pass. We present D$^2$-VLA, which combines dual
memory and dual-frequency control at the KV-cache interface of a pretrained VLA. D$^2$-VLA uses block-wise causal KV caching to encode observations incrementally and, guided by distinct temporal
attention patterns, constructs separate historical KV read views for the VLM and action expert. Between periodic VLM updates, a gated adapter incorporates fresh visual features into the latest history-conditioned KV block, while a short fast-
memory queue supports action replanning. We introduce DOMINO-Long, a ten-task
benchmark requiring
robots to use earlier visual cues when manipulating moving objects. D$^2$-VLA achieves complete-task success rates of 29.3% on DOMINO, compared with 9.6% for $\pi_{0.5}$ and 17.2% for PUMA, and 60.0% on DOMINO-Long, compared with 35.4% and 20.6%, respectively. It improves success rates on eight real-
robot tasks and reaches 97.5% on LIBERO-Long and 74.3% on RoboTwin 2.0.
Read the full abstract ▾
All authors Zijian Ye, Chengqi Wei, Wei Huang, Anlin Zheng, Chunyu Zou, Liangyu Wu, Zikang Zhao, Zhenjie Peng, Yushuo Yang, Shuman Zhao, Zhongrui Wang, Xiaojuan Qi
Machine Learning · Cross-listed from math.OC
Finite-Time Concentration and Convergence Rates for Projected Two-Time-Scale Stochastic Approximation with Markov Noise
Rahul Singh, Vivek S. Borkar and Eric Moulines
We study finite-time concentration and convergence rates for projected two-time-scale
stochastic approximation driven by a controlled Markov chain. The averaged fast map is contractive, while the slow iterate is projected onto a compact convex polyhedron. The associated projected ordinary differential equation may have a discontinuous vector field at the boundary, preventing a direct application of standard analyses based on Lipschitz vector fields. Using the Skorokhod map, we establish explicit high-probability bounds for tracking the moving fast
equilibrium and the projected slow dynamics. These bounds separate martingale fluctuations, Markov-noise residuals, and the bias due to time-scale separation. A Lipschitz Lyapunov function satisfying a uniform decrease condition over fixed time intervals yields almost-sure convergence, with explicit last-iterate rates when the decrease admits a power lower bound. Under uniform Lyapunov contraction,
polynomial step sizes yield joint fast-tracking and slow Lyapunov-error exponents arbitrarily close to $1/3$. Under the additional assumption that the reduced slow update map is a Euclidean contraction, logarithmically separated step sizes improve the joint rate to $O(n^{-1/2}\log n)$ almost surely, including for boundary
equilibria. The same rate holds under a distinct
geometric condition involving a strictly attracting face of a box and a fast
equilibrium that is constant on that face. An actor-critic application achieves an almost-sure value-gap rate of $O(n^{-1}\log n)$ relative to the optimum within the constrained policy class. Further applications include projected TD(0) and projected
stochastic gradient descent. We also extend the analysis to projection of the fast recursion under Euclidean contractivity.
Read the full abstract ▾
Cryptography and Security
CoSec: Benchmarking Agent Security in Communities
Hao Chen, Wenhui Dong, Ye Chen et al. (+12 more)
LLM
agents operate in persistent collaborative environments involving multiple users, communities,
memories, files, and tools. Community boundaries may remain fixed or evolve with changes in membership, roles, composition, and relationships.
Agents must complete legitimate tasks and prevent unauthorized disclosure of protected information. Existing evaluations do not fully examine these risks in
agent systems. We introduce CoSec, an executable
benchmark for evaluating
privacy and authorization enforcement in LLM
agent systems operating within and across communities. CoSec contains 208 canonical scenarios spanning fixed and evolving boundaries, protected information belonging to the
agent owner or other participants, and attacks through dialogue, environmental content, persistent
memory, and composed workflows. CoSec executes complete
agent systems with persistent sessions,
memory, files and tools. It verifies information flows against the active authorization state using execution traces and artifacts. Across harness and model configurations,
agents frequently complete benign tasks but violate
privacy and authorization boundaries.
Privacy behavior varies across harnesses, attack surfaces, and community states, revealing how
memory, files, tools, and workflows can carry protected information beyond its authorized scope. These findings show that task utility does not imply
privacy or authorization compliance and that authorization in community settings remains an unresolved security challenge for persistent LLM
agents.
Read the full abstract ▾
All authors Hao Chen, Wenhui Dong, Ye Chen, Jiezhi Yao, Chenbo Xia, Yuwen Qu, Renxiang Wang, Fudong Yuan, Camil Hamami, Chenglong Pan, Xinquan Yue, Ziyu Wang, Fengyu Ye, Chenyang Si, Caifeng Shan