Computation and Language
Multimodal Co-Generation With Continuous Bitstream Diffusion
Georgios Batzolis, Georgios Kontos, Bruno Trentini et al. (+4 more)
We show that continuous bitstream
diffusion (CoBit), originally developed for
language modeling, provides a unified framework for joint and conditional
multimodal generation using the same training and sampling protocol. Across image-text,
speech-text and
protein sequence-structure modeling, models are trained separately from scratch, but each uses a single input projection,
transformer backbone, bitwise prediction head and binary score-matching objective shared across its modalities. Within each setting, a single checkpoint supports joint generation and the conditional directions included in training by changing which parts of the bitstream are observed. We first validate the approach on MNIST-Sum, a controlled image-text task pairing raw binary images with equations encoded as text, achieving near-perfect conditional accuracy and high joint consistency without learned tokenizers. We then apply the same protocol at larger scale to
speech-text and
protein sequence-structure modeling. Trained without text pretraining, a single
speech-text model performs
speech synthesis, recognition, continuation, and unconditional co-generation. Jointly generated
speech and text achieve 3.5% word error rate between the generated text and automatic transcriptions of the generated
speech. In
protein modeling, a single CoBit checkpoint supports forward folding, inverse folding, and joint sequence-structure generation. For jointly generated
proteins spanning 100-500 residues, computational refolding of CoBit's sequences yields structures that agree with the generated backbones at a mean self-consistency TM-score of 0.911. Together, these results show that CoBit provides a common generative framework for substantially different modalities and forms of cross-modal dependence, without introducing a domain-specific objective or sampling procedure.
Read the full abstract ▾
All authors Georgios Batzolis, Georgios Kontos, Bruno Trentini, Pranam Chatterjee, Micha Livne, Mark Girolami, Luca Ambrogioni
Computation and Language
Evaluating LLMs as Agentic Workflow Executors in Real-World Customer Support Skills
Md Tahmid Rahman Laskar, Gundeep Singh, Xue-Yong Fu et al. (+11 more)
Enterprise customer-support
agents increasingly rely on Large
Language Models (LLMs) to execute bounded, tool-using workflows. We study this problem through the lens of Skill LLMs: specialized executors that operate within a selected skill context and must complete a narrow, stateful workflow under policy and tool constraints. We present a
benchmark construction pipeline that mines recurrent workflows from real-world transcripts, converts them into structured skill descriptions, maps them to tool requirements, simulates environment state with Model Context Protocol (MCP)-compatible mocks, and generates multi-turn tool-using dialogues. Using this
benchmark, we conduct a large-scale zero-shot evaluation of 17
language models spanning closed model families and deployment-oriented open-weight models from 1.7B to 14B parameters, each judged independently by five strong LLM judges, plus a deterministic rule-based evaluator that checks tool coverage, hallucinated tool use, and unsupported claims. We find a consistent ranking in which GPT-5.4 leads by a wide margin and QWEN3.5-9B is the strongest open-weight model (below 20B), outperforming several closed baselines under every judge. We also find that judge choice is not a negligible methodological detail: pairwise judge agreement on the pass/fail verdict averages Kappa=0.524 across 10 judge pairs. These results suggest that comprehensive zero-shot evaluation of skill execution requires judge reliability as carefully as model capability.
Read the full abstract ▾
All authors Md Tahmid Rahman Laskar, Gundeep Singh, Xue-Yong Fu, Karol Chang, Kevin Sanders, Shi Zong, Tania Habib, Julien Bouvier Tremblay, Shayna Gardiner, Harsh Saini, Matthias Lee, Elena Khasanova, Quinten McNamara, Shashi Bhushan TN
Machine Learning
On the Depth Transfer of Neural Optimizers via L-Smoothness
Maximilian Wong, Chenyu Xu, Jiajin Li et al. (+1 more)
Scaling deep learning models efficiently requires predictable performance transfer across compute regimes. Establishing reliable hyperparameter transfer allows optimal training dynamics and structural choices to be discovered on low-cost proxy models, avoiding catastrophic failures during massive-scale training. Existing depth-transfer analyses are architecture-specific and heuristic: hyperparameter transfer is only justified through toy blocks, and both rely on infinite-depth dynamics, a specific initialization, and feature-learning criteria. Building on $L$-smoothness framework of Xu et al. (2026), we understand hyperparameter transfer as how do the Lipschitz and $L$-smoothness constants of the loss scale with the depth $K$? We show that inverse-depth residual branch scaling $\alpha=\Theta(1/K)$ is sufficient to make both constants depth-independent at fixed width, uniformly over the bounded parameter domain and for arbitrary $p\to q$ geometries. Because the bounds depend on each residual branch only through its Lipschitz and smoothness constants, the argument applies to general residual networks (including branches of block depth $\geq 2$ such as
transformer and ResNet blocks) without assumptions on initialization or correlations between layer weights. Since the smoothness constant controls the
curvature penalty in the descent inequality, our bounds motivate a learning-rate scale without an additional depth-dependent
curvature correction for updates bounded in the prescribed norm. This provides a
geometric explanation of inverse-depth residual scaling and its application to the operator-norm geometries associated with AdamW,
Muon, and MOGA. Learning-rate sweeps on GPT-2 (depths 2–128) and CIFAR-100 ResNets (8–32 residual blocks) confirm that
Muon and MOGA exhibit stable learning-rate transfer under these prescriptions, with a common optimal rate across a 64-fold increase in depth.
Read the full abstract ▾
All authors Maximilian Wong, Chenyu Xu, Jiajin Li, Yiping Lu
Machine Learning
Continuous-Time Cloth Dynamics with Physics-Informed Neural ODEs
Yingdong Ru, Lipeng Zhuang, Zhuo He et al. (+1 more)
We present ContinuousCloth, a continuous-time, physics-informed neural solver for deformable cloth dynamics that enforces a spring–mass ODE within a neural trajectory model. Unlike classical time-stepping simulators that return trajectories only on a discrete integration grid, ContinuousCloth supports querying at arbitrary timestamps and computing displacement, velocity, and acceleration via automatic differentiation. We evaluate ContinuousCloth along two dimensions. In a physics-only setting, ContinuousCloth achieves trajectory accuracy comparable to high-precision solvers across six diverse boundary-condition scenarios while reducing runtime by up to 17.4 times. When fused with sparse and noisy observations, the spring–mass ODE acts as a strong inductive bias for downstream tasks: interpolation and parameter estimation. That is, ContinuousCloth reduces interpolation error by 7.3 times over a data-only neural baseline in simulation and 2.4 times on real-world cloth sequences, and achieves 2.6 times lower parameter error than a differentiable-solver baseline under high-noise conditions, demonstrating consistent robustness under realistic measurement imperfections.
Read the full abstract ▾
All authors Yingdong Ru, Lipeng Zhuang, Zhuo He, Gerardo Aragon Camarasa
Robotics
CoPlanVLM: Coordinated Path Planning for a Robot Team using Vision Language Models
David R. Martin, Jared Paquet, Mohammad Abdullah Al Faruque et al. (+4 more)
We present CoPlanVLM, which uses a
Vision Language Model (VLM) to translate a single natural-
language instruction and image into coordinated paths for a team of two
robots. While VLMs offer a flexible, natural-
language interface for commanding
robot teams, their weak spatial
reasoning has greatly limited VLM-based path planning. CoPlanVLM improves the VLM's spatial
reasoning and path planning abilities through three components: (1) an image overlay marking both free space and obstacles, (2) task classification that selects the most appropriate low-level planner and
reasoning prompt for each instruction, and (3) task-specific chain-of-thought
reasoning that
prompts the VLM to identify key objects, allocate roles, and verify routes are collision-free before committing to a plan. Together, these components enable the team to complete complex coordinated maneuvers requiring both
robots and divide large tasks between
robots to increase efficiency. CoPlanVLM also supports detailed maneuvers involving specific routes, loops, and coverage of large areas, as well as tasks requiring directional
reasoning such as north/south or left/right. We validate CoPlanVLM in both Gazebo simulation and on physical Turtlebot 4 hardware. CoPlanVLM attains an average success rate of 84.4% compared to 50.0% for the closest baseline. On trials where both methods succeed, CoPlanVLM's paths are 27% shorter on average. Project code is available at https://github.com/AICPS/CoPlanVLM.
Read the full abstract ▾
All authors David R. Martin, Jared Paquet, Mohammad Abdullah Al Faruque, Pramod Khargonekar, Kyle Volle, J. Humberto Ramos, Scott Nivison
Machine Learning · Cross-listed from math.OC
STREGO: A Scalable Implementation of a Trust-Region Framework for Noisy Bayesian Optimization
Youssef Diouane, Issmail El Hallaoui, Meisseme Kadri et al. (+1 more)
TREGO, a trust-region
Bayesian optimization (BO) method for expensive blackbox problems, has convergence guarantees, but only for exact function values. In high dimension, its performance tends to deteriorate. This paper asks whether a trust-region BO method can be extended to handle noisy evaluations while being competitive in high dimension. We first generalize TREGO into a trust-region framework for noisy BO and extend its convergence analysis to noisy function estimates that are sufficiently accurate with sufficiently high probability. We then propose STREGO, a scalable implementation of this framework. Its global phase selects a batch of candidates by solving a bi-objective problem on the posterior mean of a Gaussian process and the integrated variance reduction. Unlike the predictive variance, this second objective does not favor boundary points merely because they are far from the data. Its local phase minimizes the posterior mean of a second Gaussian process, trained only on points close to the incumbent. STREGO is compared with TREGO, TURBO, BAXUS and NSMA-BO on synthetic and real-world problems of dimension $4$ to $124$, with and without noise. It gives the best average results in every synthetic setting tested and the best or tied-best solution on four of the six real-world problems. An implementation of STREGO is available at https://github.com/Mayousk/STREGO.
Read the full abstract ▾
All authors Youssef Diouane, Issmail El Hallaoui, Meisseme Kadri, Amina Lamghari
Robotics
PILOT: Guiding Long-Horizon World-Model Planning for Robot Manipulation with Visual Subgoals and Policy Priors
Yang Liu, Hanchen Cui and Karthik Desingh
Latent world-model planning has shown promising results in goal-conditioned robotic manipulation, but remains challenging for long-horizon planning due to limited temporal guidance and inefficient action search. A distant visual goal provides little guidance for preparatory actions, while uninformed action sampling makes local optimization inefficient. We propose PILOT (Policy-Initialized Local Optimization with Temporal subgoals), which uses a
video model to generate intermediate visual subgoals and a learned policy to initialize the cross-
entropy method (CEM). CEM refines candidate action sequences toward each subgoal using an action-conditioned world model in a receding-horizon control loop. In our primary real-
robot evaluation across four UR5e manipulation tasks, PILOT achieves 68.8% success, compared with 55.0% without policy initialization, 13.8% for policy-only execution, and 0.0% for direct final-goal planning. Simulation results show that these gains persist over longer horizons: PILOT exceeds direct execution of the same policy by 30 percentage points on long Push-T tasks and by 22 percentage points on LIBERO-10. Policy initialization also accelerates local optimization: on Push-T, CEM matches the cost reached by generic initialization after 10 iterations in only 4 iterations and reduces the final planning cost by 24.1% under the same sampling budget. These results show that combining temporal subgoals with policy-guided action search improves both long-horizon task execution and local planning efficiency.
Videos are available on the webpage: https://pilot-planner.github.io/.
Read the full abstract ▾
Artificial Intelligence
Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventions
Xiaofan Li, Fengchun Qiao and Xing Gao
In Model Context Protocol (MCP) systems, LLMs select tools from metadata rather than from their implementations. Names and descriptions summarize what a tool does, while the parameter schema specifies the fields a tool accepts and how they are organized. Our analysis of metadata for 367,174 MCP tools reveals substantial schema variation among functionally similar tools. Many of these tools share names and similar descriptions but expose different parameter schemas. This raises a key question: can parameter-schema differences influence tool selection before any call is made? We study this question through controlled metadata interventions across five LLMs and five tasks. Each comparison changes exactly one metadata factor (server name, tool name, description, or parameter schema) while keeping the others fixed. We balance tool positions and use conditional logistic regression to estimate position-adjusted effects. We find that: (1) parameter-schema changes produce the largest and most consistent selection shifts among the tested interventions. Models tend to favor separate top-level parameters over grouped or composite forms. (2) Descriptions that explain parameters also tend to increase selection, while name effects are weaker and less consistent. (3) Targeted comparisons show that schema effects depend on the specific parameter names and on how the request represents the same information. When only one schema satisfies an explicit request requirement, all five models select the compatible tool despite position preferences. These results show that parameter schemas influence not only how tools are called, but also whether they are selected. For MCP developers and evaluators, schema design and tool ordering can therefore affect which implementation is invoked even when functionality is unchanged.
Read the full abstract ▾
Machine Learning
Decentralized Bandit Learning of Markov Perfect Equilibria
Felix Höfer and H. Mete Soner
We study decentralized bandit learning in discounted infinite-horizon, general-sum Markov games with continuous actions.
Agents observe the public state, but only their own realized rewards, and use simultaneous perturbation
stochastic approximation to update their actions. Under strong monotonicity of the negative running-reward gradients and sufficiently small time steps, we prove that the fully decentralized procedure converges almost surely to the unique stationary Markov perfect
equilibrium. We obtain an $O(n^{-1/2})$ convergence rate and present a natural extension to episodic finite-horizon learning.
Read the full abstract ▾
Computer Vision
TraceSampler: Making Sampling Errors Predictable and Correctable for Accelerated Generative Sampling
George Matthew, Guoqiang Zhang and Kenta Niwa
Accurate sampling from continuous-time generative models can require many model evaluations. Low-step sampling reduces cost, but coarse updates introduce endpoint errors that can accumulate and degrade generation quality. We introduce TraceSampler, a post-training method that corrects these errors without modifying the pretrained model. Rather than optimising low-step updates directly, TraceSampler models endpoint error as a sample-specific prediction problem. A higher-step reference integration over the same interval provides the supervised target, while a compact representation makes high-dimensional error predictable from the sampling trace. This representation combines shared residual structure, trace-derived directions, and a low-rank remainder, so shallow per-step MLP heads predict sample-dependent coefficients. These coefficients drive a two-stage learned residual correction: the first stage predicts the full endpoint residual, while an intermediate backbone evaluation provides new trace information to predict the remaining residual; this evaluation is reused by the following step. TraceSampler is evaluated with Rectified Flow on CIFAR-10, SiT-XL/2 on ImageNet-256, and SD3.5 Medium for text-to-image generation. At 10 steps, it reduces trajectory MSE relative to matched-step Euler by 90.5% on CIFAR-10 and 85.6% on ImageNet-256, while achieving FID scores of 2.692 and 2.189, close to 120-step Heun references of 2.601 and 2.056 with 24$\times$ fewer backbone evaluations. On SD3.5 Medium, TraceSampler improves image quality and
prompt following; at 10 steps, ImageReward rises from 0.752 to 0.906 while trajectory MSE falls by 70.7%. These gains preserve Euler's backbone-evaluation budget, add less than 0.4% inference FLOPs, and incur negligible wall-clock overhead, showing that coarse-step sampling error can be made predictable from the sampling trace and efficiently corrected.
Read the full abstract ▾
Game Theory
Assortment Optimization under the Mallows Model: Independent-Utility Choice Equivalence and Its Algorithmic Implications
Danny Segev
The Mallows model is a well-established construct in preference modeling, describing random rankings through their Kendall-Tau distance from a reference permutation. Its compact parameterization and interpretable treatment of preference uncertainty have made it influential across statistics, machine learning, probability, and revenue management. In the latter domain, recent literature has studied the
embedding of Mallows-driven choice into assortment optimization. Early approximation schemes in this context require stylized structural simplifications or restrictive parametric assumptions. In the absence of these levers, a highly involved quasi-
polynomial-time approximation scheme has subsequently been proposed, leaving wide open the question of whether near-optimal assortments can be computed in true
polynomial time. The main contribution of this focused technical paper resides in closing these gaps by uncovering a striking independent-utility structure within Mallows choices, and turning this representation into a
polynomial-time approximation scheme. Our central structural characterization explicitly constructs a single collection of mutually independent utilities, reproducing Mallows choice probabilities simultaneously for all offered sets. This exact choice equivalence connects pending Mallows-related challenges to the recent approximation framework of Bowers et al. (FOCS 2025) for assortment optimization under independent random utilities. We make this connection algorithmically feasible through a finite-support reduction that produces distributions of
polynomial size while controlling revenue loss uniformly over all assortments. Fused together, these ingredients lead to an $O(n^{\tilde{O}(1/\epsilon^2)})$-time approximation scheme for assortment optimization under the Mallows model, while allowing arbitrary product prices, concentration parameters, and modal rankings.
Read the full abstract ▾
Computer Vision
Real-Time Traffic Density Estimation: A Hybrid Computer Vision and Predictive Machine Learning Pipeline
Prince Kwabena Appiah Boadu
Traffic density estimation from camera feeds requires connecting visual perception with short-horizon forecasting. We present a two-stage pipeline that combines vehicle detection and tracking with a spatial logic layer, followed by a temporal
estimator. The spatial layer filters bounding boxes by their overlap with a precomputed road mask and extracts per-frame traffic features. LSTM, GRU, and Temporal Convolutional Network (TCN)
estimators use these features to predict five future steps from a ten-frame input window. We
benchmark ten detector variants from six families on a
dataset of 5,716 annotated frames using a single NVIDIA GPU. YOLOv5l achieves the highest mAP@50:95 of 0.524 at 56 frames per second, while YOLOv8n reaches 258 frames per second at 0.487 mAP@50:95; these rates measure detector inference without tracking or spatial filtering. Temporal evaluation on 29,243 held-out windows yields aggregate R-squared values of 0.858 for GRU, 0.856 for LSTM, and 0.840 for TCN. These metrics measure forecasts of detector-derived telemetry and are dominated by direction variance, rather than establishing absolute traffic-density accuracy. No road mask was available for the evaluation stream, limiting validation of the spatial filtering component. A live run on a third camera stream examines forecasting behavior within the complete pipeline. The results characterize detector speed-accuracy trade-offs and the limitations of short-window traffic telemetry forecasting.
Read the full abstract ▾
Computer Vision
Manifold Navigation Learning Using Recurrent Networks: Efficient Training Techniques
Sarah Harkins Dayton, Sathya Rengaswami, Vasileios Maroulas et al. (+2 more)
Position tracking is a well-studied problem in Euclidean navigation, yet comparatively little
attention has been given to navigation on general
manifolds, where motion is governed by the underlying geometry of the space. In this work, we formalize the problem of
manifold navigation as a velocity-driven tracking task and propose a coordinate-chart-based learning framework in which learning models operate entirely in a Euclidean representation. Predicted trajectories are mapped to the
manifold via an immersion function prior to error computation, allowing
geometric consistency to be evaluated without explicitly enforcing
manifold constraints during learning. Within this framework, we focus on Gated Recurrent Units (GRUs) for sequential position prediction on curved
manifolds after comparisons with alternative recurrent architectures. We evaluate our GRU models, trained with pertinent custom loss functions, against both a standard MSE loss function and a naive vector-addition baseline. We evaluate our approach on both analytically (and numerically) defined
manifolds, as well as a surface reconstructed from a
LiDAR point cloud. Our results show that recurrent models can learn to approximate
manifold-constrained dynamics from velocity sequences and consistently outperform Euclidean baselines. Our results demonstrate that certain custom loss functions significantly improve long-horizon prediction accuracy compared to conventional point-wise loss functions.
Read the full abstract ▾
All authors Sarah Harkins Dayton, Sathya Rengaswami, Vasileios Maroulas, David L. Boothe, Ioannis D. Schizas
Computer Vision
EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference
Anxhelo Diko, Jieyu Lin, Stefano Ambrogio et al. (+5 more)
Foundation models like V-JEPA are increasingly deployed across a heterogeneous edge-cloud continuum spanning smart glasses, phones,
robots, and servers. Yet, they are typically trained as rigid, fixed-capacity networks bound to one device, supporting neither reconfigurable compute budgets nor collaborative execution. Existing approaches address only one axis at a time: slimmable and NAS-based methods provide width elasticity but assume single-device inference, whereas split-computing methods enable cross-device collaboration only for fixed partitions and static capacities. We introduce Elastic V-JEPA (EV-JEPA), a single model trained once and deployable everywhere along the continuum. Its width is reconfigurable, and it is natively partitionable across heterogeneous devices. At its core is Transition-Aware Triplet Training (T3). At each step we pick a narrow and a wide width, and train three paths: each width alone, plus a hybrid path that runs the early layers narrow and the rest wide. The hybrid switch point is where one device hands off to another, so the same weights learn both single-device and cross-device execution. Across various
benchmarks, EV-JEPA falls within 1 accuracy point of independently trained baselines while improving over previous elastic training approaches by up to 11 accuracy points. Narrow-to-wide cross-device hybrid execution recovers to within 1.1 points of the highest-capacity standalone model and supports many-to-one, any-to-any, and variable-split configurations. Finally, we show that the elastic principle extends beyond the backbone to an elastic
token compression mechanism that exposes controllable trade-offs among on-device computation, latency, and accuracy.
Read the full abstract ▾
All authors Anxhelo Diko, Jieyu Lin, Stefano Ambrogio, Ziyun Li, Mahmoud Assran, Théo Moutakanni, Barbara De Salvo, Chiao Liu
Computer Vision
MvFFN: Multi-view Floor plan Feed-forward Network for Unposed Wide-Baseline Panorama Layout Reconstruction
Yuguang Li, Yichuan Deng, Zixuan Liu et al. (+5 more)
Reconstructing accurate camera poses and floor-plan layouts from sparse, unposed, wide-baseline RGB panoramas remains a challenging open problem. Direct multi-room floor-plan prediction is especially difficult in this setting: pose, geometry, semantics, and global layout structure are tightly coupled, yet a complete floor-plan is hard to learn as a single global output. Our key insight is to keep the model's predictions strictly local. Multi-view Ordered Wall Instance Segmentation (MOIS) is designed around this concept. Each panorama column predicts only the wall it observes plus the next two adjacent walls in clockwise order. The global wall partition, room layouts, and shared coordinates emerge from deterministic aggregation of overlapping local predictions. Building on MOIS, we propose Multi-view floor-plan Feed-Forward Network (MvFFN), which jointly predicts coarse camera poses, dense global geometry, and per-wall semantic outputs. A downstream Layout Chaining (LC) post processing pipeline then assembles these predictions into multi-room floor-plans with shared global coordinates, consistent wall identities, and connected room structure. The dense MOIS predictions recover cross-view wall identity and local within-room connectivity more accurately than composed modular baselines, and the assembled layouts substantially outperform prior methods on wall and junction level accuracy. We also introduce the ZInD-CrossView
dataset, which augments ZInD with globally unique wall instances and cross-view segmentation labels for this task.
Read the full abstract ▾
All authors Yuguang Li, Yichuan Deng, Zixuan Liu, Ivaylo Boyadzhiev, Xiuyuan Wu, Xinyu Zou, Linda Shapiro, Alex Colburn
Computation and Language
ASSET: Adaptive Speech Segmentation via Reinforcement Learning
Cihan Xiao, Yuan Tseng, Matthew Wiesner et al. (+4 more)
Speech large
language models (LLMs) are often built using pretrained acoustic encoders as an audio interface to an LLM decoder. However, LLMs scale poorly with input sequence length, while acoustic encoders tend to produce substantially longer sequences than typical textual inputs. This motivates learning to compress encoder outputs while preserving task-oriented information. To this end, we introduce ASSET, a reinforcement learning (RL)-based framework that learns to adaptively compress encoder outputs by first partitioning them into variable-duration segments and then pooling each segment into a continuous audio
token. We use group-relative policy optimization (GRPO) to train a segmenter with task performance feedback and a
token-rate penalty. In automatic
speech recognition (ASR) experiments, ASSET achieves performance comparable to a transcript-assisted character-boundary oracle with approximately 25% fewer audio
tokens on English LibriSpeech, while improving on a phone-boundary oracle at a similar
token rate on Mandarin AISHELL. Exploratory experiments on non-ASR tasks further illustrate how learned segmentation changes when the framework is adapted to tasks beyond transcription.
Read the full abstract ▾
All authors Cihan Xiao, Yuan Tseng, Matthew Wiesner, Desh Raj, Leibny Paola Garcia Perera, Sanjeev Khudanpur, David Harwath
Information Theory
Orthogonal SC-LDPC Codes for the IDS Channel
Frederik Walter, Jessica Bariffi and Antonia Wachter-Zeh
We propose a coding scheme for the insertion, deletion and substitution (IDS) channel tailored to the short strands of DNA-based data storage. Instead of an inner synchronisation code, a spatially coupled LDPC (SC-LDPC) code is placed across the strands, with its coupling axis aligned to the position along the strands. Every decoded position thus becomes an anchor to synchronise the neighbouring strands. The decoding waves of the SC-LDPC code carry these anchors from both ends of the strands towards the middle. We bound the achievable rate from above by the information rate of a single strand and from below by that of a separated receiver. For strands of 100 nucleotides (the relevant regime for DNA-based data storage), our scheme outperforms a rate-matched concatenated construction.
Read the full abstract ▾
Artificial Intelligence
ConicFlow: constrained Inference-time optimal control for flow matching models
Shervin Khalafi, Kasra Fallah, Sung Cho et al. (+3 more)
We propose ConicFlow, which frames constrained sampling from a flow matching model at inference time as an optimal control problem that balances deviation from the pretrained trajectory against reward, subject to hard terminal constraints. We discretize this problem and adapt receding-horizon control, optimizing only a short window of the trajectory at each step for computational tractability. We address the resulting sequence of receding-horizon subproblems with a receding-horizon primal-dual
algorithm. Its dual multipliers capture the difficulty of satisfying each constraint and are refined throughout sampling, steering generation toward feasible final samples. We further propose a terminal-estimate variant that avoids accumulating local truncation errors by evaluating the reward and constraints at the predicted terminal point rather than at the end of the current horizon. Under standard regularity assumptions, we establish finite-horizon guarantees on suboptimality. Under convex-affine structure, we derive a sharper bound that scales linearly with omitted horizon length, and we show that the terminal-estimate variant is provably tighter when the flow dynamics change slowly. Experiments in both low- and high-dimensional settings with a recent large-scale flow matching model show that ConicFlow achieves high reward and strong constraint satisfaction while deviating less from the pretrained flow than recent baselines.
Read the full abstract ▾
All authors Shervin Khalafi, Kasra Fallah, Sung Cho, Dongsheng Ding, James Anderson, Alejandro Ribeiro
Human-Computer Interaction
Plottery: A Code-Integrated GUI for Matplotlib
Brian Hempel, Ruanqianqian Huang, Devamardeep Hayatpur et al. (+2 more)
Scientists have many tools to author plots: graphical direct manipulation apps like a spreadsheet's chart builder, text-based programming tools like Matplotlib and ggplot2, or AI chat interfaces. Textual code is the most capable of these, but writing code, even with AI assistance, is a potentially poor fit for graphical chart design. We present Plottery, a Jupyter notebook extension that leverages API type annotations to add graphical direct manipulation to the popular Matplotlib plotting library. User interactions in Plottery's in-situ graphical interface are immediately reified in the user's code, so they may seamlessly mix-and-match between GUI, code, and AI interactions. In a user study, participants seamlessly alternated modalities, with the GUI used for straightforward visual changes, code for quick edits or duplicating code, and the AI for broad support. Plottery showcases graphical interaction that retains code's benefits.
Read the full abstract ▾
All authors Brian Hempel, Ruanqianqian Huang, Devamardeep Hayatpur, Sorin Lerner, Haijun Xia
Machine Learning
Learning to See through Action: Predictive Curiosity for Robots
Stelios Zarifis, Panagiotis Filntisis, George Retsinas et al. (+1 more)
Humans and animals do not learn
vision from a fixed set of data: they move, interact, and learn from the visual consequences of their actions. Motivated by this developmental view, we study whether
robots can improve visual learning by actively collecting experience through curiosity-driven interaction. We introduce DySCOVER, a self-supervised framework for active
robot visual learning. A
robot observes RGB frames, learns a visual autoencoder that encodes and reconstructs them, and trains a recurrent world model to predict the next visual latent state conditioned on the current state and action. The
agent's policy is trained only with intrinsic rewards: reconstruction error encourages visually novel observations, while the world model's error encourages transitions whose consequences are not yet predictable. This prediction error biases the policy toward more intricate trajectories that enrich the
agent's set of experiences with observations beyond those encouraged by visual novelty alone. Our experiments conducted with robotic arms as the embodied
agents show that adding predictive curiosity improves the
vision model over using visual curiosity alone. Across variations in initial conditions, experience capacity, embodiment, and scene content, adding predictive curiosity consistently improved visual learning. These results show that predictive curiosity can improve active visual learning beyond reconstruction-based curiosity alone.
Read the full abstract ▾
All authors Stelios Zarifis, Panagiotis Filntisis, George Retsinas, Petros Maragos
Cryptography and Security
ARMOR: Adversary Cost Amplification via a Schema-Preserving Password-Hardening Retrofit for Resource-Constrained Systems
Avinash Srinivasan
Internet of Things (IoT) gateways, edge authentication servers, and legacy systems need protection against offline password guessing after credential-database disclosure.
Memory-hard key derivation functions (KDFs) are the preferred defense, but their resource requirements can exceed
memory or latency budgets. ARMOR addresses this gap with a schema-preserving verifier-side transform for database-only (DB-only) compromise. Secret per-account chains of injective operations reside in an isolated Chain Registry. Fixed terminal escape encoding preserves distinct pre-hash values under each chain and moves outputs outside printable ASCII. We evaluate NAIVE adversaries using unmodified cracking tools and AWARE adversaries that enumerate chains. For the evaluated pre-hardened three-operation chains, NAIVE recovery is at most 0.10% on non-digit targets. Escape encoding yields zero recovery of digit-only targets under evaluated stock rule-based and non-rule attacks, closing a bypass permitting up to 45.3% recovery. Across 14.05 million eligible RockYou password entries, retaining duplicates, the median conditional guessing number is approximately 52 output-class tests at depth three and 317 at depth five. These values quantify the chain-search subproblem given the correct plaintext under a unit-cost model. A separate historical 14-operation AWARE evaluation matches all transformed targets at full dictionary coverage. Deployment measurements show microsecond-scale bare-metal transforms on three IoT boards, with protected storage dominating the measured TrustZone-M prototype components. Although an isolated high-
entropy pepper provides stronger cryptographic protection, ARMOR combines compact per-account chain storage with low transform cost. When resources permit and the KDF accepts the complete transformed output, ARMOR composes as KDF(ARMOR(password)).
Read the full abstract ▾
Artificial Intelligence · Cross-listed from eess.AS
RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can
Yi-Jen Shih, Puyuan Peng, Haohe Liu et al. (+2 more)
Recent
Speech Full-Duplex (FDX) models are capable of engaging in naturalistic conversation by listening and speaking at the same time. Building on this foundation, recent architectures introduce internal thinking streams to enhance their intelligence. To guarantee these streams process successfully in real time, existing methods enforce a fixed decoding rate, which assumes a hardware profile capable of a specific latency. We argue this static approach is severely suboptimal. Because internal thinking is intrinsically asynchronous, an ideal FDX model should sustain real-time conversational flow while allowing each stream to decode at its own hardware-adaptive rate. Thinking streams must process as fast as the hardware allows to deliver accurate answers faster. To solve this, we propose RUBATO, a novel multistream positional encoding that assigns independent clocks to entirely decouple the streams. This structural isolation empowers the model to zero-shot interpolate across any inference decoding speed. It drastically simplifies
dataset construction by requiring training on only two extreme decoding speeds while smoothly adapting to any intermediate rate during deployment. RUBATO demonstrates consistent robustness across five spoken
reasoning tasks. On Spoken GSM8K, a baseline model collapses to 1.7% accuracy at interpolated speeds. Conversely, RUBATO-Full sustains 48.7% to 51.1% accuracy across all rates, leveraging faster hardware to achieve a 6.5x shorter time-to-answer with a constant time-to-first-response. This advantage holds on BIG-Bench Audio and amplifies significantly during complex full-duplex interruptions. Specifically, when a user interjects while the model is actively speaking and
reasoning, RUBATO-Full achieves 40.0% to 45.1% accuracy, whereas the baseline fails completely.
Read the full abstract ▾
All authors Yi-Jen Shih, Puyuan Peng, Haohe Liu, David Harwath, Abdelrahman Mohamed
Software Engineering
Diagnosing Generalization Boundaries in AI-Based Vulnerability Detection via Calibrated Embedding Distances
Konstantin Ievtushenko, Ralf Lämmel and Jan Jürjens
High
benchmark scores in learning-based vulnerability detection mask severe degradation under distribution shifts. This paper demonstrates that calibrated
embedding geometry exposes these hidden generalization boundaries on syntactically similar vulnerable and patched code pairs. An evaluation of CodeBERT and Qwen2.5-Coder across pattern, project, and temporal shifts on a strictly paired
dataset reveals three main findings. First, whitened intermediate-layer
embeddings with local nearest-neighbour scoring separate unfamiliar samples with a mean out-of-distribution AUROC of 0.739 (RQ1). Second, distance scores reveal a shift-specific reliability gradient where accuracy drops by over 35 percentage points and cross-project shifts cause the sharpest degradation, even though temporal shifts drift geometrically farthest (RQ2). Third, false positives on patched code systematically correlate with
geometric proximity to vulnerable training neighbourhoods, increasing from 9.4% in the lowest-risk decile to 58.8% in the highest (RQ3). These calibrated distances establish a diagnostic signal that maps representation geometry directly to model degradation without requiring auxiliary models.
Read the full abstract ▾
Artificial Intelligence
Controllable Diverse Sampling through Pullback Directions
Sylvain Dehayem Kenfouo, Aaron Zweig, Mingxuan Zhang et al. (+1 more)
Conditional
diffusion models can generate high-quality images that align closely with a given
prompt. However, samples generated from different random seeds may remain visually similar, limiting creative exploration. Existing approaches can increase variation, but often require additional optimization or sampling overhead, while stronger perturbations may introduce distortions that degrade image quality or
prompt alignment. We propose a training-free method that leverages the local pullback geometry induced by the mapping from the conditioning-
embedding space to the image space to identify directions of maximal output sensitivity. By steering different sampling trajectories along these directions, we encourage distinct generations while maintaining consistency with the conditioning signal. Experiments on MS-COCO and BrushNet show the efficacy of the method in improving diversity on
prompt-conditioned image generation and inpainting.
Read the full abstract ▾
All authors Sylvain Dehayem Kenfouo, Aaron Zweig, Mingxuan Zhang, Elham Azizi
Computer Vision
GeoFix: Geometry Fixing for 4D Consistent Generation
Yeobin Hong, Hyojun Go, Dohun Lee et al. (+7 more)
Video diffusion models increasingly serve as priors for 4D generation, but the generated
videos often lack
geometric consistency. Recent approaches address this through post-training by designing
geometric rewards, yet this can degrade texture and fine detail, requiring auxiliary appearance rewards and careful reward balancing. To address this challenge, we propose GeoFix, a unified framework that improves
geometric consistency and visual quality with a single
geometric reward, applicable to both training and inference-time reward
alignment. Our reward measures cross-view depth consistency in static scene regions, targeting structural inconsistencies without directly constraining appearance or penalizing valid object motion. Crucially, depth agreement increases with camera field-of-view overlap even in real
videos and 4D
datasets, allowing a smaller camera movement to be mistaken for better geometry. We account for this effect by calibrating the reward against the overlap-dependent trend. GeoFix achieves a 39% improvement in WorldScore 3D consistency and a 4% increase in VBench Quality Score.
Read the full abstract ▾
All authors Yeobin Hong, Hyojun Go, Dohun Lee, Jeongjae Lee, Sunghwan Hong, Federico Tombari, Zhaochong An, Jong Chul Ye, Konrad Schindler, Dominik Narnhofer
Machine Learning
Layer-wise Input Recoverability in Deep Neural Networks
Maxwell Spiridonov, Alireza Rahmansetayesh and Pierre Baldi
The inner workings of deep neural networks are still not well understood. The information bottleneck framework, which characterizes deep networks by the mutual information between their hidden representations and the network's input, has been used to identify compression of input information with respect to an input image and output labels. We instead study how much of a layer's own input is recoverable from the layer's output in convolutional neural networks (CNNs). A convolutional layer applies its filters to one input patch at a time, so we evaluate recoverability on sampled input patches with a reconstruction-based probe and not on the entire feature map. The probe's size thus depends on channels and kernel size and not on feature map size, which lets our method scale to deep CNNs with wide feature maps. For each layer, we compare pretrained, random, and reconstruction-optimized encoders, using linear and linear-plus-residual-MLP decoders. We measure recoverability as $R^2$ between input patches and their reconstructions on a held-out set. Across nine ImageNet CNNs, recoverability generally decreases with depth but is more consistently predicted by the compression of the layer. Learning-attributed recoverability, the difference between pretrained and random encoders, is not uniform across architectures. We introduce single-property random controls, which transfer one pretrained property onto a random encoder, and find that each control changes recoverability with a consistent sign across models: copying pretrained singular vectors tends to raise recoverability, while copying the pretrained singular-value spectrum or activation sparsity lowers it.
Read the full abstract ▾
Cryptography and Security
Watermarking for Byzantine Robust LLM Training
Leixu Huang, Zedian Shao and Teodora Baluta
LLM watermarking has emerged as a promising approach for detecting downstream model misuse in centralized settings, but extending it to
federated learning (FL) presents unique challenges. In particular, FL systems often rely on strong robust aggregators to defend against poisoning attacks. We empirically show that a malicious server can exploit these same aggregators as effective watermark-removal mechanisms during training, causing representative existing LLM watermarking methods to fail detection in the final model. To address this limitation, we formalize the problem as a constrained optimization task and propose CamoMark, an aggregator-agnostic watermarking scheme for
embedding robust watermarks in FL. CamoMark optimizes watermarked updates to become inliers, maintaining bounded deviation from the distribution of clean updates, while preserving sufficient watermark signal for reliable detection. Evaluated on Qwen fine-tuned on the Alpaca
dataset, CamoMark achieves a detection confidence of $p$-value $= 1.82\times10^{-2}$ using only 3.3% watermarked data. The watermark also remains robust to a range of post-training removal attacks, including continued fine-tuning on clean data,
pruning, and model
distillation. At higher watermark ratios, detection confidence improves substantially, reaching $p$-value $= 5.11\times10^{-11}$ at a 13.3% watermark ratio.
Read the full abstract ▾
Systems and Control
Zero-Crossing Time Lower Bounds for the Homogeneous Super-Twisting Algorithm
Diego Gutiérrez-Oribio and Ioannis Stefanou
High-frequency switching and chattering are well-known problems in sliding-mode control. Although the super-twisting
algorithm produces a continuous control input, the sliding variable may still cross zero repeatedly near the sliding
manifold. This paper studies how fast these crossings can occur and whether they can accumulate in finite time for the scalar homogeneous super-twisting
algorithm. Two lower bounds on the time required to return to zero are derived, together with a bound on the time needed to complete a full excursion. These results give minimum separation times and upper bounds on the number of crossings associated with excursions above a prescribed amplitude. Although zero crossings may accumulate in finite time, the amplitudes of the complete excursions considered here must vanish and satisfy a summability condition. Thus, crossings associated with excursions above any prescribed nonzero amplitude cannot accumulate in finite time. When the perturbation-rate bound is zero, the estimates recover the scaling imposed by homogeneity. For the classical super-twisting
algorithm, the square-root scaling also holds for nonzero perturbation-rate bounds. Numerical simulations illustrate the results.
Read the full abstract ▾
Computer Vision
TGSR: Text-Grounded Diffusion for Faithful Image Super-Resolution
Zhiming Hu, Ran Zhang, Angela Ye et al. (+7 more)
Diffusion models have achieved remarkable success in image super-resolution (SR) by leveraging strong generative priors to enhance perceptual quality. However, for textual content, these priors often produce text
hallucinations: non-existent or distorted text, which alter scene semantics and compromise model reliability. While recent methods focus on enhancing text generation, they do so indiscriminately, leaving
hallucinations fundamentally unresolved. We argue the core problem is a lack of selective control. Therefore, we reformulate text SR as a selective generation problem: text is resolved only when reliable generation is expected, while other scene content is fully super-resolved. We introduce TGSR (Text-Grounded Super-Resolution), a test-time framework that progressively identifies and suppresses likely hallucinated text regions during generation. To enable this, we train dedicated detectors on a large-scale
dataset curated from diverse
diffusion-based SR outputs. TGSR consistently improves text fidelity across multiple backbones, increasing recognition precision by up to 17.68 points without degrading perceptual quality, underscoring the value of selective control in generative SR.
Read the full abstract ▾
All authors Zhiming Hu, Ran Zhang, Angela Ye, Raghav Goyal, Faraz Ali, Tristan Ty Aumentado-Armstrong, Samrudhdhi B. Rangrej, Konstantinos G. Derpanis, Iqbal Mohomed, Alex Levinshtein
Hardware Architecture
DFIT: A Framework for Building Instrumentation Tools Across Heterogeneous Dataflow Accelerators
Md. Azahar Alam, Abrar Hossain and Kishwar Ahmed
Emerging dataflow accelerators offer exceptional energy efficiency and throughput for machine learning workloads, yet remain largely opaque to developers for performance analysis. Unlike CPUs and GPUs, where instrumentation frameworks enable fine-grained introspection, dataflow hardware exposes little information. This lack of transparency limits the ability of compiler engineers, system designers, and application developers to analyze, tune, or debug accelerator behavior. In this paper, we introduce DFIT (DataFlow Accelerator Instrumentation Tool), an instrumentation framework designed specifically for dataflow architectures. DFIT directly instruments operations, regions, and compiler transformations at the graph IR and backend-lowering stages using semantic program representation. DFIT captures high-level, semantically meaningful data that reflect the true dataflow of computation. These data can be exported for visualization and analysis. Our approach demonstrates how instrumentation can be generalized beyond traditional hardware to support heterogeneous architectures, providing portability without modifying tool logic. By opening the black box of dataflow execution, DFIT establishes a foundation for new classes of performance analysis tools for AI accelerators. It provides a practical path toward reproducible, architecture-aware introspection across heterogeneous systems, bridging the long-standing visibility gap between compiler design, runtime scheduling, and hardware realization. Evaluation across Cerebras, SambaNova, and TPU shows practical overhead of 3.53%-13.62% across instrumentation modes with robust cross-platform portability.
Read the full abstract ▾