Sound · Cross-listed from eess.AS
Listen, Critique, and Refine: RL-Based Self-Refinement for Instruction-Following Speech Synthesis
Chee-En Yu, Yi-Cheng Lin, Sung-Feng Huang et al. (+4 more)
Large Audio
Language Models (LALMs) can follow diverse instructions to synthesize
speech in specified styles. However, complex instructions that require simultaneous control over pitch dynamics, speaking rate, and emotional tone often exceed what a single-pass generation can faithfully realize. While recent
reasoning models have shown that intermediate "thinking"
tokens improve output quality, this paradigm has been confined to the text modality. In this work, we extend
reasoning to the audio
token space by training a LALM with reinforcement learning to reason over its own
speech output. The model first generates a draft
speech as a form of audio-
token reasoning, critiques its own generation by reflecting on the acoustic realization in text, and then produces a refined version conditioned on both the first-pass
speech and the critique, all within a single model. After RL training, the refined two-hop outputs achieve a relative improvement of 7.15% on the InstructTTSEval
benchmark, demonstrating the model's reflective ability.
Read the full abstract ▾
All authors Chee-En Yu, Yi-Cheng Lin, Sung-Feng Huang, Yun-Shao Tsai, Ho-Lam Chung, Xuanjun Chen, Hung-yi Lee
Distributed Computing
MCP-GRANITE Benchmark: GRANularity Interface TEsting for MCP-Based LLM Agents
Demetris Paschalides, Moysis Symeonides, George Pallis et al. (+1 more)
As LLM
agents increasingly interact with external tools through standardized protocols such as MCP, tool-interface design becomes a critical yet underexplored factor. How fun{\psi}tionality is decomposed into tools affects whether an
agent can select the right tool and construct valid arguments. This choice is especially consequential at the edge, where resource constraints limit which models can run locally and scaling up is often not an option. We present MCP-GRANITE, an open-source extensible
benchmark framework that treats tool-interface granularity as a controlled variable for MCP-based
agents, evaluated under edge and IoT scenarios. It comprises 81 multi-step scenarios across 9 domains, instantiated at 4 granularity levels from fine-grained primitive tools to a single tool. We evaluate 9 locally deployed models (268M-20.9B parameters) across 8,748 trials using task completion, tool selection F1, argument accuracy, latency, and resource-usage metrics. Results show that a 4-tool interface offers the best trade-off, improving task completion by 16.4% over fine-grained primitives and 33.6% over a single monolithic tool, while nearly doubling argument accuracy. Model size is only weakly correlated with task completion and strongly with latency, while its association with argument accuracy is less robust, and a 3.2B model at the optimal granularity outperforms a 20.9B model at a mismatched one. These findings identify tool-interface granularity as a key design parameter for MCP-based
agents.
Read the full abstract ▾
All authors Demetris Paschalides, Moysis Symeonides, George Pallis, Marios D. Dikaiakos
Computer Vision
Relightable 3D Avatar Reconstruction with Semantic-Adaptive Motion-Illumination Responses
Jiankuo Zhao, Xiangyu Zhu, Jijie Li et al. (+3 more)
Reconstructing expressive and relightable 3D head avatars from
monocular videos remains challenging in computer
vision, as it requires accurate modeling of both non-rigid facial motion and illumination-dependent appearance. Existing Gaussian avatar methods commonly rely on globally coupled representations, in which Gaussian primitives share a unified motion or illumination response model. Such uniform modeling neglects the distinct motion patterns and material/reflectance properties of different facial semantic regions, thereby limiting fine-grained animation accuracy and reducing relighting plausibility. To address this limitation, we propose SAMIRA, a 3D Gaussian avatar framework for semantic-adaptive motion-illumination response modeling. For motion response modeling, the Semantic-Adaptive Motion Response module rasterizes current-to-reference mesh displacements into a
topology-consistent UV space and leverages facial semantics to route displacement features through semantic-specific modulators, predicting localized Gaussian
geometric residuals beyond coarse mesh binding. For illumination response modeling, the Semantic-Adaptive Illumination Response module learns compact diffuse and specular response factors for each facial region, allowing Gaussians in different regions to adapt their illumination responses to novel environment lighting. These response factors are incorporated into deferred physically based shading, providing a lightweight approximation of semantic-dependent illumination effects. Extensive experiments on self-reenactment, cross-reenactment, and relighting demonstrate that SAMIRA improves both fine-grained expression reconstruction and relighting realism over existing methods.
Read the full abstract ▾
All authors Jiankuo Zhao, Xiangyu Zhu, Jijie Li, Baiqin Wang, Shukai Chen, Zhen Lei
Computation and Language
TAC-Time: Texts as Channels For Multimodal Time Series Forecasting
Jiayi Liang, Xiaotian Gu, Xinyu Xie et al. (+2 more)
Most existing time series
forecasting methods rely solely on numerical observations, overlooking rich contextual information from auxiliary texts. Recent
multimodal approaches attempt to incorporate textual signals, but they often treat text as static features or use large
language models as
forecasting backbones, limiting their ability to capture temporal dynamics and increasing computational cost. To address these challenges, we propose TAC-Time, a unified framework that transforms textual information into additional temporal channels. By modeling text features jointly with numerical sequences in a shared temporal backbone, TAC-Time preserves temporal continuity and periodic structures while remaining efficient and scalable. This formulation also enables systematic interpretability analyses. We show strong cross-modal dependencies through
attention and frequency-domain analyses, and identify predictive textual signals whose correlation-aware
alignment yields partial
forecasting improvements. Extensive experiments on real-world
multimodal benchmarks demonstrate that TAC-Time outperforms prior methods.
Read the full abstract ▾
All authors Jiayi Liang, Xiaotian Gu, Xinyu Xie, Yuanbin Wu, Xiaoling Wang
Robotics
Object-Centric Conditioning for Visuomotor Flow Matching
Jijie Li, Xu Yang, Junhong Zou et al. (+4 more)
Robot visuomotor policies are commonly formulated as autoregressive,
diffusion-based, or more recently, flow matching models. Among them, Action-to-Action (A2A) flow matching improves inference efficiency by initializing generation from historical action priors rather than
stochastic noise. However, stale historical motion patterns and entangled global visual representations can jointly reduce robustness under spatial out-of-distribution (OOD) shifts and visual distractors. In this work, we propose SlotFlow, an object-centric flow matching policy for robust visuomotor manipulation. SlotFlow decouples scene observations into semantic ("what") features and lightweight image-plane spatial ("where") cues to provide object-aware policy conditioning and current-state grounding. The semantic representation suppresses irrelevant background correlations, while the spatial cue improves adaptation to shifted object configurations. Extensive simulation and real-world experiments demonstrate improved robustness under visual distractors and severe spatial perturbations while preserving the low-step inference efficiency of A2A. Controlled initialization and perception ablations further identify object-centric grounding as a major source of the gains and show that it complements, rather than replaces, useful historical motion priors.
Read the full abstract ▾
All authors Jijie Li, Xu Yang, Junhong Zou, Chunhai Zhao, Chaoyang Zhao, Zhen Lei, Xiangyu Zhu
Information Retrieval
Graded-Relevance Composed Multimodal Retrieval for E-commerce Visual Search at Scale
Anubhav Gupta, Hrushikesh Mohapatra, Prijith Chandra et al. (+6 more)
Visual search on large e-commerce catalogs must serve both "similarity" queries that ask for items resembling an uploaded image and "modifier" queries that comprise an image and text describing a desired modification (e.g. a color change or style swap). The latter is the setting known as composed image
retrieval (CIR). Existing CIR methods, however, treat relevance as binary and train on triplets with a single positive target - a poor fit for real catalogs where many candidates partially satisfy a user query and ranking across that partial-match spectrum drives the customer experience. We propose a methodology for training CIR retrievers on graded relevance, consisting of: (i) a VLM to curate training data, generating both queries (object detection + modifier synthesis) and 4-level relevance labels without manual annotation, (ii) an iterative relevance-feedback loop that expands the training set by mining hard negatives from the in-training retriever, and (iii) a hierarchy-aware angular objective to train the retriever directly on the graded labels rather than collapsing them to a binary split. We call this methodology GradCIR and instantiate it on a PaliGemma2 bi-encoder trained on 3.5M graded pairs curated from raw Walmart catalog data. A controlled graded-vs-binary ablation isolates the supervision granularity and shows lift of 4.9%-5.9% in NDCG@10. The same recipe applied to other
multimodal encoders lifts early-fusion backbones by up to 8.5% NDCG@10. On the public FashionIQ
benchmark, GradCIR (applied to PaliGemma2) reaches 0.6703 average recall when fine-tuned, slightly ahead of the strongest peer-reviewed supervised baseline we compare against, and matching or exceeding all published CLIP-L-class zero-shot CIR methods. The system is deployed in production at Walmart, where it's serving live visual-search user traffic.
Read the full abstract ▾
All authors Anubhav Gupta, Hrushikesh Mohapatra, Prijith Chandra, Asish Mohapatra, Anuj Garg, Arvind Maan, Sudip Datta, Venkat Bulusu, Sitesh Kumar Jalan
Computer Vision
STAR: Scene- and Task-Aware 4D Radar Preprocessing Towards End-to-End Cognitive Radar
Seung-Hyun Song, Dong-Hee Paek and Seung-Hyun Kong
Four-dimensional (4D) Radar has emerged as a key sensor for environmental perception, providing range, azimuth, elevation, and Doppler measurements while remaining robust to illumination changes and adverse weather conditions. However, conventional Radar preprocessing methods, such as constant false alarm rate (CFAR) detection, select measurements primarily based on signal-level criteria and may therefore discard information valuable for downstream perception during point cloud generation. In addition, existing 4D Radar perception pipelines typically optimize Radar data processing and downstream perception independently, preventing task objectives from directly guiding the preprocessing stage. To address these limitations, we propose a Scene- and Task-Aware Radar (
STAR) Preprocessor together with an end-to-end training framework. The
STAR Preprocessor incorporates scene context and downstream task objectives to generate task-relevant Radar points, enabling the Radar representation to be optimized directly for perception. On the K-Radar
benchmark, the proposed method achieves 74.3 AP, outperforming the previous state of the art by 5.6 AP points. Furthermore, applying the task-relevant points generated by
STAR to various existing 3D detectors improves detection performance in most evaluation settings and yields an overall positive average gain over point clouds produced by conventional preprocessing.
Read the full abstract ▾
Machine Learning
Acceptance-Aware Draft Model Training for Speculative Decoding
Tianhua Xia, Mugilan Ganesan, Yifei Feng et al. (+3 more)
Speculative decoding accelerates large
language model (LLM) inference by using a lightweight draft model to generate multiple candidate
tokens that are verified by the target model in a single forward pass. Its speedup is largely determined by the acceptance length, yet existing draft-model training methods mainly optimize cross-
entropy or Kullback-Leibler (KL) divergence as proxies. These objectives encourage distribution matching but do not directly optimize acceptance length, and the acceptance mechanism also differs between greedy and sampling-based decoding. In this work, we propose acceptance-length-aware training losses that directly optimize the expected number of accepted
tokens within a speculative window. For greedy verification, we derive an expected accepted length (EAL) loss that explicitly maximizes expected acceptance length. For sampling-based decoding, we introduce a window total variation (WTV) loss that optimizes the overlap between temperature-scaled draft and target distributions while accounting for sequential acceptance dependencies. Both objectives can be further combined with a group-relative reinforcement learning stage (GRPO) using simulated acceptance length as the reward. Experiments across different target and draft models, tasks, and decoding settings show that our losses consistently improve acceptance length over KL-based training. WTV provides particularly strong gains under sampling-based decoding, while EAL better matches greedy verification. These results show that directly optimizing the acceptance objective, with losses tailored to the decoding mode, is more effective than conventional distribution-matching objectives.
Read the full abstract ▾
All authors Tianhua Xia, Mugilan Ganesan, Yifei Feng, Haiyu Wang, Maximilian Egger, Sai Qian Zhang
Machine Learning
Mind or Message? Auditing Theory of Mind in Multi-Agent Social Simulation
Cong Li, Cheng Chen, Thomas Fung et al. (+2 more)
Language model
agents are increasingly used to simulate social interaction, and the resulting transcripts read as though the
agents understand one another. We ask whether that appearance rests on a model of the partner's mind or on the surface record of what the partner said. We build a social simulation in which both questions have exact answers: 40 multi-issue negotiations whose hidden preference weights and whose full Pareto frontier are known by construction. Two model families negotiate across 160 dyads, every transcript is frozen before any measurement, and 2880 counterfactual probes then hold the evidence byte identical while moving one factor at a time: the reader's own stake, the partner's tone, an identity label, and the order of recursion. The
agents are socially fluent and economically poor. They reach agreement in 96.2% of dyads with 0 protocol failures, yet only 0.7% of deals land on the Pareto frontier, they leave 20.5% of the available joint value unclaimed, and they miss the one issue on which their interests are perfectly aligned in 76.6% of deals; on the frontier and on that aligned issue, a package drawn at random from the set both sides would accept does as well. The probes locate the failure. Swapping only the reader's own payoff sheet, while the partner's words and offers stay identical, moves the inferred top priority by 15.0 percentage points, which is
egocentric projection rather than inference, while a tone rewrite moves it by 5.3 percentage points and an identity label by 0.0. Most tellingly, an
agent predicts what its partner believes about it 72.5% of the time while that partner's belief is itself correct only 51.2% of the time: the
agents track the conversation far better than they track the mind behind it.
Read the full abstract ▾
All authors Cong Li, Cheng Chen, Thomas Fung, Alex Rossi, Yi Li
Robotics
MimicAgent: Quadruped Skills via Prompt-to-Trajectory Generation
Lucky Kant Nayak, Narayanan Palghat Parameswaran, Neehar Peri et al. (+1 more)
We present MimicAgent, a
prompt-to-trajectory generation framework for learning dynamic quadruped skills. Although reward shaping is extensively used when training quadruped policies, navigating the resulting reward landscape is notoriously difficult, requiring hours of "graduate student descent". Eureka attempts to automate reward design with LLMs, but we find that it struggles to generalize across diverse skills and morphologies. Our key observation is that it is far easier for a human - and by association, an LLM - to generate reference motions than to shape reward functions. Our hypothesis is motivated by the success of example-guided RL for
humanoids, which exploits large-scale motion capture
datasets as references for training locomotion policies. Unlike
humanoids, quadrupeds lack such reference motion data. Towards this end, we propose MimicAgent, an agentic harness that, given a skill
prompt, generates quadruped reference trajectories with coding
agents. These coarse reference trajectories are then used to train example-guided RL policies that are deployable in simulation and in the real-world. Notably, we find that when prompting Claude Fable 5.1 within our agentic harness, 87% of
prompts yield semantically aligned reference trajectories.
Read the full abstract ▾
All authors Lucky Kant Nayak, Narayanan Palghat Parameswaran, Neehar Peri, Deva Ramanan
Machine Learning
Luck Is Not Skill: When Do Paired Rollouts Help Group-Relative RL of LLM Agents?
Nazmus Sakib
Group-relative reinforcement learning compares rollouts of the same
prompt, but independent environment noise can obscure these comparisons. We study paired rollouts, which share an event-keyed noise schedule within each group while preserving each rollout's marginal distribution. Pairing removes the between-schedule component of reward-contrast variance, but need not reduce gradient variance. For one-sided grader noise, we derive an exact condition for reduction and give a counterexample in which reward contrasts improve while gradient variance increases. A controlled study trains a 2B tool-use
agent under tool faults and grader flips, with three seeds per design. The protocol was registered with a disclosed, previously completed pilot. Under tool faults, pairing improves final noisy-test success by +5.1 percentage points on average, with all three seed differences positive, but misses the registered learning-curve criterion. The criterion is also missed under grader flips: the validation-AUC difference is +0.003 (95% interval [-0.029, +0.033]). A gradient probe on eight distinct checkpoints from two fault-trained trajectories finds lower mean-centered covariance traces under both noise types: 21 to 30% for grader flips and 40 to 63% for tool faults. These finite-sample measurements support the variance mechanism without establishing a general learning-speed benefit. The results distinguish improving reward comparisons, reducing
estimator variance, and improving learning.
Read the full abstract ▾
Numerical Analysis
A mesh- and horizon-robust preconditioner for finite element discretizations of volume-constrained nonlocal diffusion problems
Jiashu Lu, Yufeng Nie, Haolun Zhang et al. (+1 more)
In this paper, we develop a preconditioner that is robust with respect to both the mesh size and the interaction horizon for finite element discretizations of volume-constrained nonlocal
diffusion problems. Motivated by the
Fourier multiplier of the nonlocal operator, we construct the preconditioner $B_h=G_h+\beta\delta^2D_h^{-1}$, where $G_h$ is a symmetric local Poisson solver, $D_h$ is the lumped mass
matrix, and $\beta$ is a kernel-dependent constant. We rigorously prove that the spectrum of the $B_h$-preconditioned system is bounded above and away from zero uniformly with respect to the mesh size and horizon. Moreover, we develop a pair-
cell assembly for the finite element
matrix that preserves the
symmetry and positive definiteness required by the preconditioner and PCG, and establish an $L^2$-error estimate for the finite element discretization. Finally, we use a single symmetric
geometric multigrid V-cycle for $G_h$ in the numerical experiments, and the corresponding numerical results confirm the predicted convergence rate and show bounded condition numbers and PCG iteration counts over the tested meshes, horizons, and kernels.
Read the full abstract ▾
All authors Jiashu Lu, Yufeng Nie, Haolun Zhang, Pingrui Zhang
Machine Learning
CLOOPD: Closing the Learner Loop in On-Policy Distillation
Keye Zheng, Hanyu Li, Zhan Cheng et al. (+1 more)
On-policy
distillation (OPD) pays twice for each fresh batch: the student generates trajectories and a stronger teacher scores them. Existing methods improve which trajectories are scored and how the teacher signal is constructed, but usually consume it with one actor update. We introduce CLOOPD, a closed-loop framework separating teacher-signal acquisition from student-side realization. CLOOPD selects an adaptive $\alpha$ waypoint inside a KL envelope, freezes the scored batch and its advantages, re-forwards the student after each actor pass, measures realization, and allocates actor work under a separate
token budget. The framework includes deterministic two- and three-pass policies,
token-priced CLOOPD-TPMR, and a budget-matched control. Across six 300-step runs on an 8-H20 node, every CLOOPD policy improves the one-pass TOP-D anchor at comparable teacher-
token scale: macro accuracy rises from 15.41 to 17.78 with CLOOPD-Fixed2 and 19.36 with CLOOPD-Fixed3. At step 100, CLOOPD-Fixed3 reaches 15.35, nearly matching TOP-D at step 300 while using 67.2% fewer teacher-scored
tokens and 28.0% fewer GPU-hours. Earlier 8-A100 ablations show adaptive $\alpha$ eliminates observed trust-envelope violations; a third pass adds headroom. These results position CLOOPD as a framework for budgeting how fully students learn from teacher-scored
tokens.
Read the full abstract ▾
All authors Keye Zheng, Hanyu Li, Zhan Cheng, Yuan Gao
Robotics
BayesianGS-SLAM: Uncertainty-Aware Neural Rendering SLAM via Probabilistic Formulation
Kyeongsu Kang, Seongbo Ha, Sibaek Lee et al. (+1 more)
Neural-rendering-based SLAM relies on rendered RGB-D residuals for camera tracking and map optimization, but the reliability of these predictions can vary substantially because of sensor noise, limited observation coverage, and incomplete map representations. Without an explicit reliability estimate, unreliable residuals may adversely affect pose optimization, while frames already well explained by the current map may trigger redundant mapping updates. In this paper, we present BayesianGS-SLAM, an uncertainty-aware 3D Gaussian Splatting SLAM framework that estimates predictive color and depth uncertainty during mapping and consistently reuses it across the SLAM pipeline. Our tractable probabilistic formulation combines a sensor-noise uncertainty component with an opacity-induced map-representation component propagated through the rendering process. The resulting predictive uncertainty is used to augment mapping, normalize tracking residuals through a robust pose objective, and evaluate incoming frames using a predictive-surprise-based keyframe criterion. Unlike prior uncertainty-aware neural-rendering SLAM methods that primarily consider color uncertainty or use uncertainty only during mapping, our framework estimates predictive uncertainty for both color and depth and integrates it into mapping, tracking, and keyframe selection. Evaluations on real-world RGB-D
datasets demonstrate substantially improved depth uncertainty-error ranking compared with existing uncertainty-aware SLAM methods. Moreover, the proposed keyframe-selection strategy reduces the number of selected keyframes and mapping calls while maintaining competitive tracking and rendering performance.
Read the full abstract ▾
All authors Kyeongsu Kang, Seongbo Ha, Sibaek Lee, Hyeonwoo Yu
Machine Learning · Cross-listed from eess.AS
P2Flow: Phoneme-aware Progressive Flow Matching for Extreme Speech Super-Resolution
Ningyuan Yang, Yize Li, Pu Zhao et al. (+5 more)
Generative models have recently demonstrated considerable promise in
speech super-resolution (SSR). Nevertheless, the majority of existing work has concentrated on standard or versatile SSR configurations, leaving the extreme setting with severely limited spectral inputs largely unexplored. In this regime, current approaches exhibit marked performance degradation, underscoring the need for dedicated solutions. To bridge this gap, we introduce P2Flow, a phoneme-aware progressive flow matching (FM) framework designed for extreme SSR with three main strategies. First, our model leverages phonetic information to reconstruct missing spectral components. Furthermore, it employs a progressive architectural design that hierarchically restores distinct frequency regions. Finally, we incorporate post-training of the vocoder to enhance overall waveform fidelity. Extensive experiments are conducted on the TIMIT and VCTK
datasets under both 1 kHz to 16 kHz and 2 kHz to 16 kHz settings, demonstrating that P2Flow yields state-of-the-art results across multiple evaluation metrics.
Read the full abstract ▾
All authors Ningyuan Yang, Yize Li, Pu Zhao, Diego A. Cuji, Kanad Sarkar, Ryan M. Corey, Xue Lin, Andrew C. Singer
Databases
Data Agents: Agentic Data Systems
Guoliang Li, Peiyao Zhou, Xuanhe Zhou et al. (+3 more)
Traditional data systems face profound limitations in the AI era, relying on human-crafted pipelines, lacking semantic understanding of heterogeneous data, and operating through rigid, reactive processing. To address these challenges, we propose a new paradigm called the Data
Agent, designed to manage, process, and analyze data with minimal human intervention. Data
agents autonomously execute a wide range of data-related tasks, transforming traditional data systems by shifting from manual design to autonomous orchestration, from literal manipulation to semantic interpretation, and from reactive to proactive processing. Our Data
Agent system includes six components: semantic data organization, semantic operators, agentic pipeline orchestration and optimization, feedback-driven refinement,
memory management, and proactive adaptation. Building on this foundation, we also develop two specialized
agents: the data analytics
agent and the data science
agent. Experiments on real
benchmarks demonstrate significant performance gains of our data
agent over state-of-the-art methods. We identify open challenges to guide future research in building fully autonomous data systems.
Read the full abstract ▾
All authors Guoliang Li, Peiyao Zhou, Xuanhe Zhou, Ji Sun, Yuyu Luo, Ju Fan
Computer Vision
The Visual Target Matters: Learning across the Visual Hierarchy for Brain-to-Image Retrieval
Ye Wang, Haokun Ren, Hong Yu et al. (+4 more)
Brain-to-image
retrieval seeks to identify the visual stimulus that elicited a non-invasive neural response. Candidate images are typically represented by pretrained
vision models, whose internal representations vary in abstraction across depth. Existing methods usually train the neural encoder to recover a fixed final-layer visual target. Under this formulation, the visual hierarchy is reduced to a single prescribed endpoint, preventing representations at other depths from directly shaping the visual target. This limitation motivates learning how information across visual depths should contribute to the
retrieval target. To this end, we introduce NeuroGlyph, which learns a trial-independent visual target from multiple depths of a frozen visual backbone. NeuroGlyph decomposes the target into factor-specific subspaces. Each subspace learns an image-conditioned allocation over visual depth. The resulting subspaces are fused into a single
embedding for
retrieval. Across THINGS-
EEG and THINGS-MEG, NeuroGlyph outperforms final-layer supervision in all controlled comparisons. It also surpasses the post hoc best fixed-layer oracle in three of four comparisons. Parameter-matched ablations support both factorized target construction and image-conditioned depth allocation. Under comparable 200-way
retrieval protocols, NeuroGlyph achieves the strongest system-level performance in six of eight reported metrics. These results support learning
retrieval targets across the visual hierarchy rather than prescribing one visual depth.
Read the full abstract ▾
All authors Ye Wang, Haokun Ren, Hong Yu, Ruirui Li, Xiao Li, Ke Liu, Wei Wu
Cryptography and Security
Monet: Measuring the Ecosystem of Open-Source Text-to-Image Models Tailored for Harmful Services
Zihao Wang, Jiacen Xu and Zilong Lin
The open-source text-to-image (T2I) ecosystem enables rapid model development and sharing, but also hosts models intentionally tailored for harmful services, which we call Monets. Prior work has examined specific types of harmful T2I models on individual platforms, but a Monet does not exist in isolation. The broader Monet ecosystem, spanning model characteristics, cross-platform propagation, governance evasion, monetization, and downstream deployment, remains poorly understood. In this study, we present the first systematic, ecosystem-level measurement of Monets. Grounded in the policies of real-world model hubs, we construct a taxonomy of ten harmful service categories and identify 23,947 Monets across eight major T2I model hubs, with the most popular exceeding 19 million downloads. While some developers employ anti-theft mechanisms against unauthorized re-uploading, Monets propagate across platforms at scale, with 40.76% mirrored across hubs. Such propagation further enables governance evasion via cross-platform archiving, keeping 11.99% of Monets accessible after bans on their original platforms, alongside other evasion strategies including keyword obfuscation and model-level safeguard circumvention. Monets also anchor coordinated commercial campaigns—one spanning 668 models with 914 completed commissions and another advertising gray-
market account-farming service—and reach users through GitHub projects and inference APIs, raising downstream child safety concerns. These findings expose the limitations of platform-siloed defenses and highlight the need for cross-platform threat intelligence, coordinated governance, and technical safeguards.
Read the full abstract ▾
Robotics
Phrase-Level Robotic Guqin Performance: Bimanual Motion Planning and Audio-Tactile Interaction Monitoring
Zhen Wang, Zhiheng Chen, Tianyuan Bao et al. (+1 more)
Recent advances in
humanoid robotics and
embodied intelligence have enabled
robots to perform increasingly complex manipulation tasks. However, musical instrument performance remains a formidable
benchmark, demanding not only collision-free trajectory execution but also precise contact timing, asymmetric bimanual coordination, and target acoustic outcomes on physical instruments. The guqin, a seven-string fretless zither, presents unique manipulation challenges due to its millimetric string spacing, transient right-hand plucking, and sustained left-hand harmonic contacts. In this work, we present a physical heterogeneous dual-arm robotic system for phrase-level autonomous guqin performance. We formulate guqin playing as a hybrid discrete–continuous execution problem and develop a hierarchical planning framework that coordinates working finger assignment, configuration continuity, obstacle avoidance, and tight bimanual contact schedules across consecutive musical events. The system integrates
vision-guided instrument localization, tactile-based harmonic contact monitoring, and auditory feedback-informed plucking parameter calibration. Real-world experiments on a 25-event phrase demonstrate that the system reliably executes coordinated open-string and seventh-hui harmonic sequences on a physical guqin, achieving 93.6% and 96.8% event correctness across repeated trials.
Read the full abstract ▾
All authors Zhen Wang, Zhiheng Chen, Tianyuan Bao, Tianwei Zhang
Artificial Intelligence
Self-Healing Harness for Runtime Oversight of Agent Self-Modification
Sina Tayebati, Divake Kumar, Nastaran Darabi et al. (+2 more)
LLM
agents can change their own future behavior, raising a basic control question of which self-generated changes should be allowed to persist. We formulate this as admission control for self-modification. The
agent may propose changes to its operating instructions, while an external runtime gate controls persistence. We implement this principle as a model-agnostic self-healing harness that runs a Detect, Notice, Heal, Validate loop around an otherwise unmodified
agent. The
agent authors candidate behavioral rules in an external workspace, where they receive provisional execution authority during evaluation and acquire persistent cross-episode authority only after measured improvement on the triggering failure without regression beyond a fixed margin on protected cases. Replay provides matched evidence when available, forward trials provide a weaker fallback, and a corpus-level guard re-tests the accumulated active rule set. Across 16 matched Baseline and Harness runs spanning AppWorld, Terminal-Bench, and $\tau^2$-Bench, the gate rejected 383 replay-decided proposals. Of these, 211 (55%) improved their triggering failure while degrading a case that previously worked. This shows that locally beneficial self-modifications can introduce collateral regressions often enough to materially affect gate decisions, providing direct empirical motivation for external admission control. Task-completion score is higher under the Harness in all 16 pairs, with two paired bootstrap intervals excluding zero, while repeated-trial reliability is higher in 12 pairs, tied in 4, and lower in none. Because adaptation modifies the policy-inducing context while leaving model weights fixed, admitted changes remain inspectable, reversible, and compatible with closed-weight models.
Read the full abstract ▾
All authors Sina Tayebati, Divake Kumar, Nastaran Darabi, Ranganath Krishnan, Amit Ranjan Trivedi
Machine Learning · Cross-listed from stat.ML
OSCAR: Order-aware Sensitivity Correction for AI Rankings
You Liu, Yue Liu, Quanchao Lu et al. (+2 more)
LLM judges evaluate candidate
language models through pairwise comparisons of their responses. Judge-aware Bradley–Terry models aggregate pairwise comparisons by jointly estimating candidate scores and judge-specific sensitivities (how strongly each judge responds to quality differences), often interpreted as reliability. We show that this reading fails when judges prefer a display position. Even under balanced display orders or swap-and-average, omitting position attenuates fitted judge sensitivity by approximately the factor $1-D^2$ near equal quality, where $D$ is the judge's net first-slot preference between equally good responses. We propose OSCAR, which jointly estimates candidate scores, judge sensitivities and judge-specific position, length and family effects, and provides confidence intervals for pairwise win probabilities that account for dependence within
prompts. On held-out data, observed attenuation closely matches the prediction $1-D^2$ (correlation of log ratios 0.99). Across four
datasets, position adjustment lowers held-out log loss by 14-23% and changes which judges would be dropped for low sensitivity on three
datasets. On MT-Bench, it ranks GPT-4 first instead of claude-v1, matching human votes. Simulations with known parameters show improved recovery of judge sensitivities and candidate rankings, with 94.4-95.2% confidence interval coverage. Low estimated sensitivity alone is therefore not evidence of an unreliable judge.
Read the full abstract ▾
All authors You Liu, Yue Liu, Quanchao Lu, Ergan Shang, Nick Shipilov
Computer Vision
Action-Slot: Structured Action-Centric Representation Learning for Multi-Agent Atomic Activity Understanding
Yu-Ho Chang, Chi-Hsi Kung, Yi-Hsuan Tsai et al. (+1 more)
Atomic activity understanding aims to recognize and localize structured traffic behaviors that jointly encode motion patterns and their grounding in road
topology. Unlike conventional action recognition, atomic activities are multi-
agent, multi-label, and
topology-aware: multiple activities co-occur while many
agents remain inactive. We introduce Action-Slot, a structured action-centric representation learning framework. Slot
attention is widely used for object-centric decomposition, but its permutation-invariant design and object-level inductive bias are misaligned with atomic activity semantics. We reformulate slot learning as structured activity decomposition through three designs: (1) category-aligned action slots that anchor slots to predefined activity categories, (2) parallel spatio-temporal slot updating for holistic
video-level
reasoning, and (3) background and negative-slot regularization that enforces competition between foreground activities and irrelevant regions. Together these establish an activity-centric inductive bias that disentangles concurrent and asynchronous activities directly from raw
video. Beyond recognition, the learned representations encode transferable spatio-temporal grounding signals. We further propose an
attention-difference-based pseudo mask selection framework that suppresses false positives by measuring
attention changes before and after candidate region removal, enabling weakly supervised localization without dense annotations. To support systematic evaluation, we introduce TACO, a balanced synthetic
dataset with full atomic activity coverage and pixel-level annotations. Experiments on OATS, TACO, and annotated nuScenes show superior recognition, strong sim-to-real transfer, and state-of-the-art weakly supervised localization.
Read the full abstract ▾
All authors Yu-Ho Chang, Chi-Hsi Kung, Yi-Hsuan Tsai, Yi-Ting Chen
Machine Learning · Cross-listed from stat.ML
Model-Agnostic Feature Selection via LOCO-Guided Adaptive Minipatch Sampling
Xuhui Liu and Lili Zheng
Black-box machine learning models increasingly deliver strong predictions, but extracting useful information from them, such as a set of important features, remains challenging. Existing model-agnostic methods primarily estimate feature importance or conduct inference on it rather than directly selecting features, whereas many feature selection methods are model-specific or rely on the model-X assumption. We introduce LOCO-guided Adaptive Minipatch Sampling (LAMPS), a model-agnostic ensemble framework that uses any black-box regression
algorithm as its base learner to select features important for predicting the response. The base learner need only produce predictions and need not perform feature selection itself. LAMPS operates within a minipatch ensemble framework that subsamples both observations and features, allowing leave-one-
covariate-out (LOCO) feature importance scores to be easily computed. It adaptively concentrates minipatch sampling on features with high LOCO scores while maintaining exploration. The resulting sampling probabilities rapidly separate signal from noise features after a few iterations, enabling selection through simple thresholding. We establish that LAMPS achieves exact feature selection in high-dimensional settings, provided that the base predictive models are sufficiently well trained on average. Extensive experiments on synthetic and real data show that LAMPS outperforms state-of-the-art feature selection methods, with particularly strong performance in the presence of correlated features.
Read the full abstract ▾
Computer Vision
Positive Pair Geometry Matters: Optimal Transport for Contrastive Learning of Visual Representations
Akshit Nanda, Shahzad Ahmad and Ram Prasad Padhy
Contrastive self-supervised learning has achieved strong performance by learning representations from multiple augmented views of the same image. However, most existing methods construct positive pairs using independently sampled
stochastic augmentations, which may alter semantic content and ignore the intrinsic geometry of the data distribution. In this work, we propose OTCLR, an optimal transport-aware framework for contrastive learning representations that generates geometry-consistent positive samples. Instead of directly contrasting two randomly augmented views, we construct intermediate views between the original image and its augmented variants through entropic optimal-transport displacement interpolation. These transport-interpolated samples serve as positive views that better preserve image structure while explicitly modeling spatial distributional geometry. To further promote smooth representation learning, we evaluate auxiliary Sinkhorn regularization terms that encourage transport-interpolated views to remain consistent with their endpoint images. The proposed method can be incorporated into standard contrastive learning pipelines without modifying the encoder architecture. Experiments on multiple
benchmark datasets show that our approach improves representation quality and transfer learning performance compared with conventional augmentation-based contrastive learning baselines.
Read the full abstract ▾
Robotics
ActiveArena: Benchmarking and Understanding Active Perception in Robotic Manipulation
Yibo Li, Enshen Zhou, Rui Chen et al. (+7 more)
Active perception and manipulation are crucial for
robots to interact with complex scenes. Existing
benchmarks struggle to evaluate how
robots effectively acquire and maintain information in
memory in an active manner. To this end, we introduce ActiveArena-Sim, an active-perception simulator with controllable viewpoints and large-scale workspaces as the foundation. Built on this, we propose ActiveArena-Bench, which comprises 35 tasks across 5 fine-grained categories, covering visual exploration and interactive information acquisition. Each task is difficult to solve from passive observations alone, requiring multi-round evidence acquisition and
memory-based
reasoning. The
benchmark provides rich
memory annotations, standardized training data, and ID/OOD protocols featuring disjoint scenes, unseen distractor configurations, and novel backgrounds. Moreover, we present ActiveArena-VLA, a modular suite of 13
vision-
language-action configurations for controlled studies of
memory writing,
memory capacity, proprioceptive state, subtask supervision, and high-level planning in active perception.
Benchmark results reveal a substantial ID-OOD gap: uniform
memory sampling, increased
memory capacity under reliable write policies, proprioceptive inputs, and subtask supervision improve OOD generalization, while planner-guided
memory management and decision-making achieve performance close to the best-performing configuration using only sparse
memory. ActiveArena thus provides a unified testbed to develop and diagnose models for active perception and manipulation.
Read the full abstract ▾
All authors Yibo Li, Enshen Zhou, Rui Chen, Yanjun Ding, Mengzhen Liu, Yi Han, Jiabo Zhan, Lipeng Wang, Shanghang Zhang, Lu Sheng
Computation and Language
Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines
Aviral Joshi, Hanoz Bhathena, Max Nelson et al. (+1 more)
Retrieval-augmented generation (RAG) is hard to monitor in production: exhaustive relevance labels do not exist for non-stationary multi-million-passage corpora that re-index in real time. As a result,
retrieval quality is generally understudied and often deprioritised in favour of generation-oriented metrics. In this work, we propose
auditing retrieval coverage by probing for evidence of missing documents rather than enumerating every relevant one. Our method Re:CAP (
REtrieval Coverage Audit by iterative Probing) is a reference-free audit loop applied to a deployed RAG pipeline's initial answer and retrieved context: it identifies the topics already covered, generates probing questions for plausibly missing topics, retrieves candidate documents, and applies an LLM-as-judge to retain only those that introduce previously-unretrieved information. On four public
benchmarks, Re:CAP recovers 9-29% of gold labels that flat BM25 top-500 cannot reach, rising to 48% on TREC-COVID. On MuSiQue Re:CAP beats flat hybrid top-500 by +12.9 pp on recall at less than half the document budget. An ensemble BM25, dense, and hybrid baseline (top-500 each) still leaves out 21.2% of gold docs on TREC-COVID that Re:CAP recovers; human annotators judge that 78.9% of those structurally distinct documents add new information to the baseline answer (Fleiss $\kappa$ = 0.79, n = 123), and 73.9% on live production traffic (n = 180). End-to-end recall is reproducible to within $\pm$1% across three independent runs, making Re:CAP a stable instrument for periodic
retrieval audits.
Read the full abstract ▾
All authors Aviral Joshi, Hanoz Bhathena, Max Nelson, Saket Sharma
Information Theory
Exact Common Information and Exact Channel Synthesis for Correlated Discrete Sources
Lei Yu
In this paper, we provide a single-letter characterization of the exact common information for a pair of correlated discrete sources, as well as a single-letter characterization of the admissible region for the shared randomness rate and the communication rate in exact channel synthesis.
Read the full abstract ▾
Robotics
CARE: Experience-Guided Atomic Corrective Execution for Vision-Language-Action Policies
Junlan Xiao, Junwei Jiang, Zaibin Zhang et al. (+4 more)
Vision-
Language-Action (VLA) policies achieve strong performance in robotic manipulation but remain brittle once execution deviates from nominal trajectories. We propose CARE (Corrective Atomic Robotic Execution), a framework that improves recovery by learning from failures encountered during execution. Instead of generating corrective data from manually designed or random perturbations, CARE collects failed rollouts, models stage-conditioned post-failure deviations, and uses the resulting empirical distributions to synthesize representative failure states and corrective demonstrations. At inference time, CARE combines stage-wise planning with physically grounded 3D monitoring to trigger atomic adjustments or re-operations while preserving task progress. We further introduce the Failure State Recovery
Benchmark (FSR-Bench), which evaluates recovery from intermediate failure states under local deviations and structural anomalies. Experiments across multiple VLA backbones, simulation
benchmarks, and real-world dual-arm tasks show consistent improvements, with average task-success gains of 14.5 points in simulation and 15.9 points in the real world. Code, models, and data are available at https://github.com/xiaojunlan/care
Read the full abstract ▾
All authors Junlan Xiao, Junwei Jiang, Zaibin Zhang, Yifan Wang, Zhongbo Zhang, Huchuan Lu, Lijun Wang
Machine Learning
PAC-Bayesian Meta-Learning for Few-Shot Identification of Linear Dynamical Systems
Chenfeng Huang and George Michailidis
Identifying linear time-invariant (LTI) dynamical systems is challenging when trajectories are short, noisy, or high-dimensional. Traditional system identification typically treats each system independently and cannot exploit shared structure across related systems. We propose PBML-LTI, a PAC-
Bayesian meta-learning framework for few-shot LTI system identification that learns a transferable prior over task-specific dynamics while preserving task heterogeneity. Each task corresponds to an unknown LTI system, and the meta-learner uses training trajectories to learn a data-dependent prior over transition
matrices. For a new system with limited data, PBML-LTI performs
Bayesian adaptation under this prior to obtain a task-specific posterior, providing accurate estimates and principled uncertainty quantification. A key challenge is temporal dependence, since LTI trajectories violate the i.i.d. assumptions underlying most PAC-Bayes meta-learning analyses. We address this with a martingale PAC-Bayes analysis for dependent trajectory losses and derive a support-query predictive-risk bound that motivates a fit-KL meta-training objective. The bound clarifies the roles of empirical fit, posterior complexity, and prior quality in few-shot adaptation under sequential dependence. We further derive corollaries for transition-
matrix recovery and multi-step trajectory prediction, connecting uncertainty-aware meta-identification with finite-sample guarantees for dependent dynamical data.
Read the full abstract ▾
Computer Vision
Patch-to-Global: Random Patch Diffusion for Globally Consistent Megapixel Artifact Inpainting in Whole Slide Images
Hyeseong Lee, Eunsu Kim, D M Bappy et al. (+6 more)
Although deep learning has advanced Whole Slide Image (WSI) Analysis, tissue artifacts like bubbles and folds often cause silent failures by concealing essential morphology. Current pathology image restoration methods are mostly restricted to small patches, struggling to maintain global structural coherence at a megapixel scale. We introduce RestorePath, a framework for globally consistent megapixel scale inpainting that reconstructs diagnostic structures in histological image to prevent incorrect high-confidence predictions and lower error rates. Our model utilizes a Latent
Diffusion Model (LDM) conditioned on Pathology Foundation Model (PFM)
embeddings, integrating Large Kernel
Attention (LKA) to manage long-range dependencies during random patch
diffusion. Enhanced by Distance-Weighted Interpolation (DWI) and an Adaptive Guidance Scale (AGS), RestorePath ensures structural consistency and fidelity by modulating information from surrounding patches. Evaluations across TCGA-BRCA, BACH, and Camelyon16
datasets for images ranging from 512 to 4608 pixels demonstrate state-of-the-art performance in maintaining histological consistency. RestorePath significantly improves downstream Computational Pathology (CP) tasks, outperforming both raw artifact images and the conventional Detect-and-Discard (D&D) approach. The code is available at https://github.com/PathfinderLab/RestorePath
Read the full abstract ▾
All authors Hyeseong Lee, Eunsu Kim, D M Bappy, Ho Heon Kim, Youngsuk Lee, Se Young Chun, Jang-Hwan Choi, Sung Hak Lee, Sangjeong Ahn