Computer Vision
SparkDiffusion: Mitigating the High-Sparsity Trap — A Unified Framework for up to $265\times$ Single-GPU Acceleration of Visual Generation
Yuxi Liu, Haoyu Li, Zekun Zhang et al. (+12 more)
Video diffusion transformers are expensive because
attention dominates long spatiotemporal
token sequences. We identify the high-sparsity trap: at extreme
attention sparsity, step-local training losses keep decreasing while terminal generation quality stagnates or degrades. The trap is one of supervision: the dominant terminal errors originate in the high-noise structure-generation stage, and terminal-aligned training corrects terminal errors that substantially extended step-local training cannot. This yields a simple staging principle: first adapt the sparse architecture into a coarse prior, then correct the terminal distribution. We instantiate the principle as \method, a unified acceleration framework for visual generation that combines a short sparse warm-up, few-step trajectory-mixed
distillation, and FP8 quantization with fused kernels. \method sustains $97\%$
attention sparsity with strong visual quality on long-sequence 720P generation across Wan2.1/Wan2.2 backbones and T2V/I2V tasks, and $90\%$ sparsity on Wan2.1-T2V-1.3B-480P. With 3-step CFG-free inference, \method achieves a $265\times$ end-to-end speedup over the 50-step CFG dense baseline for Wan2.1-T2V-14B-720P on a single RTX~5090 ($220\times$ on H100), and denoises a Wan2.1-T2V-1.3B-480P
video in $1.3$s.
Read the full abstract ▾
All authors Yuxi Liu, Haoyu Li, Zekun Zhang, Tengxu Sun, Yixiang Cai, Jiayong Li, Yifei Xia, Tianle Liu, Baole Ai, Ang Wang, Jiamang Wang, Lin Qu, Kai Zhang, Kun Yuan, Bin Cui
Computer Vision
QwenVLConnector: A Fast, Unified Medical VLM Chatbot for Fine-Grained Clinical Perception and Text Generation
Le Thien Phuc Nguyen, Thien Nguyen, Thanh-Huy Nguyen et al. (+3 more)
Most medical
vision-
language models (VLMs) excel at open-ended report generation and VQA but provide limited support for structured, fine-grained clinical perception within a unified interface. We present QwenVLConnector, a Qwen2.5-VL-based medical chatbot that unifies classification, multi-label classification, textualized detection, counting, regression, and free-form report generation under a single next-
token objective. Our key component is a lightweight dense multi-layer Connector that aggregates low- and high-level visual features, aligns them through the pretrained
vision Merger, and fuses them with the final visual representation without increasing sequence length. This design enriches visual
tokens with complementary spatial and semantic cues while preserving efficiency. On
FLARE-2D, QwenVLConnector improves detection F1 from 0.55 to 0.85, raises single-label classification from 0.37 to 0.51, and boosts report-generation GREEN by up to 18.3 points over the Qwen2.5-VL baseline. We further explore
multimodal in-context learning for report generation, showing additional improvements without updating model parameters. Overall, QwenVLConnector offers a unified and efficient framework for combining structured medical perception with open-ended clinical text generation. Our code can be found at https://github.com/plnguyen2908/QwenConnector.
Read the full abstract ▾
All authors Le Thien Phuc Nguyen, Thien Nguyen, Thanh-Huy Nguyen, Gia Minh Hoang, Anh Mai Vu, Ulas Bagci
Computer Vision
MM-ContextFold: Context Folding for Multimodal Agentic Retrieval
Yang Tian, Fan Liu, Jingyuan Zhang et al. (+3 more)
Multimodal Agentic
Retrieval (MAR) requires
agents to solve complex information-seeking tasks by iteratively invoking external tools. Typical frameworks such as ReAct maintain raw
multimodal inputs and the accumulating interaction history in a single, ever-growing context, leading to the context explosion problem. While existing methods alleviate this issue by compressing redundant text, effective strategies for managing
token-intensive visual content remain largely underexplored. To address this gap, we first conduct a systematic empirical study of approximately 10,000 trajectories. The results show that as visual cues are progressively extracted through external tools and textualized into the context, raw images become increasingly redundant. Continued image retention is associated with higher output
entropy and can even degrade task accuracy. Motivated by these findings, we propose MM-ContextFold, a training-free framework that loads raw images only when needed. It maintains a persistent, text-only main context for high-level planning and spawns ephemeral branch contexts for image-dependent subtasks. Within each branch, the
agent loads the relevant images, completes the subtask, and folds the result back into the main context as a concise textual summary; the images and branch trace are then discarded. Experiments on seven MAR
benchmarks across five backbone models show that MM-ContextFold improves average accuracy by 6.3 percentage points over ReAct while reducing the working context length by 27.5%.
Read the full abstract ▾
All authors Yang Tian, Fan Liu, Jingyuan Zhang, Zhenyang Li, Yupeng Hu, Liqiang Nie
Computer Vision
Zan Wang, Yunhe Feng, Dong Nie et al. (+4 more)
Referring
surgical video instrument
segmentation (RSVIS) aims at segmenting the instrument in a
surgical video, given a textual description. Despite recent progress, current models are trained and assessed on relatively small-scale
benchmarks, hindering the development of more general RSVIS. In addition, existing
benchmarks support only the single-target expression that refers to one instrument in the
video, while overlooking multi-target and no-target referring expressions, restricting the applicability of RSVIS in practical scenarios. Addressing these issues, we propose LD-RSVIS, a new
benchmark aiming to facilitate more robust and general RSVIS. Specifically, LD-RSVIS consists of 3,536
surgical videos with 1.09 million frames and covers a broad set of 30 instrument classes from 25 various procedures. By including abundant
videos and classes, LD-RSVIS could benefit large-scale training and evaluation of more general RSVIS methods. Besides, unlike existing
datasets, LD-RSVIS offers diverse referring settings, including no-target, single-target, and multi-target expressions, which enables the development of more practical RSVIS models in real applications. In order to ensure high-quality annotations, all
videos in LD-RSVIS are manually labeled with multiple rounds of inspection and refinement. To our knowledge, LD-RSVIS is the largest and most diverse
benchmark for RSVIS. To analyze LD-RSVIS and to provide comparison for future research, we evaluate 12 representative methods, and the results reveal that more efforts are required for improvements. To encourage future research, we present a simple yet effective RSVIS method, dubbed Cascade-RSVIS, that first mines target-specific cues using the complementary multi-cue text information and then employs such cues and textual information for
segmentation, achieving promising performance. Our
benchmark and code will be released.
Read the full abstract ▾
All authors Zan Wang, Yunhe Feng, Dong Nie, Oluwatosin Oluwadare, Kewei Sha, Yan Huang, Heng Fan
Computer Vision
HDMamba-YOLO: Efficient State-Space Perception and Local Spatial Reconstruction for UAV Small Object
Linduo Wei, Junjie Fan, Yijun Mai et al. (+1 more)
Small-object detection in
UAV imagery is challenged by weak visual evidence, ambiguous boundaries, dense object distributions, and complex backgrounds. Effective detection therefore requires long-range contextual information for target-background discrimination while preserving explicit local two-dimensional structures for accurate localization. These requirements arise at different stages of the detection pipeline and are not naturally addressed by a uniform feature-processing strategy. We propose Hybrid Dual-domain Mamba-YOLO (HDMamba-YOLO), a stage-wise heterogeneous SSM-CNN detector organized according to a perception-reconstruction-
alignment-interaction rationale. EfficientVMamba-based EVSS establishes long-range contextual perception in the backbone, while PhasePatchMerging2D provides phase-aware hierarchical transitions. DST-Wrapper and Native C3k2-ASSAF then perform perception-to-reconstruction transition and repeated local two-dimensional reconstruction during FPN/PAN aggregation. DySample provides content-adaptive cross-scale resampling, while OS-CVTIA introduces macro-micro interaction and task-specific modulation for localization and classification. On VisDrone2019, HDMamba-YOLO-B achieves 42.737% mAP50 and 25.713% mAP50:95 with 10.042M parameters and 29.879 corrected GFLOPs. HDMamba-YOLO-Lite achieves 41.140% mAP50 and 24.741% mAP50:95 with 5.344M parameters. Under the unified AI-TOD evaluation protocol, HDMamba-YOLO-B obtains 21.621% AP and 47.881% AP50. Controlled ablations further support the stage-wise allocation of state-space perception, convolutional reconstruction, dynamic
alignment, and task interaction for
UAV small-object detection.
Read the full abstract ▾
All authors Linduo Wei, Junjie Fan, Yijun Mai, Yong Qi
Computer Vision
VDGS: Visibility-Driven Large-Scale 3D Gaussian Splatting for Aerial Scene Reconstruction
Haolin Yu, Jiadong Tang, YiXian Wang et al. (+5 more)
Large-scale scene reconstruction is a critical foundational technology in robotic autonomous systems such as 3D mapping and autonomous driving. In recent years, 3D Gaussian Splatting (3DGS) has demonstrated remarkable advantages in both visual quality and computational efficiency, making it a promising representation for large-scale scene reconstruction. However, it still faces challenges in large-scale scenes, including excessive
memory consumption and uneven viewpoint coverage caused by
UAV acquisition, limiting its real-world applications. To address this, we propose VDGS, a novel 3DGS framework that incorporates camera distribution into scene modeling. VDGS introduces visibility-driven statistics for scene anchors to quantify supervision strength. These statistics are further leveraged for scene partitioning and for gradient compensation in under-optimized regions, thereby promoting balanced optimization across different regions. Extensive experiments on multiple large-scale aerial scene
datasets demonstrate that, under imbalanced viewpoint distributions, VDGS consistently outperforms existing methods, while maintaining competitive performance in scenarios with more uniform view distributions.
Read the full abstract ▾
All authors Haolin Yu, Jiadong Tang, YiXian Wang, Yu Gao, Shi He, Zhilin Lai, Yi Yang, Mengyin Fu
Computer Vision
BrainIAC: Interactive 3D Brain Lesion Segmentation across Heterogeneous MRI Modalities with Online Adaptation
Wentian Xu, Anthony P Addison, Ziyun Liang et al. (+3 more)
Brain lesion
segmentation is a fundamental task in medical image analysis, playing a critical role in diagnosis, treatment planning, and longitudinal disease monitoring. Yet existing models still struggle to meet the demands of real clinical use, where deployments contain data distribution shifts, arising from differences in scanner hardware, imaging protocol (varying MRI modality sets), and new pathologies. We present BrainIAC (
Brain lesion Interactive Adaptive Continuously learning
segmentation), a unified framework that integrates (i) a multi-modal backbone network trained to segment multiple types of
brain lesions and handle heterogeneous sets of modalities via zero-filling and random modality dropping; (ii) 3D interactive
segmentation with bounding-box and click
prompts that preserves fully automatic prediction when no
prompt is given; and (iii) an online adaptation mechanism combining Mid-Interaction adaptation and Post-Interaction adaptation, supervised by the network's own predictions as pseudo labels and guided by an extra Click-Centered Gaussian loss. To our knowledge, this is one of the first 3D online adaptation methods for interactive
segmentation, and the first to combine handling of heterogeneous modality sets with online adaptation. Experiments across seven
brain MRI
datasets demonstrate that the proposed components provide complementary and synergistic benefits. The method consistently outperforms existing approaches and generalizes well across heterogeneous imaging modalities, including those unseen during training, as well as previously unseen
brain pathology types. The code and a 3D Slicer plug-in will be released at https://github.com/WenTXuL/BrainIAC upon publication.
Read the full abstract ▾
All authors Wentian Xu, Anthony P Addison, Ziyun Liang, Harry Anthony, Guang Yang, Konstantinos Kamnitsas
Computer Vision · Cross-listed from eess.IV
SomaNet: Weakly Supervised Learning for Instance Soma Segmentation in 3D Electron Microscopy with Partial Annotations
Mohammad Khateri, Morteza Ghahremani, Jussi Tohka et al. (+1 more)
Soma instance
segmentation, i.e., identifying and delineating individual
cell somas as distinct instances, is crucial for cellular analysis and connectomic reconstruction. Three-dimensional electron microscopy (3D EM) provides nanometer-scale resolution for capturing fine-grained soma morphology. However, dense instance-level manual annotation is prohibitively costly, limiting the scalability of fully supervised methods. To address this challenge, we propose SomaNet, a weakly supervised framework for 3D EM soma instance
segmentation under partial annotation constraints. SomaNet adopts a teacher–student learning paradigm tailored to partial labels. The teacher is trained using partially annotated data to generate pseudo-labels, while the student jointly learns from the partial ground-truth annotations and the generated pseudo-labels, progressively recovering dense instance
segmentations. To accurately delineate
cell somas under limited supervision with varying instance counts, SomaNet incorporates affinity learning, which encourages high similarity within instances and low similarity across instance boundaries. Semantic-guided affinity decoding and 2D-to-3D reconstruction then produce volumetrically consistent 3D soma instances while preserving the large receptive fields of 2D backbones. The framework is architecture-flexible and supports diverse backbones, including
vision transformers and foundation models, enabling direct transfer of pretrained visual representations to volumetric EM
segmentation. Experiments on 3D EM
brain datasets demonstrate that SomaNet achieves accurate and robust soma instance
segmentation across regions with diverse soma morphologies under partial annotation. Code is available at https://github.com/mkhateri/SomaNet.
Read the full abstract ▾
All authors Mohammad Khateri, Morteza Ghahremani, Jussi Tohka, Alejandra Sierra
Computer Vision
Reconstructed holograms and explanation-aware evaluation for low-cost computational pollen analysis in veterinary cytology
Swarn Warshaneyan, Joial Danyal, Blaž Cugmas et al. (+4 more)
Automated pollen analysis supports veterinary cytology, but brightfield microscopy is costlier and more complex than lens-less digital in-line
holographic microscopy. We evaluate whether reconstructed holograms can narrow this gap and whether model explanations remain reliable under modality change. Six pollen species were imaged by brightfield and
holographic microscopy. Raw, single back-propagation and iterative phase
retrieval holograms were evaluated with YOLOv26s detection and MobileNetV4 classification after anchor-based annotation transfer. Six attribution methods were assessed for spatial grounding and faithfulness with the Attribution Health Inspection and Repair (AHIR) protocol, which tests model brittleness under weak noise and corrects attribution-map granularity when needed. Brightfield achieved 0.6890 mAP50-95 (0.8865 mAP50) for detection and 0.9687 macro-F1 (0.9705 accuracy) for classification. Reconstructed holograms narrowed the gap with a task-dependent split: p-type was strongest for detection at 0.5324 mAP50-95 (0.8229 mAP50), while r-type was strongest for classification at 0.7695 macro-F1 (0.7866 accuracy), both far above raw-hologram baselines. Activation-based explanations localized strongly on grains, and region-based methods retained ~60 to ~80% of faithfulness under holography. The
holographic detector was highly brittle to weak perturbations, saturating deletion-based evaluation while insertion remained informative. Pixel-level gradient explanations approached random floor, yet spatial smoothing restored p-type gradient faithfulness from 0.05 to 0.51. For
holographic classification, perturbation-based explanations remained faithful while gradient-based methods fell below random floor. Reconstruction improves low-cost
holographic pollen analysis, while AHIR distinguishes genuine attribution failure from artifacts caused by model brittleness and map granularity.
Read the full abstract ▾
All authors Swarn Warshaneyan, Joial Danyal, Blaž Cugmas, Mindaugas Tamošiūnas, Edgars Kviesis-Kipge, Kirishanth Manivannan, Roberts Kadiķis
Computer Vision
CrowdCue: Specialist-Cue Conditioning for Vision-Language Crowd Counting
Moshiur Farazi, Bekir Ciftler, Abdulhalim Dandoush et al. (+1 more)
Generative
vision-
language models (VLMs) offer a counting paradigm in which one model produces both a count and a natural-
language account of the scene, yet their raw counting accuracy sits in the range of sub-million-parameter specialist regressors. The open question is whether auxiliary guidance from a pretrained specialist can lift them into useful territory, and through which channel that guidance is best routed. We evaluate Qwen2.5-VL-7B on four widely used crowd counting
benchmarks (ShanghaiTech A and B, UCF-QNRF, NWPU-Crowd). Zero-shot prompting rarely produces a parseable count, so LoRA supervised fine-tuning establishes the baseline at overall MAE 81.64. Conditioning on a P2PNet-derived density heatmap as an auxiliary visual signal fails in every encoding we tested, and an
adversarial-swap protocol shows the model reads the heatmap but applies it counterproductively. We propose CrowdCue, a family that supplies the same specialist's already-integrated integer count to the VLM as a discrete symbol. The text-channel variant reaches MAE 72.04. The visual-channel variant, which renders the integer as printed digits and supplies it as a second image, reaches MAE 62.65, the strongest result in this paper and well ahead of the cue-supplying specialist alone (84.45 on the same split). In the late-fusion VLM we study, the binding constraint is not the channel but the abstraction level at which the specialist signal is delivered.
Read the full abstract ▾
All authors Moshiur Farazi, Bekir Ciftler, Abdulhalim Dandoush, Reda Bendraou
Computer Vision
MixiMotion: One-Step Text-to-Motion Generation via Asymmetric Set Distillation
Hung Dinh, Binh Mai, Tran Quoc Bao Le et al. (+2 more)
Iterative text-to-motion generation delivers high-quality and semantically aligned motions but requires multiple network evaluations, resulting in substantial inference latency. We present MixiMotion, a strict one-step text-to-motion generation framework based on offline set
distillation. Instead of distilling a single teacher trajectory for each text
prompt, MixiMotion constructs an offline bank of multiple teacher motions and aligns teacher and student sample sets through asymmetric bidirectional matching. The teacher-to-student direction promotes coverage of diverse teacher-supported motions, while the student-to-teacher direction suppresses unsupported generations. We further introduce differentiable decoded-space kinematic supervision to complement normalized representation matching with constraints in the decoded motion space. At inference, MixiMotion generates a complete motion sequence with a single network evaluation, without teacher queries, iterative sampling, or candidate ranking. On ViMoGen, MixiMotion achieves a semantic
alignment score of $0.835$, outperforming the evaluated one-step baselines and approaching the $0.858$ score of its 50-step HY-Motion-1.0-Lite teacher. In blinded human evaluation, MixiMotion obtains an overall rating of $4.33$, compared with $4.50$ for the teacher, while outperforming the evaluated one-/few-step baselines. Meanwhile, generation latency is reduced from $829.58$\,ms to $9.30$\,ms, corresponding to an $89.2\times$ speedup. These results demonstrate an effective quality–efficiency trade-off for strict one-step text-to-motion generation.
Read the full abstract ▾
All authors Hung Dinh, Binh Mai, Tran Quoc Bao Le, Lam Nguyen, Cong Tran
Computer Vision
Compressing 3D Gaussian Splatting via Cross-Representation Priors
Yezheng Zhang, Huanxiong Liang, Chuqin Zhou et al. (+2 more)
3D Gaussian Splatting (3DGS) enables high-quality novel view synthesis but incurs high storage and transmission costs due to dense Gaussian primitives. Recent anchor-based compression reduces per-primitive redundancy, yet redundancy across anchors remains largely unexploited. We propose CRP-GS (Cross-Representation Priors for Gaussian Splatting), a rate-distortion optimized compression framework that leverages cross-representation priors to improve anchor-level
entropy modeling. First, a Correspondence-Oriented Hierarchical Structure (COHS) organizes anchors by feature correspondence rather than spatial proximity, constructing root-leaf dependencies so that selected anchors can act as informative priors to conditionally encode others, yielding more accurate likelihood prediction and lower conditional
entropy. Second, Shared Feature Aggregation (SFA) extracts globally shared features from a contextual hash grid and injects them into anchor representations, factoring out scene-consistent low-frequency information that would otherwise be redundantly embedded in individual anchors. Both modules are trained under a unified rate-distortion objective to balance bitrate reduction and rendering fidelity. Experiments across multiple
benchmarks show that CRP-GS achieves a favorable overall rate-distortion trade-off, yielding around 30% average bitrate reduction compared to anchor-based baselines while maintaining comparable rendering quality.
Read the full abstract ▾
All authors Yezheng Zhang, Huanxiong Liang, Chuqin Zhou, Guo Lu, Wenjun Zhang
Computer Vision
M3GA-Wild: A Large-Scale Dataset and Benchmark for Multi-Modal Multi-session Ground-to-Aerial Place Recognition in Forests
Ethan Griffiths, Maryam Haghighat, Simon Denman et al. (+2 more)
We present M3GA-Wild, the first
benchmark for multi-modal, multi-session ground-to-aerial place recognition in forests. M3GA-Wild unifies and extends existing forest localisation
datasets, providing a holistic
benchmark with synchronised RGB imagery and
LiDAR from ground traversals spanning 36 km, aligned high-resolution aerial imagery and multi-altitude
LiDAR covering 370 hectares, and accurate geo-referenced 6-DoF poses for precise evaluation. M3GA-Wild captures diverse forest scenes with varying viewpoints, occlusion, and environmental conditions, enabling systematic evaluation of visual,
LiDAR, cross-modal, and multi-modal methods. Baseline experiments show that
LiDAR-based approaches significantly outperform
vision-only methods under severe viewpoint differences, while current multi-modal fusion strategies yield limited gains due to poor cross-modal
alignment. By pairing aerial RGB imagery with geo-referenced aerial
LiDAR, M3GA-Wild also enables evaluation of foundation models for
monocular depth estimation as a cheap source of 3D geometry from forest imagery, with initial experiments revealing shortfalls of current methods. These results highlight key challenges in cross-platform localisation, including modality misalignment and severe domain gaps. M3GA-Wild establishes a new
benchmark to support research in robust multi-modal localisation and long-term autonomy in unstructured natural environments. The
dataset and code will be available upon acceptance.
Read the full abstract ▾
All authors Ethan Griffiths, Maryam Haghighat, Simon Denman, Clinton Fookes, Milad Ramezani
Computer Vision
General Collaborative Intelligence: Architecting Cognition for Resilient Multi-Agent Ecosystems
Lei Zhang, Chun Ye, Le Yang et al. (+4 more)
Multi-
agent unmanned systems are moving from isolated, ego-centric sensing toward collaborative intelligence, in which distributed
agents exchange compact features to overcome a local observation trap that no single
agent can escape: occlusions, finite sensor range, and environmental degradation. The field has matured across architectural, communication,
embodied, resilience, and trust dimensions, yet existing surveys examine these dimensions in isolation and rarely expose their dependencies. This review offers a unified synthesis through two complementary lenses. The first is a five-dimensional taxonomy spanning collaboration stage, communication paradigm, fusion architecture, learning strategy, and application domain. The second is three cognitive synergy conditions, Semantic Disambiguation, Pragmatic Information Exchange, and Proactive Informational Foraging, that turn cognitive synergy into operational criteria. Across these lenses we survey collaboration architectures and
topologies, neural-communication co-design that treats the channel as a differentiable pipeline component,
embodied action-perception loops via multi-
agent reinforcement learning, and resilience mechanisms for synchronization, uncertainty quantification, and label-efficient learning. We then map these advances onto four operational domains, V2X, unmanned aerial, industrial logistics, and smart cities, and onto the safety-
privacy-utility triad. To counter
benchmark saturation and evaluation fragmentation, we propose GCI-Bench, a five-pillar scoring protocol with a maturity model that makes the trade-offs of collaborative methods comparable across studies. A critical reflection on reproducibility, the sim-to-real gulf, and conditions under which collaboration degrades performance identifies open challenges and charts directions toward general collaborative intelligence under real-world uncertainty.
Read the full abstract ▾
All authors Lei Zhang, Chun Ye, Le Yang, Zhaozhong Wang, Deng-Ping Fan, Hang Dai, Binglu Wang
Computer Vision
CLEAR: Complex Learned Explicit Analytical Regularization for Ultra-Accelerated 4D Flow CMR Reconstruction
German Shâma Wache and Sebastian Neumayer
While compressed-sensing regularizers enable interpretable reconstruction of 4D Flow CMR through transparent variational objectives, their hand-crafted nature is too restrictive under high acceleration. State-of-the-art learning-based approaches mitigate this, but typically encode regularization implicitly through unrolled network modules, which limits their interpretability. To address this limitation, we propose CLEAR, designed to combine the interpretability of compressed sensing with the flexibility of learned models. To the best of our knowledge, it is the first learned regularizer for a 4D reconstruction task. In the ultra-accelerated \(10\times\)–\(50\times\) regime of the CMRx4DFlow2026 challenge, CLEAR outperforms compressed sensing locally low-rank (LLR) and the popular variational network FlowVN, while using less than 10k parameters and preserving an interpretable regularization structure.
Read the full abstract ▾
Computer Vision
RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling
Zhenchen Tang, Yang Li, Songlin Yang et al. (+6 more)
Reinforcement learning (RL) is vital for optimizing
video generation models, with a robust reward model (RM) serving as the cornerstone. However, existing
video reward models often produce unstable scalar scores because they directly map complex, subjective
video quality into a single score without explicit evaluation criteria. This leads to scalar drift, where the scoring scale collapses or shifts across different
prompts, making the reward unreliable for RL. Drawing inspiration from professional human annotation engineering, we address this problem with RewardVerse, a rubric-based
video reward framework that introduces a dynamic rubric as an intermediate representation between the evaluation query and the scorer. Instead of unconstrained direct scoring, RewardVerse first generates explicit evaluation criteria and then performs rubric-guided scoring, providing a stable semantic anchor that mitigates scalar drift. To efficiently optimize this collaborative pipeline, we propose Rubric-Guided Policy Optimization (RGPO), a two-stage training
algorithm. RGPO first warms up the scorer using self-evolving seed rubrics and then jointly optimizes the rubric generator to produce query-adaptive evaluation criteria while continuously aligning the scorer with human ratings. Extensive experiments on the 16-dimensional EvalVerse
benchmark and external
datasets demonstrate that RewardVerse mitigates scalar drift, achieves state-of-the-art performance on both pointwise and pairwise evaluation, and provides a robust and interpretable reward signal for RL in
video generation.
Read the full abstract ▾
All authors Zhenchen Tang, Yang Li, Songlin Yang, Bo Peng, Xiaotong Zhao, Shuai Li, Haotian Fan, Alan Zhao, Jing Dong
Computer Vision
An Evolutionary Agentic Approach for Open-ended Image Quality Perception
Zhenchen Tang, Bo Peng, Zichuan Wang et al. (+4 more)
Generative models are rapidly expanding image quality assessment (IQA) beyond traditional fidelity factors to emerging dimensions such as physical plausibility and text-rendering correctness. However, existing IQA models rely on fixed definitions and heavy supervision, making them difficult to extend to open-ended perceptual dimensions. We identify holistic bias as an important limitation: when scoring an unseen dimension, models reuse generic quality priors, leading to scoring errors and rank inversion. To address this, we propose PACE (Perceptual Agentic Collaborative
Evolution), a training-free multi-
agent framework that formulates open-ended IQA as explicit protocol construction. Given a target dimension, PACE uses collaborative
agents to construct an evaluation protocol composed of verifiable Visual Question Answering (VQA) probes, grounding evaluation in concrete visual evidence rather than holistic impressions. The resulting protocol is calibrated using only four human-annotated images per dimension, while a dual-track scoring mechanism aligns model perception with human scoring scales. Across traditional IQA, structural fidelity, context-aware aesthetics, and newly defined open-ended dimensions, PACE consistently improves its MLLM backbone, achieving competitive performance across diverse IQA settings, and reduces the Holistic Override Rate (HOR) from 44.4% to 8.6%.
Read the full abstract ▾
All authors Zhenchen Tang, Bo Peng, Zichuan Wang, Songlin Yang, Leilei Cao, Fengjie Zhu, Jing Dong
Computer Vision
D3GS: Depth, DINO, and RGB Diffusion Co-Guided 3D Gaussian Splatting for Sparse-View Reconstruction
Yunqi Gao, Zhanfeng Liao, Hanzhang Tu et al. (+7 more)
Novel view synthesis from sparse inputs remains challenging for 3D Gaussian Splatting (3DGS) due to ambiguous geometry, cross-view inconsistency, and missing details in under-constrained regions, resulting in degraded reconstruction and unstable rendering. To tackle these issues, we propose D$^{3}$GS, a Depth-DINO-
Diffusion guided sparse-view Gaussian reconstruction framework that jointly enhances geometry and appearance. D$^{3}$GS first recovers a high-resolution, metric depth map via
diffusion-based completion and DPT (Dense Prediction
Transformer) refinement, providing robust Gaussian initialization and
geometric constraints. Then, a DINO-guided view-consistent learning is introduced to augment Gaussian attributes with structural features, improving multi-view consistency. Finally, a
diffusion-based Gaussian refinement module injects generative priors into an iterative optimization strategy, enhancing high-frequency
geometric and appearance details within the Gaussian representation. Experiments on DTU, LLFF, and Mip-NeRF 360 show that D$^{3}$GS achieves consistent and substantial improvements over strong baselines, with ablation studies validating the effectiveness and complementary roles of each component.
Read the full abstract ▾
All authors Yunqi Gao, Zhanfeng Liao, Hanzhang Tu, Zhaoqi Su, Guoqing Zheng, Songtao Wang, Hongwen Zhang, Zhou Xue, Leyuan Liu, Yebin Liu
Computer Vision
Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generation
Guanqiao Chen, Jingru Tan, Dongxing Mao et al. (+5 more)
Generating text-rich images from
prompts requires both textual fidelity and the coherent integration of text into the surrounding image. An explicit layout can provide structured guidance about what text should appear and where, but a well-formed plan alone does not guarantee that the renderer will realize it faithfully. Existing layout-based AR-
diffusion systems typically optimize planning and rendering separately, preventing the planner's representations from being adapted jointly with image synthesis. We introduce DuetGen, an autonomous visual text generator built on DeepFusion, which jointly learns autoregressive planning and continuous
diffusion rendering. DeepFusion conditions a
diffusion transformer on the planner's
prompt and bbox-content hidden states, allowing rendering supervision to shape the representations connecting textual plans with visual outputs. Its joint objective combines autoregressive plan supervision, text-region-weighted
diffusion learning, and auxiliary coordinate supervision to maintain structured planning, emphasize text-bearing regions, and improve the spatial precision of planner representations. During inference, Phase-Aware
Attention Modulation strengthens the correspondence between image regions and their matched coordinate and content states, facilitating region-specific execution of the generated plan. With a 2B planner and a 4B single-stream DiT, DuetGen achieves 0.8293 word accuracy on CVTG-2K and 0.938 accuracy on LongText-Bench, closely matching the substantially larger Qwen-Image on both
benchmarks. These results demonstrate the value of jointly learned planning representations and region-specific rendering for autonomous visual text generation.
Read the full abstract ▾
All authors Guanqiao Chen, Jingru Tan, Dongxing Mao, Catherine Chen, Zijian Du, Libo Qin, Hu Jian Guo, Alex Jinpeng Wang
Computer Vision
AVTR-1: Open Stack for Real-Time Interactive Avatars
Artem Kravtsov, Dmitrii Ziganshin, Vsevolod Poletaev et al. (+4 more)
Talking-head and dyadic models now achieve real-time inference, yet fast motion generation alone does not produce an interactive conversation. A live system must synchronize the model's output with
speech from an external voice
agent, schedule
video frames for playback, and handle interruptions. We introduce AVTR-1, an open stack for real-time interactive avatar conversations, built around a compact 153M-parameter autoregressive flow-matching motion generator conditioned on both participants' audio. We adapt its audio encoder for streaming through self-
distillation. The stack turns the model's chunk-based generation into a continuous, synchronized audio-
video stream driven by an external voice
agent, and we analytically derive its contribution to the user-facing latencies and validate the resulting bounds with two commercial voice
agents. Further experiments demonstrate that AVTR-1 leads the compared dyadic systems on all reported visual-quality metrics and most conventional listening-motion metrics while remaining competitive in lip synchronization. Its inference runtime operates in real time on data-center and consumer GPUs. However, conventional listening metrics do not establish whether the paired speaker's
speech contributes to generated motion. We therefore introduce the Reference-Based Directed Granger Gain (R-DGG), which measures the additional predictive information carried by speaker
speech after accounting for listener history and speaker motion. R-DGG finds statistically supported predictive dependence for recorded listeners and all evaluated dyadic systems, but not for talking-head generators without paired audio or mismatched speaker-listener pairs. We release the model weights, renderer, and serving backend under component-specific licenses.
Read the full abstract ▾
All authors Artem Kravtsov, Dmitrii Ziganshin, Vsevolod Poletaev, Gleb Balitskiy, Anastasia Tikhonova, Egor Burkov, Vadim Lebedev
Computer Vision
Scout: Open-World Species Recognition on the Edge
Mohammad Mehdi Rastikerdar, Hui Guan and Deepak Ganesan
Large
vision-
language models (VLMs) enable recognition beyond a fixed class set, but their computational demands prevent them from running on many edge devices. Cloud offload makes this capability accessible, but sending every image consumes scarce bandwidth and communication energy. We ask how to bring the open-world recognition capability of VLMs to the edge while operating within tight compute, energy, and bandwidth budgets. Wildlife monitoring provides a natural setting for exploring this question because camera traps encounter species not known at deployment. We present Scout, an autonomous open-world recognition system that invokes a cloud VLM intermittently to teach new classes to a compact edge model. Given only the deployment location and empty site frames, Scout autonomously turns each species identified by the VLM into persistent, site-conditioned recognition capability in a resource-efficient edge model, without a predefined species list, human labeling, or manual tuning. Across 30 camera-trap deployments in three regions on an NVIDIA Jetson Orin Nano, the accuracy of Scout remains within 0.1-2.5% of a model given a predefined species list. On species outside its initial class set, Scout achieves 53.7-59.1% accuracy, compared with 56.5-65.1% for full cloud offload, while using 59-71% less deployment energy.
Read the full abstract ▾
Computer Vision
Combining Foundation Model Confidence and Monocular Depth for Training-Free Out-of-Distribution Segmentation
Serin Varghese, Fabian Hüger and Kira Maag
Autonomous vehicles operating in open-world scenarios are inevitably confronted with previously unknown objects, such as exotic animals or loose cargo. The reliable detection and
segmentation of these out-of-distribution (OOD) objects is therefore crucial for a safe understanding of the environment and decision-making. Most existing approaches require access to OOD training samples, retraining of the
segmentation backbone, or dedicated auxiliary architectures, limiting their practical applicability. We propose a training-free method that derives dense OOD scores directly from the confidence predictions of a foundation
segmentation model, without any task-specific fine-tuning or access to anomalous data. To improve the robustness of our OOD
segmentation,
geometric information from
monocular depth estimation is incorporated into the decision process, providing complementary cues to uncertainty-based predictions. We evaluate the proposed method on the SegmentMeIfYouCan
benchmark and additionally assess its performance on OOD tracking in
video sequences, reflecting the temporal nature of real-world perception systems. The method performs strongly on road-centered
benchmarks.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Are Coreset Selection Methods Worth Their Cost?
Yangze Liu and Zhongyi Han
Coreset selection picks a representative subset of the labeled training set to make training cheaper. However, it is usually evaluated by downstream accuracy at a fixed subset size, ignoring both the time spent selecting the subset and the training recipe behind each reported number. We introduce an end-to-end
benchmark that standardizes downstream training and charges selection and training to the same auditable wall-clock budget, spanning 4
datasets from CIFAR-10 to ImageNet-1K, 11 selectors, 5 fractions, and 3 seeds, with over 1,500 released runs. Repeated-sampling work has shown that budget-aware evaluation already favors random strategies. Our two budget studies test whether that verdict survives when every selector is granted its most favorable operating point. Across eight wall-clock budget anchors on each of CIFAR-10 and Tiny ImageNet, no anchor is won by a sophisticated selector: every winner is class-balanced random sampling, repeated random sampling, or full-data training. In fixed-budget duels on ImageNet-1K, training on all data for fewer epochs beats every selection strategy we probe while also costing the least. A per-
dataset cost audit shows that selection cost is dominated at every scale by a fixed full-
dataset scan, so it cannot be amortized away by selecting a smaller fraction, and its absolute size does not extrapolate from one
dataset to another. We further quantify when selection does pay back through subset reuse, and document 9 correctness fixes to a widely used codebase, one of which shifts a standard Herding baseline by nearly 6 points. Selection time is not free preprocessing, and an evaluation that ignores it measures the wrong quantity.
Read the full abstract ▾
Computer Vision
Planning-Aligned Pretraining of BEV Representations with Sparse Action-Conditioned Targets for End-to-End Autonomous Driving
Jaeha Song and Soonmin Hwang
End-to-end driving requires planning-relevant bird's-eye-view (BEV) representations, but existing pretraining approaches often rely on task annotations or dense scene reconstruction. We introduce PAVER, Planning-Aligned BEV Encoder Pretraining. From a single
LiDAR sweep, PAVER constructs sparse risk and unknown targets describing occupied and unobserved evidence along rule-based ego motions. A 10K-parameter head predicts these targets from masked BEV features conditioned on the action state, directing supervision toward
geometric constraints on candidate motions. Pretraining requires no driving-task annotations or dense reconstruction. Only the BEV encoder is transferred, preserving the downstream architecture and camera-only inference. On nuScenes, PAVER reduces VAD-Tiny's average collision rate from 0.51% to 0.19%, while improving planning L2, motion prediction, detection, and mapping. The selected VAD-Tiny and VAD-Base schedules use about 36% less estimated total training time than scratch training, including pretraining. On Bench2Drive Town05 Long, PAVER improves UniAD-Tiny's closed-loop Driving Score from 48.45 to 58.79. The project page is available at https://archiiive99.github.io/PAVER.
Read the full abstract ▾
Computer Vision
Image Frame Dynamic Object Segmentation and Ego Motion Estimation using Radar Image Fusion
Astik Srivastava, Suhani Grover, Avinash Sharma et al. (+1 more)
Dynamic object
segmentation and ego-motion estimation are closely coupled problems in autonomous driving, as accurate ego-motion estimation typically requires static scene observations, while identifying static observations requires knowledge of the ego motion. We present Radar-Dot, a radar–RGB framework that exploits radar Doppler measurements to address this coupling. Radar returns are first used to estimate ego velocity through a linear Doppler constraint, with residual-based static/dynamic
segmentation and robust estimation used to reduce the influence of moving objects. The estimated motion is then combined with metric depth and dense optical flow to identify image regions whose observed motion is inconsistent with the rigid scene motion. Experiments on 10 nuScenes scenes (part of nuscenes-mini) demonstrate that the resulting
geometric pipeline achieves 20.24% dynamic IoU and 33.67% F1-score over 394 frame pairs, while radar-based static-point filtering improves ego-velocity estimation compared with using all radar returns. These results demonstrate the potential of radar as a modality for jointly improving ego-motion estimation and dynamic object
segmentation.
Read the full abstract ▾
All authors Astik Srivastava, Suhani Grover, Avinash Sharma, Madhava Krishna
Computer Vision · Cross-listed from cs.RO
SmoLSTM: A Compact Vision-Language-Action Model with Recurrent Memory that Persists
Jan-Gerrit Habekost, Parsa Mastouri Kashani, Connor Gäde et al. (+5 more)
Vision-
language-action models often predict actions from only the current observation, which can leave tasks involving object occlusion or visually identical objects ambiguous without episode history. The usual countermeasure, widening the observation window, turns the horizon into a hyperparameter and lets per-step cost grow with it. We instead capture the episode in a recurrent state. SmoLSTM couples a frozen 256M-parameter SmolVLM backbone to a
matrix-
memory LSTM control layer in which observation
tokens and action queries are unified in a single causal stream that is never reset throughout the entire episode. Recurrent-state storage is therefore O(1) in episode length. A flow-matching action head predicts chunks of 10 end-effector pose deltas and gripper commands at each control step. Our single policy, trained jointly on 7,461 demonstrations across 140 tasks and evaluated on held-out initial states, performs best, reaching 85.1% subgoal coverage and 77.5% full-task success on LIBERO-Mem with 0.04B trainable parameters, surpassing both the
benchmark's own object-centric baseline and recent
memory-based approaches. Resetting the recurrent state at every control step reduces full-task success to 7.0%, showing that the trained policy relies on context carried between decisions. The same model achieves 79.6% average success on standard LIBERO.
Read the full abstract ▾
All authors Jan-Gerrit Habekost, Parsa Mastouri Kashani, Connor Gäde, Matthias Kerzel, Philipp Allgeuer, Cornelius Weber, Stefan Wermter, Jae Hee Lee
Computer Vision
LINGO: Latent Initialization and Gradient Optimization for Sparse-view X-ray Novel View Synthesis and CT Reconstruction with 3D Gaussian Splatting
Lifeng Xing, Dequan Jin, Kunpeng Bu et al. (+2 more)
In novel view synthesis and Computed
Tomography (CT) reconstruction with sparse-view X-ray imaging, insufficient angular coverage leads to structural ambiguity and accumulated noise. Integrating 3D Gaussian Splatting (3DGS) with X-ray absorption physics can achieve promising results, but it suffers from noisy initialization, positional insensitivity, and weak gradients in low-density regions. In this paper, we propose a unified Latent Initialization and Gradient Optimization (LINGO) framework to address these issues. LINGO combines latent mask-space initialization with dynamic gradient optimization to improve point cloud structural completeness while accelerating training. It constructs voxel-level 3D filters from X-ray masks to robustly suppress background noise and provide reliable
geometric priors. By employing an adaptive voxel scaling strategy and dynamically scaling loss, LINGO can adjust spatial resolution and explicitly amplify gradients in low-density structures. To evaluate the quality of initialization, we introduce the Initialization Point Cloud Structural Deviation (IPSD) metric. Experiments on the X3D
dataset indicate that for the novel view synthesis task, LINGO improves the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) by an average of 0.72 and 0.0039, respectively, over baselines under identical sparse-view settings, achieving comparable reconstruction quality within 5k steps to state-of-the-art models typically trained with 30k iterations. For the CT reconstruction task, LINGO also demonstrates consistent improvements, with average PSNR and SSIM gains of 0.36 and 0.0134. These results highlight LINGO's effectiveness in both accelerating training and enhancing reconstruction quality across different sparse-view imaging scenarios.
Read the full abstract ▾
All authors Lifeng Xing, Dequan Jin, Kunpeng Bu, Peigeng He, Shihui Ying
Computer Vision
SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagery
Changhao Zhao, Linglin Zeng and Hai Liu
Open-vocabulary semantic
segmentation (OVS) of remote sensing imagery is a challenging pixel-level task requiring strong generalization and adaptation to the spatial characteristics of remote sensing data. Although existing
vision-
language foundation models perform well in general domains, their image-level classification design weakens the spatial priors needed for high-resolution remote sensing
segmentation: structural spatial relations are degraded during deep feature transformation, and fine-grained spatial details are lost during downsampling. To address these complementary deficiencies, we propose SatOV, a training-free framework for open-vocabulary remote sensing
segmentation that restores spatial priors at two stages of the representation pipeline. Specifically, Residual QQ
Attention (ResQQ) extracts Query-Key self-
attention from an intermediate CLIP layer and fuses it with final-layer Query-Query
attention via a residual combination, restoring structural spatial priors suppressed by the final-layer representation. Spatially Modulated Upsampling (SatUp) uses the original high-resolution RGB image as spatial guidance, combining spatial feature modulation with guided cross-
attention to reconstruct pixel-level textures and boundaries. Extensive experiments on DOTA, UDD, LoveDA, and Vaihingen show that SatOV consistently improves training-free OVS and achieves competitive quantitative and qualitative results against state-of-the-art methods. These results validate the effectiveness of restoring spatial priors at both the representation and spatial-resolution stages for remote sensing open-vocabulary
segmentation.
Read the full abstract ▾
Computer Vision
C$^{2}$-INR: Customized Convolutional Implicit Neural Representation
Jinglei Shi, Xinran Chang, Jiaqi Cui et al. (+3 more)
Implicit Neural Representation (INR) leverages neural networks to represent discrete signals such as images as continuous ones, where the network weights serve as a compact form of the signal itself. Most existing INR methods adopt Multi-Layer Perceptrons (MLPs) as their backbone. Since these models render each pixel independently, they inherently fail to exploit the spatial correlations that exist between neighboring pixels. In contrast,convolutional INRs can process pixels in parallel while inherently accounting for inter-pixel dependencies, making them a more natural fit for representing images. Nevertheless, convolutional INRs remain relatively underexplored, and the majority of them rely on fixed architectural settings, leaving little room for image-specific adaptation. In this paper, we investigate network customization for convolutional INRs. We replace conventional filters with irregular directional kernels, whose allocation is guided by the directional energy in the image spectrum, i.e., directions exhibiting stronger energy are assigned a larger number of kernels, enabling content-tailored convolution settings. These kernels are further reformulated via an orthogonal basis to achieve a superior sparse representation. Moreover, we introduce an annealed Gumbel-Softmax-based mechanism for kernel-level activation function selection, which gives the most suitable activation function for each convolution kernel. Extensive experiments demonstrate that our method, namely C$^{2}$-INR, achieves superior performance against state-of-the-art approaches under comparable parameter budgets across a wide range of image processing tasks, including representation, inpainting, and super-resolution.
Read the full abstract ▾
All authors Jinglei Shi, Xinran Chang, Jiaqi Cui, Yingjie Xia, Zhaolin Xiao, Chongyi Li
Computer Vision
PixelART: Image-to-Layer Decomposition without Latents or Text-to-Image Pretraining
Zelin Jia, Zhao Zhang, Zhicong Tang et al. (+2 more)
Image-to-layer decomposition converts a flattened image into editable RGBA layers, enabling element-level editing in design workflows. Existing
diffusion-based systems typically adapt large pretrained text-to-image (T2I) models and introduce RGBA autoencoders or variable-layer architectural modules. We revisit this design choice and ask whether layer decomposition truly requires these heavyweight components. We introduce PixelART, a pixel-space rectified-flow
Transformer trained from scratch for image-to-layer (I2L) decomposition. PixelART directly denoises regional RGBA pixel patches using a single-stream multi-modal
diffusion Transformer, avoiding RGBA-VAEs, pretrained T2I backbones, and layer-specific decoders. We identify a key property of the task: high-noise timesteps determine layer assignment and coarse layer organization, while low-noise timesteps mainly refine color, alpha, texture, and boundaries. Based on this observation, we propose a terminal-boosted timestep sampling strategy to increase training coverage in the high-noise layer assignment regime. Trained on 4M multi-layer design templates, PixelART achieves state-of-the-art layer decomposition and composite reconstruction on Design-Multi-Layer-Bench and LICA with over 80% fewer parameters, 98% lower latency, and 85% lower
memory than the recent Qwen-Image-Layered model. Ablation experiments show that pixel-space $\mathbf{x}$-prediction, terminal-boosted timestep sampling, and data/model scaling are critical, while T2I pretraining brings marginal benefits to the I2L task.
Read the full abstract ▾
All authors Zelin Jia, Zhao Zhang, Zhicong Tang, Yuhui Yuan, Shixia Liu