Computer Vision
Cryo-Bench: Benchmarking Foundation Models for Cryosphere Mapping
Saurabh Kaushik, Lalit Maurya, Beth Tellman et al. (+4 more)
Geo-Foundation Models (GFMs) have been evaluated across diverse Earth observation tasks and domains, showing strong potential to produce reliable maps even with sparse labels. However, systematic benchmarking of GFMs for Cryosphere applications remains limited, primarily because suitable evaluation
datasets are scarce. We address this gap by introducing Cryo-Bench, a
benchmark comprising six semantic
segmentation datasets covering five cryospheric components: supraglacial debris, glacial lakes under two sensing configurations, sea ice, calving fronts and Antarctic ice-shelf extent. The
benchmark includes multispectral, RGB, and synthetic aperture radar observations from regions underrepresented in existing pretraining archives. We evaluate thirteen GFMs alongside U-Net and
Vision Transformer baselines trained from scratch under a unified evaluation protocol. With frozen encoders, the U-Net achieves the highest six-
dataset average mean intersection over union (mIoU) of 69.31%, exceeding TerraMind (67.86%) by 1.45 points. The paired difference has a 95% confidence interval of [+0.94, +1.98], indicating that the U-Net's lead is statistically significant. In contrast, learning-rate optimization substantially improves fine-tuning performance: DOFA reaches 93.97% mIoU on the RGB glacial lake task, ranking the U-Net fourth, while Scale-MAE and GFM-Swin surpass U-Net on calving fronts. Averaged across all six
datasets, four GFMs, TerraMind, GFM-Swin, DOFA, and Scale-MAE, exceed the U-Net baseline (69.31%). In the few-shot setting, five GFMs, DOFA, RemoteCLIP, TerraMind, GFM-Swin, and Scale-MAE, likewise outperform U-Net; averaged across all thirteen GFMs, retention is 92.5% of full-label accuracy compared with 86.1% for U-Net.
Read the full abstract ▾
All authors Saurabh Kaushik, Lalit Maurya, Beth Tellman, Swalpa Kumar Roy, Valerio Marsocci, Gustau Camps-Valls, Jocelyn Chanussot
Computer Vision · Cross-listed from cs.LG
Same Compression Principle, Different Geometry: Rate-Distortion Signatures Dissociate Biological and Artificial Visual Systems
Leyla Roksan Caglar, Pedro A.M. Mediano and Baihan Lin
Efficient coding theory predicts that biological perceptual systems compress sensory input optimally under resource constraints, with the systematic structure of errors reflecting the geometry of that compression. Here we operationalize this principle using rate-distortion theory (RDT) to characterize how any system - biological or artificial - trades representational fidelity for informational efficiency. Treating stimulus-response behavior as an effective communication channel, we infer rate-distortion (RD) frontiers directly from confusion
matrices and summarize each system with three
geometric signatures: slope (beta),
curvature (kappa), and area under the RD curve (AUC), capturing the marginal cost, abruptness, and overall efficiency of the accuracy-compression trade-off respectively. Applying this framework to human psychophysical data and 18 deep
vision models across 12 families of controlled image perturbations at graded severities, we find that both biological and artificial systems follow a common lossy-compression principle but occupy systematically different regions of RD space. Humans exhibit smooth, flexible trade-offs characteristic of near-optimal efficient coding, while deep networks operate in steeper, more brittle regimes even at matched accuracy, with geometry dissociable from performance across training regimes. Critically, behavioral RD signatures track internal representational geometry, evidenced by the behaviorally inferred compression structure correlating with internal representational dissimilarity across all models. These results establish RD geometry as a compact diagnostic of perceptual compression strategy that recovers mechanistically interpretable structure in internal representations from behavioral input alone and extends naturally to the direct characterization of compression geometry in neural population activity.
Read the full abstract ▾
Computer Vision
TopoMaskV3: 3D Mask Head with Dense Offset and Height Predictions for Road Topology Understanding
Muhammet Esat Kalfaoglu, Halil Ibrahim Ozturk, Ozsel Kilinc et al. (+1 more)
Mask-based paradigms for road
topology understanding, such as TopoMaskV2, offer a complementary alternative to query-based methods by generating centerlines via a dense rasterized intermediate representation. However, prior work was limited to 2D predictions and suffered from severe discretization artifacts, necessitating fusion with parametric heads. We introduce TopoMaskV3, which advances this pipeline into a robust, standalone 3D predictor via two novel dense prediction heads: a dense offset field for sub-grid discretization correction within the existing BEV resolution, and a dense height map for direct 3D estimation. Beyond the architecture, we are the first to address geographic data leakage in road
topology evaluation by introducing (1) geographically distinct splits to prevent memorization and ensure fair generalization, and (2) a long-range (+/-100 m)
benchmark. TopoMaskV3 achieves state-of-the-art 28.5 OLS on this geographically disjoint
benchmark, surpassing all prior methods. Our analysis shows that the mask representation is more robust to geographic overfitting than Bezier, while
LiDAR fusion is most beneficial at long range and exhibits larger relative gains on the overlapping original split, suggesting overlap-induced memorization effects.
Read the full abstract ▾
All authors Muhammet Esat Kalfaoglu, Halil Ibrahim Ozturk, Ozsel Kilinc, Alptekin Temizel
Computer Vision
Align-cDAE: Alzheimer's Disease Progression Modeling with Attention-Aligned Conditional Diffusion Auto-Encoder
Ayantika Das, Keerthi Ram and Mohanasankar Sivaprakasam
Generative AI framework-based modeling and prediction of longitudinal human
brain images offer an efficient mechanism to track neurodegenerative progression, essential for the assessment of diseases like Alzheimer's. Among the existing generative approaches, recent
diffusion-based models have emerged as an effective alternative to generate disease progression images. Incorporating multi-modal and non-imaging attributes as conditional information into
diffusion frameworks has been shown to improve controllability during such generations. However, existing methods do not explicitly ensure that information from non-imaging conditioning modalities is meaningfully aligned with image features to introduce desirable changes in the generated images, such as modulation of progression-specific regions. Further, more precise control over the generation process can be achieved by introducing progression-relevant structure into the internal representations of the model, lacking in the existing approaches. To address these limitations, we propose a
diffusion autoencoder-based framework for disease progression modeling that explicitly enforces
alignment between different modalities. The
alignment is enforced by introducing an explicit objective function that enables the model to focus on the regions exhibiting progression-related changes. Further, we devise a mechanism to better structure the latent representational space of the
diffusion auto-encoding framework. Specifically, we assign separate latent subspaces for integrating progression-related conditions and retaining subject-specific identity information, allowing better-controlled image generation. These results demonstrate that enforcing
alignment and better structuring of the latent representational space of
diffusion auto-encoding framework leads to more anatomically precise modeling of Alzheimer's disease progression.
Read the full abstract ▾
Computer Vision
Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D Representation
Jisoo Kim, Jungbin Cho, Sanghyeok Chu et al. (+9 more)
Humans learn not only how their bodies move, but also how the surrounding world responds to their actions. In contrast, while recent
Vision-
Language-Action (VLA) models exhibit impressive semantic understanding, they often fail to capture the spatiotemporal dynamics governing physical interaction. In this paper, we introduce Pri4R, a simple yet effective approach that endows VLA models with an implicit understanding of world dynamics by leveraging privileged 4D information during training. Specifically, Pri4R augments VLAs with a lightweight point track head that predicts 3D point tracks. By injecting VLA features into this head to jointly predict future 3D trajectories, the model learns to incorporate evolving scene geometry within its shared representation space, enabling more physically aware context for precise control. Due to its architectural simplicity, Pri4R is compatible with dominant VLA design patterns with minimal changes. During inference, we run the model using the original VLA architecture unchanged; Pri4R adds no extra inputs, outputs, or computational overhead. Across simulation and real-world evaluations, Pri4R significantly improves performance on challenging manipulation tasks, including a +10% gain on LIBERO-Long and a +40% gain on RoboCasa. We further show that 3D point track prediction is an effective supervision target for learning action-world dynamics, and validate our design choices through extensive ablations. Project page: https://jiiiisoo.github.io/Pri4R/
Read the full abstract ▾
All authors Jisoo Kim, Jungbin Cho, Sanghyeok Chu, Ananya Bal, Jinhyung Kim, Gunhee Lee, Sihaeng Lee, Seung Hwan Kim, Bohyung Han, Hyunmin Lee, Laszlo A. Jeni, Seungryong Kim
Computer Vision
PathMoE: Interpretable Multimodal Interaction Experts for Pediatric Brain Tumor Classification
Jian Yu, Joakim Nguyen, Jinrui Fang et al. (+10 more)
Accurate classification of pediatric central nervous system tumors remains challenging due to histological complexity and limited training data. While pathology foundation models have advanced whole-slide image (WSI) analysis, they often fail to leverage the rich, complementary information found in clinical text and tissue microarchitecture. To this end, we propose PathMoE, an interpretable
multimodal framework that integrates H&E slides, pathology reports, and
nuclei-level
cell graphs via an interaction-aware mixture-of-experts architecture built on state-of-the-art foundation models for each modality. By training specialized experts to capture modality uniqueness, redundancy, and synergy, PathMoE employs an input-dependent gating mechanism that dynamically weights these interactions, providing sample-level interpretability. We evaluate our framework on two
dataset-specific classification tasks on an internal pediatric
brain tumor
dataset (PBT) and external TCGA
datasets. PathMoE improves macro-F1 from 0.762 to 0.799 (+0.037) on PBT when integrating WSI, text, and graph modalities; on TCGA, augmenting WSI with graph knowledge improves macro-F1 from 0.668 to 0.709 (+0.041). These results demonstrate significant performance gains over state-of-the-art image-only baselines while revealing the specific modality interactions driving individual predictions. This interpretability is particularly critical for rare tumor subtypes, where transparent model
reasoning is essential for clinical trust and diagnostic validation.
Read the full abstract ▾
All authors Jian Yu, Joakim Nguyen, Jinrui Fang, Awais Naeem, Zeyuan Cao, Sanjay Krishnan, Nicholas Konz, Tianlong Chen, Chandra Krishnan, Hairong Wang, Edward Castillo, Ying Ding, Ankita Shukla
Computer Vision
Zhengtong Zhu, Jiaqing Fan, Zhixuan Liu et al. (+1 more)
Reasoning Video Object
Segmentation (ReasonVOS) is a challenging task that requires stable object
segmentation across
video sequences using implicit and complex textual inputs. Previous methods fine-tune
Multimodal Large
Language Models (MLLMs) to produce
segmentation outputs, which demand substantial resources. Additionally, some existing methods are coupled in the processing of spatio-temporal information, which affects the temporal stability of the model to some extent. To address these issues, we propose Training-Free Spatio-temporal Decoupled
Reasoning Video Segmentation with Adaptive Object
Memory (SDAM). We aim to design a training-free
reasoning video segmentation framework that outperforms existing methods requiring fine-tuning, using only pre-trained models. Meanwhile, we propose an Adaptive Object
Memory module that selects and memorizes key objects based on motion cues in different
video sequences. Finally, we propose Spatio-temporal Decoupling for stable temporal propagation. In the spatial domain, we achieve precise localization and
segmentation of target objects, while in the temporal domain, we leverage key object temporal information to drive stable cross-frame propagation. Our method achieves excellent results on five
benchmark datasets, including Ref-YouTubeVOS, Ref-DAVIS17, MeViS, ReasonVOS, and ReVOS.
Read the full abstract ▾
All authors Zhengtong Zhu, Jiaqing Fan, Zhixuan Liu, Fanzhang Li
Computer Vision
RA-Det: Towards Universal Detection of AI-Generated Images via Robustness Asymmetry
Xinchang Wang, Yunhao Chen, Yuechen Zhang et al. (+4 more)
Recent image generators produce photo-realistic content that undermines the reliability of downstream recognition systems. As visual appearance cues become less pronounced, appearance-driven detectors that rely on forensic cues or high-level representations lose stability. This motivates a shift from appearance to behavior, focusing on how images respond to controlled perturbations rather than how they look. In this work, we identify a simple and universal behavioral signal. Natural images preserve stable semantic representations under small, structured perturbations, whereas generated images exhibit markedly larger feature drift. We refer to this phenomenon as robustness asymmetry and provide a theoretical analysis that establishes a lower bound connecting this asymmetry to memorization tendencies in generative models, explaining its prevalence across architectures. Building on this insight, we introduce Robustness Asymmetry Detection (RA-Det), a behavior-driven detection framework that converts robustness asymmetry into a reliable decision signal. Evaluated across 14 diverse generative models and against more than 10 strong detectors, RA-Det achieves superior performance, improving the average performance by 7.81 percent. The method is data- and model-agnostic, requires no generator fingerprints, and transfers across unseen generators. Together, these results indicate that robustness asymmetry is a stable, general cue for synthetic-image detection and that carefully designed probing can turn this cue into a practical, universal detector. The source code is publicly available at Github.
Read the full abstract ▾
All authors Xinchang Wang, Yunhao Chen, Yuechen Zhang, Congcong Bian, Zihao Guo, Xingjun Ma, Hui Li
Computer Vision
Benchmarking Semantic Segmentation Models via Appearance and Geometry Attribute Editing
Zijin Yin, Bing Li, Kongming Liang et al. (+4 more)
Semantic
segmentation takes pivotal roles in various applications such as autonomous driving and medical image analysis. When deploying
segmentation models in practice, it is critical to test their behaviors in varied and complex scenes in advance. In this paper, we construct an automatic data generation pipeline Gen4Seg to stress-test semantic
segmentation models by generating various challenging samples with different attribute changes. Beyond previous evaluation paradigms focusing solely on global weather and style transfer, we investigate variations in both appearance and geometry attributes at the object and image level. These include object color, material, size, position, as well as image-level variations such as weather and style. To achieve this, we propose to edit visual attributes of existing real images with precise control of structural information, empowered by
diffusion models. In this way, the existing
segmentation labels can be reused for the edited images, which greatly reduces the labor costs. Using our pipeline, we construct two new
benchmarks, Pascal-EA and COCO-EA. We
benchmark a wide variety of semantic
segmentation models, spanning from closed-set models to open-vocabulary large models. We have several key findings: 1) advanced open-vocabulary models do not exhibit greater robustness compared to closed-set methods under
geometric variations; 2) data augmentation techniques, such as CutOut and CutMix, are limited in enhancing robustness against appearance variations; 3) our pipeline can also be employed as a data augmentation tool and improve both in-distribution and out-of-distribution performances. Our work suggests the potential of generative models as effective tools for automatically analyzing
segmentation models, and we hope our findings will assist practitioners and researchers in developing more robust and reliable
segmentation models.
Read the full abstract ▾
All authors Zijin Yin, Bing Li, Kongming Liang, Hao Sun, Zhongjiang He, Zhanyu Ma, Jun Guo
Computer Vision
Boosting AI Reliability with an FSM-Driven Streaming Inference Pipeline: An Industrial Case
Yutian Zhang, Zhongyi Pei, Yi Mao et al. (+3 more)
The widespread adoption of AI in industry is often hampered by its limited robustness when faced with scenarios absent from training data, leading to prediction bias and vulnerabilities. To address this, we propose a novel streaming inference pipeline that enhances data-driven models by explicitly incorporating prior knowledge. This paper presents the work on an industrial AI application that automatically counts excavator workloads from surveillance
videos. Our approach integrates an object detection model with a Finite State Machine (FSM), which encodes knowledge of operational scenarios to guide and correct the AI's predictions on streaming data. In experiments on a real-world
dataset of over 7,000 images from 12 site
videos, encompassing more than 300 excavator workloads, our method demonstrates superior performance and greater robustness compared to the original solution based on manual heuristic rules. We will release the code at https://github.com/thulab/
video-streamling-inference-pipeline.
Read the full abstract ▾
All authors Yutian Zhang, Zhongyi Pei, Yi Mao, Chen Wang, Lin Liu, Jianmin Wang
Computer Vision
Better Matching, Less Forgetting: A Quality-Guided Matcher for Transformer-based Incremental Object Detection
Qirui Wu, Shizhou Zhang, De Cheng et al. (+4 more)
Incremental Object Detection (IOD) aims to continuously learn new object classes without forgetting previously learned ones. A persistent challenge is catastrophic forgetting, primarily attributed to background shift in conventional detectors. While pseudo-labeling mitigates this in dense detectors, we identify a novel, distinct source of forgetting specific to DETR-like architectures: background foregrounding. This arises from the exhaustiveness constraint of the Hungarian matcher, which forcibly assigns every ground truth target to one prediction, even when predictions primarily cover background regions (i.e., low IoU). This erroneous supervision compels the model to misclassify background features as specific foreground classes, disrupting learned representations and accelerating forgetting. To address this, we propose a Quality-guided Min-Cost Max-Flow (Q-MCMF) matcher. To avoid forced assignments, Q-MCMF builds a flow graph and prunes implausible matches based on
geometric quality. It then optimizes for the final matching that minimizes cost and maximizes valid assignments. This strategy eliminates harmful supervision from background foregrounding while maximizing foreground learning signals. Extensive experiments on the COCO
dataset under various incremental settings demonstrate that our method consistently outperforms existing state-of-the-art approaches.
Read the full abstract ▾
All authors Qirui Wu, Shizhou Zhang, De Cheng, Yinghui Xing, Lingyan Ran, Dahu Shi, Peng Wang
Computer Vision
FACE: A Face-based Autoregressive Representation for High-Fidelity and Efficient Mesh Generation
Hanxiao Wang, Yuan-Chen Guo, Ying-Tian Liu et al. (+6 more)
Autoregressive models for 3D mesh generation suffer from a fundamental limitation: they flatten meshes into long vertex-coordinate sequences. This results in prohibitive computational costs, hindering the efficient synthesis of high-fidelity geometry. We argue this bottleneck stems from operating at the wrong semantic level. We introduce FACE, a novel Autoregressive Autoencoder (ARAE) framework that reconceptualizes the task by generating meshes at the face level. Our one-face-one-
token strategy treats each triangle face, the fundamental building block of a mesh, as a single, unified
token. This simple yet powerful design reduces the sequence length by a factor of nine, leading to an unprecedented compression ratio of 0.11, halving the previous state-of-the-art. This dramatic efficiency gain does not compromise quality; by pairing our face-level decoder with a powerful VecSet encoder, FACE achieves state-of-the-art reconstruction quality on standard
benchmarks. The versatility of the learned latent space is further demonstrated by training a latent
diffusion model that achieves high-fidelity, single-image-to-mesh generation. FACE provides a simple, scalable, and powerful paradigm that lowers the barrier to high-quality structured 3D content creation.
Read the full abstract ▾
All authors Hanxiao Wang, Yuan-Chen Guo, Ying-Tian Liu, Zi-Xin Zou, Biao Zhang, Weize Quan, Ding Liang, Yan-Pei Cao, Dong-Ming Yan
Computer Vision
Retrieval, Refinement, and Ranking for Text-to-Video Generation via Prompt Optimization and Test-Time Scaling
Zillur Rahman, Alex Sheng and Cristian Meo
While large-scale
datasets have driven significant progress in Text-to-
Video (T2V) generative models, these models remain highly sensitive to input
prompts, demonstrating that
prompt design is critical to generation quality. Current methods for improving
video output often fall short: they either depend on complex, post-editing models, risking the introduction of artifacts, or require expensive fine-tuning of the core generator, which severely limits both scalability and accessibility. In this work, we introduce 3R, a novel RAG based
prompt optimization framework. 3R utilizes the power of current state-of-the-art T2V
diffusion model and
vision language model. It can be used with any T2V model without any kind of model training. The framework leverages three key strategies: RAG-based modifiers extraction for enriched contextual grounding,
diffusion-based Preference Optimization for aligning outputs with human preferences, and temporal frame interpolation for producing temporally consistent visual contents. Together, these components enable more accurate, efficient, and contextually aligned text-to-
video generation. Experimental results demonstrate the efficacy of 3R in enhancing the static fidelity and dynamic coherence of generated
videos, underscoring the importance of optimizing user
prompts.
Read the full abstract ▾
Computer Vision
OMG-Avatar: One-shot Multi-LOD Gaussian Head Avatar
Jianqiang Ren, Lin Liu and Steven Hoi
We propose OMG-Avatar, a novel One-shot method that leverages a Multi-LOD (Level-of-Detail) Gaussian representation for animatable 3D head reconstruction from a single image in 0.2s. Our method enables LOD head avatar modeling using a unified model that accommodates diverse hardware capabilities and inference speed requirements. To capture both global and local facial characteristics, we employ a
transformer-based architecture for global feature extraction and projection-based sampling for local feature acquisition. These features are effectively fused under the guidance of a depth buffer, ensuring occlusion plausibility. We further introduce a coarse-to-fine learning paradigm to support Level-of-Detail functionality and enhance the perception of hierarchical details. To address the limitations of 3DMMs in modeling non-head regions such as the shoulders, we introduce a multi-region decomposition scheme in which the head and shoulders are predicted separately and then integrated through cross-region combination. Extensive experiments demonstrate that OMG-Avatar outperforms state-of-the-art methods in reconstruction quality, reenactment performance, and computational efficiency. The project homepage is https://human3daigc.github.io/OMGAvatar_project_page/ .
Read the full abstract ▾
Computer Vision
Tri-path DINO: Feature Complementary Learning for Remote Sensing Multi-Class Change Detection
Kai Zheng, Hang-Cheng Dong, Zhenkai Wu et al. (+3 more)
In remote sensing imagery, multi-class change detection (MCD) is crucial for fine-grained monitoring, yet it has long been plagued by feature ambiguity in change regions, which obscures subtle changes and hinders fine-grained discrimination under complex scene variations and scarce annotations. To address this, we propose the Tri-path DINO architecture, which adopts a multi-path decoupled complementary feature learning strategy to separately capture coarse-grained semantic features and fine-grained target-edge features. Specifically, we employ the DINOv3 pre-trained model as the backbone feature extraction network to learn coarse-grained features. An auxiliary path also adopts a siamese structure, progressively aggregating intermediate features from the siamese encoder to enhance the learning of fine-grained features. Finally, a multi-scale
attention mechanism is introduced to augment the decoder network, where parallel convolutions adaptively capture and enhance contextual information under different receptive fields. The proposed method achieves state-of-the-art performance on Gaza-change, SECOND, and LandsatSCD, improving mIoU/$F_{\mathrm{scd}}$ by 1.01%/2.93%, 0.67%/2.84%, and 1.22%/1.51%, respectively, over the best competing method. Grad-CAM visualizations further confirm that the main and auxiliary paths naturally focus on coarse-grained semantic changes and fine-grained structural details, respectively. This synergistic complementarity provides a robust and interpretable solution for advanced change detection tasks, offering a basis for rapid and accurate damage assessment.
Read the full abstract ▾
All authors Kai Zheng, Hang-Cheng Dong, Zhenkai Wu, Fupeng Wei, Qin Huang, Wei Zhang
Computer Vision · Cross-listed from cs.IR
PhotoBench: Beyond Visual Matching Towards Personalized Intent-Driven Photo Retrieval
Tianyi Xu, Rong Shan, Junjie Wu et al. (+11 more)
Personal photo albums are not merely collections of static images but living, ecological archives defined by temporal continuity, social
entanglement, and rich metadata, which makes the personalized photo
retrieval non-trivial. However, existing
retrieval benchmarks rely heavily on context-isolated web snapshots, failing to capture the multi-source
reasoning required to resolve authentic, intent-driven user queries. To bridge this gap, we introduce PhotoBench, the first
benchmark constructed from authentic, personal albums. It is designed to shift the paradigm from visual matching to personalized multi-source intent-driven
reasoning. Based on a rigorous multi-source profiling framework, which integrates visual semantics, spatial-temporal metadata, social identity, and temporal events for each image, we synthesize complex intent-driven queries rooted in users' life trajectories. Extensive evaluation on PhotoBench exposes two critical limitations: the modality gap, where unified
embedding models collapse on non-visual constraints, and the source fusion paradox, where agentic systems perform poor tool orchestration. These findings indicate that the next frontier in personal
multimodal retrieval lies beyond unified
embeddings, necessitating robust agentic
reasoning systems capable of precise constraint satisfaction and multi-source fusion. Our PhotoBench is available.
Read the full abstract ▾
All authors Tianyi Xu, Rong Shan, Junjie Wu, Jiadeng Huang, Teng Wang, Jiachen Zhu, Wenteng Chen, Minxin Tu, Quantao Dou, Zhaoxiang Wang, Changwang Zhang, Weinan Zhang, Jun Wang, Jianghao Lin
Computer Vision
Radiometrically Consistent Gaussian Surfels for Inverse Rendering
Kyu Beom Han, Jaeyoon Kim, Woo Jae Kim et al. (+2 more)
Inverse rendering with Gaussian Splatting has advanced rapidly, but accurately disentangling material properties from complex global illumination effects, particularly indirect illumination, remains a major challenge. Existing methods often query indirect radiance from Gaussian primitives pre-trained for novel-view synthesis. However, these pre-trained Gaussian primitives are supervised only towards limited training viewpoints, thus lack supervision for modeling indirect radiances from unobserved views. To address this issue, we introduce radiometric consistency, a novel physically-based constraint that provides supervision towards unobserved views by minimizing the residual between each Gaussian primitive's learned radiance and its physically-based rendered counterpart. Minimizing the residual for unobserved views establishes a self-correcting feedback loop that provides supervision from both physically-based rendering and novel-view synthesis, enabling accurate modeling of inter-reflection. We then propose Radiometrically Consistent Gaussian Surfels (RadioGS), an inverse rendering framework built upon our principle by efficiently integrating radiometric consistency by utilizing Gaussian surfels and 2D Gaussian ray tracing. We further propose a finetuning-based relighting strategy that adapts Gaussian surfel radiances to new illuminations within minutes, achieving low rendering cost (<10ms). Extensive experiments on existing inverse rendering
benchmarks show that RadioGS outperforms existing Gaussian-based methods in inverse rendering, while retaining the computational efficiency.
Read the full abstract ▾
All authors Kyu Beom Han, Jaeyoon Kim, Woo Jae Kim, Jinhwan Seo, Sung-eui Yoon
Computer Vision
ATA: Bridging Implicit Reasoning with Attention-Guided and Action-Guided Inference for Vision-Language Action Models
Cheng Yang, Jianhao Jiao, Lingyi Huang et al. (+8 more)
Vision-
Language-Action (VLA) models rely on current observations, including images,
language instructions, and
robot states, to predict actions and complete tasks. While accurate visual perception is crucial for precise action prediction and execution, recent work has attempted to further improve performance by introducing explicit
reasoning during inference. However, such approaches face significant limitations. They often depend on data-intensive resources such as Chain-of-Thought (CoT) style annotations to decompose tasks into step-by-step
reasoning, and in many cases require additional visual grounding annotations (e.g., bounding boxes or masks) to highlight relevant image regions. Moreover, they involve time-consuming
dataset construction, labeling, and retraining, which ultimately results in longer inference sequences and reduced efficiency. To address these challenges, we propose ATA, a novel training-free framework that introduces implicit
reasoning into VLA inference through complementary
attention-guided and action-guided strategies. Unlike CoT or explicit visual-grounding methods, ATA formulates
reasoning implicitly by integrating
attention maps with an action-based region of interest (RoI), thereby adaptively refining visual inputs without requiring extra training or annotations. ATA is a plug-and-play implicit
reasoning approach for VLA models, lightweight yet effective. Extensive experiments show that it consistently improves task success and robustness while preserving, and even enhancing, inference efficiency.
Read the full abstract ▾
All authors Cheng Yang, Jianhao Jiao, Lingyi Huang, Jinqi Xiao, Zhexiang Tang, Yu Gong, Yibiao Ying, Yang Sui, Jintian Lin, Wen Huang, Bo Yuan
Computer Vision
SCATR: Mitigating New Instance Suppression in LiDAR-based Tracking-by-Attention via Second Chance Assignment and Track Query Dropout
Brian Cheong, Letian Wang, Sandro Papais et al. (+1 more)
LiDAR-based tracking-by-
attention (TBA) frameworks inherently suffer from high false negative errors, leading to a significant performance gap compared to traditional
LiDAR-based tracking-by-detection (TBD) methods. This paper introduces SCATR, a novel
LiDAR-based TBA model designed to address this fundamental challenge systematically. SCATR leverages recent progress in
vision-based tracking and incorporates targeted training strategies specifically adapted for
LiDAR. Our work's core innovations are two architecture-agnostic training strategies for TBA methods: Second Chance Assignment and Track Query Dropout. Second Chance Assignment is a novel ground truth assignment that concatenates unassigned track queries to the proposal queries before bipartite matching, giving these track queries a second chance to be assigned to a ground truth object and effectively mitigating the conflict between detection and tracking tasks inherent in tracking-by-
attention. Track Query Dropout is a training method that diversifies supervised object query configurations to efficiently train the decoder to handle different track query sets, enhancing robustness to missing or newborn tracks. Experiments on the nuScenes tracking
benchmark demonstrate that SCATR achieves state-of-the-art performance among
LiDAR-based TBA methods, outperforming previous works by 7.6% AMOTA and successfully bridging the long-standing performance gap between
LiDAR-based TBA and TBD methods. Ablation studies further validate the effectiveness and generalization of Second Chance Assignment and Track Query Dropout. Code can be found at the following link: \href{https://github.com/TRAILab/SCATR}{https://github.com/TRAILab/SCATR}
Read the full abstract ▾
All authors Brian Cheong, Letian Wang, Sandro Papais, Steven L. Waslander
Computer Vision
WildCross: A Cross-Modal Large Scale Benchmark for Place Recognition and Metric Depth Estimation in Natural Environments
Joshua Knights, Joseph Reid, Kaushik Roy et al. (+3 more)
Recent years have seen a significant increase in demand for robotic solutions in unstructured natural environments, alongside growing interest in bridging 2D and 3D scene understanding. However, existing robotics
datasets are predominantly captured in structured urban environments, making them inadequate for addressing the challenges posed by complex, unstructured natural settings. To address this gap, we propose WildCross, a cross-modal
benchmark for place recognition and metric depth estimation in large-scale natural environments. WildCross comprises over 476K sequential RGB frames with semi-dense depth and surface normal annotations, each aligned with accurate 6DoF poses and synchronized dense
lidar submaps. We conduct comprehensive experiments on visual,
lidar, and cross-modal place recognition, as well as metric depth estimation, demonstrating the value of WildCross as a challenging
benchmark for multi-modal robotic perception tasks. We provide access to the code repository and
dataset at https://csiro-robotics.github.io/WildCross.
Read the full abstract ▾
All authors Joshua Knights, Joseph Reid, Kaushik Roy, David Hall, Mark Cox, Peyman Moghadam
Computer Vision
UltraStar: Semantic-Aware Star Graph Modeling for Echocardiography Navigation
Teng Wang, Haojun Jiang, Chenxi Li et al. (+6 more)
Echocardiography is critical for diagnosing cardiovascular diseases, yet the shortage of skilled sonographers hinders timely patient care, due to high operational difficulties. Consequently, research on automated probe
navigation has significant clinical potential. To achieve robust
navigation, it is essential to leverage historical scanning information, mimicking how experts rely on past feedback to adjust subsequent maneuvers. Practical scanning data collected from sonographers typically consists of noisy trajectories inherently generated through trial-and-error exploration. However, existing methods typically model this history as a sequential chain, forcing models to overfit these noisy paths, leading to performance degradation on long sequences. In this paper, we propose UltraStar, which reformulates probe
navigation from path regression to anchor-based global localization. By establishing a
Star Graph, UltraStar treats historical keyframes as spatial anchors connected directly to the current view, explicitly modeling
geometric constraints for precise positioning. We further enhance the
Star Graph with a semantic-aware sampling strategy that actively selects the representative landmarks from massive history logs, reducing redundancy for accurate anchoring. Extensive experiments on a
dataset with over 1.31 million samples demonstrate that UltraStar outperforms baselines and scales better with longer input lengths, revealing a more effective
topology for history modeling under noisy exploration. Code is available at https://github.com/LeapLabTHU/UltraStar.
Read the full abstract ▾
All authors Teng Wang, Haojun Jiang, Chenxi Li, Diwen Wang, Yihang Tang, Zhenguo Sun, Yujiao Deng, Shiji Song, Gao Huang
Computer Vision
From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents
Niu Lian, Yuting Wang, Hanshu Yao et al. (+5 more)
While
multimodal large
language models have demonstrated impressive short-term
reasoning, they struggle with long-horizon
video understanding due to limited context windows and static
memory mechanisms that fail to mirror human cognitive efficiency. Existing paradigms typically fall into two extremes:
vision-centric methods that incur high latency and redundancy through dense visual accumulation, or text-centric approaches that suffer from detail loss and
hallucination via aggressive captioning. To bridge this gap, we propose MM-Mem, a pyramidal
multimodal memory architecture grounded in Fuzzy-Trace Theory. MM-Mem structures
memory hierarchically into a Sensory Buffer, Episodic Stream, and Symbolic Schema, enabling the progressive
distillation of fine-grained perceptual traces (verbatim) into high-level semantic schemas (gist). Furthermore, to govern the dynamic construction of
memory, we derive a Semantic Information Bottleneck objective and introduce SIB-GRPO to optimize the trade-off between
memory compression and task-relevant information retention. In inference, we design an
entropy-driven top-down
memory retrieval strategy. Extensive experiments across 4
benchmarks confirm that MM-Mem achieves state-of-the-art performance on both offline and streaming tasks, demonstrating robust generalization and validating the effectiveness of cognition-inspired
memory organization. Code and associated configurations are publicly available at https://github.com/EliSpectre/MM-Mem.
Read the full abstract ▾
All authors Niu Lian, Yuting Wang, Hanshu Yao, Jinpeng Wang, Bin Chen, Yaowei Wang, Min Zhang, Shu-Tao Xia
Computer Vision
VidDoS: Universal Denial-of-Service Attack on Video-based Large Language Models
Duoxun Tang, Dasen Dai, Jiyao Wang et al. (+3 more)
Video-LLMs are increasingly deployed in safety-critical applications but are vulnerable to Energy-Latency Attacks (ELAs) that exhaust computational resources. Current image-centric methods fail because temporal aggregation mechanisms dilute individual frame perturbations. Additionally, real-time demands make instance-wise optimization impractical for continuous
video streams. We introduce VidDoS, which is the first universal ELA framework tailored for
Video-LLMs. Our method leverages universal optimization to create instance-agnostic triggers that require no inference-time gradient calculation. We achieve this through $\textit{masked teacher forcing}$ to steer models toward expensive target sequences, combined with a $\textit{refusal penalty}$ and $\textit{early-termination suppression}$ to override conciseness priors. Testing across three mainstream
Video-LLMs and three
video datasets, which include
video question answering and autonomous driving scenarios, shows extreme degradation. VidDoS induces a
token expansion of more than 205$\times$ and inflates the inference latency by more than 15$\times$ relative to clean baselines. Simulations of real-time autonomous driving streams further reveal that this induced latency leads to critical safety violations. We urge the community to recognize and mitigate these high-hazard ELA in
Video-LLMs.
Read the full abstract ▾
All authors Duoxun Tang, Dasen Dai, Jiyao Wang, Xiao Yang, Jianyu Wang, Siqi Cai
Computer Vision
Deepfake Forensics Adapter: A Dual-Stream Network for Generalizable Deepfake Detection
Jianfeng Liao, Yichen Wei, Raymond Chan Ching Bon et al. (+3 more)
The rapid advancement of deepfake generation techniques poses significant threats to public safety and causes societal harm through the creation of highly realistic synthetic facial media. While existing detection methods demonstrate limitations in generalizing to emerging forgery patterns, this paper presents Deepfake Forensics Adapter (DFA), a novel dual-stream framework that synergizes
vision-
language foundation models with targeted forensics analysis. Our approach integrates a pre-trained CLIP model with three core components to achieve specialized deepfake detection by leveraging the powerful general capabilities of CLIP without changing CLIP parameters: 1) A Global Feature Adapter is used to identify global inconsistencies in image content that may indicate forgery, 2) A Local Anomaly Stream enhances the model's ability to perceive local facial forgery cues by explicitly leveraging facial structure priors, and 3) An Interactive Fusion Classifier promotes deep interaction and fusion between global and local features using a
transformer encoder. Extensive evaluations of frame-level and
video-level
benchmarks demonstrate the superior generalization capabilities of DFA, particularly achieving state-of-the-art performance in the challenging DFDC
dataset with frame-level AUC/EER of 0.816/0.256 and
video-level AUC/EER of 0.836/0.251, representing a 4.8%
video AUC improvement over previous methods. Our framework not only demonstrates state-of-the-art performance, but also points out a feasible and effective direction for developing a robust deepfake detection system with enhanced generalization capabilities against the evolving deepfake threats. Our code is available at https://github.com/Liao330/DFA.git
Read the full abstract ▾
All authors Jianfeng Liao, Yichen Wei, Raymond Chan Ching Bon, Shulan Wang, Kam-Pui Chow, Kwok-Yan Lam
Computer Vision · Cross-listed from eess.IV
Revisiting Global Token Mixing in Task-Dependent MRI Restoration: Insights from Minimal Gated CNN Baselines
Xiangjian Hou, Chao Qin, Chang Ni et al. (+3 more)
Global
token mixing, implemented via self-
attention or state-space sequence models, has become a popular model design choice for MRI restoration. However, MRI restoration tasks differ substantially in how their degradations vary over image and k-space domains, and in the degree to which global coupling is already imposed by physics-driven data consistency terms. In this work, we ask the question whether global
token mixing is actually beneficial in each individual task across three representative settings: accelerated MRI reconstruction with explicit data consistency, MRI super-resolution with k-space center cropping, and denoising of clinical carotid MRI data with spatially heteroscedastic noise. To reduce confounding factors, we establish a controlled testbed comparing a minimal local gated CNN and its large-field variant, benchmarking them directly against state-of-the-art global models under aligned training and evaluation protocols. For accelerated MRI reconstruction, the minimal unrolled gated-CNN baseline is already highly competitive compared to recent
token-mixing approaches in public reconstruction
benchmarks, suggesting limited additional benefits when the forward model and data-consistency steps provide strong global constraints. For super-resolution, where low-frequency k-space data are largely preserved by the controlled low-pass degradation, local gated models remain competitive, and a lightweight large-field variant yields only modest improvements. In contrast, for denoising with pronounced spatially heteroscedastic noise,
token-mixing models achieve the strongest overall performance, consistent with the need to estimate spatially varying reliability. In conclusion, our results demonstrate that the utility of global
token mixing in MRI restoration is task-dependent, and it should be tailored to the underlying imaging physics and degradation structure.
Read the full abstract ▾
All authors Xiangjian Hou, Chao Qin, Chang Ni, Xin Wang, Chun Yuan, Xiaodong Ma
Computer Vision
Unifying Language-Action Understanding and Generation for Autonomous Driving
Xinyang Wang, Qian Liu, Wenjie Ding et al. (+7 more)
Vision-
Language-Action (VLA) models are emerging as a promising paradigm for end-to-end autonomous driving, valued for their potential to leverage world knowledge and reason about complex driving scenes. However, existing methods suffer from two critical limitations: a persistent misalignment between
language instructions and action outputs, and the inherent inefficiency of typical auto-regressive action generation. In this paper, we introduce LinkVLA, a novel architecture that directly addresses these challenges to enhance both
alignment and efficiency. First, we establish a structural link by unifying
language and action
tokens into a shared discrete codebook, processed within a single multi-modal model. This structurally enforces cross-modal consistency from the ground up. Second, to create a deep semantic link, we introduce an auxiliary action understanding objective that trains the model to generate descriptive captions from trajectories, fostering a bidirectional
language-action mapping. Finally, we replace the slow, step-by-step generation with a two-step coarse-to-fine generation method C2F that efficiently decodes the action sequence, saving 86% inference time. Experiments on closed-loop driving
benchmarks show consistent gains in instruction following accuracy and driving performance, alongside reduced inference latency.
Read the full abstract ▾
All authors Xinyang Wang, Qian Liu, Wenjie Ding, Zhao Yang, Wei Li, Chang Liu, Bailin Li, Kun Zhan, Xianpeng Lang, Wei Chen
Computer Vision
DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis
Zengqi Zhao, Weidi Xia, En Wei et al. (+7 more)
We present DOCFORGE-BENCH, the first unified zero-shot
benchmark for document forgery detection, evaluating 14 methods across eight
datasets spanning text tampering, receipt forgery, and identity document manipulation. Unlike fine-tuning-oriented evaluations such as ForensicHub [Du et al., 2025], DOCFORGE-BENCH applies all methods with their published pretrained weights and no domain adaptation – a deliberate design choice that reflects the realistic deployment scenario where practitioners lack labeled document training data. Our central finding is a pervasive calibration failure invisible under single-threshold protocols: methods achieve moderate Pixel-AUC (>=0.76) yet near-zero Pixel-F1. This AUC-F1 gap is not a discrimination failure but a score-distribution shift: tampered regions occupy only 0.27-4.17% of pixels in document images – an order of magnitude less than in natural image
benchmarks – making the standard tau=0.5 threshold catastrophically miscalibrated. Oracle-F1 is 2-10x higher than fixed-threshold Pixel-F1, confirming that calibration, not representation, is the bottleneck. A controlled calibration experiment validates this: adapting a single threshold on N=10 domain images recovers 39-55% of the Oracle-F1 gap, demonstrating that threshold adaptation – not retraining – is the key missing step for practical deployment. Overall, no evaluated method works reliably out-of-the-box on diverse document types, underscoring that document forgery detection remains an unsolved problem. We further note that all eight
datasets predate the era of generative AI editing;
benchmarks covering
diffusion- and LLM-based document forgeries represent a critical open gap on the modern attack surface.
Read the full abstract ▾
All authors Zengqi Zhao, Weidi Xia, En Wei, Yan Zhang, Jane Mo, Tiannan Zhang, Yuanqin Dai, Zexi Chen, Yiran Tao, Simiao Ren
Computer Vision
SeaVIS: Sound-Enhanced Association for Online Audio-Visual Instance Segmentation
Yingjian Zhu, Ying Wang, Yuyang Hong et al. (+5 more)
Recently, an audio-visual instance
segmentation (AVIS) task has been introduced, aiming to identify, segment and track individual sounding instances in
videos. However, prevailing methods primarily adopt the offline paradigm, that cannot associate detected instances across consecutive clips, making them unsuitable for real-world scenarios that involve continuous
video streams. To address this limitation, we introduce SeaVIS, the first online framework designed for audio-visual instance
segmentation. SeaVIS leverages the Causal Cross
Attention Fusion (CCAF) module to enable efficient online processing, which integrates visual features from the current frame with the entire audio history under strict causal constraints. A major challenge for conventional VIS methods is that appearance-based instance association fails to distinguish between an object's sounding and silent states, resulting in the incorrect
segmentation of silent objects. To tackle this, we employ an Audio-Guided Contrastive Learning (AGCL) strategy to generate instance prototypes that encode not only visual appearance but also sounding activity. In this way, instances preserved during per-frame prediction that do not emit sound can be effectively suppressed during instance association process, thereby significantly enhancing the audio-following capability of SeaVIS. Extensive experiments conducted on the AVISeg
dataset demonstrate that SeaVIS surpasses existing state-of-the-art models across multiple evaluation metrics while maintaining a competitive inference speed suitable for real-time processing.
Read the full abstract ▾
All authors Yingjian Zhu, Ying Wang, Yuyang Hong, Ruohao Guo, Kun Ding, Xin Gu, Bin Fan, Shiming Xiang
Computer Vision
UniTalking: A Unified Audio-Video Framework for Talking Portrait Generation
Hebeizi Li, Zihao Liang, Benyuan Sun et al. (+4 more)
While state-of-the-art audio-
video generation models like Veo3 and Sora2 demonstrate remarkable capabilities, their closed-source nature makes their architectures and training paradigms inaccessible. To bridge this gap in accessibility and performance, we introduce UniTalking, a unified, end-to-end
diffusion framework for generating high-fidelity
speech and lip-synchronized
video. At its core, our framework employs Multi-Modal
Transformer Blocks to explicitly model the fine-grained temporal correspondence between audio and
video latent
tokens via a shared self-
attention mechanism. By leveraging powerful priors from a pre-trained
video generation model, our framework ensures state-of-the-art visual fidelity while enabling efficient training. Furthermore, UniTalking incorporates a personalized voice cloning capability, allowing the generation of
speech in a target style from a brief audio reference. Qualitative and quantitative results demonstrate that our method produces highly realistic talking portraits, achieving superior performance over existing open-source approaches in lip-sync accuracy, audio naturalness, and overall perceptual quality.
Read the full abstract ▾
All authors Hebeizi Li, Zihao Liang, Benyuan Sun, Zihao Yin, Xiao Sha, Chenliang Wang, Yi Yang
Computer Vision
UETrack: A Unified and Efficient Framework for Single Object Tracking
Ben Kang, Jie Zhao, Xin Chen et al. (+5 more)
With growing real-world demands, efficient tracking has received increasing
attention. However, most existing methods are limited to RGB inputs and struggle in multi-modal scenarios. Moreover, current multi-modal tracking approaches typically use complex designs, making them too heavy and slow for resource-constrained deployment. To tackle these limitations, we propose UETrack, an efficient framework for single object tracking. UETrack demonstrates high practicality and versatility, efficiently handling multiple modalities including RGB, Depth, Thermal, Event, and
Language, and addresses the gap in efficient multi-modal tracking. It introduces two key components: a
Token-Pooling-based Mixture-of-Experts mechanism that enhances modeling capacity through feature aggregation and expert specialization, and a Target-aware Adaptive
Distillation strategy that selectively performs
distillation based on sample characteristics, reducing redundant supervision and improving performance. Extensive experiments on 12
benchmarks across 3 hardware platforms show that UETrack achieves a superior speed-accuracy trade-off compared to previous methods. For instance, UETrack-B achieves 69.2% AUC on LaSOT and runs at 163/56/60 FPS on GPU/CPU/AGX, demonstrating strong practicality and versatility. Code is available at https://github.com/kangben258/UETrack.
Read the full abstract ▾
All authors Ben Kang, Jie Zhao, Xin Chen, Wanting Geng, Bin Zhang, Lu Zhang, Dong Wang, Huchuan Lu