Computer Vision
4DSurf: High-Fidelity Dynamic Scene Surface Reconstruction
Renjie Wu, Hongdong Li, Jose M. Alvarez et al. (+1 more)
This paper addresses the problem of dynamic scene surface reconstruction using Gaussian Splatting (GS), aiming to recover temporally consistent geometry. While existing GS-based dynamic surface reconstruction methods can yield superior reconstruction, they are typically limited to either a single object or objects with only small deformations, struggling to maintain temporally consistent surface reconstruction of large deformations over time. We propose “4DSurf”, a novel and unified framework for generic dynamic surface reconstruction that does not require specifying the number or types of objects in the scene, can handle large surface deformations and temporal inconsistency in reconstruction. The key innovation of our framework is the introduction of Gaussian deformations induced Signed Distance Function Flow Regularization that constrains the motion of Gaussians to align with the evolving surface. To handle large deformations, we introduce an Overlapping Segment Partitioning strategy that divides the sequence into overlapping segments with small deformations and incrementally passes
geometric information across segments through the shared overlapping timestep. Experiments on two challenging dynamic scene
datasets, Hi4D and CMU Panoptic, demonstrate that our method outperforms state-of-the-art surface reconstruction methods by 49% and 19% in Chamfer distance, respectively, and achieves superior temporal consistency under sparse-view settings.
Read the full abstract ▾
All authors Renjie Wu, Hongdong Li, Jose M. Alvarez, Miaomiao Liu
Computer Vision
Object Detection Based on Distributed Convolutional Neural Networks
Liang Sun
Based on the Distributed Convolutional Neural Network(DisCNN), a straightforward object detection method is proposed. The modules of the output vector of a DisCNN with respect to a specific positive class are positively monotonic with the presence probabilities of the positive features. So, by identifying all high-scoring patches across all possible scales, the positive object can be detected by overlapping them to form a bounding box. The essential idea is that the object is detected by detecting its features on multiple scales, ranging from specific sub-features to abstract features composed of these sub-features. Training DisCNN requires only object-centered image data with positive and negative class labels. The detection process for multiple positive classes can be conducted in parallel to significantly accelerate it, and also faster for single-object detection because of its lightweight model architecture.
Read the full abstract ▾
Computer Vision
Drift-AR: Single-Step Visual Autoregressive Generation via Anti-Symmetric Drifting
Zhen Zou, Xiaoxiao Ma, Mingde Yao et al. (+3 more)
Autoregressive (AR)-
Diffusion hybrid paradigms combine AR's structured semantic modeling with
diffusion's high-fidelity synthesis, yet suffer from a dual speed bottleneck: the sequential AR stage and the iterative multi-step denoising of the
diffusion vision decode stage. Existing methods address each in isolation without a unified principle design. We observe that the per-position prediction
entropy of continuous-space AR models naturally encodes spatially varying generation uncertainty, which simultaneously governing draft prediction quality in the AR stage and reflecting the corrective effort required by
vision decoding stage, which is not fully explored before. Since
entropy is inherently tied to both bottlenecks, it serves as a natural unifying signal for joint acceleration. In this work, we propose Drift-AR, which leverages
entropy signal to accelerate both stages: 1) for AR acceleration, we introduce
Entropy-Informed Speculative Decoding that align draft-target
entropy distributions via a causal-normalized
entropy loss, resolving the
entropy mismatch that causes excessive draft rejection; 2) for visual decoder acceleration, we reinterpret
entropy as the physical variance of the initial state for an anti-symmetric drifting field – high-
entropy positions activate stronger drift toward the data
manifold while low-
entropy positions yield vanishing drift – enabling single-step (1-NFE) decoding without iterative denoising or
distillation. Moreover, both stages share the same
entropy signal, which is computed once with no extra cost. Experiments on MAR, TransDiff, and NextStep-1 demonstrate 3.8-5.5$\times$ speedup with genuine 1-NFE decoding, matching or surpassing original quality. Code will be available at https://github.com/aSleepyTree/Drift-AR.
Read the full abstract ▾
All authors Zhen Zou, Xiaoxiao Ma, Mingde Yao, Jie Huang, LinJiang Huang, Feng Zhao
Computer Vision
Event6D: Event-based Novel Object 6D Pose Tracking
Jae-Young Kang, Hoonhee Cho, Taeyeop Lee et al. (+4 more)
Event cameras provide microsecond latency, making them suitable for 6D object pose tracking in fast, dynamic scenes where conventional RGB and depth pipelines suffer from motion blur and large pixel displacements. We introduce EventTrack6D, an event-depth tracking framework that generalizes to novel objects without object-specific training by reconstructing both intensity and depth at arbitrary timestamps between depth frames. Conditioned on the most recent depth measurement, our dual reconstruction recovers dense photometric and
geometric cues from sparse event streams. Our EventTrack6D operates at over 120 FPS and maintains temporal consistency under rapid motion. To support training and evaluation, we introduce a comprehensive
benchmark suite: a large-scale synthetic
dataset for training and two complementary evaluation sets, including real and simulated event
datasets. Trained exclusively on synthetic data, EventTrack6D generalizes effectively to real-world scenarios without fine-tuning, maintaining accurate tracking across diverse objects and motion patterns. Our method and
datasets validate the effectiveness of event cameras for event-based 6D pose tracking of novel objects. Code and
datasets are publicly available at https://chohoonhee.github.io/Event6D.
Read the full abstract ▾
All authors Jae-Young Kang, Hoonhee Cho, Taeyeop Lee, Minjun Kang, Bowen Wen, Youngho Kim, Kuk-Jin Yoon
Computer Vision · Cross-listed from cs.RO
CARLA-Air: Fly Drones Inside a CARLA World – A Unified Infrastructure for Air-Ground Embodied Intelligence
Tianle Zeng, Yanci Wen and Hong Zhang
The convergence of low-altitude economies,
embodied intelligence, and air-ground cooperative systems creates growing demand for simulation infrastructure capable of jointly modeling aerial and ground
agents within a single physically coherent environment. Existing open-source platforms remain domain-segregated: driving simulators lack aerial dynamics, while multirotor simulators lack realistic ground scenes. Bridge-based co-simulation introduces synchronization overhead and cannot guarantee strict spatial-temporal consistency. We present CARLA-Air, an open-source infrastructure that unifies high-fidelity urban driving and physics-accurate multirotor flight within a single Unreal Engine process. The platform preserves both CARLA and AirSim native Python APIs and ROS 2 interfaces, enabling zero-modification code reuse. Within a shared physics tick and rendering pipeline, CARLA-Air delivers photorealistic environments with rule-compliant traffic, socially-aware pedestrians, and aerodynamically consistent
UAV dynamics, synchronously capturing up to 18 sensor modalities across all platforms at each tick. The platform supports representative air-ground
embodied intelligence workloads spanning cooperation,
embodied navigation and
vision-
language action, multi-modal perception and
dataset construction, and reinforcement-learning-based policy training. An extensible asset pipeline allows integration of custom
robot platforms into the shared world. By inheriting AirSim's aerial capabilities – whose upstream development has been archived – CARLA-Air ensures this widely adopted flight stack continues to evolve within a modern infrastructure. Released with prebuilt binaries and full source: https://github.com/louiszengCN/CarlaAir
Read the full abstract ▾
Computer Vision
Effort-Based Criticality Metrics for Evaluating 3D Perception Errors in Autonomous Driving
Sharang Kaul, Simon Bultmann, Mario Berk et al. (+1 more)
Criticality metrics such as time-to-collision (TTC) quantify collision urgency but do not distinguish the operational consequences of false-positive (FP) and false-negative (FN) perception errors. We formulate two error-specific effort metrics: False Speed Reduction (FSR), the cumulative velocity loss associated with persistent phantom detections, and Maximum Deceleration Rate (MDR), the peak braking demand associated with missed objects under a longitudinal kinematic model. These longitudinal metrics are complemented by Lateral Evasion Acceleration (LEA), adapted from prior lateral-evasion kinematics and coupled with reachability-based collision timing. The collision check quantifies the minimum steering effort required to avoid a predicted collision. A dynamically conservative, semantically unfiltered reachability gate selects candidate interactions before frame-level scoring and track-level aggregation. Evaluation on nuScenes and Argoverse 2 shows that 65% to 93% of errors fall below the chosen criticality thresholds. Correlation and threshold analysis indicate that the proposed metrics provide complementary rankings for screening and mining perception failures and are not substitutes for closed-loop safety validation.
Read the full abstract ▾
All authors Sharang Kaul, Simon Bultmann, Mario Berk, Abhinav Valada
Computer Vision
Efficient Domain Adaptation for Text Line Recognition via Decoupled Language Models
Arundhathi Dev and Justin Zhan
Optical character recognition remains critical infrastructure for document digitization, yet state-of-the-art performance is often restricted to well-resourced institutions by prohibitive computational barriers. End-to-end
transformer architectures achieve strong accuracy but demand hundreds of GPU hours for domain adaptation, limiting accessibility for practitioners and digital humanities scholars. We present a modular detection-and-correction framework that achieves near-SOTA accuracy with single-GPU training. Our approach decouples lightweight visual character detection (domain-agnostic) from domain-specific linguistic correction using pretrained sequence models including T5, ByT5, and BART. By training the correctors entirely on synthetic noise, we enable annotation-free domain adaptation without requiring labeled target images. Evaluating across modern clean handwriting, cursive script, and historical documents, we identify a critical "Pareto frontier" in architecture selection: T5-Base excels on modern text with standard vocabulary, whereas ByT5-Base dominates on historical documents by reconstructing archaic spellings at the byte level. Our results demonstrate that this decoupled paradigm matches end-to-end
transformer accuracy while reducing compute by approximately 95%, establishing a viable, resource-efficient alternative to monolithic OCR architectures.
Read the full abstract ▾
Computer Vision
Adapting SAM to Nuclei Instance Segmentation and Classification via Cooperative Fine-Grained Refinement
Jingze Su, Tianle Zhu, Jiaxin Cai et al. (+6 more)
Nuclei instance
segmentation is critical in computational pathology for cancer diagnosis and prognosis. Recently, the Segment Anything Model has demonstrated exceptional performance in various
segmentation tasks, leveraging its rich priors and powerful global context modeling capabilities derived from large-scale pre-training on natural images. However, directly applying SAM to the medical imaging domain faces significant limitations: it lacks sufficient perception of the local structural features that are crucial for
nuclei segmentation, and full fine-tuning for downstream tasks requires substantial computational costs. To efficiently transfer SAM's robust prior knowledge to
nuclei instance
segmentation while supplementing its task-aware local perception, we propose a parameter-efficient fine-tuning framework, named Cooperative Fine-Grained Refinement of SAM, consisting of three core components: 1) a Multi-scale Adaptive Local-aware Adapter, which enables effective capability transfer by augmenting the frozen SAM backbone with minimal parameters and instilling a powerful perception of local structures through dynamically generated, multi-scale convolutional kernels; 2) a Hierarchical Modulated Fusion Module, which dynamically aggregates multi-level encoder features to preserve fine-grained spatial details; and 3) a Boundary-Guided Mask Refinement, which integrates multi-context boundary cues with semantic features through explicit supervision, producing a boundary-focused signal to refine initial mask predictions for sharper delineation. These three components work cooperatively to enhance local perception, preserve spatial details, and refine boundaries, enabling SAM to perform accurate
nuclei instance
segmentation directly.
Read the full abstract ▾
All authors Jingze Su, Tianle Zhu, Jiaxin Cai, Zhiyi Wang, Qi Li, Xiao Zhang, Tong Tong, Shu Wang, Wenxi Liu
Computer Vision
SegRGB-X: General RGB-X Semantic Segmentation Model
Jiong Liu, Yingjie Xu, Xingcheng Zhou et al. (+4 more)
Semantic
segmentation across arbitrary sensor modalities faces significant challenges due to diverse sensor characteristics, and the traditional configurations for this task result in redundant development efforts. We address these challenges by introducing a universal arbitrary-modal semantic
segmentation framework that unifies
segmentation across multiple modalities. Our approach features three key innovations: (1) the Modality-aware CLIP (MA-CLIP), which provides modality-specific scene understanding guidance through LoRA fine-tuning; (2) Modality-aligned
Embeddings for capturing fine-grained features; and (3) the Domain-specific Refinement Module (DSRM) for dynamic feature adjustment. Evaluated on five diverse
datasets with different complementary modalities (event, thermal, depth, polarization, and light field), our model surpasses specialized multi-modal methods and achieves state-of-the-art performance with a mIoU of 65.03%. The codes will be released upon acceptance.
Read the full abstract ▾
All authors Jiong Liu, Yingjie Xu, Xingcheng Zhou, Rui Song, Walter Zimmer, Alois Knoll, Hu Cao
Computer Vision
Physically Inspired Gaussian Splatting for HDR Novel View Synthesis
Huimin Zeng, Yue Bai, Hailing Wang et al. (+1 more)
High dynamic range novel view synthesis (HDR-NVS) reconstructs scenes with dynamic details by fusing multi-exposure low dynamic range (LDR) views, yet it struggles to capture ambient illumination-dependent appearance. Implicitly supervising HDR content by constraining tone-mapped results fails in correcting abnormal HDR values, and results in limited gradients for Gaussians in under/over-exposed regions. To this end, we introduce PhysHDR-GS, a physically inspired HDR-NVS framework that models scene appearance via intrinsic reflectance and adjustable ambient illumination. PhysHDR-GS employs a complementary image-exposure (IE) branch and Gaussian-illumination (GI) branch to faithfully reproduce standard camera observations and capture illumination-dependent appearance changes, respectively. During training, the proposed cross-branch HDR consistency loss provides explicit supervision for HDR content, while an illumination-guided gradient scaling strategy mitigates exposure-biased gradient starvation and reduces under-densified representations. Experimental results across realistic and synthetic
datasets demonstrate our superiority in reconstructing HDR details (e.g., a PSNR gain of 2.04 dB over HDR-GS), while maintaining real-time rendering speed (up to 76 FPS). Code and models are available at https://huimin-zeng.github.io/PhysHDR-GS/.
Read the full abstract ▾
All authors Huimin Zeng, Yue Bai, Hailing Wang, Yun Fu
Computer Vision
Energy-Aware Imitation Learning for Steering Prediction Using Events and Frames
Hu Cao, Jiong Liu, Xingzhuo Yan et al. (+5 more)
In autonomous driving, relying solely on frame-based cameras can lead to inaccuracies caused by factors like long exposure times, high-speed motion, and challenging lighting conditions. To address these issues, we introduce a bio-inspired
vision sensor known as the event camera. Unlike conventional cameras, event cameras capture sparse, asynchronous events that provide a complementary modality to mitigate these challenges. In this work, we propose an energy-aware imitation learning framework for steering prediction that leverages both events and frames. Specifically, we design an Energy-driven Cross-modality Fusion Module (ECFM) and an energy-aware decoder to produce reliable and safe predictions. Extensive experiments on two public real-world
datasets, DDD20 and DRFuser, demonstrate that our method outperforms existing state-of-the-art (SOTA) approaches. The codes and trained models will be released upon acceptance.
Read the full abstract ▾
All authors Hu Cao, Jiong Liu, Xingzhuo Yan, Rui Song, Yan Xia, Walter Zimmer, Guang Chen, Alois Knoll
Computer Vision
DipGuava: Disentangling Personalized Gaussian Features for 3D Head Avatars from Monocular Video
Jeonghaeng Lee, Seok Keun Choi, Zhixuan Li et al. (+2 more)
While recent 3D head avatar creation methods attempt to animate facial dynamics, they often fail to capture personalized details, limiting realism and expressiveness. To fill this gap, we present DipGuava (Disentangled and Personalized Gaussian UV Avatar), a novel 3D Gaussian head avatar creation method that successfully generates avatars with personalized attributes from
monocular video. DipGuava is the first method to explicitly disentangle facial appearance into two complementary components, trained in a structured two-stage pipeline that significantly reduces learning ambiguity and enhances reconstruction fidelity. In the first stage, we learn a stable geometry-driven base appearance that captures global facial structure and coarse expression-dependent variations. In the second stage, the personalized residual details not captured in the first stage are predicted, including high-frequency components and nonlinearly varying features such as wrinkles and subtle skin deformations. These components are fused via dynamic appearance fusion that integrates residual details after deformation, ensuring spatial and semantic
alignment. This disentangled design enables DipGuava to generate photorealistic, identity-preserving avatars, consistently outperforming prior methods in both visual quality and quantitativeperformance, as demonstrated in extensive experiments.
Read the full abstract ▾
All authors Jeonghaeng Lee, Seok Keun Choi, Zhixuan Li, Weisi Lin, Sanghoon Lee
Computer Vision
CLIP-AUTT: Test-Time Personalization with Action Unit Prompting for Fine-Grained Video Emotion Recognition
Muhammad Osama Zeeshan, Masoumeh Sharafi, Benoit Savary et al. (+3 more)
Personalization in
emotion recognition (ER) is essential for accurate interpretation of subtle and subject-specific expressive patterns. Recent advances in
vision-
language models (VLMs), such as CLIP, demonstrate strong potential for leveraging joint image-text representations in ER. However, existing CLIP-based methods either rely on CLIP's contrastive pretraining or use LLMs to generate descriptive text
prompts, which can be noisy, computationally expensive, and often fail to capture fine-grained expressions, leading to degraded performance. In this work, Action Units (AUs) are leveraged as structured textual
prompts within CLIP to model fine-grained facial expressions. AUs encode the subtle muscle activations underlying expressions, providing localized and interpretable semantic cues for more robust facial expression recognition (FER). We introduce CLIP-AU, a lightweight AU-guided temporal learning method that integrates interpretable AU semantics into CLIP. It learns generic, subject-agnostic representations by aligning AU
prompts with facial dynamics, enabling fine-grained FER without CLIP fine-tuning or LLM-generated text supervision. Although CLIP-AU models fine-grained AU semantics, it does not adapt to subject-specific variability in subtle expressions. To address this limitation, we propose CLIP-AUTT, a
video-based test-time personalization method that dynamically adapts AU
prompts to
videos from unseen subjects. By combining
entropy-guided temporal window selection with
prompt tuning, CLIP-AUTT enables subject-specific adaptation while preserving temporal consistency. Our experiments on three challenging
video-based
datasets, BioVid, StressID, and BAH, indicate that CLIP-AU and CLIP-AUTT outperform state-of-the-art CLIP-based FER and TTA methods.
Read the full abstract ▾
All authors Muhammad Osama Zeeshan, Masoumeh Sharafi, Benoit Savary, Alessandro Lameiras Koerich, Marco Pedersoli, Eric Granger
Computer Vision
UniDA3D: A Unified Domain-Adaptive Framework for Multi-View 3D Object Detection
Hongjing Wu, Cheng Chi, Jinlin Wu et al. (+3 more)
Camera-only 3D object detection is critical for autonomous driving, offering a cost-effective alternative to
LiDAR based methods. In particular, multi-view 3D object detection has emerged as a promising direction due to its balanced trade-off between performance and cost. However, existing methods often suffer significant performance degradation under complex environmental conditions such as nighttime, fog, and rain, primarily due to their reliance on training data collected mostly in ideal conditions. To address this challenge, we propose UniDA3D, a unified domain-adaptive multi-view 3D object detector designed for robust perception under diverse adverse conditions. UniDA3D formulates nighttime, rainy, and foggy scenes as a unified multi target domain adaptation problem and leverages a novel query guided domain discrepancy mitigation (QDDM) module to align object features between source and target domains at both batch and global levels via query-centric
adversarial and contrastive learning. Furthermore, we introduce a domain-adaptive teacher student training pipeline with an exponential-moving-average teacher and dynamically updated high-quality pseudo labels to enhance consistency learning and suppress background noise in unlabeled target domains. In contrast to prior approaches that require separate training for each condition, UniDA3D performs a single unified training process across multiple domains, enabling robust all-weather 3D perception. On a synthesized multi-view 3D
benchmark constructed by generating nighttime, rainy, and foggy counterparts from nuScenes (nuScenes-Night, nuScenes-Rain, and nuScenes-Haze), UniDA3D consistently outperforms state of-the-art camera-only multi-view 3D detectors under extreme conditions, achieving substantial gains in mAP and NDS while maintaining real-time inference efficiency.
Read the full abstract ▾
All authors Hongjing Wu, Cheng Chi, Jinlin Wu, Yanzhao Su, Zhen Lei, Wenqi Ren
Computer Vision
Jiachen Li, Hongyun Wang, Jinyu Xu et al. (+5 more)
Referring image
segmentation aims to localize and segment a target object in an image based on a free-form referring expression. The core challenge lies in effectively bridging linguistic descriptions with object-level visual representations, especially when referring expressions involve detailed attributes and complex inter-object relationships. Existing methods either rely on cross-modal
alignment or employ Semantic
Segmentation Prompts, but they often lack explicit
reasoning mechanisms for grounding
language descriptions to target regions in the image. To address these limitations, we propose PPCR, a Progressive
Prompt-guided Cross-modal
Reasoning framework for referring image
segmentation. PPCR explicitly structures the
reasoning process as a Semantic Understanding-Spatial Grounding-Instance
Segmentation pipeline. Specifically, PPCR first employs
multimodal large
language models (MLLMs) to generate Semantic
Segmentation Prompt that capture key semantic cues of the target object. Based on this semantic context, Spatial
Segmentation Prompt are further generated to reason about object location and spatial extent, enabling a progressive transition from semantic understanding to spatial grounding. The Semantic and Spatial
Segmentation prompts are then jointly integrated into the
segmentation module to guide accurate target localization and
segmentation. Extensive experiments on standard referring image
segmentation benchmarks demonstrate that PPCR consistently outperforms existing methods. The code will be publicly released to facilitate reproducibility.
Read the full abstract ▾
All authors Jiachen Li, Hongyun Wang, Jinyu Xu, Wenbo Jiang, Yanchun Ma, Yongjian Liu, Qing Xie, Bolong Zheng
Computer Vision
Tongfei Liu, Yufan Liu, Bing Li et al. (+1 more)
The high cost and accessibility problem associated with large
datasets hinder the development of large-scale visual recognition systems.
Dataset Distillation addresses these problems by synthesizing compact surrogate
datasets for efficient training, storage, transfer, and
privacy preservation. The existing state-of-the-art
diffusion-based
dataset distillation methods face three issues: lack of theoretical justification, poor efficiency in scaling to high data volumes, and failure in data-free scenarios. To address these issues, we establish a theoretical framework that justifies the use of
diffusion models by proving the equivalence between
dataset distillation and distribution matching, and reveals an inherent efficiency limit in the
dataset distillation paradigm. We then propose a
Dataset Concentration (DsCo) framework that uses a
diffusion-based Noise-Optimization (NOpt) method to synthesize a small yet representative set of samples, and optionally augments the synthetic data via "Doping", which mixes selected samples from the original
dataset with the synthetic samples to overcome the efficiency limit of
dataset distillation. DsCo is applicable in both data-accessible and data-free scenarios, achieving SOTA performances for low data volumes, and it extends well to high data volumes, where it nearly reduces the
dataset size by half with no performance degradation.
Read the full abstract ▾
All authors Tongfei Liu, Yufan Liu, Bing Li, Weiming Hu
Computer Vision · Cross-listed from cs.CR
FedFG: Privacy-Preserving and Robust Federated Learning via Flow-Matching Generation
Ruiyang Wang, Rong Pan and Zhengan Yao
Federated learning (FL) enables distributed clients to collaboratively train a global model using local private data. Nevertheless, recent studies show that conventional FL
algorithms still exhibit deficiencies in
privacy protection, and the server lacks a reliable and stable aggregation rule for updating the global model. This situation creates opportunities for adversaries: on the one hand, they may eavesdrop on uploaded gradients or model parameters, potentially leaking benign clients' private data; on the other hand, they may compromise clients to launch poisoning attacks that corrupt the global model. To balance accuracy and security, we propose FedFG, a robust FL framework based on flow-matching generation that simultaneously preserves client
privacy and resists sophisticated poisoning attacks. On the client side, each local network is decoupled into a private feature extractor and a public classifier. Each client is further equipped with a flow-matching generator that replaces the extractor when interacting with the server, thereby protecting private features while learning an approximation of the underlying data distribution. Complementing the client-side design, the server employs a client-update verification scheme and a novel robust aggregation mechanism driven by synthetic samples produced by the flow-matching generator. Experiments on MNIST, FMNIST, and CIFAR-10 demonstrate that, compared with prior work, our approach adapts to multiple attack strategies and achieves higher accuracy while maintaining strong
privacy protection.
Read the full abstract ▾
Computer Vision
CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models
Kesheng Chen, Yamin Hu, Qi Zhou et al. (+2 more)
Vision-
language models (VLMs) achieve strong performance on many
benchmarks, yet a basic reliability question remains underexplored: when visual evidence conflicts with commonsense, do models follow what is shown or what commonsense suggests? A characteristic failure in this setting is that the model overrides visual evidence and outputs the commonsense alternative. We term this phenomenon commonsense-driven
hallucination (CDH). To evaluate it, we introduce CDH-Bench, a
benchmark designed to create explicit visual evidence–commonsense conflicts. CDH-Bench covers three dimensions: counting anomalies, relational anomalies, and attribute anomalies. We evaluate frontier VLMs under binary Question Answering (QA) and multiple-choice QA, and report metrics including Counterfactual Accuracy (CF-Acc), Commonsense Accuracy (CS-Acc), Counterfactual Accuracy Drop (CFAD), Commonsense Collapse Rate (CCR), and Relative Prior Dependency (RPD). Results show that even strong models remain vulnerable to prior-driven normalization under visual evidence–commonsense conflict. CDH-Bench provides a controlled diagnostic of visual fidelity under visual evidence–commonsense conflict.
Read the full abstract ▾
All authors Kesheng Chen, Yamin Hu, Qi Zhou, Zhenqian Zhu, Wenjian Luo
Computer Vision
RetinexDualV2: Physically-Grounded Dual Retinex for Generalized UHD Image Restoration
Mohab Kishawy and Jun Chen
We propose RetinexDualV2, a unified, physically grounded dual-branch framework for diverse Ultra-High-Definition (UHD) image restoration. Unlike generic models, our method employs a Task-Specific Physical Grounding Module (TS-PGM) to extract degradation-aware priors (e.g., rain masks and dark channels). These explicitly guide a Retinex decomposition network via a novel Physical-Conditioned Multi-head Self-
Attention (PC-MSA) mechanism, enabling robust reflection and illumination correction. This physical conditioning allows a single architecture to handle various complex degradations seamlessly, without task-specific structural modifications. RetinexDualV2 demonstrates exceptional generalizability, securing 4th place in the NTIRE 2026 Day and Night Raindrop Removal Challenge and 5th place in the Joint Noise Low-light Enhancement (JNLLIE) Challenge. Extensive experiments confirm the state-of-the-art performance and efficiency of our physically motivated approach. Code is available at https://github.com/ErrorLogic1211/RetinexDual/tree/master/RetinexDualV2
Read the full abstract ▾
Computer Vision
AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifiers
Nghia Vu, Tuong Do, Khang Nguyen et al. (+8 more)
Affordance learning is a complex challenge in many applications, where existing approaches primarily focus on the
geometric structures, visual knowledge, and affordance labels of objects to determine interactable regions. However, extending this learning capability to a scene is significantly more complicated, as incorporating object- and scene-level semantics is not straightforward. In this work, we introduce AffordBridge, a large-scale
dataset with 291,637 functional interaction annotations across 685 high-resolution indoor scenes in the form of point clouds. Our affordance annotations are complemented by RGB images that are linked to the same instances within the scenes. Building upon our
dataset, we propose AffordMatcher, an affordance learning method that establishes coherent semantic correspondences between image-based and point cloud-based instances for keypoint matching, enabling a more precise identification of affordance regions based on cues, so-called visual signifiers. Experimental results on our
dataset demonstrate the effectiveness of our approach compared to other methods.
Read the full abstract ▾
All authors Nghia Vu, Tuong Do, Khang Nguyen, Baoru Huang, Nhat Le, Binh Xuan Nguyen, Erman Tjiputra, Quang D. Tran, Ravi Prakash, Te-Chuan Chiu, Anh Nguyen
Computer Vision
Hg-I2P: Bridging Modalities for Generalizable Image-to-Point-Cloud Registration via Heterogeneous Graphs
Pei An, Junfeng Ding, Jiaqi Yang et al. (+3 more)
Image-to-point-cloud (I2P) registration aims to align 2D images with 3D point clouds by establishing reliable 2D-3D correspondences. The drastic modality gap between images and point clouds makes it challenging to learn features that are both discriminative and generalizable, leading to severe performance drops in unseen scenarios. We address this challenge by introducing a heterogeneous graph that enables refining both cross-modal features and correspondences within a unified architecture. The proposed graph represents a mapping between segmented 2D and 3D regions, which enhances cross-modal feature interaction and thus improves feature discriminability. In addition, modeling the consistency among vertices and edges within the graph enables
pruning of unreliable correspondences. Building on these insights, we propose a heterogeneous graph embedded I2P registration method, termed Hg-I2P. It learns a heterogeneous graph by mining multi-path feature relationships, adapts features under the guidance of heterogeneous edges, and prunes correspondences using graph-based projection consistency. Experiments on six indoor and outdoor
benchmarks under cross-domain setups demonstrate that Hg-I2P significantly outperforms existing methods in both generalization and accuracy. Code is released on https://github.com/anpei96/hg-i2p-demo.
Read the full abstract ▾
All authors Pei An, Junfeng Ding, Jiaqi Yang, Yulong Wang, Jie Ma, Liangliang Nan
Computer Vision
Learning Multi-View Spatial Reasoning from Cross-View Relations
Suchae Jeong, Jaehwi Song, Haeone Lee et al. (+9 more)
Vision-
language models (VLMs) have achieved impressive results on single-view
vision tasks, but lack the multi-view spatial
reasoning capabilities essential for
embodied AI systems to understand 3D environments and manipulate objects across different viewpoints. In this work, we introduce Cross-View Relations (XVR), a large-scale
dataset designed to teach VLMs spatial
reasoning across multiple views. XVR comprises 100K
vision-question-answer samples derived from 18K diverse 3D scenes and 70K robotic manipulation trajectories, spanning three fundamental spatial
reasoning tasks: Correspondence (matching objects across views), Verification (validating spatial relationships), and Localization (identifying object positions). VLMs fine-tuned on XVR achieve substantial improvements on established multi-view and robotic spatial
reasoning benchmarks (MindCube and RoboSpatial). When integrated as backbones in
Vision-
Language-Action models, XVR-trained representations improve success rates on RoboCasa. Our results demonstrate that explicit training on cross-view spatial relations significantly enhances multi-view
reasoning and transfers effectively to real-world robotic manipulation.
Read the full abstract ▾
All authors Suchae Jeong, Jaehwi Song, Haeone Lee, Hanna Kim, Jian Kim, Dongjun Lee, Dong Kyu Shin, Changyeon Kim, Dongyoon Hahm, Woogyeol Jin, Juheon Choi, Kimin Lee
Computer Vision
ExFusion: Efficient Transformer Training via Multi-Experts Fusion
Jiacheng Ruan, Daize Dong, Xiaoye Qu et al. (+5 more)
Mixture-of-Experts (MoE) models substantially improve performance by increasing the capacity of dense architectures. However, directly training MoE models requires considerable computational resources and introduces extra overhead in parameter storage and deployment. Therefore, it is critical to develop an approach that leverages the multi-expert capability of MoE to enhance performance while incurring minimal additional cost. To this end, we propose a novel pre-training approach, termed ExFusion, which improves the efficiency of
Transformer training through multi-expert fusion. Specifically, during the initialization phase, ExFusion upcycles the feed-forward network (FFN) of the
Transformer into a multi-expert configuration, where each expert is assigned a weight for later parameter fusion. During training, these weights allow multiple experts to be fused into a single unified expert equivalent to the original FFN, which is subsequently used for forward computation. As a result, ExFusion introduces multi-expert characteristics into the training process while incurring only marginal computational cost compared to standard dense training. After training, the learned weights are used to integrate multi-experts into a single unified expert, thereby eliminating additional overhead in storage and deployment. Extensive experiments on a variety of computer
vision and natural
language processing tasks demonstrate the effectiveness of the proposed method.
Read the full abstract ▾
All authors Jiacheng Ruan, Daize Dong, Xiaoye Qu, Tong Zhu, Ting Liu, Yuzhuo Fu, Yu Cheng, Suncheng Xiang
Computer Vision · Cross-listed from cs.LG
Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
Surendra Pathak and Bo Han
Although Large
Vision Language Models (LVLMs) have demonstrated impressive
multimodal reasoning capabilities, their scalability and deployment are constrained by massive computational requirements. In particular, the massive amount of visual
tokens from high-resolution input data aggravates the situation due to the quadratic complexity of
attention mechanisms. To address these issues, the research community has developed several optimization frameworks. This paper presents a comprehensive survey of the current state-of-the-art techniques for accelerating LVLM inference. We introduce a systematic taxonomy that categorizes existing optimization frameworks into four primary dimensions: visual
token compression,
memory management and serving, efficient architectural design, and advanced decoding strategies. Furthermore, we critically examine the limitations of these current methodologies and identify critical open problems to inspire future research directions in efficient
multimodal systems.
Read the full abstract ▾
Computer Vision
MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generation
Ruiyao Liu, Hui Shen, Ping Zhang et al. (+16 more)
Modern generative models have demonstrated the ability to solve challenging mathematical problems. In many real-world settings, however, mathematical solutions must be expressed visually through diagrams, plots,
geometric constructions, and structured symbolic layouts, where correctness depends on precise visual composition. This naturally raises the question of whether generative models can still do so when the answer must be rendered visually rather than written in text? To study this problem, we introduce MathGen, a rigorous
benchmark of 420 problems spanning seven core domains, including 350 Clean-Scene problems and 70 paired Open-Scene problems. Each problem is evaluated under a Script-as-a-Judge protocol with problem-specific verification criteria implemented through reusable executable scripts for deterministic and reproducible evaluation. Experiments on representative open-source and proprietary text-to-image models show that mathematical fidelity remains a major bottleneck: even the best closed-source model reaches only 53.7% overall accuracy, while open-source models achieve just 1–11%, often near 0% on structured tasks, particularly those requiring precise
geometric and functional rendering. Overall, current T2I models remain far from reliable at even elementary mathematical visual generation.
Read the full abstract ▾
All authors Ruiyao Liu, Hui Shen, Ping Zhang, Yunta Hsieh, Yifan Zhang, Jing Xu, Qi Han, Junchen Li, Jiawei Lu, Jianing Ma, Jiaqi Mo, Sicheng Chen, Zhen Zhang, Zhongwei Wan, Jing Xiong, Xin Wang, Ziyuan Liu, Hangrui Cao, Ngai Wong
Computer Vision
RehearsalNeRF: Decoupling Intrinsic Neural Fields of Dynamic Illuminations for Scene Editing
Changyeon Won, Hyunjun Jung, Jungu Cho et al. (+3 more)
Although there has been significant progress in neural radiance fields, an issue on dynamic illumination changes still remains unsolved. Different from relevant works that parameterize time-variant/-invariant components in scenes, subjects' radiance is highly entangled with their own emitted radiance and lighting colors in spatio-temporal domain. In this paper, we present a new effective method to learn disentangled neural fields under the severe illumination changes, named RehearsalNeRF. Our key idea is to leverage scenes captured under stable lighting like rehearsal stages, easily taken before dynamic illumination occurs, to enforce
geometric consistency between the different lighting conditions. In particular, RehearsalNeRF employs a learnable vector for lighting effects which represents illumination colors in a temporal dimension and is used to disentangle projected light colors from scene radiance. Furthermore, our RehearsalNeRF is also able to reconstruct the neural fields of dynamic objects by simply adopting off-the-shelf interactive masks. To decouple the dynamic objects, we propose a new regularization leveraging optical flow, which provides coarse supervision for the color disentanglement. We demonstrate the effectiveness of RehearsalNeRF by showing robust performances on novel view synthesis and scene editing under dynamic illumination conditions. Our source code and
video datasets will be publicly available.
Read the full abstract ▾
All authors Changyeon Won, Hyunjun Jung, Jungu Cho, Seonmi Park, Chi-Hoon Lee, Hae-Gon Jeon
Computer Vision
JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding
Koki Maeda and Naoaki Okazaki
Japanese scene text poses challenges that
multilingual benchmarks often fail to capture, including mixed scripts, frequent vertical writing, and a character inventory far larger than the Latin alphabet. Although Japanese is included in several
multilingual benchmarks, these resources do not adequately capture the
language-specific complexities. Meanwhile, existing Japanese visual text
datasets have primarily focused on scanned documents, leaving in-the-wild scene text underexplored. To fill this gap, we introduce JaWildText, a diagnostic
benchmark for evaluating
vision-
language models (VLMs) on Japanese scene text understanding. JaWildText contains 3,241 instances from 2,961 images newly captured in Japan, with 1.12 million annotated characters spanning 3,643 unique character types. It comprises three complementary tasks that vary in visual organization, output format, and writing style: (i) Dense Scene Text Visual Question Answering (STVQA), which requires
reasoning over multiple pieces of visual text evidence; (ii) Receipt Key Information Extraction (KIE), which tests layout-aware structured extraction from mobile-captured receipts; and (iii) Handwriting OCR, which evaluates page-level transcription across various media and writing directions. We evaluate 14 open-weight VLMs and find that the best model achieves an average score of 0.64 across the three tasks. Error analyses show recognition remains the dominant bottleneck, especially for kanji. JaWildText enables fine-grained, script-aware diagnosis of Japanese scene text capabilities, and will be released with evaluation code.
Read the full abstract ▾
Computer Vision
A Cross-Scale Decoder with Token Refinement for Off-Road Semantic Segmentation
Seongkyu Choi Jhonghyun An
Off-road semantic
segmentation is fundamentally challenged by irregular terrain, vegetation clutter, and inherent annotation ambiguity. Unlike urban scenes with crisp object boundaries, off-road environments exhibit strong class-level similarity among terrain categories, resulting in thick and uncertain transition regions that degrade boundary coherence and destabilize training. Rare or thin structures, such as narrow traversable gaps or isolated obstacles, further receive sparse and unreliable supervision and are easily overwhelmed by dominant background textures. Existing decoder designs either rely on low-scale bottlenecks that oversmooth fine structural details, or repeatedly fuse high-detail features, which tends to amplify annotation noise and incur substantial computational cost. We present a cross-scale decoder that explicitly addresses these challenges through three complementary mechanisms. First, a global–local
token refinement module consolidates semantic context on a compact bottleneck
lattice, guided by boundary-aware regularization to remain robust under ambiguous supervision. Second, a gated detail bridge selectively injects fine-scale structural cues only once through cross-scale
attention, preserving boundary and texture information while avoiding noise accumulation. Third, an uncertainty-guided class-aware point refinement selectively updates the least reliable pixels, improving rare and ambiguous structures with minimal computational overhead. The resulting framework achieves noise-robust and boundary-preserving
segmentation tailored to off-road environments, recovering fine structural details while maintaining deployment-friendly efficiency. Experimental results on standard off-road
benchmarks demonstrate consistent improvements over prior approaches without resorting to heavy dense feature fusion.
Read the full abstract ▾
Computer Vision
ForestSim: A Synthetic Benchmark for Intelligent Vehicle Perception in Unstructured Forest Environments
Pragat Wagle, Zheng Chen and Lantao Liu
Robust scene understanding is essential for intelligent vehicles operating in natural, unstructured environments. While semantic
segmentation datasets for structured urban driving are abundant, the
datasets for extremely unstructured wild environments remain scarce due to the difficulty and cost of generating pixel-accurate annotations. These limitations hinder the development of perception systems needed for intelligent ground vehicles tasked with forestry automation, agricultural robotics, disaster response, and all-terrain mobility. To address this gap, we present ForestSim, a high-fidelity synthetic
dataset designed for training and evaluating semantic
segmentation models for intelligent vehicles in forested off-road and no-road environments. ForestSim contains 2094 photorealistic images across 25 diverse environments, covering multiple seasons, terrain types, and foliage densities. Using Unreal Engine environments integrated with Microsoft AirSim, we generate consistent, pixel-accurate labels across 20 classes relevant to autonomous
navigation. We
benchmark ForestSim using state-of-the-art architectures and report strong performance despite the inherent challenges of unstructured scenes. ForestSim provides a scalable and accessible foundation for perception research supporting the next generation of intelligent off-road vehicles. The
dataset and code are publicly available:
Dataset: https://vailforestsim.github.io Code: https://github.com/pragatwagle/ForestSim
Read the full abstract ▾
Computer Vision
FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generation
Liuzhou Zhang, Zeyu Zhang, Biao Wu et al. (+10 more)
Sign
language plays a crucial role in bridging communication gaps between the deaf and hard-of-hearing communities. However, existing sign
language video generation models often rely on complex intermediate representations, which limits their flexibility and efficiency. In this work, we propose a novel pose-free framework for real-time sign
language video generation. Our method eliminates the need for intermediate pose representations by directly mapping natural
language text to sign
language videos using a
diffusion-based approach. We introduce two key innovations: (1) a pose-free generative model based on the a state-of-the-art
diffusion backbone, which learns implicit text-to-gesture
alignments without pose estimation, and (2) a Trainable Sliding Tile
Attention (T-STA) mechanism that accelerates inference by exploiting spatio-temporal locality patterns. Unlike previous training-free sparsity approaches, T-STA integrates trainable sparsity into both training and inference, ensuring consistency and eliminating the train-test gap. This approach significantly reduces computational overhead while maintaining high generation quality, making real-time deployment feasible. Our method increases
video generation speed by 3.07x without compromising
video quality. Our contributions open new avenues for real-time, high-quality, pose-free sign
language synthesis, with potential applications in inclusive communication tools for diverse communities. Code: https://github.com/AIGeeksGroup/FlashSign.
Read the full abstract ▾
All authors Liuzhou Zhang, Zeyu Zhang, Biao Wu, Luyao Tang, Zirui Song, Hongyang He, Renda Han, Guangzhen Yao, Huacan Wang, Ronghao Chen, Xiuying Chen, Guan Huang, Zheng Zhu