Computer Vision · Cross-listed from cs.RO
MVP-Nav: Multi-layer Value Map Planner Navigator
Wenyuan Xie, Shaokai Wu, Yijin Zhou et al. (+7 more)
Zero-shot Object Goal
Navigation (ZSON) with RGB-only perception poses a fundamental challenge for
embodied agents, as the absence of explicit depth information introduces severe physical uncertainty and semantic-physical misalignment. Existing approaches either rely on high-level semantic
reasoning without
geometric grounding or learn end-to-end policies that lack explicit physical constraints, often resulting in semantically plausible but physically unsafe behaviors. In this paper, we propose MVP-Nav, a physical-aware RGB-only
navigation framework that aligns perception, planning, and control with the real 3D world. MVP-Nav reconstructs explicit physical occupancy from
monocular observations by leveraging 3D foundation models to project 2D semantic instances into 3D oriented bounding boxes, forming a global spatial semantic representation. To unify high-level semantic
reasoning and low-level physical constraints, we introduce a Multi-layer Value Map (MVM) that integrates semantic priorities and reconstructed geometry into a shared cost space, enabling physically grounded
geometric planning. Extensive experiments on zero-shot object
navigation benchmarks demonstrate that MVP-Nav significantly outperforms existing depth-free methods, achieving state-of-the-art performance and validating that structured physical priors can effectively compensate for the absence of active depth sensors.
Read the full abstract ▾
All authors Wenyuan Xie, Shaokai Wu, Yijin Zhou, Yanbiao Ji, Guodong Zhang, Bayram Bayramli, Qiuchang Li, Xunchu Zhou, Yue Ding, Hongtao Lu
Computer Vision
DriveWeaver: Point-Conditioned Video Inpainting for Controllable Vehicle Insertion in Autonomous Driving Simulation
Junzhe Jiang, Zipei Ma, Zijie Pan et al. (+1 more)
A pivotal step in autonomous driving simulation involves inserting foreground vehicles with predefined trajectories into simulated scenes. This process enhances scene diversity and facilitates the creation of various corner cases for testing and improving autonomous driving models. However, existing methods often rely on pre-reconstructed 3D assets, which frequently lead to lighting inconsistencies between the inserted foreground and the background. Moreover, the reliance on limited, manually-curated 3D assets hinders large-scale deployment. To address these challenges, we propose DriveWeaver, a novel framework for controllable vehicle insertion in autonomous driving simulation. Specifically, for a masked target insertion area, DriveWeaver performs
video inpainting conditioned on vehicle point clouds to generate high-quality, temporally consistent vehicles. This
video-inpainting-based approach ensures seamless blending between the foreground and background, while the readily available point cloud conditions enable superior generalization. To support long-term generation, we further design a global-to-local hierarchical inpainting strategy, ensuring the consistent identity and appearance of the inserted vehicles. Meanwhile, we extract explicit 3D Gaussian representations of the inserted vehicles through an urban reconstruction pipeline to enable real-time rendering for autonomous driving simulation. Extensive experiments across diverse
datasets demonstrate that our method outperforms existing baselines in visual realism and
geometric consistency, providing a robust tool for scalable autonomous driving scene augmentation.
Read the full abstract ▾
All authors Junzhe Jiang, Zipei Ma, Zijie Pan, Li Zhang
Computer Vision
Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference
Zhaoyang Luo, Runmin Dong, Miao Yang et al. (+4 more)
Multimodal large
language models (MLLMs) increasingly process long visual-
token sequences, increasing the overall inference computation. Existing acceleration methods usually remove visual
tokens or skip visual-
token updates in entire layers, but these coarse strategies may discard fine-grained evidence or suppress useful operators together with redundant ones. In this paper, we study visual-
token computation from an answer-observable perspective and find that late visual-
token updates can remain large while having little effect on answer-
token representations. Motivated by this answer-silent redundancy, we decompose each
Transformer layer into
attention and FFN operators and show that useful visual computation is often operator-dominant and layer-dependent. We propose an operator-level visual-
token skipping framework that preserves the full visual-
token sequence while selectively bypassing redundant
attention, FFN, or both. Experiments across three MLLM architectures and 10 VQA
benchmarks show that our method achieves strong efficiency-accuracy trade-offs, reducing 33.7% TFLOPs on Qwen3-VL while retaining 99.5% of the vanilla model performance.
Read the full abstract ▾
All authors Zhaoyang Luo, Runmin Dong, Miao Yang, Fan Wei, Yushan Lai, Bin Luo, Haohuan Fu
Computer Vision
RESOLVE: A Multi-Resolution and Multi-Modal Dataset for Roadside Cooperative Perception
Shaozu Ding, Linan Song, Marco De Vincenzi et al. (+1 more)
LiDAR has increasingly been integrated into traffic cameras to expand coverage and mitigate occlusion in roadside cooperative perception. However, how unimodal and camera-
LiDAR fusion architectures behave under variations in
LiDAR point sparsity induced by sensor configurations and scene-dependent sensing conditions remains underexplored. We introduce RESOLVE, a large-scale real-world
benchmark dataset featuring multi-resolution roadside
LiDAR and synchronized camera-
LiDAR sensing for systematic evaluation of unimodal and fusion-based architectures in roadside 3D detection and tracking. RESOLVE contains over 100k images and 26k point cloud frames with 220k manually annotated bounding boxes, captured at a real-world urban intersection across diverse lighting and weather conditions and spanning 10 classes of traffic participants. In particular, RESOLVE enables controlled evaluation across three
LiDAR resolution levels while keeping all other sensing and environmental factors fixed. This allows fair cross-architecture comparisons under point cloud distribution shifts resulting from resolution variations, sensing distance, and training-inference resolution mismatches. Results from extensive
benchmark experiments reveal insights into how
multimodal fusion can compensate for
LiDAR point sparsity, offering clues for designing cost-efficient roadside
multimodal perception. The
dataset and
benchmark codes are available at https://github.com/ASU-Suo-Lab/RESOLVE.
Read the full abstract ▾
All authors Shaozu Ding, Linan Song, Marco De Vincenzi, Dajiang Suo
Computer Vision · Cross-listed from cs.AI
Harnessing Textual Refusal Directions for Multimodal Safety
Moreno D'Incà, Nicu Sebe and Massimiliano Mancini
Safety in
Multimodal Large
Language Models (MLLMs) depends on unsafe
multimodal data for post-training
alignment or activation steering. However, such data is hard to collect, must be gathered anew for each modality, and its collection and distribution raise ethical and legal concerns. In this work, we relax this constraint and investigate whether textual refusal directions, extracted directly from the LLM backbone, generalize across modalities (i.e., image,
video). Preliminary findings confirm this ability, though effectiveness is conditioned by layer selection, steering strength, and cross-modal
alignment, with the latter causing safe
multimodal inputs to be spuriously steered toward refusal. Building on this, we introduce Modality-Agnostic Refusal Steering (MARS), a lightweight training-free approach that injects
multimodal safety without the need for
multimodal safety data. MARS corrects modality misalignment via activation centering, adaptively scales steering strength within a geometrically defined trust region, and selects the optimal intervention layer, operating at the first generated
token. Evaluated on five SOTA MLLMs across safety, utility, and
video jailbreak, MARS achieves consistent safety gains while preserving utility up to over-refusal. MARS reveals that safety-relevant structure is shared across modalities and that textual refusal directions are a powerful and underexplored foundation for
multimodal alignment.
Read the full abstract ▾
Computer Vision
SENSE-VAD: Sentient and Semantic Video Anomaly Detection for Autonomous Driving
Nghia T. Nguyen, Lokman Bekit and Yasin Yilmaz
Autonomous vehicles (AVs) must navigate not only motion-based hazards but also socially complex situations whose danger is constituted by inter-
agent relationships rather than movement statistics alone. A child running away from a guardian, a person being carried by another, or a pursuer chasing a pedestrian across a sidewalk are all anomalous in social context, yet none produces an obvious motion signal that current anomaly detectors are equipped to flag. We introduce SENSE-VAD, the first synthetic
video anomaly detection
benchmark for autonomous driving explicitly designed around socially complex anomalies. Using the CARLA simulator and Unreal Engine (UE), we generate distinct anomaly scenarios across multiple categories: individual behaviors, group behaviors, person–object interactions, cyclist interactions, vehicle &
agent, each annotated with per-frame binary labels. A key design principle is the separation of social anomaly from motion-based or appearance-based anomaly: many scenarios involve motion of objects that appears unremarkable in isolation but is anomalous in relational context. We additionally provide real-world normal and anomalous
videos as a sim-to-real transfer probe. We evaluate state-of-the-art
video anomaly detection baselines and demonstrate that socially complex anomalies constitute a distinct and currently unsolved challenge. Our
dataset, annotations, and generation code are publicly available.
Read the full abstract ▾
Computer Vision
Towards Voxel Spacing Consistency for Medical Image Segmentation
Xin You, Runze Yang, Minghui Zhang et al. (+6 more)
Volumetric medical image
segmentation is essential for both preoperative diagnosis and intraoperative guidance. While recent years have witnessed rapid progress in
segmentation architectures, comparatively little
attention is paid to the physical voxel spacing of anatomical data. Indeed, volumetric image resampling is a ubiquitous preprocessing step before
segmentation, yet its interaction with downstream
segmentation has not been systematically exploited. In this work, we study the correlation between image resampling and
segmentation, and propose Consispace, a semantic-aware resampling framework that achieves consistent voxel spacing in the axial direction while preserving anatomical and semantic consistency. Consispace introduces an ODE-based anatomical constraint to model inter-slice dynamics with a continuous interpolator, enabling faithful reconstruction under complex anatomical transitions beyond discrete interpolation. To further couple resampling with
segmentation objectives, we leverage dense features from a pretrained
vision model to build intra-slice semantic correlation maps and inject class-wise semantic consistency via feature reweighting during resampling. Both intra-slice and inter-slice constraints are integrated into an implicit neural network, supporting arbitrary-scale resampling. Extensive experiments on multiple
datasets demonstrate that Consispace achieves superior reconstruction quality and perceptual fidelity, produces smoother inter-slice anatomy, and improves downstream
segmentation performance when used as a preprocessing step.
Read the full abstract ▾
All authors Xin You, Runze Yang, Minghui Zhang, Hanxiao Zhang, Han Li, Yi Yu, Jie Yang, Nassir Navab, Yun Gu
Computer Vision
Real-Time Source-Free Object Detection
Sairam VCR, Varun Gopal, Poornima Jain et al. (+2 more)
Real-world detectors for autonomous driving, surveillance, and robotics must handle domain-shifts under strict latency and
memory constraints, yet existing source-free object detection (SFOD) methods rely on heavyweight architectures that prioritize accuracy alone. We show this trade-off is unnecessary: building on YOLOv10, an NMS-free dual-head detector, we achieve state-of-the-art adaptation accuracy while being faster and more compact. We observe that directly applying vanilla mean-teacher self-training to dual-head detectors leads to suboptimal adaptation performance due to two key factors. First, simple pseudo-label generation strategies, such as using a single head or directly combining high-confidence predictions from both heads, yield suboptimal supervision under domain-shift. We propose DHF (Dual-Head Pseudo-Label Fusion) which selectively admits one-to-one (O2O) and one-to-many (O2M) head predictions, preserving precision and recovering missed objects. Second, we observe domain-shift collapses multi-scale feature discriminability. We propose the use of our MARD (Multi-scale Adaptive Representation Diversification) loss which mitigates this by enforcing detection-aware variance and covariance constraints on multi-scale feature maps. Both modules are training-time only, leaving inference unchanged. Across domain-shift
benchmarks, our method, RT-SFOD yields 1.4 to 3.5% mAP gains, 1.3$\times$ higher throughput, with $\sim$2$\times$ fewer parameters than prior state-of-the-art SFOD methods, thus advancing the Pareto frontier of the speed-accuracy-model size trade-off. We report main results with YOLOv10, and demonstrate generalizability with additional YOLO- and DETR-based dual-head detectors. Code is available here: https://github.com/Sairam13001/RT-SFOD/
Read the full abstract ▾
All authors Sairam VCR, Varun Gopal, Poornima Jain, Vineeth N Balasubramanian, Muhammad Haris Khan
Computer Vision
PriorEye: Geospatial Visual Priors for End-to-End Autonomous Driving
Kyuhwan Yeon, Benjamin Ramtoula and Daniele De Martini
Most end-to-end autonomous driving methods rely solely on instantaneous sensor observations, limiting them to reactive behavior without the anticipatory foresight human drivers employ through prior experience. We introduce geospatial visual priors, street-level visual context anchored to the intended driving route, providing visual-spatial foresight independent of real-time sensors. We propose a
memory augmentation module featuring a dual-
memory architecture and an adaptive
memory gate, which can be easily integrated into existing end-to-end approaches. This design pairs a contextual
memory for retrieved priors with a persistent fallback
memory, and dynamically regulates the influence of
memories based on current state compatibility. Evaluated on the NAVSIM-v2
benchmark, our approach consistently improves performance across diverse end-to-end baselines. Furthermore, because these priors are independent of onboard sensors, our method inherently improves robustness against sensor corruption, while the dual-
memory design ensures safe fallback when the retrieved priors themselves become unreliable. Our project page is available at https://ori-mrg.github.io/PriorEye.
Read the full abstract ▾
Computer Vision
Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
Junha Jung, Minbyul Jeong, Suhyeon Lim et al. (+5 more)
Recent
multimodal large
language models have shown great promise in clinical image
reasoning, but existing post-training pipelines remain predominantly outcome-centric, relying on final answer correctness or sequence-level preferences. This suffers from sparse credit assignment, making it difficult to optimize the
reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early-stage
reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA)
benchmarks. Motivated by this, we propose Medical
Reasoning-aware Policy Optimization (MRPO), an RL
algorithm that incorporates step-wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to
tokens in earlier invalid
reasoning steps, breaking failure cascades without compromising successful paths. Across four
multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Thinking even surpasses substantially larger medical MLLMs such as HuatuoGPT-
Vision-34B by 4.59 points. Moreover, MRPO reduces early-stage
reasoning failures from 58.6% to 13.4%, showing that targeted mitigation of cascading failures improves both
reasoning quality and final answer accuracy. Our code is available at https://github.com/dmis-lab/MRPO
Read the full abstract ▾
All authors Junha Jung, Minbyul Jeong, Suhyeon Lim, Sungwook Jung, Jaehoon Yun, Taeyun Roh, Mujeen Sung, Jaewoo Kang
Computer Vision
Absorption-Feature-Guided Distance-Decoupled Estimation and Band Selection for LWIR Hyperspectral Passive Ranging
Shuo Liu, Chen Fan, Zhihe Chen et al. (+2 more)
Long-wave infrared (LWIR) hyperspectral observations contain distance-dependent atmospheric absorption signatures, providing a physical basis for long-range passive ranging. However, in natural scenes, these signatures are nonlinearly coupled with target temperature, material emissivity, and path radiance, making distance inversion from observed radiance ill posed. Existing methods typically rely on full-band measurements and pixel-wise joint optimization, which is computationally expensive and does not explicitly exploit sharp atmospheric absorption structures. This paper proposes an Absorption-Guided Distance-Decoupled Estimation and Refinement (ADER) framework for LWIR hyperspectral passive ranging. ADER represents emissivity with B-spline control points under a smoothness prior, suppressing overfitting to atmospheric absorption structures and enabling distance-decoupled estimation. It further uses ozone-absorption cues to classify pixels into emission-dominant and reflection-dominant groups. For emission-dominant pixels, ADER compensates path radiance and transmittance and estimates distance by one-dimensional absorption-residual minimization. For reflection-dominant pixels, ADER refines the initial estimate using downwelling-radiance compensation based on the complete radiative model. To reduce spectral redundancy, ADER also introduces a greedy band selection strategy based on multi-scene effective Fisher information for the distance parameter. Experiments on real scenes show that ADER recovers
LiDAR-consistent spatial distance structures under both full-band and 20-band settings, improves ranging accuracy in the evaluated regions, and achieves approximately two orders of magnitude speedup over a public full-band hyperspectral ranging method.
Read the full abstract ▾
All authors Shuo Liu, Chen Fan, Zhihe Chen, Xiaolin Huang, Lilian Zhang
Computer Vision
Generative Lane Topology Reasoning via Autoregressive Model with Geometry Prior
Jiahui Fu, Zehao Huang, Han Li et al. (+2 more)
Lane
topology reasoning aims to construct a lane graph from onboard sensor observations. Existing methods follow a detection and association paradigm that treats each lane instance independently, leading to
geometric inconsistency at connected endpoints and incomplete graphs due to visual occlusions. To address these issues, we propose TopoGPT, a generative framework that learns the geometry prior from typical lane graph structures through autoregressive sequence modeling. Specifically, we construct a large-scale map
dataset comprising 3.3M scenes. For each lane graph, a lane tokenizer serializes it into discrete
tokens, while a scene context encoder converts it into a rasterized image and extracts global features as scene
tokens. We pre-train an autoregressive lane sequence
transformer via scene-conditioned next-
token prediction, endowing the model with the geometry prior over lane graph structures. Building upon this prior, a perception adapter aligns BEV features from multi-view images with the pre-trained scene condition, transferring the learned geometry prior to sensor-based lane graph prediction. On the OpenLane-V2
benchmark, TopoGPT outperforms existing methods by an average of +6.4 on lane-level and +11.6 on point-level metrics, and produces geometrically consistent and structurally complete lane graphs.
Read the full abstract ▾
All authors Jiahui Fu, Zehao Huang, Han Li, Naiyan Wang, Si Liu
Computer Vision
MuSViT: A Foundation Vision Model for Sheet Music Representation
Carlos Penarrubia, Antonio Rios-Vila, Eliseo Fuentes-Martinez et al. (+4 more)
Foundation models have transformed
vision and
language processing by providing rich, reusable representations that transfer across diverse tasks. Sheet music, as a visual encoding of musical
language, lacks such a strong domain-specific backbone. We introduce MuSViT (Music Score
Vision Transformer): the first foundation
vision model for sheet music representation – a ViT encoder pre-trained via Masked Autoencoders on 9.7 million pages from the IMSLP. To handle the complexity of real-world scores, we adopt a two-stage curriculum: a synthetic warm-up on typeset scores followed by large-scale training on the full IMSLP corpus. We evaluate MuSViT on four downstream tasks – full-page and staff-level music score recognition, music symbol detection, and score difficulty classification – under two scenarios: linear probing (frozen encoder) and fine-tuning. Under linear probing, MuSViT consistently outperforms modern
vision encoders, revealing that general-purpose representations, regardless of scale, fall systematically short on the structured symbolic properties of musical notation. Under fine-tuning, MuSViT generally improves upon task-specific state-of-the-art methods. An additional
embedding-transcription consistency analysis reveals that MuSViT encodes symbolic musical structure directly in its representation space – unlike other encoders, whose
embeddings do not correlate with music notation content. These results establish MuSViT as a foundation backbone for sheet music understanding.
Read the full abstract ▾
All authors Carlos Penarrubia, Antonio Rios-Vila, Eliseo Fuentes-Martinez, Juan C. Martinez-Sevilla, Francisco J. Castellanos, María Alfaro-Contreras, Jorge Calvo-Zaragoza
Computer Vision
Self-Supervised Temporal Regularization for Landmark-Based Cardiac Segmentation with Automatic AHA Regional Mapping
David Montalvo-García, Nicolás Gaggion, María J. Ledesma-Carbayo et al. (+1 more)
Graph-based cardiac
segmentation with implicit anatomical correspondences provides topological guarantees and population-level analysis capabilities, but models trained on independent frames of image sequences exhibit temporal discontinuities that affect reliable clinical measurements, particularly in cardiac ultrasound. In this work, we introduce self-supervised temporal regularization as a post-training refinement stage that exploits the temporal coherence in image sequences to enforce consistent cardiac
segmentation and motion estimation over time, without requiring per-frame annotations. By penalizing velocity and acceleration discontinuities across consecutive frames, our method achieves temporally consistent
segmentations while maintaining the learned anatomical correspondences. We further leverage these correspondences to automatically map landmarks to the AHA 17-segment clinical standard, enabling standardized regional assessment and detection of pathological myocardial motion patterns. Validation on CAMUS
dataset demonstrates the clinical utility of combining temporal consistency with automatic regional mapping. The code is publicly available at https://github.com/david-montalvoo/MaskHybridGNet-TempReg
Read the full abstract ▾
All authors David Montalvo-García, Nicolás Gaggion, María J. Ledesma-Carbayo, Enzo Ferrante
Computer Vision
SpikeLogBERT: Energy-Efficient Log Parsing Using Spiking Transformer Networks
Thuan Bui, Duong Do, Tung Vu et al. (+2 more)
Log parsing is a fundamental step in automated log analysis, transforming raw system logs into structured event templates for downstream tasks such as anomaly detection and system monitoring. Existing log parsing methods range from rule-based and clustering-based approaches to neural models that learn semantic representations from log messages. However, neural approaches typically rely on dense
matrix multiplications, which can result in high computational cost and energy consumption. This paper presents SpikeLogBERT, a spiking neural network framework for energy-efficient log parsing. The proposed model integrates a spiking
transformer architecture with knowledge
distillation from a BERT teacher model, enabling spike-driven computation while preserving semantic representation capability. By leveraging sparse spike activations and event-driven processing, the number of active operations during inference can be significantly reduced. As an initial
benchmark study, experiments on the HDFS
dataset demonstrate that SpikeLogBERT outperforms ANN-based neural log parsing models with a parsing accuracy of 0.99997, while reducing estimated theoretical energy consumption by up to 62.6% under standard 45nm CMOS assumptions.
Read the full abstract ▾
All authors Thuan Bui, Duong Do, Tung Vu, Duc-Tho Mai, Cong-Kha Pham
Computer Vision
Mesh BDF: Barycentric Dominance Field for 3D Native Mesh Generation
Gaochao Song, Haohan Weng, Luo Zhang et al. (+2 more)
Autoregressive (AR) modeling has recently achieved remarkable progress in native 3D mesh generation, largely due to its natural ability to handle variable-length, discrete data structures. However, the inherent constraints of the AR paradigm severely restrict the generated meshes, leading to limited face counts, bounded vertex resolutions, and difficulties in supporting textures. To overcome these bottlenecks, we propose the Barycentric Dominance Field (BDF), a continuous representation defined on triangular mesh surfaces that elegantly encodes vertex topological connectivity. BDF bridges the fundamental gap between discrete mesh
topology and continuous
diffusion-based generative modeling by transforming connectivity into a continuous surface signal. As an intrinsic mesh property, BDF shares strong similarities with texture maps, enabling its seamless integration into existing 3D
diffusion pipelines without requiring architectural modifications. Extensive experiments demonstrate that BDF empowers
diffusion models to generate native meshes with significantly higher quality, greater scalability, and stronger robustness compared to state-of-the-art autoregressive methods.
Read the full abstract ▾
All authors Gaochao Song, Haohan Weng, Luo Zhang, Zibo Zhao, Shenghua Gao
Computer Vision · Cross-listed from cs.GR
NURBS Splatting: A Unified Differentiable Rendering Framework for Vector Graphics
Jingye Qiu and Shizhe Zhou
Differentiable rendering of planar rational splines remains largely underexplored, despite their widespread use in vector graphics and design. Existing differentiable vector renderers primarily focus on Bézier curves and rely on analytic rasterization, which can suffer from gradient instability and limited flexibility. We propose NURBS Splatting, a unified framework that represents planar rational curves as continuous Gaussian fields. By sampling Gaussians along the curve parameter domain and inside closed regions, rendering is reformulated as a smooth accumulation process with stable gradients. Our method naturally supports long splines, rational weights, non-uniform
knots, and closed-region filling. We demonstrate its effectiveness in calligraphy reconstruction, vectorization frameworks, and long-spline image abstraction, showing improved stability and reconstruction quality over existing approaches.
Read the full abstract ▾
Computer Vision
Estimating Velocity and Spin of Spherical Objects from Rolling-Shutter Image(s)
Wenjie Xue, Jun Yang, Jingmin Wang et al. (+1 more)
Rolling-shutter cameras introduce characteristic distortions when imaging fast moving objects, and these effects are typically treated as artifacts to be corrected. In this work, we instead leverage rolling-shutter distortions as a valuable source of temporal information to estimate the 3D translational and angular velocities of rapidly moving spherical objects from a single rolling-shutter frame. We design a robust and easily detectable spherical pattern and propose a correspondence-free formulation that recovers motion by enforcing
geometric consistency in a back-projection framework. By exploiting the geometry of the sphere, translational and rotational motions are decoupled and estimated through a two-stage optimization process, enabling reliable velocity recovery even for textureless objects. Extensive experiments on both synthetic and real
datasets demonstrate accurate and robust estimation of motion parameters under challenging high-speed conditions.
Read the full abstract ▾
All authors Wenjie Xue, Jun Yang, Jingmin Wang, Limin Shang
Computer Vision
JL1-CC&QA: Extending the JL1-CD Benchmark with Change Captioning and Question Answering
Ziyuan Liu, Ruifei Zhu, Ouqiao Ma et al. (+1 more)
Remote sensing change detection (CD) traditionally focuses on pixel-level binary
segmentation, which identifies where changes occur but neither what nor why. To bridge this semantic gap, we introduce JL1-CC&QA, a multi-task
benchmark that extends the JL1-CD
dataset with two complementary annotation layers: change captioning (CC) and change question answering (QA). Built upon 5,000 bi-temporal image pairs acquired by the Jilin-1 satellite at 0.5-0.75m ground sample distance, the
benchmark comprises: (i) JL1-CC, providing 17,021 quality-verified captions that describe diverse land-cover transformations; and (ii) JL1-QA, offering 20,060 question-answer pairs across eight question types, enabling fine-grained, interactive interrogation of surface changes. All annotations are produced via a three-stage pipeline consisting of multi-modal large
language model (LLM) generation,
vision-grounded LLM judging, and human expert verification. We hope that JL1-CC&QA, as a
benchmark unifying binary change masks, change captions, and change-oriented QA over the same image set, will serve as a valuable resource for the community to advance multi-task change understanding in remote sensing. The
dataset is available at https://github.com/circleLZY/JL1-CD.
Read the full abstract ▾
All authors Ziyuan Liu, Ruifei Zhu, Ouqiao Ma, Yuantao Gu
Computer Vision
Rhythm-Structured Predictive Learning for Remote Photoplethysmography
Ba-Thinh Nguyen, Huu-Dung Nguyen, Thi-Duyen Ngo et al. (+1 more)
Remote photoplethysmography (rPPG) estimates physiological signals from facial
videos by analyzing subtle pulse induced skin color variations. Despite recent progress, existing self-supervised rPPG methods mainly reconstruct masked pixels or low-level visual representations, which can bias the model toward facial appearance rather than latent physiological dy namics. Moreover, most recent Mamba-based approaches scan facial
video tokens only in chronological order, limiting their ability to exploit the cyclic structure of pulse signals. To ad dress these limitations, we propose RhythmJEPA, a rhythm structured joint-
embedding predictive learning framework for rPPG. Instead of reconstructing RGB frames, RhythmJEPA predicts latent teacher representations from masked facial
videos, thereby encouraging physiology-aware representation learning in the
embedding space. To explicitly model pulse-related tem poral structure, we introduce a Cyclic Rhythm-State Plan ner (CRSP), which estimates frame-wise latent physiological states and decodes the most plausible cyclic state path via dynamic programming with a constrained transition grammar. Guided by the decoded states, we further design a Dual Order Mamba Encoder (DOM), which combines conventional chronological scanning with state-ordered scanning to capture both local temporal continuity and long-range rhythm-consistent dependencies. Finally, a lightweight Spatial Pulse Mixer (SPM) extracts compact pulse-sensitive facial
tokens with a favorable balance between complexity and performance. Experiments on PURE, UBFC-rPPG, and MMPD show competitive performance over representative rPPG methods. The codes are available at https://github.com/deconasser/RhythmJEPA.
Read the full abstract ▾
All authors Ba-Thinh Nguyen, Huu-Dung Nguyen, Thi-Duyen Ngo, Thanh-Ha Le
Computer Vision
MemLearner: Learning to Query Context memory for Video World Models
Jiwen Yu, Jianxiong Gao, Jianhong Bai et al. (+7 more)
Video World Models are interactive
video generation models that predict future world states based on user actions and history
video frames. A critical challenge in
video world models is the lack of
memory, causing inconsistent generated scenes over extended durations. Previous methods explored rule-based context frame
retrieval as
memory, but they fail to generalize in scenarios with scene occlusions and dynamic objects. We propose MemLearner, a learning-based adaptive context query method using query
tokens to bridge context and predicted
tokens. By leveraging the
video generation model itself for context querying, MemLearner exploits pre-trained visual priors without training additional modules from scratch, and incorporates efficient strategies for training and inference. We collect a
dataset of long
videos with scene occlusions and dynamic objects, paired with camera pose annotations, and propose a multi-
dataset training strategy leveraging both annotated rendered and unannotated real-world
videos. Extensive experiments demonstrate that MemLearner significantly outperforms prior
video world models in terms of scene consistency and
memory, particularly under challenging occlusion and dynamic scenarios.
Read the full abstract ▾
All authors Jiwen Yu, Jianxiong Gao, Jianhong Bai, Yiran Qin, Kaiyi Huang, Quande Liu, Xintao Wang, Pengfei Wan, Kun Gai, Xihui Liu
Computer Vision
UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization
Yaozhi Zheng, Yilei Jiang, Manyuan Zhang et al. (+5 more)
Visual-to-Code generation, which transforms scientific plots, vector graphics, and webpages into executable scripts, demands a level of pixel-precise
alignment that standard
Multimodal Large
Language Models (MLLMs) fail to achieve through Supervised Fine-Tuning (SFT) alone. While Reinforcement Learning (RL) offers a theoretical pathway to bridge this gap, its application is hindered by two fundamental obstacles: (1) Reward Coarseness, where semantic metrics like CLIP scores fail to penalize fine-grained element deviations, and (2) Exploration Stagnation, where the sparse, heterogeneous code search space prevents the policy from bootstrapping valid trajectories. To overcome these limitations, we introduce UniCoder, a unified RL framework that integrates two novel mechanisms. First, we propose Symbolic Attribute
Alignment, which employs a lightweight auxiliary LLM to parse generated code into discrete visual attributes (e.g., hex colors, coordinate limits), enabling dense, element-wise reward computation. Second, to escape local optima, we devise Reference-Guided Code Optimization, a strategy that dynamically injects ground-truth trajectories into low-performing rollout groups, transforming blind exploration into guided policy improvement. Extensive experiments on ChartMimic, UniSVG, Design2Code and ScreenBench
benchmarks demonstrate that our 8B-parameter model not only surpasses all open-source baselines but also achieves state-of-the-art performance comparable to proprietary models, establishing a new paradigm for generalized visual-to-code synthesis.
Read the full abstract ▾
All authors Yaozhi Zheng, Yilei Jiang, Manyuan Zhang, Yuxuan Wan, Kaituo Feng, Tianshuo Peng, Bo Zhang, Xiangyu Yue
Computer Vision · Cross-listed from cs.NI
Exploiting Overlapping Fields of View for Redundancy-Aware Uplink Transmission in Vehicular 6G
Hamidreza Mazandarani, Masoud Shokrnezhad, Tarik Taleb et al. (+1 more)
Emerging uplink-dominant 6G use cases, such as cooperative vehicular streaming, require efficient transmission of high-volume visual data over limited wireless resources. While semantic communications can reduce traffic by prioritizing task-relevant content, most existing approaches treat users independently and therefore overlook spatial redundancy among nearby devices' observations. This paper proposes a semantic-aware multiple access scheme that exploits overlapping fields of view among vehicular users to reduce redundant uplink transmissions. We formulate a joint perception and transmission control problem in which users decide which image patches to transmit, when to transmit them, and over which channel, subject to communication constraints. To address the resulting complexity, we introduce a practical two-phase approach. First, nearby vehicles share selected observation patches over Vehicle-to-Vehicle (V2V) links to calculate inter-user spatial redundancy. Second, users transmit only semantically important, non-redundant patches to the base station, where observations can be reconstructed using the received patches and complementary views from neighboring vehicles. Simulation results in a dense urban vehicular scenario demonstrate that our approach improves the proportion of users who achieve high-fidelity reconstruction, highlighting the potential of semantic-aware multiple access for sustainable and resource-efficient 6G uplink systems.
Read the full abstract ▾
All authors Hamidreza Mazandarani, Masoud Shokrnezhad, Tarik Taleb, Onur Günlü
Computer Vision
WIDER-FAIR: An Annotated Version of the WIDER-FACE Dataset for Fairness Evaluation
Maxime Moussi, Benoît Ronval, Siegfried Nijssen et al. (+1 more)
The deployment of face detection models in real-world applications raises important
fairness concerns, as these systems may showcase performance disparities across demographic groups. A key obstacle to studying and mitigating such biases is the lack of face detection
datasets with sensitive feature annotations. To address this gap, we introduce WIDER-FAIR, a new
dataset built on the widely used WIDER-FACE
benchmark, manually annotated with the perceived ethnicity and sex of each face. The
dataset contains 16,256 images annotated across four ethnic groups: Asian, Black, Indian, and White, and two sex categories. We assess the quality and coherence of the annotations using face
embeddings, a K-Nearest Neighbors classifier, and a t-SNE visualization, all of which support the consistency of the labeling process. As a demonstration of the
dataset's potential, we train a YOLOv5 model and perform ablation studies on each sensitive feature. Among other findings, our experiments show that detection performance is notably lower for faces of Black individuals, and that excluding this group from training increases
fairness disparity more than excluding any other ethnic group. These observations illustrate the value of demographically annotated
datasets for understanding and evaluating bias in face detection models.
Read the full abstract ▾
All authors Maxime Moussi, Benoît Ronval, Siegfried Nijssen, Félicien Schiltz
Computer Vision
Phantom: A Unified Face-Swap Deepfake Protection Framework with Latent and Spatial Constraints
Jungkon Kim, Cheolseung Jung, Jong-Min Choi et al. (+1 more)
Face-swapping deepfakes pose an escalating threat to personal
privacy by enabling unauthorized identity manipulation. While
adversarial approaches have demonstrated success against black-box face recognition (FR) models, their applicability to face-swapping scenarios remains underexplored. In particular, reliance on fixed or random targets yields ambiguous latent guidance, and the lack of explicit spatial constraints causes perturbations to spill into identity-irrelevant regions. These issues are further exacerbated by identity-style disentanglement, which suppresses
adversarial signals during deepfake generation. In this paper, we present Phantom, a unified face-swap deepfake protection framework that jointly constrains perturbations in latent and spatial domains. Phantom adaptively synthesizes identity-shifted yet attribute-preserving targets to guide identity-aware latent optimization, and applies masked perturbations confined to semantically relevant facial regions. Extensive experiments on state-of-the-art face-swapping deepfakes demonstrate that Phantom improves protection success rates in dodging scenarios by 27.8%, 25.6%, and 16.6% on UniFace, INSwapper, and SimSwap, respectively, while also enhancing visual quality. Furthermore, Phantom generalizes to impersonation scenario, yielding up to 10.2% higher protection while improving perceptual fidelity. These results underscore the effectiveness of jointly leveraging latent and spatial constraints for robust and coherent facial
privacy protection.
Read the full abstract ▾
All authors Jungkon Kim, Cheolseung Jung, Jong-Min Choi, Juseong Lee
Computer Vision
Look But Don't Touch with Sparse Autoencoders for Unlearning in Diffusion Models
Enrico Cassano, Riccardo Renzulli, Rayyan Ahmed et al. (+2 more)
Sparse autoencoders (SAEs) have recently been proposed as interpretable tools for concept-level manipulation, under the assumption that isolated features can serve as controllable intervention points. In this work, we systematically evaluate this assumption in the context of object erasure and steering in
diffusion models. We show that while SAEs reliably detect and localize semantic concepts within
diffusion model activations, direct intervention in their latent space frequently induces out-of-distribution activations, resulting in severe visual artifacts. To disentangle detection from intervention, we use SAE activations purely as semantic detectors to identify image regions containing the target object, and replace those patch
embeddings with the ones that do not contain it. This detection-based replacement preserves the
diffusion model's activation statistics and produces significantly cleaner erasure results than latent steering. Our findings reveal a fundamental gap between concept detection and concept intervention in
diffusion models: monosemantic or sparse features are not inherently suitable as control knobs for steering. These results position SAEs as powerful interpretability tools for analyzing generative models, but highlight important limitations when used for direct manipulation, such as unlearning.
Read the full abstract ▾
All authors Enrico Cassano, Riccardo Renzulli, Rayyan Ahmed, Marco Grangetto, Stephan Alaniz
Computer Vision
Intrinsically Stable Spiking Neural Networks: Overcoming the Performance Barrier in the Absence of Batch Normalization
Ruichen Ma, Xiaoyang Zhang, Jian Bai et al. (+5 more)
The performance of deep spiking neural networks (SNNs) often relies on batch normalization (BN). However, the advanced dynamic BN variants used in state-of-the-art models introduce runtime multiplications, which weaken the hardware-efficiency motivation of SNNs. To address this tension, we identify catastrophic firing-rate decay as a primary cause of severe performance degradation in normalization-free SNNs. Guided by this insight, this work proposes the Intrinsically Stable SNN (IS-SNN) architecture, which removes activation-normalization layers by enforcing signal homeostasis through
topology-aware weight standardization and modified residual connections. By folding the standardization operations into static weights offline, IS-SNN removes the runtime statistics tracking and multiplications introduced by activation normalization, restoring an accumulation-oriented inference datapath. Comprehensive experiments show that IS-SNN achieves performance competitive with or superior to computationally expensive dynamic BN techniques across VGG, ResNet, and
Transformer-based models. Notably, it achieves a competitive accuracy of 68.05% on ImageNet and overcomes the severe depth limitations of prior BN-free attempts. Together with a 96.4% reduction in FPGA lookup table resource consumption for
neuron implementations, these results support IS-SNN as a practical framework for building accurate and hardware-friendly deep neuromorphic systems.
Read the full abstract ▾
All authors Ruichen Ma, Xiaoyang Zhang, Jian Bai, Guanchao Qiao, Liwei Meng, Ning Ning, Yang Liu, Shaogang Hu
Computer Vision · Cross-listed from cs.RO
RCT: A Robot-Collected Touch-Vision-Language Dataset for Tactile Generalization
Jingbo He, Michael Färber and Roberto Calandra
For
robots manipulating open-world objects, tactile representations must generalize to unseen materials. We introduce RCT (Robotic Contact Tactile), a
robot-collected touch-
vision-
language dataset with 29,279 tactile frames from full
robot presses on 122 industrial reference materials in 7 categories, recorded with three DIGIT sensors at multiple contact positions. RCT preserves each press as a contact sequence, enabling held-out evaluation across materials, categories, sensors, contact positions, and contact sequences. Frames from one press are strongly correlated: frame-random splits can place near-duplicate observations of the same physical interaction in both training and test. With the encoder held fixed, removing contact-sequence overlap reduces tactile-to-text Recall@1 by 17.7 percentage points. When materials are additionally held out at training time, performance drops sharply, leaving held-out-material Recall@1 at 25.1 +/- 6.1% averaged over three held-out draws. The public TVL/HCT split shows the same structure: every test contact sequence appears in training, and raw-pixel nearest neighbors recover the correct sequence in 98.3% of cases. Uniformly sampling a press improves contrastive training, and RCT-trained
embeddings improve category probes on unseen materials. RCT makes contact-sequence-aware, held-out-material evaluation reproducible and exposes novel-material generalization as a central challenge for robotic tactile perception. The RCT
dataset is open-sourced at https://faerber-lab.github.io/RCT/
Read the full abstract ▾
Computer Vision
Semantic Occupancy Prediction with Dual Range-Voxel Representation
Sitao Chen, Zhuangwei Zhuang, Hui Luo et al. (+3 more)
LiDAR-based 3D semantic occupancy prediction, which aims to provide accurate and comprehensive scene representation, is crucial for autonomous driving systems. As point clouds suffer from sparsity and incompleteness, leading to insufficient semantic learning and difficult occupancy perception, existing methods often stack multi-sweep point clouds to obtain dense spatial information. However, such a naive strategy also results in efficiency (e.g., additional computational burden) and robustness (e.g., pose transformation noise) concerns, which hinder their practical applications. In this work, we propose a Dual Range-Voxel Representation (DRVR) that leverages the range-view context and voxel-view geometry of single-sweep point clouds for 3D semantic occupancy prediction, eliminating the concerns associated with the multi-sweeps. Specifically, we use the range-view encoder to extract the compact context of the scene. To fully exploit the spatial information, we design a geometry-aware voxel-view encoder that extracts multi-scale voxel-view features separately and combines them for better
geometric occupancy prediction. Moreover, we propose a range-voxel fusion module to cooperate range- and voxel-view features via voxel-to-range and range-to-voxel fusions. Extensive experiments on nuScenes-Occupancy, SemanticKITTI and SemanticPOSS show the superiority of our method. Especially on nuScenes-Occupancy, our single-sweep DRVR achieves 5.4% improvement in mIoU and 2.1x acceleration compared to the multi-sweep method.
Read the full abstract ▾
All authors Sitao Chen, Zhuangwei Zhuang, Hui Luo, Lizhao Liu, Qingyao Wu, Mingkui Tan
Computer Vision
Histogram-constrained Image Generation
Haoming Liu, Yuanhe Guo, Yijia Cao et al. (+2 more)
Diffusion models have emerged as a dominant paradigm in generative modeling, enabling high-fidelity sampling from complex data distributions. Despite impressive capabilities, controlling
diffusion models to produce outputs aligned with user intent remains an open challenge, especially when balancing global coherence with local precision. Existing control mechanisms vary in the granularity of their conditioning signals. For example, textual
prompts guide generation globally through high-level semantics, while ControlNet-like approaches secure precise local structure via dense conditions. In this work, we introduce Histogram-constrained Image Generation (HIG), a novel control mechanism that falls into the middle ground of control granularity. Our framework enforces user-specified distributional constraints (e.g., color histograms or latent
token distributions) during the generation process with exact precision. We model such control as an optimal transport (OT) problem and apply explicit guidance transformations during sampling, thereby driving the
diffusion trajectory to align with the desired histogram. We demonstrate the versatility of HIG across diverse applications, including constrained generation via color/latent histograms and high-capacity information
embedding through histogram-level encoding. Our findings underscore the promise of distributional control, a flexible and interpretable control scheme that is fully compatible with existing control mechanisms, diversifying the hybrid strategies for controllable image generation. Our project page is available at: https://maps-research.github.io/hig/.
Read the full abstract ▾
All authors Haoming Liu, Yuanhe Guo, Yijia Cao, Shenji Wan, Hongyi Wen