Computer Vision · Cross-listed from cs.RO
VLM-CASE: Vision-Language Model Enabled Context-Adaptive Safety Envelopes for Anticipatory Safe Autonomous Driving
Tianjia Yang, Ke Li, Ruwen Qin et al. (+1 more)
Adverse driving conditions, such as bad weather, remain a principal barrier to autonomous driving because they degrade two things at once: what the vehicle can perceive and what it can physically do. Human drivers cope by anticipation,
reasoning about the scene and re-budgeting speed, following distance, and steering before grip or sight is lost, whereas current autonomous driving systems at best react after the fact. This paper proposes VLM-CASE, a framework that gives an autonomous vehicle this anticipatory capacity while keeping its motion bounded by a formal safety model at all times. A
vision-
language model (VLM), fine-tuned with low-rank adaptation (LoRA), reasons about the scene from the front-camera image and reports the road surface and visibility conditions. This output parametrizes a context-adaptive safety envelope (CASE), derived from physical limits and the guarantees of responsibility-sensitive safety, that couples braking and steering through a shared friction budget. A model predictive controller then drives freely within the envelope, while the VLM runs asynchronously so it never blocks the real-time control loop. We validate the framework in closed-loop CARLA simulation on tasks that demand both lateral and longitudinal control, across a range of weather, road-surface, and lighting conditions. The resulting controller, VLM-CASE-MPC, completes all trials, outperforming a conventional MPC baseline and a state-of-the-art VLM-integrated controller. Ablations confirm that the gains come from context adaptation, with the friction and visibility adaptations proving complementary. Furthermore, the framework is controller-agnostic and pairs with almost any low-level controller, offering a promising direction for safe autonomous driving. The
dataset and supplementary materials for VLM-CASE are available at https://github.com/ytj254/VLM-CASE.
Read the full abstract ▾
All authors Tianjia Yang, Ke Li, Ruwen Qin, Xianbiao Hu
Computer Vision
FSDC-DETR: A Frequency-Spatial Domain Collaborative DETR for Small Object Detection
Aiwen Liu, Chengguang Zhu, Gang Wang et al. (+5 more)
Small object detection (SOD) remains a challenging task in real-world applications. Despite recent advances, existing detectors remain limited by rigid processing that entangle spatial aggregation with implicit frequency aliasing and truncation, leading to inadequate preservation of high-frequency components for SOD. To tackle these limitations, we propose a Frequency-Spatial Domain Collaborative Detection
Transformer (FSDC-DETR), a novel collaborative framework that explicitly models complementary spatial and frequency representations. Specifically, we first introduce Dual-Branch Frequency-Spatial Adaptive Fusion (DBFSAF) to enhance frequency diversity and adaptively capture frequency-spatial domain discriminative representations. Building on these representations, a frequency-spatial interaction scheme is further explored within the hybrid encoder to enable progressive feature propagation to the decoder. In particular, structure-aware frequency-spatial aggregation is achieved through Shunt Frequency-Spatial Feature Fusion (SFS-FF), establishing bidirectional interaction and progressive cross-scale propagation between frequency and spatial representations for coherent discriminative modeling. Meanwhile, informative high-frequency responses are preserved during scale transitions through Frequency-Spatial Dynamic Downsampling (FSD-Down), thereby minimizing frequency degradation throughout multi-scale fusion for the precise SOD. Experimental results demonstrate that FSDC-DETR achieves state-of-the-art performance, improving AP by 6.4 on VisDrone-DET2019 and 6.6 on AITODv2, with gains of 6.8 and 6.9 AP for small objects. The code is available at github.com/nevereverinsomnia/FSDC-DETR.
Read the full abstract ▾
All authors Aiwen Liu, Chengguang Zhu, Gang Wang, Dandan Zhu, Haodong Lin, Yan Wang, Huiyu Zhou, Zhengyi Pan
Computer Vision
Claim-Level Rubric Rewards for Video Caption Reinforcement Learning
Mingqi Gao, Hongyuan Dong, Yifei Chen et al. (+6 more)
In this paper, we introduce Claim-Level Rubric Rewards (CuRe), a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense
video captioning. Existing reward designs generally fall into two categories: holistic response-level judgment across heterogeneous criteria, or
alignment-based evaluation against reference captions. However, both paradigms suffer from fundamental limitations. Holistic rewards struggle to ensure factual accuracy and are prone to stylistic reward hacking, while reference-based rewards overly rely on rigid textual
alignment, failing to preserve the completeness and diversity inherent to open-ended generation tasks. To address these challenges, CuRe reformulates reward modeling as fine-grained claim-level verification. Specifically, CuRe decomposes captions into category-aware atomic claims through a structured rubric, converting holistic evaluation into simpler and more reliable claim-level verification.
Read the full abstract ▾
All authors Mingqi Gao, Hongyuan Dong, Yifei Chen, Zhisheng Zhong, Zheng Ruan, Wenjin Hou, Yu Chen, Han Hu, Yansong Tang
Computer Vision
Fully Rotation-Equivariant Spectral-Spatial Learning for Multispectral Object Detection
Peng Zhang, Tingfa Xu, Shuaihao Han et al. (+1 more)
Existing multispectral detectors are limited by discrete spectral processing, a scale-dependent shift in the relative reliability of spectral and spatial cues across pyramid levels, and the lack of explicit rotation-equivariant
geometric priors for arbitrarily oriented objects. To tackle these limitations, we propose FressDet, a fully rotation-equivariant spectral-spatial learning framework for multispectral object detection, capable of capturing the continuous, ordered nature of spectral structure and enabling reliable spectral-spatial fusion across pyramid levels under arbitrary in-plane rotations. FressDet integrates three complementary components. Spectral Implicit Warp (SpeIW) enables query-based spectral resampling via a coordinate-conditioned implicit field, yielding a monotone, order-preserving warp. Rotation-Equivariant Consistency Weighting (ReCoW) adaptively fuses spectral and spatial branches based on branch reliability, reinforcing informative cues while suppressing noise across pyramid levels. The oriented-aware head exploits group-indexed features to stably predict oriented objects without parameter replication. Taken together, FressDet learns more discriminative and robust spectral-spatial representations even under rotational perturbations. By achieving state-of-the-art performance with 93% fewer parameters on five public
benchmarks, FressDet demonstrates its effectiveness and generalizability.
Read the full abstract ▾
All authors Peng Zhang, Tingfa Xu, Shuaihao Han, Jianan Li
Computer Vision
UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation
Mengmeng Liu, Diankun Zhang, Jiuming Liu et al. (+7 more)
World Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving by using future
video prediction as dense supervision for scene dynamics and temporal causality. However, it remains unclear which architecture better transfers
video-modeling benefits to trajectory generation. Existing cascaded or dual-DiT designs separate
video imagination from action prediction, weakening the transfer of
video-learned world dynamics to the trajectory branch: the action model may still overfit
dataset-specific driving priors, while the
video model only indirectly regularizes planning. We propose UNIVERSE, a unified
video-action model built upon a single mask-modulated
Diffusion Transformer. By co-training future
video latents and ego-trajectory
tokens within shared generative parameters, UNIVERSE allows dense
video supervision to directly shape trajectory denoising, leading to stronger cross-domain action generalization. To ensure causal validity and efficient deployment, we introduce a Modality-Decoupling Visibility Mask, which shares historical context across modalities while blocking mutual
attention between future
video and trajectory
tokens. This prevents future-target leakage and enables trajectory-only inference by removing future-
video denoising at test time, achieving a $4.3\times$ speedup over joint
video-action rollout while maintaining comparable planning accuracy. The same model also supports
video-only and joint
video-action rollouts. Experiments show that UNIVERSE achieves 91.0 PDMS on NAVSIM (vs. 89.6 for the Two-DiT variant), and demonstrates strong zero-shot transfer to nuScenes and Bench2Drive without fine-tuning, while ablations confirm the importance of single-DiT unification,
video co-training, and mask-based modality decoupling.
Read the full abstract ▾
All authors Mengmeng Liu, Diankun Zhang, Jiuming Liu, Jianfeng Cui, Hongwei Xie, Guang Chen, Hangjun Ye, Francesco Nex, Hao Cheng, Michael Ying Yang
Computer Vision · Cross-listed from cs.AI
ASSEMCAD: Production-Ready CAD Assembly Generation from Natural Language
Yurui Dong, Shu Zou, Siqi Li et al. (+7 more)
Recent advances in large
language models and programmatic CAD have significantly improved Text-to-CAD generation for individual parts. However, production-ready mechanical assembly generation remains largely unsolved. Unlike single-part modeling, assemblies require coordinated
reasoning over multiple components, functional interfaces, assembly relations, engineering principles, and physical consistency. Consequently, directly generating executable CAD code is insufficient for constructing mechanically valid and reusable assemblies. We present AssemCAD, an axiom-grounded framework for production-ready CAD assembly generation from natural
language. Instead of representing an assembly as monolithic CAD code, AssemCAD first constructs an axiomatic Assembly Specification consisting of typed parts, geometry-backed ports, executable mates, and engineering axioms. Each assembly relation is explicitly grounded in one or more engineering principles, making the resulting specification interpretable, reusable, and verifiable. To realize this specification, AssemCAD introduces a port- and mate-based CAD assembly library that executes symbolic assembly relations through deterministic mate transformations and validates declared interfaces using concrete B-Rep
geometric evidence. Built on this representation and library, AssemCAD further supports on-demand synthesis of reusable parametric component factories for both standard and open-world geometries. Experiments on AssemBench show that AssemCAD substantially improves assembly preservation and physical validity over code-centric CAD generation baselines, while generalizing across different foundation-model backbones. By combining axiom-grounded assembly
reasoning with deterministic
geometric execution, AssemCAD extends Text-to-CAD from isolated part generation toward production-ready mechanical assembly design.
Read the full abstract ▾
All authors Yurui Dong, Shu Zou, Siqi Li, Nianchen Deng, Hongbin Zhou, Xuemeng Yang, Pinlong Cai, Licheng Wen, Xinyu Cai, Botian Shi
Computer Vision
Green for Go, Red for No: Visual Grounding via Semantic Segmentation for VLA Navigation Policies
Adrian Szvoren, Dimitrios Kanoulas and Nilufer Tuptuk
Vision-
language-action (VLA) models enable
robot navigation from natural
language and visual goals, but remain susceptible to perceptual distractions and ambiguous scene interpretations. This paper presents the first empirical evaluation of visual grounding for VLA
navigation policies. We propose a real-time
segmentation-based grounding method that highlights traversable areas in green and non-traversable areas in red using SegFormer. Two variants are evaluated: observation-only
segmentation and joint observation-goal augmentation. Using OmniVLA on the Grand Tour
dataset, we show that visual grounding reduces the mean waypoint error by 27-44% at the farthest waypoint, depending on the instruction length. The benefits are greater for long instructions than for short instructions, and grounding provides little improvement for image goals. Normalized error analysis indicates that grounding primarily acts as a trajectory length regularizer, reducing the predicted path length by 30% without improving per-unit-distance
reasoning. Our results indicate that visual grounding offers a simple, computationally inexpensive method to improve VLA
navigation without model retraining, although it cannot compensate for missing training signals in out-of-distribution instructions.
Read the full abstract ▾
Computer Vision
Video-Text Temporal Localization via Multi-Scale Convolution and Dynamic Routing
Gengtian Shi, Jinze Yu, Chenhao Wu et al. (+5 more)
Video-text temporal localization requires precise
alignment between natural
language queries and corresponding
video segments, a fundamental challenge in
multimodal understanding. We present a novel framework that addresses two critical limitations of existing methods: inadequate modeling of hierarchical temporal structure and inability to handle complex many-to-many correspondences between modalities. Our approach introduces a multi-scale temporal convolutional encoder that captures motion patterns across different temporal granularities - from instantaneous frame transitions to extended action sequences. We further propose a capsule-based dynamic routing mechanism that iteratively refines segment-query associations through structured agreement updates, enabling flexible modeling of non-monotonic
alignments. These components are unified through a multi-task learning objective that jointly optimizes temporal boundary regression, cross-modal semantic
alignment, and capsule diversity. Extensive experiments on ActivityNet Captions demonstrate significant improvements, achieving 42.9% Recall@0.5 and 41.1% mean IoU, surpassing strong
transformer-based baselines while maintaining computational efficiency. Our results validate that combining hierarchical temporal modeling with structured semantic routing provides an effective solution for fine-grained
video-
language understanding.
Read the full abstract ▾
All authors Gengtian Shi, Jinze Yu, Chenhao Wu, Shaofei Wang, Eiji Fukuzawa, Junjie Tang, Hiroshi Onoda, Jiang Liu
Computer Vision
Be Indiscrete: The Benefits of Learning Continuous Spine Degeneration Severity Scores
Maria Monzon, Andrew Zisserman, Robin Y. Park et al. (+2 more)
Lumbar spine degeneration is a major contributor to chronic low back pain and is routinely assessed on MRI using ordinal grading systems, e.g. normal, mild, moderate, severe. Consequently, most approaches to train models to grade these MRIs formulate grading as a multi-class classification problem, treating ordinal grades as categorical, ignoring differences in misclassification severity, and imposing hard decision boundaries on a continuous disease process. This work explores modeling spinal degeneration as a continuous severity ranking problem. We introduce SpineRankNet, a framework that learns scalar severity scores from lumbar spinal MRI, and compare it against multi-class classification and ordinal regression. Using multiple degeneration measures from the Genodisc
dataset, we show that a model trained using a ranking loss to produce a continuous score enables fine-grained ordering of MRI scans. Furthermore, the ordinal grading classes can be recovered from the score with comparable accuracy to those from a model trained directly for classification. The score learned by ranking even improves discrimination between more distant classes. Source code is available at https://github.com/spinetools/spineranknet.
Read the full abstract ▾
All authors Maria Monzon, Andrew Zisserman, Robin Y. Park, Catherine R. Jutzeler, Amir Jamaludin
Computer Vision
TimeThink: Reasoning with Time for Video LLMs
Handong Li, Longteng Guo, Zikang Liu et al. (+8 more)
Video reasoning requires models to identify and verify temporally localized evidence within long
video sequences. Recent
Video Large
Language Models (
Video-LLMs) have shown promising
reasoning abilities when aligned with reinforcement learning, yet existing approaches typically rely on outcome-based rewards that supervise only the final prediction. Such supervision provides limited guidance on how models should discover the relevant temporal evidence during intermediate
reasoning. In this work, we propose TimeThink, a reinforcement learning framework that explicitly guides temporal evidence discovery in
Video-LLMs. Our key idea is to treat temporal clue steps as the fundamental optimization primitive of
video reasoning, where each
reasoning step references a candidate time interval in the
video. We introduce a step-wise temporal process reward that provides localized credit assignment for these clues and a joint process–outcome optimization objective that balances
reasoning fidelity with task correctness. To enable scalable training, we construct TimeThink-RFT-20K, a
dataset with automatically derived temporal evidence segments. Extensive experiments across
video reasoning, temporal grounding, and general
video understanding
benchmarks show that TimeThink consistently improves both temporal localization and
reasoning performance, achieving state-of-the-art results among open-source
video RL models.
Read the full abstract ▾
All authors Handong Li, Longteng Guo, Zikang Liu, Dongze Hao, Yepeng Tang, Zijia Zhao, Jie Jiang, Zhiwei Jin, Chen Chen, Haonan Lu, Jing Liu
Computer Vision
RADIANCE: Relative Adaptive Denoising with IP-Adapter for Novel Concept Enhancement
Zi-Xiang Ni, Bo-Lun Huang, Teng-Fang Hsiao et al. (+2 more)
Text-to-image (T2I)
diffusion models have achieved striking progress but still struggle to synthesize rare concepts involving unusual attribute-object pairings, often resulting in concept omission or semantic drift where a dominant entity overwhelms the generation. Tracing these failures to a lack of compositional balance during the denoising trajectory, we propose RADIANCE, a training-free framework that treats inference as a closed-loop feedback process. RADIANCE augments pretrained backbones with three modular components: (1) a Compositional Similarity Monitor (CSM) that tracks the emergence of objects and attributes in intermediate latents via CLIP-based feedback; (2) a Bidirectional Scale Controller (BSC) that applies a reactive "restoring force" using positive and negative IP-Adapter scales to rebalance biased trajectories; and (3) a Feedback Guidance Scheduler (FGS) that coordinates these updates across timesteps without additional training. We further extend the framework to multi-object
prompts via Delayed Adapter Activation (DAA) and Layer-wise Alternating Guidance (LAG) to prevent premature concept fusion. By overlapping monitoring and denoising through pipelined execution, RADIANCE maintains competitive latency while significantly enhancing the per-sample success rate and effective throughput. Experiments on RareBench and T2I-CompBench demonstrate that RADIANCE consistently enhances compositional
alignment and perceptual quality over state-of-the-art baselines.
Read the full abstract ▾
All authors Zi-Xiang Ni, Bo-Lun Huang, Teng-Fang Hsiao, Bo-Kai Ruan, Hong-Han Shuai
Computer Vision
LangLoc: "Tell Me What You See"
Shaurya Kishore Panwar, Roham Zendehdel Nobari, Shirley Feng Yi Lau et al. (+4 more)
We tackle fine-grained indoor localization from natural
language: given a free-form description of one's surroundings, estimate the observer's 2D position and heading within a known 3D environment.
Language queries are lightweight,
privacy-preserving, and need no camera - yet prior work stops at coarse scene
retrieval and cannot resolve an intra-scene pose. We close this gap with LangLoc, a three-stage pipeline that (i) retrieves the correct scene via a dual-branch GATv2 encoder with CLIP semantic features, surpassing the previous best by 8 percentage points in Top-1 recall; (ii) estimates position and heading by scoring a dense floor grid through ray-cast object visibility, reaching a median error of 0.95 m; and (iii) resolves residual ambiguity through a
Bayesian dialog module that asks targeted yes/no questions and updates a pose posterior until the location is pinpointed. To support this task we contribute a
benchmark of $13{,}000{+}$ pose-indexed natural-
language descriptions over $1{,}300{+}$ indoor 3D scans.
Read the full abstract ▾
All authors Shaurya Kishore Panwar, Roham Zendehdel Nobari, Shirley Feng Yi Lau, Abu Bakr Rahman Shaik, Manuel Günther, Marc Pollefeys, Daniel Barath
Computer Vision
Consistent and Editable: A Balanced Framework for Text-Guided Video Editing
Tao Jin and Li Xiao
Recently,
diffusion models have achieved considerable success in the text-guided
video editing domain. However, existing works often struggle to balance the trade-off between temporal consistency and editability in
video editing, with consistency and editability typically being inversely related. To address this, we propose a high-quality
video editing framework enhanced for consistency and editability, named EquiEdit, which improves coordinatively the temporal consistency and editability of the edited
videos while achieving a balance between the two. In terms of temporal consistency, the proposed temporal Mamba module with a tailored temporal-aware scanning scans fused
video sequences following four designed directions, effectively enhancing the inter-frame consistency of edited
videos. For editability, we design a noise injection strategy based on the spectral transformation to increase editing flexibility, where the
Fourier transform is used to preserve the hidden structure in the initial latent noise used for editing, ensuring inter-frame consistency of the edited
video and fidelity to the input
video. Extensive qualitative and quantitative experiments demonstrate the effectiveness of our method in terms of temporal consistency and editability, as well as its great fidelity to the input
video itself.
Read the full abstract ▾
Computer Vision
RUFNet: Query-Guided Support Mask Refinement and Uncertainty Fusion based on Hybrid Mamba for Few-Shot Brain Tumor Segmentation
Dongyi He, Xiangkai Wang, Binbing Xu et al. (+5 more)
Few-shot
brain tumor
segmentation remains challenging due to noisy support masks, inter-patient variations between support and query images, and the lack of pixel-wise confidence estimation. This study proposes RUFNet, a Hybrid Mamba-based few-shot framework that combines support mask refinement with uncertainty-aware posterior fusion. To preserve support-query dependencies with manageable cost, RUFNet adopts a Hybrid Mamba interaction backbone with linear complexity. To reduce support-mask noise, an
Attention-Guided Mask Refinement module (AGMR) uses query features to recalibrate support masks and improve prototype consistency. To handle ambiguous predictions, an Uncertainty-Aware Posterior Fusion module (UAPF) estimates pixel-wise variance and adaptively balances few-shot predictions with query-aligned priors. On the
Brain Tumor
Segmentation Challenge (BraTS) 2020
dataset, RUFNet achieves Dice coefficients of 84.3% and 86.1% in the 1-way 1-shot and 1-way 5-shot settings, respectively, outperforming the compared state-of-the-art methods. These results suggest that Hybrid Mamba interaction, mask refinement and uncertainty modelling can improve the robustness of few-shot medical image
segmentation. The official implementation code is available at https://github.com/hdy6438/RUFNet.
Read the full abstract ▾
All authors Dongyi He, Xiangkai Wang, Binbing Xu, Bin Jiang, Hongjie Yan, Weixiang Liu, Wai Ting Siok, Nizhuan Wang
Computer Vision · Cross-listed from cs.LG
Beyond Modality Fusion: Deep Ensembles for Multimodal Classification
Ilya Burenko and Dmitry Vetrov
In
multimodal classification, late-fusion approaches classify concatenated modality-specific features extracted by unimodal neural networks. When modality imbalance is pronounced, various regularization techniques have been proposed to balance the learning process and overcome the inferior performance of late-fusion networks. In contrast, this work demonstrates that
multimodal data can be effectively classified without any explicit modality fusion, using deep ensembles of unimodal networks. We systematically compare deep ensembles to late-fusion networks at equal parameter count and show that ensembles consistently outperform state-of-the-art late-fusion methods designed to address modality imbalance. This advantage also holds over intermediate-fusion techniques we evaluated and over hybrid methods that combine unimodal and
multimodal predictions. We propose and empirically validate a method for selecting the number of models per modality in an ensemble, avoiding computationally expensive exhaustive search. Under extreme modality imbalance and small ensemble sizes, the heuristic indicates that ensembles of unimodal models trained solely on the stronger modality are preferable; as the ensemble scales up, incorporating models from the weaker modality becomes beneficial. Both predictions align with our empirical findings. To systematically explore the challenges of optimizing
multimodal models, we propose a synthetic
multimodal framework that allows control over both the number of modalities and their predictive strength; our findings are consistent across synthetic and real-world
datasets. Finally, by fitting scaling laws to bimodal
datasets, we estimate the asymptotic performance of ensembles.
Read the full abstract ▾
Computer Vision
Comparison of Loss Functions for Robust Deep Learning-based Echocardiography Segmentation when Learning with Partially Labelled Data from Multiple Domains
Iman Islam, Esther Puyol-Antón, Bram Ruijsink et al. (+2 more)
Echocardiography is the first imaging modality used for assessing cardiac function, and accurate
segmentation of cardiac structures is essential for deriving biomarkers. However, the development of effective automated
segmentation models for multiple cardiac structures is challenged by the difficulty of training on
datasets from different sources that are often partially-labelled. This study aims to address this challenge by evaluating the performance of three loss functions - adaptive categorical cross
entropy (aCCE) loss, marginal loss, and the adaptive binary cross
entropy (aBCE) loss - in handling partially-labelled data. We conduct a comprehensive comparison of these loss functions across multiple scenarios and network architectures: intra-domain and inter-domain tasks, with both single and multiple partial-labels, and varying proportions of fully-labelled to partially-labelled data. Our experiments reveal that all three loss functions exhibit strong performance in intra-domain
segmentation tasks, effectively handling label variations within the same domain. For inter-domain tasks, where models are trained on
datasets with a domain shift, the aBCE and marginal losses show superior performance when dealing with the case of one label being missing from some training examples. In scenarios involving more than one label being missing, marginal loss outperforms the other methods, demonstrating its robustness in such complex conditions. These results highlight the strengths of each loss function depending on the labelling scenario, emphasizing the importance of selecting the appropriate loss function to optimize model performance. This study represents the first investigation of techniques for handling partially-labelled data from multiple different domains in echocardiography
segmentation and provides a comprehensive comparison of loss-based solutions.
Read the full abstract ▾
All authors Iman Islam, Esther Puyol-Antón, Bram Ruijsink, Andrew J. Reader, Andrew P. King
Computer Vision
Unsupervised Pixel-Level Semantic Left-Right Understanding of In-the-Wild Images
Weikang Wang, Tobias Weißberg and Florian Bernard
While various works address reflective
symmetry understanding in 3D data and images, pixel-level semantic left-right prediction of in-the-wild images remains challenging, due to certain difficulties including the lack of 3D information, occlusion, object pose variation, partiality, etc. In this work, we propose an unsupervised learning framework to tackle this challenge. Leveraging recent advances in vertex-wise semantic left-right understanding of 3D data, our unsupervised learning method jointly utilises 3D shape and image
datasets to infer pixel-wise semantic left-right predictions in single-view images. In particular, we show that a medium-scale 3D shape
dataset comprising mainly of human- and quadruped animal-like shapes, combined with diverse in-the-wild image data, are sufficient to achieve high-quality semantic left-right prediction in images, even for entirely unseen 3D object categories, such as cars or trains. Overall, our approach achieves superior performance in dense pixel-wise semantic left-right predictions on both rendered and in-the-wild image
datasets when compared to existing state-of-the-art methods.
Read the full abstract ▾
Computer Vision
Geometry-aware Depth-guided Representation Learning for Structure-preserving Low-light Image Enhancement
Fang Gao, Jiongkai Qin, Jiabao Wang et al. (+5 more)
Low-light degradation reduces image visibility and weakens structural cues that are important for visual representation and scene understanding. Existing low-light image enhancement methods mainly focus on appearance restoration, while insufficiently exploiting scene geometry to preserve structural consistency. To address this limitation, this paper proposes a Depth-guided Multi-scale
Attention Network (DMSA-Net) for geometry-aware low-light image enhancement. DMSA-Net introduces depth-related structural priors into low-light representation learning through reflectance-geometry interaction. A Retinex-based decomposition module is first used to obtain illumination-invariant reflectance representations, from which depth cues are inferred to characterize scene structure under degraded illumination. A multi-scale depth-guided fusion strategy is then embedded into a hierarchical encoder-decoder architecture, where depth-aware
attention adaptively integrates
geometric and appearance features. Experiments on several
benchmark datasets show that DMSA-Net achieves effective low-light restoration while improving structural preservation. Moreover, we construct LOL-D, a depth-augmented low-light
dataset, to facilitate research on geometry-aware low-light
vision.
Read the full abstract ▾
All authors Fang Gao, Jiongkai Qin, Jiabao Wang, Jingfeng Tang, Ming Cheng, Hanbo Zheng, Qingbao Huang, Cheng Wu
Computer Vision
Virtual Category-Guided Continual Generalized Category Discovery
Jiahui Xiong, Qiuxia Lai and Hongsong Wang
Continual Generalized Category Discovery (C-GCD) aims to incrementally identify novel categories from sequential unlabeled data while preserving recognition of known classes, which is an essential capability for open-world visual learning. A major bottleneck lies in ambiguous unlabeled samples that cannot be confidently assigned to known classes nor reliably grouped as novel ones, making pseudo-labeling brittle and often biasing learning toward familiar categories. In this work, we introduce Virtual Category-Guided Continual Generalized Category Discovery by adapting Virtual Category Learning (VCL) to the continual setting. Our method identifies uncertain samples and assigns them to temporary virtual categories, enabling safe and informative learning from unlabeled streams without injecting noisy labels, while improving unlabeled data utilization and mitigating prediction bias. To further stabilize discovery across sessions and enhance class separation, we augment VCL with Expanded Neighborhood Contrastive Learning (ENCL), which exploits extended neighborhood relations and an adaptive margin to learn more discriminative and well-separated representations for both old and emerging classes. Extensive experiments on CIFAR-100, Tiny ImageNet, and ImageNet-100 demonstrate that our approach consistently outperforms state-of-the-art methods, establishing a scalable and effective solution for C-GCD.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Inference
Ruslan Rakhimov, George Bredis, Yuriy Maksyuta et al. (+1 more)
Joint-
Embedding Predictive Architectures (JEPAs) underpin a growing family of latent world models for control from raw pixels. Published JEPA world models for pixel-based control (PLDM, DINO-WM, LeWM, V-JEPA-2-AC) each commit at training time to a single inference paradigm, trajectory optimisation in a learned dynamics model, so a checkpoint trained to plan cannot be read off as a policy without retraining. Deferring that choice to inference would let the constraints that bind at deployment, per-step latency and whether a goal observation can be supplied, pick the path. We present Qantara, an end-to-end JEPA of about 21M parameters whose joint training objective pairs a Brownian-bridge interpolant between consecutive clean latents on the state axis with noise-to-data flow matching on the action axis. One checkpoint serves three inference paradigms without retraining: latent planning, behaviour-cloning action sampling, and a
video-inverse composition that predicts the next latent without action conditioning, then recovers the action bridging the two. Training samples the two noise levels along the edges of the (action-noise, state-noise) square, one edge per inference dispatch, so each path is trained where it is queried. Replacing that sampler with uniform sampling over the square at matched compute drops Push-T planning from 90.1 to 53.3 success rate (SR). On the LeWM control suite at a matched 10-epoch budget, Qantara averages 91.2 SR over the four environments and reaches 93.7 on OGBench-Cube, +7.7 over DINO-WM and +17.0 over LeWM, each measured against the stronger of that baseline's published score and our retrain of it. From the same weights the two goal-blind paths, which receive no goal image, reach 82.1-83.2 SR on Push-T and 70.8-72.8 on Cube at roughly 15-65x lower inference cost, and fall to near-noise on Reacher, whose released training corpus comes from a random policy.
Read the full abstract ▾
All authors Ruslan Rakhimov, George Bredis, Yuriy Maksyuta, Daniil Gavrilov
Computer Vision
MemPose: Category-level Object Pose Estimation with Memory
Xiao Lin, Minghao Zhu, Yun Peng et al. (+4 more)
In the pursuit of robust and generalizable category-level object pose estimation, most existing methods adopt parametric formulations that learn effective representations from data, yet they primarily encode category-level patterns into fixed shape priors or static parameter weights, which limits their scalability to highly diverse instances. In this paper, we rethink category-level pose estimation from a
memory-centric perspective and present MemPose, a
memory-augmented framework that explicitly incorporates category-level
geometric memory into the pose estimation pipeline. We introduce an external
memory buffer that stores and dynamically updates structural representations from previously observed instances, enabling the model to leverage accumulated experience to support current perception. Extensive experiments on four challenging
benchmarks (REAL275, CAMERA25, Housecat6D and Wild6D) demonstrate the superiority of our proposed method over previous state-of-the-art approaches.
Read the full abstract ▾
All authors Xiao Lin, Minghao Zhu, Yun Peng, Liuyi Wang, Qiyi Wang, Chengju Liu, Qijun Chen
Computer Vision
UniSpine-GS: An Efficient Physics-Aware Gaussian Framework for Cross-Modality Multi-view Spine Image Synthesis
Qiuhua Chen, Changning Yu, Na Huang et al. (+2 more)
The diagnosis of spinal diseases is often assisted by 3D imaging techniques in clinical practice. However, precise 3D spinal assessment is limited by the high costs of 3D imaging hardware and the challenges posed by the physical differences between imaging modalities, which hinder the generalizability of models. To address these issues, we propose UniSpine-GS, an efficient, physics-aware Gaussian framework designed for novel-view projection rendering in multi-view spine imaging via a 3D-aware representation. Instead of performing explicit 3D reconstruction, our approach learns a geometry-aware Gaussian representation that ensures anatomical consistency across different views. We introduce SPWM, a structure-guided loss reweighting strategy to improve boundary fidelity and local details. We evaluate our method on the CTSpine3D
dataset and a newly constructed 3D fetal ultrasound
dataset, FeSpine3D. Our results demonstrate that UniSpine-GS significantly outperforms existing methods across all metrics, offering a practical and cost-effective solution for unified multi-view medical imaging. Our code is publicly available at https://github.com/orangeisland66/UniSpine-GS.
Read the full abstract ▾
All authors Qiuhua Chen, Changning Yu, Na Huang, Chao Sun, Bo Du
Computer Vision
Efficient Perception in Automotive Detection and Tracking Using Neuromorphic Computing
Manish Kolachalam and Rani Malhotra
Deep learning
algorithms are notorious for their high carbon footprint and computational demands that limit their deployment on edge devices and raise concerns about their long-term sustainability. Neuromorphic computing and Spiking Neural Networks (SNNs) offer a promising alternative to traditional Von Neumann architectures, providing energy-efficient performance, massively parallel computation, and on-chip learning capabilities. Autonomous machines represent a critical application domain where these advantages are particularly valuable. We present the first comprehensive evaluation of SNNs for real-world automotive multi-object detection and tracking. Using transfer learning with the SpikeYOLO architecture, we achieve mean Average Precision of 0.937 on the KITTI
dataset and 0.771 on BDD100K MOT2020
dataset for object detection and a Higher Order Tracking Accuracy score of 0.701 (KITTI) and 0.445 (BDD100K MOT2020) for object tracking–results competitive with conventional deep learning methods. Our results demonstrate that SNNs can deliver high-performance object detection and tracking in an energy efficient manner, establishing their viability for perception in real-world autonomous systems.
Read the full abstract ▾
Computer Vision
Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction
Johannes Kiechle, Richard Osuala, Daniel M. Lang et al. (+5 more)
In patients with breast cancer, pathological complete response (pCR) has been established as a clinically meaningful surrogate marker for long-term outcomes. While commonly treated with neoadjuvant chemotherapy (NACT), effective treatment decision-making remains challenging, as therapeutic response can vary substantially across patients, calling for predictive models capable of accurately estimating individualized treatment response. To address this, we propose an imaging-based 3D spatio-temporal framework for treatment response prediction that integrates a state-of-the-art graph neural network with relational modeling of temporal interactions across timepoints alongside three novel complementary self-supervised treatment trajectory representation learning objectives. Experiments across a cohort of 585 patients from the public ISPY-2
dataset demonstrate that our method substantially outperforms both
vision and self-supervised learning baselines across several classification metrics. Alongside establishing a breast cancer pCR prediction
benchmark, we include a principled ablation of our method and further introduce and empirically assess the impact of the available number of DCE-MRI timepoints per patient trajectory and the inclusion of inter-scan time-differences. Overall, our study substantiates the utility of clinically meaningful longitudinal medical imagaging modeling for predicting NACT-induced pCR. We will publicly share our code repository and a user-friendly PyPI library for
dataset curation upon publication, effectively promoting reproducible open-source research.
Read the full abstract ▾
All authors Johannes Kiechle, Richard Osuala, Daniel M. Lang, Stefan M. Fischer, Ivana Janíčková, Karim Lekadir, Julia A. Schnabel, Jan C. Peeken
Computer Vision
3DMPE: 3D Multi-Perspective Embedding
Vahan Huroyan, Md Rahat-uz-Zaman and Stephen Kobourov
We study 3D point cloud reconstruction from multiple partially observed 2D projections. Given two or more projections of an unknown 3D point cloud, together with cross-view point correspondences and visibility information, our goal is to recover a consistent 3D configuration when different views contain different subsets of points. We propose 3D Multi-Perspective
Embedding (3DMPE), an optimization-based, training-free method that reconstructs the 3D point cloud and, in the variable-projection setting, jointly estimates the projection maps. 3DMPE extends Multi-Perspective Simultaneous
Embedding to accommodate missing points and incomplete pairwise distance information across views. We consider both fixed-projection and variable-projection settings. Unlike learning-based reconstruction methods that infer shape from raw images and often depend on training data, 3DMPE operates on
geometric observations with established correspondences and does not require category-specific training. Experiments on ShapeNet and Pix3D evaluate reconstruction quality using Chamfer Distance, Earth Mover Distance, and RMSE-Optimize-Align (ROA), and examine the effects of initialization, the number of views, point visibility, and several noise regimes, including noisy distances and erroneous correspondences. The results demonstrate that 3DMPE can effectively reconstruct point clouds from partial multi-view
geometric observations.
Read the full abstract ▾
Computer Vision
ProCon: Projection-Consistency Memory for Training-Free Anomaly Detection
Joongwon Chae, Lihui Luo, Yang Liu et al. (+4 more)
Memory-based anomaly detection is attractive because it localizes defects from normal images without training a decoder or synthesizing pseudo anomalies. However, most
memory methods still use the
memory bank as a nearest-neighbor lookup table: a test patch is treated as normal if it has one nearby normal anchor. This hard
retrieval view is vulnerable to false-normal matches and does not test whether the patch is consistently supported by a local normal neighborhood. We propose ProCon, a training-free framework that turns
memory retrieval into decoder-free reconstruction. ProCon softly projects each test patch onto nearby normal
memory vectors and uses the projection residual as anomaly evidence. To stabilize this residual, it constructs seed-perturbed layer-wise
memories, aggregates bank residuals by a median, and fuses depth-specific residual maps by layer consensus. ProCon requires no decoder training, backbone fine-tuning, learned fusion weights, or pseudo-anomaly supervision. Across MVTec-AD, VisA, and Real-IAD under the single-category evaluation protocol, ProCon achieves strong image- and pixel-level performance under seven standard metrics, including image AUROC scores of 99.8%, 99.2%, and 93.2%, respectively. Ablations show that the gains come from replacing hard
retrieval with soft normal projection and stabilizing the residuals through
memory and depth consensus. The code is available at https://github.com/jw-chae/Procon
Read the full abstract ▾
All authors Joongwon Chae, Lihui Luo, Yang Liu, Dongmei Yu, Peiwu Qin, Runming Wang, Ilmoon Chae
Computer Vision
HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
Gengluo Li, Xingyu Wan, Shangpin Peng et al. (+20 more)
We present HunyuanOCR-1.5, a lightweight end-to-end OCR-specialized
vision-
language model. HunyuanOCR unifies document parsing, text spotting, information extraction, text-image translation, and multi-image document understanding within a single end-to-end VLM. Building upon the lightweight architecture of HunyuanOCR-1.0, HunyuanOCR-1.5 does not redesign the backbone, but systematically improves both efficiency and capability. For efficiency, we adapt DFlash to OCR decoding, significantly reducing the latency of long structured outputs such as dense documents, tables, and formulas while preserving output distribution. Powered by DFlash, HunyuanOCR-1.5 achieves a 6.37x
Transformer inference speedup and a 2.14x speedup under vLLM, delivering the fastest inference among lightweight OCR VLMs. For capability, we propose Agentic Data Flow, an
agent-driven data construction system that transforms model weaknesses into executable data requirements and autonomously performs material search, quality verification, and pipeline development. It substantially improves long-tail capabilities in ancient-script OCR, fine-grained chart and table parsing, multi-image text-centric QA, low-resource
multilingual parsing, and document
hallucination evaluation. HunyuanOCR-1.5 ranks among the top-tier end-to-end OCR solutions on OmniDocBench v1.6 while achieving new performance milestones across these long-tail tasks. Combined with an upgraded pretraining and post-training recipe, HunyuanOCR-1.5 further extends its capability in high-resolution, long-context, and multi-task scenarios. Experiments demonstrate faster inference, broader OCR capability coverage, and the deployment advantages of a lightweight end-to-end model. We will release the model weights and training code to support future research and real-world OCR applications.
Read the full abstract ▾
All authors Gengluo Li, Xingyu Wan, Shangpin Peng, Weinong Wang, Hao Feng, Yongkun Du, Binghong Wu, Zheng Ruan, Zhiqiong Lu, Liang Wu, Pengyuan Lyu, Huawen Shen, Zibin Lin, Shijing Hu, Jieneng Yang, Hongbing Wen, Guanghua Yu, Hong Liu, Bochao Wang, Can Ma, Han Hu, Chengquan Zhang, Yu Zhou
Computer Vision
Unsupervised Detection of Underground Tunnels in Ground-Penetrating Radar Using Depth-Restricted Reconstruction Scoring
Muhammad Junaid, Shoab A. Khan and Nisar Ahmed
Clandestine tunneling beneath oil and gas pipelines enables fuel theft, smuggling, and sabotage, yet conventional monitoring detects damage only after a pipeline has been compromised. Ground-penetrating radar (GPR) can image such tunnels non-invasively, but manual radargram interpretation does not scale to continuous corridor surveillance, and supervised detectors require tunnel examples that are scarce in practice. We present a fully unsupervised detection pipeline trained exclusively on normal subsurface radargrams collected at a purpose-built field site containing three buried tunnels at 1.5-3 m depth. A denoising convolutional autoencoder learns the structure of anomaly-free ground; at inference, tunnels are flagged by reconstruction error. Our central contribution is a depth-restricted top-k anomaly score, which pools the highest reconstruction errors only within the depth band where tunnels can physically occur. This physically motivated rule raises AUC from 0.986 to 0.994 and cuts missed detections from 74 to 17 of 634 tunnel windows, relative to whole-image scoring, without any retraining or labels. We further show that the optimal top-k fraction interacts with the depth restriction - 1% pooling is best on full images, 5% once scoring is depth-restricted - and that spatial voting across overlapping survey windows helps weak per-image detectors but offers no benefit once the scoring rule is strong. The final system attains AUC 0.994, F1 0.975, recall 0.973, and precision 0.976 on 1,600 field test windows spanning 55 survey lines, at a 1.6% false-alarm rate, using no tunnel labels for training, scoring, or threshold calibration.
Read the full abstract ▾
Computer Vision
EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization
Youngkil Song, Yoonjae Baek, Dongwon Kim et al. (+3 more)
Reasoning temporal localization (RTL) requires a model to generate an answer that itself contains the time interval supporting it, coupling high-level
reasoning with temporal grounding in a single response. To tackle this challenge, we propose the first event-centric
video chain-of-thought framework, dubbed EventCoT. EventCoT first performs event-centric tokenization, converting the
video into compact event
tokens that enable efficient identification of question-relevant events. It then reasons within these events to generate the answer, grounding the time interval via
embedding matching that aligns placeholder
tokens with visual
embeddings. EventCoT achieves state-of-the-art results on ActivityNet-RTL while using substantially fewer visual
tokens than previous work, and attains strong zero-shot results on the grounded
video question answering
benchmark ReXTime. Our code will be released for research purposes.
Read the full abstract ▾
All authors Youngkil Song, Yoonjae Baek, Dongwon Kim, Inho Kim, Dongkeun Kim, Suha Kwak
Computer Vision
PAGE: Towards Practical Human-level Gaze Target Estimation
Zhoutong Ye, Chengwen Zhang, Zhaibin Cui et al. (+10 more)
Gaze target estimation, the task of predicting where a person is looking in a scene, is crucial to understanding human
attention and intent. It is a challenging task that combines high-level understanding of global scene semantics and precise spatial
reasoning using human appearance (e.g. pose, eye orientation). As a result, human-level performance remains elusive for existing models, limiting their practical application. To this end, we propose PaGE (Practical Gaze
Estimator), a gaze estimation model that explicitly models the complex interaction between scene and head features. Using a PaGE model with a large ViT-H+ backbone as the teacher, we further distill student models with lighter backbones on a much larger and more diverse unlabeled
dataset. The architectural improvements and novel training recipe allow PaGE to achieve state-of-the-art performance on several gaze estimation tasks, outperforming humans in 7 out of 9 metrics while reducing the human-AI gap by at least 60% in the remaining 2. The distilled student models retain most of the teacher's performance while being lightweight enough for practical deployment on
robots and consumer devices. The code and model checkpoints are available at our project page.
Read the full abstract ▾
All authors Zhoutong Ye, Chengwen Zhang, Zhaibin Cui, Mingze Sun, Jiaqi Liu, Xiangwu Li, Qingyang Wan, Chang Liu, Xutong Wang, Huan-ang Gao, Yu Mei, Chun Yu, Yuanchun Shi