Computer Vision · Cross-listed from cs.RO
PROBE: Probabilistic Occupancy BEV Encoding with Analytical Translation Robustness for 3D Place Recognition
Jinseop Lee, Byoungho Lee and Gichul Yoo
We present PROBE (PRobabilistic Occupancy BEV Encoding), a learning-free
LiDAR place recognition descriptor that models each BEV
cell's occupancy as a Bernoulli random variable. Rather than relying on discrete point-cloud perturbations, PROBE analytically marginalizes over continuous Cartesian translations via the polar Jacobian, yielding a distance-adaptive angular uncertainty $\sigma_\theta = \sigma_t / r$ in $\mathcal{O}(R{\cdot}S)$ time. The primary parameter $\sigma_t$ represents the expected translational uncertainty in meters, a sensor-independent physical quantity that enhances cross-sensor generalization while reducing the need for extensive per-
dataset tuning. Pairwise similarity combines a Bernoulli-KL Jaccard with exponential uncertainty gating and FFT-based height cosine similarity for rotation
alignment. Evaluated on four
datasets spanning four diverse
LiDAR types, PROBE achieves the highest accuracy among handcrafted descriptors in multi-session evaluation and competitive single-session performance relative to both handcrafted and supervised baselines. The source code and supplementary materials are available at https://sites.google.com/view/probe-pr.
Read the full abstract ▾
Computer Vision
QATMA: Quantization-Aware Training with Multimodal Alignment for Open-Vocabulary Object Detection
Jinyeong Park, Donghwa Kang, Seunghwan An et al. (+4 more)
Quantizing open-vocabulary object detection (OVOD) models reduces their
memory and computational costs, but extremely low-bit quantization severely degrades both cross-modal (region-text) and intra-modal (region-region)
alignments. This
multimodal degradation is a unique challenge that prior quantization methods for closed-vocabulary detectors fail to resolve. To overcome this, we propose Quantization-Aware Training with
Multimodal Alignment (QATMA), the first
multimodal-aware and architecture-agnostic QAT framework tailored for OVOD. QATMA integrates two key components: (i) Curriculum QAT, which partitions the detector by functional roles and progressively expands the quantization scope to suppress error accumulation and ensure stable optimization; and (ii) Text-anchored Pairwise Similarity
Distillation, which transfers both region-text and region-region
alignments from a full-precision teacher model via pairwise cosine similarities in the joint
embedding space. Experimental results on LVIS and COCO zero-shot
benchmarks demonstrate that QATMA significantly outperforms existing QAT baselines under extremely low-bit settings, achieving gains of up to 4.3 and 7.6 AP, respectively.
Read the full abstract ▾
All authors Jinyeong Park, Donghwa Kang, Seunghwan An, Insoo Kim, Brent ByungHoon Kang, Hyeongboo Baek, Jibum Kim
Computer Vision
Skeleton-to-Image Encoding: Enabling Skeleton Representation Learning via Vision-Pretrained Models
Siyuan Yang, Jun Liu, Hao Cheng et al. (+5 more)
Recent advances in large-scale pretrained
vision models have demonstrated impressive capabilities across a wide range of downstream tasks, including cross-modal and multi-modal scenarios. However, their direct application to 3D human skeleton data remains challenging due to fundamental differences in data format. Moreover, the scarcity of large-scale skeleton
datasets and the need to incorporate skeleton data into multi-modal action recognition without introducing additional model branches present significant research opportunities. To address these challenges, we introduce Skeleton-to-Image Encoding (S2I), a novel representation that transforms skeleton sequences into image-like data by partitioning and arranging joints based on body-part semantics and resizing to standardized image dimensions. This encoding enables, for the first time, the use of powerful
vision-pretrained models for self-supervised skeleton representation learning, effectively transferring rich visual-domain knowledge to skeleton analysis. While existing skeleton methods often design models tailored to specific, homogeneous skeleton formats, they overlook the structural heterogeneity that naturally arises from diverse data sources. In contrast, our S2I representation offers a unified image-like format that naturally accommodates heterogeneous skeleton data. Extensive experiments on NTU-60, NTU-120, and PKU-MMD demonstrate the effectiveness and generalizability of our method for self-supervised skeleton representation learning, including under challenging cross-format evaluation settings.
Read the full abstract ▾
All authors Siyuan Yang, Jun Liu, Hao Cheng, Chong Wang, Shijian Lu, Hedvig Kjellstrom, Weisi Lin, Alex C. Kot
Computer Vision
A novel Framework for Open-Vocabulary Multi-Object Recognition using CLIP
Wei Yu Chen and Ying Dai
To address the limitations of existing open-vocabulary object recognition methods, including high system complexity, substantial training costs, and limited generalization capability, this paper proposes a novel Open-Vocabulary Object Recognition (OVOR) framework based on a streamlined two-stage strategy: object
segmentation followed by recognition. The proposed framework eliminates the need for complex retraining procedures and labor-intensive annotation. After extracting object regions, object-level image
embeddings and category-level text
embeddings are generated using CLIP, enabling recognition over arbitrary vocabularies. To reduce dependence on CLIP and enhance encoding flexibility, we further introduce a CNN/MLP-based approach that extracts convolutional neural network (CNN) feature maps and employs a multilayer perceptron (MLP) to align visual features with text
embeddings. The resulting
embeddings are then concatenated for cross-modal matching. Finally, object recognition is performed through similarity matching between image and text
embeddings. Experiments on COCO, Pascal VOC, and ADE20K demonstrate that CLIP-based image encoding achieves the highest average AP, outperforming current state-of-the-art methods. Meanwhile, the results reveal the potential of CLIP-independent image encoding as a promising alternative for OVOR.
Read the full abstract ▾
Computer Vision
OVGGT: O(1) Constant-Cost Streaming Visual Geometry Transformer
Si-Yu Lu, Po-Ting Chen, Hui-Che Hsu et al. (+3 more)
Reconstructing 3D geometry from streaming
video requires continuous inference under bounded resources. Recent
geometric foundation models achieve impressive reconstruction quality through all-to-all
attention, yet their quadratic cost confines them to short, offline sequences. Causal-
attention variants such as StreamVGGT enable single-pass streaming but accumulate an ever-growing KV cache, exhausting GPU
memory within hundreds of frames and precluding the long-horizon deployment that motivates streaming inference in the first place. We present OVGGT, a training-free framework that bounds both
memory and compute to a fixed budget regardless of sequence length. Our approach combines Self-Selective Caching, which leverages FFN residual magnitudes to compress the KV cache while remaining fully compatible with FlashAttention, with Dynamic Anchor Protection, which shields coordinate-critical
tokens from eviction to suppress
geometric drift over extended trajectories. Extensive experiments on indoor, outdoor, and ultra-long sequence
benchmarks demonstrate that OVGGT processes arbitrarily long
videos within a constant VRAM envelope while achieving state-of-the-art 3D
geometric accuracy. Project page: https://vaisr.github.io/OVGGT/ Code: https://github.com/VAISR/OVGGT
Read the full abstract ▾
All authors Si-Yu Lu, Po-Ting Chen, Hui-Che Hsu, Sin-Ye Jhong, Wen-Huang Cheng, Yung-Yao Chen
Computer Vision
Unify the Views: View-Consistent Prototype Learning for Few-Shot Segmentation
Hongli Liu, Yu Wang and Shengjie Zhao
Few-shot
segmentation (FSS) has gained significant
attention for its ability to generalize to novel classes with limited supervision, yet remains challenged by structural misalignment and cross-view inconsistency under large appearance or viewpoint variations. This paper tackles these challenges by introducing VINE (View-Informed NEtwork), a unified framework that jointly models structural consistency and foreground discrimination to refine class-specific prototypes. Specifically, VINE introduces a spatial-view graph on backbone features, where the spatial graph captures local
geometric topology and the view graph connects features from different perspectives to propagate view-invariant structural semantics. To further alleviate foreground ambiguity, we derive a discriminative prior from the support-query feature discrepancy to capture category-specific contrast, which reweights SAM features by emphasizing salient regions and recalibrates backbone activations for improved structural focus. The foreground-enhanced SAM features and structurally enriched ResNet features are progressively integrated through masked cross-
attention, yielding class-consistent prototypes used as adaptive
prompts for the SAM decoder to generate accurate masks. Extensive experiments on multiple FSS
benchmarks validate the effectiveness and robustness of VINE, particularly under challenging scenarios with viewpoint shifts and complex structures. The code is available at https://github.com/HongliLiu1/VINE-main.
Read the full abstract ▾
Computer Vision
Energy-Driven Adaptive Visual Token Pruning for Efficient Vision-Language Models
Jialuo He and Huangxun Chen
Visual
token reduction is critical for accelerating
Vision-
Language Models (VLMs), since visual inputs are represented as
token sequences that introduce substantial computational overhead in the LLM backbone. However, most
pruning pipelines treat efficiency primarily as a
token selection problem and retain a fixed visual
token budget across inputs, overlooking the substantial variation in image information density. We propose E-AdaPrune, an energy driven adaptive
pruning framework that determines an image specific
token budget from the singular value spectrum of the visual feature
matrix and passes this budget to existing
token selectors. By preserving a certain proportion of spectral energy, our method allocates more
tokens to information dense scenes while assigning fewer
tokens to redundant scenes, without introducing additional learnable parameters. We evaluate E-AdaPrune across four VLM backbones, three
token selectors, and nine
benchmarks under matched average
token budgets. Results show that E-AdaPrune removes a substantial amount of redundant computation from simple cases and converts the saved budget into larger gains on information rich cases. Notably, on SQA$^\mathrm{I}$ with Qwen2.5-VL-3B, E-AdaPrune uses 35.8% fewer
tokens for simple cases with only a 0.52% relative performance decrease. The saved budget is redirected to hard cases, which receive 52.5% more
tokens and achieve a 1.94% relative performance improvement.
Read the full abstract ▾
Computer Vision
LucidNFT: LR-Anchored Multi-Reward Preference Optimization for Flow-Based Real-World Super-Resolution
Song Fei, Tian Ye, Sixiang Chen et al. (+3 more)
Generative real-world image super-resolution (Real-ISR) can synthesize visually convincing details from severely degraded low-resolution (LR) inputs, but
stochastic sampling can introduce semantic or structural
hallucinations unsupported by the LR evidence. Sampling multiple outputs per LR input exposes trade-offs between perceptual quality and faithfulness that reinforcement learning (RL) can optimize beyond paired supervision. Effective
alignment is limited by the lack of a degradation-robust LR-referenced faithfulness signal, the loss of per-objective reward contrasts when rewards are combined before normalization in
Diffusion Negative-aware FineTuning (DiffusionNFT), and limited coverage of real degradations. We propose LucidNFT, a multi-reward RL framework for flow-matching Real-ISR. LucidNFT introduces LucidConsistency, a degradation-invariant LR-referenced evaluator trained with content-consistent degradation pools and hard negatives created by inpainting to detect localized
hallucinations. Its decoupled reward normalization strategy normalizes each reward within each LR-conditioned rollout group before fusion, preserving per-objective contrasts across candidates. LucidLR provides a large-scale collection of real-world degraded images for RL fine-tuning. Experiments show that LucidNFT improves perceptual quality on strong flow-based Real-ISR baselines while generally maintaining LR-referenced consistency across diverse real-world scenarios. In a reward-independent human study, LucidNFT outputs were judged more faithful to the LR inputs in 72.5 percent of comparisons.
Read the full abstract ▾
All authors Song Fei, Tian Ye, Sixiang Chen, Zhaohu Xing, Jianyu Lai, Lei Zhu
Computer Vision
Adaptive Radial Projection on Fourier Magnitude Spectrum for Document Image Skew Estimation
Luan Pham, Phu Hao Hoang, Xuan Toan Mai et al. (+1 more)
Skew estimation is one of the vital tasks in document processing systems, especially for scanned document images, because its performance impacts subsequent steps directly. Over the years, an enormous number of researches focus on this challenging problem in the rise of digitization age. In this research, we first propose a novel skew estimation method that extracts the dominant skew angle of the given document image by applying an Adaptive Radial Projection on the 2D Discrete
Fourier Magnitude spectrum. Second, we introduce a high quality skew estimation
dataset DISE-2021 to assess the performance of different
estimators. Finally, we provide comprehensive analyses that focus on multiple improvement aspects of
Fourier-based methods. Our results show that the proposed method is robust, reliable, and outperforms all compared methods. The source code is available at https://github.com/phamquiluan/jdeskew.
Read the full abstract ▾
All authors Luan Pham, Phu Hao Hoang, Xuan Toan Mai, Tuan Anh Tran
Computer Vision
SLER-IR: Spherical Layer-wise Expert Routing for All-in-One Image Restoration
Peng Shurui, Xin Lin, Shi Luo et al. (+5 more)
Image restoration under diverse degradations remains challenging for unified all-in-one frameworks due to feature interference and insufficient expert specialization. We propose SLER-IR, a spherical layer-wise expert routing framework that dynamically activates specialized experts across network layers. To ensure reliable routing, we introduce a Spherical Uniform Degradation
Embedding with contrastive learning, which maps degradation representations onto a hypersphere to eliminate geometry bias in linear
embedding spaces. In addition, a Global-Local Granularity Fusion (GLGF) module integrates global semantics and local degradation cues to address spatially non-uniform degradations and the train-test granularity gap. Experiments on three-task and five-task
benchmarks demonstrate that SLER-IR achieves consistent improvements over state-of-the-art methods in both PSNR and SSIM. Code and models will be publicly released.
Read the full abstract ▾
All authors Peng Shurui, Xin Lin, Shi Luo, Jincen Ou, Dizhe Zhang, Lu Qi, Truong Nguyen, Chao Ren
Computer Vision
Facial Expression Recognition Using Residual Masking Network
Luan Pham, The Huynh Vu and Tuan Anh Tran
Automatic facial expression recognition (FER) has gained much
attention due to its applications in human-computer interaction. Among the approaches to improve FER tasks, this paper focuses on deep architecture with the
attention mechanism. We propose a novel Masking idea to boost the performance of CNN in facial expression task. It uses a
segmentation network to refine feature maps, enabling the network to focus on relevant information to make correct decisions. In experiments, we combine the ubiquitous Deep Residual Network and Unet-like architecture to produce a Residual Masking Network. The proposed method holds state-of-the-art (SOTA) accuracy on the well-known FER2013 and private VEMO
datasets. The source code is available at https://github.com/phamquiluan/ResidualMaskingNetwork.
Read the full abstract ▾
Computer Vision
OD-RASE: Ontology-Driven Risk Assessment and Safety Enhancement for Autonomous Driving
Kota Shimomura, Masaki Nambata, Atsuya Ishikawa et al. (+4 more)
Although autonomous driving systems demonstrate high perception performance, they still face limitations when handling rare situations or complex road structures. Such road infrastructures are designed for human drivers, safety improvements are typically introduced only after accidents occur. This reactive approach poses a significant challenge for autonomous systems, which require proactive risk mitigation. To address this issue, we propose OD-RASE, a framework for enhancing the safety of autonomous driving systems by detecting road structures that cause traffic accidents and connecting these findings to infrastructure development. First, we formalize an ontology based on specialized domain knowledge of road traffic systems. In parallel, we generate infrastructure improvement proposals using a large-scale visual
language model (LVLM) and use ontology-driven data filtering to enhance their reliability. This process automatically annotates improvement proposals on pre-accident road images, leading to the construction of a new
dataset. Furthermore, we introduce the Baseline approach (OD-RASE model), which leverages LVLM and a
diffusion model to produce both infrastructure improvement proposals and generated images of the improved road environment. Our experiments demonstrate that ontology-driven data filtering enables highly accurate prediction of accident-causing road structures and the corresponding improvement plans. We believe that this work contributes to the overall safety of traffic environments and marks an important step toward the broader adoption of autonomous driving systems.
Read the full abstract ▾
All authors Kota Shimomura, Masaki Nambata, Atsuya Ishikawa, Ryota Mimura, Takayuki Kawabuchi, Takayoshi Yamashita, Koki Inoue
Computer Vision
FTSplat: Feed-forward Triangle Splatting Network
Xiong Jinlin, Li Can, Shen Jiawei et al. (+3 more)
High-fidelity three-dimensional (3D) reconstruction is essential for robotics and simulation. While Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) achieve impressive rendering quality, their reliance on time-consuming per-scene optimization limits real-time deployment. Emerging feed-forward Gaussian splatting methods improve efficiency but often lack explicit,
manifold geometry required for direct simulation. To address these limitations, we propose a feed-forward framework for triangle primitive generation that directly predicts continuous triangle surfaces from calibrated multi-view images. Our method produces simulation-ready models in a single forward pass, obviating the need for per-scene optimization or post-processing. We introduce a pixel-aligned triangle generation module and incorporate relative 3D point cloud supervision to enhance
geometric learning stability and consistency. Experiments demonstrate that our method achieves efficient reconstruction while maintaining seamless compatibility with standard graphics and robotic simulators.
Read the full abstract ▾
All authors Xiong Jinlin, Li Can, Shen Jiawei, Qi Zhigang, Sun Lei, Zhao Dongyang
Computer Vision
Beyond Static Frames: Temporal Aggregate-and-Restore Vision Transformer for Human Pose Estimation
Hongwei Fang, Jiahang Cai, Xun Wang et al. (+1 more)
Vision Transformers (ViTs) have recently achieved state-of-the-art performance in 2D human pose estimation due to their strong global modeling capability. However, existing ViT-based pose
estimators are designed for static images and process each frame independently, thereby ignoring the temporal coherence that exists in
video sequences. This limitation often results in unstable predictions, especially in challenging scenes involving motion blur, occlusion, or defocus. In this paper, we propose TAR-ViTPose, a novel Temporal Aggregate-and-Restore
Vision Transformer tailored for
video-based 2D human pose estimation. TAR-ViTPose enhances static ViT representations by aggregating temporal cues across frames in a plug-and-play manner, leading to more robust and accurate pose estimation. To effectively aggregate joint-specific features that are temporally aligned across frames, we introduce a joint-centric temporal aggregation (JTA) that assigns each joint a learnable query
token to selectively attend to its corresponding regions from neighboring frames. Furthermore, we develop a global restoring
attention (GRA) to restore the aggregated temporal features back into the
token sequence of the current frame, enriching its pose representation while fully preserving global context for precise keypoint localization. Extensive experiments demonstrate that TAR-ViTPose substantially improves upon the single-frame baseline ViTPose, achieving a +2.3 mAP gain on the PoseTrack2017
benchmark. Moreover, our approach outperforms existing state-of-the-art
video-based methods, while also achieving a noticeably higher runtime frame rate in real-world applications. Project page: https://github.com/zgspose/TARViTPose.
Read the full abstract ▾
All authors Hongwei Fang, Jiahang Cai, Xun Wang, Wenwu Yang
Computer Vision
Towards Driver Behavior Understanding: Weakly-Supervised Risk Perception in Driving Scenes
Nakul Agarwal, Yi-Ting Chen and Behzad Dariush
Achieving zero-collision mobility remains a key objective for intelligent vehicle systems, which requires understanding driver risk perception-a complex cognitive process shaped by voluntary response of the driver to external stimuli and the attentiveness of surrounding road users towards the ego-vehicle. To support progress in this area, we introduce RAID (Risk Assessment In Driving scenes)-a large-scale
dataset specifically curated for research on driver risk perception and contextual risk assessment. RAID comprises 4,691 annotated
video clips, covering diverse traffic scenarios with labels for driver's intended maneuver, road
topology, risk situations (e.g., crossing pedestrians), driver responses, and pedestrian attentiveness. Leveraging RAID, we propose a weakly supervised risk object identification framework that models the relationship between driver's intended maneuver and responses to identify potential risk sources. Additionally, we analyze the role of pedestrian
attention in estimating risk and demonstrate the value of the proposed
dataset. Experimental evaluations demonstrate that our method achieves 20.6% and 23.1% performance gains over prior state-of-the-art approaches on the RAID and HDDS
datasets, respectively.
Read the full abstract ▾
Computer Vision
RAC: Rectified Flow Auto Coder
Sen Fang, Yalin Feng, Yanxin Zhang et al. (+6 more)
In this paper, we propose a Rectified Flow Auto Coder (RAC) inspired by Rectified Flow to replace the traditional VAE: 1. It achieves multi-step decoding by applying the decoder to flow timesteps. Its decoding path is straight and correctable, enabling step-by-step refinement. 2. The model inherently supports bidirectional inference, where the decoder serves as the encoder through time reversal (hence Coder rather than encoder or decoder), reducing parameter count by nearly 41%. 3. This generative decoding method improves generation quality since the model can correct latent variables along the path, partially addressing the reconstruction–generation gap. Experiments show that RAC achieves a Pareto improvement over SOTA VAEs, where even a 10$\times$ parameter-reduced decoder exceeds full-scale VAE performance in both reconstruction and generation quality, validating the effectiveness of our approach.
Read the full abstract ▾
All authors Sen Fang, Yalin Feng, Yanxin Zhang, Yihao Quan, Juyi Lin, Yifan Shen, Ziwei Dong, Sisong Bei, Dimitris N. Metaxas
Computer Vision
BlackMirror: Black-Box Backdoor Detection for Text-to-Image Models via Instruction-Response Deviation
Feiran Li, Qianqian Xu, Shilong Bao et al. (+4 more)
This paper investigates the challenging task of detecting backdoored text-to-image models under black-box settings and introduces a novel detection framework BlackMirror. Existing approaches typically rely on analyzing image-level similarity, under the assumption that
backdoor-triggered generations exhibit strong consistency across samples. However, they struggle to generalize to recently emerging
backdoor attacks, where backdoored generations can appear visually diverse. BlackMirror is motivated by an observation: across
backdoor attacks, {only partial semantic patterns within the generated image are steadily manipulated, while the rest of the content remains diverse or benign. Accordingly, BlackMirror consists of two components: MirrorMatch, which aligns visual patterns with the corresponding instructions to detect semantic deviations; and MirrorVerify, which evaluates the stability of these deviations across varied
prompts to distinguish true
backdoor behavior from benign responses. BlackMirror is a general, training-free framework that can be deployed as a plug-and-play module in Model-as-a-Service (MaaS) applications. Comprehensive experiments demonstrate that BlackMirror achieves accurate detection across a wide range of attacks. Code is available at https://github.com/Ferry-Li/BlackMirror.
Read the full abstract ▾
All authors Feiran Li, Qianqian Xu, Shilong Bao, Zhiyong Yang, Xilin Zhao, Xiaochun Cao, Qingming Huang
Computer Vision
CORE-Seg: Reasoning-Driven Segmentation for Complex Lesions via Reinforcement Learning
Yuxin Xie, Yuming Chen, Yishan Yang et al. (+5 more)
Medical image
segmentation is undergoing a paradigm shift from conventional visual pattern matching to cognitive
reasoning analysis. Although
Multimodal Large
Language Models (MLLMs) have shown promise in integrating linguistic and visual knowledge, significant gaps remain: existing general MLLMs possess broad common sense but lack the specialized visual
reasoning required for complex lesions, whereas traditional
segmentation models excel at pixel-level
segmentation but lack logical interpretability. In this paper, we introduce ComLesion-14K, the first diverse Chain-of-Thought (CoT)
benchmark for
reasoning-driven complex lesion
segmentation. To accomplish this task, we propose CORE-Seg, an end-to-end framework integrating
reasoning with
segmentation through a Semantic-Guided
Prompt Adapter. We design a progressive training strategy from SFT to GRPO, equipped with an adaptive dual-granularity reward mechanism to mitigate reward sparsity. Our Method achieves state-of-the-art results with a mean Dice of 37.06% (14.89% higher than the second-best baseline), while reducing the failure rate to 18.42%. Project Page: https://xyxl024.github.io/CORE-Seg.github.io/
Read the full abstract ▾
All authors Yuxin Xie, Yuming Chen, Yishan Yang, Yi Zhou, Tao Zhou, Zhen Zhao, Jiacheng Liu, Huazhu Fu
Computer Vision
Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image
Zidian Qiu and Ancong Wu
Current compositional image-to-3D scene generation approaches construct 3D scenes by time-consuming iterative layout optimization or inflexible joint object-layout generation. Moreover, most methods rely on limited field-of-view perspective images, hindering the creation of complete 360-degree environments. To address these limitations, we design Pano3DComposer, an efficient feed-forward framework for panoramic images. To decouple object generation from layout estimation, we propose a plug-and-play Object-World Transformation Predictor. This module converts the 3D objects generated by off-the-shelf image-to-3D models from local to world coordinates. To achieve this, we adapt the VGGT architecture to
Alignment-VGGT by using target object crop, multi-view object renderings and camera parameters to predict the transformation. The predictor is trained using pseudo-
geometric supervision to address the shape discrepancy between generated and ground-truth objects. For input images from unseen domains, we further introduce a Coarse-to-Fine (C2F)
alignment mechanism for Pano3DComposer that iteratively refines
geometric consistency with feedback of scene rendering. Our method achieves superior
geometric accuracy for image/text-to-3D tasks on synthetic and real-world
datasets. It can generate a high-fidelity 3D scene in approximately 20 seconds on an RTX 4090 GPU. Project page: https://qiuzidian.github.io/pano3dcomposer-page/.
Read the full abstract ▾
Computer Vision
Beyond Geometry: Artistic Disparity Synthesis for Immersive 2D-to-3D
Ping Chen, Zezhou Chen, Xingpeng Zhang et al. (+8 more)
Current 2D-to-3D conversion methods achieve
geometric accuracy but are artistically deficient, failing to replicate the immersive and emotionally resonant experience of professional 3D cinema. This is because
geometric reconstruction paradigms mistake deliberate artistic intent, such as strategic zero-plane shifts for pop-out effects and local depth sculpting, for data noise or ambiguity. This paper argues for a new paradigm: Artistic Disparity Synthesis, shifting the goal from physically accurate disparity estimation to artistically coherent disparity synthesis. We propose Art3D, a preliminary framework exploring this paradigm. Art3D uses a dual-path architecture to decouple global depth parameters (macro-intent) from local artistic effects (visual brushstrokes) and learns from professional 3D film data via indirect supervision. We also introduce a preliminary evaluation method to quantify cinematic
alignment. Experiments show our approach demonstrates potential in replicating key local out-of-screen effects and aligning with the global depth styles of cinematic 3D content, laying the groundwork for a new class of artistically-driven conversion tools.
Read the full abstract ▾
All authors Ping Chen, Zezhou Chen, Xingpeng Zhang, Yanlin Qian, Huan Hu, Xiang Liu, Zipeng Wang, Xin Wang, Zhaoxiang Liu, Kai Wang, Shiguo Lian
Computer Vision
CollabOD: Collaborative Multi-Backbone with Cross-scale Vision for UAV Small Object Detection
Xuecheng Bai, Yuxiang Wang, Chuanzhi Xu et al. (+3 more)
Small object detection in unmanned aerial vehicle (
UAV) imagery is challenging because high-altitude viewpoints produce severe scale variation, weak structural cues, and tight computational budgets. Existing lightweight detectors usually fuse multi-scale features after downsampling, where boundary and texture details have already been attenuated and heterogeneous feature streams may be spatially misaligned. To address these issues, we propose CollabOD, a collaborative detection framework that preserves structural details, aligns cross-path features before fusion, and keeps the detection head lightweight at inference time. CollabOD combines a Dual-Path Fusion Stem, a Dense Aggregation Block, a Bilateral Reweighting Module, and a Unified Detail-Aware Head to strengthen localization-oriented representation while limiting extra computation. On VisDrone, CollabOD obtains 52.4 AP50, 30.8 AP75, and 29.9 AP50:95 with 65.5 GFLOPs; on UAVDT it reaches 31.2 AP50 and 17.4 AP50:95; and on AI-TOD it reaches 45.4 AP50 and 20.0 AP50:95 at 137 FPS. The code is available at: https://github.com/Bai-Xuecheng/CollabOD.
Read the full abstract ▾
All authors Xuecheng Bai, Yuxiang Wang, Chuanzhi Xu, Kang Han, Jun Guo, Pengfei Ye
Computer Vision
Mitigating Bias in Concept Bottleneck Models for Fair and Interpretable Image Classification
Schrasing Tong, Antoine Salaun, Vincent Yuan et al. (+2 more)
Ensuring
fairness in image classification prevents models from perpetuating and amplifying bias. Concept bottleneck models (CBMs) map images to high-level, human-interpretable concepts before making predictions via a sparse, one-layer classifier. This structure enhances interpretability and, in theory, supports
fairness by masking sensitive attribute proxies such as facial features. However, CBM concepts have been known to leak information unrelated to concept semantics and early results reveal only marginal reductions in gender bias on
datasets like ImSitu. We propose three bias mitigation techniques to improve
fairness in CBMs: 1. Decreasing information leakage using a top-k concept filter, 2. Removing biased concepts, and 3.
Adversarial debiasing. Our results outperform prior work in terms of
fairness-performance tradeoffs, indicating that our debiased CBM provides a significant step towards fair and interpretable image classification.
Read the full abstract ▾
All authors Schrasing Tong, Antoine Salaun, Vincent Yuan, Annabel Adeyeri, Lalana Kagal
Computer Vision
InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generation
Yuxin Qin, Ke Cao, Haowei Liu et al. (+13 more)
E-commerce product poster generation aims to automatically synthesize a single image that effectively conveys product information by presenting a subject, text, and a designed style. Recent
diffusion models with fine-grained and efficient controllability have advanced product poster synthesis, yet they typically rely on multi-stage pipelines, and simultaneous control over subject, text, and style remains underexplored. Such naive multi-stage pipelines also show three issues: poor subject fidelity, inaccurate text, and inconsistent style. To address these issues, we propose InnoAds-Composer, a single-stage framework that enables efficient tri-conditional control
tokens over subject, glyph, and style. To alleviate the quadratic overhead introduced by naive tri-conditional
token concatenation, we perform importance analysis over layers and timesteps and route each condition only to the most responsive positions, thereby shortening the active
token sequence. Besides, to improve the accuracy of Chinese text rendering, we design a Text Feature Enhancement Module (TFEM) that integrates features from both glyph images and glyph crops. To support training and evaluation, we also construct a high-quality e-commerce product poster
dataset and
benchmark, which is the first
dataset that jointly contains subject, text, and style conditions. Extensive experiments demonstrate that InnoAds-Composer significantly outperforms existing product poster methods without obviously increasing inference latency.
Read the full abstract ▾
All authors Yuxin Qin, Ke Cao, Haowei Liu, Ao Ma, Fengheng Li, Honghe Zhu, Zheng Zhang, Run Ling, Wei Feng, Xuanhua He, Zhanjie Zhang, Zhen Guo, Haoyi Bian, Jingjing Lv, Junjie Shen, Ching Law
Computer Vision
PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstruction
Xiang Zhang, Sohyun Yoo, Hongrui Wu et al. (+3 more)
We introduce PixARMesh, a method to autoregressively reconstruct complete 3D indoor scene meshes directly from a single RGB image. Unlike prior methods that rely on implicit signed distance fields and post-hoc layout optimization, PixARMesh jointly predicts object layout and geometry within a unified model, producing coherent and artist-ready meshes in a single forward pass. Building on recent advances in mesh generative models, we augment a point-cloud encoder with pixel-aligned image features and global scene context via cross-
attention, enabling accurate spatial
reasoning from a single image. Scenes are generated autoregressively from a unified
token stream containing context, pose, and mesh, yielding compact meshes with high-fidelity geometry. Experiments on synthetic and real-world
datasets show that PixARMesh achieves state-of-the-art reconstruction quality while producing lightweight, high-quality meshes ready for downstream applications.
Read the full abstract ▾
All authors Xiang Zhang, Sohyun Yoo, Hongrui Wu, Chuan Li, Jianwen Xie, Zhuowen Tu
Computer Vision
CylinderSplat: 3D Gaussian Splatting with Cylindrical Triplanes for Panoramic Novel View Synthesis
Qiwei Wang, Xianghui Ze, Jingyi Yu et al. (+1 more)
Feed-forward 3D Gaussian Splatting (3DGS) has shown great promise for real-time novel view synthesis, but its application to panoramic imagery remains challenging. Existing methods often rely on multi-view cost volumes for
geometric refinement, which struggle to resolve occlusions in sparse-view scenarios. Furthermore, standard volumetric representations like Cartesian Triplanes are poor in capturing the inherent geometry of $360^\circ$ scenes, leading to distortion and aliasing. In this work, we introduce CylinderSplat, a feed-forward framework for panoramic 3DGS that addresses these limitations. The core of our method is a new {cylindrical Triplane} representation, which is better aligned with panoramic data and real-world structures adhering to the Manhattan-world assumption. We use a dual-branch architecture: a pixel-based branch reconstructs well-observed regions, while a volume-based branch leverages the cylindrical Triplane to complete occluded or sparsely-viewed areas. Our framework is designed to flexibly handle a variable number of input views, from single to multiple panoramas. Extensive experiments demonstrate that CylinderSplat achieves state-of-the-art results in both single-view and multi-view panoramic novel view synthesis, outperforming previous methods in both reconstruction quality and
geometric accuracy.
Read the full abstract ▾
All authors Qiwei Wang, Xianghui Ze, Jingyi Yu, Yujiao Shi
Computer Vision
Systematic Evaluation of Novel View Synthesis for Video Place Recognition
Muhammad Zawad Mahmud, Samiha Islam and Damian Lyons
The generation of synthetic novel views has the potential to positively impact
robot navigation in several ways. In image-based
navigation, a novel overhead view generated from a scene taken by a ground
robot could be used to guide an aerial
robot to that location. In
Video Place Recognition (VPR), novel views of ground locations from the air can be added that enable a
UAV to identify places seen by the ground
robot, and similarly, overhead views can be used to generate novel ground views. This paper presents a systematic evaluation of synthetic novel views in VPR using five public VPR image databases and seven typical image similarity methods. We show that for small synthetic additions, novel views improve VPR recognition statistics. We find that for larger additions, the magnitude of viewpoint change is less important than the number of views added and the type of imagery in the
dataset.
Read the full abstract ▾
Computer Vision
Shifting Adaptation from Weight Space to Memory Space: A Memory-Augmented Agent for Medical Image Segmentation
Bowen Chen, Qiaohui Gao, Shaowen Wan et al. (+6 more)
Medical image
segmentation is fundamental to clinical workflows, yet models trained on a single
dataset often fail to generalize across institutions, scanners, or patient populations. While
vision foundation models have shown great promise in addressing this challenge, their deployment typically requires task-specific fine-tuning, which introduces substantial communication overhead in
federated learning and prevents continuous knowledge
evolution during deployment. In this work, we propose a
memory-augmented
segmentation agent (MemSeg-
Agent) that shifts adaptation from weight space to
memory space, enabling few-shot learning,
federated supervised learning, and test-time adaptation within a unified architecture. MemSeg-
Agent conditions a fixed backbone with lightweight static, few-shot, and test-time working
memories, which are dynamically composed by an agentic controller. In
federated settings, we update compact
memory units instead of model parameters, substantially reducing communication overhead. Experiments on four public
datasets demonstrate strong performance and robustness to domain shift: Static
memory alone matches or surpasses strong supervised baselines with high parameter efficiency, and test-time working
memory further improves in-domain and cross-domain performance without fine-tuning. Overall, MemSeg-
Agent introduces a new paradigm for scalable and adaptive medical image
segmentation in the era of agentic AI.
Read the full abstract ▾
All authors Bowen Chen, Qiaohui Gao, Shaowen Wan, Ruiyu Yan, Shanhui Sun, Wei Liu, Xiang Li, Tianming Liu, Lin Zhao
Computer Vision
PatchCue: Enhancing Vision-Language Model Reasoning with Patch-Based Visual Cues
Yukun Qi, Pei Fu, Hang Li et al. (+5 more)
Vision-
Language Models (VLMs) have achieved remarkable progress on a wide range of challenging
multimodal understanding and
reasoning tasks. However, existing
reasoning paradigms, such as the classical Chain-of-Thought (CoT), rely solely on textual information and often underutilize important visual cues. While prior work has incorporated pixel-level visual cues, these representations require precise spatial localization, introducing additional learning complexity. To address this, we propose PatchCue, a novel patch-based visual cue paradigm designed to significantly enhance the visual
reasoning capabilities of VLMs. By partitioning images into patches and representing cues at the patch level, PatchCue aligns better with human perceptual habits and leverages the patch-tokenized input of modern VLMs. We train VLMs using a two-stage approach: cold-start supervised fine-tuning to output patch-level cues, followed by reinforcement learning with a process-supervised cue reward that guides intermediate visual
reasoning steps. Extensive experiments on multiple VLMs and diverse
benchmarks, including general visual question answering, complex
reasoning, and document understanding, demonstrate that PatchCue consistently improves overall model performance. Our results show that patch-level cues outperform both pixel-level bounding boxes and point-based cues, providing a more effective and cognitively aligned visual
reasoning paradigm.
Read the full abstract ▾
All authors Yukun Qi, Pei Fu, Hang Li, Yuhan Liu, Chao Jiang, Bin Qin, Zhenbo Luo, Jian Luan
Computer Vision
TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis
Sijing Li, Zhongwei Qiu, Jiang Liu et al. (+22 more)
Accurate tumor analysis is central to clinical radiology and precision oncology, where early detection, reliable lesion characterization, and pathology-level risk assessment guide diagnosis and treatment planning. Chain-of-Thought (CoT)
reasoning is particularly important in this setting because it enables step-by-step interpretation from imaging findings to clinical impressions and pathology conclusions, improving traceability and reducing diagnostic errors. Here, we target the clinical tumor analysis task and build a large-scale
benchmark that operationalizes a
multimodal reasoning pipeline, spanning findings, impressions, and pathology predictions. We curate TumorCoT, a large-scale
dataset of 1.5M CoT-labeled VQA instructions paired with 3D CT scans, with step-aligned rationales and cross-modal
alignments along the trajectory from findings to impression to pathology, enabling evaluation of both answer accuracy and
reasoning consistency. We further propose TumorChain, a
multimodal interleaved
reasoning framework that tightly couples 3D imaging encoders, clinical text understanding, and organ-level
vision-
language alignment. Through cross-modal
alignment and iterative interleaved causal
reasoning, TumorChain grounds visual evidence, aggregates conclusions, and issues pathology predictions after multiple rounds of self-refinement, improving traceability and reducing
hallucination risk. Experiments show consistent improvements over strong baselines in lesion detection, impression generation, and pathology classification, and demonstrate strong generalization on the DeepTumorVQA
benchmark. These results highlight the potential of
multimodal reasoning for reliable and interpretable tumor analysis in clinical practice. Detailed information about our project can be found on our project homepage at https://github.com/ZJU4HealthCare/TumorChain.
Read the full abstract ▾
All authors Sijing Li, Zhongwei Qiu, Jiang Liu, Wenqiao Zhang, Tianwei Lin, Yihan Xie, Jianxiang An, Boxiang Yun, Chenglin Yang, Jun Xiao, Guangyu Guo, Jiawen Yao, Wei Liu, Yuan Gao, Ke Yan, Weiwei Cao, Zhilin Zheng, Tony C. W. Mok, Kai Cao, Yu Shi, Jiuyu Zhang, Jian Zhou, Beng Chin Ooi, Yingda Xia, Ling Zhang
Computer Vision · Cross-listed from cs.AI
Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery
Lin Fan, Pengyu Dai, Zhipeng Deng et al. (+4 more)
Clinical image interpretation is inherently multi-step and tool-centric: clinicians iteratively combine visual evidence with patient context, quantify findings, and refine their decisions through a sequence of specialized procedures. While LLM-based
agents promise to orchestrate such heterogeneous medical tools, existing systems treat tool sets and invocation strategies as static after deployment. This design is brittle under real-world domain shifts, across tasks, and evolving diagnostic requirements, where predefined tool chains frequently degrade and demand costly manual re-design. We propose MACRO, a self-evolving, experience-augmented medical
agent that shifts from static tool composition to experience-driven tool discovery. From verified execution trajectories, the
agent autonomously identifies recurring effective multi-step tool sequences, synthesizes them into reusable composite tools, and registers these as new high-level primitives that continuously expand its behavioral repertoire. A lightweight image-feature
memory grounds tool selection in a visual-clinical context, while a GRPO-like training loop reinforces reliable invocation of discovered composites, enabling closed-loop self-improvement with minimal supervision. Extensive experiments across diverse medical imaging
datasets and tasks demonstrate that autonomous composite tool discovery consistently improves multi-step orchestration accuracy and cross-domain generalization over strong baselines and recent state-of-the-art agentic methods, bridging the gap between brittle static tool use and adaptive, context-aware clinical AI assistance. Code will be available upon acceptance.
Read the full abstract ▾
All authors Lin Fan, Pengyu Dai, Zhipeng Deng, Haolin Wang, Xun Gong, Yefeng Zheng, Yafei Ou