Computer Vision
AeroDeshadow: Physics-Guided Shadow Synthesis and Penumbra-Aware Deshadowing for Aerospace Imagery
Wei Lu, Zi-Yang Bo, Fei-Fei Sang et al. (+3 more)
Shadows are prevalent in high-resolution aerospace imagery (ASI). They often cause spectral distortion and information loss, which degrade downstream interpretation tasks. While deep learning methods have advanced natural-image shadow removal, their direct application to ASI faces two primary challenges. First, strictly paired training data are severely lacking. Second, homogeneous shadow assumptions fail to handle the broad penumbra transition zones inherent in aerospace scenes. To address these issues, we propose AeroDeshadow, a unified two-stage framework integrating physics-guided shadow synthesis and penumbra-aware restoration. In the first stage, a Physics-aware Degradation Shadow Synthesis Network (PDSS-Net) explicitly models illumination decay and spatial attenuation. This process constructs AeroDS-Syn, a large-scale paired
dataset featuring soft boundary transitions. Constrained by this physical formulation, a Penumbra-aware Cascaded DeShadowing Network (PCDS-Net) then decouples the input into umbra and penumbra components. By restoring these regions progressively, PCDS-Net alleviates boundary artifacts and over-correction. Trained solely on the synthetic AeroDS-Syn, the network generalizes to real-world ASI without requiring paired real annotations. Experimental results indicate that AeroDeshadow achieves state-of-the-art quantitative accuracy and visual fidelity across synthetic and real-world
datasets. The
datasets and code will be made publicly available at: https://github.com/AeroVILab-AHU/AeroDeshadow.
Read the full abstract ▾
All authors Wei Lu, Zi-Yang Bo, Fei-Fei Sang, Yi Liu, Xue Yang, Si-Bao Chen
Computer Vision
PolarMAE: Efficient Fetal Ultrasound Pre-training via Semantic Screening and Polar-Guided Masking
Meng Lv, Yapeng Li, Hang Su et al. (+2 more)
Intelligent fetal ultrasound (US) interpretation is crucial for prenatal diagnosis, but high annotation costs and operator-induced variance make unsupervised pre-training a highly promising paradigm. However, existing pre-training methods largely ignore US-specific characteristics – severe data redundancy, fan-shaped locality, and polar coordinate beamforming – limiting their effectiveness in downstream tasks. To address this, we propose PolarMAE, a novel and efficient pre-training framework tailored for US images. Specifically, to mitigate continuous scanning redundancy, we introduce a Progressive Visual-Semantic Screening (PVSS) that adaptively extracts high-value samples, significantly boosting pre-training efficiency. Furthermore, we design an Acoustic-Bounded Region Constraint (ABRC) to accommodate US locality, forcing the model to focus strictly on valid acoustic regions rather than invalid dark backgrounds. Finally, leveraging the beamforming prior and local details, we propose a Polar-Texture Collaborative Masking (PTCM), enabling the model to capture underlying radial imaging patterns and critical tissue structures. Extensive experiments across diverse
datasets and downstream interpretation tasks demonstrate that our method achieves state-of-the-art performance with strong pre-training scalability and efficiency.
Read the full abstract ▾
All authors Meng Lv, Yapeng Li, Hang Su, Juhua Liu, Bo Du
Computer Vision
CLOTH-HUGS: Cloth Aware Human Gaussian Splatting
Sadia Mubashshira, Nazanin Amini and Kevin Desai
We present Cloth-HUGS, a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing. Unlike prior methods that absorb clothing into a single body representation and struggle with loose garments and complex deformations, Cloth-HUGS represents the performer using separate Gaussian layers for body and cloth within a shared canonical space. The canonical volume jointly encodes body, cloth, and scene primitives and is deformed through SMPL-driven articulation with learned linear blend skinning weights. To improve cloth realism, we initialize cloth Gaussians from mesh
topology and apply physics-inspired constraints, including simulation-consistency, ARAP regularization, and mask supervision. We further introduce a depth-aware multi-pass rendering strategy for robust body-cloth-scene compositing, enabling real-time rendering at over 60 FPS. Experiments on multiple
benchmarks show that Cloth-HUGS improves perceptual quality and
geometric fidelity over state-of-the-art baselines, reducing LPIPS by up to 28% while producing temporally coherent cloth dynamics.
Read the full abstract ▾
Computer Vision
UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs
Lifan Jiang, Tianrun Wu, Yuhang Pei et al. (+3 more)
The evaluation of visual editing models remains fragmented across methods and modalities. Existing
benchmarks are often tailored to specific paradigms, making fair cross-paradigm comparisons difficult, while
video editing lacks reliable evaluation
benchmarks. Furthermore, common automatic metrics often misalign with human preference, yet directly deploying large
multimodal models (MLLMs) as evaluators incurs prohibitive computational and financial costs. We present UniEditBench, a unified
benchmark for image and
video editing that supports reconstruction-based and instruction-driven methods under a shared protocol. UniEditBench includes a structured taxonomy of nine image operations (Add, Remove, Replace, Change, Stroke-based, Extract, Adjust, Count, Reorder) and eight
video operations, with coverage of challenging compositional tasks such as counting and spatial reordering. To enable scalable evaluation, we distill a high-capacity MLLM judge (Qwen3-VL-235B-A22B Instruct) into lightweight 4B/8B evaluators that provide multi-dimensional scoring over structural fidelity, text
alignment, background consistency, naturalness, and temporal-spatial consistency (for
videos). Experiments show that the distilled evaluators maintain strong agreement with human judgments and substantially reduce deployment cost relative to the teacher model. UniEditBench provides a practical and reproducible protocol for benchmarking modern visual editing methods. Our
benchmark and the associated reward models are publicly available at https://github.com/wesar1/UniEditBench.
Read the full abstract ▾
All authors Lifan Jiang, Tianrun Wu, Yuhang Pei, Chenyang Wang, Boxi Wu, Deng Cai
Computer Vision
Splats in Splats++: Robust and Generalizable 3D Gaussian Splatting Steganography
Yijia Guo, Wenkai Huang, Tong Hu et al. (+9 more)
3D Gaussian Splatting (3DGS) has recently redefined the paradigm of 3D reconstruction, striking an unprecedented balance between visual fidelity and computational efficiency. As its adoption proliferates, safeguarding the copyright of explicit 3DGS assets has become paramount. However, existing invisible message
embedding frameworks struggle to reconcile secure and high-capacity data
embedding with intrinsic asset utility, often disrupting the native rendering pipeline or exhibiting vulnerability to structural perturbations. In this work, we present \textbf{Splats in Splats++}, a unified and pipeline-agnostic steganography framework that seamlessly embeds high-capacity 3D/4D content directly within the native 3DGS representation. Grounded in a principled analysis of the frequency distribution of Spherical Harmonics (SH), we propose an importance-graded SH coefficient encryption scheme that achieves imperceptible
embedding without compromising the original expressive power. To fundamentally resolve the
geometric ambiguities that lead to message leakage, we introduce a Hash-Grid Guided Opacity Mapping mechanism. Coupled with a novel Gradient-Gated Opacity Consistency Loss, our formulation enforces a stringent spatial-attribute coupling between the original and hidden scenes, effectively projecting the discrete attribute mapping into a continuous, attack-resilient latent
manifold. Extensive experiments demonstrate that our method substantially outperforms existing approaches, achieving up to 6.28 db higher message fidelity, \textbf{3$\times$} faster rendering, and exceptional robustness against aggressive 3D-targeted structural attacks (e.g., GSPure). Furthermore, our framework exhibits remarkable versatility, generalizing seamlessly to 2D image
embedding, 4D dynamic scene steganography, and diverse downstream tasks.
Read the full abstract ▾
All authors Yijia Guo, Wenkai Huang, Tong Hu, Gaolei Li, Yang Li, Yuxin Hong, Liwen Hu, Xitong Ling, Jianhua Li, Shengbo Chen, Tiejun Huang, Lei Ma
Computer Vision
AHS: Adaptive Head Synthesis via Synthetic Data Augmentations
Taewoong Kang, Hyojin Jang, Sohyun Jeong et al. (+4 more)
Recent digital media advancements have created increasing demands for sophisticated portrait manipulation techniques, particularly head swapping, where one's head is seamlessly integrated with another's body. However, current approaches predominantly rely on face-centered cropped data with limited view angles, significantly restricting their real-world applicability. They struggle with diverse head expressions, varying hairstyles, and natural blending beyond facial regions. To address these limitations, we propose Adaptive Head Synthesis (AHS), which effectively handles full upper-body images with varied head poses and expressions. AHS incorporates a novel head reenacted synthetic data augmentation strategy to overcome self-supervised training constraints, enhancing generalization across diverse facial expressions and orientations without requiring paired training data. Comprehensive experiments demonstrate that AHS achieves superior performance in challenging real-world scenarios, producing visually coherent results that preserve identity and expression fidelity across various head orientations and hairstyles. Notably, AHS shows exceptional robustness in maintaining facial identity while drastic expression changes and faithfully preserving accessories while significant head pose variations.
Read the full abstract ▾
All authors Taewoong Kang, Hyojin Jang, Sohyun Jeong, Seunggi Moon, Gihwi Kim, Hoon Jin Jung, Jaegul choo
Computer Vision
Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection
Irem Ulku, Erdem Akagündüz and Ömer Özgür Tanr{\i}över
Multimodal remote sensing data provide complementary information for semantic
segmentation, but in real-world deployments, some modalities may be unavailable due to sensor failures, acquisition issues, or challenging atmospheric conditions. Existing
multimodal segmentation models typically address missing modalities by learning a shared representation across inputs. However, this approach can introduce a trade-off by compromising modality-specific complementary information and reducing performance when all modalities are available. In this paper, we tackle this limitation with CBC-SLP, a
multimodal semantic
segmentation model designed to preserve both modality-invariant and modality-specific information. Inspired by the theoretical results on modality
alignment, which state that perfectly aligned
multimodal representations can lead to sub-optimal performance in downstream prediction tasks, we propose a novel structured latent projection approach as an architectural inductive bias. Rather than enforcing this strategy through a loss term, we incorporate it directly into the architecture. In particular, to use the complementary information effectively while maintaining robustness under random modality dropout, we structure the latent representations into shared and modality-specific components and adaptively transfer them to the decoder according to the random modality availability mask. Extensive experiments on three
multimodal remote sensing image sets demonstrate that CBC-SLP consistently outperforms state-of-the-art
multimodal models across full and missing modality scenarios. Besides, we empirically demonstrate that the proposed strategy can recover the complementary information that may not be preserved in a shared representation. The code is available at https://github.com/iremulku/Multispectral-Semantic-
Segmentation-via-Structured-Latent-Projection-CBC-SLP-.
Read the full abstract ▾
Computer Vision
Learning to Look before Learning to Like: Incorporating Human Visual Cognition into Aesthetic Quality Assessment
Liwen Yu, Chi Liu, Xiaotong Han et al. (+3 more)
Automated Aesthetic Quality Assessment (AQA) treats images primarily as static pixel vectors, aligning predictions with human-rating scores largely through semantic perception. However, this paradigm diverges from human aesthetic cognition, which arises from dynamic visual exploration shaped by scanning paths, processing fluency, and the interplay between bottom-up salience and top-down intention. We introduce AestheticNet, a novel cognitive-inspired AQA paradigm that integrates human-like visual cognition and semantic perception with a two-pathway architecture. The visual
attention pathway, implemented as a gaze-aligned visual encoder (GAVE) pre-trained offline on eye-tracking data using resource-efficient contrast gaze
alignment, models
attention from human
vision system. This pathway augments the semantic pathway, which uses a fixed semantic encoder such as CLIP, through cross-
attention fusion. Visual
attention provides a cognitive prior reflecting foreground/background structure, color cascade, brightness, and lighting, all of which are determinants of aesthetic perception beyond semantics. Experiments validated by hypothesis testing show a consistent improvement over the semantic-alone baselines, and demonstrate the gaze module as a model-agnostic corrector compatible with diverse AQA backbones, supporting the necessity and modularity of human-like visual cognition for AQA. Our code is available at https://github.com/keepgallop/AestheticNet.
Read the full abstract ▾
All authors Liwen Yu, Chi Liu, Xiaotong Han, Congcong Zhu, Minghao Wang, Sheng Shen
Computer Vision
Beyond Text Prompts: Precise Concept Erasure through Text-Image Collaboration
Jun Li, Lizhi Xiong, Ziqiang Li et al. (+4 more)
Text-to-image generative models have achieved impressive fidelity and diversity, but can inadvertently produce unsafe or undesirable content due to implicit biases embedded in large-scale training
datasets. Existing concept erasure methods, whether text-only or image-assisted, face trade-offs: textual approaches often fail to fully suppress concepts, while naive image-guided methods risk over-erasing unrelated content. We propose TICoE, a text-image Collaborative Erasing framework that achieves precise and faithful concept removal through a continuous convex concept
manifold and hierarchical visual representation learning. TICoE precisely removes target concepts while preserving unrelated semantic and visual content. To objectively assess the quality of erasure, we further introduce a fidelity-oriented evaluation strategy that measures post-erasure usability. Experiments on multiple
benchmarks show that TICoE surpasses prior methods in concept removal precision and content fidelity, enabling safer, more controllable text-to-image generation. Our code is available at https://github.com/OpenAscent-L/TICoE.git
Read the full abstract ▾
All authors Jun Li, Lizhi Xiong, Ziqiang Li, Weiwei Jiang, Zhangjie Fu, Yong Li, Guo-Sen Xie
Computer Vision
SSFT: A Lightweight Spectral-Spatial Fusion Transformer for Generic Hyperspectral Classification
Alexander Musiat, Nikolas Ebert and Oliver Wasenmüller
Hyperspectral imaging enables fine-grained recognition of materials by capturing rich spectral signatures, but learning robust classifiers is challenging due to high dimensionality, spectral redundancy, limited labeled data, and strong domain shifts. Beyond earth observation, labeled HSI data is often scarce and imbalanced, motivating compact models for generic hyperspectral classification across diverse acquisition regimes. We propose the lightweight Spectral-Spatial Fusion
Transformer (SSFT), which factorizes representation learning into spectral and spatial pathways and integrates them via cross-
attention to capture complementary wavelength-dependent and structural information. We evaluate our SSFT on the challenging HSI-
Benchmark, a heterogeneous multi-
dataset benchmark covering earth observation, fruit condition assessment, and fine-grained material recognition. SSFT achieves state-of-the-art overall performance, ranking first while using less than 2% of the parameters of the previous leading method. We further evaluate transfer to the substantially larger SpectralEarth
benchmark under the official protocol, where SSFT remains competitive despite its compact size. Ablation studies show that both spectral and spatial pathways are crucial, with spatial modeling contributing most, and that SSFT remains robust without data augmentation.
Read the full abstract ▾
Computer Vision
Watching Movies Like a Human: Egocentric Emotion Understanding for Embodied Companions
Ze Dong, Hao Shi, Zejia Gao et al. (+3 more)
Embodied robotic
agents often perceive movies through an
egocentric screen-view interface rather than native cinematic footage, introducing domain shifts such as viewpoint distortion, scale variation, illumination changes, and environmental interference. However, existing research on movie
emotion understanding is almost exclusively conducted on cinematic footage, limiting cross-domain generalization to real-world viewing scenarios. To bridge this gap, we introduce EgoScreen-
Emotion (ESE), the first
benchmark dataset for
egocentric screen-view movie
emotion understanding. ESE contains 224 movie trailers captured under controlled
egocentric screen-view conditions, producing 28,667 temporally aligned key-frames annotated by multiple raters with a confidence-aware multi-label protocol to address emotional ambiguity. We further build a
multimodal long-context
emotion reasoning framework that models temporal visual evidence, narrative summaries, compressed historical context, and audio cues. Cross-domain experiments reveal a severe domain gap: models trained on cinematic footage drop from 27.99 to 16.69 Macro-F1 when evaluated on realistic
egocentric screen-view observations. Training on ESE substantially improves robustness under realistic viewing conditions. Our approach achieves competitive performance compared with strong closed-source
multimodal models, highlighting the importance of domain-specific data and long-context
multimodal reasoning.
Read the full abstract ▾
All authors Ze Dong, Hao Shi, Zejia Gao, Zhonghua Yi, Kaiwei Wang, Lin Wang
Computer Vision
Continual Hand-Eye Calibration for Open-world Robotic Manipulation
Fazeng Li, Gan Sun, Chenxi Liu et al. (+3 more)
Hand-eye calibration through visual localization is a critical capability for robotic manipulation in open-world environments. However, most deep learning-based calibration models suffer from catastrophic forgetting when adapting into unseen data amongst open-world scene changes, while simple rehearsal-based continual learning strategy cannot well mitigate this issue. To overcome this challenge, we propose a continual hand-eye calibration framework, enabling
robots to adapt to sequentially encountered open-world manipulation scenes through spatially replay strategy and structure-preserving
distillation. Specifically, a Spatial-Aware Replay Strategy (SARS) constructs a geometrically uniform replay buffer that ensures comprehensive coverage of each scene pose space, replacing redundant adjacent frames with maximally informative viewpoints. Meanwhile, a Structure-Preserving Dual
Distillation (SPDD) is proposed to decompose localization knowledge into coarse scene layout and fine pose precision, and distills them separately to alleviate both types of forgetting during continual adaptation. As a new manipulation scene arrives, SARS provides geometrically representative replay samples from all prior scenes, and SPDD applies structured
distillation on these samples to retain previously learned knowledge. After training on the new scene, SARS incorporates selected samples from the new scene into the replay buffer for future rehearsal, allowing the model to continuously accumulate multi-scene calibration capability. Experiments on multiple public
datasets show significant anti scene forgetting performance, maintaining accuracy on past scenes while preserving adaptation to new scenes, confirming the effectiveness of the framework.
Read the full abstract ▾
All authors Fazeng Li, Gan Sun, Chenxi Liu, Yao He, Wei Cong, Yang Cong
Computer Vision
Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow
Chengxin Liu, Wonseok Choi, Chenshuang Zhang et al. (+1 more)
Vision-
Language Models (VLMs) have demonstrated strong capability in a wide range of tasks such as visual recognition, document parsing, and visual grounding. Nevertheless, recent work shows that while VLMs often manage to capture the correct image region corresponding to the question, they do not necessarily produce the correct answers. In this work, we demonstrate that this misalignment could be attributed to suboptimal information flow within VLMs, where text
tokens distribute too much
attention to irrelevant visual
tokens, leading to incorrect answers. Based on the observation, we show that modulating the information flow during inference can improve the perception capability of VLMs. The idea is that text
tokens should only be associated with important visual
tokens during decoding, eliminating the interference of irrelevant regions. To achieve this, we propose a
token dynamics-based method to determine the importance of visual
tokens, where visual
tokens that exhibit distinct activation patterns during different decoding stages are viewed as important. We apply our approach to representative open-source VLMs and evaluate on various
datasets, including visual question answering, visual grounding and counting, optical character recognition, and object
hallucination. The results show that our approach significantly improves the performance of baselines. Project page: https://cxliu0.github.io/AIF/.
Read the full abstract ▾
All authors Chengxin Liu, Wonseok Choi, Chenshuang Zhang, Tae-Hyun Oh
Computer Vision
Beyond a Single Frame: Multi-Frame Spatially Grounded Reasoning Across Volumetric MRI
Lama Moukheiber, Caleb M. Yeung, Haotian Xue et al. (+3 more)
Spatial
reasoning and visual grounding are core capabilities for
vision-
language models (VLMs), yet most medical VLMs produce predictions without transparent
reasoning or spatial evidence. Existing
benchmarks also evaluate VLMs on isolated 2D images, overlooking the volumetric nature of clinical imaging, where findings can span multiple frames or appear on only a few slices. We introduce Spatially Grounded MRI Visual Question Answering (SGMRI-VQA), a 41,307-pair
benchmark for multi-frame, spatially grounded
reasoning on volumetric MRI. Built from expert radiologist annotations in the fastMRI+
dataset across
brain and knee studies, each QA pair includes a clinician-aligned chain-of-thought trace with frame-indexed bounding box coordinates. Tasks are organized hierarchically across detection, localization, counting/classification, and captioning, requiring models to jointly reason about what is present, where it is, and across which frames it extends. We
benchmark 10 VLMs and show that supervised fine-tuning of Qwen3-VL-8B with bounding box supervision consistently improves grounding performance over strong zero-shot baselines, indicating that targeted spatial supervision is an effective path toward grounded clinical
reasoning.
Read the full abstract ▾
All authors Lama Moukheiber, Caleb M. Yeung, Haotian Xue, Alec Helbling, Zelin Zhao, Yongxin Chen
Computer Vision
Fed3D: Federated 3D Object Detection
Suyan Dai, Chenxi Liu, Fazeng Li et al. (+1 more)
3D object detection models trained in one server plays an important role in autonomous driving, robotics manipulation, and augmented reality scenarios. However, most existing methods face severe
privacy concern when deployed on a multi-
robot perception network to explore large-scale 3D scene. Meanwhile, it is highly challenging to employ conventional
federated learning methods on 3D object detection scenes, due to the 3D data heterogeneity and limited communication bandwidth. In this paper, we take the first attempt to propose a novel
Federated 3D object detection framework (i.e., Fed3D), to enable distributed learning for 3D object detection with
privacy preservation. Specifically, considering the irregular input 3D object in local
robot and various category distribution between
robots could cause local heterogeneity and global heterogeneity, respectively. We then propose a local-global class-aware loss for the 3D data heterogeneity issue, which could balance gradient back-propagation rate of different 3D categories from local and global aspects. To reduce communication cost on each round, we develop a
federated 3D
prompt module, which could only learn and communicate the
prompts with few learnable parameters. To the end, several extensive experiments on
federated 3D object detection show that our Fed3D model significantly outperforms state-of-the-art
algorithms with lower communication cost when providing the limited local training data.
Read the full abstract ▾
All authors Suyan Dai, Chenxi Liu, Fazeng Li, Peican Lin
Computer Vision
SegMix:Shuffle-based Feedback Learning for Semantic Segmentation of Pathology Images
Zhiling Yan, Sicheng Chen, Tianyi Zhang et al. (+3 more)
Segmentation is a critical task in computational pathology, as it identifies areas affected by disease or abnormal growth and is essential for diagnosis and treatment. However, acquiring high-quality pixel-level supervised
segmentation data requires significant workload demands from experienced pathologists, limiting the application of deep learning. To overcome this challenge, relaxing the label conditions to image-level classification labels allows for more data to be used and more scenarios to be enabled. One approach is to leverage Class Activation Map (CAM) to generate pseudo pixel-level annotations for semantic
segmentation with only image-level labels. However, this method fails to thoroughly explore the essential characteristics of pathology images, thus identifying only small areas that are insufficient for pseudo masking. In this paper, we propose a novel shuffle-based feedback learning method inspired by curriculum learning to generate higher-quality pseudo-semantic
segmentation masks. Specifically, we perform patch level shuffle of pathology images, with the model adaptively adjusting the shuffle strategy based on feedback from previous learning. Experimental results demonstrate that our proposed approach outperforms state-of-the-arts on three different
datasets.
Read the full abstract ▾
All authors Zhiling Yan, Sicheng Chen, Tianyi Zhang, Nan Ying, Yanli Lei, Guanglei Zhang
Computer Vision
PLAF: Pixel-wise Language-Aligned Feature Extraction for Efficient 3D Scene Understanding
Junjie Wen, Junlin He, Fei Ma et al. (+1 more)
Accurate open-vocabulary 3D scene understanding requires semantic representations that are both
language-aligned and spatially precise at the pixel level, while remaining scalable when lifted to 3D space. However, existing representations struggle to jointly satisfy these requirements, and densely propagating pixel-wise semantics to 3D often results in substantial redundancy, leading to inefficient storage and querying in large-scale scenes. To address these challenges, we present PLAF, a Pixel-wise
Language-Aligned Feature extraction framework that enables dense and accurate semantic
alignment in 2D without sacrificing open-vocabulary expressiveness. Building upon this representation, we further design an efficient semantic storage and querying scheme that significantly reduces redundancy across both 2D and 3D domains. Experimental results show that PLAF provides a strong semantic foundation for accurate and efficient open-vocabulary 3D scene understanding. The codes are publicly available at https://github.com/RockWenJJ/PLAF.
Read the full abstract ▾
All authors Junjie Wen, Junlin He, Fei Ma, Jinqiang Cui
Computer Vision · Cross-listed from cs.CL
TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language Models
Jinlun Ye, Jiang Liao, Runhe Lai et al. (+4 more)
Vision-
language models (VLMs) such as CLIP exhibit strong Out-of-distribution (OOD) detection capabilities by aligning visual and textual representations. Recent CLIP-based test-time adaptation methods further improve detection performance by incorporating external OOD labels. However, such labels are finite and fixed, while the real OOD semantic space is inherently open-ended. Consequently, fixed labels fail to represent the diverse and evolving OOD semantics encountered in test streams. To address this limitation, we introduce Test-time Textual Learning (TTL), a framework that dynamically learns OOD textual semantics from unlabeled test streams, without relying on external OOD labels. TTL updates learnable
prompts using pseudo-labeled test samples to capture emerging OOD knowledge. To suppress noise introduced by pseudo-labels, we introduce an OOD knowledge purification strategy that selects reliable OOD samples for adaptation while suppressing noise. In addition, TTL maintains an OOD Textual Knowledge Bank that stores high-quality textual features, providing stable score calibration across batches. Extensive experiments on two standard
benchmarks with nine OOD
datasets demonstrate that TTL consistently achieves state-of-the-art performance, highlighting the value of textual adaptation for robust test-time OOD detection. Our code is available at https://github.com/figec/TTL.
Read the full abstract ▾
All authors Jinlun Ye, Jiang Liao, Runhe Lai, Xinhua Lu, Jiaxin Zhuang, Zhiyong Gan, Ruixuan Wang
Computer Vision
Concept-wise Attention for Fine-grained Concept Bottleneck Models
Minghong Zhong, Guoshuai Zou, Kanghao Chen et al. (+2 more)
Recently impressive performance has been achieved in Concept Bottleneck Models (CBM) by utilizing the image-text
alignment learned by a large pre-trained
vision-
language model (i.e. CLIP). However, there exist two key limitations in concept modeling. Existing methods often suffer from pre-training biases, manifested as granularity misalignment or reliance on structural priors. Moreover, fine-tuning with Binary Cross-
Entropy (BCE) loss treats each concept independently, which ignores mutual exclusivity among concepts, leading to suboptimal
alignment. To address these limitations, we propose Concept-wise
Attention for Fine-grained Concept Bottleneck Models (CoAt-CBM), a novel framework that achieves adaptive fine-grained image-concept
alignment and high interpretability. Specifically, CoAt-CBM employs learnable concept-wise visual queries to adaptively obtain fine-grained concept-wise visual
embeddings, which are then used to produce a concept score vector. Then, a novel concept contrastive optimization guides the model to handle the relative importance of the concept scores, enabling concept predictions to faithfully reflect the image content and improved
alignment. Extensive experiments demonstrate that CoAt-CBM consistently outperforms state-of-the-art methods. The codes will be available upon acceptance.
Read the full abstract ▾
All authors Minghong Zhong, Guoshuai Zou, Kanghao Chen, Dexia Chen, Ruixuan Wang
Computer Vision
RefereeBench: Are Video MLLMs Ready to be Multi-Sport Referees
Yichen Xu, Yuanhang Liu, Chuhan Wang et al. (+5 more)
While
Multimodal Large
Language Models (MLLMs) excel at generic
video understanding, their ability to support specialized, rule-grounded decision-making remains insufficiently explored. In this paper, we introduce RefereeBench, the first large-scale
benchmark for evaluating MLLMs as automatic sports referees. Spanning 11 sports with 925 curated
videos and 6,475 QA pairs, RefereeBench evaluates five core officiating abilities: foul existence, foul and penalty classification, foul and penalty
reasoning, entity perception, and temporal grounding. The
benchmark is fully human-annotated to ensure high-quality annotations grounded in authentic officiating logic and
multimodal evidence. Extensive evaluations of state-of-the-art MLLMs show that even the strongest models, such as Doubao-Seed-1.8 and Gemini-3-Pro, achieve only around 60% accuracy, while the strongest open-source model, Qwen3-VL, reaches only 47%. These results indicate that current models remain far from being reliable sports referees. Further analysis shows that while models can often identify incidents and involved entities, they struggle with rule application and temporal grounding, and frequently over-call fouls on normal clips. Our
benchmark highlights the need for future MLLMs that better integrate domain knowledge and
multimodal understanding, advancing trustworthy AI-assisted officiating and broader
multimodal decision-making.
Read the full abstract ▾
All authors Yichen Xu, Yuanhang Liu, Chuhan Wang, Zihan Zhao, jinghan luo, Jianzhe Ma, Wenxuan Wang, Qin Jin
Computer Vision
Sketch and Text Synergy: Fusing Structural Contours and Descriptive Attributes for Fine-Grained Image Retrieval
Siyuan Wang, Hanchen Gao, Guangming Zhu et al. (+5 more)
Fine-grained image
retrieval via hand-drawn sketches or textual descriptions remains a critical challenge due to inherent modality gaps. While hand-drawn sketches capture complex structural contours, they lack color and texture, which text effectively provides despite omitting spatial contours. Motivated by the complementary nature of these modalities, we propose the Sketch and Text Based Image
Retrieval (STBIR) framework. By synergizing the rich color and texture cues from text with the structural outlines provided by sketches, STBIR achieves superior fine-grained
retrieval performance. First, a curriculum learning driven robustness enhancement module is proposed to enhance the model's robustness when handling queries of varying quality. Second, we introduce a category-knowledge-based feature space optimization module, thereby significantly boosting the model's representational power. Finally, we design a multi-stage cross-modal feature
alignment mechanism to effectively mitigate the challenges of cross modal feature
alignment. Furthermore, we curate the fine-grained STBIR
benchmark dataset to rigorously validate the efficacy of our proposed framework and to provide data support as a reference for subsequent related research. Extensive experiments demonstrate that the proposed STBIR framework significantly outperforms state of the art methods.
Read the full abstract ▾
All authors Siyuan Wang, Hanchen Gao, Guangming Zhu, Jiang Lu, Yiyue Ma, Tianci Wu, Jincai Huang, Liang Zhang
Computer Vision
MambaBack: Bridging Local Features and Global Contexts in Whole Slide Image Analysis
Sicheng Chen, Chad Wong, Tianyi Zhang et al. (+3 more)
Whole Slide Image (WSI) analysis is pivotal in computational pathology, enabling cancer diagnosis by integrating morphological and architectural cues across magnifications. Multiple Instance Learning (MIL) serves as the standard framework for WSI analysis. Recently, Mamba has become a promising backbone for MIL, overtaking
Transformers due to its efficiency and global context modeling capabilities originating from Natural
Language Processing (NLP). However, existing Mamba-based MIL approaches face three critical challenges: (1) disruption of 2D spatial locality during 1D sequence flattening; (2) sub-optimal modeling of fine-grained local cellular structures; and (3) high
memory peaks during inference on resource-constrained edge devices. Studies like MambaOut reveal that Mamba's SSM component is redundant for local feature extraction, where Gated CNNs suffice. Recognizing that WSI analysis demands both fine-grained local feature extraction akin to natural images, and global context modeling akin to NLP, we propose MambaBack, a novel hybrid architecture that harmonizes the strengths of Mamba and MambaOut. First, we propose the
Hilbert sampling strategy to preserve the 2D spatial locality of tiles within 1D sequences, enhancing the model's spatial perception. Second, we design a hierarchical structure comprising a 1D Gated CNN block based on MambaOut to capture local cellular features, and a BiMamba2 block to aggregate global context, jointly enhancing multi-scale representation. Finally, we implement an asymmetric chunking design, allowing parallel processing during training and chunking-streaming accumulation during inference, minimizing peak
memory usage for deployment. Experimental results on five
datasets demonstrate that MambaBack outperforms seven state-of-the-art methods. Source code and
datasets are publicly available.
Read the full abstract ▾
All authors Sicheng Chen, Chad Wong, Tianyi Zhang, Enhui Chai, Zeyu Liu, Fei Xia
Computer Vision
Diffusion Autoencoder for Unsupervised Artifact Restoration in Handheld Fundus Images
Mathumetha Palani, Kavya Puthumana, Ayantika Das et al. (+1 more)
The advent of handheld fundus imaging devices has made ophthalmologic diagnosis and disease screening more accessible, efficient, and cost-effective. However, images captured from these setups often suffer from artifacts such as flash reflections, exposure variations, and motion-induced blur, which degrade image quality and hinder downstream analysis. While generative models have been effective in image restoration, most depend on paired supervision or predefined artifact structures, making them less adaptable to unstructured degradations commonly observed in handheld fundus images. To address this, we propose an unsupervised
diffusion autoencoder that integrates a context encoder with the denoising process to learn semantically meaningful representations for artifact restoration. The model is trained only on high-quality table-top fundus images and infers to restore artifact-affected handheld acquisitions. We validate the restorations through quantitative and qualitative evaluations, and have shown that diagnostic accuracy increases to 81.17% on an unseen
dataset and multiple artifact conditions
Read the full abstract ▾
All authors Mathumetha Palani, Kavya Puthumana, Ayantika Das, Ganapathy Krishnamurthi
Computer Vision
NeuroLip: An Event-driven Spatiotemporal Learning Framework for Cross-Scene Lip-Motion-based Visual Speaker Recognition
Junguang Yao, Wenye Liu, Stjepan Picek et al. (+1 more)
Visual speaker recognition based on lip motion offers a silent, hands-free, and behavior-driven biometric solution that remains effective even when acoustic cues are unavailable. Compared to traditional methods that rely heavily on appearance-dependent representations, lip motion encodes subject-specific behavioral dynamics driven by consistent articulation patterns and muscle coordination, offering inherent stability across environmental changes. However, capturing these robust, fine-grained dynamics is challenging for conventional frame-based cameras due to motion blur and low dynamic range. To exploit the intrinsic stability of lip motion and address these sensing limitations, we propose NeuroLip, an event-based framework that captures fine-grained lip dynamics under a strict yet practical cross-scene protocol: training is performed under a single controlled condition, while recognition must generalize to unseen viewing and lighting conditions. NeuroLip features a 1) Temporal-aware Voxel Encoding module with adaptive event weighting, 2) Structure-aware Spatial Enhancer that amplifies discriminative behavioral patterns by suppressing noise while preserving vertically structured motion information, and 3) Polarity Consistency Regularization mechanism to retain motion-direction cues encoded in event polarities. To facilitate systematic evaluation, we introduce DVSpeaker, a comprehensive event-based lip-motion
dataset comprising 50 subjects recorded under four distinct viewpoint and illumination scenarios. Extensive experiments demonstrate that NeuroLip achieves near-perfect matched-scene accuracy and robust cross-scene generalization, attaining over 71% accuracy on unseen viewpoints and nearly 76% under low-light conditions, outperforming representative existing methods by at least 8.54%. The
dataset and code are publicly available at https://github.com/JiuZeongit/NeuroLip.
Read the full abstract ▾
All authors Junguang Yao, Wenye Liu, Stjepan Picek, Yue Zheng
Computer Vision
SSMamba: A Self-Supervised Hybrid State Space Model for Pathological Image Classification
Enhui Chai, Sicheng Chen, Tianyi Zhang et al. (+2 more)
Pathological diagnosis is highly reliant on image analysis, where Regions of Interest (ROIs) serve as the primary basis for diagnostic evidence, while whole-slide image (WSI)-level tasks primarily capture aggregated patterns. To extract these critical morphological features, ROI-level Foundation Models (FMs) based on
Vision Transformers (ViTs) and large-scale self-supervised learning (SSL) have been widely adopted. However, three core limitations remain in their application to ROI analysis: (1) cross-magnification domain shift, as fixed-scale pretraining hinders adaptation to diverse clinical settings; (2) inadequate local-global relationship modeling, wherein the ViT backbone of FMs suffers from high computational overhead and imprecise local characterization; (3) insufficient fine-grained sensitivity, as traditional self-
attention mechanisms tend to overlook subtle diagnostic cues. To address these challenges, we propose SSMamba, a hybrid SSL framework that enables effective fine-grained feature learning without relying on large external
datasets. This framework incorporates three domain-adaptive components: Mamba Masked Image Modeling (MAMIM) for mitigating domain shift, a Directional Multi-scale (DMS) module for balanced local-global modeling, and a Local Perception Residual (LPR) module for enhanced fine-grained sensitivity. Employing a two-stage pipeline, SSL pretraining on target ROI
datasets followed by supervised fine-tuning (SFT), SSMamba outperforms 11 state-of-the-art (SOTA) pathological FMs on 10 public ROI
datasets and surpasses 8 SOTA methods on 6 public WSI
datasets. These results validate the superiority of task-specific architectural designs for pathological image analysis.
Read the full abstract ▾
All authors Enhui Chai, Sicheng Chen, Tianyi Zhang, Xingyu Li, Tianxiang Cui
Computer Vision
APC: Transferable and Efficient Adversarial Point Counterattack for Robust 3D Point Cloud Recognition
Geunyoung Jung, Soohong Kim, Inseok Kong et al. (+1 more)
The advent of deep neural networks has led to remarkable progress in 3D point cloud recognition, but they remain vulnerable to
adversarial attacks. Although various defense methods have been studied, they suffer from a trade-off between robustness and transferability. We propose
Adversarial Point Counterattack (APC) to achieve both simultaneously. APC is a lightweight input-level purification module that generates instance-specific counter-perturbations for each point, effectively neutralizing attacks. Leveraging clean-
adversarial pairs, APC enforces
geometric consistency in data space and semantic consistency in feature space. To improve generalizability across diverse attacks, we adopt a hybrid training strategy using
adversarial point clouds from multiple attack types. Since APC operates purely on input point clouds, it directly transfers to unseen models and defends against attacks targeting them without retraining. At inference, a single APC forward pass provides purified point clouds with negligible time and parameter overhead. Extensive experiments on two 3D recognition
benchmarks demonstrate that the APC achieves state-of-the-art defense performance. Furthermore, cross-model evaluations validate its superior transferability. The code is available at https://github.com/gyjung975/APC.
Read the full abstract ▾
All authors Geunyoung Jung, Soohong Kim, Inseok Kong, Jiyoung Jung
Computer Vision
LP$^{2}$DH: A Locality-Preserving Pixel-Difference Hashing Framework for Dynamic Texture Recognition
Ruxin Ding, Jianfeng Ren, Heng Yu et al. (+2 more)
Spatiotemporal Local Binary Pattern (STLBP) is a widely used dynamic texture descriptor, but it suffers from extremely high dimensionality. To tackle this, STLBP features are often extracted on three orthogonal planes, which sacrifice inter-plane correlation. In this work, we propose a Locality-Preserving Pixel-Difference Hashing (LP$^{2}$DH) framework that jointly encodes pixel differences in the full spatiotemporal neighbourhood. LP$^{2}$DH transforms Pixel-Difference Vectors (PDVs) into compact binary codes with maximal discriminative power. Furthermore, we incorporate a locality-preserving
embedding to maintain the PDVs' local structure before and after hashing. Then, a curvilinear search strategy is utilized to jointly optimize the hashing
matrix and binary codes via gradient descent on the Stiefel
manifold. After hashing, dictionary learning is applied to encode the binary vectors into codewords, and the resulting histogram is utilized as the final feature representation. The proposed LP$^{2}$DH achieves state-of-the-art performance on three major dynamic texture recognition
benchmarks: 99.80% against DT-GoogleNet's 98.93% on UCLA, 98.52% against HoGF$^{3D}$'s 97.63% on DynTex++, and 96.19% compared to STS's 95.00% on YUPENN. The source code is available at: https://github.com/drx770/LP2DH.
Read the full abstract ▾
All authors Ruxin Ding, Jianfeng Ren, Heng Yu, Jiawei Li, Xudong Jiang
Computer Vision
P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models
Geunyoung Jung, Soohong Kim, Kyungwoo Song et al. (+1 more)
With the rise of pre-trained models in the 3D point cloud domain for a wide range of real-world applications, adapting them to downstream tasks has become increasingly important. However, conventional full fine-tuning methods are computationally expensive and storage-intensive. Although
prompt tuning has emerged as an efficient alternative, it often suffers from overfitting, thereby compromising generalization capability. To address this issue, we propose Prototypical Point-level
Prompt Tuning (P$^3$T), a parameter-efficient
prompt tuning method designed for pre-trained 3D
vision-
language models (VLMs). P$^3$T consists of two components: 1) Point Prompter, which generates instance-aware point-level
prompts for the input point cloud, and 2) Text Prompter, which employs learnable
prompts into the input text instead of hand-crafted ones. Since both prompters operate directly on input data, P$^3$T enables task-specific adaptation of 3D VLMs without sacrificing generalizability. Furthermore, to enhance
embedding space
alignment, which is key to fine-tuning 3D VLMs, we introduce a prototypical loss that reduces intra-category variance. Extensive experiments demonstrate that our method matches or outperforms full fine-tuning in classification and few-shot learning, and further exhibits robust generalization under data shift in the cross-
dataset setting. The code is available at \textcolor{violet}{https://github.com/gyjung975/P3T}.
Read the full abstract ▾
All authors Geunyoung Jung, Soohong Kim, Kyungwoo Song, Jiyoung Jung
Computer Vision
Self-Supervised Angular Deblurring in Photoacoustic Reconstruction via Noisier2Inverse
Markus Haltmeier, Nadja Gruber and Gyeongha Hwang
Photoacoustic
tomography (PAT) is an emerging imaging modality that combines the complementary strengths of optical contrast and ultrasonic resolution. A central task is image reconstruction, where measured acoustic signals are used to recover the initial pressure distribution. For ideal point-like or line-like detectors, several efficient and fast reconstruction
algorithms exist, including
Fourier methods, filtered backprojection, and time reversal. However, when applied to data acquired with finite-size detectors, these methods yield systematically blurred images. Although sharper images can be obtained by compensating for finite-detector effects, supervised learning approaches typically require ground-truth images that may not be available in practice. We propose a self-supervised reconstruction method based on Noisier2Inverse that addresses finite-size detector effects without requiring ground-truth data. Our approach operates directly on noisy measurements and learns to recover high-quality PAT images in a ground-truth-free manner. Its key components are: (i) PAT-specific modeling that recasts the problem as angular deblurring; (ii) a Noisier2Inverse formulation in the polar domain that leverages the known angular point-spread function; and (iii) a novel, statistically grounded early-stopping rule. In experiments, the proposed method consistently outperforms alternative approaches that do not use supervised data and achieves performance close to supervised
benchmarks, while remaining practical for real acquisitions with finite-size detectors.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Hierarchical Active Inference using Successor Representations
Prashant Rangarajan and Rajesh P. N. Rao
Active inference, a neurally-inspired model for inferring actions based on the free energy principle (FEP), has been proposed as a unifying framework for understanding perception, action, and learning in the
brain. Active inference has previously been used to model ecologically important tasks such as
navigation and planning, but scaling it to solve complex large-scale problems in real-world environments has remained a challenge. Inspired by the existence of multi-scale hierarchical representations in the
brain, we propose a model for planning of actions based on hierarchical active inference. Our approach combines a hierarchical model of the environment with successor representations for efficient planning. We present results demonstrating (1) how lower-level successor representations can be used to learn higher-level abstract states, (2) how planning based on active inference at the lower-level can be used to bootstrap and learn higher-level abstract actions, and (3) how these learned higher-level abstract states and actions can facilitate efficient planning. We illustrate the performance of the approach on several planning and reinforcement learning (RL) problems including a variant of the well-known four rooms task, a key-based
navigation task, a partially observable planning problem, the Mountain Car problem, and PointMaze, a family of
navigation tasks with continuous state and action spaces. Our results represent, to our knowledge, the first application of learned hierarchical state and action abstractions to active inference in FEP-based theories of
brain function.
Read the full abstract ▾