Computer Vision
Annotation-Free Furniture Codes: What They Encode, and How Far They Transfer
Benjamin Friedman
Layout-based 3D scene synthesizers place each object using two human-annotated channels: a categorical class label and a canonical-pose convention. We ask whether a single self-supervised
token derived from object geometry can replace both, and study such
tokens directly as a representation, decoupled from any synthesizer. A Finite Scalar Quantization (FSQ) point-cloud autoencoder is chamfer-trained on placed 3D-FUTURE furniture with no labels or pose annotations. Diagnostic probes recover fine-category (62.6 +/- 0.5%), super-category (85.6 +/- 1.3%), and yaw (52.7 +/- 0.5 deg) from the codes alone. Swapping the chamfer target from the rotated to the un-rotated point cloud collapses the yaw signal while raising class recovery, showing the codes' rotation content can be set by the training objective. Scaling across asset libraries needs codes that transfer; on an unseen
dataset (ShapeNet),
alignment is category-dependent: box-like furniture transfers, organically-shaped furniture does not, and a target-blind augmentation partly closes the gap.
Read the full abstract ▾
Computer Vision
BOCCHI: A More Realistic and Challenging Benchmark for Local Motion Blur Detection with MSDCT-UNet
Kuan-Lin Chen, Yuan-Kang Lee, Cheng-Yuan Chiang et al. (+1 more)
Local motion blur detection requires pixel-level localization of blurred regions. Existing
benchmarks let models rely on gradient shortcuts that fail to transfer. We introduce BOCCHI (Blurred Objects Captured across Cameras with Human-annotated Imagery), a real-captured
benchmark whose sharp regions overlap the blur gradient distribution and defeat these shortcuts, and propose MSDCT-UNet (Multi-Scale Discrete Cosine Transform UNet), a frequency-aware encoder-decoder injecting multi-scale DCT priors through DCT
Attention and FiLM. MSDCT-UNet ranks first in in-domain mIoU and boundary localization on BOCCHI, and BOCCHI-trained models outperform every other training source on cross-
dataset transfer with only 633 training images. Our project page is available at https://brianchen1120.github.io/project/bocchi/.
Read the full abstract ▾
All authors Kuan-Lin Chen, Yuan-Kang Lee, Cheng-Yuan Chiang, Jian-Jiun Ding
Computer Vision
SPORT: Structure-Aware Prototype Disentanglement for Incomplete Multi-View Clustering
Yaoyuan Guo, Zhibin Gu, Songhe Feng et al. (+2 more)
Prototype-based Incomplete Multi-view Clustering has recently attracted increasing
attention by exploiting prototypes as semantic anchors for missing-view imputation. However, existing approaches are still limited in three aspects. First, they typically focus on enforcing cross-view prototype consistency, while ignoring view-specific information embedded in prototypes, thus limiting multi-view expressiveness. Second, most methods rely on instance-level contrastive learning that only aligns paired samples across views, failing to preserve cluster-level relational structures. Third, missing-view imputation is usually performed using global prototypes alone, without considering local
geometric neighborhood structures, leading to inaccurate recovery of missing representations. To address these limitations, we propose a novel framework termed Structure-aware PrOtotype disentanglement foR incomplete multi-view clusTering (SPORT), which explicitly disentangles shared and view-specific components of prototypes while preserving cluster-level relational structures. Specifically, we decouple prototypes into orthogonal shared and view-specific components, aligning only shared components to capture consensus semantics while de-correlating view-specific components to preserve complementary information. Meanwhile, a structure-aware contrastive learning mechanism is incorporated to explicitly model cluster-level relationships during cross-view representation learning. Furthermore, a hybrid imputation strategy integrates global prototype matching with local neighborhood matching, enabling joint exploitation of semantic prototypes and
manifold structures for missing-view recovery. Extensive experiments on six
benchmark datasets show that SPORT achieves superior performance over state-of-the-art methods under various missing rates.
Read the full abstract ▾
All authors Yaoyuan Guo, Zhibin Gu, Songhe Feng, Yuhui Zheng, Bing Li
Computer Vision
GNOCHI: Generative Neural mOdel for Close Human-Human Interactions
Gonzalo Gómez-Nogales, Marc Comino-Trinidad, Andrés Casado-Elvira et al. (+1 more)
Creating realistic 3D human-human interactions in virtual environments is challenging due to the high degrees of freedom in the human body and the need for physically accurate poses that do not collide with each other. Traditional methods for human-human interaction are based on motion tracking or 3D body reconstruction, but lack generative capabilities. Recent generative methods enable the synthesis of individual or interacting motions via text or image input, but generally fall short in modeling close interactions. This paper introduces a novel generative model for close 3D human-human interactions using a conditional variational autoencoder (cVAE), which generates poses for one human conditioned on the pose of another, allowing for controlled and diverse interaction synthesis. To train our model, we address two underlying long-standing challenges in the field of human-human interaction: data scarcity, for which we propose an automated supervised data augmentation strategy that generates synthetic yet realistic interaction poses; and collision awareness in generative approaches, for which we propose a self-supervised loss based on a collision resolution technique using volumetric proxies to ensure physically correct interactions. We extensively evaluate the capabilities of our model, and demonstrate a wide variety of plausible and physically correct interactions, not possible to generate with current state-of-the-art methods.
Read the full abstract ▾
All authors Gonzalo Gómez-Nogales, Marc Comino-Trinidad, Andrés Casado-Elvira, Dan Casas
Computer Vision
TVT-PAPD: Pathology-Aware Prototype Distillation for Self-Supervised Whole Slide Image Classification
Ramesh Naidu Laveti, Jaya Sreevalsan-Nair and T K Srikanth
Self-supervised learning (SSL) has emerged as an effective paradigm for learning transferable representations from large-scale unlabeled whole slide images (WSIs). However, existing SSL methods primarily learn generic visual features and often fail to explicitly capture pathology-specific morphological patterns that are critical for disease characterization. To address this limitation, we propose Tiny
Vision Transformer with Pathology-Aware Prototype
Distillation (TVT-PAPD). This self-supervised pathology representation learning framework integrates a Tiny
Vision Transformer (TVT) with a novel Pathology-Aware Prototype
Distillation (PAPD) module. PAPD employs a learnable pathology prototype bank to discover and preserve representative tissue morphology patterns, encouraging semantically similar pathological regions to learn consistent and discriminative representations. The proposed framework enhances pathology-aware feature learning while maintaining computational efficiency with 90M parameters. Experiments on the Cancer Genome Atlas (TCGA) low-grade glioma (LGG)/glioblastoma (GBM)
dataset and the Indian Pathology
Brain (IPD-
Brain)
dataset demonstrate that TVT-PAPD achieves weighted F1-scores of 93.02% and 90.23%, respectively, for LGG-GBM classification, while exhibiting strong cross-cohort generalization across independent glioma
datasets.
Read the full abstract ▾
Computer Vision
SynthDocBench: Controlled Benchmark for Long-Context Visual Document Understanding
Abhigya Verma, Khyati Mahajan, Amit Kumar Saha et al. (+4 more)
Vision language models (VLMs) have achieved strong performance on visual document understanding
benchmarks such as DocVQA, ChartQA, and MMLongBench-Doc. However, real-world documents combine multiple factors such as length, layout complexity, modality, and question difficulty, which makes it difficult to attribute model failures to specific causes. We introduce SynthDocBench, a fully synthetic
benchmark for long-context visual document understanding that systematically controls factors including document length, layout structure, modality composition, and question type. The
benchmark is constructed using a combinatorial design, each factor is varied independently across generated documents, enabling controlled analysis of model behavior. Documents are generated end to end using an LLM pipeline across six layout archetypes, with a 40 percent random override to prevent models from exploiting spurious correlations. Additionally, SynthDocBench spans long-context documents with substantially greater length and structural diversity than existing
benchmarks. Evaluating seven frontier VLMs, we uncover three failure modes that existing
benchmarks cannot surface: sharp degradation with document length, a systematic positional sensitivity in which the middle third of a document is hardest for five of six models and five of six models show a negative Early-to-Late trend (steepest decline: 8.3 percentage points), and breakdown of chart comprehension in long-document settings. These results suggest that current models may be overfitting to
benchmark artifacts rather than achieving robust long-context visual document understanding.
Read the full abstract ▾
All authors Abhigya Verma, Khyati Mahajan, Amit Kumar Saha, Shruthan Radhakrishna, Sagar Davasam, Vikas Yadav, Sai Rajeswar Mudumba
Computer Vision
Self-supervised Automatic Matting
Xiaonan Hu, Zhiyuan Lu, Jingdong Zhao et al. (+1 more)
High-quality alpha mattes are notoriously expensive to annotate, creating a fundamental data bottleneck for deep image matting. While prior work attempts to reduce annotation cost using coarser labels like trimaps or masks, they remain reliant on costly per-pixel supervision, limiting scalability and generalization. In this work, we push the boundary further and ask: can we train an automatic matting model using only RGB images, with no manual annotation at all? We answer this by presenting SSMatte, a self-supervised framework that for the first time achieves performance on par with fully-supervised automatic matting. Our key insight is to decompose the problem into semantic anchoring and detail matting. SSMatte first generates a semantic matting
prompt from frozen self-supervised ViT features by propagating class-
token seeds via a novel, training-efficient semantic anchoring loss based on a generalized Rayleigh quotient. This
prompt then anchors a detail matting network, which is optimized via a fixed-point-based loss that enforces alpha-RGB consistency. Extensive experiments show SSMatte outperforms prior weakly-supervised methods, matches the performance of fully-supervised models on portrait
benchmarks, and demonstrates favorable scaling and generalization behaviors with additional data. Our work pushes automatic matting to an fresh, fully annotation-free paradigm. Code will be available.
Read the full abstract ▾
All authors Xiaonan Hu, Zhiyuan Lu, Jingdong Zhao, Hao Lu
Computer Vision
Vertical Fusion: Condensing Internal Representations for Robust ViT Classification
Francesco Di Salvo, Shyam Nandan Rai, Hamed Damirchi et al. (+4 more)
Despite exposing rich intermediate representations,
Vision Transformers (ViTs) are almost exclusively utilized as black-box feature extractors, where only the last layer is considered for downstream tasks. We challenge this convention by introducing the notion of recoverability: the capacity of intermediate representations to correct last-layer failures. By evaluating independent classification probes at every model depth across 16
datasets, we observe that intermediate probes correctly classify 18% to 76% of samples that the last-layer probe misclassifies. We show that these gains are not primarily driven by predictive diversity, but by a redundancy-correctness correspondence, where the internal hierarchy acts as a series of stable, redundant probes of a shared discriminative signal. While established horizontal ensemble strategies (i.e., across multiple models) can improve performance, they incur high computational cost and ignore this vertical signal within a single model. To bridge this gap, we propose VFusion, a principled vertical aggregation strategy employing a learnable mapping into a low-dimensional latent space that synthesizes features across the internal ViT hierarchy. VFusion substantially outperforms established aggregation baselines in both in-distribution and out-of-distribution settings, notably closing 45% of the accuracy gap between the best individual layer and a theoretical oracle performance. Our gains consistently generalize across model sizes and pre-training regimes, confirming that VFusion offers a robust and efficient alternative to horizontal ensemble methods. The code is available at https://github.com/francescodisalvo05/vit-vertical-fusion.
Read the full abstract ▾
All authors Francesco Di Salvo, Shyam Nandan Rai, Hamed Damirchi, Ignacio Meza De la Jara, Sebastian Doerrich, Marco Lents, Christian Ledig
Computer Vision
ABot-N1: Toward a General Visual Language Navigation Foundation Model
Ruiyan Gong, Yingnan Guo, Junjun Hu et al. (+43 more)
Visual
Language Navigation foundation models aim to unify deep
reasoning for grounded spatial decisions with broad versatility for diverse
embodied tasks. Current approaches typically achieve this integration via monolithic policies that map observations directly to actions, yet they often suffer from coordinate drift and poor handling of long-tail semantics. Furthermore, these black-box mappings lack interpretability, hindering the simultaneous achievement of generality, robustness, and transparency. We present ABot-N1, a step toward a general Visual
Language Navigation foundation model, that addresses these challenges by decoupling cognition from control via a slow-fast architecture guided by dual visual-
language signals. More specifically, a slow
vision-
language reasoner performs explicit Chain-of-Thought
reasoning while producing a pixel goal. This compact set of image-space anchor points serves as a universal interface for diverse tasks, including point-goal, object-goal, poi-goal, instruction-following, and person-following. Subsequently, a fast action expert leverages both the textual cues and the pixel guidance to generate continuous waypoints at the native control frequency. By bridging high-level intents and low-level control through pixel-grounded anchors paired with explicit linguistic traces, our approach ensures robust, generalizable, and interpretable
navigation across simulation and real-world
benchmarks. ABot-N1 establishes new state-of-the-art records, delivering massive gains specifically in urban-scale
navigation: boosting POI arrival by 35.0% (to 77.3%) and achieving 95.4%/92.9% SR in complex indoor and outdoor scenes. It also maintains superior robustness across object-reaching, person-following, and instruction-following tasks. New Point-Goal/POI-Goal
benchmarks are released as open source to advance the field of urban-scale
navigation.
Read the full abstract ▾
All authors Ruiyan Gong, Yingnan Guo, Junjun Hu, Jintao Kong, Xiaoxu Leng, Tianlun Li, Weize Li, Fei Liu, Zhicheng Liu, Jia Lu, Minghua Luo, Chenlin Ming, Yanfen Shen, Jiyue Tao, Zhengbo Wang, Mingyang Yin, Minqi Gu, Zihao Guan, Wei Guo, Guoqing Liu, Huachong Pang, Menglin Yang, Zeqian Ye, Xiaoxiao Geng, Zhining Gu, Honglin Han, Di Jing, Hongyu Pan, Mingchao Sun, Kuan Yang, Jianfang Zhang, Yanghong Chen, Ye He, Wei Mei, Jiahao Shi, Xiangpo Yang, Yanqing Zhu, Yang Cai, Jingjing Ma, Shihui Su, Zixiao Tang, Linbo Zheng, Zedong Chu, Xiaolong Wu, Wenbin Tang, Mu Xu
Computer Vision · Cross-listed from cs.RO
Robotic Contextual Awareness for Human-Robot Collaboration and Environmental Understanding
Federico Rollo
The transition of autonomous mobile
robots from controlled industrial settings to dynamic, human-centric environments, such as manufacturing, logistics, and healthcare, has made their safe and autonomous operation a critical area of research. These sophisticated machines must be capable of perceiving, understanding, and interacting with their surroundings to navigate freely and perform complex tasks. A significant obstacle to achieving this is the lack of comprehensive contextual awareness, which requires a
robot to recognize its spatial environment and identify the objects and actors within it. Without this perceptual knowledge,
robots struggle to plan adaptive behaviors or engage in meaningful interaction with humans. This thesis presents novel solutions to this challenge by exploring two distinct but complementary research directions. The first direction involves human re-identification and tracking to improve Human-
Robot Collaboration. Our developed approach enables a mobile
robot to recognize a specific person, facilitating targeted collaboration while ignoring other individuals. The second direction focuses on enhancing the
robot's overall perceptual capabilities to understand its environment geometrically and semantically.
Geometric information is vital for motion planning and collision avoidance, while semantic knowledge provides the
robot with a richer understanding for more advanced interaction. Both solutions are driven by the improvement of the semantical understanding of
robots that enhance their knowledge of their surroundings, allowing a smoother and more natural interaction between
robots, humans, and the environment. The contributions of this work in human re-identification and environmental understanding represent a significant step toward a future where
robots are more contextually aware, enabling safer coexistence and more effective collaboration.
Read the full abstract ▾
Computer Vision
Neural Motion Blending Across Arbitrary Character Topologies
Luca Cazzola, Giulia Martinelli and Nicola Conci
Motion blending in character animation enables the synthesis of new motions by interpolating between existing examples. Current methods are typically restricted to fixed skeleton
topologies, requiring identical or near-identical skeletal structures across characters. We present a novel framework for motion blending across heterogeneous skeletons. The proposed architecture combines a semantic encoder, which extracts per-frame latent representations of the motion state, with a
diffusion-based decoder, which reconstructs character-specific motion conditioned on this latent code. At inference, blended motions are obtained by interpolating the latent representations of two input motions. We train and evaluate the method on the Truebones Zoo
dataset using motions defined on both same and distinct skeleton
topologies, demonstrating the ability to achieve smooth and plausible blending in a variety of scenarios.
Read the full abstract ▾
Computer Vision
Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformers
Christopher Buratti, Michele Marchetti, Federica Parlapiano et al. (+3 more)
Vision Transformers (ViTs) are difficult to interpret because current methods of relevance propagation and
attention flow do not fully consider some key architectural features, such as the uneven importance of
attention heads and residual connections. Prior approaches typically assume uniform importance across
attention heads; furthermore, they model skip connections as identity paths, leading to inaccurate relevance attribution. To address these issues, we introduce GradSkip, a novel relevance propagation method for ViTs based on adaptive head weighting and skip-aware propagation. GradSkip models the different importance of the
attention heads and dynamically distributes relevance between the
attention and residual paths. Experiments on ImageNet1K and BloodMNIST demonstrate a state-of-the-art faithfulness of GradSkip while requiring over 14 times fewer GFLOPs than the best-performing existing approaches. Additional evaluations using
transformer-based
segmentation confirm improved localization and
alignment with ground-truth regions.
Read the full abstract ▾
All authors Christopher Buratti, Michele Marchetti, Federica Parlapiano, Davide Traini, Domenico Ursino, Luca Virgili
Computer Vision
Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
Giang Nguyen, Raghav Mehta, Emma A.M. Stanley et al. (+4 more)
Foundation models are increasingly used as image feature extractors for mammography, but their robustness under external domain shift remains unclear. We
benchmark 15 foundation-model backbones across breast density, BI-RADS severity, and cancer status using a unified frozen-backbone linear-probe protocol, training on 3 source
datasets and evaluating on 12 task-compatible out-of-distribution (OOD)
datasets after label harmonization. Mammography-specific
vision-
language models (Mammo-FM and MaMA) provide the strongest mean OOD performance, but robustness is not explained by mammography exposure alone. DINOv3 remains a competitive
vision-only baseline, and mammography-adapted pretraining does not consistently improve generalization.
Dataset-level analysis further shows that even leading models show heterogeneous performance across
datasets. Feature-space inspection reveals that useful representations can preserve clinical signal while retaining
dataset and acquisition structure. These findings highlight
dataset-level OOD evaluation as a central criterion for assessing mammography representations. Our code is publicly available: https://github.com/biomedia-mira/mammo-ood.
Read the full abstract ▾
All authors Giang Nguyen, Raghav Mehta, Emma A.M. Stanley, Tian Xia, Thi Hao Nguyen, Hieu Pham, Ben Glocker
Computer Vision
GRC-ProbNet: Uncertainty-aware Feature Extraction for Cardiovascular Disease Classification
Yash Shah, Omar Todd, Philipp Seeböck et al. (+3 more)
The automatic detection and classification of cardiovascular disease (CVD) from computed
tomography (CT) images plays an important role in clinical practice. Recently, a hybrid pipeline (GRC-Net) for CVD classification was proposed, which leverages a deep-learning-based
segmentation and registration method to extract radiomic and
geometric features. However, GRC-Net relies on a deterministic
segmentation mask, without considering the inherent ambiguity associated with cardiac anatomy. In this paper, we propose GRC-ProbNet, which takes advantage of a deep ensemble to produce multiple
segmentation masks for a given input. From these masks, we extract multiple uncertainty features. We analyze these uncertainty features for both their correlation with
segmentation error and their propagation effects on downstream CVD classification performance. Our experiments on the publicly available MM-WHS and ASOCA
datasets show that the uncertainty measure that best reflects
segmentation quality is not necessarily the one that provides the strongest signal for downstream CVD classification. Overall, our results demonstrate that GRC-ProbNet utilizing uncertainty features substantially improves CVD classification AUROC (92.92\) compared to the baseline GRC-Net model (91.25%). Our code is publicly available: https://github.com/biomedia-mira/GRC-ProbNet.
Read the full abstract ▾
All authors Yash Shah, Omar Todd, Philipp Seeböck, Georg Langs, Ben Glocker, Raghav Mehta
Computer Vision · Cross-listed from cs.RO
PrismAD: Decoupled Planning via Semantic Mixture-of-Planners for End-to-End Autonomous Driving
Kang Ding, Zhigui Lin, Hongsong Wang et al. (+5 more)
This letter presents PrismAD, a decoupled end-to-end autonomous driving framework based on a Semantic Mixture-of-Planners. Existing planners usually aggregate heterogeneous scene
tokens into a coupled representation space, forcing a single planning branch to jointly model
agent interaction, road geometry, and driving intention. Such coupling may weaken factor-specific
reasoning and obscure the contribution of different planning cues. To address this limitation, PrismAD partitions scene
tokens into interaction, geometry, and intent groups, and assigns them to independent planning experts with the same architecture but separate parameters. Each expert learns a specialized motion-planning representation, while a semantics-aware router adaptively aggregates expert predictions with separate routing weights for motion prediction and ego planning. Sparse top-$K$ activation with noisy gating is further introduced to improve routing robustness and reduce unnecessary expert computation. Extensive experiments on the nuScenes open-loop
dataset and NeuroNCAP closed-loop
benchmark demonstrate that PrismAD exhibits competitive performance. Our code will be released soon.
Read the full abstract ▾
All authors Kang Ding, Zhigui Lin, Hongsong Wang, Jie Gui, Qi Liu, Zhe Wang, Luqi Tang, Lei He
Computer Vision
Imperceptible and Reversible Adversarial Examples against Vision-Language Models for Privacy Protection
Qi Lu, Ziqi Zhou, Yufei Song et al. (+5 more)
Vision Language Models (VLMs) offer powerful
multimodal ability but also expose users to text-based
privacy attacks where adversaries crawl online photos and query VLMs to extract sensitive attributes. Existing reversible
adversarial example (RAE) methods protect images in purely visual tasks but fail in
multimodal settings, and current
adversarial examples on VLMs rely on high frequency noise that severely degrades visual quality. We propose CloakDiff, the first framework for reversible, high fidelity
privacy protection against text-based query attacks in VLMs. CloakDiff produces imperceptible
adversarial examples by combining
diffusion based
adversarial editing with an invertible network that embeds the original image for lossless recovery. It perturbs both pixel space
embeddings and manipulates latent cross
attention maps to ensure strong cross-model and cross-
prompt transferability while preserving global visual structure. To further enhance fidelity, we design EDM Heuristic Sampling, a principled
diffusion schedule for
adversarial guidance. Experiments on multiple
datasets and VLMs demonstrate that CloakDiff delivers
multimodal privacy preservation with high visual quality and reversibility.
Read the full abstract ▾
All authors Qi Lu, Ziqi Zhou, Yufei Song, Zijing Li, Lulu Xue, Minghui Li, Shengshan Hu, Leo Yu Zhang
Computer Vision
Generalize LMMs to Versatile Visual Modalities via Fabricated Modality Synthesis
Shihao Yuan, Yuanze Li, Ruyi Zhang et al. (+2 more)
Despite the advancements of Large
Multimodal Models (LMMs) in RGB
vision, their ability to generalize to unseen visual modalities remains a largely unexplored challenge. We argue that different visual modalities are merely distinct samplings of the same physical world. Therefore, effective generalization requires models to possess both modality-agnostic perception of scene semantics and the adaptability to modality-specific characteristics. To achieve this, we propose a training framework, VVM-Tuning, to equip LMMs with these capabilities through modality synthesis and modality contexts. Specifically, we synthesize diverse appearance-varied images from RGB scenes, training the model to disentangle invariant semantics from varying visual appearances, and align these appearances with
language for visual concepts decoupled from modalities. We then introduce modality contexts in the
prompt and use instruction tuning to assist the model in mapping these appearance variations back to modality-related attributes, enabling zero-shot adaptation to unseen modalities during inference. To facilitate research in this direction, we introduce VVM-Bench, a comprehensive
benchmark featuring 6 real and synthetic modalities to evaluate semantic perception and modality understanding. Experiments demonstrate that, via our training on synthetic modalities, 5 tested models exhibit consistent improvements on both real-world and novel synthetic modalities without in-modality training. Source code and data will be publicly available at https://github.com/Hunter-Will/VVM-Tuning.
Read the full abstract ▾
All authors Shihao Yuan, Yuanze Li, Ruyi Zhang, Ming Liu, Wangmeng Zuo
Computer Vision
ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing
Jiakang Yu, Yixuan Chai, Tianci Wang et al. (+7 more)
Generative image editing models struggle with structured statistical charts when data modifications require
geometric synchronization. We formalize this task as Visuo-Logical Cascading Editing (VLCE). However, existing methods remain confined to localized text substitutions and struggle with dependency-aware cascading updates. To systematically evaluate this capability, we introduce ChartSync, an expert-validated
benchmark constructed via a programmatic rendering pipeline that guarantees deterministic visuo-logical coupling for the ground truth. ChartSync comprises 870 triplets across 9 chart categories and 4 task types, including 235 geometry-coupled VLCE instances that specifically test cascading text-to-geometry synchronization. We further evaluate these instances via a two-tier framework combining objective visual metrics with a
vision-
language model judge paradigm to assess low-level fidelity alongside
multimodal comprehension and
reasoning. Evaluating 14 image editing models and one code-mediated pipeline reveals a nuanced capability gap: most open-source models suffer severe drops in
geometric synchronization, while only two frontier proprietary models show emerging VLCE capability, with their residual errors mainly involving semantic isolation and background corruption. Our detailed error analysis deconstructs these failure paradigms to identify core meta-abilities for guiding future
multimodal architectures. The ChartSync
dataset and code are publicly released at https://github.com/kaka-yjk/ChartSyncCodebase.
Read the full abstract ▾
All authors Jiakang Yu, Yixuan Chai, Tianci Wang, Rihui Jin, Guangkai Xu, Hongtao Deng, Xun Zhu, Wang Gao, Xinrun Guo, Haipang Wu
Computer Vision
Structured Evidence Selection for Weakly Supervised Video Anomaly Detection
Chenglizhao Chen, Tianxiang Nan, Wen Li et al. (+4 more)
Weakly supervised
video anomaly detection relies solely on
video-level labels for training, making it difficult to accurately localize anomalous events in complex scenes. In real-world
videos, anomalous behaviors exhibit large variations in appearance and temporal duration, while scene appearance and action dynamics are often tightly entangled. Consequently, existing models tend to rely on scene-related statistical cues rather than true behavioral deviations, resulting in unstable detection performance. To address this challenge, we propose a Structured Evidence Selection framework (SESAD) that reformulates anomaly detection as a structured
reasoning process over clip-level visual evidence. Instead of directly mapping aggregated features to anomaly scores, SESAD reorganizes clip representations into semantically structured candidate evidence and performs context-conditioned selection under scene and action constraints. This mechanism adaptively emphasizes anomaly-relevant semantics while suppressing scene interference, thereby alleviating semantic
entanglement under weak supervision. Furthermore, we introduce a lightweight
geometric discrimination module that constructs a dual-prototype structure in the
embedding space, enabling anomaly decisions through relative
geometric relations. Extensive experiments on UBnormal, ShanghaiTech, and UCF-Crime show that SESAD achieves 67.92, 97.99, and 88.46 AUC, respectively, while maintaining high computational efficiency and overall consistently stable anomaly discrimination.
Read the full abstract ▾
All authors Chenglizhao Chen, Tianxiang Nan, Wen Li, Xinyu Liu, Guisheng Zhang, Mengke Song, Xiaomin Yu
Computer Vision
InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation
Yichen Peng, Jyun-Ting Song, Chen-Chieh Liao et al. (+3 more)
Human-pet interaction estimation and generation remain underexplored due to the absence of a high-quality large-scale
dataset. We present InterPet4D, the first
multimodal dataset capturing natural interactions between humans and dogs. Using a synchronized multi-view capture system, we record human-dog obedience tasks and provide annotations for both humans and dogs, including multi-view and
egocentric videos,
segmentations, 2D and 3D keypoints, meshes, and audio tracks. InterPet4D consists of 6.8 million frames collected from 13 dogs of 11 breeds interacting with 23 human participants. We further introduce the InterPetMoGen framework for human-pet interaction motion generation. Our proposed model achieves an FID score of 11.21 and substantially outperforms the Seq2Seq and DiT baselines, demonstrating the effectiveness of InterPet4D for modeling realistic human-pet interactions.
Read the full abstract ▾
All authors Yichen Peng, Jyun-Ting Song, Chen-Chieh Liao, Kris Kitani, Hideki Koike, Erwin Wu
Computer Vision
Geometry-aware Gaussian Prior and Axial Attention for Cervical Cytology Image Classification
Yating Li, Cheng Ye, Nenan Lyu et al. (+2 more)
Accurate cervical cytology image classification is a key component of automated cervical cancer screening, where reliable recognition of normal, precancerous, and cancer-associated cellular patterns from Pap smear images can improve screening efficiency and diagnostic consistency. However, this task remains challenging because cervical
cells exhibit complex morphology, subtle intra-class variations, and strong inter-class similarities. Existing convolution-based models capture local texture well but have limited ability to model long-range relationships, whereas
attention-based models provide broader context but often lack explicit structural guidance. To address these limitations, we propose a geometry-aware classification framework for cervical cancer screening-oriented cytology image analysis, incorporating semantic abstraction and structural priors learned from pre-trained
vision-
language features. The method uses Gaussian expert modules to generate axis-wise priors from global semantic information, capturing structural regularities such as
nuclear alignment and cellular spatial organization. These priors are embedded into an axial self-
attention module to modulate similarity computation along horizontal and vertical directions, improving long-range dependency modeling and structure-sensitive feature interaction. Experiments on the Mendeley
liquid-based cytology and SIPaKMeD
datasets show that the proposed method achieves 99.48% accuracy on the former and 96.08% on the latter, with balanced gains in recall, precision, and overall classification performance. Visual analysis further shows that the learned priors highlight diagnostically relevant cellular regions, demonstrating the potential of the proposed framework as a screening-oriented decision-support tool for cervical cytology.
Read the full abstract ▾
All authors Yating Li, Cheng Ye, Nenan Lyu, Weidong Chen, Zhendong Mao
Computer Vision
What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks
Guanhua Ye, Niu Jingbin, Yan Li et al. (+4 more)
Short-answer VQA
benchmarks conflate two distinct quantities: whether a model's answer is semantically correct, and whether that answer matches the surface form expected by the automatic evaluator. We study this conflation across six
vision–
language models and six
benchmarks, using a human-validated semantic judge (97.6% precision) to audit over 37k official errors. A second text-only judge reproduces the same
benchmark-level false-negative pattern, showing that the effect is not an artifact of a single audit model. On text-rich
benchmarks, up to half of these errors are semantically acceptable answers penalized purely for surface-form mismatch. This instability is structured by answer type: extractive and multi-span answers are far more evaluator-sensitive than scalar answers. Benign
prompt and context rewrites further destabilize official outcomes, flipping item-level correctness at substantial rates without changing the underlying task. A deterministic CPU-only contract repair confirms that the undercount is partially recoverable. These findings imply that official short-answer VQA scores should be accompanied by semantic audits and answer-type diagnostics to remain interpretable.
Read the full abstract ▾
All authors Guanhua Ye, Niu Jingbin, Yan Li, Meiyu Liang, Zhe Xue, Yingxia Shao, Yawen Li
Computer Vision
Zhiyan Zhang, Peipei Song, Jinpeng Hu et al. (+3 more)
Video emotion analysis is typically framed as a static classification problem, treating each clip as an independent labeled unit. However, such a formulation overlooks a key psychological fact:
emotions change as a result of cumulative reactions to consecutive causal events. To bridge this gap, we introduce Dynamic Affective
Reasoning, the first large-scale
benchmark for viewer-centric affect transitions and causal
reasoning over consecutive
video events. DAR contains 15,087
videos and 36,908 event-aligned affective segments annotated with 27
emotion categories. Unlike existing
video-based
emotion datasets, DAR presents a new viewer-centric perspective on fine-grained emotional expressions and transitions, and provides dense, temporally grounded, and causally explicit
reasoning chains. Based on DAR, we formally define three challenging tasks: affective
segmentation, fine-grained
emotion classification, and affective
reasoning. Complementing this
benchmark, we propose DAR-R1, a two-stage framework that combines supervised fine-tuning with Group Relative Policy Optimization. Experiments across 10+ MLLMs show that DAR-R1 sets a new state-of-the-art for dynamic affective
reasoning, in terms of both emotional localization and affective
reasoning. Project page: https://github.com/Zhang-Zhiyan/DAR.
Read the full abstract ▾
All authors Zhiyan Zhang, Peipei Song, Jinpeng Hu, Jingyang Jia, Xun Yang, Xiaojun Chang
Computer Vision
CoSAG: Compact Semantic Anchor Gaussians via Training-Free Rate-Distortion Coding
Yuang Jia, Jinlong Wang, Junhong Lin et al. (+2 more)
Open-vocabulary 3D scene understanding is commonly achieved by
embedding 2D
vision-
language features such as CLIP into a 3D Gaussian Splatting scene, turning it into a text-queryable semantic field. However, attaching a high-dimensional feature to each of millions of Gaussians inflates a single scene to gigabytes, which makes storage and deployment the real bottleneck of these fields. Existing compact methods each learn and ship a per-scene codec, an autoencoder, a quantized codebook, or a distilled feature field, entangling field construction with field storage and never compressing the per-Gaussian assignment that holds the bulk of the cost. We argue that construction and storage should be decoupled, and that storage is a rate-distortion problem over the per-Gaussian binding to a small anchor table, a structure no prior open-vocabulary method compresses. We present CoSAG, which constructs the field without any per-scene training through a closed-form transmittance-weighted lift, spatially grounded semantic anchors, and multi-view denoising, and stores it with a spatially predictive
entropy coder that ships no decoder. Because the anchors are spatially grounded, the binding is predictable and therefore highly compressible. The transmittance-weighted lift and multi-view denoising yield a clean, view-consistent assignment, so the
entropy coder spends almost no rate on correcting noise and instead codes only the residual against its spatial prediction. CoSAG reaches sub-megabyte storage while matching or exceeding the state of the art across the 2D-rendered, 3D-selection, and dense-LSeg protocols, reducing field size by 37 to 76x relative to LangSplatV2 at higher accuracy.
Read the full abstract ▾
All authors Yuang Jia, Jinlong Wang, Junhong Lin, Ruiting Dai, Wei Gao
Computer Vision
PhenoEmbed: Self-Supervised Multispectral UAV Time-Series Embeddings for Individual Tree Crown Phenology
Taimur Khan
Tree crowns are a challenging target for resilient AI because they are not static objects: their spectral response, internal texture, translucency, and apparent boundaries change substantially across the growing season. We develop PhenoEmbed, a self-supervised crown-centric temporal
embedding model trained with contrastive and masked reconstruction objectives on HeideBench, an 18-date
UAV multispectral time-series
benchmark for forest crown phenology in Dölauer Heide. The model treats seasonal crown dynamics as phenological appearance change driven by leaf emergence, canopy closure, senescence, and leaf-off conditions. Segmented tree crown polygons are retained as object anchors to extract aligned crown-centered crops through time, allowing one 256-dimensional vector summarizing seasonal crown appearance to be learned per tree. On 5,885 crop-safe crowns, the exported
embeddings show structured low-dimensional organization, with the first two principal components explaining 25.1% of variance and nearest-neighbor
retrieval producing a median top-1 cosine similarity of 0.946. Compared with handcrafted temporal features and a learned mean-pooling baseline, PhenoEmbed yields substantially more compact nearest-neighbor structure, while ablations show that the contrastive loss, masked reconstruction loss, and explicit seasonal time features each affect the structure of the learned
embedding space. These results support PhenoEmbed as a reusable forest crown representation learner and motivate future downstream tests of whether such features improve tree-level models under seasonal change.
Read the full abstract ▾
Computer Vision
ScratNet: A Swin-Based Multi-Scale Dilated Network with Precision Refinement for Semiconductor Scratch Segmentation
Sachin Ranjan and Hoon Kim
Surface scratch defects in
semiconductor manufacturing pose significant challenges due to their irregular shapes, low contrast, and varying scales. Traditional inspection methods often struggle to detect such defects reliably, especially in complex imaging scenarios. While deep learning approaches based on Convolutional Neural Networks (CNNs) have improved accuracy, they often fail to capture fine-grained edge details. To address these limitations, we propose ScratNet, a novel end-to-end scratch
segmentation framework that integrates a modified Swin
Transformer backbone with a tailored decoder. The decoder incorporates a Multi-Scale Dilated Aggregation (MDA) module to capture both local and global context, a Stem Integration Module (SIM) to restore spatial detail, and a Precision Refinement (PR) branch that enhances boundary sharpness using anisotropic convolutions. Through this stage-adaptive feature aggregation and boundary-aware refinement, ScratNet achieves superior accuracy on thin and irregular defects. Extensive experiments demonstrate that ScratNet consistently outperforms existing methods, providing a scalable and robust solution for automated scratch inspection in high-precision manufacturing.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning
Wenyuan Wang, Lianyu Hu, Hao Wang et al. (+1 more)
Video-
language models (VLMs) have achieved remarkable performance on
video understanding and visual question answering, yet they remain unreliable in
reasoning about physical plausibility, where understanding object interactions, causal dynamics, and fundamental physical principles is essential. This limitation is particularly evident on challenging physical
reasoning benchmarks, revealing a persistent gap in physical commonsense
reasoning. To address this challenge, we propose PhysMRV, a training-free physical
memory and verification framework for physical plausibility
reasoning. Unlike
retrieval-augmented VLMs that retrieve semantically similar
videos as additional context, PhysMRV transforms training
videos into a Hierarchical
Memory Bank of structured physical knowledge comprising three complementary levels: scene descriptions capturing visual context, physical-event graphs modeling object interactions and causal structure, and physics-rule summaries distilling reusable physical principles and cues. During inference, PhysMRV retrieves physically relevant
memories and leverages their structured physical evidence to guide a frozen VLM in verifying physical plausibility, requiring neither fine-tuning nor parameter updates. We evaluate PhysMRV on three challenging physical
reasoning benchmarks, ImplausiBench, IntPhys2, and GRASP Level 2, across multiple state-of-the-art VLMs. Experimental results demonstrate consistent improvements over direct prompting across diverse VLMs and evaluation
benchmarks, showing that structured physical
memories provide an effective and scalable means of enhancing physical plausibility
reasoning without additional training.
Read the full abstract ▾
All authors Wenyuan Wang, Lianyu Hu, Hao Wang, Yang Liu
Computer Vision
BiLoG-Net: A Bi-Context Location-Guided Network for Breast Mass Segmentation and Malignancy Classification in Mammography
Abu Fatema Mohammad Abdun Noor, Md Imam Ahasan, Md Samiul Ahasan et al. (+3 more)
Breast cancer remains the most commonly diagnosed malignancy among women worldwide, yet accurate detection and characterization of breast masses in mammography remain challenging due to subtle intensity variations, heterogeneous tissue densities, and indistinct lesion boundaries that complicate radiological interpretation. To address these limitations, we propose BiLoG-Net, a deep learning framework that jointly performs breast mass
segmentation and malignancy classification through bi-context location-aware feature modeling and
segmentation-guided
attention mechanisms. Our architecture integrates a novel encoder-decoder paradigm with Fire-based feature extraction, lightweight global and local feature enhancement modules, and adaptive location-aware gating to simultaneously capture long-range contextual dependencies and fine-grained boundary-sensitive details. Unlike conventional multi-stage pipelines, our tightly coupled multi-task design enables mutual reinforcement between pixel-level localization and image-level diagnosis, reducing error propagation while producing spatially grounded malignancy predictions. Evaluated on CBIS-DDSM and INBreast
benchmarks, BiLoG-Net achieves state-of-the-art performance with Dice scores of 94.20% and 93.10%, classification accuracies of 95.20% and 93.60%, and AUC values of 97.10% and 96.00%, respectively, substantially outperforming existing CNN and
transformer-based baselines. By combining precise boundary delineation with reliable malignancy assessment in a single end-to-end model, this work holds strong potential for clinical computer-aided detection systems, helping radiologists prioritize suspicious cases and improve screening efficiency in busy clinical settings.
Read the full abstract ▾
All authors Abu Fatema Mohammad Abdun Noor, Md Imam Ahasan, Md Samiul Ahasan, Kah Ong Michael Goh, S M Hasan Mahmud, Raihana Zannat
Computer Vision
EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generation
Dexiang Hong, Yijie Guo, Weidong Chen et al. (+4 more)
Emotion-aware artistic image generation requires an image to match the input
prompt, follow the specified artistic style, and convey the target
emotion. In this challenge, the main difficulty is that the visual and affective attributes available in the training data are not explicitly provided at test time. Without these attributes, the generator has to decide not only what to depict, but also how the target
emotion should be expressed through color, lighting, brushwork, composition, line, and layout. This creates a control gap between the available test
prompt and the fine-grained conditions needed for
emotion-aware artistic generation. To bridge this gap, we propose EmoStyle, a Z-Image-based framework that converts the input
prompt into a structured generation state. An LLM reasoner first predicts affective cues (valence-arousal, dominant
emotion, and therapeutic-effect labels) and an aspect-ratio decision. Instead of using these predictions only as additional
prompt text, we encode the affective fields into an affective condition vector and inject it into the denoising blocks through AdaLN-style modulation. This allows the inferred control variables to directly guide the generation of intermediate features. Since emotional expression is also style-dependent, we further train a dedicated LoRA adapter for each artistic style bucket and select the corresponding expert during inference, enabling the same affective cues to be rendered with bucket-specific priors for color, texture, brushwork, and composition. Finally, a lightweight VLM-guided candidate selection step ranks the generated images based on
prompt alignment, style consistency, emotional expression, and visual quality. In Track 1 of the AffectiveArt Challenge 2026, our USTC\_PI\_LAB\_TEAM submission achieved first place.
Read the full abstract ▾
All authors Dexiang Hong, Yijie Guo, Weidong Chen, Xinyan Liu, Zixuan Zou, Zhendong Mao, Yongdong Zhang
Computer Vision
REVA-PO: Stabilizing Reinforcement Learning for Chest X-ray Report Generation
Li Guo, Anas M. Tahir and Z. Jane Wang
Automated chest X-ray report generation has recently benefited from reinforcement learning (RL) and large
language models. However, RL training often suffers from instability or limited exploration due to fixed Kullback-Leibler (KL) regularization and a static reference policy that accumulates KL pressure over time. We propose Response-Weighted and Validation-Anchored Policy Optimization (REVA-PO), a RL framework that stabilizes long-term training via Response-Weighted Regularization (RER) and Validation-Anchored Policy Reset (VAPR). RER dynamically adjusts per-response KL weights based on advantage and reference-policy
entropy, relaxing constraints for high-quality responses while tightening them for low-quality ones. Complementarily, VAPR periodically synchronizes the reference and current policies to the best validation checkpoint, resetting accumulated regularization pressure to expand the viable exploration space. To ensure a robust starting point, we employ a three-stage pipeline consisting of warm-up training, classifier-guided supervised fine-tuning, and RL. Extensive evaluations on MIMIC-CXR and IU-Xray demonstrate that REVA-PO sets new state-of-the-art
benchmarks in both linguistic quality and clinical accuracy. Notably, BLEU-4 improves by 5.1% on MIMIC-CXR and 3.6% on IU-Xray, while CheXpert F1 and RadGraph F1 scores increase by 4.5% and 12.8%, respectively, over prior leading methods. The code is publicly available at https://github.com/LiGuo12/REVA_PO/.
Read the full abstract ▾