Computer Vision
MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models
Eunkyu Park, Wesley Hanwen Deng, Cheyon Jin et al. (+8 more)
Vision-
Language Models increasingly make socially consequential judgments from image-text inputs, yet current evaluations often stop at verdict-level accuracy: whether the final label is human-aligned, not whether it follows from the correct evidence. We introduce MM-SCALE (
Multimodal Moral SCALE), a
benchmark for evidence-grounded moral judgment built around a design choice: each image is paired with multiple action scenarios, so models must compare how different actions interact with the same visual context rather than score images in isolation. MM-SCALE contains 8,444 image contexts and 21,977 action scenarios, each annotated with a 5-point moral acceptability rating and a modality-grounding label indicating whether the human judgment relied on text, image, or both. Three tasks evaluate whether models (i) assign calibrated scalar moral scores, (ii) preserve human orderings within a shared visual context, and (iii) ground rationales in the evidence source humans found critical to judgment. Across models, aggregate metrics overstate
alignment: NDCG@5 remains high across models, while within-image pairwise accuracy remains only 0.53-0.62 even on scenario pairs whose human mean ratings differ by at least one point. CoT has inconsistent effects on calibration and does not reliably improve within-image ordering. Evidence grounding evaluations further show that models often describe visual context without making it critical for the verdict. These results reveal an evidence-grounding failure in current VLMs and position MM-SCALE as an evaluation testbed for moral judgment under shared image contexts. This
dataset includes potentially harmful and sensitive visual content. Images are intended solely for evaluation.
Read the full abstract ▾
All authors Eunkyu Park, Wesley Hanwen Deng, Cheyon Jin, Matheus Kunzler Maldaner, Jordan Wheeler, Jason I. Hong, Hong Shen, Adam Perer, Ken Holstein, Motahhare Eslami, Gunhee Kim
Computer Vision
SPWOOD: Sparse Partial Weakly-Supervised Oriented Object Detection
Wei Zhang, Xiang Liu, Ningjing Liu et al. (+4 more)
A consistent trend throughout the research of oriented object detection has been the pursuit of maintaining comparable performance with fewer and weaker annotations. This is particularly crucial in the remote sensing domain, where the dense object distribution and a wide variety of categories contribute to prohibitively high costs. Based on the supervision level, existing oriented object detection
algorithms can be broadly grouped into fully supervised, semi-supervised, and weakly supervised methods. Within the scope of this work, we further categorize them to include sparsely supervised and partially weakly-supervised methods. To address the challenges of large-scale labeling, we introduce the first Sparse Partial Weakly-Supervised Oriented Object Detection framework, designed to efficiently leverage only a few sparse weakly-labeled data and plenty of unlabeled data. Our framework incorporates three key innovations: (1) We design a Sparse-annotation-Orientation-and-Scale-aware Student (SOS-Student) model to separate unlabeled objects from the background in a sparsely-labeled setting, and learn orientation and scale information from orientation-agnostic or scale-agnostic weak annotations. (2) We construct a novel Multi-level Pseudo-label Filtering strategy that leverages the distribution of model predictions, which is informed by the model's multi-layer predictions. (3) We propose a unique sparse partitioning approach, ensuring equal treatment for each category. Extensive experiments on the DOTA and DIOR
datasets show that our framework achieves a significant performance gain over traditional oriented object detection methods mentioned above, offering a highly cost-effective solution. Our code is publicly available at https://github.com/VisionXLab/SPWOOD.
Read the full abstract ▾
All authors Wei Zhang, Xiang Liu, Ningjing Liu, Mingxin Liu, Wei Liao, Chunyan Xu, Xue Yang
Computer Vision
Multi-Objective Optimization for Synthetic-to-Real Style Transfer
Estelle Chigot, Thomas Oberlin, Manon Huguenin et al. (+1 more)
Semantic
segmentation networks require large amounts of pixel-level annotated data, which are costly to obtain for real-world images. Computer graphics engines can generate synthetic images alongside their ground-truth annotations. However, models trained on such images can perform poorly on real images due to the domain gap between real and synthetic images. Style transfer methods can reduce this difference by applying a realistic style to synthetic images. Choosing effective data transformations and their sequence is difficult due to the large combinatorial search space of style transfer operators. Using multi-objective genetic
algorithms, we optimize pipelines to balance structural coherence and style similarity to target domains. We study the use of paired-image metrics on individual image samples during
evolution to enable rapid pipeline evaluation, as opposed to standard distributional metrics that require the generation of many images. After optimization, we evaluate the resulting Pareto front using distributional metrics and
segmentation performance. We apply this approach to standard
datasets in synthetic-to-real domain adaptation: from the
video game GTA5 to real image
datasets Cityscapes and ACDC, focusing on adverse conditions. Results demonstrate that evolutionary
algorithms can propose diverse augmentation pipelines adapted to different objectives. The contribution of this work is the formulation of style transfer as a sequencing problem suitable for evolutionary optimization and the study of efficient metrics that enable feasible search in this space. The source code is available at: https://github.com/echigot/MOOSS.
Read the full abstract ▾
All authors Estelle Chigot, Thomas Oberlin, Manon Huguenin, Dennis Wilson
Computer Vision
Quasi-multimodal-based pathophysiological feature learning for retinal disease diagnosis
Lu Zhang, Huizhen Yu, Zuowei Wang et al. (+4 more)
Retinal diseases spanning a broad spectrum can be effectively identified and diagnosed using complementary signals from
multimodal data. However,
multimodal diagnosis in ophthalmic practice is typically challenged in terms of data heterogeneity, potential invasiveness, registration complexity, and so on. As such, a unified framework that integrates
multimodal data synthesis and fusion is proposed for retinal disease classification and grading. Specifically, the synthesized
multimodal data incorporates fundus fluorescein angiography (FFA), multispectral imaging (MSI), and saliency maps that emphasize latent lesions as well as optic disc/cup regions. Parallel models are independently trained to learn modality-specific representations that capture cross-pathophysiological signatures. These features are then adaptively calibrated within and across modalities to perform information
pruning and flexible integration according to downstream tasks. The proposed learning system is thoroughly interpreted through visualizations in both image and feature spaces. Extensive experiments on two public
datasets demonstrated the superiority of our approach over state-of-the-art ones in the tasks of multi-label classification (F1-score: 0.683, AUC: 0.953) and diabetic retinopathy grading (Accuracy:0.842, Kappa: 0.861). This work not only enhances the accuracy and efficiency of retinal disease screening but also offers a scalable framework for data augmentation across various medical imaging modalities.
Read the full abstract ▾
All authors Lu Zhang, Huizhen Yu, Zuowei Wang, Fu Gui, Yatu Guo, Wei Zhang, Mengyu Jia
Computer Vision
KTV: Keyframes and Key Tokens Selection for Efficient Training-Free Video LLMs
Baiyang Song, Jun Peng, Yuxin Zhang et al. (+3 more)
Training-free
video understanding leverages the strong image comprehension capabilities of pre-trained
vision language models (VLMs) by treating a
video as a sequence of static frames, thus obviating the need for costly
video-specific training. However, this paradigm often suffers from severe visual redundancy and high computational overhead, especially when processing long
videos. Crucially, existing keyframe selection strategies, especially those based on CLIP similarity, are prone to biases and may inadvertently overlook critical frames, resulting in suboptimal
video comprehension. To address these significant challenges, we propose KTV, a novel two-stage framework for efficient and effective training-free
video understanding. In the first stage, KTV performs question-agnostic keyframe selection by clustering frame-level visual features, yielding a compact, diverse, and representative subset of frames that mitigates temporal redundancy. In the second stage, KTV applies key visual
token selection,
pruning redundant or less informative
tokens from each selected keyframe based on
token importance and redundancy, which significantly reduces the number of
tokens fed into the LLM. Extensive experiments on the Multiple-Choice VideoQA task demonstrate that KTV outperforms state-of-the-art training-free baselines while using significantly fewer visual
tokens, e.g., only 504 visual
tokens for a 60-min
video with 10800 frames, achieving $44.8\%$ accuracy on the MLVU-Test
benchmark. In particular, KTV also exceeds several training-based approaches on certain
benchmarks.
Read the full abstract ▾
All authors Baiyang Song, Jun Peng, Yuxin Zhang, Guangyao Chen, Feidiao Yang, Jianyuan Guo
Computer Vision
A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures
Basile Terver, Randall Balestriero, Megi Dervishi et al. (+8 more)
We present EB-JEPA, an open-source library for learning representations and world models using Joint-
Embedding Predictive Architectures (JEPAs). JEPAs learn to predict in representation space rather than pixel space, avoiding the pitfalls of generative modeling while capturing semantically meaningful features suitable for downstream tasks. Our library provides modular, self-contained implementations that illustrate how representation learning techniques developed for image-level self-supervised learning can transfer to
video, where temporal dynamics add complexity, and ultimately to action-conditioned world models, where the model must additionally learn to predict the effects of control inputs. Each example is designed for single-GPU training within a few hours, making energy-based self-supervised learning accessible for research and education. We provide ablations of JEA components on CIFAR-10. Probing these representations yields 91% accuracy, indicating that the model learns useful features. Extending to
video, we include a multi-step prediction example on Moving MNIST that demonstrates how the same principles scale to temporal modeling. Finally, we show how these representations can drive action-conditioned world models, achieving a 97% planning success rate on the Two Rooms
navigation task. Comprehensive ablations reveal the critical importance of each regularization component for preventing representation collapse. Code is available at https://github.com/facebookresearch/eb_jepa.
Read the full abstract ▾
All authors Basile Terver, Randall Balestriero, Megi Dervishi, David Fan, Quentin Garrido, Tushar Nagarajan, Koustuv Sinha, Wancong Zhang, Mike Rabbat, Yann LeCun, Amir Bar
Computer Vision
Refer-Agent: A Collaborative Multi-Agent System with Reasoning and Reflection for Referring Video Object Segmentation
Haichao Jiang, Tianming Liang, Wei-Shi Zheng et al. (+1 more)
Referring
Video Object
Segmentation (RVOS) aims to segment objects in
videos based on textual queries. Current methods mainly rely on large-scale supervised fine-tuning (SFT) of Multi-modal Large
Language Models (MLLMs). However, this paradigm suffers from heavy data dependence and limited scalability against the rapid
evolution of MLLMs. Although recent zero-shot approaches offer a flexible alternative, their performance remains significantly behind SFT-based methods, due to the straightforward workflow designs. To address these limitations, we propose Refer-
Agent, a collaborative multi-
agent system with alternating
reasoning-reflection mechanisms. This system decomposes RVOS into step-by-step
reasoning process. During
reasoning, we introduce a Coarse-to-Fine frame selection strategy to ensure the frame diversity and textual relevance, along with a Dynamic Focus Layout that adaptively adjusts the
agent's visual focus. Furthermore, we propose a Chain-of-Reflection mechanism, which employs a Questioner-Responder pair to generate a self-reflection chain, enabling the system to verify intermediate results and generates feedback for next-round
reasoning refinement. Extensive experiments on five challenging
benchmarks demonstrate that Refer-
Agent significantly outperforms state-of-the-art methods, including both SFT-based models and zero-shot approaches. Moreover, Refer-
Agent is flexible and enables fast integration of new MLLMs without any additional fine-tuning costs. Code will be released at https://github.com/iSEE-Laboratory/Refer-
Agent.
Read the full abstract ▾
All authors Haichao Jiang, Tianming Liang, Wei-Shi Zheng, Jian-Fang Hu
Computer Vision
TIPS Over Tricks: Simple Prompts for Effective Zero-shot Anomaly Detection
Alireza Salehi, Ehsan Karami, Sepehr Noey et al. (+4 more)
Anomaly detection identifies departures from expected behavior in safety-critical settings. When target-domain normal data are unavailable, zero-shot anomaly detection (ZSAD) leverages
vision-
language models (VLMs). However, CLIP's coarse image-text
alignment limits both localization and detection due to (i) spatial misalignment and (ii) weak sensitivity to fine-grained anomalies; prior works compensate with complex auxiliary modules yet largely overlook the choice of backbone. We revisit the backbone and use TIPS-a VLM trained with spatially aware objectives. While TIPS alleviates CLIP's issues, it exposes a distributional gap between global and local features. We address this with decoupled
prompts-fixed for image-level detection and learnable for pixel-level localization-and by injecting local evidence into the global score. Without CLIP-specific tricks, our TIPS-based pipeline improves image-level performance by 1.1-3.9% and pixel-level by 1.5-6.9% across seven industrial
datasets, delivering strong generalization with a lean architecture. Code is available at github.com/AlirezaSalehy/Tipsomaly.
Read the full abstract ▾
All authors Alireza Salehi, Ehsan Karami, Sepehr Noey, Sahand Noey, Makoto Yamada, Reshad Hosseini, Mohammad Sabokrou
Computer Vision
High-Resolution Underwater Camouflaged Object Detection: GBU-UCOD Dataset and Topology-Aware and Frequency-Decoupled Networks
Wenji Wu, Shuo Ye, Yiyu Liu et al. (+3 more)
Underwater Camouflaged Object Detection (UCOD) is a challenging task due to the extreme visual similarity between targets and backgrounds across varying marine depths. Existing methods often struggle with topological fragmentation of slender creatures in the deep sea and the subtle feature extraction of transparent organisms. In this paper, we propose DeepTopo-Net, a novel framework that integrates
topology-aware modeling with frequency-decoupled perception. To address physical degradation, we design the Water-Conditioned Adaptive Perceptor (WCAP), which employs Riemannian metric tensors to dynamically deform convolutional sampling fields. Furthermore, the Abyssal-
Topology Refinement Module (ATRM) is developed to maintain the structural connectivity of spindly targets through skeletal priors. Specifically, we first introduce GBU-UCOD, the first high-resolution (2K)
benchmark tailored for marine vertical zonation, filling the data gap for hadal and abyssal zones. Extensive experiments on MAS3K, RMAS, and our proposed GBU-UCOD
datasets demonstrate that DeepTopo-Net achieves state-of-the-art performance, particularly in preserving the morphological integrity of complex
underwater patterns. The
datasets and codes will be released at https://github.com/Wuwenji18/GBU-UCOD.
Read the full abstract ▾
All authors Wenji Wu, Shuo Ye, Yiyu Liu, Jiguang He, Zhuo Wang, Zitong Yu
Computer Vision
SlowFocus: Enhancing Fine-grained Temporal Understanding in Video LLM
Ming Nie, Dan Ding, Chunwei Wang et al. (+4 more)
Large
language models (LLMs) have demonstrated exceptional capabilities in text understanding, which has paved the way for their expansion into
video LLMs (Vid-LLMs) to analyze
video data. However, current Vid-LLMs struggle to simultaneously retain high-quality frame-level semantic information (i.e., a sufficient number of
tokens per frame) and comprehensive
video-level temporal information (i.e., an adequate number of sampled frames per
video). This limitation hinders the advancement of Vid-LLMs towards fine-grained
video understanding. To address this issue, we introduce the SlowFocus mechanism, which significantly enhances the equivalent sampling frequency without compromising the quality of frame-level visual
tokens. SlowFocus begins by identifying the query-related temporal segment based on the posed question, then performs dense sampling on this segment to extract local high-frequency features. A multi-frequency mixing
attention module is further leveraged to aggregate these local high-frequency details with global low-frequency contexts for enhanced temporal comprehension. Additionally, to tailor Vid-LLMs to this innovative mechanism, we introduce a set of training strategies aimed at bolstering both temporal grounding and detailed temporal
reasoning capabilities. Furthermore, we establish FineAction-CGR, a
benchmark specifically devised to assess the ability of Vid-LLMs to process fine-grained temporal understanding tasks. Comprehensive experiments demonstrate the superiority of our mechanism across both existing public
video understanding
benchmarks and our proposed FineAction-CGR.
Read the full abstract ▾
All authors Ming Nie, Dan Ding, Chunwei Wang, Yuanfan Guo, Jianhua Han, Hang Xu, Li Zhang
Computer Vision
ELIQ: A Label-Free Framework for Quality Assessment of Evolving AI-Generated Images
Xinyue Li, Zhiming Xu, Min Tang et al. (+5 more)
Generative text-to-image models are advancing at an unprecedented pace, continuously shifting the perceptual quality ceiling and rendering previously collected labels unreliable for newer generations. To address this, we present ELIQ, a Label-free Framework for Quality Assessment of Evolving AI-generated Images. Specifically, ELIQ focuses on visual quality and
prompt-image
alignment, automatically constructs positive and aspect-specific negative pairs to cover both conventional distortions and AIGC-specific distortion modes, enabling transferable supervision without human annotations. Building on these pairs, ELIQ adapts a pre-trained
multimodal model into a quality-aware critic via instruction tuning and predicts two-dimensional quality using lightweight gated fusion and a Quality Query
Transformer. Experiments across multiple
benchmarks demonstrate that ELIQ consistently outperforms existing label-free methods, generalizes from AI-generated content (AIGC) to user-generated content (UGC) scenarios without modification, and paves the way for scalable and label-free quality assessment under continuously evolving generative models. The code will be released upon publication.
Read the full abstract ▾
All authors Xinyue Li, Zhiming Xu, Min Tang, Zhaolin Cai, Sijing Wu, Xiongkuo Min, Yitong Chen, Guangtao Zhai
Computer Vision
Cut to the Mix: Simple Data Augmentation Outperforms Elaborate Ones in Limited Organ Segmentation Datasets
Chang Liu, Fuxin Fan, Annette Schwarz et al. (+1 more)
Multi-organ
segmentation is a widely applied clinical routine and automated organ
segmentation tools dramatically improve the pipeline of the radiologists. Recently, deep learning (DL) based
segmentation models have shown the capacity to accomplish such a task. However, the training of the
segmentation networks requires large amount of data with manual annotations, which is a major concern due to the data scarcity from clinic. Working with limited data is still common for researches on novel imaging modalities. To enhance the effectiveness of DL models trained with limited data, data augmentation (DA) is a crucial regularization technique. Traditional DA (TDA) strategies focus on basic intra-image operations, i.e. generating images with different orientations and intensity distributions. In contrast, the interimage and object-level DA operations are able to create new images from separate individuals. However, such DA strategies are not well explored on the task of multi-organ
segmentation. In this paper, we investigated four possible inter-image DA strategies: CutMix, CarveMix, ObjectAug and AnatoMix, on two organ
segmentation datasets. The result shows that CutMix, CarveMix and AnatoMix can improve the average dice score by 4.9, 2.0 and 1.9, compared with the state-of-the-art nnUNet without DA strategies. These results can be further improved by adding TDA strategies. It is revealed in our experiments that Cut-Mix is a robust but simple DA strategy to drive up the
segmentation performance for multi-organ
segmentation, even when CutMix produces intuitively 'wrong' images. Our implementation is publicly available for future
benchmarks.
Read the full abstract ▾
All authors Chang Liu, Fuxin Fan, Annette Schwarz, Andreas Maier
Computer Vision · Cross-listed from cs.RO
AffordanceGrasp-R1:Leveraging Reasoning-Based Affordance Segmentation with Reinforcement Learning for Robotic Grasping
Dingyi Zhou, Mu He, Zhuowei Fang et al. (+4 more)
We introduce AffordanceGrasp-R1, a
reasoning-driven affordance
segmentation framework for robotic grasping that combines a chain-of-thought (CoT) cold-start strategy with reinforcement learning to enhance deduction and spatial grounding. In addition, we redesign the grasping pipeline to be more context-aware by generating grasp candidates from the global scene point cloud and subsequently filtering them using instruction-conditioned affordance masks. Extensive experiments demonstrate that AffordanceGrasp-R1 consistently outperforms state-of-the-art (SOTA) methods on
benchmark datasets, and real-world robotic grasping evaluations further validate its robustness and generalization under complex
language-conditioned manipulation scenarios.
Read the full abstract ▾
All authors Dingyi Zhou, Mu He, Zhuowei Fang, Xiangtong Yao, Yinlong Liu, Alois Knoll, Hu Cao
Computer Vision
Constrained Dynamic Gaussian Splatting
Zihan Zheng, Zhenlong Wu, Xuanxuan Wang et al. (+5 more)
While Dynamic Gaussian Splatting enables high-fidelity 4D reconstruction, its deployment is severely hindered by a fundamental dilemma: unconstrained densification leads to excessive
memory consumption incompatible with edge devices, whereas heuristic
pruning fails to achieve optimal rendering quality under preset Gaussian budgets. In this work, we propose Constrained Dynamic Gaussian Splatting (CDGS), a novel framework that formulates dynamic scene reconstruction as a budget-constrained optimization problem to enforce a strict, user-defined Gaussian budget during training. Our key insight is to introduce a differentiable budget controller as the core optimization driver. Guided by a multi-modal unified importance score, this controller fuses
geometric, motion, and perceptual cues for precise capacity regulation. To maximize the utility of this fixed budget, we further introduce an adaptive static-dynamic allocation strategy that separates the Gaussian representation into static and dynamic branches and distributes the shared global capacity between them according to motion complexity. Furthermore, we implement a three-phase training strategy to seamlessly integrate these constraints, ensuring precise adherence to the target count. After training, a dual-mode hybrid compression scheme further reduces storage overhead. CDGS therefore not only strictly adheres to the specified Gaussian-count budget (error<2%) but also achieves favorable rate-distortion performance. Extensive experiments demonstrate that CDGS delivers optimal rendering quality under varying capacity limits and favorable rate-distortion performance, achieving over 3x model compression compared with the state-of-the-art method Ex4DGS.
Read the full abstract ▾
All authors Zihan Zheng, Zhenlong Wu, Xuanxuan Wang, Houqiang Zhong, Xiaoyun Zhang, Qiang Hu, Guangtao Zhai, Wenjun Zhang
Computer Vision
PnP-U3D: Plug-and-Play 3D Framework Bridging Autoregression and Diffusion for Unified Understanding and Generation
Yongwei Chen, Tianyi Wei, Yushi Lan et al. (+4 more)
The rapid progress of large
multimodal models has inspired efforts toward unified frameworks that couple understanding and generation. While such paradigms have shown remarkable success in 2D, extending them to 3D remains largely underexplored. Existing attempts to unify 3D tasks under a single autoregressive (AR) paradigm lead to significant performance degradation due to forced signal quantization and prohibitive training cost. Our key insight is that the essential challenge lies not in enforcing a unified autoregressive paradigm, but in enabling effective information interaction between generation and understanding while minimally compromising their inherent capabilities and leveraging pretrained models to reduce training cost. Guided by this perspective, we present the first unified framework for 3D understanding and generation that combines autoregression with
diffusion. Specifically, we adopt an autoregressive next-
token prediction paradigm for 3D understanding, and a continuous
diffusion paradigm for 3D generation. A lightweight
transformer bridges the feature space of large
language models and the conditional space of 3D
diffusion models, enabling effective cross-modal information exchange while preserving the priors learned by standalone models. Extensive experiments demonstrate that our framework achieves state-of-the-art performance across diverse 3D understanding and generation
benchmarks, while also excelling in 3D editing tasks. These results highlight the potential of unified AR+
diffusion models as a promising direction for building more general-purpose 3D intelligence.
Read the full abstract ▾
All authors Yongwei Chen, Tianyi Wei, Yushi Lan, Zhaoyang Lyu, Shangchen Zhou, Xudong Xu, Xingang Pan
Computer Vision · Cross-listed from cs.LG
Robust Representation Learning in Masked Autoencoders
Anika Shrivastava, Renu Rameshan and Samar Agnihotri
Masked Autoencoders (MAEs) achieve impressive performance in image classification tasks, yet the internal representations they learn remain less understood. This work started as an attempt to understand the strong downstream classification performance of MAE. In this process we discover that representations learned with the pretraining and fine-tuning, are quite robust – demonstrating a good classification performance in the presence of degradations, such as blur and occlusions. Through layer-wise analysis of
token embeddings, we show that pretrained MAE progressively constructs its latent space in a class-aware manner across network depth:
embeddings from different classes lie in subspaces that become increasingly separable. We further observe that MAE exhibits early and persistent global
attention across encoder layers, in contrast to standard
Vision Transformers (ViTs). To quantify feature robustness, we introduce two sensitivity indicators: directional
alignment between clean and perturbed
embeddings, and head-wise retention of active features under degradations. These studies help establish the robust classification performance of MAEs.
Read the full abstract ▾
Computer Vision
Interpretable Logical Anomaly Classification via Constraint Decomposition and Instruction Fine-Tuning
Xufei Zhang, Xinjiao Zhou, Ziling Deng et al. (+2 more)
Logical anomalies are violations of predefined constraints on object quantity, spatial layout, and compositional relationships in industrial images. While prior work largely treats anomaly detection as a binary decision, such formulations cannot indicate which logical rule is broken and therefore offer limited value for quality assurance. We introduce Logical Anomaly Classification (LAC), a task that unifies anomaly detection and fine-grained violation classification in a single inference step. To tackle LAC, we propose LogiCls, a
vision-
language framework that decomposes complex logical constraints into a sequence of verifiable subqueries. We further present a data-centric instruction synthesis pipeline that generates chain-of-thought (CoT) supervision for these subqueries, coupling precise grounding annotations with diverse image-text augmentations to adapt
vision language models (VLMs) to logic-sensitive
reasoning. Training is stabilized by a difficulty-aware resampling strategy that emphasizes challenging subqueries and long tail constraint types. Extensive experiments demonstrate that LogiCls delivers robust, interpretable, and accurate industrial logical anomaly classification, providing both the predicted violation categories and their evidence trails.
Read the full abstract ▾
All authors Xufei Zhang, Xinjiao Zhou, Ziling Deng, Dongdong Geng, Jianxiong Wang
Computer Vision
Semantic Routing: Exploring Multi-Layer LLM Feature Weighting for Diffusion Transformers
Bozhou Li, Yushuo Guan, Haolin Li et al. (+7 more)
Recent DiT-based text-to-image models increasingly adopt LLMs as text encoders, yet text conditioning remains largely static and often utilizes only a single LLM layer, despite pronounced semantic hierarchy across LLM layers and non-stationary denoising dynamics over both
diffusion time and network depth. To better match the dynamic process of DiT generation and thereby enhance the
diffusion model's generative capability, we introduce a unified normalized convex fusion framework equipped with lightweight gates to systematically organize multi-layer LLM hidden states via time-wise, depth-wise, and joint fusion. Experiments establish Depth-wise Semantic Routing as the superior conditioning strategy, consistently improving text-image
alignment and compositional generation (e.g., +9.97 on the GenAI-Bench Counting task). Conversely, we find that purely time-wise fusion can paradoxically degrade visual generation fidelity. We attribute this to a train-inference trajectory mismatch: under classifier-free guidance, nominal timesteps fail to track the effective SNR, causing semantically mistimed feature injection during inference. Overall, our results position depth-wise routing as a strong and effective baseline and highlight the critical need for trajectory-aware signals to enable robust time-dependent conditioning.
Read the full abstract ▾
All authors Bozhou Li, Yushuo Guan, Haolin Li, Bohan Zeng, Yiyan Ji, Yue Ding, Pengfei Wan, Kun Gai, Yuanxing Zhang, Wentao Zhang
Computer Vision
Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance
Yingjie Zhu, Xuefeng Bai, Kehai Chen et al. (+4 more)
Reasoning over table images remains challenging for Large
Vision-
Language Models (LVLMs) due to complex layouts and tightly coupled structure-content information. Existing solutions often depend on expensive supervised training, reinforcement learning, or external tools, limiting efficiency and scalability. This work addresses a key question: how to adapt LVLMs to table
reasoning with minimal annotation and no external tools? Specifically, we first introduce DiSCo, a Disentangled Structure-Content
alignment framework that explicitly separates structural abstraction from semantic grounding during
multimodal alignment, efficiently adapting LVLMs to tables structures. Building on DiSCo, we further present Table-GLS, a Global-to-Local Structure-guided
reasoning framework that performs table
reasoning via structured exploration and evidence-grounded inference. Extensive experiments across diverse
benchmarks demonstrate that our framework efficiently enhances LVLM's table understanding and
reasoning capabilities, particularly generalizing to unseen table structures. Our data and code are available at https://github.com/AAAndy-Zhu/TableVLM.
Read the full abstract ▾
All authors Yingjie Zhu, Xuefeng Bai, Kehai Chen, Yang Xiang, Youcheng Pan, Xiaoqiang Zhou, Min Zhang
Computer Vision · Cross-listed from cs.LG
Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Experts
Meng Lou, Yunxiang Fu and Yizhou Yu
Continual learning, especially class-incremental learning (CIL), on the basis of a pre-trained model (PTM) has garnered substantial research interest in recent years. However, how to effectively learn both discriminative and comprehensive feature representations while maintaining stability and plasticity over very long task sequences remains an open problem. We propose CaRE, a scalable {C}ontinual Le{a}rner with efficient Bi-Level {R}outing Mixture-of-{E}xperts (BR-MoE). The core idea of BR-MoE is a bi-level routing mechanism: a router selection stage that dynamically activates relevant task-specific routers, followed by an expert routing phase that dynamically activates and aggregates experts, aiming to inject discriminative and comprehensive representations into every intermediate network layer. On the other hand, we introduce a challenging
dataset, OmniBenchmark-1K, for CIL performance evaluation on very long task sequences with hundreds of tasks. Extensive experiments show that CaRE demonstrates leading performance across a variety of
datasets and task settings, including commonly used CIL
datasets with classical CIL settings (e.g., 5-20 tasks). To the best of our knowledge, CaRE is the first continual learner that scales to very long task sequences (ranging from 100 to over 300 non-overlapping tasks), while outperforming all baselines by a large margin on such task sequences. We hope that this work will inspire further research into continual learning over extremely long task sequences. Code and
dataset are publicly released at https://github.com/LMMMEng/CaRE.
Read the full abstract ▾
Computer Vision
Inlier-Centric Post-Training Quantization for Object Detection Models
Minsu Kim, Dongyeun Lee, Jaemyung Yu et al. (+3 more)
Object detection is pivotal in computer
vision, yet its immense computational demands make deployment slow and power-hungry, motivating quantization. However, task-irrelevant morphologies such as background clutter and sensor noise induce redundant activations (or anomalies). These anomalies expand activation ranges and skew activation distributions toward task-irrelevant responses, complicating bit allocation and weakening the preservation of informative features. Without a clear criterion to distinguish anomalies, suppressing them can inadvertently discard useful information. To address this, we present InlierQ, an inlier-centric post-training quantization approach that separates anomalies from informative inliers. InlierQ computes gradient-aware volume saliency scores, classifies each volume as an inlier or anomaly, and fits a posterior distribution over these scores using the Expectation-Maximization (EM)
algorithm. This design suppresses anomalies while preserving informative features. InlierQ is label-free, drop-in, and requires only 64 calibration samples. Experiments on the COCO and nuScenes
benchmarks show consistent reductions in quantization error for camera-based (2D and 3D) and
LiDAR-based (3D) object detection.
Read the full abstract ▾
All authors Minsu Kim, Dongyeun Lee, Jaemyung Yu, Jiwan Hur, Giseop Kim, Junmo Kim
Computer Vision
Contextualized Visual Personalization in Vision-Language Models
Yeongtak Oh, Sangwon Yu, Junsung Park et al. (+3 more)
Despite recent progress in
vision-
language models (VLMs), existing approaches often fail to generate personalized responses based on the user's specific experiences, as they lack the ability to associate visual inputs with a user's accumulated visual-textual context. We newly formalize this challenge as contextualized visual personalization, which requires the visual recognition and textual
retrieval of personalized visual experiences by VLMs when interpreting new images. To address this issue, we propose CoViP, a unified framework that treats personalized image captioning as a core task for contextualized visual personalization and improves this capability through reinforcement-learning-based post-training and caption-augmented generation. We further introduce diagnostic evaluations that explicitly rule out textual shortcut solutions and verify whether VLMs truly leverage visual context. Extensive experiments demonstrate that existing open-source and proprietary VLMs exhibit substantial limitations, while CoViP not only improves personalized image captioning but also yields holistic gains across downstream personalization tasks. These results highlight CoViP as a crucial stage for enabling robust and generalizable contextualized visual personalization.
Read the full abstract ▾
All authors Yeongtak Oh, Sangwon Yu, Junsung Park, Han Cheol Moon, Jisoo Mok, Sungroh Yoon
Computer Vision
Hierarchical Concept-to-Appearance Guidance for Multi-Subject Image Generation
Yijia Xu, Zihao Wang, Haokun Gui et al. (+1 more)
Multi-subject image generation aims to synthesize images that faithfully preserve the identities of multiple reference subjects while following textual instructions. However, existing methods often suffer from identity inconsistency and limited compositional control, as they rely on
diffusion models to implicitly associate text
prompts with reference images. In this work, we propose Hierarchical Concept-to-Appearance Guidance (CAG), a framework that provides explicit, structured supervision from high-level concepts to fine-grained appearances. At the conceptual level, we introduce a VAE dropout training strategy that randomly omits reference VAE features, encouraging the model to rely more on robust semantic signals from a Visual
Language Model (VLM) and thereby promoting consistent concept-level generation in the absence of complete appearance cues. At the appearance level, we integrate the VLM-derived correspondences into a correspondence-aware masked
attention module within the
Diffusion Transformer (DiT). This module restricts each text
token to attend only to its matched reference regions, ensuring precise attribute binding and reliable multi-subject composition. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the multi-subject image generation, substantially improving
prompt following and subject consistency.
Read the full abstract ▾
All authors Yijia Xu, Zihao Wang, Haokun Gui, Jinshi Cui
Computer Vision · Cross-listed from cs.RO
HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Traffic
Yu-Hsiang Chen, Wei-Jer Chang, Christian Kotulla et al. (+7 more)
We present HetroD, a
dataset and
benchmark for developing autonomous driving systems in heterogeneous environments. HetroD targets the critical challenge of navi- gating real-world heterogeneous traffic dominated by vulner- able road users (VRUs), including pedestrians, cyclists, and motorcyclists that interact with vehicles. These mixed
agent types exhibit complex behaviors such as hook turns, lane splitting, and informal right-of-way negotiation. Such behaviors pose significant challenges for autonomous vehicles but remain underrepresented in existing
datasets focused on structured, lane-disciplined traffic. To bridge the gap, we collect a large- scale drone-based
dataset to provide a holistic observation of traffic scenes with centimeter-accurate annotations, HD maps, and traffic signal states. We further develop a modular toolkit for extracting per-
agent scenarios to support downstream task development. In total, the
dataset comprises over 65.4k high- fidelity
agent trajectories, 70% of which are from VRUs. HetroD supports modeling of VRU behaviors in dense, het- erogeneous traffic and provides standardized
benchmarks for
forecasting, planning, and simulation tasks. Evaluation results reveal that state-of-the-art prediction and planning models struggle with the challenges presented by our
dataset: they fail to predict lateral VRU movements, cannot handle unstructured maneuvers, and exhibit limited performance in dense and multi-
agent scenarios, highlighting the need for more robust approaches to heterogeneous traffic. See our project page for more examples: https://hetroddata.github.io/HetroD/
Read the full abstract ▾
All authors Yu-Hsiang Chen, Wei-Jer Chang, Christian Kotulla, Thomas Keutgens, Steffen Runde, Tobias Moers, Christoph Klas, Wei Zhan, Masayoshi Tomizuka, Yi-Ting Chen
Computer Vision
Dual-Grained Policy Optimization for Hallucination Reduction in Flow-Based RL
Xiaofeng Tan, Jun Liu, Yuanting Fan et al. (+7 more)
Reinforcement Fine-Tuning (RFT) on flow-based models is crucial for
alignment. However, they often introduce visual
hallucinations like over-optimized details and semantic misalignment. This work explains why visual
hallucinations arise and shows how to reduce them. We first investigate RFT methods from a unified perspective and trace
hallucinations to two mechanisms. Limited exploration from
stochastic differential equation (SDE) rollouts yields detail-dominated within-group differences, leading to fine-grained optimization that over-emphasizes local details and neglects global semantics. Trajectory imitation writes reward-selected SDE noise into the velocity field, leading to inconsistent optimization that degrades cross-step consistency. Building on this, we propose DGPO, a general framework to mitigate
hallucinations. Specifically, we design Dynamic Granularity Rollout (DGR) to balance global semantics and local details by dynamically scheduling noise sources. We then introduce a Consistent Policy Gradient Optimization that preserves the consistency by aligning the policy with a more stable prior. Experiments show that DGPO reduces low-/high-level
hallucinations by 49%/38% and improves the average of out-of-domain (OoD) metrics by +4.9% on FLUX.1.dev, compared with the baseline's -0.7%. Project page: https://xiaofeng-tan.github.io/projects/ConsistentRFT/
Read the full abstract ▾
All authors Xiaofeng Tan, Jun Liu, Yuanting Fan, Bin-Bin Gao, Xi Jiang, Xiaochen Chen, Jinlong Peng, Chengjie Wang, Hongsong Wang, Feng Zheng
Computer Vision · Cross-listed from cs.CR
Origin Lens: A Privacy-First Mobile Framework for Cryptographic Image Provenance and AI Detection
Alexander Loth, Dominique Conceicao Rosario, Peter Ebinger et al. (+2 more)
The proliferation of generative AI poses challenges for information integrity assurance, requiring systems that connect model governance with end-user verification. We present Origin Lens, a
privacy-first mobile framework that targets visual disinformation through a layered verification architecture. Unlike server-side detection systems, Origin Lens performs cryptographic image provenance verification and metadata-based AI detection locally on the device via a Rust/Flutter hybrid architecture. Our system integrates multiple signals - including cryptographic provenance, generative-model metadata and watermarks, and optional
retrieval-augmented verification - to provide users with graded confidence indicators at the point of consumption. We discuss the framework's
alignment with regulatory requirements (EU AI Act, DSA) and its role in verification infrastructure that complements platform-level mechanisms.
Read the full abstract ▾
All authors Alexander Loth, Dominique Conceicao Rosario, Peter Ebinger, Martin Kappes, Marc-Oliver Pahl
Computer Vision
Socratic-Geo: Synthetic Data Generation and Geometric Reasoning via Multi-Agent Interaction
Zhengbo Jiao, Shaobo Wang, Zifan Zhang et al. (+4 more)
Multimodal Large
Language Models (MLLMs) have significantly advanced
vision-
language understanding. However, even state-of-the-art models struggle with
geometric reasoning, revealing a critical bottleneck: the extreme scarcity of high-quality image-text pairs. Human annotation is prohibitively expensive, while automated methods fail to ensure fidelity and training effectiveness. Existing approaches either passively adapt to available images or employ inefficient random exploration with filtering, decoupling generation from learning needs. We propose Socratic-Geo, a fully autonomous framework that dynamically couples data synthesis with model learning through multi-
agent interaction. The Teacher
agent generates parameterized Python scripts with reflective feedback (Reflect for solvability, RePI for visual validity), ensuring image-text pair purity. The Solver
agent optimizes
reasoning through preference learning, with failure paths guiding Teacher's targeted augmentation. Independently, the Generator learns image generation capabilities on accumulated "image-code-instruction" triplets, distilling programmatic drawing intelligence into visual generation. Starting from only 108 seed problems, Socratic-Solver achieves 49.11 on six
benchmarks using one-quarter of baseline data, surpassing strong baselines by 2.43 points. Socratic-Generator achieves 42.4% on GenExam, establishing new state-of-the-art for open-source models, surpassing Seedream-4.0 (39.8%) and approaching Gemini-2.5-Flash-Image (43.1%).
Read the full abstract ▾
All authors Zhengbo Jiao, Shaobo Wang, Zifan Zhang, Wei Wang, Bing Zhao, Hu Wei, Linfeng Zhang
Computer Vision
UnHype: CLIP-Guided Hypernetworks for Dynamic LoRA Unlearning
Piotr Wójcik, Maksym Petrenko, Wojciech Gromski et al. (+2 more)
Recent advances in large-scale
diffusion models have intensified concerns about their potential misuse, particularly in generating realistic yet harmful or socially disruptive content. This challenge has spurred growing interest in effective machine unlearning, the process of selectively removing specific knowledge or concepts from a model without compromising its overall generative capabilities. Among various approaches, Low-Rank Adaptation (LoRA) has emerged as an effective and efficient method for fine-tuning models toward targeted unlearning. However, LoRA-based methods often exhibit limited adaptability to concept semantics and struggle to balance removing closely related concepts with maintaining generalization across broader meanings. Moreover, these methods face scalability challenges when multiple concepts must be erased simultaneously. To address these limitations, we introduce UnHype, a framework that incorporates hypernetworks into single- and multi-concept LoRA training. The proposed architecture can be directly plugged into Stable
Diffusion as well as modern flow-based text-to-image models, where it demonstrates stable training behavior and effective concept control. During inference, the hypernetwork dynamically generates adaptive LoRA weights based on the CLIP
embedding, enabling more context-aware, scalable unlearning. We evaluate UnHype across several challenging tasks, including object erasure, celebrity erasure, and explicit content removal, demonstrating its effectiveness and versatility. See the code on GitHub: https://github.com/gmum/UnHype.
Read the full abstract ▾
All authors Piotr Wójcik, Maksym Petrenko, Wojciech Gromski, Przemysław Spurek, Maciej Zieba
Computer Vision
From Vicious to Virtuous Cycles: Synergistic Representation Learning for Unsupervised Video Object-Centric Learning
Hyun Seok Seong, WonJun Moon and Jae-Pil Heo
Unsupervised object-centric learning models, particularly slot-based architectures, have shown great promise in decomposing complex scenes. However, their reliance on reconstruction-based training creates a fundamental conflict between the sharp, high-frequency
attention maps of the encoder and the spatially consistent but blurry reconstruction maps of the decoder. We identify that this discrepancy gives rise to a vicious cycle: the noisy feature map from the encoder forces the decoder to average over possibilities and produce even blurrier outputs, while the gradient computed from blurry reconstruction maps lacks high-frequency details necessary to supervise encoder features. To break this cycle, we introduce Synergistic Representation Learning (SRL) that establishes a virtuous cycle where the encoder and decoder mutually refine one another. SRL leverages the encoder's sharpness to deblur the semantic boundary within the decoder output, while exploiting the decoder's spatial consistency to denoise the encoder's features. This mutual refinement process is stabilized by a warm-up phase with a slot regularization objective that initially allocates distinct entities per slot. By bridging the representational gap between the encoder and decoder, SRL achieves state-of-the-art results on
video object-centric learning
benchmarks. Codes are available at https://github.com/hynnsk/SRL.
Read the full abstract ▾
Computer Vision
Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models
Hao Fang, Jinyu Li, Jiawei Kong et al. (+4 more)
While
multimodal reasoning models (MLRMs) have exhibited impressive capabilities, they remain prone to
hallucinations, and effective solutions are still underexplored. In this paper, we experimentally analyze the
hallucination cause and propose C3PO, a training-based mitigation framework comprising Chain-of-Thought Compression and Contrastive Preference Optimization. Firstly, we identify that introducing
reasoning mechanisms exacerbates models' reliance on
language priors while overlooking visual inputs, which can produce CoTs with reduced visual cues but redundant text
tokens. To this end, we propose to selectively filter redundant thinking
tokens for a more compact and signal-efficient CoT representation that preserves task-relevant information while suppressing noise. In addition, we observe that the quality of the
reasoning trace largely determines whether
hallucination emerges in subsequent responses. To leverage this insight, we introduce a
reasoning-enhanced preference tuning scheme that constructs training pairs using high-quality AI feedback. We further design a
multimodal hallucination-inducing mechanism that elicits models' inherent
hallucination patterns via carefully crafted inducers, yielding informative negative signals for contrastive correction. We provide theoretical justification for the effectiveness and demonstrate consistent
hallucination reduction across diverse MLRMs and
benchmarks.
Read the full abstract ▾
All authors Hao Fang, Jinyu Li, Jiawei Kong, Tianqu Zhuang, Kuofeng Gao, Bin Chen, Shu-Tao Xia