Computer Vision
Koo-Fu CLIP: Closed-Form Adaptation of Vision-Language Models via Fukunaga-Koontz Linear Discriminant Analysis
Matej Suchanek, Klara Janouskova, Ondrej Vasatko et al. (+1 more)
Visual-
language models such as CLIP provide powerful general-purpose representations, but their raw
embeddings are not optimized for supervised classification, often exhibiting limited class separation and excessive dimensionality. We propose Koo-Fu CLIP, a supervised CLIP adaptation method based on Fukunaga-Koontz Linear Discriminant Analysis, which operates in a whitened
embedding space to suppress within-class variation and enhance between-class discrimination. The resulting closed-form linear projection reshapes the geometry of CLIP
embeddings, improving class separability while performing effective dimensionality reduction, and provides a lightweight and efficient adaptation of CLIP representations. Across large-scale ImageNet
benchmarks, nearest visual prototype classification in the Koo-Fu CLIP space improves top-1 accuracy from 75.1% to 79.1% on ImageNet-1K, with consistent gains persisting as the label space expands to 14K and 21K classes. The method supports substantial compression by up to 10-12x with little or no loss in accuracy, enabling efficient large-scale classification and
retrieval.
Read the full abstract ▾
All authors Matej Suchanek, Klara Janouskova, Ondrej Vasatko, Jiri Matas
Computer Vision
LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditions
Jingjing Wang, Qirui Hu, Chong Bao et al. (+4 more)
Inverse rendering in urban scenes is pivotal for applications like autonomous driving and digital twins. Yet, it faces significant challenges due to complex illumination conditions, including multi-illumination and indirect light and shadow effects. However, the effects of these challenges on intrinsic decomposition and 3D reconstruction have not been explored due to the lack of appropriate
datasets. In this paper, we present LightCity, a novel high-quality synthetic urban
dataset featuring diverse illumination conditions with realistic indirect light and shadow effects. LightCity encompasses over 300 sky maps with highly controllable illumination, varying scales with street-level and aerial perspectives over 50K images, and rich properties such as depth, normal, material components, light and indirect light, etc. Besides, we leverage LightCity to
benchmark three fundamental tasks in the urban environments and conduct a comprehensive analysis of these
benchmarks, laying a robust foundation for advancing related research.
Read the full abstract ▾
All authors Jingjing Wang, Qirui Hu, Chong Bao, Yuke Zhu, Hujun Bao, Zhaopeng Cui, Guofeng Zhang
Computer Vision · Cross-listed from cs.RO
KAN We Flow? Advancing Robotic Manipulation with 3D Flow Matching via KAN & RWKV
Zhihao Chen, Yiyuan Ge, Ziyang Wang et al. (+1 more)
Diffusion-based visuomotor policies excel at modeling action distributions but are inference-inefficient, since recursively denoising from noise to policy requires many steps and heavy UNet backbones, which hinders deployment on resource-constrained
robots. Flow matching alleviates the sampling burden by learning a one-step vector field, yet prior implementations still inherit large UNet-style architectures. In this work, we present KAN-We-Flow, a flow-matching policy that draws on recent advances in Receptance Weighted Key Value (RWKV) and Kolmogorov-Arnold Networks (KAN) from
vision to build a lightweight and highly expressive backbone for 3D manipulation. Concretely, we introduce an RWKV-KAN block: an RWKV first performs efficient time/channel mixing to propagate task context, and a subsequent GroupKAN layer applies learnable spline-based, groupwise functional mappings to perform feature-wise nonlinear calibration of the action mapping on RWKV outputs. Moreover, we introduce an Action Consistency Regularization (ACR), a lightweight auxiliary loss that enforces
alignment between predicted action trajectories and expert demonstrations via Euler extrapolation, providing additional supervision to stabilize training and improve policy precision. Without resorting to large UNets, our design reduces parameters by 86.8%, maintains fast runtime, and achieves state-of-the-art success rates on Adroit, Meta-World, and DexArt
benchmarks. Our project page can be viewed in \href{https://zhihaochen-2003.github.io/KAN-We-Flow.github.io/}{\textcolor{red}{link}}
Read the full abstract ▾
All authors Zhihao Chen, Yiyuan Ge, Ziyang Wang, Youwei Zhang
Computer Vision
Robust Harmful Meme Detection under Missing Modalities via Shared Representation Learning
Felix Breiteneder, Mohammad Belal, Muhammad Saad Saeed et al. (+5 more)
Internet memes are powerful tools for communication, capable of spreading political, psychological, and sociocultural ideas. However, they can be harmful and can be used to disseminate hate toward targeted individuals or groups. Although previous studies have focused on designing new detection methods, these often rely on modal-complete data, such as text and images. In real-world settings, however, modalities like text may be missing due to issues like poor OCR quality, making existing methods sensitive to missing information and leading to performance deterioration. To address this gap, in this paper, we present the first-of-its-kind work to comprehensively investigate the behavior of harmful meme detection methods in the presence of modal-incomplete data. Specifically, we propose a new baseline method that learns a shared representation for multiple modalities by projecting them independently. These shared representations can then be leveraged when data is modal-incomplete. Experimental results on two
benchmark datasets demonstrate that our method outperforms existing approaches when text is missing. Moreover, these results suggest that our method allows for better integration of visual features, reducing dependence on text and improving robustness in scenarios where textual information is missing. Our work represents a significant step forward in enabling the real-world application of harmful meme detection, particularly in situations where a modality is absent.
Read the full abstract ▾
All authors Felix Breiteneder, Mohammad Belal, Muhammad Saad Saeed, Shahed Masoudian, Usman Naseem, Kulshrestha Juhi, Markus Schedl, Shah Nawaz
Computer Vision
PandaPose: 3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space
Jinghong Zheng, Changlong Jiang, Yang Xiao et al. (+7 more)
3D human pose lifting from a single RGB image is a challenging task in 3D
vision. Existing methods typically establish a direct joint-to-joint mapping from 2D to 3D poses based on 2D features. This formulation suffers from two fundamental limitations: inevitable error propagation from input predicted 2D pose to 3D predictions and inherent difficulties in handling self-occlusion cases. In this paper, we propose PandaPose, a 3D human pose lifting approach via propagating 2D pose prior to 3D anchor space as the unified intermediate representation. Specifically, our 3D anchor space comprises: (1) Joint-wise 3D anchors in the canonical coordinate system, providing accurate and robust priors to mitigate 2D pose estimation inaccuracies. (2) Depth-aware joint-wise feature lifting that hierarchically integrates depth information to resolve self-occlusion ambiguities. (3) The anchor-feature interaction decoder that incorporates 3D anchors with lifted features to generate unified anchor queries encapsulating joint-wise 3D anchor set, visual cues and
geometric depth information. The anchor queries are further employed to facilitate anchor-to-joint ensemble prediction. Experiments on three well-established
benchmarks (i.e., Human3.6M, MPI-INF-3DHP and 3DPW) demonstrate the superiority of our proposition. The substantial reduction in error by $14.7\%$ compared to SOTA methods on the challenging conditions of Human3.6M and qualitative comparisons further showcase the effectiveness and robustness of our approach.
Read the full abstract ▾
All authors Jinghong Zheng, Changlong Jiang, Yang Xiao, Jiaqi Li, Haohong Kuang, Hang Xu, Ran Wang, Zhiguo Cao, Min Du, Joey Tianyi Zhou
Computer Vision
Differential Vector Erasure: Unified Training-Free Concept Erasure for Flow Matching Models
Zhiqi Zhang, Xinhao Zhong, Yi Sun et al. (+5 more)
Text-to-image
diffusion models have demonstrated remarkable capabilities in generating high-quality images, yet their tendency to reproduce undesirable concepts, such as NSFW content, copyrighted styles, or specific objects, poses growing concerns for safe and controllable deployment. While existing concept erasure approaches primarily focus on DDPM-based
diffusion models and rely on costly fine-tuning, the recent emergence of flow matching models introduces a fundamentally different generative paradigm for which prior methods are not directly applicable. In this paper, we propose Differential Vector Erasure (DVE), a training-free concept erasure method specifically designed for flow matching models. Our key insight is that semantic concepts are implicitly encoded in the directional structure of the velocity field governing the generative flow. Leveraging this observation, we construct a differential vector field that characterizes the directional discrepancy between a target concept and a carefully chosen anchor concept. During inference, DVE selectively removes concept-specific components by projecting the velocity field onto the differential direction, enabling precise concept suppression without affecting irrelevant semantics. Extensive experiments on FLUX demonstrate that DVE consistently outperforms existing baselines on a wide range of concept erasure tasks, including NSFW suppression, artistic style removal, and object erasure, while preserving image quality and diversity. Code is available at https://github.com/zhiqizhang23/DVE.
Read the full abstract ▾
All authors Zhiqi Zhang, Xinhao Zhong, Yi Sun, Shuoyang Sun, Jiahui Wang, Bin Chen, Shu-Tao Xia, Xuan Wang
Computer Vision
MedAD-R1: Consistency-Reinforced Policy Optimization for Interpretable Medical Anomaly Detection
Haitao Zhang, Yingying Wang, Jiaxiang Wang et al. (+5 more)
Medical Anomaly Detection (MedAD) offers a promising direction for medical image analysis with Large
Multimodal Models (LMMs). However, progress is limited by fragmented
datasets and the tendency of Supervised Fine-Tuning (SFT) to learn superficial image-text correlations rather than verifiable diagnostic
reasoning. Consequently, current models often generate fluent explanations that are either insufficiently grounded in the image or inconsistent with their final answers, limiting their reliability in high-stakes medical applications. To address these issues, we introduce MedAD-38K, a large-scale,
multimodal, and multicenter
benchmark containing structured Visual Question Answering pairs and quality-controlled diagnostic Chain-of-Thought annotations across five core MedAD tasks. Based on this
benchmark, we propose a two-stage framework. Cognitive Injection first uses SFT to inject domain-specific medical knowledge and establish a structured think-then-answer format. Consistency Group Relative Policy Optimization (Con-GRPO) then employs an Evidence-Aware Consistency Reward to reinforce
reasoning that remains grounded in the image and logically supports the final answer. The resulting MedAD-R1 achieves state-of-the-art performance on MedAD-38K and consistently outperforms all evaluated baselines across five source-disjoint external
datasets spanning diverse imaging modalities. Beyond accuracy, it achieves higher
reasoning-answer consistency and visual-grounding scores across multiple backbones. With only 0.8B parameters, MedAD-R1 offers practical potential for resource-constrained deployment. These results demonstrate the generality of evidence-aware consistency optimization for interpretable MedAD. Project resources are available at https://github.com/zhtstar/MedAD-R1.
Read the full abstract ▾
All authors Haitao Zhang, Yingying Wang, Jiaxiang Wang, Haote Xu, Hongyang Zhang, Yirong Chen, Yue Huang, Xinghao Ding
Computer Vision
Haopeng Li, Shitong Shao, Wenliang Zhong et al. (+4 more)
Diffusion Transformers are fundamental for
video and image generation, but their efficiency is bottlenecked by the quadratic complexity of
attention. While block sparse
attention accelerates computation by attending only critical key-value blocks, it suffers from degradation at high sparsity by discarding context. In this work, we discover that
attention scores of non-critical blocks exhibit distributional stability, allowing them to be approximated accurately and efficiently rather than discarded, which is essentially important for sparse
attention design. Motivated by this key insight, we propose PISA, a training-free Piecewise Sparse
Attention that covers the full
attention span with sub-quadratic complexity. Unlike the conventional keep-or-drop paradigm that directly drop the non-critical block information, PISA introduces a novel exact-or-approximate strategy: it maintains exact computation for critical blocks while efficiently approximating the remainder through block-wise Taylor expansion. This design allows PISA to serve as a faithful proxy to full
attention, effectively bridging the gap between speed and quality. Experimental results demonstrate that PISA achieves 1.91 times and 2.57 times speedups on Wan2.1-14B and Hunyuan-
Video, respectively, while consistently maintaining the highest quality among sparse
attention methods. Notably, even for image generation on FLUX, PISA achieves a 1.2 times acceleration without compromising visual quality. Code is available at: https://github.com/xie-lab-ml/piecewise-sparse-
attention.
Read the full abstract ▾
All authors Haopeng Li, Shitong Shao, Wenliang Zhong, Zikai Zhou, Lichen Bai, Hui Xiong, Zeke Xie
Computer Vision
PDE-Constrained Optimization for Neural Image Segmentation with Physics Priors
Seema K. Poudel and Sunny K. Khadka
Segmentation of microscopy images constitutes an ill-posed inverse problem due to measurement noise, weak object boundaries, and limited labeled data. Although deep neural networks provide flexible nonparametric
estimators, unconstrained empirical risk minimization often leads to unstable solutions and poor generalization. In this work, image
segmentation is formulated as a PDE-constrained optimization problem that integrates physically motivated priors into deep learning models through variational regularization. The proposed framework minimizes a composite objective function consisting of a data fidelity term and penalty terms derived from reaction-
diffusion equations and phase-field interface energies, all implemented as differentiable residual losses. Experiments are conducted on the LIVECell
dataset, a high-quality, manually annotated collection of phase-contrast microscopy images. Training is performed on two
cell types, while evaluation is carried out on a distinct, unseen
cell type to assess generalization. A UNet architecture is used as the unconstrained baseline model. Experimental results demonstrate consistent improvements in
segmentation accuracy and boundary fidelity compared to unconstrained deep learning baselines. Moreover, the PDE-regularized models exhibit enhanced stability and improved generalization in low-sample regimes, highlighting the advantages of incorporating structured priors. The proposed approach illustrates how PDE-constrained optimization can strengthen data-driven learning frameworks, providing a principled bridge between variational methods, statistical learning, and scientific machine learning.
Read the full abstract ▾
Computer Vision
DRFormer: A Dual-Regularized Bidirectional Transformer for Person Re-identification
Ying Shu, Pujian Zhan, Huiqi Yang et al. (+3 more)
Both fine-grained discriminative details and global semantic features can contribute to solving person re-identification challenges, such as occlusion and pose variations.
Vision foundation models (e.g., DINO) excel at mining local textures, and
vision-
language models (e.g., CLIP) capture strong global semantic difference. Existing methods predominantly rely on a single paradigm, neglecting the potential benefits of their integration. In this paper, we analyze the complementary roles of these two architectures and propose a framework to synergize their strengths by a Dual-Regularized Bidirectional
Transformer (DRFormer). The dual-regularization mechanism ensures diverse feature extraction and achieves a better balance in the contributions of the two models. Extensive experiments on five
benchmarks show that our method effectively harmonizes local and global representations, achieving competitive performance against state-of-the-art methods.
Read the full abstract ▾
All authors Ying Shu, Pujian Zhan, Huiqi Yang, Hehe Fan, Youfang Lin, Kai Lv
Computer Vision
Radioactive 3D Gaussian Ray Tracing for Tomographic Reconstruction
Ling Chen and Bao Yang
3D Gaussian Splatting (3DGS) has recently emerged in computer
vision as a promising rendering technique. By adapting the principles of Elliptical Weighted Average (EWA) splatting to a modern differentiable pipeline, 3DGS enables real-time, high-quality novel view synthesis. Building upon this, R2-Gaussian extended the 3DGS paradigm to tomographic reconstruction by rectifying integration bias, achieving state-of-the-art performance in computed
tomography (CT). To enable differentiability, R2-Gaussian adopts a local affine approximation: each 3D Gaussian is locally mapped to a 2D Gaussian on the detector and composed via alpha blending to form projections. However, the affine approximation can degrade reconstruction quantitative accuracy and complicate the incorporation of nonlinear
geometric corrections. To address these limitations, we propose a tomographic reconstruction framework based on 3D Gaussian ray tracing. Our approach provides two key advantages over splatting-based models: (i) it computes the line integral through 3D Gaussian primitives analytically, avoiding the local affine collapse and thus yielding a more physically consistent forward projection model; and (ii) the ray-tracing formulation gives explicit control over ray origins and directions, which facilitates the precise application of nonlinear
geometric corrections, e.g., arc-correction used in positron emission
tomography (PET). These properties extend the applicability of Gaussian-based reconstruction to a wider range of realistic
tomography systems while improving projection accuracy.
Read the full abstract ▾
Computer Vision
Baseline Method of the Foundation Model Challenge for Ultrasound Image Analysis
Bo Deng, Yitong Tang, Jiake Li et al. (+7 more)
Ultrasound (US) imaging exhibits substantial heterogeneity across anatomical structures and acquisition protocols, posing significant challenges to the development of generalizable analysis models. Most existing methods are task-specific, limiting their suitability as clinically deployable foundation models. To address this limitation, the Foundation Model Challenge for Ultrasound Image Analysis (FM\_UIA~2026) introduces a large-scale multi-task
benchmark comprising 27 subtasks across
segmentation, classification, detection, and regression. In this paper, we present the official baseline for FM\_UIA~2026 based on a unified Multi-Head Multi-Task Learning (MH-MTL) framework that supports all tasks within a single shared network. The model employs an ImageNet-pretrained EfficientNet–B4 backbone for robust feature extraction, combined with a Feature Pyramid Network (FPN) to capture multi-scale contextual information. A task-specific routing strategy enables global tasks to leverage high-level semantic features, while dense prediction tasks exploit spatially detailed FPN representations. Training incorporates a composite loss with task-adaptive learning rate scaling and a cosine annealing schedule. Validation results demonstrate the feasibility and robustness of this unified design, establishing a strong and extensible baseline for ultrasound foundation model research. The code and
dataset are publicly available at \href{https://github.com/lijiake2408/Foundation-Model-Challenge-for-Ultrasound-Image-Analysis}{GitHub}.
Read the full abstract ▾
All authors Bo Deng, Yitong Tang, Jiake Li, Yuxin Huang, Li Wang, Yu Zhang, Yufei Zhan, Hua Lu, Xiaoshen Zhang, Jieyun Bai
Computer Vision
Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidance
Xinrong Chen, Xu Chu, Yingmin Qiu et al. (+8 more)
Large
Vision-
Language Models (LVLMs) can reason from image-text inputs and perform well in various
multimodal tasks. Despite this success, they are affected by
language priors and often produce
hallucinations.
Hallucinations denote generated content that is grammatically and syntactically coherent, yet bears no match or direct relevance to visual input. To address this problem, we propose Residual Decoding (ResDec). It is a novel training-free method that uses historical information to aid decoding. The method relies on the internal implicit
reasoning mechanism and
token logits
evolution mechanism of LVLMs to correct biases. Extensive experiments demonstrate that ResDec effectively suppresses
hallucinations induced by
language priors, significantly improves visual grounding, and reduces object
hallucinations. In addition to mitigating
hallucinations, ResDec also performs exceptionally well on comprehensive LVLM
benchmarks, highlighting its broad applicability.
Read the full abstract ▾
All authors Xinrong Chen, Xu Chu, Yingmin Qiu, Hengyuan Zhang, Jing Xiong, Shiyu Tang, Shuai Liu, Shaokang Yang, Cheng Yang, Hayden Kwok-Hay So, Ngai Wong
Computer Vision
ReLayout: Versatile and Structure-Preserving Design Layout Editing via Relation-Aware Design Reconstruction
Jiawei Lin, Shizhao Sun, Danqing Huang et al. (+3 more)
Automated redesign without manual adjustments marks a key step forward in the design workflow. In this work, we focus on a foundational redesign task termed design layout editing, which seeks to autonomously modify the
geometric composition of a design based on user intents. To overcome the ambiguity of user needs expressed in natural
language, we introduce four basic and important editing actions and standardize the format of editing operations. The underexplored task presents a unique challenge: satisfying specified editing operations while simultaneously preserving the layout structure of unedited elements. Besides, the scarcity of triplet (original design, editing operation, edited design) samples poses another formidable challenge. To this end, we present ReLayout, a novel framework for versatile and structure-preserving design layout editing that operates without triplet data. Specifically, ReLayout first introduces the relation graph, which contains the position and size relationships among unedited elements, as the constraint for layout structure preservation. Then, relation-aware design reconstruction (RADR) is proposed to bypass the data challenge. By learning to reconstruct a design from its elements, a relation graph, and a synthesized editing operation, RADR effectively emulates the editing process in a self-supervised manner. A multi-modal large
language model serves as the backbone for RADR, unifying multiple editing actions within a single model and thus achieving versatile editing after fine-tuning. Qualitative, quantitative results and user studies show that ReLayout significantly outperforms the baseline models in terms of editing quality, accuracy, and layout structure preservation.
Read the full abstract ▾
All authors Jiawei Lin, Shizhao Sun, Danqing Huang, Ting Liu, Ji Li, Jiang Bian
Computer Vision
From Videos to Conversations: Egocentric Instructions for Task Assistance
Lavisha Aggarwal, Vikas Bahirwani and Andrea Colaco
Many everyday tasks, ranging from appliance repair and cooking to car maintenance, require expert knowledge, particularly for complex, multi-step procedures. Despite growing interest in AI
agents for augmented reality (AR) assistance, progress remains limited by the scarcity of large-scale
multimodal conversational
datasets grounded in real-world task execution, in part due to the cost and logistical complexity of human-assisted data collection. In this paper, we present a framework to automatically transform single person instructional
videos into two-person
multimodal task-guidance conversations. Our fully automatic pipeline, based on large
language models, provides a scalable and cost efficient alternative to traditional data collection approaches. Using this framework, we introduce HowToDIV, a
multimodal dataset comprising 507 conversations, 6,636 question answer pairs, and 24 hours of
video spanning multiple domains. Each session consists of a multi-turn expert-novice interaction. Finally, we report baseline results using Gemma 3 and Qwen 2.5 on HowToDIV, providing an initial
benchmark for
multimodal procedural task assistance.
Read the full abstract ▾
Computer Vision
VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models
Guangshuo Qin, Zhiteng Li, Zheng Chen et al. (+3 more)
Mixture-of-Experts(MoE)
Vision-
Language Models (VLMs) offer remarkable performance but incur prohibitive
memory and computational costs, making compression essential. Post-Training Quantization (PTQ) is an effective training-free technique to address the massive
memory and computation overhead. Existing quantization paradigms fall short as they are oblivious to two critical forms of heterogeneity: the inherent discrepancy between
vision and
language tokens, and the non-uniform contribution of different experts. To bridge this gap, we propose Visual Expert Quantization (VEQ), a dual-aware quantization framework designed to simultaneously accommodate cross-modal differences and heterogeneity between experts. Specifically, VEQ incorporates 1)Modality-expert-aware Quantization, which utilizes expert activation frequency to prioritize error minimization for pivotal experts, and 2)Modality-affinity-aware Quantization, which constructs an enhanced Hessian
matrix by integrating
token-expert affinity with modality information to guide the calibration process. Extensive experiments across diverse
benchmarks verify that VEQ consistently outperforms state-of-the-art baselines. Specifically, under the W3A16 configuration, our method achieves significant average accuracy gains of 2.04% on Kimi-VL and 3.09% on Qwen3-VL compared to the previous SOTA quantization methods, demonstrating superior robustness across various
multimodal tasks. Our code will be available at https://github.com/guangshuoqin/VEQ.
Read the full abstract ▾
All authors Guangshuo Qin, Zhiteng Li, Zheng Chen, Weihang Zhang, Linghe Kong, Yulun Zhang
Computer Vision
FUSE-Flow: Scalable Real-Time Multi-View Point Cloud Reconstruction Using Confidence
Chentian Sun
Real-time multi-view point cloud reconstruction is a core problem in 3D
vision and immersive perception, with wide applications in VR, AR, robotic
navigation, digital twins, and computer interaction. Despite advances in multi-camera systems and high-resolution depth sensors, fusing large-scale multi-view depth observations into high-quality point clouds under strict real-time constraints remains challenging. Existing methods relying on voxel-based fusion, temporal accumulation, or global optimization suffer from high computational complexity, excessive
memory usage, and limited scalability, failing to simultaneously achieve real-time performance, reconstruction quality, and multi-camera extensibility. We propose FUSE-Flow, a frame-wise, stateless, and linearly scalable point cloud streaming reconstruction framework. Each frame independently generates point cloud fragments, fused via two weights, measurement confidence and 3D distance consistency to suppress noise while preserving
geometric details. For large-scale multi-camera efficiency, we introduce an adaptive spatial hashing-based weighted aggregation method: 3D space is adaptively partitioned by local point cloud density, representative points are selected per
cell, and weighted fusion is performed to handle both sparse and dense regions. With GPU parallelization, FUSE-Flow achieves high-throughput, low-latency point cloud generation and fusion with linear complexity. Experiments demonstrate that the framework improves reconstruction stability and
geometric fidelity in overlapping, depth-discontinuous, and dynamic scenes, while maintaining real-time frame rates on modern GPUs, verifying its effectiveness, robustness, and scalability.
Read the full abstract ▾
Computer Vision
GMAC: Global Multi-View Constraint for Automatic Multi-Camera Extrinsic Calibration
Chentian Sun
Automatic calibration of multi-camera systems, namely the accurate estimation of spatial extrinsic parameters, is fundamental for 3D reconstruction, panoramic perception, and multi-view data fusion. Existing methods typically rely on calibration targets, explicit
geometric modeling, or task-specific neural networks. Such approaches often exhibit limited robustness and applicability in complex dynamic environments or online scenarios, making them difficult to deploy in practical applications. To address this, this paper proposes GMAC, a multi-camera extrinsic estimation framework based on the implicit
geometric representations learned by multi-view reconstruction networks. GMAC models extrinsics as global variables constrained by the latent multi-view
geometric structure and prunes and structurally reconfigures existing networks so that their latent features can directly support extrinsic prediction through a lightweight regression head, without requiring a completely new network design. Furthermore, GMAC jointly optimizes cross-view reprojection consistency and multi-view cycle consistency, ensuring
geometric coherence across cameras while improving prediction accuracy and optimization stability. Experiments on both synthetic and real-world multi-camera
datasets demonstrate that GMAC achieves accurate and stable extrinsic estimation without explicit 3D reconstruction or manual calibration, providing a new solution for efficient deployment and online calibration of multi-camera systems.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models
Kaiyuan Cui, Yige Li, Yutao Wu et al. (+4 more)
Vision-
language models (VLMs) extend large
language models (LLMs) with
vision encoders, enabling text generation conditioned on both images and text. However, this
multimodal integration expands the attack surface by exposing the model to image-based jailbreaks crafted to induce harmful responses. Existing gradient-based jailbreak methods transfer poorly, as
adversarial patterns overfit to a single white-box surrogate and fail to generalise to black-box models. In this work, we propose Universal and transferable jailbreak (UltraBreak), a framework that constrains
adversarial patterns through transformations and regularisation in the
vision space, while relaxing textual targets through semantic-based objectives. By defining its loss in the textual
embedding space of the target LLM, UltraBreak discovers universal
adversarial patterns that generalise across diverse jailbreak objectives. This combination of
vision-level regularisation and semantically guided textual supervision mitigates surrogate overfitting and enables strong transferability across both models and attack targets. Extensive experiments show that UltraBreak consistently outperforms prior jailbreak methods. Further analysis reveals why earlier approaches fail to transfer, highlighting that smoothing the loss landscape via semantic objectives is crucial for enabling universal and transferable jailbreaks. The code is publicly available in our \href{https://github.com/kaiyuanCui/UltraBreak}{GitHub repository}.
Read the full abstract ▾
All authors Kaiyuan Cui, Yige Li, Yutao Wu, Xingjun Ma, Sarah Erfani, Christopher Leckie, Hanxun Huang
Computer Vision
Effectiveness of Automatically Curated Dataset in Thyroid Nodules Classification Algorithms Using Deep Learning
Jichen Yang, Jikai Zhang, Benjamin Wildman-Tobriner et al. (+1 more)
The diagnosis of thyroid nodule cancers commonly utilizes ultrasound images. Several studies showed that deep learning
algorithms designed to classify benign and malignant thyroid nodules could match radiologists' performance. However, data availability for training deep learning models is often limited due to the significant effort required to curate such
datasets. Our previous study proposed a method (MADLaP) to curate thyroid nodule
datasets automatically. It was tested to have a 63% yield rate and 83% accuracy. However, the usefulness of the generated data for training deep learning models remains unknown. In this study, we conducted experiments to determine whether using an automatically-curated
dataset improves deep learning
algorithms' performance. The expert annotations of 378 patients either trained a model directly (803 images) or built MADLaP, which curated 5,228 images from 3,981 patients; the same network and training method were used throughout, and 10-model cross-validation ensembles were tested on 378 nodules from an earlier period. The model trained on the automatically-curated
dataset had a significantly higher AUC than the model trained on the manually selected
dataset (0.657 vs 0.607; difference 0.051, 95% CI: 0.020, 0.081; P = .001), an advantage seen in all 22 configuration pairs. On the same test set, this model was within the AUC range of four radiologists (0.63-0.66) and not significantly different from a
benchmark trained on 1,278 expert-curated nodules (0.69; difference -0.036, 95% CI: -0.083, 0.011), whereas manually curated models fell significantly below it. The Stage-1 subset (86% accuracy) did not differ significantly from the full MADLaP
dataset (difference 0.015, 95% CI: -0.006, 0.036). In conclusion, we showed that using an automatically-curated
dataset can substantially increase the performance of deep learning
algorithms.
Read the full abstract ▾
All authors Jichen Yang, Jikai Zhang, Benjamin Wildman-Tobriner, Maciej A. Mazurowski
Computer Vision
LocalScore: Local Density-Aware Similarity Scoring for Biometrics
Yiyang Su, Minchul Kim, Jie Zhu et al. (+4 more)
Open-set biometrics faces challenges with probe subjects who may not be enrolled in the gallery, as traditional biometric systems struggle to detect these non-mated probes. Despite the growing prevalence of multi-sample galleries in real-world deployments, most existing methods collapse intra-subject variability into a single global representation, leading to suboptimal decision boundaries and poor open-set robustness. To address this issue, we propose LocalScore, a simple yet effective scoring
algorithm that explicitly incorporates the local density of the gallery feature distribution using the k-th nearest neighbors. LocalScore is architecture-agnostic, loss-independent, and incurs negligible computational overhead, making it a plug-and-play solution for existing biometric systems. Extensive experiments across multiple modalities demonstrate that LocalScore consistently achieves substantial gains in open-set
retrieval (FNIR@FPIR reduced from 53% to 40%) and verification (TAR@FAR improved from 51% to 74%). We further provide theoretical analysis and empirical validation explaining when and why the method achieves the most significant gains based on
dataset characteristics.
Read the full abstract ▾
All authors Yiyang Su, Minchul Kim, Jie Zhu, Christopher Perry, Feng Liu, Anil Jain, Xiaoming Liu
Computer Vision
SRVAU-R1: Enhancing Video Anomaly Understanding via Reflection-Aware Learning
Zihao Zhao, Shengting Cao and Muchao Ye
Multi-modal large
language models (MLLMs) have demonstrated significant progress in
reasoning capabilities and shown promising effectiveness in
video anomaly understanding (VAU) tasks. However, existing MLLM-based approaches remain largely focused on surface-level descriptions of anomalies, lacking deep
reasoning over abnormal behaviors like explicit self-reflection and self-correction. To address that, we propose Self-Reflection-Enhanced
Reasoning for
Video Anomaly Understanding (SRVAU-R1), a reflection-aware learning framework that incorporates reflection in MLLM
reasoning. Specifically, SRVAU-R1 introduces the first reflection-oriented Chain-of-Thought
dataset tailored for VAU, providing structured supervision with initial
reasoning, self-reflection, and revised
reasoning. Based on that, it includes a novel reflection-aware learning paradigm with supervised fine-tuning and reinforcement fine-tuning to enhance multi-modal
reasoning for VAU. Extensive experiments on multiple
video anomaly
benchmarks demonstrate that SRVAU-R1 consistently outperforms existing methods, achieving significant improvements in both temporal anomaly localization accuracy and
reasoning quality.
Read the full abstract ▾
Computer Vision
CortiNet: A Physics-Perception Hybrid Cortical-Inspired Dual-Stream Network for Gallbladder Disease Diagnosis from Ultrasound
Vagish Kumar and Souvik Chakraborty
Ultrasound imaging is the primary diagnostic modality for detecting Gallbladder diseases due to its non-invasive nature, affordability, and wide accessibility. However, the low resolution and speckle noise inherent to ultrasound images hinder diagnostic reliability, prompting the use of large convolutional neural networks that are difficult to deploy in routine clinical settings. In this work, we propose CortiNet, a lightweight, cortical-inspired dual-stream neural architecture for gallbladder disease diagnosis that integrates physically interpretable multi-scale signal decomposition with perception-driven feature learning. Inspired by parallel processing pathways in the human visual cortex, CortiNet explicitly separates low-frequency structural information from high-frequency perceptual details and processes them through specialized encoding streams. By operating directly on structured, frequency-selective representations rather than raw pixel intensities, the architecture embeds strong physics-based inductive bias, enabling efficient feature learning with a significantly reduced parameter footprint. A late-stage cortical-style fusion mechanism integrates complementary structural and textural cues while preserving computational efficiency. Additionally, we propose a structure-aware explainability framework wherein gradient-weighted class activation mapping is only applied to the structural branch of the proposed CortiNet architecture. This choice allows the model to only focus on the structural features, making it robust against speckle noise. We evaluate CortiNet on 10,692 expert-annotated images spanning nine clinically relevant gallbladder disease categories. Experimental results demonstrate that CortiNet achieves high diagnostic accuracy (98.74%) with only a fraction of the parameters required by conventional deep convolutional models.
Read the full abstract ▾
Computer Vision
VAMOS-OCTA: Vessel-Aware Multi-Axis Orthogonal Supervision for Inpainting Motion-Corrupted OCT Angiography Volumes
Nick DiSanto, Ehsan Khodapanah Aghdam, Han Liu et al. (+4 more)
Handheld Optical Coherence
Tomography Angiography (OCTA) enables noninvasive retinal imaging in uncooperative or pediatric subjects, but is highly susceptible to motion artifacts that severely degrade volumetric image quality. Sudden motion during 3D acquisition can lead to unsampled retinal regions across entire B-scans (cross-sectional slices), resulting in blank bands in en face projections. We propose VAMOS-OCTA, a deep learning framework for inpainting motion-corrupted B-scans using vessel-aware multi-axis supervision. We employ a 2.5D U-Net architecture that takes a stack of neighboring B-scans as input to reconstruct a corrupted center B-scan, guided by a novel Vessel-Aware Multi-Axis Orthogonal Supervision (VAMOS) loss. This loss combines vessel-weighted intensity reconstruction with axial and lateral projection consistency, encouraging vascular continuity in native B-scans and across orthogonal planes. Unlike prior work that focuses primarily on restoring the en face MIP, VAMOS-OCTA jointly enhances both cross-sectional B-scan sharpness and volumetric projection accuracy, even under severe motion corruptions. We trained our model on both synthetic and real-world corrupted volumes and evaluated its performance using both perceptual quality and pixel-wise accuracy metrics. VAMOS-OCTA consistently outperforms prior methods, producing reconstructions with sharp capillaries, restored vessel continuity, and clean en face projections. These results demonstrate that multi-axis supervision offers a powerful constraint for restoring motion-degraded 3D OCTA data. Our source code is available at https://github.com/MedICL-VU/VAMOS-OCTA.
Read the full abstract ▾
All authors Nick DiSanto, Ehsan Khodapanah Aghdam, Han Liu, Jacob Watson, Yuankai K. Tao, Hao Li, Ipek Oguz
Computer Vision
Navigating Simply, Aligning Deeply: Winning Solutions for Mouse vs. AI 2025
Phu-Hoa Pham, Chi-Nguyen Tran, Dao Sy Duy Minh et al. (+2 more)
Visual robustness and neural
alignment remain critical challenges in developing artificial
agents that can match biological
vision systems. We present the winning approaches from Team HCMUS_TheFangs for both tracks of the NeurIPS 2025 Mouse vs. AI: Robust Visual Foraging Competition. For Track 1 (Visual Robustness), we demonstrate that architectural simplicity combined with targeted components yields superior generalization, achieving 95.4% final score with a lightweight two-layer CNN enhanced by Gated Linear Units and observation normalization. For Track 2 (Neural
Alignment), we develop a deep ResNet-like architecture with 16 convolutional layers and GLU-based gating that achieves top-1 neural prediction performance with 17.8 million parameters. Our systematic analysis of ten model checkpoints trained between 60K to 1.14M steps reveals that training duration exhibits a non-monotonic relationship with performance, with optimal results achieved around 200K steps. Through comprehensive ablation studies and failure case analysis, we provide insights into why simpler architectures excel at visual robustness while deeper models with increased capacity achieve better neural
alignment. Our results challenge conventional assumptions about model complexity in visuomotor learning and offer practical guidance for developing robust, biologically-inspired visual
agents.
Read the full abstract ▾
All authors Phu-Hoa Pham, Chi-Nguyen Tran, Dao Sy Duy Minh, Nguyen Lam Phu Quy, Huynh Trung Kiet
Computer Vision
Unveiling the Cognitive Compass: Theory-of-Mind-Guided Multimodal Emotion Reasoning
Meng Luo, Bobo Li, Shanqing Xu et al. (+8 more)
Despite rapid progress in
multimodal large
language models (MLLMs), their capability for deep emotional understanding remains limited. We argue that genuine affective intelligence requires explicit modeling of Theory of Mind (ToM), the cognitive substrate from which
emotions arise. To this end, we introduce HitEmotion, a ToM-grounded hierarchical
benchmark that diagnoses capability breakpoints across increasing levels of cognitive depth. Second, we propose a ToM-guided
reasoning chain that tracks mental states and calibrates cross-modal evidence to achieve faithful emotional
reasoning. We further introduce TMPO, a reinforcement learning method that uses intermediate mental states as process-level supervision to guide and strengthen model
reasoning. Extensive experiments show that HitEmotion exposes deep emotional
reasoning deficits in state-of-the-art models, especially on cognitively demanding tasks. In evaluation, the ToM-guided
reasoning chain and TMPO improve end-task accuracy and yield more faithful, more coherent rationales. In conclusion, our work provides the research community with a practical toolkit for evaluating and enhancing the cognition-based emotional understanding capabilities of MLLMs. Our
dataset and code are available at: https://HitEmotion.github.io/.
Read the full abstract ▾
All authors Meng Luo, Bobo Li, Shanqing Xu, Shize Zhang, Qiuchan Chen, Menglu Han, Wenhao Chen, Yanxiang Huang, Hao Fei, Mong-Li Lee, Wynne Hsu
Computer Vision
Hybrid Topological and Deep Feature Fusion for Accurate MRI-Based Alzheimer's Disease Severity Classification
Faisal Ahmed
Early and accurate diagnosis of Alzheimer's disease (AD) remains a critical challenge in neuroimaging-based clinical decision support systems. In this work, we propose a novel hybrid deep learning framework that integrates Topological Data Analysis (TDA) with a DenseNet121 backbone for four-class Alzheimer's disease classification using structural MRI data from the OASIS
dataset. TDA is employed to capture complementary topological characteristics of
brain structures that are often overlooked by conventional neural networks, while DenseNet121 efficiently learns hierarchical spatial features from MRI slices. The extracted deep and topological features are fused to enhance class separability across the four AD stages. Extensive experiments conducted on the OASIS-1 Kaggle MRI
dataset demonstrate that the proposed TDA+DenseNet121 model significantly outperforms existing state-of-the-art approaches. The model achieves an accuracy of 99.93% and an AUC of 100%, surpassing recently published CNN-based, transfer learning, ensemble, and multi-scale architectures. These results confirm the effectiveness of incorporating topological insights into deep learning pipelines and highlight the potential of the proposed framework as a robust and highly accurate tool for automated Alzheimer's disease diagnosis.
Read the full abstract ▾
Computer Vision
Data Augmentation for High-Fidelity Generation of CAR-T/NK Immunological Synapse Images
Xiang Zhang, Boxuan Zhang, Alireza Naghizadeh et al. (+5 more)
Chimeric antigen receptor (CAR)-T and NK
cell immunotherapies have transformed cancer treatment, and recent studies suggest that the quality of the CAR-T/NK
cell immunological synapse (IS) may serve as a functional biomarker for predicting therapeutic efficacy. Accurate detection and
segmentation of CAR-T/NK IS structures using artificial neural networks (ANNs) can greatly increase the speed and reliability of IS quantification. However, a persistent challenge is the limited size of annotated microscopy
datasets, which restricts the ability of ANNs to generalize. To address this challenge, we integrate two complementary data-augmentation frameworks. First, we employ Instance Aware Automatic Augmentation (IAAA), an automated, instance-preserving augmentation method that generates synthetic CAR-T/NK IS images and corresponding
segmentation masks by applying optimized augmentation policies to original IS data. IAAA supports multiple imaging modalities (e.g., fluorescence and brightfield) and can be applied directly to CAR-T/NK IS images derived from patient samples. In parallel, we introduce a Semantic-Aware AI Augmentation (SAAA) pipeline that combines a
diffusion-based mask generator with a Pix2Pix conditional image synthesizer. This second method enables the creation of diverse, anatomically realistic
segmentation masks and produces high-fidelity CAR-T/NK IS images aligned with those masks, further expanding the training corpus beyond what IAAA alone can provide. Together, these augmentation strategies generate synthetic images whose visual and structural properties closely match real IS data, significantly improving CAR-T/NK IS detection and
segmentation performance. By enhancing the robustness and accuracy of IS quantification, this work supports the development of more reliable imaging-based biomarkers for predicting patient response to CAR-T/NK immunotherapy.
Read the full abstract ▾
All authors Xiang Zhang, Boxuan Zhang, Alireza Naghizadeh, Mohab Mohamed, Dongfang Liu, Ruixiang Tang, Dimitris Metaxas, Dongfang Liu
Computer Vision
ConsensusDrop: Fusing Visual and Cross-Modal Saliency for Efficient Vision Language Models
Dhruv Parikh, Haoyang Fan, Rajgopal Kannan et al. (+1 more)
Vision-
Language Models (VLMs) are expensive because the LLM processes hundreds of largely redundant visual
tokens. Existing
token reduction methods typically exploit either
vision-encoder saliency (broad but query-agnostic) or LLM cross-
attention (query-aware but sparse and costly). We show that neither signal alone is sufficient: fusing them consistently improves performance compared to unimodal visual
token selection (ranking). However, making such fusion practical is non-trivial: cross-modal saliency is usually only available inside the LLM (too late for efficient pre-LLM
pruning), and the two signals are inherently asymmetric, so naive fusion underutilizes their complementary strengths. We propose ConsensusDrop, a training-free framework that derives a consensus ranking by reconciling
vision encoder saliency with query-aware cross-
attention, retaining the most informative
tokens while compressing the remainder via encoder-guided
token merging. Across LLaVA-1.5/NeXT,
Video-LLaVA, and other open-source VLMs, ConsensusDrop consistently outperforms prior
pruning methods under identical
token budgets and delivers a stronger accuracy-efficiency Pareto frontier – preserving near-baseline accuracy even at aggressive
token reductions while reducing TTFT and KV cache footprint. Our code will be open-sourced.
Read the full abstract ▾
All authors Dhruv Parikh, Haoyang Fan, Rajgopal Kannan, Viktor Prasanna