Computer Vision
DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation
Ziyang Gao, Zhizhuo Jiang, Jingjing Chang et al. (+5 more)
Referring remote sensing image
segmentation (RRSIS) aims to delineate targets specified by natural
language expressions in remote sensing imagery. Existing methods mainly follow joint fusion
segmentation (JFS) or decoupled
prompt segmentation (DPS). JFS is efficient but often suffers from limited accuracy because referent localization and mask delineation are optimized under a unified objective, whereas DPS separates localization from mask generation using spatial
prompts and foundation segmenters at the cost of higher
memory consumption and inference latency. To bridge this gap, we propose DiCoR, a decoupled referent disambiguation and contour recalibration framework built on an efficient JFS pipeline. DiCoR addresses two key challenges: distinguishing the correct referent from ambiguous candidates and refining coarse masks after localization. A disambiguation-aware localization guidance strategy ranks salient candidate regions with adaptive linguistic cues and injects the resulting localization prior into fused features. A lightweight contour recalibration module further predicts residual corrections to coarse logits under localized contour supervision, improving mask quality with limited computational overhead. Experiments on RefSegRS, RRSIS-D, and RISBench show that DiCoR achieves the best
segmentation accuracy across all three
benchmarks. On RefSegRS, it improves mIoU and gIoU by 4.90% and 2.16% over a competitive JFS method while running 2.4-6.5 times faster than the evaluated DPS method, demonstrating a favorable accuracy-efficiency trade-off. Code is available at https://github.com/zyGao1126/DiCoR.
Read the full abstract ▾
All authors Ziyang Gao, Zhizhuo Jiang, Jingjing Chang, Yixin Yang, Yuwen Pan, Yong-Qiang Mao, Yu Liu, Hai-Bao Chen
Computer Vision
Bias Mitigation in Face Recognition via Demographic-based Supervised Contrastive Learning
Yu Linghu, Salman Mohammad, Xinyi Zhang et al. (+1 more)
Face recognition systems have been shown to be biased toward certain demographic groups by exhibiting different error rates across gender, age, or ethnicity. Though the imbalance of the training data with respect to these demographics is one cause of this bias, training on artificially balanced groups does not completely mitigate the problem. For deployment, face recognition typically works at operating points allowing very low false match rates and, hence, on the tail of the non-match score distribution. While class balancing can improve the means of these distributions, the aim of our approach is to improve
fairness by addressing the behavior in the tail. Particularly, we propose the Demographic-based Supervised Contrastive loss (DeSCon) for face recognition, which relies on a well-designed composition of training batches and demographic-aware pair selection. Our experimental evaluation on both demographically-labeled
datasets and standard verification
benchmarks shows that DeSCon can improve
fairness beyond balancing training
datasets while maintaining competitive verification performance. Source code is available upon request.
Read the full abstract ▾
All authors Yu Linghu, Salman Mohammad, Xinyi Zhang, Manuel Günther
Computer Vision
A Deep RL based Framework for Targeted White Matter Tractography
Ankita Joshi
Fiber tractography's ability to reconstruct the
brain's structural pathways, has made it a crucial component of modern neuroimaging, enabling detailed, non-invasive mapping of structural connectivity and supporting a wide range of neurological research and clinical applications. However, despite its importance, tractography remains a challenging task due to the inherent complexity of white matter structure and its susceptibility to false positives, which can lead to the misrepresentation of critical pathways. To overcome these limitations, in this thesis, we propose a hybrid framework that integrates reinforcement learning with supervised learning for refining RL policies, specifically tailored for tract-specific tractography. Notably, our framework does not rely on ground-truth fibers for training. Moreover, the tract-specific formulation bypasses the need for an explicit
segmentation process, simplifying the overall pipeline. Our work includes two main contributions, each building upon the previous. First, we introduce a hybrid approach that combines reinforcement learning with supervised learning (specifically, GPT-based policy learning) to refine policies in a tract-specific context. Second, we propose a scalable framework for data-driven multi-policy fusion, which leverages the complementary strengths of multiple RL policies to improve tractography performance and robustness. We demonstrate the effectiveness of our framework through extensive validation on
benchmark public
datasets including TractoInferno, HCP, and ISMRM-2015, highlighting its ability to generalize across data sources and accurately reconstruct
brain white matter tracts. We believe that these contributions represent significant advancements in the field of tractography, improving robustness, reliability, and accuracy while reducing dependence on ground-truth annotations.
Read the full abstract ▾
Computer Vision
TennisVAR: A Stroke-Evidence-Grounded Multimodal Large Language Model for Tactical Reasoning in Tennis Videos
Yifan Mei, Qingling Shi, Changli Wu et al. (+3 more)
Sports-
video understanding is moving beyond event recognition toward explaining how actions collectively shape match progression, however, existing tennis-
video methods either perceive individual strokes without modeling their tactical dependencies or generate high-level analyses without grounding them in the underlying events. To bridge this perception-to-understanding gap, we formulate stroke-evidence-grounded tactical
reasoning, a new rally-level task that requires models to jointly predict an open-ended answer, a hierarchical tactic label, an ordered sequence of supporting strokes, and decisive key actions, with each evidence stroke anchored to its racket-ball contact frame. We further introduce TRACE (Tactical
Reasoning with Action-Chain Evidence in Tennis), a large-scale expert-annotated
benchmark containing 11,189 rally
videos, 41,485 stroke events, 25,429 tactical units, and 11,189 question-answer pairs, which unifies fine-grained stroke attributes, cross-stroke tactical relations, hierarchical tactic annotations, and evidence-grounded questions across factual perception, tactical understanding, and decision
reasoning. Building on TRACE, we propose TennisVAR (Tennis
Video Action-chain Reasoner), an evidence-grounded
multimodal large
language model that follows an "event-relation-evidence-tactic"
reasoning paradigm, where an Event Parsing Module converts continuous rallies into explicit stroke-event sequences while a Tactical Graph-Guided Temporal Reasoner jointly models rally progression and same-player decision dependencies to identify question-relevant evidence and decisive actions.
Read the full abstract ▾
All authors Yifan Mei, Qingling Shi, Changli Wu, Jiayuan Rao, Jiayi Ji, Liujuan Cao
Computer Vision
Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs
Beining Xu, Hairui Wang, Jiaxin Wang et al. (+2 more)
While the
privacy risks of
multimodal large
language models (MLLMs) have drawn significant
attention, the unique vulnerabilities of domain-specific MLLMs remain largely underexplored. Focusing on document understanding MLLMs for identity document processing, this paper investigates the
privacy issues inherent in Key Information Extraction (KIE) tasks. We reveal that when input images lack sufficient visual evidence, these models often rely on memorized field relations from training data to infer missing content, thereby leaking multiple correlated fields containing sensitive personal information. To mitigate this risk, we make three key contributions.First, we propose the Dynamic Relational Unlearning Framework (DRUF) which comprises a Relational Decoupling Unlearning (RDU) module and a dynamic set update mechanism. It suppresses the leakage of high-risk field pairs while preserving KIE performance.Second, we introduce DocPrivacyBench, a novel
benchmark to systematically evaluate a model's susceptibility to
privacy leakage under conditions of absent or minimal visual evidence.Third, we evaluate three MLLMs and six unlearning methods using this
benchmark, assessing both post-unlearning leakage suppression and utility preservation.Our results demonstrate that existing MLLMs consistently exhibit
privacy leakage when visual evidence is scarce, particularly on noisier
datasets. In contrast, DRUF outperforms the strongest baseline by improving leakage suppression by 4.8 percentage points, effectively mitigating
privacy risks while maintaining robust document information extraction performance.
Read the full abstract ▾
All authors Beining Xu, Hairui Wang, Jiaxin Wang, Changsheng Chen, Anirban Chakraborty
Computer Vision
HounsWorld: A Multimodal World Model for Hidden Patient-State Readout, Reconstruction, and Simulation
Yunhao Bai, Zhongwei Qiu, Guangyu Guo et al. (+5 more)
Clinical intelligence requires estimating a patient's underlying condition from incomplete observations rather than learning isolated mappings from scans to answers. Volumetric medical images provide dense observations of anatomy, attenuation, and lesions, whereas clinical
language provides sparse but complementary semantic observations. We formulate CT-centered intelligence as inference over a shared latent patient state, under which readout, reconstruction, and simulation all become state-dependent prediction problems. To operationalize this view, we introduce HounsBench, a computed
tomography (CT) centric patient-state
benchmark that unifies these three task families with patient-disjoint splits and per-family metrics, and HounsWorld, a 3B
multimodal world model that treats volumetric scans and
language as observations of the shared state through Joint Understanding-Generation Learning. A shared
transformer forms an implicit patient-state estimate and supports three outputs: query-conditioned answers that read out the state, reports and captions that reconstruct it in
language, and condition-specific CT volumes for low-dose denoising, virtual contrast enhancement, and anatomy-constrained text-and-mask-to-volume generation. Zero-initialized CT adapters preserve pretrained
multimodal mappings, while condition-explicit Hounsfield-unit window sampling exposes clinically meaningful density observations. HounsWorld shows strong performance across all three task families while consistently improving CT understanding through clinically structured completion. Our project is available at https://github.com/byhwhite/HounsWorld.git
Read the full abstract ▾
All authors Yunhao Bai, Zhongwei Qiu, Guangyu Guo, Yiming Huang, Tony C.W. Mok, Qinji Yu, Ling Zhang, Yan Wang
Computer Vision
TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
Peng Cai, Zhaofan Zou, Shifa Liu et al. (+7 more)
Document parsing aims to transform unstructured documents into structured and machine-readable representations. Recent advances in
Vision-
Language Models (VLMs) have significantly advanced document parsing. However, existing approaches still face two major challenges. First, decoupled VLM-based methods heavily rely on accurate layout analysis, where
geometric distortions in camera-captured documents can introduce cascading errors. Second, although end-to-end VLM-based methods alleviate the dependence on explicit layout detection, they often suffer from redundant generation,
hallucinations, and insufficient structural
reasoning in high-resolution scenarios. To address these challenges, we propose TeleOCR, a unified framework for document parsing. TeleOCR introduces deformation-aware learning to incorporate
geometric perception into VLMs and proposes an adaptive sampling mechanism for complex layout representation. Furthermore, a content-structure decoupled learning strategy is developed to explicitly model formula grammars and table structures, enabling more effective structured representation learning. Extensive experiments demonstrate that TeleOCR achieves state-of-the-art performance across diverse document parsing
benchmarks. It obtains overall scores of 96.87, 88.53 and 78.41 on OmniDocBench v1.6, Wild-OmniDocBench, and PureDocBench, respectively, and ranks first in the ICDAR 2026 Sci-ImageMiner Challenge. These results validate the effectiveness and generalization capability of TeleOCR in complex document parsing scenarios.
Read the full abstract ▾
All authors Peng Cai, Zhaofan Zou, Shifa Liu, Yikun Wang, Jiawei Tang, Kaicheng Yang, Meng Tong, MingKun Jiang, Zhongjiang He, Hao Sun
Computer Vision
SPARED: Reasoning-Based AI-Generated Image Detection via Adversarially Edited Data
Yicheng Bao, Xiahui Guo, Xuhong Wang et al. (+1 more)
Detecting AI-generated images is only half the task: a deployed detector must also justify its verdict, yet existing detectors inherit three failure modes from their training data: real and fake images collected from different sources invite provenance shortcuts, supervised explanation corpora teach templated rationales, and a static forgery corpus leaves the decision boundary standing still while generators keep moving. We introduce \methodname{}, an
adversarial reinforcement learning framework that pits two heterogeneous models against each other. A
diffusion image editor learns to edit real photographs into fake counterparts of those same photographs that fool the current detector, while a
reasoning MLLM learns to expose them with a verdict grounded in free-form
reasoning. Both rewards are shortcut-proof by design: the attacker is credited only when its edit is faithfully executed, and the defender only when its verdict is correct. As the two models alternate, each round's attacker regenerates a harder training pool aimed at the current detector's blind spots, so the detector must generalize rather than memorize any fixed artifact distribution. Although the explanation is never rewarded, its quality rises round over round as a side effect of accuracy-only training. A detector trained within this loop improves monotonically across rounds on each of three external
benchmarks.
Read the full abstract ▾
All authors Yicheng Bao, Xiahui Guo, Xuhong Wang, Xin Tan
Computer Vision · Cross-listed from cs.RO
BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving
Bing Zhan, Shuyao Shang, Shuo Lu et al. (+6 more)
Autonomous driving requires planning under both semantic constraints and predictive dynamics. Existing end-to-end driving approaches, however, typically emphasize only one side of this requirement:
Vision-
Language-Action (VLA) models exploit VLM priors for semantic
reasoning, while World Action Models (WAMs) provide future-aware prediction through generative world modeling. This naturally motivates a unified planner that can leverage both semantic priors and predictive dynamics. However, we find that a naive combination through joint
token-level
attention suffers from an
attention-allocation mismatch, where semantic shortcuts dominate the shared
attention space and suppress predictive dynamics. Inspired by neuroscience evidence that complex behavior arises from coordination among functionally specialized systems, we propose BrainWAM, a structured action-space coordination framework that converts semantic
reasoning and predictive world modeling into two specialized action-oriented pathways, and aligns them at the level of compact action representations. We further introduce an asynchronous rectified-flow inference strategy with decoupled
video and action denoising, which shortens inference latency while preserving planning-relevant predictive context. BrainWAM reaches state-of-the-art performance on both NAVSIM v1 (89.5 PDMS) and NAVSIM v2 (89.6 EPDMS), consistently outperforming VLA-only or WAM-only methods, highlighting BrainWAM as a practical and promising direction for autonomous driving systems.
Read the full abstract ▾
All authors Bing Zhan, Shuyao Shang, Shuo Lu, Yuan Xu, Zhao Wang, Yida Wang, Xueyang Zhang, Kun Zhan, Jiahao Gu
Computer Vision
Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval
Huu-An Vu, Cam Tu Tran Thi, Thanh Toan Le Ngo et al. (+5 more)
Text-based person anomaly
retrieval aims to retrieve pedestrians exhibiting anomalous behaviors from a large image gallery using natural
language descriptions. Compared with conventional text-based person
retrieval, this task requires fine-grained
reasoning over pedestrian appearance, behaviors, object interactions, and scene context, making robust cross-modal matching significantly more challenging. This paper presents the GENAI4E team's solution to AI City Challenge 2026 Track 4. Our framework builds upon a strong
retrieval backbone and progressively integrates heterogeneous
vision-
language embedding models through score
alignment and iterative ensemble fusion, followed by disagreement-aware VLM reranking for ambiguous queries. On the official Pedestrian Anomaly Behavior (PAB)
benchmark, our approach achieves 90.92% mAP, 85.13% Recall@1, 97.72% Recall@5, and 98.68% Recall@10, demonstrating the effectiveness of combining complementary
vision-
language representations with selective
multimodal reasoning for large-scale text-based person anomaly
retrieval.
Read the full abstract ▾
All authors Huu-An Vu, Cam Tu Tran Thi, Thanh Toan Le Ngo, Hoang Vo, Do Trung Hieu, Hieu Dinh Trung Pham, Khang Minh Le, Huy Minh Nhat Nguyen
Computer Vision
Semantic Steering for Controllable Generation: Tuning-Free Concept Erasure in Multimodal Diffusion Transformers
Qiao Li, Xiaomeng Fu, Yuanshu Zhao et al. (+3 more)
Multimodal Diffusion Transformers (MM-DiTs) have demonstrated remarkable text-to-image generation performance, surpassing traditional U-Net-based
diffusion models. Nevertheless, their powerful generative capabilities also raise significant safety concerns, as they may generate sensitive or inappropriate content. While existing concept erasure methods aim to mitigate such risks, most require modifying model parameters, which are often architecture-specific and impractical for deployed larger models. Several tuning-free approaches face challenges when applied to advanced large-scale MM-DiTs due to their deeply embedded knowledge, broad semantic space, and context-dependent text encoders. To address these challenges, we propose to erase concepts by directly manipulating the model's internal representations. Our key insight, derived from an in-depth analysis of MM-DiT's block-wise generative roles, is that text-conditioned semantic representations are most salient in the middle blocks of MM-DiTs. Based on this, we extract representations of an unwanted concept and a desirable safe one from the middle block, construct a steering vector from their difference, and inject this single vector into consecutive early and middle blocks. By operating exclusively on the sparse text-branch
tokens and leveraging the straight sampling trajectory of rectified flow, our method achieves effective concept erasure with negligible overhead and without any training. Extensive experiments across MM-DiT models demonstrate that our method achieves state-of-the-art performance in erasing diverse concepts, enables effective control over the final output, and remains robust to
adversarial attacks.
Read the full abstract ▾
All authors Qiao Li, Xiaomeng Fu, Yuanshu Zhao, Qipeng Wang, Jiao Dai, Jizhong Han
Computer Vision
Validation of Smartphone-Based Photogrammetric 3D Body Scanning for Automated Anthropometric Measurements Compared with a Commercial Depth-Sensor-Based Body Scanner
Ruting Cheng, Boyuan Feng, Chuhui Qiu et al. (+4 more)
3D body scanning has become an important tool in healthcare applications because of its rapid and non-invasive nature. While smartphone-based photogrammetric reconstruction provide a low-cost and accessible alternative to commercial 3D body scanners, their performance for whole-body scanning remains insufficiently validated. Thus, we designed this study to comprehensively validate the photogrammetric 3D scanning application by evaluating automatically extracted whole-body measurements and longitudinal body-shape monitoring. We evaluated a representative application, PolyCam, against the commercial depth-sensor-based Fit3D ProScanner using 144 pregnant participants scanned longitudinally throughout pregnancy. We designed an automatic circumference extraction pipeline to get measurements at four anatomical landmarks from paired 3D scans. A linear mixed-effects model was used to evaluate scanner effects and longitudinal body-shape changes. Measurement consistency was assessed using repeated PolyCam scans and tape measurements on a rigid mannequin. PolyCam demonstrated strong agreement with Fit3D, with average biases below 16 mm, intraclass correlation coefficients above 0.8, and Pearson correlation coefficients above 0.9 across all landmarks. Both systems captured comparable longitudinal body-shape changes. Mannequin experiments showed mean biases below 3.5 mm and no significant differences from tape measurements. These findings support smartphone photogrammetry as a potential accessible alternative to commercial body scanners and applicable for longitudinal 3D body-shape assessment.
Read the full abstract ▾
All authors Ruting Cheng, Boyuan Feng, Chuhui Qiu, Joaquin A. Calderon, Qing Pan, Yufan Liu, James K. Hahn
Computer Vision
LocusGS: Spatially Grounded Tokens for Feed-Forward 3D Gaussian Splatting
Wenyu Li, Sidun Liu, Tongrui Hu et al. (+2 more)
Recent query-based feed-forward 3DGS methods represent a scene using learnable queries, each aggregating multi-view evidence and decoding a group of Gaussians. Ideally, different queries should specialize in coherent local regions of the scene. However, we observe that Gaussians decoded from the same query often scatter across distant scene regions, resulting in weak query-level spatial coherence and poor
alignment with the scene structure. We attribute this behavior to the purely latent representation of existing Gaussian queries. To address this limitation, we introduce LocusGS, which augments each Gaussian query with a 3D anchor state consisting of a center and a support radius. The anchor state is progressively refined across decoder layers and is used throughout query interaction, multi-view feature aggregation, and Gaussian generation. Specifically, an anchor-to-ray
geometric bias guides each query toward spatially relevant image observations, while anchor-centered decoding organizes its Gaussians within a local region. Experiments on novel view synthesis
benchmarks show that LocusGS improves rendering quality over query-based Gaussian
token baselines under the same Gaussian budget. Further analysis shows that the learned anchors form coherent spatial layouts and lead to more structured Gaussian distributions, demonstrating that explicit anchor states improve the spatial organization. Our project page: https://leo-frank.github.io/LocusGS_viewer.
Read the full abstract ▾
All authors Wenyu Li, Sidun Liu, Tongrui Hu, Peng Qiao, Yong Dou
Computer Vision
Structured Local Differential Modeling for AI-Generated Image Detection
Jiazhen Yang, Ruijin Jin, Junjun Zheng et al. (+3 more)
The rapid advancement of AI-generated content has made the reliable detection of generated images an increasingly critical challenge. Existing detection methods are often dominated during training by semantically salient components with high signal-to-noise ratios (SNRs), thereby suppressing subtler forensic cues associated with the underlying generation mechanisms and embedded in low-level statistical structures. From an information-theoretic perspective, we present a key insight: effective detection in the low-level statistical space requires mitigating the dominance of semantic components while emphasizing and amplifying responses to low-SNR forgery traces. Building on this insight, we propose RippleNet, an AI-generated image detection framework based on local differential signals. RippleNet adaptively identifies forgery-sensitive regions and constructs multi-directional, multi-scale differential representations within local neighborhoods, explicitly characterizing anomalous patterns in neighborhood statistics. More importantly, we refine the
attention mechanism to operate within the local differential representation space, enabling the model to establish explicit dependencies at a finer statistical granularity. This design facilitates the capture of pixel-level forgery traces that are difficult to model using conventional convolutions or image-wide patch-level
attention. Extensive experiments on multiple public
benchmarks and under cross-generator evaluation settings demonstrate that RippleNet achieves consistently competitive performance.
Read the full abstract ▾
All authors Jiazhen Yang, Ruijin Jin, Junjun Zheng, Xiangheng Kong, Zunlei Feng, Jie Lei
Computer Vision
Erase but Preserve: Controllable Removal of Copyrighted Animation Characters via Optimized Semantic Anchors
Qiao Li, Xiaomeng Fu, Wangjia Yu et al. (+4 more)
The exceptional generation capabilities of text-to-image
diffusion models have raised copyright concerns, particularly the unauthorized reproduction of animation characters. Existing concept erasure methods fall short for animation character erasure: model modification methods struggle to identify suitable anchors for diverse, highly distinctive characters;
prompt-based steering methods lack fine-grained control for precise intervention. These approaches often yield incomplete erasure and degraded image fidelity, hindering real-world deployment. In this paper, we propose a controllable method operating on the model's continuous textual representation to erase target characters during generation. We optimizes an anchor
embedding via structural and detailed constraints to serve as a character surrogate, then replaces target-related
embeddings with the anchor via a structure-aware adaptive strategy. Experiments show that our method achieves state-of-the-art erasure effectiveness and image fidelity preservation, while supporting controllable erasure degree, multi-target removal, and model transferability. Moreover, our optimized anchors are plug-and-play with current model modification baselines to improve their erasure performance.
Read the full abstract ▾
All authors Qiao Li, Xiaomeng Fu, Wangjia Yu, Runze He, Baisen Wang, Jiao Dai, Jizhong Han
Computer Vision
Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs
Xinming Wang, Weinong Wang, Hongming Yang et al. (+13 more)
Hybrid-thinking
multimodal large
language models (MLLMs) allow a single model to alternate between deliberative thinking and latency-efficient non-thinking inference. Although these modes differ in
reasoning budget, their delivered responses should satisfy the same user-facing standard. Correctness alone may not characterize this response quality; we therefore evaluate task accuracy and response-pattern failures as complementary outcomes. We study this gap through response-pattern
alignment: whether thinking and non-thinking interfaces preserve acceptable final-response behavior. We introduce PatternEval, a failure-enriched diagnostic
benchmark comprising 2,415
multimodal prompts spanning visual perception and grounding, structured image understanding, and
multimodal knowledge
reasoning. PatternEval tests four recurrent failures: chain-of-thought leakage, response repetition, logical contradiction, and performative
reasoning. Response-pattern failures are widespread across models from different providers, with non-thinking inference exhibiting substantially higher failure rates and thereby creating systematic misalignment between thinking and non-thinking interfaces. Motivated by this diagnosis, we develop PatternRM, a response-level reward model, and PatternRL, which introduces pattern-specific penalties during reinforcement learning. Experiments on Qwen3-VL-4B and Qwen3-VL-8B show that incorporating pattern-specific penalties into reinforcement learning can mitigate cross-mode misalignment while incurring a marginal task performance trade-off. Together, PatternEval and PatternRL provide an evaluation-and-training framework for aligning user-visible response patterns across hybrid-thinking interfaces.
Read the full abstract ▾
All authors Xinming Wang, Weinong Wang, Hongming Yang, Yansong Lin, Zheng Ruan, Shangpin Peng, Qiming Peng, Nan Qiao, Fengyuan Lu, Guoqing Ma, Marito Li, Songyang Zhang, Saiyong Yang, Han Hu, Yonglong Tian, Xu-Yao Zhang
Computer Vision
SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention
Qi Zhao, Qirui Li, Hanlin Tang et al. (+10 more)
Diffusion Transformers (DiTs) incur quadratic self-
attention cost over spatiotemporal
tokens. Existing training-free sparse
attention methods often construct sparse masks from block-level or cluster-level proxy scores, which can obscure fine-grained differences among keys and miss high contribution keys under aggressive sparsity. Moreover, such proxy scores may yield overly concentrated softmax distributions, causing Top-$p$ to retain too few keys for some query clusters. Although a fixed Top-$k$ minimum alleviates this failure mode, a shared value cannot adapt to variations across heads and inputs. To address both limitations, we propose SCOPE, a training-free sparse
attention framework that combines 3D-RoPE-aligned key subspace clustering with online per-head Top-$k$ estimation for efficient
video-DiT inference. SCOPE partitions post-RoPE keys into temporal, height, and width subspaces, clusters them independently, and aggregates the corresponding centroid scores through lookup tables to obtain per key proxy scores for each query cluster. Building on existing hybrid Top-$p$/fixed Top-$k$ selection, SCOPE derives a head-specific Top-$k$ value online by averaging the initial retained key counts within each head, weighted by query cluster size, and selects additional keys only for query clusters whose initial retained key counts fall below this value. Sparse
attention is then computed over the selected original keys and values. Across six model–task configurations, SCOPE consistently outperforms existing training-free baselines in both fidelity and latency, achieving up to a $1.99\times$ end-to-end speedup on 720p HunyuanVideo with $28.46$ dB PSNR relative to dense
attention.
Read the full abstract ▾
All authors Qi Zhao, Qirui Li, Hanlin Tang, Yiduo Li, Zhen Guo, Cuifeng Shen, Chao Xu, Zhaosheng Chi, Xiaojin Lu, Kan Liu, Tao Lan, Lin Qu, Xi Li
Computer Vision
CW-BASS v2: Saturation-Aware Pseudo-Label Selection for Semi-Supervised Segmentation under Foundation-Model Teachers
Ebenezer Tarubinga
Semi-supervised semantic
segmentation has long turned on one question, which pseudo-labels to trust, and a generation of selection rules, dynamic thresholds, per-class curricula, soft confidence weights, answered it for the noisy, under-confident ResNet teachers of their day. Self-supervised foundation encoders change the regime: with a DINOv2 teacher, confidence saturates, so the filtering that helped a weak teacher can hurt a strong one. We propose CW-BASS v2, a saturation-aware pseudo-label selection method that reads the teacher's confidence regime rather than committing to one rule. It pairs held-out calibration, an unbiased per-class noise estimate, with a self-adaptive confidence floor that provably bounds retention away from 1, and combines them in a one-pass gate: measure the reliability of the teacher's confident set, pi_kept = Pr[correct | c >= tau], on a held-out slice, and filter strictly when it meets the confidence demanded (pi_kept >= tau), falling back to the adaptive floor otherwise. The boundary is the pre-existing operating threshold, not a value tuned to mIoU, and across six DINOv2 teachers it makes the correct strict-vs-floor call blind. CW-BASS v2 thus recovers the UniMatch V2 operating point on the saturated
benchmarks by selecting strict (Pascal VOC 1/8 87.4 against its reported 87.9; Cityscapes within 0.5), and improves on it where the confident set is unreliable (pi_kept ~ 89%, ADE20K), where the floor edges ahead (+1.5 mIoU, single seed). The gate is principled because the failure it avoids is measured, not assumed: on a reliable, saturated teacher the confidence distribution's dynamic range collapses (98% of Pascal pixels >= 0.95), so an adaptive cutoff floods the retention mask and self-training decays into confirmation bias.
Read the full abstract ▾
Computer Vision
PatchGen: Learning Soft Intra-Image Predictive Subsets for Visual Generalization
Zhaorui Tan, Weimiao Yu and Xi Yang
Visual classifiers are expected to generalize under data shifts, target shifts, and their combinations, yet most existing methods focus on domain invariance while failing to address intra-image predictive sufficiency. We investigate the structural hypothesis that each image contains a sample-adaptive oracle intra-image predictive subset sufficient for label prediction, while the remaining patches form non-essential complementary context that may correlate with the label. The theoretical analysis shows that restricting prediction to this oracle subset preserves the Bayes risk achievable by the full-patch representation while admitting a complexity bound that tightens with the oracle-subset size. Based on this view, we propose PatchGen, a text-free module that learns a sample-dependent soft predictive-subset mask as a task-driven proxy for the unobserved oracle subset mask. Specifically, histopathology visualizations suggest that PatchGen assigns higher scores to tumor-consistent regions than to some frequently co-occurring inflammatory context. Extensive experiments on natural and histopathological image
benchmarks spanning all three shift settings show that PatchGen improves average performance over matched-backbone baselines in most evaluated configurations, enhances generalization to unknown classes, and remains competitive with
vision-
language methods without text supervision.
Read the full abstract ▾
Computer Vision
Scaling Representation Diversity: Modulated Attention and Reconstructive Regularization for Visual Grounding
Junyi Hu, Tian Bai, Fengyi Wu et al. (+7 more)
Referring Expression Comprehension (REC) is commonly studied under
dataset-specific fine-tuning, resulting in specialist models with limited cross-
dataset generalization. In this work, we revisit REC from the perspective of unified open-vocabulary grounding and identify representation degeneration as a key obstacle to scaling a single generalist model. To preserve representation diversity, we propose a holistic data-model co-design framework. Architecturally, we introduce the Modulated
Attention-Contrastive Head (mACH) for efficient
token-level
vision-
language alignment and a text-conditioned JEPA auxiliary stream that provides complementary gradient support to preserve
alignment-active representations without inference overhead. On the data side, we introduce Objects365-Caption, enriching Objects365 with context-aware referring expressions for large-scale
language supervision. We further provide a theoretical analysis showing that complementary gradient subspaces preserve
alignment capacity and thereby scale representation diversity. Extensive experiments demonstrate that our single-checkpoint framework achieves highly competitive performance on standard REC
benchmarks while exhibiting strong generalization across heterogeneous grounding
datasets without
benchmark-specific adaptation.
Read the full abstract ▾
All authors Junyi Hu, Tian Bai, Fengyi Wu, Yian Huang, Wei Wen, Zaoli Li, Junli Lin, Xingchen Li, Zhenming Peng, Yi Zhang
Computer Vision
Dual-Stream Cross-Anchor Correction Grounding Long-Form Captions and the Domain Limits of Object-Level Anchors
Lingkai Bu, Qian Gao, Jun Fan et al. (+4 more)
Object
hallucination in
multimodal large
language models arises when
language priors and corpus co-occurrence bias outweigh the visual evidence, with nothing tying an object mention to the image. Most remedies intervene at decoding time, yet under a unified protocol their benefit is confined to short captions; supervised fine-tuning (SFT) on a detail-rich corpus lengthens captions, but over forty percent still name absent objects. This paper proposes Dual-Stream Cross-Anchor Correction (DSCC). Unlike work that post-processes decoding, DSCC injects object-level visual anchors into the
language model itself during fine-tuning: a perception stream aligns object-level hidden states at an intermediate layer to frozen text anchors by a bidirectional contrastive objective; a cognition stream lets deeper layers query those anchors by cross-
attention at every generation step; and a two-stage curriculum gate couples them, making evidence
retrieval a structural constraint on generation. Under one backbone and one scoring protocol, experiments span long-caption
hallucination, object-existence discrimination and cross-domain generalisation, with vanilla SFT on the same corpus and schedule as a length- and density-matched control separating the data effect from the architectural gain. DSCC alone reaches the long-caption, low-
hallucination region: captions roughly 1.9 times the baseline length at 88.19% precision per object mention, the highest under a density-independent criterion. Ablations expose a synergy: the perception stream alone degrades precision yet reverses sign when stacked on the cognition stream. No universal superiority is claimed: three out-of-domain
benchmarks yield a predictable, falsifiable domain-conditionality, the synergy being bound to the anchors' semantic domain and breaking on charts and illusions.
Read the full abstract ▾
All authors Lingkai Bu, Qian Gao, Jun Fan, Guohui Ding, Zhenyu Yang, Yuteng Xiao, Jinyi Liang
Computer Vision
Dual-Manifold Geometry Guided Representation Learning: Adaptive Coupling between Kernel and Data Spaces
Wencong Zhang, Yue Zhang, Meiyan Huang et al. (+2 more)
Deep representation learning has primarily focused on how features evolve across network layers, while largely overlooking the structured geometry embedded in network parameters. We introduce a dual-
manifold perspective in which each convolutional layer contains two coupled
geometric spaces: a Kernel
Manifold induced by convolutional filters and a Data
Manifold characterized by intermediate feature representations. Because these
manifolds share the same channel space, parameter geometry can provide complementary structural information to guide feature
evolution. Based on this insight, we propose Kernel-Guided Feature Transform (KGFT), a lightweight module that derives a
geometric guidance
matrix from the kernel Gram
matrix and uses it to transform the covariance structure of feature representations. Unlike conventional
attention mechanisms that reweight feature responses, KGFT explicitly reshapes feature relationships by transferring
geometric information from the kernel
manifold to the data
manifold. To accommodate network hierarchy, we further introduce Exploit and Explore modes with a depth-aware scheduling strategy and a learnable guidance strength that adaptively controls the contribution of
geometric transformation. This design promotes
geometric alignment in shallow layers while encouraging feature diversity in deeper layers, without imposing excessive constraints on representation learning. Theoretical analysis establishes the validity of the proposed transformation and characterizes its effect on feature covariance. Extensive experiments across CNN- and
Transformer-based architectures, including ResNet, ViT, and LLaMA-7B, demonstrate consistent improvements on image classification and arithmetic
reasoning tasks, validating the generality and effectiveness of kernel-guided dual-
manifold representation learning. Code will be publicly available.
Read the full abstract ▾
All authors Wencong Zhang, Yue Zhang, Meiyan Huang, Wei Yang, Qianjin Feng
Computer Vision
A Generative Approach for Improving Multi-Label Defect Classification in Photovoltaic Modules
Abdul Mueez, Yogesh S. Rawat and Shruti Vyas
This paper addresses the challenge of multi-label defect classification in electroluminescence (EL) images of photovoltaic (PV)
cells. Training models on images where multiple defects co-occur creates learning ambiguity, making it difficult to disentangle visual features for specific defect types, a problem compounded by the scarcity of examples for individual classes. To tackle this, we introduce Generative Defect Isolation (GDI), utilizing the LaMa inpainting model with Fast
Fourier Convolutions to remove selected defects and generate realistic, single-defect training samples. Extensive experiments on
Vision Transformer (ViT-S, ViT-L) and EfficientNetV2-L architectures demonstrate that GDI significantly outperforms baselines. The performance gains are most pronounced in low-data scenarios; class-wise analysis shows substantial improvements, boosting the F1-Score for rare defect classes by up to 63.6%. Furthermore, GDI effectively resolves learning ambiguity from co-occurring defects, yielding a 26% reduction in such co-occurring classification errors. Our work establishes GDI as an effective method for maximizing the value of existing
segmentation datasets and sets a new performance
benchmark for multi-label classification in this domain.
Read the full abstract ▾
Computer Vision
VOS-Agent: The 1st Place Solution for the 8th LSVOS Challenge (MOSEv2 Track)
Canyang Wu, Jinrong Zhang, Xusheng He et al. (+3 more)
Complex
video object
segmentation requires robust target propagation under severe occlusion, disappearance and reappearance. Although SAM3 provides strong promptable mask propagation, a uniform inference path remains unreliable for tiny targets with insufficient visual evidence and semantic-dominated targets whose identities depend on explicit attributes. To this end, we present VOS-
Agent, a collaborative framework that retains SAM3 as the shared dense
segmentation module and conditionally activates specialized
agents according to target characteristics. A Target Perception and Routing
Agent assigns each sequence to a regular, tiny, or semantic-dominated route. Tiny targets are supported by a Visual Tracking
Agent through confidence-aware box
prompts, while semantic-dominated targets are handled by an MLLM-based Semantic
Agent through description-guided localization and candidate verification. On the MOSEv2 test set, VOS-
Agent achieves 69.82% on the official $\mathcal{J}\&\dot{\mathcal{F}}$ metric and ranks first in the MOSEv2 Track of the 8th LSVOS Challenge at ECCV 2026.
Read the full abstract ▾
All authors Canyang Wu, Jinrong Zhang, Xusheng He, Ce Bian, Xianjing Han, Jianlong Wu
Computer Vision
Towards Sparsely Annotated Open-World Object Detection
HeeJu Han, AJeong Kim and Jinsun Park
Real-world object detection operates under ambiguous supervision, where unlabeled regions may correspond to missing annotations of known objects or genuinely unknown categories. These challenges have been addressed separately in Sparsely Annotated Object Detection (SAOD) and Open-World Object Detection (OWOD). In practice, their co-occurrence remains an open problem. To address this problem, we introduce Sparsely Annotated Open-World Object Detection (SA-OWOD), a new task that jointly considers sparse supervision and the presence of unseen categories. We propose Dual-Perspective Object Discovery (DPOD), a unified framework that jointly models unlabeled known and unknown instances via two complementary mechanisms. The Known Target Recovery Module (KTRM) recovers supervision for unlabeled known instances and explicitly regularizes the feature space to separate known and unknown representations. Complementarily, the Dual-Disagreement Target Generator (DDTG) identifies reliable unknown candidates through cross-view semantic inconsistency. By integrating these modules, DPOD resolves contradictory supervision signals caused by ambiguous unlabeled regions. As a result, it prevents misclassification between known and unknown objects and stabilizes the decision boundaries. Experimental results on sparsely annotated open-world
benchmarks demonstrate that the proposed method outperforms existing open-world detection methods, particularly in detecting unknown objects.
Read the full abstract ▾
Computer Vision
Class Geometry as Supervision for Sample-Efficient Open-World Detection
Akash Rao, Zhou Chen, Revanth Reddy Palem et al. (+3 more)
Open-world object detection requires models to recognize known categories, reject unfamiliar objects, and incorporate new classes over time. This is especially challenging in scarce-data settings such as biomedical and scientific imaging, where rare categories may have only a few annotated examples and fine-grained classes differ by subtle morphology. Prototype-based detectors are natural for this regime, but they typically learn class prototypes as independent anchors, ignoring relational structure among classes. We propose class-geometry supervision (CGS), a general framework that constrains learned prototype or class-representation spaces to preserve visual or semantic class dissimilarities estimated from training data. CGS introduces a dissimilarity-preserving objective that aligns pairwise distances among learned class representations with a target class-geometry
matrix while retaining the standard task loss. We instantiate the same objective across prototype recognition, few-shot biomedical object detection, open-set detection, novel-class insertion, and OWOD adaptation on COCO. Experiments show that CGS improves sample efficiency in recognition and ova detection, substantially strengthens novel-class insertion, and improves unknown recall on COCO while retaining much of the known-class detection performance. Ablations show that meaningful visual geometry provides the most reliable gains, while random geometry can help novel separation but is less consistent for few-shot detection. These results suggest that relational class geometry is an effective supervisory signal for building calibrated and extensible open-world detectors under limited supervision.
Read the full abstract ▾
All authors Akash Rao, Zhou Chen, Revanth Reddy Palem, Udhav Ramachandran, Ruth Scimeca, Sathyanarayanan N. Aakur
Computer Vision
Mr3D-VL: A generalist vision language foundation model for Multiparametric 3D Magnetic Resonance Imaging
Zhi Qiao, Xintong Wu, Yichu He et al. (+1 more)
Multi-parametric
magnetic resonance imaging (mpMRI) is a cornerstone for
brain tumor diagnosis and treatment, yet current AI models face critical limitations: their lack of natural
language interaction and interpretability impedes spatial information integration and cross-modal
reasoning required clinically. Key challenges arise from significant physical meaning differences across modalities, spatial misalignment due to scan intervals, and the need for complex multi-feature interpretation in tasks like glioma grading. While visual-
language models (VLMs) show promise in cross-modal understanding, existing methods focus mainly on 2D image modeling, neglecting direct perception of 3D volumetric space. Although 3D VLMs have been proposed for report generation and feature
alignment in 3D CT imaging, mpMRI applications demand collaborative inference across multiple imaging modalities-a requirement unmet by current solutions. To address this, we introduce Mr3D-VL, a dedicated visual-
language foundation model for multi-parametric 3D MRI. With 4 billion parameters, it employs an unsupervised pre-trained shared 3D encoder and 4D rotational positional
embedding for dual modality-spatial integration. Its cross-modal projection layer uses a multi-resolution feature implantation strategy to enhance feature perception across resolutions. Experimental results show significant improvements over existing 4B/7B/30B domain-specific and general-purpose models in text generation tasks, achieving a BERTScore of 0.856 for report generation, with question-answering accuracy at 0.713 and multiple-choice accuracy at 0.912.
Read the full abstract ▾
All authors Zhi Qiao, Xintong Wu, Yichu He, Feng Shi
Computer Vision · Cross-listed from cs.RO
FUSE: Active Functional Affordance Grounding through Adaptive Semantic-Geometric Evidence Acquisition
Zhou Chen and Sathyanarayanan N. Aakur
Embodied agents must often identify and interact with objects based on their function rather than their identity, requiring them to actively acquire observations that reveal discriminative functional evidence. Existing affordance grounding methods operate from fixed viewpoints and lack mechanisms for deciding where to look when functional cues are occluded or incomplete. We introduce Active Functional Affordance Grounding, a new task in which an
agent sequentially explores a scene to identify and spatially ground an object satisfying a functional query. To address this problem, we propose FUSE, an adaptive semantic-
geometric evidence acquisition framework that combines explicit uncertainty-driven exploration with a learned amortized planner to efficiently select informative viewpoints. We further introduce a Habitat-based
benchmark for evaluating active functional grounding. Experiments show that FUSE achieves the highest observed non-oracle grounding performance while reducing computation by 1.33x relative to fully explicit exploration, and remains effective across multiple affordance knowledge sources.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
The Role of Natural Language Understanding in Multimodal Video-Based Dengue Diagnosis
Danial Sharifrazi, Saadat Behzadi, Julakha Jahan Jui et al. (+5 more)
Detecting infection-related behavioral changes in mosquitoes from
video data is challenging because mosquitoes are small, move rapidly and irregularly, and are affected by environmental factors such as background, lighting, and shadows, which can make reliable feature extraction difficult. In this study, a YOLO- and Contrastive
Language-Image Pre-training (CLIP)-based
vision-
language framework is proposed to classify mosquito flight frames of uninfected and Dengue virus serotype 2 (DENV2)-infected mosquitoes. First, YOLO is used to isolate mosquito regions from the background. Then, visual features extracted from
video frames are aligned with biologically meaningful textual
prompts in a shared
embedding space. The
multimodal model was fine-tuned using supervised bidirectional contrastive learning and evaluated through frame-level image-text similarity-based classification. The results show that the proposed method achieved 98.54% accuracy and 99.91% sensitivity at the frame level. After temporal aggregation of frame-level information, the model achieved complete
video-level performance. The ablation results showed that fine-tuning and CLIP-based representations were essential for this domain, while the textual branch provided semantic image-text
alignment rather than an accuracy advantage over the
vision-only model. These findings suggest that
vision-
language models can provide a useful framework for analyzing infection-related biological behaviors from
video data.
Read the full abstract ▾
All authors Danial Sharifrazi, Saadat Behzadi, Julakha Jahan Jui, Mojtaba Mohammadi, Nouman Javed, Roohallah Alizadehsani, Prasad N. Paradkar, Asim Bhatti