Computer Vision
PC-CrossDiff: Point-Cluster Dual-Level Cross-Modal Differential Attention for Unified 3D Referring and Segmentation
Wenbin Tan, Jiawen Lin, Fangyong Wang et al. (+4 more)
3D Visual Grounding (3DVG) aims to localize the referent of natural
language referring expressions through two core tasks: Referring Expression Comprehension (3DREC) and
Segmentation (3DRES). While existing methods achieve high accuracy in simple, single-object scenes, they suffer from severe performance degradation in complex, multi-object scenes that are common in real-world settings, hindering practical deployment. Existing methods face two key challenges in complex, multi-object scenes: inadequate parsing of implicit localization cues critical for disambiguating visually similar objects, and ineffective suppression of dynamic spatial interference from co-occurring objects, resulting in degraded grounding accuracy. To address these challenges, we propose PC-CrossDiff, a unified dual-task framework with a dual-level cross-modal differential
attention architecture for 3DREC and 3DRES. Specifically, the framework introduces: (i) Point-Level Differential
Attention (PLDA) modules that apply bidirectional differential
attention between text and point clouds, adaptively extracting implicit localization cues via learnable weights to improve discriminative representation; (ii) Cluster-Level Differential
Attention (CLDA) modules that establish a hierarchical
attention mechanism to adaptively enhance localization-relevant spatial relationships while suppressing ambiguous or irrelevant spatial relations through a localization-aware differential
attention block. Our method achieves state-of-the-art performance on the ScanRefer, NR3D, and SR3D
benchmarks. Notably, on the Implicit subsets of ScanRefer, it improves the Overall@0.50 score by +10.16% for the 3DREC task, highlighting its strong ability to parse implicit spatial cues.
Read the full abstract ▾
All authors Wenbin Tan, Jiawen Lin, Fangyong Wang, Yuan Xie, Yong Xie, Yachao Zhang, Yanyun Qu
Computer Vision
Concept-to-Pixel: Prompt-Free Universal Medical Image Segmentation
Haoyun Chen, Fenghe Tang, Wenxin Ma et al. (+1 more)
Universal medical image
segmentation seeks to use a single foundational model to handle diverse tasks across multiple imaging modalities. However, existing approaches often rely heavily on manual visual
prompts or retrieved reference images, which limits their automation and robustness. In addition, naive joint training across modalities often fails to address large domain shifts. To address these limitations, we propose Concept-to-Pixel (C2P), a novel
prompt-free universal
segmentation framework. C2P explicitly separates anatomical knowledge into two components:
Geometric and Semantic representations. It leverages
Multimodal Large
Language Models (MLLMs) to distill abstract, high-level medical concepts into learnable Semantic
Tokens and introduces explicitly supervised
Geometric Tokens to enforce universal physical and structural constraints. These disentangled
tokens interact deeply with image features to generate input-specific dynamic kernels for precise mask prediction. Furthermore, we introduce a Geometry-Aware Inference Consensus mechanism, which utilizes the model's predicted
geometric constraints to assess prediction reliability and suppress outliers. Extensive experiments and analysis on a unified
benchmark comprising eight diverse
datasets across seven modalities demonstrate the significant superiority of our jointly trained approach, compared to universe- or single-model approaches. Remarkably, our unified model demonstrates strong generalization, achieving impressive results not only on zero-shot tasks involving unseen cases but also in cross-modal transfers across similar tasks. Code is available at: https://github.com/Yundi218/Concept-to-Pixel
Read the full abstract ▾
All authors Haoyun Chen, Fenghe Tang, Wenxin Ma, Shaohua Kevin Zhou
Computer Vision
TAPESTRY: From Geometry to Appearance via Consistent Turntable Videos
Yan Zeng, Haoran Jiang, Kaixin Yao et al. (+4 more)
Automatically generating photorealistic and self-consistent appearances for untextured 3D models is a critical challenge in digital content creation. The advancement of large-scale
video generation models offers a natural approach: directly synthesizing 360-degree turntable
videos (TTVs), which can serve not only as high-quality dynamic previews but also as an intermediate representation to drive texture synthesis and neural rendering. However, existing general-purpose
video diffusion models struggle to maintain strict
geometric consistency and appearance stability across the full range of views, making their outputs ill-suited for high-quality 3D reconstruction. To this end, we introduce TAPESTRY, a framework for generating high-fidelity TTVs conditioned on explicit 3D geometry. We reframe the 3D appearance generation task as a geometry-conditioned
video diffusion problem: given a 3D mesh, we first render and encode multi-modal
geometric features to constrain the
video generation process with pixel-level precision, thereby enabling the creation of high-quality and consistent TTVs. Building upon this, we also design a method for downstream reconstruction tasks from the TTV input, featuring a multi-stage pipeline with 3D-Aware Inpainting. By rotating the model and performing a context-aware secondary generation, this pipeline effectively completes self-occluded regions to achieve full surface coverage. The
videos generated by TAPESTRY are not only high-quality dynamic previews but also serve as a reliable, 3D-aware intermediate representation that can be seamlessly back-projected into UV textures or used to supervise neural rendering methods like 3DGS. This enables the automated creation of production-ready, complete 3D assets from untextured meshes. Experimental results demonstrate that our method outperforms existing approaches in both
video consistency and final reconstruction quality.
Read the full abstract ▾
All authors Yan Zeng, Haoran Jiang, Kaixin Yao, Qixuan Zhang, Longwen Zhang, Lan Xu, Jingyi Yu
Computer Vision
SARE: Sample-wise Adaptive Reasoning for Training-free Fine-grained Visual Recognition
Jingxiao Yang, DaLin He, Miao Pan et al. (+7 more)
Recent advances in Large
Vision-
Language Models (LVLMs) have enabled training-free Fine-Grained Visual Recognition (FGVR). However, effectively exploiting LVLMs for FGVR remains challenging due to the inherent visual ambiguity of subordinate-level categories. Existing methods predominantly adopt either
retrieval-oriented or
reasoning-oriented paradigms to tackle this challenge, but both are constrained by two fundamental limitations:(1) They apply the same inference pipeline to all samples without accounting for uneven recognition difficulty, thereby leading to suboptimal accuracy and efficiency; (2) The lack of mechanisms to consolidate and reuse error-specific experience causes repeated failures on similar challenging cases. To address these limitations, we propose SARE, a Sample-wise Adaptive textbfREasoning framework for training-free FGVR. Specifically, SARE adopts a cascaded design that combines fast candidate
retrieval with fine-grained
reasoning, invoking the latter only when necessary. In the
reasoning process, SARE incorporates a self-reflective experience mechanism that leverages past failures to provide transferable discriminative guidance during inference, without any parameter updates. Extensive experiments across 14
datasets substantiate that SARE achieves state-of-the-art performance while substantially reducing computational overhead.
Read the full abstract ▾
All authors Jingxiao Yang, DaLin He, Miao Pan, Kaixiang Yao, Ge Su, Wenqi Zhang, Yifeng Hu, Tangwei Li, Yuke Li, Xuhong Zhang
Computer Vision
DiffVP: Differential Visual Semantic Prompting for LLM-Based CT Report Generation
Yuhe Tian, Kun Zhang, Haoran Ma et al. (+4 more)
While large
language models (LLMs) have advanced CT report generation, existing methods typically encode 3D volumes holistically, failing to distinguish informative cues from redundant anatomical background. Inspired by radiological cognitive subtraction, we propose Differential Visual Prompting (DiffVP), which conditions report generation on explicit, high-level semantic scan-to-reference differences rather than solely on absolute visual features. DiffVP employs a hierarchical difference extractor to capture complementary global and local semantic discrepancies into a shared latent space, along with a difference-to-
prompt generator that transforms these signals into learnable visual prefix
tokens for LLM conditioning. These difference
prompts serve as structured conditioning signals that implicitly suppress invariant anatomy while amplifying diagnostically relevant visual evidence, thereby facilitating accurate report generation without explicit lesion localization. On two large-scale
benchmarks, DiffVP consistently outperforms prior methods, improving the average BLEU-1-4 by +10.98 and +4.36, respectively, and further boosts clinical efficacy on RadGenome-ChestCT (F1 score 0.421). All codes will be released at https://github.com/ArielTYH/DiffVP/.
Read the full abstract ▾
All authors Yuhe Tian, Kun Zhang, Haoran Ma, Rui Yan, Yingtai Li, Rongsheng Wang, Shaohua Kevin Zhou
Computer Vision
Eye image segmentation using visual and concept prompts with Segment Anything Model 3 (SAM3)
Diederick C. Niehorster and Marcus Nyström
Previous work has reported that
vision foundation models show promising zero-shot performance in eye image
segmentation. Here we examine whether the latest iteration of the Segment Anything Model, SAM3, offers better eye image
segmentation performance than SAM2, and explore the performance of its new concept (text) prompting mode. Eye image
segmentation performance was evaluated using diverse
datasets encompassing both high-resolution high-quality
videos from a lab environment and the TEyeD
dataset consisting of challenging eye
videos acquired in the wild. Results show that in most cases SAM3 with either visual or concept
prompts did not perform better than SAM2, for both lab and in-the-wild
datasets. Since SAM2 not only performed better but was also faster, we conclude that SAM2 remains the best option for eye image
segmentation. We provide our adaptation of SAM3's codebase that allows processing
videos of arbitrary duration.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigation
Jingzhi Huang, Junkai Huang, Haoyang Yang et al. (+2 more)
Zero-Shot Object
Navigation (ZSON) in unknown multi-floor environments presents a significant challenge. Recent methods, mostly based on semantic value greedy waypoint selection, spatial
topology-enhanced
memory, and
Multimodal Large
Language Model (MLLM) as a decision-making framework, have led to improvements. However, these architectures struggle to balance exploration and exploitation for ZSON when encountering unseen environments, especially in multi-floor settings, such as
robots getting stuck at narrow intersections, endlessly wandering, or failing to find stair entrances. To overcome these challenges, we propose AERR-Nav, a Zero-Shot Object
Navigation framework that dynamically adjusts its state based on the
robot's environment. Specifically, AERR-Nav has the following two key advantages: (1) An Adaptive Exploration-Recovery-Reminiscing Strategy, enables
robots to dynamically transition between three states, facilitating specialized responses to diverse
navigation scenarios. (2) An Adaptive Exploration State featuring Fast and Slow-Thinking modes helps
robots better balance exploration, exploitation, and higher-level
reasoning based on evolving environmental information. Extensive experiments on the HM3D and MP3D
benchmarks demonstrate that our AERR-Nav achieves state-of-the-art performance among zero-shot methods. Comprehensive ablation studies further validate the efficacy of our proposed strategy and modules.
Read the full abstract ▾
All authors Jingzhi Huang, Junkai Huang, Haoyang Yang, Haoang Li, Yi Wang
Computer Vision
Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentation
Haocheng Li, Juepeng Zheng, Shuangxi Miao et al. (+4 more)
Multimodal remote sensing semantic
segmentation enhances scene interpretation by exploiting complementary physical cues from heterogeneous data. Although pretrained
Vision Foundation Models (VFMs) provide strong general-purpose representations, adapting them to
multimodal tasks often incurs substantial computational overhead and is prone to modality imbalance, where the contribution of auxiliary modalities is suppressed during optimization. To address these challenges, we propose MoBaNet, a parameter-efficient and modality-balanced symmetric fusion framework. Built upon a largely frozen VFM backbone, MoBaNet adopts a symmetric dual-stream architecture to preserve generalizable representations while minimizing the number of trainable parameters. Specifically, we design a Cross-modal
Prompt-Injected Adapter (CPIA) to enable deep semantic interaction by generating shared
prompts and injecting them into bottleneck adapters under the frozen backbone. To obtain compact and discriminative
multimodal representations for decoding, we further introduce a Difference-Guided Gated Fusion Module (DGFM), which adaptively fuses paired stage features by explicitly leveraging cross-modal discrepancy to guide feature selection. Furthermore, we propose a Modality-Conditional Random Masking (MCRM) strategy to mitigate modality imbalance by masking one modality only during training and imposing hard-pixel auxiliary supervision on modality-specific branches. Extensive experiments on the ISPRS Vaihingen and Potsdam
benchmarks demonstrate that MoBaNet achieves state-of-the-art performance with significantly fewer trainable parameters than full fine-tuning, validating its effectiveness for robust and balanced
multimodal fusion. The source code in this work is available at https://github.com/sauryeo/MoBaNet.
Read the full abstract ▾
All authors Haocheng Li, Juepeng Zheng, Shuangxi Miao, Ruibo Lu, Guosheng Cai, Haohuan Fu, Jianxi Huang
Computer Vision · Cross-listed from cs.GR
DancingBox: A Lightweight MoCap System for Character Animation from Physical Proxies
Haocheng Yuan, Adrien Bousseau, Hao Pan et al. (+2 more)
Creating compelling 3D character animations typically requires either expert use of professional software or expensive motion capture systems operated by skilled actors. We present DancingBox, a lightweight,
vision-based system that makes motion capture accessible to novices by reimagining the process as digital puppetry. Instead of tracking precise human motions, DancingBox captures the approximate movements of everyday objects manipulated by users with a single webcam. These coarse proxy motions are then refined into realistic character animations by conditioning a generative motion model on bounding-box representations, enriched with human motion priors learned from large-scale
datasets. To overcome the lack of paired proxy-animation data, we synthesize training pairs by converting existing motion capture sequences into proxy representations. A user study demonstrates that DancingBox enables intuitive and creative character animation using diverse proxies, from plush toys to bananas, lowering the barrier to entry for novice animators.
Read the full abstract ▾
All authors Haocheng Yuan, Adrien Bousseau, Hao Pan, Lei Zhong, Changjian Li
Computer Vision
Learning Transferable Temporal Primitives for Video Reasoning via Synthetic Videos
Songtao Jiang, Sibo Song, Chenyi Zhou et al. (+12 more)
The transition from image to
video understanding requires
vision-
language models (VLMs) to shift from recognizing static patterns to
reasoning over temporal dynamics such as motion trajectories, speed changes, and state transitions. Yet current post-training methods fall short due to two critical limitations: (1) existing
datasets often lack temporal-centricity, where answers can be inferred from isolated keyframes rather than requiring holistic temporal integration; and (2) training data generated by proprietary models contains systematic errors in fundamental temporal perception, such as confusing motion directions or misjudging speeds. We introduce SynRL, a post-training framework that teaches models temporal primitives, the fundamental building blocks of temporal understanding including direction, speed, and state tracking. Our key insight is that these abstract primitives, learned from programmatically generated synthetic
videos, transfer effectively to real-world scenarios. We decompose temporal understanding into short-term perceptual primitives (speed, direction) and long-term cognitive primitives, constructing 7.7K CoT and 7K RL samples with ground-truth frame-level annotations through code-based
video generation. Despite training on simple
geometric shapes, SynRL achieves substantial improvements across 15
benchmarks spanning temporal grounding, complex
reasoning, and general
video understanding. Remarkably, our 7.7K synthetic CoT samples outperform
Video-R1 with 165K real-world samples. We attribute this to fundamental temporal skills, such as tracking frame by frame changes and comparing velocity, that transfer effectively from abstract synthetic patterns to complex real-world scenarios. This establishes a new paradigm for
video post-training:
video temporal learning through carefully designed synthetic data provides a more cost efficient scaling path.
Read the full abstract ▾
All authors Songtao Jiang, Sibo Song, Chenyi Zhou, Yuan Wang, Ruizhe Chen, Tongkun Guan, Ruilin Luo, Yan Zhang, Zhihang Tang, Yuchong Sun, Hang Zhang, Zhibo Yang, Shuai Bai, Junyang Lin, Zuozhu Liu
Computer Vision
Does YOLO Really Need to See Every Training Image in Every Epoch?
Xingxing Xie, Jiahua Dong, Junwei Han et al. (+1 more)
YOLO detectors are known for their fast inference speed, yet training them remains unexpectedly time-consuming due to their exhaustive pipeline that processes every training image in every epoch, even when many images have already been sufficiently learned. This stands in clear contrast to the efficiency suggested by the “You Only Look Once” philosophy. This naturally raises an important question: Does YOLO really need to see every training image in every epoch? To explore this, we propose an Anti-Forgetting Sampling Strategy (AFSS) that dynamically determines which images should be used and which can be skipped during each epoch, allowing the detector to learn more effectively and efficiently. Specifically, AFSS measures the learning sufficiency of each training image as the minimum of its detection recall and precision, and dynamically categorizes training images into easy, medium, or hard levels accordingly. Easy training images are sparsely resampled during training in a continuous review manner, with priority given to those that have not been used for a long time to reduce redundancy and prevent forgetting. Moderate training images are partially selected, prioritizing recently unused ones and randomly choosing the rest from unselected images to ensure coverage and prevent forgetting. Hard training images are fully sampled in every epoch to ensure sufficient learning. The learning sufficiency of each training image is periodically updated, enabling detectors to adaptively shift its focus toward the informative training images over time while progressively discarding redundant ones. On widely used natural image detection
benchmarks (MS COCO 2017 and PASCAL VOC 2007) and remote sensing detection
datasets (DOTA-v1.0 and DIOR-R), AFSS achieves more than $1.43\times$ training speedup for YOLO-series detectors while also improving accuracy.
Read the full abstract ▾
All authors Xingxing Xie, Jiahua Dong, Junwei Han, Gong Cheng
Computer Vision
Wanjun Du, Zifeng Yuan, Tingting Chen et al. (+3 more)
Existing
vision-
language models (VLMs) have demonstrated impressive performance in
reasoning-based
segmentation. However, current
benchmarks are primarily constructed from high-quality images captured under idealized conditions. This raises a critical question: when visual cues are severely degraded by adverse weather conditions such as rain, snow, or fog, can VLMs sustain reliable
reasoning segmentation capabilities? In response to this challenge, we introduce WeatherReasonSeg, a
benchmark designed to evaluate VLM performance in
reasoning-based
segmentation under adverse weather conditions. It consists of two complementary components. First, we construct a controllable
reasoning dataset by applying synthetic weather with varying severity levels to existing
segmentation datasets, enabling fine-grained robustness analysis. Second, to capture real-world complexity, we curate a real-world adverse-weather
reasoning segmentation dataset with semantically consistent queries generated via mask-guided LLM prompting. We further broaden the evaluation scope across five
reasoning dimensions, including functionality, application scenarios, structural attributes, interactions, and requirement matching. Extensive experiments across diverse VLMs reveal two key findings: (1) VLM performance degrades monotonically with increasing weather severity, and (2) different weather types induce distinct vulnerability patterns. We hope WeatherReasonSeg will serve as a foundation for advancing robust, weather-aware
reasoning.
Read the full abstract ▾
All authors Wanjun Du, Zifeng Yuan, Tingting Chen, Fucai Ke, Beibei Lin, Shunli Zhang
Computer Vision
Illumination-Aware Contactless Fingerprint Spoof Detection via Paired Flash-Non-Flash Imaging
Roja Sahoo and Anoop Namboodiri
Contactless fingerprint recognition enables hygienic and convenient biometric authentication but poses new challenges for spoof detection due to the absence of physical contact and traditional liveness cues. Most existing methods rely on single-image acquisition and appearance-based features, which often generalize poorly across devices, capture conditions, and spoof materials. In this work, we study paired flash-non-flash contactless fingerprint acquisition as a lightweight active sensing mechanism for spoof detection. Through a preliminary empirical analysis, we show that flash illumination accentuates material- and structure-dependent properties, including ridge visibility, subsurface scattering, micro-geometry, and surface oils, while non-flash images provide a baseline appearance context. We analyze lighting-induced differences using interpretable metrics such as inter-channel correlation, specular reflection characteristics, texture realism, and differential imaging. These complementary features help discriminate genuine fingerprints from printed, digital, and molded presentation attacks. We further examine the limitations of paired acquisition, including sensitivity to imaging settings,
dataset scale, and emerging high-fidelity spoofs. Our findings demonstrate the potential of illumination-aware analysis to improve robustness and interpretability in contactless fingerprint presentation attack detection, motivating future work on paired acquisition and physics-informed feature design. Code is available in the repository.
Read the full abstract ▾
Computer Vision
DeepCORO-CLIP: A Multi-View Foundation Model for Comprehensive Coronary Angiography Video-Text Analysis and External Validation
Sarra Harrabi, Yichen Wu, Geoffrey H. Tison et al. (+6 more)
Coronary angiography is the reference standard for evaluating coronary artery disease, yet visual interpretation remains variable between readers. Existing artificial intelligence methods typically analyze single frames or projections and focus mainly on stenosis, limiting comprehensive coronary assessment. We present DeepCORO-CLIP, a multi-view foundation model trained with
video-text contrastive learning on 203,808 angiography
videos from 28,117 patients across 32,473 studies at the Montreal Heart Institute and externally validated on 4,249 studies from the University of California, San Francisco. DeepCORO-CLIP integrates multiple projections with
attention-based pooling for study-level assessment across diagnostic, prognostic, and disease progression tasks. For significant stenosis detection, the model achieved an AUROC of 0.888 internally and 0.89 on external validation. Mean absolute error against core laboratory quantitative coronary angiography was 13.6%, lower than clinical reports at 19.0%. The model also performed strongly for chronic total occlusion, intracoronary thrombus, and coronary calcification detection. Transfer learning enabled prediction of one-year major adverse cardiovascular events with AUROC 0.79 and estimation of left ventricular ejection fraction with mean absolute error 7.3%.
Embeddings also captured disease progression across serial examinations. With a mean inference time of 4.2 seconds in hospital deployment, DeepCORO-CLIP provides a foundation for automated coronary angiography interpretation at the point of care. Code, sample data, model weights, and deployment infrastructure are publicly released.
Read the full abstract ▾
All authors Sarra Harrabi, Yichen Wu, Geoffrey H. Tison, Minhaj Ansari, Milos Vukadinovic, David Ouyang, Joshua P. Barrios, Jacques Delfrate, Robert Avram
Computer Vision
Few-Step Diffusion Sampling Through Instance-Aware Discretizations
Liangyu Yuan, Ruoyu Wang, Tong Zhao et al. (+4 more)
Diffusion and flow matching models generate high-fidelity data by simulating paths defined by Ordinary or
Stochastic Differential Equations (ODEs/SDEs), starting from a tractable prior distribution. The probability flow ODE formulation enables the use of advanced numerical solvers to accelerate sampling. Orthogonal yet vital to solver design is the discretization strategy. While early approaches employed handcrafted heuristics and recent methods adopt optimization-based techniques, most existing strategies enforce a globally shared timestep schedule across all samples. This uniform treatment fails to account for instance-specific complexity in the generative process, potentially limiting performance. Motivated by controlled experiments on synthetic data, which reveals the suboptimality of global schedules under instance-specific dynamics, we propose an instance-aware discretization framework. Our method learns to adapt timestep allocations based on input-dependent priors, extending gradient-based discretization search to the conditional generative setting. Empirical results across diverse settings, including synthetic data, pixel-space
diffusion, latent-space images and
video flow matching models, demonstrate that our method consistently improves generation quality with marginal tuning cost compared to training and negligible inference overhead.
Read the full abstract ▾
All authors Liangyu Yuan, Ruoyu Wang, Tong Zhao, Dingwen Fu, Mingkun Lei, Beier Zhu, Chi Zhang
Computer Vision
FINER: MLLMs Hallucinate under Fine-grained Negative Queries
Rui Xiao, Sanghwan Kim, Yongqin Xian et al. (+2 more)
Multimodal large
language models (MLLMs) struggle with
hallucinations, particularly with fine-grained queries, a challenge underrepresented by existing
benchmarks that focus on coarse image-related questions. We introduce FIne-grained NEgative queRies (FINER), alongside two
benchmarks: FINER-CompreCap and FINER-DOCCI. Using FINER, we analyze
hallucinations across four settings: multi-object, multi-attribute, multi-relation, and “what” questions. Our
benchmarks reveal that MLLMs hallucinate when fine-grained mismatches co-occur with genuinely present elements in the image. To address this, we propose FINER-Tuning, leveraging Direct Preference Optimization (DPO) on FINER-inspired data. Finetuning four frontier MLLMs with FINER-Tuning yields up to 24.2% gains (InternVL3.5-14B) on
hallucinations from our
benchmarks, while simultaneously improving performance on eight existing
hallucination suites, and enhancing general
multimodal capabilities across six
benchmarks. Code,
benchmark, and models are available at \href{https://explainableml.github.io/finer-project/}{https://explainableml.github.io/finer-project/}.
Read the full abstract ▾
All authors Rui Xiao, Sanghwan Kim, Yongqin Xian, Zeynep Akata, Stephan Alaniz
Computer Vision
Interpretable Cross-Domain Few-Shot Learning with Rectified Target-Domain Local Alignment
Yaze Zhao, Yixiong Zou, Yuhua Li et al. (+1 more)
Cross-Domain Few-Shot Learning (CDFSL) adapts models trained with large-scale general data (source domain) to downstream target domains with only scarce training data, where the research on
vision-
language models (e.g., CLIP) is still in the early stages. Typical downstream domains, such as medical diagnosis, require fine-grained visual cues for interpretable recognition, but we find that current fine-tuned CLIP models can hardly focus on these cues, albeit they can roughly focus on important regions in source domains. Although current works have demonstrated CLIP's shortcomings in capturing local subtle patterns, in this paper, we find that the domain gap and scarce training data further exacerbate such shortcomings, much more than that of holistic patterns, which we call the local misalignment problem in CLIP-based CDFSL. To address this problem, due to the lack of supervision in aligning local visual features and text semantics, we turn to self-supervision information. Inspired by the translation task, we propose the CC-CDFSL method with cycle consistency, which translates local visual features into text features and then translates them back into visual features (and vice versa), and constrains the original features close to the translated back features. To reduce the noise imported by richer information in the visual modality, we further propose a Semantic Anchor mechanism, which first augments visual features to provide a larger corpus for the text-to-image mapping, and then shrinks the image features to filter out irrelevant image-to-text mapping. Extensive experiments on various
benchmarks, backbones, and fine-tuning methods show we can (1) effectively improve the local
vision-
language alignment, (2) enhance the interpretability of learned patterns and model decisions by visualizing patches, and (3) achieve state-of-the-art performance.
Read the full abstract ▾
All authors Yaze Zhao, Yixiong Zou, Yuhua Li, Ruixuan Li
Computer Vision · Cross-listed from cs.RO
VectorWorld: Efficient Streaming World Model via Diffusion Flow on Vector Graphs
Chaokang Jiang, Desen Zhou, Jiuming Liu et al. (+1 more)
Closed-loop evaluation of autonomous-driving policies requires interactive simulation beyond log replay. However, existing generative world models often degrade in closed loop due to (i) history-free initialization that mismatches policy inputs, (ii) multi-step sampling latency that violates real-time budgets, and (iii) compounding kinematic infeasibility over long horizons. We propose VectorWorld, a streaming world model that incrementally generates ego-centric $64 \mathrm{m}\times 64\mathrm{m}$ lane–
agent vector-graph tiles during rollout. VectorWorld aligns initialization with history-conditioned policies by producing a policy-compatible interaction state via a motion-aware gated VAE. It enables real-time outpainting via solver-free one-step masked completion with an edge-gated relational DiT trained with interval-conditioned MeanFlow and JVP-based large-step supervision. To stabilize long-horizon rollouts, we introduce $\Delta$Sim, a physics-aligned non-ego (NPC) policy with hybrid discrete–continuous actions and differentiable kinematic logit shaping. On Waymo open motion and nuPlan, VectorWorld improves map-structure fidelity and initialization validity, and supports stable, real-time $1\mathrm{km}+$ closed-loop rollouts (\href{https://github.com/jiangchaokang/VectorWorld}{code}).
Read the full abstract ▾
All authors Chaokang Jiang, Desen Zhou, Jiuming Liu, Kevin Li Sun
Computer Vision
Anchoring and Rescaling Attention for Semantically Coherent Inbetweening
Tae Eun Choi, Sumin Shim, Junhyeok Kim et al. (+1 more)
Generative inbetweening (GI) seeks to synthesize realistic intermediate frames between the first and last keyframes beyond mere interpolation. As sequences become sparser and motions larger, previous GI models struggle with inconsistent frames with unstable pacing and semantic misalignment. Since GI involves fixed endpoints and numerous plausible paths, this task requires additional guidance gained from the keyframes and text to specify the intended path. Thus, we give semantic and temporal guidance from the keyframes and text onto each intermediate frame through Keyframe-anchored
Attention Bias. We also better enforce frame consistency with Rescaled Temporal RoPE, which allows self-
attention to attend to keyframes more faithfully. TGI-Bench, the first
benchmark specifically designed for text-conditioned GI evaluation, enables challenge-targeted evaluation to analyze GI models. Without additional training, our method achieves state-of-the-art frame consistency, semantic fidelity, and pace stability for both short and long sequences across diverse challenges.
Read the full abstract ▾
All authors Tae Eun Choi, Sumin Shim, Junhyeok Kim, Seong Jae Hwang
Computer Vision
Part-Aware Open-Vocabulary 3D Affordance Grounding via Prototypical Semantic and Geometric Alignment
Dongqiang Gou and Xuming He
Grounding natural
language questions to functionally relevant regions in 3D objects – termed
language-driven 3D affordance grounding – is essential for
embodied intelligence and human-AI interaction. Existing methods, while progressing from label-based to
language-driven approaches, still face challenges in open-vocabulary generalization, fine-grained
geometric alignment, and part-level semantic consistency. To address these issues, we propose a novel two-stage cross-modal framework that enhances both semantic and
geometric representations for open-vocabulary 3D affordance grounding. In the first stage, large
language models generate part-aware instructions to recover missing semantics, enabling the model to link semantically similar affordances. In the second stage, we introduce two key components: Affordance Prototype Aggregation (APA), which captures cross-object
geometric consistency for each affordance, and Intra-Object Relational Modeling (IORM), which refines
geometric differentiation within objects to support precise semantic
alignment. We validate the effectiveness of our method through extensive experiments on a newly introduced
benchmark, as well as two existing
benchmarks, demonstrating superior performance in comparison with existing methods.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
DSS-GAN: Directional State Space GAN with Mamba backbone for Class-Conditional Image Synthesis
Aleksander Ogonowski, Konrad Klimaszewski and Przemysław Rokita
We present DSS-GAN, the first generative
adversarial network to employ Mamba as a hierarchical generator backbone for noise-to-image synthesis. The central contribution is Directional Latent Routing (DLR), a novel conditioning mechanism that decomposes the latent vector into direction-specific subvectors, each jointly projected with a class
embedding to produce a feature-wise affine modulation of the corresponding Mamba scan. Unlike conventional class conditioning that injects a global signal, DLR couples class identity and latent structure along distinct spatial axes of the feature map, applied consistently across all generative scales. DSS-GAN achieves improved FID, KID, and precision-recall scores compared to StyleGAN2-ADA across multiple tested
datasets. Analysis of the latent space reveals that directional subvectors exhibit measurable specialization: perturbations along individual components produce structured, direction-correlated changes in the synthesized image.
Read the full abstract ▾
Computer Vision
A Multi-Agent System for Building-Age Cohort Mapping to Support Urban Energy Planning
Kundan Thota, Thorsten Schlachter and Veit Hagenmeyer
Determining the age distribution of the urban building stock is crucial for sustainable municipal heat planning and upgrade prioritization. However, existing approaches often rely on
datasets gathered via sensors or remote sensing techniques, leaving inconsistencies and gaps in data. We present a multi-
agent LLM system comprising three key
agents, the Zensus
agent, the OSM
agent, and the Monument
agent, that fuse data from heterogeneous sources. A data orchestrator and harmonizer geocodes and deduplicates building imprints. Using this fused ground truth, we introduce BuildingAgeCNN, a satellite-only classifier based on a ConvNeXt backbone augmented with a Feature Pyramid Network (FPN), CoordConv spatial channels, and Squeeze-and-Excitation (SE) blocks. Under spatial cross validation, BuildingAgeCNN attains an overall accuracy of 90.69% but a modest macro-F1 of 67.25%, reflecting strong class imbalance and persistent confusions between adjacent historical cohorts. To mitigate risk for planning applications, the address-to prediction pipeline includes calibrated confidence estimates and flags low-confidence cases for manual review. This multi-
agent LLM system not only assists in gathering structured data but also helps energy demand planners optimize district-heating networks and target low-carbon sustainable energy systems.
Read the full abstract ▾
Computer Vision
S-VGGT: Structure-Aware Subscene Decomposition for Scalable 3D Foundation Models
Xinze Li, Pengxu Chen, Yiyuan Wang et al. (+2 more)
Feed-forward 3D foundation models face a key challenge: the quadratic computational cost introduced by global
attention, which severely limits scalability as input length increases. Concurrent acceleration methods, such as
token merging, operate at the
token level. While they offer local savings, the required nearest-neighbor searches introduce undesirable overhead. Consequently, these techniques fail to tackle the fundamental issue of structural redundancy dominant in dense capture data. In this work, we introduce S-VGGT, a novel approach that addresses redundancy at the structural frame level, drastically shifting the optimization focus. We first leverage the initial features to build a dense scene graph, which characterizes structural scene redundancy and guides the subsequent scene partitioning. Using this graph, we softly assign frames to a small number of subscenes, guaranteeing balanced groups and smooth
geometric transitions. The core innovation lies in designing the subscenes to share a common reference frame, establishing a parallel
geometric bridge that enables independent and highly efficient processing without explicit
geometric alignment. This structural reorganization provides strong intrinsic acceleration by cutting the global
attention cost at its source. Crucially, S-VGGT is entirely orthogonal to
token-level acceleration methods, allowing the two to be seamlessly combined for compounded speedups without compromising reconstruction fidelity. Code is available at https://github.com/Powertony102/S-VGGT.
Read the full abstract ▾
All authors Xinze Li, Pengxu Chen, Yiyuan Wang, Weifeng Su, Wentao Cheng
Computer Vision
ReLaGS: Relational Language Gaussian Splatting
Yaxu Xie, Abdalla Arafa, Alireza Javanmardi et al. (+5 more)
Achieving unified 3D perception and
reasoning across tasks such as
segmentation,
retrieval, and relation understanding remains challenging, as existing methods are either object-centric or rely on costly training for inter-object
reasoning. We present a novel framework that constructs a hierarchical
language-distilled Gaussian scene and its 3D semantic scene graph without scene-specific training. A Gaussian
pruning mechanism refines scene geometry, while a robust multi-view
language alignment strategy aggregates noisy 2D features into accurate 3D object
embeddings. On top of this hierarchy, we build an open-vocabulary 3D scene graph with
Vision Language derived annotations and Graph Neural Network-based relational
reasoning. Our approach enables efficient and scalable open-vocabulary 3D
reasoning by jointly modeling hierarchical semantics and inter/intra-object relationships, validated across tasks including open-vocabulary
segmentation, scene graph generation, and relation-guided
retrieval. Project page: https://dfki-av.github.io/ReLaGS/
Read the full abstract ▾
All authors Yaxu Xie, Abdalla Arafa, Alireza Javanmardi, Christen Millerdurai, Jia Cheng Hu, Shaoxiang Wang, Alain Pagani, Didier Stricker
Computer Vision
Trust the Unreliability: Inward Backward Dynamic Unreliability Driven Coreset Selection for Medical Image Classification
Yan Liang, Ziyuan Yang, Zhuxin Lei et al. (+3 more)
Efficiently managing and utilizing large-scale medical imaging
datasets with limited resources presents significant challenges. While coreset selection helps reduce computational costs, its effectiveness in medical data remains limited due to inherent complexity, such as large intra-class variation and high inter-class similarity. To address this, we revisit the training process and observe that neural networks consistently produce stable confidence predictions and better remember samples near class centers in training. However, concentrating on these samples may complicate the modeling of decision boundaries. Hence, we argue that the more unreliable samples are, in fact, the more informative in helping build the decision boundary. Based on this, we propose the Dynamic Unreliability-Driven Coreset Selection(DUCS) strategy. Specifically, we introduce an inward-backward unreliability assessment perspective: 1) Inward Self-Awareness: The model introspects its behavior by analyzing the
evolution of confidence during training, thereby quantifying uncertainty of each sample. 2) Backward
Memory Tracking: The model reflects on its training tracking by tracking the frequency of forgetting samples, thus evaluating its retention ability for each sample. Next, we select unreliable samples that exhibit substantial confidence fluctuations and are repeatedly forgotten during training. This selection process ensures that the chosen samples are near the decision boundary, thereby aiding the model in refining the boundary. Extensive experiments on public medical
datasets demonstrate our superior performance compared to state-of-the-art(SOTA) methods, particularly at high compression rates.
Read the full abstract ▾
All authors Yan Liang, Ziyuan Yang, Zhuxin Lei, Mengyu Sun, Yingyu Chen, Yi Zhang
Computer Vision
Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editing
Seongrae Noh, SeungWon Seo, Gyeong-Moon Park et al. (+1 more)
Editing a 3D indoor scene from natural
language is conceptually straightforward but technically challenging. Existing open-vocabulary systems often regenerate large portions of a scene or rely on image-space edits that disrupt spatial structure, resulting in unintended global changes or physically inconsistent layouts. These limitations stem from treating editing primarily as a generative task. We take a different view. A user instruction defines a desired world state, and editing should be the minimal sequence of actions that makes this state true while preserving everything else. This perspective motivates Edit-As-Act, a framework that performs open-vocabulary scene editing as goal-regressive planning in 3D space. Given a source scene and free-form instruction, Edit-As-Act predicts symbolic goal predicates and plans in EditLang, a PDDL-inspired action
language that we design with explicit preconditions and effects encoding support, contact, collision, and other
geometric relations. A
language-driven planner proposes actions, and a validator enforces goal-directedness, monotonicity, and physical feasibility, producing interpretable and physically coherent transformations. By separating
reasoning from low-level generation, Edit-As-Act achieves instruction fidelity, semantic consistency, and physical plausibility - three criteria that existing paradigms cannot satisfy together. On E2A-Bench, our
benchmark of 63 editing tasks across 9 indoor environments, Edit-As-Act significantly outperforms prior approaches across all edit types and scene categories.
Read the full abstract ▾
All authors Seongrae Noh, SeungWon Seo, Gyeong-Moon Park, HyeongYeop Kang
Computer Vision
LoGSAM: Parameter-Efficient Cross-Modal Grounding for MRI Segmentation
Mohammad Robaitul Islam Bhuiyan, Sheethal Bhat, Melika Qahqaie et al. (+4 more)
Precise localization and delineation of
brain tumors using
magnetic resonance imaging (MRI) are essential for planning therapy and guiding
surgical decisions. To address this, we propose LoGSAM, a parameter-efficient, detection-driven framework that transforms radiologist dictation into text
prompts for foundation-model-based localization and
segmentation. Radiologist
speech is first transcribed and translated using a pretrained Whisper ASR model, followed by negation-aware clinical NLP to extract tumor-specific textual
prompts. These
prompts guide text-conditioned tumor localization via a LoRA-adapted
vision-
language detection model, Grounding DINO (GDINO). The predicted bounding boxes are used as
prompts for MedSAM to generate pixel-level tumor masks without any additional fine-tuning. On BRISC 2025, LoGSAM attains a Dice score of 80.32%, reaching 98.6% of a fully fine-tuned GDINO + MedSAM baseline while training fewer than 5% of its parameters, indicating a favorable accuracy/parameter trade-off. In addition, we evaluate the full pipeline using German dictations from a board-certified radiologist on unseen MRI scans, achieving 91.7% case-level class-extraction accuracy. These results highlight the feasibility of constructing a modular
speech-to-
segmentation pipeline from pretrained foundation models with minimal parameter updates.
Read the full abstract ▾
All authors Mohammad Robaitul Islam Bhuiyan, Sheethal Bhat, Melika Qahqaie, Andreas Maier, Tri-Thien Nguyen, Paula Andrea Pérez-Toro, Tomás Arias-Vergara
Computer Vision
PanoVGGT: Feed-Forward 3D Reconstruction from Panoramic Imagery
Yijing Guo, Mengjun Chao, Luo Wang et al. (+5 more)
Panoramic imagery offers a full 360{\deg} field of view and is increasingly common in consumer devices. However, it introduces non-pinhole distortions that challenge joint pose estimation and 3D reconstruction. Existing feed-forward models, built for perspective cameras, generalize poorly to this setting. We propose PanoVGGT, a permutation-equivariant
Transformer framework that jointly predicts camera poses, depth maps, and 3D point clouds from one or multiple panoramas in a single forward pass. The model incorporates spherical-aware positional
embeddings and a panorama-specific three-axis SO(3) rotation augmentation, enabling effective
geometric reasoning in the spherical domain. To resolve inherent global-frame ambiguity, we further introduce a
stochastic anchoring strategy during training. In addition, we contribute PanoCity, a large-scale outdoor panoramic
dataset with dense depth and 6-DoF pose annotations. Extensive experiments on PanoCity and standard
benchmarks demonstrate that PanoVGGT achieves competitive accuracy, strong robustness, and improved cross-domain generalization. Code and
dataset will be released.
Read the full abstract ▾
All authors Yijing Guo, Mengjun Chao, Luo Wang, Tianyang Zhao, Haizhao Dai, Yingliang Zhang, Jingyi Yu, Yujiao Shi
Computer Vision
Face anonymization preserving facial expressions and photometric realism
Luigi Celona, Simone Bianco and Raimondo Schettini
The widespread sharing of face images on social media platforms and in large-scale
datasets raises pressing
privacy concerns, as biometric identifiers can be exploited without consent. Face anonymization seeks to generate realistic facial images that irreversibly conceal the subject's identity while preserving their usefulness for downstream tasks. However, most existing generative approaches focus on identity removal and image realism, often neglecting facial expressions as well as photometric consistency – specifically attributes such as illumination and skin tone – that are critical for applications like relighting, color constancy, and medical or affective analysis. In this work, we propose a feature-preserving anonymization framework that extends DeepPrivacy by incorporating dense facial landmarks to better retain expressions, and by introducing lightweight post-processing modules that ensure consistency in lighting direction and skin color. We further establish evaluation metrics specifically designed to quantify expression fidelity, lighting consistency, and color preservation, complementing standard measures of image realism, pose accuracy, and re-identification resistance. Experiments on the CelebA-HQ
dataset demonstrate that our method produces anonymized faces with improved realism and significantly higher fidelity in expression, illumination, and skin tone compared to state-of-the-art baselines. These results underscore the importance of feature-aware anonymization as a step toward more useful, fair, and trustworthy
privacy-preserving facial data.
Read the full abstract ▾
Computer Vision
FrescoDiffusion: 4K Image-to-Video with Prior-Regularized Tiled Diffusion
Hugo Caselles-Dupré, Mathis Koroglu, Guillaume Jeanneret et al. (+2 more)
Diffusion-based image-to-
video (I2V) models are increasingly effective, yet they struggle to scale to ultra-high-resolution inputs (e.g., 4K). Generating
videos at the model's native resolution often loses fine-grained structure, whereas high-resolution tiled denoising preserves local detail but breaks global layout consistency. This failure mode is particularly severe in the fresco animation setting: monumental artworks containing many distinct characters, objects, and semantically different sub-scenes that must remain spatially coherent over time. We introduce FrescoDiffusion, a training-free method for coherent large-format I2V generation from a single complex image. The key idea is to augment tiled denoising with a precomputed latent prior: we first generate a low-resolution
video at the underlying model resolution and upsample its latent trajectory to obtain a global reference that captures long-range temporal and spatial structure. For 4K generation, we compute per-tile noise predictions and fuse them with this reference at every
diffusion timestep by minimizing a single weighted least-squares objective in model-output space. The objective combines a standard tile-merging criterion with our regularization term, yielding a closed-form fusion update that strengthens global coherence while retaining fine detail. We additionally provide a spatial regularization variable that enables region-level control over where motion is allowed. Experiments on the VBench-I2V
dataset and our proposed fresco I2V
dataset show improved global consistency and fidelity over tiled baselines, while being computationally efficient. Our regularization enables explicit controllability of the trade-off between creativity and consistency.
Read the full abstract ▾
All authors Hugo Caselles-Dupré, Mathis Koroglu, Guillaume Jeanneret, Arnaud Dapogny, Matthieu Cord