Computer Vision
DSFlash: Comprehensive Panoptic Scene Graph Generation in Realtime
Julian Lorenz, Vladyslav Kovganko, Elias Kohout et al. (+3 more)
Scene Graph Generation (SGG) aims to extract a detailed graph structure from an image, a representation that holds significant promise as a robust intermediate step for complex downstream tasks like
reasoning for
embodied agents. However, practical deployment in real-world applications - especially on resource constrained edge devices - requires speed and resource efficiency, challenges that have received limited
attention in existing research. To bridge this gap, we introduce DSFlash, a low-latency model for panoptic scene graph generation designed to overcome these limitations. DSFlash can process a
video stream at 56 frames per second on a standard RTX 3090 GPU, without compromising performance against existing state-of-the-art methods. Crucially, unlike prior approaches that often restrict themselves to salient relationships, DSFlash computes comprehensive scene graphs, offering richer contextual information while maintaining its superior latency. Furthermore, DSFlash is light on resources, requiring less than 24 hours to train on a single, nine-year-old GTX 1080 GPU. This accessibility makes DSFlash particularly well-suited for researchers and practitioners operating with limited computational resources, empowering them to adapt and fine-tune SGG models for specialized applications.
Read the full abstract ▾
All authors Julian Lorenz, Vladyslav Kovganko, Elias Kohout, Mrunmai Phatak, Daniel Kienzle, Rainer Lienhart
Computer Vision
Sparse Task Vector Mixup with Hypernetworks for Efficient Knowledge Transfer in Whole-Slide Image Prognosis
Pei Liu, Xiangxiang Zeng, Tengfei Ma et al. (+3 more)
Whole-Slide Images (WSIs) are widely used for estimating the prognosis of cancer patients. Current studies generally follow a cancer-specific learning paradigm. However, the available training samples for one cancer type are usually scarce in pathology. Consequently, the model often struggles to learn generalizable knowledge, thus performing worse on the tumor samples with inherent high heterogeneity. Although multi-cancer joint learning and knowledge transfer approaches have been explored recently to address it, they either rely on large-scale joint training or extensive inference across multiple models, posing new challenges in computational efficiency. To this end, this paper proposes a new scheme, Sparse Task Vector Mixup with Hypernetworks (STEPH). Unlike previous ones, it efficiently absorbs generalizable knowledge from other cancers for the target via model merging: i) applying task vector mixup to each source-target pair and then ii) sparsely aggregating task vector mixtures to obtain an improved target model, driven by hypernetworks. Extensive experiments on 13 cancer
datasets show that STEPH improves over cancer-specific learning and an existing knowledge transfer baseline by 5.14% and 2.01%, respectively. Moreover, it is a more efficient solution for learning prognostic knowledge from other cancers, without requiring large-scale joint training or extensive multi-model inference. Code is publicly available at https://github.com/liupei101/STEPH.
Read the full abstract ▾
All authors Pei Liu, Xiangxiang Zeng, Tengfei Ma, Yucheng Xing, Xuanbai Ren, Yiping Liu
Computer Vision
Visually-Guided Controllable Medical Image Generation via Fine-Grained Semantic Disentanglement
Xin Huang, Junjie Liang, Qingshan Hou et al. (+4 more)
Medical image synthesis is crucial for alleviating data scarcity and
privacy constraints. However, fine-tuning general text-to-image (T2I) models remains challenging, mainly due to the significant modality gap between complex visual details and abstract clinical text. In addition, semantic
entanglement persists, where coarse-grained text
embeddings blur the boundary between anatomical structures and imaging styles, thus weakening controllability during generation. To address this, we propose a Visually-Guided Text Disentanglement framework. We introduce a cross-modal latent
alignment mechanism that leverages visual priors to explicitly disentangle unstructured text into independent semantic representations. Subsequently, a Hybrid Feature Fusion Module (HFFM) injects these features into a
Diffusion Transformer (DiT) through separated channels, enabling fine-grained structural control. Experimental results in three
datasets demonstrate that our method outperforms existing approaches in terms of generation quality and significantly improves performance on downstream classification tasks. The source code is available at https://github.com/hx111/VG-MedGen.
Read the full abstract ▾
All authors Xin Huang, Junjie Liang, Qingshan Hou, Peng Cao, Jinzhu Yang, Xiaoli Liu, Osmar R. Zaiane
Computer Vision
UHD Image Deblurring via Autoregressive Flow with Ill-conditioned Constraints
Yucheng Xin, Dawei Zhao, Xiang Chen et al. (+6 more)
Ultra-high-definition (UHD) image deblurring poses significant challenges for UHD restoration methods, which must balance fine-grained detail recovery and practical inference efficiency. Although prominent discriminative and generative methods have achieved remarkable results, a trade-off persists between computational cost and the ability to generate fine-grained detail for UHD image deblurring tasks. To further alleviate these issues, we propose a novel autoregressive flow method for UHD image deblurring with an ill-conditioned constraint. Our core idea is to decompose UHD restoration into a progressive, coarse-to-fine process: at each scale, the sharp estimate is formed by upsampling the previous-scale result and adding a current-scale residual, enabling stable, stage-wise refinement from low to high resolution. We further introduce Flow Matching to model residual generation as a conditional vector field and perform few-step ODE sampling with efficient Euler/Heun solvers, enriching details while keeping inference affordable. Since multi-step generation at UHD can be numerically unstable, we propose an ill-conditioning suppression scheme by imposing condition-number regularization on a feature-induced
attention matrix, improving convergence and cross-scale consistency. Our method demonstrates promising performance on blurred images at 4K (3840$\times$2160) or higher resolutions.
Read the full abstract ▾
All authors Yucheng Xin, Dawei Zhao, Xiang Chen, Chen Wu, Pu Wang, Dianjie Lu, Guijuan Zhang, Xiuyi Jia, Zhuoran Zheng
Computer Vision · Cross-listed from cs.CR
Naïve Exposure of Generative AI Capabilities Undermines Deepfake Detection
Sunpill Kim, Chanwoo Hwang, Minsu Kim et al. (+1 more)
Generative AI systems increasingly expose powerful
reasoning and image refinement capabilities through user-facing chatbot interfaces. In this work, we show that the naïve exposure of such capabilities fundamentally undermines modern deepfake detectors. Rather than proposing a new image manipulation technique, we study a realistic and already-deployed usage scenario in which an adversary uses only benign, policy-compliant
prompts and commercial generative AI systems. We demonstrate that state-of-the-art deepfake detection methods fail under semantic-preserving image refinement. Specifically, we show that generative AI systems articulate explicit authenticity criteria and inadvertently externalize them through unrestricted
reasoning, enabling their direct reuse as refinement objectives. As a result, refined images simultaneously evade detection, preserve identity as verified by commercial face recognition APIs, and exhibit substantially higher perceptual quality. Importantly, we find that widely accessible commercial chatbot services pose a significantly greater security risk than open-source models, as their superior realism, semantic controllability, and low-barrier interfaces enable effective evasion by non-expert users. Our findings reveal a structural mismatch between the threat models assumed by current detection frameworks and the actual capabilities of real-world generative AI. While detection baselines are largely shaped by prior
benchmarks, deployed systems expose unrestricted authenticity
reasoning and refinement despite stringent safety controls in other domains.
Read the full abstract ▾
All authors Sunpill Kim, Chanwoo Hwang, Minsu Kim, Jae Hong Seo
Computer Vision
IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translation
Jiahao Lyu, Pei Fu, Zhenhang Li et al. (+7 more)
End-to-end In-Image Machine Translation (IIMT) aims to convert text embedded within an image into a target
language while preserving the original visual context, layout, and rendering style. However, existing IIMT
benchmarks are largely synthetic and thus fail to reflect real-world complexity, while current evaluation protocols focus on single-modality metrics and overlook cross-modal faithfulness between rendered text and model outputs. To address these shortcomings, we present In-image Machine Translation
Benchmark (IMTBench), a new
benchmark of 2,500 image translation samples covering four practical scenarios and nine
languages. IMTBench supports multi-aspect evaluation, including translation quality, background preservation, overall image quality, and a cross-modal
alignment score that measures consistency between the translated text produced by the model and the text rendered in the translated image. We
benchmark strong commercial cascade systems, and both closed- and open-source unified multi-modal models, and observe large performance gaps across scenarios and
languages, especially on natural scenes and resource-limited
languages, highlighting substantial headroom for end-to-end image text translation. We hope IMTBench establishes a standardized
benchmark to accelerate progress in this emerging task.
Read the full abstract ▾
All authors Jiahao Lyu, Pei Fu, Zhenhang Li, Weichao Zeng, Shaojie Zhang, Jiahui Yang, Can Ma, Yu Zhou, Zhenbo Luo, Jian Luan
Computer Vision
Spatial self-supervised Peak Learning and correlation-based Evaluation of peak picking in Mass Spectrometry Imaging
Philipp Weigand, Nikolas Ebert, Shad A. Mohammed et al. (+3 more)
Mass spectrometry imaging (MSI) enables label-free visualization of molecular distributions across tissue samples but generates large and complex
datasets that require effective peak picking to reduce data size while preserving meaningful biological information. Existing peak picking approaches perform inconsistently across heterogeneous
datasets, and their evaluation is often limited to synthetic data or manually selected ion images that do not fully represent real-world challenges in MSI. To address these limitations, we propose an autoencoder-based spatial self-supervised peak learning neural network that selects spatially structured peaks by learning an
attention mask leveraging both spatial and spectral information. We further introduce an evaluation procedure based on expert-annotated
segmentation masks, allowing a more representative and spatially grounded assessment of peak picking performance. We evaluate our approach on four diverse public MSI
datasets using our proposed evaluation procedure. Our approach consistently outperforms state-of-the-art peak picking methods by selecting spatially structured peaks, thus demonstrating its efficacy. These results highlight the value of our spatial self-supervised network in comparison to contemporary state-of-the-art methods. The evaluation procedure can be readily applied to new MSI
datasets, thereby providing a consistent and robust framework for the comparison of spatially structured peak picking methods across different
datasets.
Read the full abstract ▾
All authors Philipp Weigand, Nikolas Ebert, Shad A. Mohammed, Denis Abu Sammour, Carsten Hopf, Oliver Wasenmüller
Computer Vision
StructDamage:A Large Scale Unified Crack and Surface Defect Dataset for Robust Structural Damage Detection
Misbah Ijaz, Saif Ur Rehman Khan, Abd Ur Rehman et al. (+3 more)
Automated detection and classification of structural cracks and surface defects is a critical challenge in civil engineering, infrastructure maintenance, and heritage preservation. Recent advances in Computer
Vision (CV) and Deep Learning (DL) have significantly improved automatic crack detection. However, these methods rely heavily on large, diverse, and carefully curated
datasets that include various crack types across different surface materials. Many existing public crack
datasets lack geographic diversity, surface types, scale, and labeling consistency, making it challenging for trained
algorithms to generalize effectively in real world conditions. We provide a novel
dataset, StructDamage, a curated collection of approximately 78,093 images spanning nine surface types: walls, tile, stone, road, pavement, deck, concrete, and brick. The
dataset was constructed by systematically aggregating, harmonizing, and reannotating images from 32 publicly available
datasets covering concrete structures, asphalt pavements, masonry walls, bridges, and historic buildings. All images are organized in a folder level classification hierarchy suitable for training Convolutional Neural Networks (CNNs) and
Vision Transformers. To highlight the practical value of the
dataset, we present baseline classification results using fifteen DL architectures from six model families, with twelve achieving macro F1-scores over 0.96. The best performing model DenseNet201 achieves 98.62% accuracy. The proposed
dataset provides a comprehensive and versatile resource suitable for classification tasks. With thorough documentation and a standard structure, it is designed to promote reproducible research and support the development and fair evaluation of robust crack damage detection approaches.
Read the full abstract ▾
All authors Misbah Ijaz, Saif Ur Rehman Khan, Abd Ur Rehman, Sebastian Vollmer, Andreas Dengel, Muhammad Nabeel Asim
Computer Vision
Fighting Hallucinations with Counterfactuals: Diffusion-Guided Perturbations for LVLM Hallucination Suppression
Hamidreza Dastmalchi, Aijun An, Ali Cheraghian et al. (+1 more)
While large
vision-
language models (LVLMs) achieve strong performance on
multimodal tasks, they frequently generate
hallucinations – unfaithful outputs misaligned with the visual input. To address this issue, we introduce CIPHER (Counterfactual Image Perturbations for
Hallucination Extraction and Removal), a training-free method that suppresses
vision-induced
hallucinations via lightweight feature-level correction. Unlike prior training-free approaches that primarily focus on text-induced
hallucinations, CIPHER explicitly targets
hallucinations arising from the visual modality. CIPHER operates in two phases. In the offline phase, we construct OHC-25K (Object-Hallucinated Counterfactuals, 25,000 samples), a counterfactual
dataset consisting of
diffusion-edited images that intentionally contradict the original ground-truth captions. We pair these edited images with the unchanged ground-truth captions and process them through an LVLM to extract
hallucination-related representations. Contrasting these representations with those from authentic (image, caption) pairs reveals structured, systematic shifts spanning a low-rank subspace characterizing
vision-induced
hallucination. In the inference phase, CIPHER suppresses
hallucinations by projecting intermediate hidden states away from this subspace. Experiments across multiple
benchmarks show that CIPHER significantly reduces
hallucination rates while preserving task performance, demonstrating the effectiveness of counterfactual visual perturbations for improving LVLM faithfulness. Code and additional materials are available at https://hamidreza-dastmalchi.github.io/cipher-cvpr2026/.
Read the full abstract ▾
All authors Hamidreza Dastmalchi, Aijun An, Ali Cheraghian, Hamed Barzamini
Computer Vision
UniPINN: A Unified PINN Framework for Multi-task Learning of Diverse Navier-Stokes Equations
Dengdi Sun, Jie Chen, Xiao Wang et al. (+1 more)
Physics-Informed Neural Networks (PINNs) have shown promise in solving incompressible Navier-Stokes equations, yet existing approaches are predominantly designed for single-flow settings. When extended to multi-flow scenarios, these methods face three key challenges: (1) difficulty in simultaneously capturing both shared physical principles and flow-specific characteristics, (2) susceptibility to inter-task negative transfer that degrades prediction accuracy, and (3) unstable training dynamics caused by disparate loss magnitudes across heterogeneous flow regimes. To address these limitations, we propose UniPINN, a unified multi-flow PINN framework that integrates three complementary components: a shared-specialized architecture that disentangles universal physical laws from flow-specific features, a cross-flow
attention mechanism that selectively reinforces relevant patterns while suppressing task-irrelevant interference, and a dynamic weight allocation strategy that adaptively balances loss contributions to stabilize multi-objective optimization. Extensive experiments on three canonical flows demonstrate that UniPINN effectively unifies multi-flow learning, achieving superior prediction accuracy and balanced performance across heterogeneous regimes while successfully mitigating negative transfer. The source code of this paper will be released on https://github.com/Event-AHU/OpenFusion
Read the full abstract ▾
All authors Dengdi Sun, Jie Chen, Xiao Wang, Jin Tang
Computer Vision · Cross-listed from cs.SD
MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR
Tianyu Xu, Sieun Kim, Qianhui Zheng et al. (+6 more)
In Extended Reality (XR), complex acoustic environments often overwhelm users, compromising both scene awareness and social engagement due to entangled sound sources. We introduce MoXaRt, a real-time XR system that uses audio-visual cues to separate these sources and enable fine-grained sound interaction. MoXaRt's core is a cascaded architecture that performs coarse, audio-only separation in parallel with visual detection of sources (e.g., faces, instruments). These visual anchors then guide refinement networks to isolate individual sources, separating complex mixes of up to 5 concurrent sources (e.g., 2 voices + 3 instruments) with ~2 second processing latency. We validate MoXaRt through a technical evaluation on a new
dataset of 30 one-minute recordings featuring concurrent
speech and music, and a 22-participant user study. Empirical results indicate that our system significantly enhances
speech intelligibility, yielding a 36.2% (p < 0.01) increase in listening comprehension within
adversarial acoustic environments while substantially reducing cognitive load (p < 0.001), thereby paving the way for more perceptive and socially adept XR experiences.
Read the full abstract ▾
All authors Tianyu Xu, Sieun Kim, Qianhui Zheng, Ruoyu Xu, Tejasvi Ravi, Anuva Kulkarni, Katrina Passarella-Ward, Junyi Zhu, Adarsh Kowdle
Computer Vision
Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning
Yushuo Zheng, Huiyu Duan, Zicheng Zhang et al. (+2 more)
Geolocation, the task of identifying the geographic location of an image, requires abundant world knowledge and complex
reasoning abilities. Though advanced large
multimodal models (LMMs) have shown superior aforementioned capabilities, their performance on the geolocation task remains unexplored. To this end, we introduce WanderBench, the first open access global geolocation
benchmark designed for actionable geolocation
reasoning in
embodied scenarios. WanderBench contains over 32K panoramas across six continents, organized as navigable graphs that enable physical actions such as rotation and movement, transforming geolocation from static recognition into interactive exploration. Building on this foundation, we propose GeoAoT (Action of Thought), a \underline{Geo}location framework with \underline{A}ction of \underline{T}hough, which couples
reasoning with
embodied actions. Instead of generating textual
reasoning chains, GeoAoT produces actionable plans such as, approaching landmarks or adjusting viewpoints, to actively reduce uncertainty. We further establish an evaluation protocol that jointly measures geolocation accuracy and difficulty-aware geolocation questioning ability. Experiments on 19 large
multimodal models show that GeoAoT achieves superior fine-grained localization and stronger generalization in dynamic environments. WanderBench and GeoAoT define a new paradigm for actionable,
reasoning driven geolocation in
embodied visual understanding.
Read the full abstract ▾
All authors Yushuo Zheng, Huiyu Duan, Zicheng Zhang, Xiaohong Liu, Xiongkuo Min
Computer Vision
LCAMV: High-Accuracy 3D Reconstruction of Color-Varying Objects Using LCA Correction and Minimum-Variance Fusion in Structured Light
Wonbeen Oh and Jae-Sang Hyun
Accurate 3D reconstruction of colored objects with structured light (SL) is hindered by lateral chromatic aberration (LCA) in optical components and uneven noise characteristics across RGB channels. This paper introduces lateral chromatic aberration correction and minimum-variance fusion (LCAMV), a robust 3D reconstruction method that operates with a single projector-camera pair without additional hardware or acquisition constraints. LCAMV analytically models and pixel-wise compensates LCA in both the projector and camera, then adaptively fuses multi-channel phase data using a Poisson-Gaussian noise model and minimum-variance estimation. Unlike existing methods that require extra hardware or multiple exposures, LCAMV enables fast acquisition. Experiments on planar and non-planar colored surfaces show that LCAMV outperforms grayscale conversion and conventional channel-weighting, reducing depth error by up to 43.6%. These results establish LCAMV as an effective solution for high-precision 3D reconstruction of nonuniformly colored objects.
Read the full abstract ▾
Computer Vision
SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning
Jianhe Low, Alexandre Symeonidis-Herzig, Maksym Ivashechkin et al. (+2 more)
Sign
Language Production (SLP) faces a fundamental trade-off: direct text-to-pose models suffer from regression-to-the-mean effects, while dictionary-
retrieval methods produce disjointed transitions. To resolve this, we propose a novel training paradigm that leverages sparse keyframes to capture the underlying kinematic distribution of human signing. By generating dense motion from discrete anchors, our approach mitigates regression-to-the-mean while ensuring fluid articulation. To achieve this at scale, we introduce FAST, an ultra-efficient sign
segmentation model that automatically mines precise temporal boundaries. We then present SignSparK, a Conditional Flow Matching (CFM) framework that utilizes these temporal anchors to synthesize 3D signing sequences. This keyframe-driven formulation also unlocks Keyframe-to-Pose (KF2P) generation, making precise spatiotemporal editing of signing sequences possible. Furthermore, SignSparK scales across four distinct sign
languages, constituting the largest
multilingual SLP framework to date, and integrates 3D Gaussian Splatting for photorealistic rendering. Extensive evaluations demonstrate that SignSparK achieves state-of-the-art across diverse SLP tasks and
multilingual benchmarks. Our code is available at https://github.com/JianHe0628/SignSparK.
Read the full abstract ▾
All authors Jianhe Low, Alexandre Symeonidis-Herzig, Maksym Ivashechkin, Ozge Mercanoglu Sincan, Richard Bowden
Computer Vision · Cross-listed from cs.LG
Unlearning the Unpromptable: Prompt-free Instance Unlearning in Diffusion Models
Kyungryeol Lee, Kyeonghyun Lee, Seongmin Hong et al. (+2 more)
Machine unlearning aims to remove specific outputs from trained models, often at the concept level, such as forgetting all occurrences of a particular celebrity or filtering content via text
prompts. However, many undesired outputs, such as an individual's face or generations culturally or factually misinterpreted, cannot often be specified by text
prompts. We address this underexplored setting of instance unlearning for outputs that are undesired but unpromptable, where the goal is to forget target outputs selectively while preserving the rest. To this end, we introduce an effective surrogate-based unlearning method that leverages image editing, timestep-aware weighting, and gradient surgery to guide trained
diffusion models toward forgetting specific outputs. Experiments on conditional (Stable
Diffusion 3) and unconditional (DDPM-CelebA)
diffusion models demonstrate that our
prompt-free method uniquely unlearns unpromptable outputs, such as faces and culturally inaccurate depictions, with preserved integrity, unlike
prompt-based and
prompt-free baselines. Our proposed method would serve as a practical hotfix for
diffusion model providers to ensure
privacy protection and ethical compliance.
Read the full abstract ▾
All authors Kyungryeol Lee, Kyeonghyun Lee, Seongmin Hong, Byung Hyun Lee, Se Young Chun
Computer Vision · Cross-listed from cs.RO
AsyncMDE: Real-Time Monocular Depth Estimation via Asynchronous Spatial Memory
Lianjie Ma, Yuquan Li, Bingzheng Jiang et al. (+3 more)
Foundation-model-based
monocular depth estimation offers a viable alternative to active sensors for
robot perception, yet its computational cost often prohibits deployment on edge platforms. Existing methods perform independent per-frame inference, wasting the substantial computational redundancy between adjacent viewpoints in continuous
robot operation. This paper presents AsyncMDE, an asynchronous depth perception system consisting of a frozen foundation model and a lightweight fast path that amortizes the foundation model's computational cost over time. The foundation model periodically produces high-quality spatial features in the background, while the lightweight fast path runs asynchronously in the foreground, fusing cached
memory with current observations through complementary fusion, outputting depth estimates, and autoregressively updating
memory. This enables cross-frame feature reuse with bounded accuracy degradation. With 3.83M trainable fast-path parameters and a 97.5M frozen slow path, AsyncMDE's fast path operates at 237 FPS on an RTX 4090, recovering 77% of the accuracy gap to the foundation model. Across indoor static, dynamic, and synthetic extreme-motion
benchmarks, AsyncMDE degrades predictably and reaches 161 FPS fast-path inference on a TensorRT-optimized Jetson AGX Orin, supporting real-time edge deployment.
Read the full abstract ▾
All authors Lianjie Ma, Yuquan Li, Bingzheng Jiang, Ziming Zhong, Han Ding, Lijun Zhu
Computer Vision
World2Act: Latent Action Post-Training from World Model Dynamics
An Dinh Vuong, Tuan Van Vo, Abdullah Sohail et al. (+6 more)
World Models (WMs) offer a promising mechanism for post-training
Vision-
Language-Action (VLA) policies by providing dynamics priors that improve generalization under task and scene variation. However, most WM-based post-training methods rely on pixel-space supervision, making policies sensitive to visual artifacts introduced by imperfect WM rollouts. We present World2Act, a latent-space post-training framework that transfers WM dynamics to the VLA policy without pixel-space supervision. World2Act operates in two stages: 1) it induces a shared
video-action latent space by contrastively aligning WM-dynamics latents with action
embeddings, and 2) it post-trains the VLA by guiding policy action representations toward WM-imagined dynamics rather than decoded pixels. Built on GR00T-N1.6, World2Act delivers absolute success-rate gains of up to +2.5% on simulation
benchmarks (RoboCasa, LIBERO, Bridge-SIMPLER) and +6.7% on a real
robot over finetuned VLA baselines. Notably, it outperforms pixel-space WM supervision by up to +6.0%, including on LIBERO where pixel supervision degrades the baseline, suggesting that latent WM dynamics offer a more stable WM-based post-training alternative to pixel-space transfer.
Read the full abstract ▾
All authors An Dinh Vuong, Tuan Van Vo, Abdullah Sohail, Haoran Ding, Liang Ma, Xiaodan Liang, Anqing Duan, Ivan Laptev, Ian Reid
Computer Vision
TractoRC: A Unified Probabilistic Learning Framework for Joint Tractography Registration and Clustering
Yijie Li, Xi Zhu, Junyi Wang et al. (+3 more)
Diffusion MRI tractography enables in vivo reconstruction of white matter (WM) pathways. Two key tasks in tractography analysis include: 1) tractogram registration that aligns streamlines across individuals, and 2) streamline clustering that groups streamlines into compact fiber bundles. Although both tasks share the goal of capturing geometrically similar structures to characterize consistent WM organization, they are typically performed independently. In this work, we propose TractoRC, a unified probabilistic framework that jointly performs tractogram registration and streamline clustering within a single optimization scheme, enabling the two tasks to leverage complementary information. TractoRC learns a latent
embedding space for streamline points, which serves as a shared representation for both tasks. Within this space, both tasks are formulated as probabilistic inference over structural representations: registration learns the distribution of anatomical landmarks as probabilistic keypoints to align tractograms across subjects, and clustering learns streamline structural prototypes that capture
geometric similarity to form coherent streamline clusters. To support effective learning of this shared space, we introduce a transformation-equivariant self-supervised strategy to learn geometry-aware and transformation-invariant
embeddings. Experiments demonstrate that jointly optimizing registration and clustering significantly improves performance in both tasks over state-of-the-art methods that treat them independently. Code will be made publicly available at https://github.com/yishengpoxiao/TractoRC .
Read the full abstract ▾
All authors Yijie Li, Xi Zhu, Junyi Wang, Ye Wu, Lauren J. O'Donnell, Fan Zhang
Computer Vision
Frames2Residual: Spatiotemporal Decoupling for Self-Supervised Video Denoising
Mingjie Ji, Zhan Shi, Kailai Zhou et al. (+2 more)
Self-supervised
video denoising methods typically extend image-based frameworks into the temporal dimension, yet they often struggle to integrate inter-frame temporal consistency with intra-frame spatial specificity. Existing
Video Blind-Spot Networks (BSNs) require noise independence by masking the center pixel, this constraint prevents the use of spatial evidence for texture recovery, thereby severing spatiotemporal correlations and causing texture loss. To address this, we propose Frames2Residual (F2R), a spatiotemporal decoupling framework that explicitly divides self-supervised training into two distinct stages: blind temporal consistency modeling and non-blind spatial texture recovery. In Stage 1, a blind temporal
estimator learns inter-frame consistency using a frame-wise blind strategy, producing a temporally consistent anchor. In Stage 2, a non-blind spatial refiner leverages this anchor to safely reintroduce the center frame and recover intra-frame high-frequency spatial residuals while preserving temporal stability. Extensive experiments demonstrate that our decoupling strategy allows F2R to outperform existing self-supervised methods on both sRGB and raw
video benchmarks.
Read the full abstract ▾
All authors Mingjie Ji, Zhan Shi, Kailai Zhou, Zixuan Fu, Xun Cao
Computer Vision
Motion Forcing: A Decoupled Framework for Robust Video Generation in Motion Dynamics
Tianshuo Xu, Zhifei Chen, Leyi Wu et al. (+2 more)
The ultimate goal of
video generation is to satisfy a fundamental trilemma: achieving high visual quality, maintaining rigorous physical consistency, and enabling precise controllability. While recent models can maintain this balance in simple, isolated scenarios, we observe that this
equilibrium is fragile and often breaks down as scene complexity increases (e.g., involving collisions or dense traffic). To address this, we introduce Motion Forcing, a framework designed to stabilize this trilemma even in complex generative tasks. Our key insight is to explicitly decouple physical
reasoning from visual synthesis via a hierarchical “Point-Shape-Appearance” paradigm. This approach decomposes generation into verifiable stages: modeling complex dynamics as sparse
geometric anchors (Point), expanding them into dynamic depth maps that explicitly resolve 3D geometry (Shape), and finally rendering high-fidelity textures (Appearance). Furthermore, to foster robust physical understanding, we employ a Masked Point Recovery strategy. By randomly masking input anchors during training and enforcing the reconstruction of complete dynamic depth, the model is compelled to move beyond passive pattern matching and learn latent physical laws (e.g., inertia) to infer missing trajectories. Extensive experiments on autonomous driving
benchmarks show that Motion Forcing significantly outperforms state-of-the-art baselines, maintaining trilemma stability across complex scenes. Evaluations on physics and robotics further confirm our framework's generality.
Read the full abstract ▾
All authors Tianshuo Xu, Zhifei Chen, Leyi Wu, Hao Lu, Ying-cong Chen
Computer Vision
Multi-Person Pose Estimation Evaluation Using Optimal Transportation and Improved Pose Matching
Takato Moriki, Hiromu Taketsugu and Norimichi Ukita
In Multi-Person Pose Estimation, many metrics place importance on ranking of pose detection confidence scores. Current metrics tend to disregard false-positive poses with low confidence, focusing primarily on a larger number of high-confidence poses. Consequently, these metrics may yield high scores even when many false-positive poses with low confidence are detected. For fair evaluation taking into account a tradeoff between true-positive and false-positive poses, this paper proposes Optimal Correction Cost for pose (OCpose), which evaluates detected poses against pose annotations as an optimal transportation. For the fair tradeoff between true-positive and false-positive poses, OCpose equally evaluates all the detected poses regardless of their confidence scores. In OCpose, on the other hand, the confidence score of each pose is utilized to improve the reliability of matching scores between the estimated pose and pose annotations. As a result, OCpose provides a different perspective assessment than other confidence ranking-based metrics.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Variance-Aware Adaptive Weighting for Diffusion Model Training
Nanlong Sun and Lei Shi
Diffusion models have recently achieved remarkable success in generative modeling, yet their training dynamics across different noise levels remain highly imbalanced, which can lead to inefficient optimization and unstable learning behavior. In this work, we investigate this imbalance from the perspective of loss variance across log-SNR levels and propose a variance-aware adaptive weighting strategy to address it. The proposed approach dynamically adjusts training weights based on the observed variance distribution, encouraging a more balanced optimization process across noise levels. Extensive experiments on CIFAR-10 and CIFAR-100 demonstrate that the proposed method consistently improves generative performance over standard training schemes, achieving lower Fréchet Inception Distance (FID) while also reducing performance variance across random seeds. Additional analysis, including loss-log-SNR visualization, variance heatmaps, and ablation studies, further reveal that the adaptive weighting effectively stabilizes training dynamics. These results highlight the potential of variance-aware training strategies for improving
diffusion model optimization.
Read the full abstract ▾
Computer Vision
Ruiheng Liu, Haihong Hao, Mingfei Han et al. (+4 more)
Advancing towards artificial superintelligence requires rich and intelligent perceptual capabilities. A critical frontier in this pursuit is overcoming the limited spatial understanding of
Multimodal Large
Language Models (MLLMs), where geometry information is essential. Existing methods often address this by rigidly injecting
geometric signals into every input, while ignoring their necessity and adding computation overhead. Contrary to this paradigm, our framework endows the model with an awareness of perceptual insufficiency, empowering it to autonomously engage
geometric features in
reasoning when 2D cues are deemed insufficient. To achieve this, we first introduce an independent geometry input channel to the model architecture and conduct
alignment training, enabling the effective utilization of
geometric features. Subsequently, to endow the model with perceptual awareness, we curate a dedicated spatial-aware supervised fine-tuning
dataset. This serves to activate the model's latent internal cues, empowering it to autonomously determine the necessity of
geometric information. Experiments across multiple spatial
reasoning benchmarks validate this approach, demonstrating significant spatial gains without compromising 2D visual
reasoning capabilities, offering a path toward more robust, efficient and self-aware multi-modal intelligence.
Read the full abstract ▾
All authors Ruiheng Liu, Haihong Hao, Mingfei Han, Xin Gu, Kecheng Zhang, Changlin Li, Xiaojun Chang
Computer Vision
Hangyu Liu, Jianyong Wang and Yutao Sun
Latent
diffusion models have established a new state-of-the-art in high-resolution visual generation. Integrating
Vision Foundation Model priors improves generative efficiency, yet existing latent designs remain largely heuristic. These approaches often struggle to unify semantic discriminability, reconstruction fidelity, and latent compactness. In this paper, we propose
Geometric Autoencoder (GAE), a principled framework that systematically addresses these challenges. By analyzing various
alignment paradigms, GAE constructs an optimized low-dimensional semantic supervision target from VFMs to provide guidance for the autoencoder. Furthermore, we leverage latent normalization that replaces the restrictive KL-divergence of standard VAEs, enabling a more stable latent
manifold specifically optimized for
diffusion learning. To ensure robust reconstruction under high-intensity noise, GAE incorporates a dynamic noise sampling mechanism. Empirically, GAE achieves compelling performance on the ImageNet-1K $256 \times 256$
benchmark, reaching a gFID of 1.82 at only 80 epochs and 1.31 at 800 epochs without Classifier-Free Guidance, significantly surpassing existing state-of-the-art methods. Beyond generative quality, GAE establishes a superior
equilibrium between compression, semantic depth and robust reconstruction stability. These results validate our design considerations, offering a promising paradigm for latent
diffusion modeling. Code and models are publicly available at https://github.com/sii-research/GAE.
Read the full abstract ▾
Computer Vision
One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucination
Zhan Fa, Yue Duan, Jian Zhang et al. (+2 more)
Current training-free methods tackle MLLM
hallucination with separate strategies: either enhancing visual signals or suppressing text inertia. However, these separate methods are insufficient due to critical trade-offs: simply enhancing
vision often fails against strong
language prior, while suppressing
language can introduce extra image-irrelevant noise. Moreover, we find their naive combination is also ineffective, necessitating a unified framework. We propose such a framework by focusing on the core asset: the
vision token. Our design leverages two key insights: (1) augmented images offer complementary visual semantics, and (2) removing
vision tokens (information-gap) isolates
hallucination tendencies more precisely than distorting images (modality-gap). Based on these, our framework uses
vision tokens in two distinct ways, both operating on latent representations: our Synergistic Visual Calibration (SVC) module incorporates augmented
tokens to strengthen visual representations, while our Causal Representation Calibration (CRC) module uses pruned
tokens to create latent-space negative samples for correcting internal model biases. By harmonizing these two roles, our framework effectively restores the
vision-
language balance, significantly reducing object
hallucinations, improving POPE accuracy by an average of 2% absolute on LLaVA-1.5 across multiple
benchmarks with only a 1.06x inference latency overhead.
Read the full abstract ▾
All authors Zhan Fa, Yue Duan, Jian Zhang, Lei Qi, Yinghuan Shi
Computer Vision
StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References
Boyu He, Yunfan Ye, Chang Liu et al. (+3 more)
Despite the advancements in
diffusion-based image style transfer, existing methods are commonly limited by 1) semantic gap: the style reference could miss proper content semantics, causing uncontrollable stylization; 2) reliance on extra constraints (e.g., semantic masks) restricting applicability; 3) rigid feature associations lacking adaptive global-local
alignment, failing to balance fine-grained stylization and global content preservation. These limitations, particularly the inability to flexibly leverage style inputs, fundamentally restrict style transfer in terms of personalization, accuracy, and adaptability. To address these, we propose StyleGallery, a training-free and semantic-aware framework that supports arbitrary reference images as input and enables effective personalized customization. It comprises three core stages: semantic region
segmentation (adaptive clustering on latent
diffusion features to divide regions without extra inputs); clustered region matching (block filtering on extracted features for precise
alignment); and style transfer optimization (energy function-guided
diffusion sampling with regional style loss to optimize stylization). Experiments on our introduced
benchmark demonstrate that StyleGallery outperforms state-of-the-art methods in content structure preservation, regional stylization, interpretability, and personalized customization, particularly when leveraging multiple style references.
Read the full abstract ▾
All authors Boyu He, Yunfan Ye, Chang Liu, Weishang Wu, Fang Liu, Zhiping Cai
Computer Vision
EmoStory: Emotion-Aware Story Generation
Jingyuan Yang, Rucong Chen, Weibin Luo et al. (+1 more)
Story generation aims to produce image sequences that depict coherent narratives while maintaining subject consistency across frames. Although existing methods have excelled in producing coherent and expressive stories, they remain largely
emotion-neutral, focusing on what subject appears in a story while overlooking how
emotions shape narrative interpretation and visual presentation. As stories are intended to engage audiences emotionally, we introduce
emotion-aware story generation, a new task that aims to generate subject-consistent visual stories with explicit emotional directions. This task is challenging due to the abstract nature of
emotions, which must be grounded in concrete visual elements and consistently expressed across a narrative through visual composition. To address these challenges, we propose EmoStory, a two-stage framework that integrates
agent-based story planning and region-aware story generation. The planning stage transforms target
emotions into coherent story
prompts with
emotion agent and writer
agent, while the generation stage preserves subject consistency and injects
emotion-related elements through region-aware composition. We evaluate EmoStory on a newly constructed
dataset covering 25 subjects and 600 emotional stories. Extensive quantitative and qualitative results, along with user studies, show that EmoStory outperforms state-of-the-art story generation methods in
emotion accuracy,
prompt alignment, and subject consistency.
Read the full abstract ▾
All authors Jingyuan Yang, Rucong Chen, Weibin Luo, Hui Huang
Computer Vision
Overcoming Visual Clutter in Vision Language Action Models via Concept-Gated Visual Distillation
Sangmim Song, Sarath Kodagoda, Marc Carmichael et al. (+1 more)
Vision-
Language-Action (VLA) models demonstrate impressive zero-shot generalization but frequently suffer from a "Precision-
Reasoning Gap" in cluttered environments. This failure is driven by background-induced feature dilution, where high-frequency semantic noise corrupts the
geometric grounding required for precise manipulation. To bridge this gap, we propose Concept-Gated Visual
Distillation (CGVD), a training-free, model-agnostic inference framework that stabilizes VLA policies. CGVD operates by parsing instructions into safe and distractor sets, utilizing a two-layer target refinement process–combining cross-validation and spatial disambiguation–to explicitly penalize false positives and isolate genuine manipulation targets. We then process the scene via
Fourier-based inpainting, generating a clean observation that actively suppresses semantic distractors while preserving critical spatial geometry and visual proprioception. Extensive evaluations in highly cluttered manipulation tasks demonstrate that CGVD prevents performance collapse. In environments with dense semantic distractors, our method significantly outperforms state-of-the-art baselines, achieving a 77.5% success rate compared to the baseline's 43.0%. By enforcing strict attribute adherence, CGVD establishes inference-time visual
distillation as a critical prerequisite for robust robotic manipulation in the clutter.
Read the full abstract ▾
All authors Sangmim Song, Sarath Kodagoda, Marc Carmichael, Karthick Thiyagarajan
Computer Vision
Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models
Yuedong Yang, Xiwen Wei, Mustafa Munir et al. (+1 more)
Reasoning Large Multi-modality Models (LMMs) have become the de facto choice for many applications. However, these models rely on a Chain-of-Thought (CoT) process that is lengthy and unpredictable at runtime, often resulting in inefficient use of computational resources (due to
memory fragmentation) and sub-optimal accuracy (due to under- and over-thinking). We observe empirically that the CoT process follows a very simple form, whose behavior is independent of the specific generated samples. This suggests that the CoT length can be estimated ahead of time based on a hidden parameter representing the amount of "fuel" available to support the
reasoning process. Based on this insight, we propose Fuel Gauge, the first method which extracts this hidden signal and predicts CoT length ahead of time. We demonstrate the utility on the Fuel Gauge on two downstream tasks: predictive KV cache allocation, which addresses
memory fragmentation in LMM serving systems, and CoT length modulation, which mitigates under-thinking and over-thinking. Extensive experiments on LMMs across text-only, image-text, and
video-text question answering
benchmarks demonstrate the effectiveness, generalizability, and practical value of our Fuel Gauge. For example, on the GPQA-Diamond
benchmark, our Fuel Gauge achieves less than half the CoT length prediction error compared to the baseline; this translates into a 13.37x reduction in the
memory allocation frequency.
Read the full abstract ▾
All authors Yuedong Yang, Xiwen Wei, Mustafa Munir, Radu Marculescu
Computer Vision · Cross-listed from cs.CR
The Orthogonal Vulnerabilities of Generative AI Watermarks: A Comparative Empirical Benchmark of Spatial and Latent Provenance
Jesse Yu and Nicholas Wei
As open-weights generative AI rapidly proliferates, the ability to synthesize hyper-realistic media has introduced profound challenges to digital trust. Automated disinformation and AI-generated imagery have made robust digital provenance a critical cybersecurity imperative. Currently, state-of-the-art invisible watermarks operate within one of two primary mathematical
manifolds: the spatial domain (post-generation pixel
embedding) or the latent domain (pre-generation frequency
embedding). While existing literature frequently evaluates these models against isolated, classical distortions, there is a critical lack of rigorous, comparative benchmarking against modern generative AI editing tools. In this study, we empirically evaluate two leading representative paradigms, RivaGAN (Spatial) and Tree-Ring (Latent), utilizing an automated Attack Simulation Engine across 30 intensity intervals of
geometric and generative perturbations. We formalize an "
Adversarial Evasion Region" (AER) framework to measure cryptographic degradation against semantic visual retention (OpenCLIP > 75.0). Our statistical analysis ($n=100$ per interval, $MOE = \pm 3.92\%$) reveals that these domains possess mutually exclusive, mathematically orthogonal vulnerabilities. Spatial watermarks experience severe cryptographic degradation under algorithmic pixel-rewriting (exhibiting a 67.47% AER evasion rate under Img2Img translation), whereas latent watermarks exhibit profound fragility against
geometric misalignment (yielding a 43.20% AER evasion rate under static cropping). By proving that single-domain watermarking is fundamentally insufficient against modern
adversarial toolsets, this research exposes a systemic vulnerability in current digital provenance standards and establishes the foundational exigence for future multi-domain cryptographic architectures.
Read the full abstract ▾