Computer Vision
DVD: Discrete Voxel Diffusion for 3D Generation and Editing
Zhengrui Xiang, Jiaqi Wu, Fupeng Sun et al. (+2 more)
We introduce Discrete Voxel
Diffusion (DVD), a discrete
diffusion framework to generate, assess, and edit sparse voxels for SLat (Structured LATent) based 3D generative pipelines. Although discrete
diffusion has not generally displaced continuous
diffusion in image-like generation, we show that it can be an effective first-stage prior for sparse voxel scaffolds. By treating voxel occupancy as a native discrete variable, DVD avoids continuous-to-discrete thresholding and provides a simple framework for voxel generation, uncertainty estimation, and editing. Beyond quality gains, DVD provides more interpretable generation dynamics through explicit categorical modeling. Furthermore, we leverage the predictive
entropy as a robust uncertainty metric to identify ambiguous voxel regions and complicated samples, facilitating tasks such as data filtering and quality assessment. Finally, we propose a lightweight fine-tuning strategy using block-structured perturbation patterns. This approach empowers the model to inpaint and edit voxels within a single sampling round, requiring negligible auxiliary computation and no additional model evaluations. Code is available at https://github.com/TeCai/DVD.
Read the full abstract ▾
All authors Zhengrui Xiang, Jiaqi Wu, Fupeng Sun, Heliang Zheng, Yingzhen Li
Computer Vision
TimeLesSeg: Unified Contrast-Agnostic Cross-Sectional and Longitudinal MS Lesion Segmentation via a Stochastic Generative Model
Vicent Caselles-Ballester, Eloy Martínez-Heras, Giuseppe Pontillo et al. (+9 more)
Multiple sclerosis (MS) expresses substantial clinical and radiological heterogeneity, which poses significant challenges for automatic lesion
segmentation. The current deep learning-based SOTA is highly susceptible to changes in both distribution, e.g., changes in scanner; as well as the structure of inputs, evident in the current divide between cross-sectional and longitudinal approaches. We introduce TimeLesSeg, a unified contrast-agnostic framework designed to segment MS lesions regardless of the presence of a temporal dimension in its inputs, with a single convolutional neural network. Our approach models pathological priors through lesion masks, which are processed together with the current scan. Cross-sectional processing is enabled by exposing the model to training cases where no prior information is available, which are modeled with an empty mask, allowing it to operate seamlessly in both scenarios. To overcome the scarcity and inconsistency of longitudinal
datasets, we propose a novel generative pipeline in which patterns of lesion
evolution are simulated by stochastically deforming each individual lesion with morphological operations, producing realistic prior timepoints. In parallel, we achieve contrast agnosticism through Gaussian mixture model-based domain randomization, enabling the network to experience a wide spectrum of intensity profiles. Results on three publicly available and two in-house
datasets show that TimeLesSeg outperforms the contrast-agnostic state of the art on single-modality inputs across overlap- and distance-based metrics. In longitudinal processing, our method outperforms SAMSEG, and captures lesion load dynamics more accurately than both the former and LST-AI. All source code related to the development of TimeLesSeg is available at https://github.com/NeuroADaS-Lab/TimeLesSeg.
Read the full abstract ▾
All authors Vicent Caselles-Ballester, Eloy Martínez-Heras, Giuseppe Pontillo, Zoe Mendelsohn, Elena M. Marrón, Juan Luis García Fernández, Laia Subirats, Jon Stutters, Jeremy Chataway, Frederik Barkhof, Sara Llufriu, Ferran Prados
Computer Vision
Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions
Marwane Hariat, Antoine Manzanera and David Filliat
We present CoopNet, an approach that improves the cooperation of co-trained networks by dynamically adapting the apportionment of gradient, to ensure equitable learning progress. It is applied to motion-aware self-supervised prediction of depth maps, by introducing a new hybrid loss, based on a distribution model of photo-metric reconstruction errors made by, on the one hand the depth + odometry paired networks, and on the other hand the optical flow network. This model essentially assumes that the pixels from moving objects (that must be discarded for training depth and odometry), correspond to those where the two reconstructions strongly disagree. We justify this model by theoretical considerations and experimental evidences. A comparative evaluation on KITTI and CityScapes
datasets shows that CoopNet improves or is comparable to the state-of-the-art in depth, odometry and optical flow predictions.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
TAVIS: A Benchmark for Egocentric Active Vision and Anticipatory Gaze in Imitation Learning
Giacomo Spigler
Active
vision – where a policy controls its own gaze during manipulation – has emerged as a key capability for imitation learning, with multiple independent systems demonstrating its benefits in the past year. Yet there is no shared
benchmark to compare approaches or quantify what active
vision contributes, on which task types, and under what conditions. We introduce TAVIS, evaluation infrastructure for active-
vision imitation learning, with two complementary task suites – TAVIS-Head (5 tasks, global search via pan/tilt necks) and TAVIS-Hands (3 tasks, local occlusion via wrist cameras) – on two
humanoid torso embodiments (GR1T2, Reachy2), built on IsaacLab. TAVIS provides three evaluation primitives: a paired headcam-vs-fixedcam protocol on identical demonstrations; GALT (Gaze-Action Lead Time), a novel metric grounded in cognitive science and HRI that quantifies anticipatory gaze in learned policies; and procedural ID/OOD splits. Baseline experiments with
Diffusion Policy and $\pi_0$ reveal that (i) active
vision generally helps, consistently across training runs, but benefits are task-conditional rather than uniform; (ii) multi-task policies degrade sharply under controlled distribution shifts on both suites; and (iii) imitation alone yields anticipatory gaze that lands on the task-relevant object, with lead times comparable to those of the human demonstrations, although head motion is less smooth than in the demonstrations, an effect of action chunking that success rate does not reveal. Code and evaluation scripts are released at https://github.com/spiglerg/tavis; demonstrations (LeRobot v3.0; ~2200 episodes) and trained baselines at https://huggingface.co/tavis-
benchmark.
Read the full abstract ▾
Computer Vision
Delta-Adapter: Scalable Exemplar-Based Image Editing with Single-Pair Supervision
Jiacheng Chen, Songze Li, Han Fu et al. (+5 more)
Exemplar-based image editing applies a transformation defined by a source-target image pair to a new query image. Existing methods rely on a pair-of-pairs supervision paradigm, requiring two image pairs sharing the same edit semantics to learn the target transformation. This constraint makes training data difficult to curate at scale and limits generalization across diverse edit types. We propose Delta-Adapter, a method that learns transferable editing semantics under single-pair supervision, requiring no textual guidance. Rather than directly exposing the exemplar pair to the model, we leverage a pre-trained
vision encoder to extract a semantic delta that encodes the visual transformation between the two images. This semantic delta is injected into a pre-trained image editing model via a Perceiver-based adapter. Since the target image is never directly visible to the model, it can serve as the prediction target, enabling single-pair supervision without requiring additional exemplar pairs. This formulation allows us to leverage existing large-scale editing
datasets for training. To further promote faithful transformation transfer, we introduce a semantic delta consistency loss that aligns the semantic change of the generated output with the ground-truth semantic delta extracted from the exemplar pair. Extensive experiments demonstrate that Delta-Adapter consistently improves both editing accuracy and content consistency over four strong baselines on seen editing tasks, while also generalizing more effectively to unseen editing tasks. Code will be available at https://delta-adapter.github.io.
Read the full abstract ▾
All authors Jiacheng Chen, Songze Li, Han Fu, Baoquan Zhao, Wei Liu, Yanyan Liang, Li Qing, Xudong Mao
Computer Vision
One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policy
Zuojin Tang, Shengchao Yuan, Xiaoxin Bai et al. (+4 more)
Vision-
language-action (VLA) models can use visual prediction to anticipate future states, but dense visual features make the generative sequence grow with the number of camera views, prediction horizon, and encoder resolution. Whether such dense representations are necessary for effective control remains unclear. We introduce OneWM-VLA, which represents each retained camera view with one predictive
token per future step. Adaptive
Attention Pooling compresses visual features into compact latents, which are jointly generated with
robot actions under a conditional flow-matching objective. Future observations provide the latent targets during training and are not required at inference. This design incorporates visual prediction into a pretrained VLA policy while keeping the generative sequence compact. On MetaWorld~MT50, OneWM-VLA improves the average success rate of the $\pi_0$ backbone from $47.91\%$ to $61.53\%$, reaching $72.01\%$ after 60k training steps. It also achieves $98.1\%$ success on LIBERO and raises Fold Cloth success on a real Piper arm from $20.0\%$ to $60.0\%$ relative to $\pi_0$. Comparisons on two additional VLA backbones consistently favor one
token over three across the evaluated checkpoints. A matched ablation at a longer action horizon further shows that removing the latent loss reduces success from $58.09\%$ to $21.64\%$, supporting the benefit of future supervision for policy learning.
Read the full abstract ▾
All authors Zuojin Tang, Shengchao Yuan, Xiaoxin Bai, Zhiyuan Jing, De Ma, Gang Pan, Bin Liu
Computer Vision
MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
Hanqi Jiang, Junhao Chen, Mingyu Kang et al. (+12 more)
Medical
vision–
language models (VLMs) are usually evaluated on intact image–question pairs, but trustworthy clinical use requires a stronger property: a model must recognise when the evidential basis for an answer has failed. We study this through silent failures under perturbed evidence, where a
vision-required medical question is paired with a false premise, wording perturbation, knowledge-only rewrite, or ROI-corrupted image, yet the model returns a fluent non-refusal answer. We introduce medvigil, a 300-case evaluation suite drawn from four public medical VQA sources, supervised end to end by four board-certified radiologists: every gold answer, refusal option, candidate-answer set, paraphrase, false-premise trap, ROI box, and clinical risk tier is clinician-authored. Two attending radiologists annotate every case in parallel, a senior radiologist consolidates the released manifest, and a separate fourth radiologist independent of construction answers every probe to provide the human reference baseline. The release contains 2556 MCQ probes, 240 counterfactual triplets, physician-adjudicated risk-tier and answerability flags, ROI boxes, and a paired open-ended variant. We report seven correctness-conditioned audit metrics that summarise into the medvigil Composite Score (MCS), and audit 16
vision-capable models plus two text-only baselines. The independent radiologist scores MCS 83.3 at silent-failure rate 5.8%, leaving a 14.1-point composite headroom above the strongest audited model (Claude Opus 4.7 at 69.2). The
benchmark and evaluation harness are publicly released.
Read the full abstract ▾
All authors Hanqi Jiang, Junhao Chen, Mingyu Kang, Hyeokjae Kwon, Yi Pan, Lifeng Chen, Weihang You, Haozhen Gong, Ruiyu Yan, Jinglei Lv, Lin Zhao, Hui Ren, Quanzheng Li, Tianming Liu, Xiang Li
Computer Vision
What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion
Zhengrong Yue, Taihang Hu, Mengting Chen et al. (+8 more)
Tokenizers are a crucial component of latent
diffusion models, as they define the latent space in which
diffusion models operate. However, existing tokenizers are primarily designed to improve reconstruction fidelity or inherit pretrained representations, leaving unclear what kind of latent space is truly friendly for generative modeling. In this paper, we study this question from the perspective of latent
manifold organization. By constructing controlled tokenizer variants, we identify three key properties of a
diffusion-friendly latent
manifold: coherent spatial structure, local
manifold continuity, and global
manifold semantics. We find that these properties are more consistent with downstream generation quality than reconstruction fidelity. Motivated by this finding, we propose the Prior-Aligned AutoEncoder (PAE), which explicitly shapes the latent
manifold instead of leaving
diffusion-friendly
manifold to emerge indirectly from reconstruction or inheritance. Specifically, PAE leverages refined priors derived from VFMs and perturbation-based regularization to turn spatial structure, local continuity, and global semantics into explicit training objectives. On ImageNet 256x256, PAE improves both training efficiency and generation quality over existing tokenizers, reaching performance comparable to RAE with up to 13x faster convergence under the same training setup and achieving a new state-of-the-art gFID of 1.03. These results highlight the importance of organizing the latent
manifold for latent
diffusion models.
Read the full abstract ▾
All authors Zhengrong Yue, Taihang Hu, Mengting Chen, Haiyu Zhang, Zihao Pan, Tao Liu, Zikang Wang, Jinsong Lan, Xiaoyong Zhu, Bo Zheng, Yali Wang
Computer Vision · Cross-listed from cs.LG
Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning
Aristotelis Ballas and Christos Diou
Sharpness-aware and gradient-
alignment methods have been shown to improve generalization, however each family of methods targets a single
geometric property of the loss landscape, while ignoring the other. In this paper, we show that this omission is structurally unavoidable and that both flatness and gradient
alignment should be considered in multi-distribution learning settings. Specifically, we derive an excess-risk decomposition that yields two additive leading-order terms: (i) an
alignment term, controlled by the trace of $\bar{H}^{-1}\Sigma_g$ and (ii) a
curvature term, controlled by $\bar{H}$, where $\bar{H}$ is the average Hessian and $\Sigma_g$ is the covariance of the gradient across distributions. Notably, $\bar{H}$ appears inverted in one and non-inverted in the other. We further show, via a counterexample, that neither quantity bounds the other in general, so no
algorithm targeting only one term can guarantee low excess risk. Motivated by this decomposition, we propose SAGE (Spectral-Aware Gradient-Aligned Exploration) that targets both terms. The
curvature component replaces SAM's gradient-scaled perturbation with the polar factor of each layer's gradient
matrix, computed via Newton-Schulz iteration, so that the ascent step probes all directions with similar magnitude. On the other hand, the
alignment component injects isotropic noise at the descent step, the magnitude of which scales with cross-distribution gradient disagreement. Experiments on five domain-generalization and two multi-task learning
benchmarks show that the proposed method establishes a new state-of-the-art on DomainBed and acts as a general-purpose improvement to base MTL solvers, remaining competitive with, or even surpassing, state-of-the-art methods.
Read the full abstract ▾
Computer Vision
One World, Dual Timeline: Decoupled Spatio-Temporal Gaussian Scene Graph for 4D Cooperative Driving Reconstruction
Yulong Chen, Xiaoyun Dong, Haoyu Zhang et al. (+6 more)
Reconstructing dynamic scenes from Vehicle-to-Infrastructure Cooperative Autonomous Driving (VICAD) data is fundamentally complicated by temporal asynchrony: vehicle and infrastructure cameras operate on independent clocks, capturing the same dynamic
agent such as cars and pedestrians at different physical times. Existing Gaussian Scene Graph methods implicitly assume synchronized observations and assign a single pose per
agent per frame, which is an assumption that breaks in cooperative settings, where the resulting gradient conflicts cause severe ghosting on dynamic
agents. We identify this as a representation-level failure, not an optimization artifact: we prove that any single-timeline formulation incurs an irreducible photometric loss scaling quadratically with
agent velocity and cross-source time offset. To resolve this, we propose
Dust (DecoUpled Spatio-Temporal) Gaussian Scene Graph for 4D Cooperative Driving Reconstruction.
DUST Gaussian Scene Graph shares a canonical Gaussian set per
agent for appearance consistency, while maintaining decouple pose trajectories aligned to each source's true capture timestamps. We prove that this decoupling enables the pose-gradient kernel block-diagonal, eliminating cross-source interference entirely. To make
Dust practical, we further introduce a static anchor-based pose correction pipeline that corrects spatio misalignment between vehicle and infrastructure annotations, and a pose-regularized joint optimization scheme that prevents trajectory jitter and drift during early training. On 26 sequences from V2X-Seq,
DUST achieves state-of-the-art performance, improving dynamic-area PSNR by 3.2 dB over the strongest baseline and reducing Fréchet
Video Distance by 37.7%, with keeping robustness under larger temporal asynchrony.
Read the full abstract ▾
All authors Yulong Chen, Xiaoyun Dong, Haoyu Zhang, Zongxian Yang, Lewei Xie, Xinke Li, Yifan Zhang, Kai Wang, Jianping Wang
Computer Vision · Cross-listed from stat.ML
Consistency Regularised Gradient Flows for Inverse Problems
Alessio Spagnoletti, Tim Y. J. Wang, Marcelo Pereyra et al. (+1 more)
Vision-
Language Latent
Diffusion Models (LDMs) (Rombach et al., 2022) provide powerful generative priors for inverse problems. However, existing LDM-based inverse solvers typically require a large number of neural function evaluations (NFEs) and backpropagation through large pretrained components, leading to substantial computational costs and, in some cases, degraded reconstruction quality. We propose a unified Euclidean-Wasserstein-2 gradient-flow framework that jointly performs posterior sampling and
prompt optimization in the latent space through a single flow that aligns the prior and posterior with the observed data. Combined with few-step latent text-to-image models, this formulation enables low-NFE inference without backpropagation through autoencoders. Experiments across several canonical imaging inverse problems show that our method achieves state-of-the-art performance with significantly reduced computational cost.
Read the full abstract ▾
All authors Alessio Spagnoletti, Tim Y. J. Wang, Marcelo Pereyra, O. Deniz Akyildiz
Computer Vision
Semantic-Aware Adaptive Visual Memory for Streaming Video Understanding
Hang Wu, Sherin Mary Mathews, Yujun Cai et al. (+2 more)
Online streaming
video understanding requires models to process continuous visual inputs and respond to user queries in real time, where the unbounded stream and unpredictable query timing turn
memory management into a central challenge. Existing methods typically compress visual
tokens via visual similarity heuristics, or augment compression with KV-cache-level
retrieval. However, compression decisions rarely incorporate semantic signals, and
retrieval is often added after compression is finalized, making the two stages hard to coordinate. We present SAVEMem, a training-free dual-stage framework that brings semantic awareness into
memory generation and lets the
retrieval scope adapt per query. In Stage~1, SAVEMem builds a three-tier streaming
memory online under a constant
memory budget. A fixed pseudo-question bank provides a lightweight semantic prior, so that long-term retention is shaped by semantic salience rather than visual similarity alone. In Stage~2, SAVEMem performs query-aware
retrieval over this
memory. An anchor-conditioned recency gate adapts the
retrieval scope from short-term to mid- and long-term
memory based on whether the query targets the present or the distant past. Within this scope, late interaction between query and
memory tokens selects candidate frames for answering. Applied to Qwen2.5-VL without training, SAVEMem improves the OVO-Bench overall score from 52.27 to 62.69 and yields consistent gains on StreamingBench and ODV-Bench, while reducing peak GPU
memory by 48% at 128 frames over the backbone.
Read the full abstract ▾
All authors Hang Wu, Sherin Mary Mathews, Yujun Cai, Ming-Hsuan Yang, Yiwei Wang
Computer Vision · Cross-listed from cs.LG
Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysis
Ozgu Goksu and Nicolas Pugeault
Federated Learning (FL) enables decentralised model training across distributed clients without requiring data centralisation. However, the generalisation performance of the global model is usually degraded by data heterogeneity across clients, particularly under limited data availability and class imbalance. To address this challenge, we propose FedQuad, a novel method that explicitly enforces minimising intra-class representations while enabling inter-class splits across clients. By jointly minimising distances between positive pairs and maximising distances between negative pairs, the proposed approach mitigates representation misalignment introduced during model aggregation. We evaluate our method on CIFAR-10, CIFAR-100, and Tiny-ImageNet under diverse non-IID settings and varying numbers of clients, demonstrating consistent improvements over existing baselines. Additionally, we provide a comprehensive analysis of metric learning-based approaches in both centralised and
federated environments, highlighting their effectiveness in alleviating representation collapse under heterogeneous data distributions.
Read the full abstract ▾
Computer Vision
Video Understanding Reward Modeling: A Robust Benchmark and Performant Reward Models
Yuancheng Wei, Linli Yao, Lei Li et al. (+4 more)
Multimodal reward models have advanced substantially in text and image domains, yet progress in
video understanding reward modeling remains severely limited by the lack of robust evaluation
benchmarks and high-quality preference data. To address this, we propose a unified framework spanning
benchmark design, data construction, and reward model training. We introduce
Video Understanding Reward Bench (VURB), a
benchmark featuring 2,100 preference pairs with long chain-of-thought
reasoning traces (averaging 1,143
tokens) and majority voting evaluation across general, long, and
reasoning-oriented
video tasks. We further construct
Video Understanding Preference
Dataset (VUP-35K) via a fully automated pipeline, providing large-scale high-quality supervision for
video reward training. Building on the data, we train VideoDRM and VideoGRM, a discriminative and a generative reward model, both achieving state-of-the-art performance on VURB and VideoRewardBench. Further analysis confirms that VUP-35K enhances both reward performance and model
reasoning capability, while VideoDRM and VideoGRM yield significant gains under best-of-$N$ test-time scaling.
Read the full abstract ▾
All authors Yuancheng Wei, Linli Yao, Lei Li, Haojie Zhang, Hao Zhou, Fandong Meng, Xu Sun
Computer Vision
From Synthetic to Real: Toward Identity-Consistent Makeup Transfer with Synthetic and Real Data
Yue Yu, Jiayu Wang, Jiajia Shi et al. (+3 more)
Makeup transfer aims to apply the makeup style of a reference portrait to a source portrait while preserving identity and background. Early methods formulate this task as unsupervised image-to-image translation, relying on surrogate objectives and often yielding limited performance. Recent
diffusion- and flow-based approaches instead exploit synthetic data for supervised training, leading to significant improvements. However, these methods still face two critical challenges: synthetic supervision frequently fails to faithfully preserve identity, and the domain gap between synthetic and real data limits generalization, resulting in degraded performance in complex real-world scenarios. To address these issues, this paper first proposes ConsistentBeauty, a novel data curation pipeline that ensures makeup fidelity and strict identity consistency within the synthesized data. Second, we propose RealBeauty, a synthetic-to-real post-training framework. Beyond supervised learning on curated synthetic data, we further adapt the model to real-world scenarios through reinforcement learning and design novel verifiable rewards tailored to the makeup transfer task. It allows the model to further benefit from real makeup patterns beyond synthetic supervision. In addition, we establish a new diverse
benchmark for makeup transfer, covering a wide range of skin tones, ages, genders, poses, and makeup styles, thereby enabling a more comprehensive evaluation of model performance under diverse real-world conditions. Extensive experiments show that our method achieves state-of-the-art performance on multiple
benchmarks and demonstrates clear advantages in identity preservation and performance on complex real-world cases.
Read the full abstract ▾
All authors Yue Yu, Jiayu Wang, Jiajia Shi, Zhiyu Tan, Hao Li, Jingjing Chen
Computer Vision
EyeCue: Driver Cognitive Distraction Detection via Gaze-Empowered Egocentric Video Understanding
Lang Zhang, JinYi Yoon, Matthew Corbett et al. (+2 more)
Driver cognitive distraction is a major cause of road collisions and remains difficult to detect. Unlike manual or visual distraction, cognitive distraction is diverted by thoughts unrelated to driving, even when the driver appears visually attentive and exhibits no explicit physical movements. In this work, we propose EyeCue, a gaze-empowered
egocentric video understanding framework, to detect driver cognitive distraction. A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context. To capture this interaction, EyeCue integrates eye gaze with
egocentric video to enable context-aware modeling of the driver's
attention over time. Furthermore, to tackle the limited scale and diversity of existing
datasets, we introduce CogDrive, a comprehensive multi-scenario
dataset that augments four existing driving
datasets with cognitive distraction annotations. Through extensive evaluations on CogDrive, we show that EyeCue achieves the highest accuracy of 74.38%, outperforming 11 baselines from 6 model families by over 7%. Notably, EyeCue can achieve an accuracy of over 70% across various driving scenarios (different road types, times of day, and weather conditions) with strong generalizability. These results highlight the importance of modeling gaze-context interactions and the effectiveness of cross-modal interaction modeling for
multimodal cognitive distraction detection. Our codes and CogDrive
dataset resources are available at https://github.com/langzhang2000/EyeCue.
Read the full abstract ▾
All authors Lang Zhang, JinYi Yoon, Matthew Corbett, Abhijit Sarkar, Bo Ji
Computer Vision
BRIDGE: Background Routing and Isolated Discrete Gating for Coarse-Mask Local Editing
Peilin Xiong, Honghui Yuan, Junwen Chen et al. (+1 more)
Coarse-mask local editing must preserve the background while allowing new content to depart from the mask contour. Existing mask-conditioned editors often entangle these goals and treat the coarse mask as a shape prior. We introduce BRIDGE, a dual-path architecture with a full-canvas Main Path, a local Subject Path, and a learned binary gate that routes subject-
token positional
embeddings between subject-centric and background-anchored coordinates. On Qwen-Image-Edit-2511, BRIDGE keeps the mask outside the DiT backbone and trains both targets with independent noise. On BRIDGE-Bench, it achieves 0.503 local SigLIP2-T, 0.902 DINO, and 0.175 DreamSim, outperforming ACE++ and FLUX.1-Fill in local precision. We then instantiate BRIDGE on FLUX.2 [klein] 9B Base with full-
transformer training and subject-reference conditioning. Dense BBox supervision preserves Sub-image quality; optional mask-cropped inference removes out-of-mask Sub
tokens, tightening user support without training a sparse model that overfits the mask silhouette. On supported ICE-Bench tasks, FLUX.2 BRIDGE averages 0.5941 over four local edits and 0.6254 over subject-guided edits, leading the compared open-source models on the latter. Across Qwen and FLUX.2, BRIDGE separates support localization from subject geometry and improves local generation while preserving the surrounding scene. Code is available at https://github.com/XIONGPEILIN/BRIDGE
Read the full abstract ▾
All authors Peilin Xiong, Honghui Yuan, Junwen Chen, Keiji Yanai
Computer Vision
Explainable Part-Based Vehicle Classifier with Spatial Awareness
Andreas Caduff, Klaus Zahn, Jonas Hofstetter et al. (+2 more)
In the area of Intelligent Transportation Systems (ITS), fine-grained vehicle classification systems play an essential role. Recently, the authors have presented a novel
vision-based classification approach in which standard end-to-end Convolutional Neural Networks (CNNs) have been decomposed into 1) a CNN-based detector for semantically strong vehicle parts, followed by 2) feature construction and 3) final classification by a decision tree. In contrast to conventional CNNs, this allows both easy extensibility to new vehicle categories - without the need to fully retrain the part detector - and an important step towards the interpretability of the model, removing partially the black-box nature inherent to CNNs. Here we present an important extension of this approach that now incorporates spatial awareness of the vehicle parts: while the feature construction 2) of the previous approach used a binary decision for each feature (present vs. absent), now a full spatial probability map is constructed to condition the presence of each individual part with respect to a given vehicle category. The classification is performed using a softmax regression approach for the overall vehicle probabilities. This method shows a considerably improved robustness against false (part-)detections, a point that is crucial for practical application. Comparative analyses with a state-of-the-art end-to-end CNN indicate that our part-based methods achieve comparable accuracy, effectively challenging the presumed trade-off between accuracy and explainability. This research represents a significant advance in vehicle classification for ITS and forms the basis for systems that combine high accuracy with intuitive interpretability.
Read the full abstract ▾
All authors Andreas Caduff, Klaus Zahn, Jonas Hofstetter, Martin Rechsteiner, Patrick Flaig
Computer Vision · Cross-listed from cs.MM
Anisotropic Modality Align
Xiaomin Yu, Yijiang Li, Yuhui Zhang et al. (+8 more)
Training
multimodal large
language models has long been limited by the scarcity of high-quality paired
multimodal data. Recent studies show that the shared representation space of pretrained
multimodal contrastive models can serve as a bridge, enabling models to perform
multimodal training with unimodal data. However, the key premise of this paradigm remains insufficiently understood: can representations from different modalities be reliably interchanged? The core obstacle lies in the persistent Modality Gap in the shared space. In this work, we revisit the
geometric nature of the modality gap. We find that modality representations already share compatible dominant semantic geometry. What truly hinders modality interchangeability is not a simple global shift, but an anisotropic residual structure concentrated along a small number of dominant directions. Based on this finding, we further propose the principle of anisotropic modality gap
alignment: effective modality
alignment should align with the target-modality distribution while preserving the semantic structure of the source modality. Guided by this principle, we propose an anisotropic
geometric correction framework, AnisoAlign, for unpaired modality
alignment. This framework leverages the internal
geometric prior of the target modality and performs bounded correction on source-modality representations, thereby constructing substitute representations in the target modality. Experiments confirm its benefits in both
geometric diagnostics and text-only MLLM training. Overall, this work recasts the modality gap from an empirical observation into a correctable, structured
geometric phenomenon and provides a new representation
alignment perspective for training
multimodal models with unimodal data.
Read the full abstract ▾
All authors Xiaomin Yu, Yijiang Li, Yuhui Zhang, Hanzhen Zhao, Yue Yang, Hao Tang, Yue Song, Xiaobin Hu, Chengwei Qin, Shuicheng Yan, Hui Xiong
Computer Vision
Mitigating Simplicity Bias in OOD Detection through Object Co-occurrence Analysis
Boyang Dai, Chaoqi Chen and Yizhou Yu
Out-of-distribution (OOD) detection is crucial for ensuring the reliability of deep learning models. Existing methods mostly focus on regular entangled representations to discriminate in-distribution (ID) and OOD data, neglecting the rich contextual information within images. This issue is particularly challenging for detecting near-OOD, as models with simplicity bias struggle to learn discriminative features in disentangled representations. The human visual system can use the co-occurrence of objects in the natural environment to facilitate scene understanding. Inspired by this, we propose an Object-Centric OOD detection framework that learns to capture Object CO-occurrence (OCO) patterns within images. The proposed method introduces a new OOD detection paradigm that understands object co-occurrence within an image by predicting disentangled representations for the test sample, then adaptively divides patterns into three scenarios based on object co-occurrence patterns observed in ID training data, and finally performs OOD detection in a divide-and-conquer manner. By doing so, OCO can distinguish near-OOD by considering the semantic contextual relationships present in their images, avoiding the tendency to focus solely on simple, easily learnable regions. We evaluate OCO through experiments across challenging and full-spectrum OOD settings, demonstrating competitive results and confirming its ability to address both semantic and
covariate shifts. Code is released at https://github.com/Michael-McQueen/OCO.
Read the full abstract ▾
Computer Vision
Brown Ebouky, Gabriele Carrino, Niccolo Avogaro et al. (+3 more)
Human visual
reasoning is governed by active
vision, a process where meta-cognitive control drives top-down goal-directed
attention, dynamically routing foveal focus toward task-relevant details while maintaining peripheral awareness of the global scene. In contrast, modern
Vision-
Language Models (VLMs) process visual information passively, relying on the static accumulation of massive
token contexts that dilute the visual evidence as the
reasoning chain grows. Here we propose GazeVLM, a VLM that learns to exert this oversight over its
attention resources by generating gaze actions in its
reasoning chain, in the form of <LOOK> tags specifying the coordinates of the region to inspect. During training, each <LOOK> block triggers a continuous suppression bias on the
attention logits that dampens features outside the regions gazed at so far, and that lifts at the end of the block, restoring the global view. The model is trained with this bias as a teaching signal, first by Supervised Fine-Tuning on curated gaze-
reasoning traces, then through Group Relative Policy Optimization (GRPO) with rewards for correct answers and valid grounding. At deployment, the external suppression is removed and the model, running its unmodified forward pass, steers its own
attention to the regions indicated by the <LOOK> blocks, retaining most of the effect of the bias and emulating top-down control of spatial
attention. This model thus transitions between global spatial awareness and localized focal
reasoning without relying on external agentic tools like cropping, and without adding visual
tokens from re-encoded patches. Applied to 4B-parameter backbones, GazeVLM demonstrates strong high-resolution
multimodal reasoning on HRBench-4k and HRBench-8k, surpassing its base models by about 4 points, and outperforming agentic
multimodal pipelines built around thinking with images by 5 to 12 points.
Read the full abstract ▾
All authors Brown Ebouky, Gabriele Carrino, Niccolo Avogaro, Christoph Studer, Andrea Bartezzaghi, Mattia Rigotti
Computer Vision
ICDAR 2026 Competition on Writer Identification and Pen Classification from Hand-Drawn Circles
Thomas Gorges, Janne van der Loop, Lukas Hüttner et al. (+4 more)
This paper presents CircleID, a large-scale ICDAR 2026 competition on writer identification and pen classification from scanned hand-drawn circles. The primary objective is to investigate how biometric writer characteristics and physical pen features naturally entangle within minimal, static traces. CircleID comprises two distinct tasks: (1) open-set writer identification, requiring models to recognize known writers while explicitly rejecting unknown ones, and (2) cross-writer pen classification, evaluated across both seen and unseen writers. Participants were provided with a new, controlled
dataset of 46,155 tightly cropped circle images, digitized at 400 DPI and annotated for writer identity and pen type. The
dataset comprises samples from 44 known and 22 unknown writers using eight different pens. Hosted on Kaggle as two separate tracks with public and private leaderboards, the competition provided participants with a ResNet baseline. In total, 389 teams (436 participants) made 3,185 submissions for the pen classification task, and 113 teams (141 participants) made 1,737 submissions for the writer identification track. The best-performing private leaderboard submissions achieved a Top-1 accuracy of 64.801% for writer identification and 92.726% for pen classification. This paper details the
dataset, evaluates the winning methodologies, and analyzes the impact of out-of-distribution writers on model generalization and feature disentanglement. In this large-scale competition, CircleID establishes a new baseline for minimal-trace analysis.
Read the full abstract ▾
All authors Thomas Gorges, Janne van der Loop, Lukas Hüttner, Linda-Sophie Schneider, Fei Wu, Mathias Seuret, Vincent Christlein
Computer Vision · Cross-listed from physics.optics
Pre-training Enables Extraordinary All-optical Image Denoising
Xudong Lv, Yuxiang Sun, Shuo Wang et al. (+3 more)
Optical neural networks are emerging as powerful machine learning and information processing tools because of their potential advantages in speed and energy efficiency. The training methods of these physical models, however, remain underexplored compared to their digital counterparts and are leading to suboptimal performance. This paper reports a pre-training-driven approach that leads to snapshot image denoising with substantially improved quality. We demonstrated effective free-space optical denoising by a diffractive network optimized by a two-step process including (1) pre-training using a massive
dataset of 3.45 million diverse but simple images and (2) fine-tuning with the corresponding task-specific
datasets. Compared to conventional
Fourier-domain filtering and directly trained diffractive networks, such a transfer learning process exhibited prominent advantages for denoising images degraded by severe noise, peak signal-to-noise ratio (PSNR) below 8 dB, while preserving fine image features and improving the PSNR to above 18 dB. Importantly, the same pre-trained optical network could be consistently fine-tuned to process degraded images from highly diverse styles ranging from handwritten digits (MNIST) and chest X-rays (ChestMNIST) to CIFAR-10 images and human faces (CelebA). We further demonstrated the critical role of our optical denoisers in
vision-based applications, including face detection, plate recognition, and localization of
UAVs in noisy conditions.
Read the full abstract ▾
All authors Xudong Lv, Yuxiang Sun, Shuo Wang, Nanxing Chen, Jun Guan, Jingtian Hu
Computer Vision
Text-to-CAD Evaluation with CADTests
Dimitrios Mallis, Marco Wang, Ahmet Serdar Karadeniz et al. (+3 more)
Text-to-CAD has recently emerged as an important task with the potential to substantially accelerate design workflows. Despite its significance, there has been surprisingly little work on Text-to-CAD evaluation, and assessing CAD model generation performance remains a considerable challenge. In this work, we introduce a new evaluation perspective for Text-to-CAD based on automated testing. We propose CADTestBench, the first test-based
benchmark for Text-to-CAD, based on CADTests, executable software tests that verify whether a generated CAD model satisfies the
geometric and topological requirements of the input
prompt. Using CADTestBench, we conduct comprehensive benchmarking of recent Text-to-CAD methods and further demonstrate that CADTests can also guide CAD model generation, yielding simple baselines that surpass performance of current methods. CADTestBench code and data are available at GitHub and Hugging Face
dataset.
Read the full abstract ▾
All authors Dimitrios Mallis, Marco Wang, Ahmet Serdar Karadeniz, Elisa Ricci, Anis Kacem, Djamila Aouada
Computer Vision
SARA: Semantically Adaptive Relational Alignment for Video Diffusion Models
Jiesong Lian, Zixiang Zhou, Ruizhe Zhong et al. (+6 more)
Recent
video diffusion models (VDMs) synthesize visually convincing clips, yet still drop entities, mis-bind attributes, and weaken the interactions specified in the
prompt. Representation-
alignment objectives such as VideoREPA and MoAlign improve fine-grained text following by distilling spatio-temporal
token relations from a frozen visual foundation model, but their pairwise supervision budget is allocated by visual or motion cues rather than by how relevant each pair is to the
prompt. We present SARA, Semantically Adaptive Relational
Alignment, which keeps
token-relation
distillation (TRD) on a frozen VFM target and adds a text-conditioned saliency that decides which
token pairs carry supervision. A lightweight Stage~1 aligner is trained with per-entity SAM~3.1 mask supervision and an InfoNCE regulariser, and its continuous saliency is fused into TRD through a pair-routing operator that assigns each
token pair a weight whenever either of its two endpoints is salient, thereby routing supervision toward subject-subject and subject-background pairs and away from background-background ones. In the Wan2.2 continual-training setting, SARA improves both text
alignment and motion quality over SFT, VideoREPA, and MoAlign on a 13-dimension VLM rubric, on the public VBench
benchmarks, and in a blind user study. Project page: https://saradit.github.io/.
Read the full abstract ▾
All authors Jiesong Lian, Zixiang Zhou, Ruizhe Zhong, Yuan Zhou, Qinglin Lu, Rui Wang, Long Hu, Yixue Hao, Baoru Huang
Computer Vision · Cross-listed from cs.LG
Hessian Surgery: Class-Targeted Post-Hoc Rebalancing via Hessian Spike Perturbation
Hugo Vigna and Samuel Bontemps
The Hessian spectrum of trained deep networks exhibits a characteristic structure: a continuous bulk of near-zero
eigenvalues and a small number of large outlier
eigenvalues (spikes), confirming the relevance of Random
Matrix Theory in deep learning. The spike count matches the number of classes minus one. While prior work has described this structure, no method has exploited it operationally to improve classification performance. We propose Hessian Surgery, a post-hoc optimization method that directly perturbs model weights along spike eigenvectors to rebalance per-class accuracy without retraining. We introduce (i) a spike-class sensitivity
matrix that quantifies the directional derivative of each class's accuracy along each spike eigenvector, (ii) a constrained optimization of perturbation coefficients that targets weak classes while preserving strong ones, and (iii) an adaptive amplitude control that raises or lowers the perturbation budget based on iteration-level improvement signals. We obtain encouraging results on CIFAR-10 and ISIC-2019 on both balanced accuracy and standard deviation.
Read the full abstract ▾
Computer Vision
APEX: Assumption-free Projection-based Embedding eXamination Metric for Generative Distribution Evaluation
Caterina Gallegati, Monica Bianchini, Franco Scarselli et al. (+2 more)
As generative models achieve unprecedented visual quality, the gold standard for image evaluation remains traditional feature-distribution metrics. However, these metrics are provably hindered by the closed-vocabulary bottleneck of outdated features and the assumptive bias of rigid parametric formulations. Recent alternatives exploit modern backbones to solve the feature bottleneck, yet continue to suffer from parametric limitations. To close this gap, we introduce APEX (Assumption-free Projection-based
Embedding eXamination), a novel evaluation framework leveraging two open-vocabulary foundation models (CLIP and DINOv2) as feature extractors and the Sliced Wasserstein Distance (SWD) as similarity measure. Since SWD requires no assumptions about the compared data distributions, APEX operates as an assumption-free metric. Moreover, we prove with theoretical and empirical evidences that it inherits effective scalability to high-dimensional spaces. Benchmarking APEX metrics against established baselines, we show that they exhibit strong robustness to visual degradations, intra- and cross-
dataset stability, ensuring highly stable evaluations on out-of-domain
datasets.
Read the full abstract ▾
All authors Caterina Gallegati, Monica Bianchini, Franco Scarselli, Vittorio Murino, Barbara Toniella Corradini
Computer Vision
Radiologist-Guided Causal Concept Bottleneck Models for Chest X-Ray Interpretation
Amy Rafferty, Rishi Ramaesh and Ajitha Rajan
Concept Bottleneck Models (CBMs) in medical imaging aim to improve model interpretability by predicting intermediate clinical concepts before final diagnoses. However, most existing CBMs treat concepts as discriminative predictors of pathology labels, without explicitly modelling the underlying clinical generative process where diseases produce observable radiographic findings. We propose XpertCausal, a radiologist-guided causal CBM for chest X-ray interpretation which models pathology-to-concept relationships using a probabilistic noisy-OR framework. This generative model is then inverted via
Bayesian inference to estimate pathology probabilities from predicted concepts. Radiologist-curated concept-pathology associations are used to constrain model structure to radiologist-defined clinically plausible
reasoning pathways. We evaluate XpertCausal on MIMIC-CXR across pathology classification performance, calibration, explanation quality, and
alignment with radiologist-defined
reasoning pathways. Compared with both a non-causal CBM baseline and a causal ablation with unconstrained learned associations, XpertCausal achieves improved AUROC, calibration, and clinically relevant explanation quality, while learning concept-pathology relationships that more closely align with expert knowledge. These results demonstrate that incorporating clinically motivated causal structure and expert domain knowledge into CBMs can lead to more accurate, interpretable, and clinically aligned models for CXR interpretation.
Read the full abstract ▾
Computer Vision
Differentiable Ray Tracing with Gaussians for Unified Radio Propagation Simulation and View Synthesis
Niklas Vaara, Lam Huynh, Pekka Sangi et al. (+2 more)
Explicit neural representations such as 3D Gaussian Splatting (3DGS) enable high-fidelity and real-time novel view synthesis, yet optimize for alpha-composited optical appearance rather than ray-intersectable geometry. In contrast,
radio-frequency (RF) digital twins require deterministic multi-bounce paths, where the geometry dictates trajectories and their associated attenuation and delay. We introduce a framework enabling differentiable RF propagation simulation directly within visually reconstructed neural scenes, allowing point-to-point path computation between arbitrary 3D locations while preserving high-quality visual rendering. Unlike conventional RF simulation pipelines that rely on manually constructed meshes, we embed Gaussian primitives into a hardware-accelerated ray tracing structure as the underlying spatial representation. By extracting physically meaningful channel impulse responses from visual-only reconstructions, we provide cross-modal evidence that neural reconstructions can serve as unified spatial representations for both electromagnetic propagation simulation and photorealistic view synthesis.
Read the full abstract ▾
All authors Niklas Vaara, Lam Huynh, Pekka Sangi, Miguel Bordallo López, Janne Heikkilä
Computer Vision
SIMI: Self-information Mining Network for Low-light Image Enhancement
Xuanshuo Fu, Lei Kang and Javier Vazquez-Corral
Poor lighting conditions significantly impact image quality, posing substantial challenges for image editing and visualization. Many existing enhancement methods aim at proposing complex models while neglecting the intrinsic information contained within low-light images. In this work, we propose the Self-Information Mining (SIMI) network, an innovative unsupervised framework that decomposes low-light images into multiple components based on bit-plane decomposition. Our approach allows mining intrinsic information without relying on external data. This not only accelerates model convergence but also improves performance and reduces computational overhead. The unsupervised nature of our method facilitates real-world applicability. Experiments conducted on standard
benchmarks demonstrate that SIMI achieves state-of-the-art performance.
Read the full abstract ▾