Computer Vision
OWMDrive: Causality-Aware End-to-End Autonomous Driving via 4D Occupancy World Model
Junjie Cheng, Ruiqi Song, Ye Wu et al. (+3 more)
Autonomous driving systems are steadily moving toward end-to-end paradigms to mitigate the limited adaptability of rule-based pipelines in complex traffic environments. However, most existing learning-based methods still make decisions from static representations of the current scene, without explicit future rollouts or modeling of the temporal causal dynamics in traffic interactions. This limitation often results in unstable or overly conservative planning under high-uncertainty conditions, such as occlusions and unexpected events. To overcome these challenges, we introduce OWMDrive, a generative end-to-end driving framework built upon an Occupancy World Model for multi-step 3D occupancy
forecasting, which serves as a conditional prior to guide
diffusion-based planning. Conditioned on both current observations and predicted future states, the planner iteratively refines trajectory candidates to generate a reinforced driving trajectory. By explicitly modeling scene
evolution over future horizons, OWMDrive captures key spatiotemporal causal dependencies, which leads to more foresighted and robust trajectory generation. Extensive experiments demonstrate that OWMDrive significantly improves planning reliability and safety, especially in challenging and partially observable driving scenarios.
Read the full abstract ▾
All authors Junjie Cheng, Ruiqi Song, Ye Wu, Nanxing Zeng, Ximiao Li, Yunfeng Ai
Computer Vision
Beyond Point Estimates for Glaucoma Visual Field Forecasting with Diffusion Models
Marta Colmenar Herrera, Pablo Márquez Neila, Şerife Seda Kucur Ergünay et al. (+2 more)
Forecasting visual fields (VFs) is critical for personalized monitoring and treatment planning in glaucoma. This is inherently uncertain due to heterogeneous disease progression and measurement variability, yet most existing methods produce single deterministic predictions that fail to represent this uncertainty. We formulate VF
forecasting as a probabilistic prediction problem and the use of conditioned denoising
diffusion models to generate distributions of plausible future VFs from longitudinal observations with irregular follow-up intervals. Experiments on two independent VF cohorts show that
diffusion-based predictions produce well-calibrated distributions for clinically relevant VF measures. When reduced to a standard point-estimate, the proposed approach achieves state-of-the-art accuracy compared to clinical baselines and prior learning-based methods. Our results highlight the advantages of distributional modeling for VF
forecasting and support a shift from point-estimate prediction toward uncertainty-aware, clinically interpretable risk assessment in glaucoma.
Read the full abstract ▾
All authors Marta Colmenar Herrera, Pablo Márquez Neila, Şerife Seda Kucur Ergünay, Martin S. Zinkernagel, Raphael Sznitman
Computer Vision
SA-Homo: Scale Adaptive Homography Estimation for Scale Variation Scenarios
Shangxuan Xie, Haifeng Wu, Yuhang Wang et al. (+2 more)
Homography estimation, as one of the fundamental problems in computer
vision, remains challenged by scale variation scenarios where image pairs potentially exhibit significant scale discrepancies. Existing deep learning frameworks frequently suffer from a significant performance degradation in such cases, as they rely on limited displacement assumptions and local feature consistency that might not hold under large scale gaps. In this paper, we propose SA-Homo, a novel scale-adaptive homography estimation framework designed to achieve robust
alignment across a wide range of scale discrepancy ratios. We adopt a hierarchical scale
alignment strategy that transitions from the global perspective with a heavy module to a local perspective with a light module. Specifically, we introduce the Scale-aware Discrepancy Bridging Module (SDBM) for initial
alignment, which utilizes a Multi-scale Linear
Attention Cascade (MLAC) to capture long-range dependencies and mitigate feature inconsistencies, along with a global Cross-scale Similarity
Matrix Block (CSMB) for scale robust correlation representation. Once the initial scale gap is bridged, a lightweight Iterative Homography Estimation Refinement Module (IHERM) progressively polishes the result using local correlations. To facilitate this research, we contribute the HMSA
dataset, a high-resolution, multi-modal satellite
benchmark specifically tailored for scale-variant challenges. Extensive experiments demonstrate that SA-Homo maintains high precision even under 8$\times$ scale discrepancies, outperforming state-of-the-art methods in both conventional scale-similar scenarios and challenging scale variation scenarios. Code and collected
datasets are available at https://github.com/shangxuanx330/SA_Homo
Read the full abstract ▾
All authors Shangxuan Xie, Haifeng Wu, Yuhang Wang, Huarong Jia, Wen Li
Computer Vision
SADL: What to Ignore? A Benchmark for Subject-Aware Distractor Localization
Cao-Tri Nguyen, Nguyen-Khoa Luong, Vinh-Tiep Nguyen et al. (+1 more)
Photographs frequently contain visual distractors besides foregrounds and backgrounds of the intended subject, competing for
attention and weakening composition. While modern editing tools streamline object removal, identifying which objects to remove remains a mostly manual process. Existing saliency models and open-vocabulary detectors operate without subject awareness, failing to adapt to shifting user intent. Furthermore, context-agnostic removal may disrupt the scene's semantic coherence (e.g., keep the person but remove the chair they are sitting on). To address these limitations, we formalize the task of subject-aware distractor localization, which identifies distractors while retaining compositionally essential objects. This paper introduces SADL, the first real-world
benchmark for this task, comprising 1,800 subject-aware cases across 1,000 photographs to enable systematic evaluation and facilitate future research. In total, there are 14,617 annotated candidates, including a robust set of 1,938 hard negatives to stress-test exclusion calibration. We evaluate seven proprietary and open-weight
Vision-
Language Models (VLMs) on a sequential pipeline of distractor classification followed by exclusion filtering, structured around five inclusion factors and three contextual exclusion rules. Our analysis reveals that VLMs are highly capable of identifying distractors, but then over-apply exclusion, which systematically suppresses true distractors at scale. By exposing this critical bottleneck, SADL provides a foundational diagnostic tool to advance subject-conditioned
reasoning in
multimodal systems.
Read the full abstract ▾
All authors Cao-Tri Nguyen, Nguyen-Khoa Luong, Vinh-Tiep Nguyen, Minh-Triet Tran
Computer Vision · Cross-listed from cs.GR
RenderFormer++: Scalable and Physics-Informed Feed-Forward Neural Rendering
Huangsheng Du, Haoran Zhu, Youcheng Cai et al. (+2 more)
We present RenderFormer++, a scalable and physics-informed feed-forward neural rendering framework for global illumination in mesh scenes. Existing
Transformer-based neural rendering methods such as RenderFormer achieve promising cross-scene generalization, but lack explicit transport priors and scale poorly due to quadratic triangle-level
attention. To address these issues, we introduce Physics-Informed Transport Guidance (PITG), which embeds rendering-equation-inspired inductive biases into the
attention mechanism and introduces a transport consistency loss, encouraging physics-informed light transport modeling. We further propose Hierarchical Object-Centric Tokenization (HOCT), which aggregates triangle-level features into compact object-level
tokens via cross-
attention with learnable queries, substantially reducing computational and
memory costs. Extensive experiments demonstrate that RenderFormer++ achieves scalable and generalizable feed-forward global illumination rendering across complex large-scale scenes with competitive rendering quality and substantially improved efficiency over RenderFormer. The code will be made publicly available upon acceptance.
Read the full abstract ▾
All authors Huangsheng Du, Haoran Zhu, Youcheng Cai, Jingyang Meng, Ligang Liu
Computer Vision
OmniCoT: A Benchmark for Global and Multi-Step Panoramic Reasoning
Haocong He, Chenfei Liao, Zichen Wen et al. (+13 more)
Multimodal Large
Language Models (MLLMs) have demonstrated promising spatial
reasoning capabilities, while these abilities remain underexplored in the emerging visual modality of panoramic imagery. The full 360{\deg}$\times$180{\deg} field of view of panoramas essentially supports complex global multi-step
reasoning, which is also the fundamental advantage of panoramas in applications such as
embodied intelligence. However, existing panoramic
benchmarks largely focus on simplistic queries that rely on local cues or single-/few-step
reasoning, thereby ignoring the fundamental advantage of panoramas and failing to fully exploit their potential. To address this gap, we introduce OmniCoT, a panoramic spatial
reasoning suite designed to enable MLLMs to use global evidence and perform multi-step inference across viewpoints. It includes OmniCoT-B (6.7K data) for evaluation, which measures both answer accuracy and
reasoning quality, OmniCoT-Real (1K data) as a manually annotated real-world subset to quantify the Sim-to-Real gap. For training, OmniCoT-T (14.3K data) is purpose-built with structured stepwise Chain-of-Thought annotations that explicitly link intermediate
reasoning steps to panoramic evidence. Based on OmniCoT-T, we introduce OmniCoT-R1 and adopt a two-stage training strategy tailored to the geometrically complex panoramic space, where Supervised Fine-tuning (SFT) anchors
reasoning to panoramic evidence (e.g., bearings, proximity) and GRPO penalizes geometrically incoherent paths to consolidate global 360{\deg} spatial consistency. Through OmniCoT, we aim to recalibrate the difficulty of panoramic spatial
reasoning to better align with the intrinsic capabilities of panoramic imagery, thereby fostering meaningful progress in this research area.
Read the full abstract ▾
All authors Haocong He, Chenfei Liao, Zichen Wen, Zihao Dongfang, Xu Zheng, Bin Ren, Chang Su, Zixin Zhang, Harold Haodong Chen, Hongfei Zhang, Weijia Li, Kailun Yang, Conghui He, Xuming Hu, Nicu Sebe, Linfeng Zhang
Computer Vision · Cross-listed from cs.LG
FlowAWR: Online Adaptive Flow Reinforcement via Advantage-Weighted Rectification
Zheming Fu, Ruizhe He, Wei Shang et al. (+5 more)
Aligning generative flow models on continuous spaces via online reinforcement learning is constrained by intractable trajectory likelihoods. Existing density-approximated policy gradient methods rely on
stochastic SDE samplers to construct tractable transition kernels, which introduce training-inference inconsistencies and necessitates Classifier-Free Guidance (CFG). While implicit frameworks such as DiffusionNFT directly optimize forward-process velocity fields, its heuristic fixed-magnitude corrections prevent optimization strength from relative intra-group quality. We propose Flow Advantage-Weighted Rectification (FlowAWR), a paradigm that recasts continuous generative policy optimization as supervised regression toward a theoretically optimal velocity field. Starting from the optimal policy of a KL-constrained reward maximization, FlowAWR derives the optimal velocity field that admits a magnitude-aware, advantage-weighted rectification form, yielding SDE-free optimization and CFG-free generation. In comparative evaluations on SD3.5-Medium, FlowAWR achieves improved
alignment performance alongside a 2$\times$ to 5$\times$ convergence acceleration over DiffusionNFT (e.g., reaching a 24.12 PickScore in 1.2k steps, versus 23.82 in 2.0k steps for DiffusionNFT and 23.50 in $>$4k steps for FlowGRPO). Under multi-reward constraints, FlowAWR sustains generation quality, satisfying structural rules while maintaining stable out-of-domain performance. Code at \href{https://github.com/Fuzm7/FlowAWR}{FlowAWR}.
Read the full abstract ▾
All authors Zheming Fu, Ruizhe He, Wei Shang, Xiaoxiao Ma, Lianrui Mu, Lei Wang, Chang Liu, Siming Fu
Computer Vision
Set-Inclusive Uncertainty Modeling for Robust Brain Tumor Segmentation
Seunghun Baek, Jihwan Park, Jaeyoon Sim et al. (+3 more)
Multimodal MRI is essential for accurate
brain tumor
segmentation. However, acquiring all modalities at inference is often challenging in practice, which causes intrinsic uncertainty due to unavoidable information loss. Without modeling this uncertainty, existing methods encode incomplete evidence into deterministic representations that appear plausible but lack reliability. In this regime, we propose a probabilistic representation framework that models representations as Gaussian distributions, where their mean captures task information and their variance measures uncertainty from missing evidence. To make variance reflect information deficiency, we regularize the mean from each partial configuration toward its full-modality counterpart, while scaling the variance with the discrepancy between their aligned means. We further introduce a set-inclusive strategy that exploits the hierarchical structure of modality subsets and enforces an ordering constraint to maintain their consistent uncertainty relationships. Extensive experiments on BraTS 2018 and 2020 demonstrate that our approach offers superior performance over baselines across diverse missing-modality scenarios. Code and model checkpoint are available at https://github.com/atlas-sky/SIUM.
Read the full abstract ▾
All authors Seunghun Baek, Jihwan Park, Jaeyoon Sim, Hoseok Lee, Seungjoo Lee, Won Hwa Kim
Computer Vision
MUSE: Unlocking Timestep as Native Task Steering for One-Step Dense Prediction
Shuo Zhou, Zhaoxin Li and Xiujuan Chai
Monocular dense prediction has recently seen remarkable success by repurposing pre-trained
diffusion models. This opens a promising yet challenging avenue for more efficient multi-task learning paradigm. However, existing multi-task
diffusion methods often introduce parameter-heavy adapters, experts, or learnable task
tokens, leading to computational redundancy. In this paper, we reveal an inherent mechanism within one-step
diffusion models: the native, fixed sinusoidal timestep
embedding can be repurposed as an endogenous task steering signal. Based on this discovery, we propose Multi-task Unified eStimation via timestep
Embedding (MUSE), a parameter-free, single-model multi-tasking approach for dense prediction. We interpret this mechanism via
Manifold Decoupling, where discrete, fixed timestep values deterministically steer the generation process towards decoupled, task-specific
manifolds in the latent space. Extensive experiments across 10
datasets demonstrate that MUSE achieves highly competitive performance on both
monocular depth and normal estimation, and its efficacy generalizes across U-Net and DiT architectures. Our work offers a concise and efficient path toward generalist
vision models by simply unlocking the latent potential of existing generation infrastructure.
Read the full abstract ▾
Computer Vision
CouCE: A Unified Causal Framework for Debiased Deep Metric Learning
Xin Yuan, Zhenyang Niu, Meiqi Wan et al. (+3 more)
Deep Metric Learning (DML) often struggles with zero-shot generalization because standard objectives inherently capture what co-occurs rather than what causes similarity. Consequently, DML models are vulnerable to shortcut learning driven by two structurally distinct confounders: background spurious correlations (which create
backdoor paths via scene context) and foreground nuisance perturbations (which inject non-semantic variations like pose or illumination). Although existing methods have proposed targeted solutions for each pathway individually, none can simultaneously address both due to their fundamentally distinct causal roles. To bridge this gap, we propose the Counterfactual Causal
Embedding (CouCE), a unified causal framework that explicitly models and neutralizes both confounders. Specifically, we introduce Orthogonal Dictionary-Based
Backdoor Adjustment (ODBA), which isolates spurious background patterns into a variance-gated dictionary and stably disentangles them from the learned
embeddings via soft orthogonal regularization. Simultaneously, we propose Multi-Scale Randomized Causal Intervention (MSRCI) to enforce causal invariance against foreground nuisances through multi-scale
Fourier amplitude randomization and a symmetric KL invariance constraint. Notably, CouCE seamlessly integrates with any proxy-based loss, incurring modest training overhead without requiring architectural modifications during inference. Extensive experiments on CUB-200-2011, Cars-196, and Stanford Online Products demonstrate that CouCE consistently achieves state-of-the-art performance, providing a principled and robust solution for debiased DML.
Read the full abstract ▾
All authors Xin Yuan, Zhenyang Niu, Meiqi Wan, Huilin Zhu, Xin Xu, Kui Jiang
Computer Vision · Cross-listed from cs.RO
ReactiveBFM: Reactive Closed-Loop Motion Planning Towards Universal Humanoid Whole-Body Control
Xiao Chen, Weishuai Zeng, Xiaojie Niu et al. (+12 more)
While current Behavior Foundation Models (BFMs) provide robust control priors for
humanoids, they only execute pre-defined reference motions. As a result, they are vulnerable to environmental shifts and incapable of reactive whole-body coordination. Naively cascading them with generative motion planners fails to achieve true reactivity, as inevitable tracking discrepancies induce fatal cumulative exposure bias. To bridge this gap, we propose ReactiveBFM, a real-time closed-loop planning-control framework. At its core, we effectively mitigate exposure bias via a scheduled prefix sampling curriculum, forcing the generative planner to actively learn error-recovery behaviors from imperfect physical states rather than ground-truth trajectories. Systematically, to reconcile the severe latency mismatch between auto-regressive planning and high-frequency tracking, we introduce an asynchronous replanning mechanism. Combined with trajectory chunking to temporally ensemble spatial references, our system guarantees spatio-temporally fluid execution without physical jitter. Deployed on the Unitree G1
humanoid, ReactiveBFM demonstrates unprecedented physical agility across a vast repertoire of text-conditioned closed-loop motions. Notably, ReactiveBFM achieves zero-shot moving target reaching, showcasing intricate whole-body coordination and on-the-fly replanning. In sim-to-sim benchmarking under severe perturbations, ReactiveBFM achieves a 93.1% success rate, significantly outperforming cascaded open-loop baselines by 28.6%.
Read the full abstract ▾
All authors Xiao Chen, Weishuai Zeng, Xiaojie Niu, Zirui Wang, Jianan Li, Huayi Wang, Furui Xu, Jiahe Chen, Weixiang Zhong, Lihe Ding, Kailin Li, Jiangmiao Pang, Tai Wang, Tianfan Xue, Jingbo Wang
Computer Vision · Cross-listed from cs.LG
On the Vulnerability of Parameter-Level Defenses to Model Merging
Kuangpu Guo, Qingyan Zheng, Jian Liang et al. (+4 more)
The training-free integration of expert models via model merging has exposed significant security risks, enabling free-riders to combine specialized models without authorization. Recent works propose parameter-level defenses that employ linear parameter transformations to neutralize this threat. In this paper, we systematically analyze such defenses and reveal that their protected task vectors are inherently small in magnitude. Consequently, the protected weights remain overwhelmingly dominated by the pretrained model. Based on this observation, we designate the pretrained model as a static reference anchor and propose the Anchor-Guided Attack (AGA) to circumvent existing safeguards. Specifically, AGA aligns the protected model with this anchor to recover the transformation
matrix analytically. Extensive evaluations validate that AGA consistently bypasses both individual and composite defenses under realistic defense-agnostic scenarios. Furthermore, we provide Anchor-Repulsive Fine-tuning (ARF), a defense method to mitigate the anchor dominance leveraged by AGA. Empirical results confirm that ARF effectively defeats the proposed attack. Our code is available at https://github.com/krumpguo/secure-merge-attack.
Read the full abstract ▾
All authors Kuangpu Guo, Qingyan Zheng, Jian Liang, Yongcan Yu, Zilei Wang, Ran He, Tieniu Tan
Computer Vision
Residual-Guided Expert Specialization for Incomplete Multimodal Learning
Seunghun Baek, Jihwan Park, Jaeyoon Sim et al. (+3 more)
As real-world prediction systems often face missing modalities at inference, incomplete
multimodal learning (IML) remains a practical challenge. While prior methods aim to learn representations robust to missing inputs, representations from incomplete modalities inevitably deviate from their full-modality counterparts due to missing evidence. To explicitly leverage these deviations, we propose MARS (Missingness-Aware Residual-guided Specialization), a mixture-of-experts framework that guides expert specialization based on how representations are reshaped by missingness. By contrasting task representations derived from incomplete inputs with their complete counterparts during training, we derive a privileged residual signal that captures this representational gap. The residual signal guides a residual router to assign samples to experts specialized for the corresponding deviation patterns. In parallel, a feature router learns to imitate this routing behavior using only incomplete inputs, enabling deployment without access to full modalities. To mitigate this train-test router gap, we develop a discrepancy-aware noise regularization that adaptively perturbs the residual router's decisions when the feature router deviates, enhancing expert robustness under imperfect imitation. Experiments on
multimodal classification (CASIA-SURF, CREMA-D, UPMC Food-101) and
segmentation (MCubeS) under missing scenarios show that MARS consistently surpasses baselines while remaining efficient and extensible to diverse backbones and tasks.
Read the full abstract ▾
All authors Seunghun Baek, Jihwan Park, Jaeyoon Sim, Minjae Jeong, Hoseok Lee, Won Hwa Kim
Computer Vision
FastPano3D: Feed-Forward Indoor Panoramic 3D Reconstruction from a Single Image
Jianqiang Li, Liumei Zhang, Wenjia Guo et al. (+5 more)
Recent advances in 3D scene reconstruction have highlighted the intricate trade-offs among rendering quality, inference efficiency, and data dependency. To address the challenge of rapidly reconstructing detailed 3D indoor scenes from minimal input, we introduce FastPano3D, an end-to-end framework that directly generates renderable 3D Gaussian representations from a single panoramic image. Unlike perspective-based methods, panoramic images inherently suffer from equirectangular projection distortions and spatially non-uniform feature distributions, making direct feed-forward Gaussian generation particularly challenging. In contrast to existing Gaussian Splatting based methods that rely on multi-view supervision or per-scene optimization, FastPano3D employs a lightweight feature encoder, adaptive Gaussian sampling, and a point-cloud-guided refinement strategy to achieve efficient and accurate scene generation without any test-time optimization. Our approach reconstructs high-fidelity 3D scenes within seconds, achieving up to 156 times faster inference than prior state-of-the-art methods such as Pano2Room, while using only half the parameters. Extensive experiments demonstrate that FastPano3D delivers rendering quality comparable to NeRF- and 3DGS-based reconstructions, establishing a new
benchmark for rapid, single-view 3D scene inference.
Read the full abstract ▾
All authors Jianqiang Li, Liumei Zhang, Wenjia Guo, Tianlong Feng, Yongzhi Liao, Di Lu, Hanchi Ren, Jingjing Deng
Computer Vision
FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Images
Jianjiang Yao, Ke Xian, Renxiang Dai et al. (+1 more)
We present FFAvatar, a
Transformer-based 3D Gaussian framework for fast construction of high-quality and animatable 4D head avatars from one or more reference portrait images. Unlike existing feed-forward approaches that require a fixed number of input views, FFAvatar supports incremental reconstruction, progressively refining the avatar representation as additional reference images become available. At the core of our method is an alternating
attention mechanism that disentangles identity appearance from expression and viewpoint variations, enabling the reconstruction of a canonical 3D appearance that remains consistent across poses and facial expressions. To balance visual fidelity and computational efficiency, we introduce a sparse-to-dense learning paradigm. Coarse appearance features are first learned using sparse primitives anchored to the FLAME vertex level and are subsequently densified in the UV domain to capture fine-grained
geometric and texture details. We further propose a plug-and-play motion refinement module that enables subject-specific dynamic personalization by modeling residual motion beyond parametric deformation. Extensive experiments demonstrate that FFAvatar efficiently produces high-fidelity and controllable 4D head avatars, achieving superior flexibility, driving efficiency, and identity-consistent rendering across diverse expressions and viewpoints.
Read the full abstract ▾
All authors Jianjiang Yao, Ke Xian, Renxiang Dai, Robert Caiming Qiu
Computer Vision
Early Cue Precision Shapes Visual Shortcut Learning in Controlled Cue-Manipulation Benchmarks
Chanho Park, Woochan Lee, Janyeong Oh et al. (+4 more)
Visual classifiers can achieve high matched-distribution accuracy while relying on low-level cues that fail under conflict or suppression. We test whether this failure is shaped by early cue precision: the reliability with which a low-level cue predicts the label during early learning or downstream probe fitting. Across synthetic shape-texture tasks, sequential digit training, a 10-class frozen-representation audit, and a CIFAR-10 natural-image-based texture-overlay
benchmark, we manipulate object-texture match probability and evaluate matched-ID accuracy, conflict accuracy, texture-choice rate, and suppression behavior. Degraded-but-predictive input does not substitute for cue decorrelation. In 10-class digit probes, conflict accuracy drops from 0.589 under chance-like cue precision to 0.005 under target-perfect texture. In CIFAR-10 frozen probes, conflict accuracy drops from 0.569 to 0.114, while texture choice rises from 0.049 to 0.855; this ordering persists across texture-overlay strengths alpha in {0.15,0.25,0.35,0.50}. End-to-end CIFAR-10 training shows that low early cue precision improves pre-target conflict behavior, but shortcut-rich fine-tuning can rapidly overwrite this benefit. Cue decorrelation must therefore be maintained during downstream adaptation rather than treated as a one-time inoculation.
Read the full abstract ▾
All authors Chanho Park, Woochan Lee, Janyeong Oh, Geongho Gong, Minshu Kim, Yeachan Kwak, Seongim Choi
Computer Vision
A Classifier-Agnostic Zero-Shot Adversarial Attack Detection via CLIP
Hodaya Krakover, Meir Yossef Levi, Eyal Gofer et al. (+1 more)
Adversarial attacks pose a challenge to the reliability of deep learning models, motivating effective detection methods. Existing techniques often rely on attack-specific assumptions, access to
adversarial samples, or knowledge of the underlying classifier (white-box). We propose $A^4D$ Attack- and Architecture-Agnostic
Adversarial Detector, a completely black-box, zero-shot
adversarial attack detection framework that utilizes
prompt-based similarity scores derived from CLIP. To the best of our knowledge this is the first attempt to utilize CLIP for such a task. The method is based on two key observations: (i) CLIP is sensitive even to small imperceptible non-semantic perturbations; (ii) The shift in CLIP
embedding space is not arbitrary and can be used as a robust attack indicator. Experiments across multiple attacks,
datasets and classifiers validate that $A^4D$ achieves SOTA detection results in the attack-agnostic and classifier-agnostic setting.
Read the full abstract ▾
All authors Hodaya Krakover, Meir Yossef Levi, Eyal Gofer, Guy Gilboa
Computer Vision
UniGP: Taming Diffusion Transformer for Prior-Preserved Unified Generation and Perception
Qin Guo, Hao Luo, Dongxu Yue et al. (+4 more)
Recent advances in
diffusion models have shown impressive performance in controllable image generation and dense prediction tasks. However, existing approaches typically treat
diffusion-based controllable generation and dense prediction as separate tasks, overlooking the potential benefits of jointly modeling the heterogeneous distributions. In this work, we introduce UniGP, a framework built upon MMDiT, which unifies controllable generation and dense prediction through simple joint training, without the need for complex task-specific designs or losses, while preserving the backbone's versatile priors. By learning controllable generation and prediction under different conditions, our model effectively captures the joint distribution of image-geometry pairs. UniGP is capable of versatile controllable generation, dense prediction, and joint generation. Specifically, the proposed UniGP consists of DUGP and a unified
dataset training strategy. The former, following the principle of Occam's razor, uses only a copied image branch of MMDiT to model dense distributions beyond RGB, while the latter integrates heterogeneous
datasets into a unified training framework to jointly model generation and perception tasks. Extensive experiments demonstrate that our unified model surpasses prior unified approaches and performs on par with specialized methods. Furthermore, we demonstrate that multi-task joint training provides complementary benefits: generative priors enrich perceptual details, while perceptual learning improves structural
alignment in generation.
Read the full abstract ▾
All authors Qin Guo, Hao Luo, Dongxu Yue, Weixuan Jin, Xiao Fu, Fan Wang, Dan Xu
Computer Vision
Optimizing Image Preparation and Compression for Face Recognition within 1024 Bytes
Paul Andreas, Torsten Schlett and Christoph Busch
ICAO-compliant machine readable travel documents enable automated biometric face verification. The biometric reference is stored on an RFID chip included in form of a JPEG or JPEG 2000 compressed facial image. In contrast, temporary travel documents lack of machine readability, which excludes the owner from such automated processes. This disadvantage could be solved by equipping such documents with 2D barcodes. This technology offers a resource-saving alternative to expensive RFID chips, while still offering machine readability and fast issuing processes. However, this solution introduces the challenge of storing the face images at significantly smaller storage capacities, creating the need for reducing the file size of the included facial image to a maximum of 1024 bytes. This study examines preprocessing steps and compression configurations, using JPEG, JPEG 2000, JPEG XL, JPEG AI, HEIF, AVIF, and WebP for image compression to this target size, while still preserving as much face recognition performance as possible. While the reference sample must always comply with ICAO specifications, the individual samples may or may not meet these requirements, depending on the application. This work optimizes compression steps for both of these prerequisites. It is shown that the recently standardised JPEG AI, when using optimized settings, provides the best face recognition performance, in particular when the comparison includes only images with high face image quality. AVIF and WebP also provide good results. The losses caused by the strong lossy compression are comparatively small. For the comparison of ICAO-compliant face images only, converting the images to grayscale proves to be a helpful preprocessing step, whereas for comparisons involving less suitable samples, preserving color is preferable. In addition, smoothing and resizing the images beforehand also turns out to be beneficial.
Read the full abstract ▾
Computer Vision
BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language
Haitao Wu, Qirui Zhang, Zhouheng Yao et al. (+8 more)
Modeling the bidirectional correspondence between external sensory stimuli and internal neural activity has emerged as a critical frontier in neuroscience. However, existing approaches predominantly treat
brain encoding and decoding as isolated tasks, relying heavily on unimodal
alignment and external priors while overlooking the
brain's intrinsic nature as a
multimodal integration system. To address these limitations, we propose BrainJanus, the first unified
brain model that integrates
brain,
vision, and
language within a single framework. Specifically, we introduce a Unified
Brain Tokenizer to quantize continuous neural dynamics into discrete
tokens aligned with visual and linguistic representations in a shared Omni space. Building on this, we utilize an All-in-One autoregressive architecture that leverages next-
token prediction to enable seamless any-to-any generation, which encompasses image-to-
brain and text-to-
brain encoding, and
brain-to-image and
brain-to-text decoding. Extensive experiments demonstrate that BrainJanus achieves superior performance across diverse
benchmarks. Furthermore, our framework exhibits zero-shot generalization and preserves interpretable biological topography, highlighting its potential as a general-purpose
brain modeling paradigm. The code is available at \href{https://github.com/HaitaoWuTJU/BrainJanus}{GitHub}.
Read the full abstract ▾
All authors Haitao Wu, Qirui Zhang, Zhouheng Yao, Shangquan Sun, Qihao Zheng, Mianxin Liu, Chi Zhang, Wanli Ouyang, Chunfeng Song, Changqing Zhang, Jiamin Wu
Computer Vision
Real-Time Underwater Image Enhancement via Frequency-Guided Dual-Path Attention
Leshen Zhang, Ao Li and Ce Zhu
Real-time
underwater image enhancement (UIE) is crucial for mobile
underwater photography and autonomous robotic systems, where practical deployment typically requires low latency and compact models under constrained computational resources. Recent ultra-lightweight CNNs based on structural re-parameterization meet these constraints but operate purely in the spatial domain, ignoring the frequency-sensitive nature of
underwater degradation. To address this, we propose a lightweight UIE framework that integrates two key components: a Multi-Branch Reparameterizable Convolution with Fixed DCT Priors (MBRConv-DCT) that injects structured directional frequency priors during training, and a Frequency-Guided Dual-Path
Attention (FGDPA) module that fuses spatial and spectral cues via a dual-path design for adaptive feature modulation. Both components are fully compatible with structural re-parameterization: the convolution branch introduces zero additional inference cost after re-parameterization, while the
attention module incurs only a minimal computational overhead. Experiments show our model achieves state-of-the-art performance with only 4.23K parameters and 600+ FPS, outperforming much larger methods in both quantitative metrics and visual quality. Code is available at https://github.com/LethyZhang/FGDPA.
Read the full abstract ▾
Computer Vision
TRACE: A Concept Bottleneck Model for Longitudinal 3D Glioblastoma Response Assessment
Alia Tarek, Hamsa Saberr, Hamza Elghonemy et al. (+6 more)
Longitudinal glioblastoma response assessment requires comparing subtle tumor changes across MRI time points using structured clinical criteria such as RANO. However, most deep learning methods predict response labels directly from imaging features, which limits clinical inspection, verification, and correction. We introduce TRACE, a RANO 2.0-aligned concept bottleneck model for interpretable 4-class glioblastoma response classification on longitudinal 3D MRI. TRACE processes paired baseline and follow-up
multimodal MRI scans with a shared 3D
vision encoder, predicts clinically meaningful tumor measurements as root concepts, computes downstream RANO-derived concepts through deterministic rules, and incorporates scan interval and new-lesion information as passthrough concepts. This design frames response assessment as structured concept
reasoning rather than direct image-to-label prediction. Using 5-fold patient-wise cross-validation on the LUMIERE
dataset, TRACE achieves a 4-class macro F1 of 0.4769 and a binary progression-versus-non-progression macro F1 of 0.7085. It improves over a concept bottleneck baseline and remains within the range of published non-interpretable deep learning approaches. Ablation studies show that the expert RANO graph and intervention-consistency training are important for performance, while intervention experiments demonstrate that correcting concepts can improve downstream predictions. These results suggest that structured concept bottlenecks offer a transparent and clinically aligned direction for longitudinal glioblastoma response assessment, while highlighting the need for larger protocol-aligned
datasets and external validation.
Read the full abstract ▾
All authors Alia Tarek, Hamsa Saberr, Hamza Elghonemy, Youssef Afify, Tamer Basha, Omair Shahzad Bhatti, Abdulrahman M. Selim, Hasan Md Tusfiqur Alam, Daniel Sonntag
Computer Vision
A Point Cloud Transformer for Remote Monitoring and Automated Assessment of Physical Rehabilitation Exercises
Kazi Rafat, Md. Ismail Hossain, M M Lutfe Elahi et al. (+4 more)
Rehabilitation exercises are essential in restoring lost physical functions of patients suffering from various diseases (e.g., Parkinson's, back pain). Carrying out these rehabilitation exercises, often prescribed by health experts, is costly, unavailable, and requires expert supervision. The availability of RGBD images and movement/position data of joints along with expert annotation of exercise data has prompted the use of automatic assessment of the quality of rehabilitation exercises, which is cost-effective and can be carried out at home. However, existing approaches do not extract relevant features, lack practical application, require expensive pre-processing, or overlook crucial features. This study proposes a
transformer-based framework for point clouds to extract features and assess rehabilitation exercises by analyzing joint positions collected through RGBD data. We adapt and utilize a curve-based point-cloud feature aggregation technique to augment point-cloud information that aids model output. The
transformer architecture also uses axial self-
attention, recognizing important joints and their roles to assist users in performing the exercise better. The guided system outperforms existing approaches and is also practically relevant due to its small size, fast inference, and generalization on specific joints in similar exercises. We conduct our experiments on three crucial baseline
datasets for rehabilitation exercises: Kimore, UI-PRMD, and IRDS.
Read the full abstract ▾
All authors Kazi Rafat, Md. Ismail Hossain, M M Lutfe Elahi, Sifat Momen, Fuad Rahman, Nabeel Mohammed, Shafin Rahman
Computer Vision
The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction
Yuxi Wang, Chengkai Jin, Yufei Liu et al. (+6 more)
4D hand motion reconstruction from
egocentric video is bottlenecked by clear limitations of existing methods: image-based pipelines depend on a detector that fails under heavy occlusion, while
video-based methods rely on temporal modules learned only from scarce hand-pose annotations, a narrow signal insufficient to model motion dynamics, occlusion
reasoning, and hand-object interaction. These capabilities, however, are exactly what
video generative models must implicitly acquire when trained to synthesize coherent
video at internet scale. Motivated by this, we present ViDiHand, which leverages the representations of a pretrained
video diffusion model to reconstruct 4D two-hand pose. We adapt it via a hand-overlay rendering objective that specializes its features for hands while preserving its world priors. A decoder then recovers metric-scale pose from the adapted features. The whole pipeline operates directly on full frames–no detector, no infiller, and no test-time optimization. On ARCTIC, HOT3D, and HOI4D, ViDiHand substantially outperforms prior methods, establishing
video diffusion models as a powerful new foundation for hand motion reconstruction and a promising route to scalable in-the-wild data collection for
embodied AI. Project page: https://vidihand.github.io.
Read the full abstract ▾
All authors Yuxi Wang, Chengkai Jin, Yufei Liu, Wenqi Ouyang, Tianyi Wei, Zhiwei Zeng, Siyuan Huang, Zhiqi Shen, Xingang Pan
Computer Vision · Cross-listed from cs.LG
DreamForge-World 0.1 Preview: A Low-Compute Real-Time Controllable World Model
Daniyel Ayupov and Artur Markov-Tsoy
We present DreamForge-World 0.1 Preview, a preview foundational world model for real-time interactive world simulation. The system adapts the LongLive 1 autoregressive
video stack, itself derived from Wan2.1-T2V-1.3B, with a residual action pathway inspired by the
Matrix-Game family. DreamForge-World 0.1 Preview focuses on a complementary axis to frontier-scale world simulators: low-compute adaptation, consumer-GPU runtime, and broad interactive capability coverage. It supports live keyboard and mouse control,
multimodal initialization, mid-stream reprompting, dual-view operation, and minute-scale interactive rollouts at native 480p resolution, reaching up to 14 to 15 FPS FPS on a single RTX 4090 with a low
memory footprint. By leveraging open
video backbones and applying targeted adaptation runs, we build the preview system with high cost-efficiency. DF-World 0.1 Preview is not yet a
memory-complete or frontier-quality world simulator, but demonstrates a practical low-compute route toward real-time controllable world-model previews on consumer GPUs.
Read the full abstract ▾
Computer Vision
VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context
Xiaoqian Shen and Mohamed Elhoseiny
Large
Vision Language Models (LVLMs) have achieved remarkable success on
vision-
language tasks, yet fine-grained perception over high-resolution images and long-context
videos remains challenging. As the number of visual
tokens increases, the visual
attention sink phenomenon becomes increasingly severe, causing irrelevant
tokens to absorb a disproportionate amount of
attention mass. Recent approaches attempt to mitigate this issue by explicitly predicting bounding boxes or temporal spans and re-encoding the cropped visual regions. Such methods depend on unreliable numeric localization in the discrete
token space and incur significant computational overhead due to additional forward passes. In this work, we propose **VisReflect**, a simple yet effective framework that improves fine-grained perception in long visual contexts through latent visual reflection. Instead of decoding intermediate predictions into discrete
tokens, the model generates continuous visual reflection that represents question-relevant visual features in the latent space. These reflections selectively emphasize salient regions or frames, guiding
attention towards relevant visual
tokens within a single forward pass. We conduct comprehensive evaluations on challenging high-resolution image
benchmarks, including BLINK, V*, and HRBench-4K/8K, as well as
video understanding
benchmarks such as MVBench, VideoMME, and MLVU. Our method consistently improves over strong baselines, achieving gains of 4.1% on image
benchmarks and 1.8% on
video benchmarks. Compared with zooming-based methods, our model achieves comparable performance while reducing inference time by roughly 44% on
video understanding.
Read the full abstract ▾
Computer Vision
Intermediate Text Representation Guided Text-to-Image Generation for Enhancing One-and-Only Alignment
Soyoun Won, Aryan Yazdan Parast, Basim Azam et al. (+2 more)
Text-to-image (T2I)
diffusion models often fail to faithfully render explicit textual descriptions, instead defaulting to strongly learned visual priors due to a phenomenon referred to as concept association bias. We show that such bias is particularly strong for one-and-only (OAO) objects, entities that exist in a single canonical form, such as celestial bodies, landmarks, and artworks. The deeply ingrained visual identity for these concepts often resists modification through prompting alone. Addressing this challenge, we first identify through an information-theoretic analysis that the final text
embedding discards concept-level information present in the intermediate-layer text representations, reducing the mutual information available to the subsequent denoising process. We then propose Intermediate Text Representation (IR)-guided
diffusion, which injects intermediate hidden states of the text encoder into the conditioning signal during early denoising steps, recovering suppressed concepts without any additional training, optimization, or external models. To systematically evaluate the challenging task of aligning generative outputs with unusual
prompts for OAO objects, we introduce OAO-AttackBench, a
benchmark comprising counterfactual
prompts that directly conflict with the core visual identity of OAO objects. Experiments on four
benchmarks, including OAO-AttackBench, show that our method achieves up to a 19.1 percentage-point improvement in VQAScore while preserving generation fidelity and human preference. Project page: https://soyoun-won.github.io/one-and-only-ir-guidance/.
Read the full abstract ▾
All authors Soyoun Won, Aryan Yazdan Parast, Basim Azam, Jean Honorio, Naveed Akhtar
Computer Vision
Your Data Manifold is Secretly a Reward Model: Shell-LCC for Text-to-Video Generation
Shihao Zhang, Yunzhi Li, Yuguang Yan et al. (+4 more)
Recent text-to-
video (T2V)
diffusion models rely heavily on auxiliary reward signals (e.g., via reward models or DPO) to align generated content with human aesthetics and improve realism. These signals, however, incur substantial computational overhead, require costly human annotations, and often yield limited improvement in fine-grained local details. In this paper, we argue that your data
manifold is secretly a reward model. By explicitly modeling the
manifold structure of high-quality Supervised Fine-Tuning (SFT) data and encouraging
video latents to lie on this
manifold, we derive dense, differentiable, and nearly cost-free reward signals that significantly improve
video quality, particularly in mitigating low-level distortions. Our modeling builds upon Local Coordinate Coding (LCC), which captures the `skeleton' of the
manifold. However, directly applying LCC suffers from mean regression, pulling latents toward the
geometric mean and losing high-frequency details. We therefore extend it to Shell Local Coordinate Coding (Shell-LCC), which models the
manifold `surface' as an isotropic shell to align with the true high-density region. Experiments demonstrate that our approach improves realism, enhances high-frequency details, reduces over-smoothing artifacts, and alleviates motion blur.
Read the full abstract ▾
All authors Shihao Zhang, Yunzhi Li, Yuguang Yan, Junzhe Zhang, Wei Zhao, Bohan Wang, Hanwang Zhang
Computer Vision
Semantic-Driven Scale and Spatial Selection for Efficient Cross-Modal Alignment in Referring Remote Sensing Image Segmentation
Kun Li, Shengxi Gui, Francesco Nex et al. (+1 more)
Referring Remote Sensing Image
Segmentation (RRSIS) seeks to localize and segment the target object or region specified by a natural
language expression in a remote sensing image. While existing RRSIS models have benefited from large-scale foundation models, they predominantly rely on full fine-tuning. These approaches are computationally intensive and may weaken the generalization ability of pre-trained models, as extensive fine-tuning on significantly smaller downstream
datasets can distort the well-structured feature representations learned during large-scale pre-training. Although Parameter-Efficient Tuning (PET) offers a potential alternative, existing PET frameworks primarily focus on single-modal optimization, failing to capture the complex cross-modal dependencies required for
multimodal reasoning, while simultaneously struggling to bridge the substantial domain gap between natural scenes and aerial imagery. To address these limitations, we propose a novel framework, Semantic-driven Scale and Spatial Selection for Efficient Cross-modal
Alignment (S4ECA), which enables effective and efficient cross-modal interaction through parameter-efficient adaptation. Specifically, we design a dual-encoder adapter architecture. The textual adapter employs learnable queries to distill highly semantic
language proxies from word-level
embeddings, facilitating early grounding. Simultaneously, the visual adapter refines hierarchical feature representations through a multi-scale dense extractor, followed by a
language-guided scale and spatial selection mechanism that dynamically emphasizes relevant visual contexts, ensuring precise cross-modal
alignment. By updating only 2.4% of the backbone parameters, our proposed model achieves state-of-the-art performance on the RRSIS-D and RefSegRS
datasets, demonstrating superior efficiency and precision in complex aerial scenarios.
Read the full abstract ▾
All authors Kun Li, Shengxi Gui, Francesco Nex, Michael Ying Yang
Computer Vision
From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA
Sena Korkut, María Alejandra Bravo Sarmiento, Sanghwan Kim et al. (+1 more)
High
benchmark accuracy does not guarantee genuine use of visual evidence. We study this problem in traffic accident
Video Question Answering (VideoQA), where correct answers should depend on scene-specific visual evidence but may instead be inferred from textual shortcuts. Through an audit of four public
benchmarks, we find that several recent open-weight
Vision-
Language Models (VLMs) perform competitively, and sometimes better, without
video input. On the MM-AU
benchmark, removing
video consistently improves accuracy, and adding more frames further degrades performance. To quantify visual dependence, we introduce two
dataset-level diagnostics: Blind Gap, measuring above-chance text-only performance, and Visual Gain, measuring the marginal benefit of adding
video. We further propose an instance-level Shortcut Score that combines text-only confidence with visual necessity signals, enabling continuous, training-free filtering of shortcut-prone questions. The resulting subsets reduce shortcut bias and improve visual grounding. Our findings reveal large differences in grounding quality across
benchmarks and show that visually grounded evaluation, not just high accuracy, is essential in safety-critical VideoQA.
Read the full abstract ▾
All authors Sena Korkut, María Alejandra Bravo Sarmiento, Sanghwan Kim, Zeynep Akata