Computer Vision · Cross-listed from cs.RO
DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning
Mengqi Zhang, Sahil Khose, Simar Kareer et al. (+3 more)
Generalizable
robot manipulation requires policies that can anticipate how visual scenes evolve while executing
language instructions. While recent
Vision-
Language-Action models benefit from large-scale pretraining, their predominantly static pretraining objectives provide limited supervision for physical dynamics and temporal causality, leaving control-relevant knowledge to be learned from downstream
robot demonstrations.
Video generative models offer a promising foundation by encoding rich spatiotemporal priors through future predictions. However, existing
Video-Action Models either couple
video and action prediction in a shared backbone, making policy adaptation harder to optimize, or under-utilize
video information when guiding the action branch. In this work, we introduce DeVA, a Decoupled
Video-Action model with specialized
video and action experts, multi-level feature transfer, and physically salient guidance. DeVA transfers representations from multiple
video layers to the action expert, enabling rich information exchange while making policy learning more tractable. It further supervises intermediate
video features and the action stream with physically salient guidance (affordance/depth). Experiments on both simulation
benchmarks and real-world deployment demonstrate strong performance with limited data, faster convergence than a unified architecture, and clear performance gains from physical guidance.
Read the full abstract ▾
All authors Mengqi Zhang, Sahil Khose, Simar Kareer, Yuchen Song, Unnat Jain, Judy Hoffman
Computer Vision
UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
Zhipeng Bao, Zhen Zhu, Nupur Kumari et al. (+4 more)
Modern computer
vision pipelines remain fragmented, with tasks such as text-to-image generation, editing, restoration, and classical perception handled by separate models. We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified
multimodal interface. While
diffusion-based systems dominate UVG due to strong quality and controllability, their iterative sampling incurs substantial inference latency, limiting practical deployment. To address these limitations, we propose UniGen-AR, a framework that pairs a general-purpose multi-modal
language model (MLLM) with an efficient next-scale visual auto-regressive (VAR) decoder. This design retains the flexibility of MLLM-based conditioning while leveraging the sampling efficiency and latent unification properties of VAR models. In our framework, the MLLM encodes free-form instructions and control signals into a unified sequence, which guides the VAR decoder to generate image-valued outputs for over 15 tasks spanning four families. Empirically, UniGen-AR achieves up to $19 \times$ lower inference latency than
diffusion-based baselines while maintaining or improving output quality. Our ablations further reveal that VQ-VAE tokenizer design, particularly codebook size and hierarchy, is a critical factor for VAR scalability in UVG. These results establish visual auto-regressive modeling as a compelling and efficient backbone for unified visual generation. Our project page is at https://zpbao.github.io/projects/unigenar.
Read the full abstract ▾
All authors Zhipeng Bao, Zhen Zhu, Nupur Kumari, Anurag Bagchi, Yu-Xiong Wang, Pavel Tokmakov, Martial Hebert
Computer Vision
Image Inpainting via Stochastic Dynamics
Jiaqi Kuang, Zihao Guo and Zhongmin Qian
Image inpainting aims to recover missing regions while preserving structural consistency. We propose a non-parametric method without network training based on data-guided
stochastic dynamics. Starting from a masked image, the missing pixels are evolved through a reverse-time
stochastic differential equation with a kernel-weighted correction estimated directly from a reference
dataset. This empirical correction guides the reconstruction toward high-density regions of the data distribution without training a neural network or fitting a parametric density model. Experiments on MNIST, Fashion-MNIST, and MVTec show that the proposed method outperforms Mean Fill, Telea, and Navier-Stokes inpainting in PSNR, SSIM, and visual quality. On CelebA, it remains competitive and produces plausible completions for structure-sensitive occlusions. These results demonstrate the effectiveness of empirical reference statistics as a non-parametric prior for image inpainting.
Read the full abstract ▾
Computer Vision
LoTA-N2N: Local Trace Adaptation for Zero-Shot Self-Supervised Image Denoising
Jintong Hu, Bin Xia, Junlin Liu et al. (+2 more)
Single-image self-supervised denoising replaces unavailable clean targets with surrogate targets constructed from noisy observations. Its effectiveness therefore depends on how closely the surrogate objective remains aligned with supervised denoising, especially when noise is correlated, spatially nonstationary, or unknown. We express the discrepancy between a broad class of MSE-based self-supervised objectives and supervised MSE as a parameter-independent constant and a trace interaction between the surrogate-target residual and the prediction error. The corresponding gradient discrepancy is determined by the gradient of this interaction. This formulation provides a common view of paired-noise, blind-spot, weak-noise, re-corruption, and sub-image methods, while revealing that a small global interaction may conceal substantial positive and negative regional interactions through spatial cancellation. Building on these observations, we propose LoTA-N2N, a two-stage zero-shot adaptation framework. Stage 1 trains a denoiser on complementary sub-image pairs and freezes it to construct detached clean-sub-image proxies. Stage 2 estimates the residual–prediction interaction using these proxies and suppresses its patch-wise absolute magnitude. We show that the local construction prevents spatial cancellation and upper-bounds the magnitude of the corresponding global interaction. Experiments across natural, confocal, and X-ray images, complemented by iteration-matched controls, controlled noise shifts, and gradient diagnostics, show consistent gains over MSE-only adaptation under IID, spatially varying, and mixed noise. Overall, LoTA-N2N demonstrates that estimated local interaction and spatial cancellation control provide effective design principles for single-image self-supervised denoising without paired clean targets, repeated acquisitions, or a predefined re-corruption model.
Read the full abstract ▾
All authors Jintong Hu, Bin Xia, Junlin Liu, Jiayue Liu, Wenming Yang
Computer Vision
ViDS: Video Diffusion Shader using 3D Face Tracking
Wenbo Ji, Davide Davoli, Zhe Chen et al. (+3 more)
We introduce ViDS, a
Video Diffusion Shader that leverages 3D face tracking for expressive and identity-preserving portrait animation. We first reconstruct the identity-specific 3DMM mesh from the reference image, and then animate it using expression and pose parameters from a driving
video. Leveraging dense
geometric cues from 3DMM normal maps, we employ a
video diffusion model as a neural shader to synthesize lifelike portrait animations while preserving the appearance and identity of the reference image. We find that more accurate 3DMM tracking enables finer-grained expression control. We also introduce an autoregressive
diffusion sampling process that extends generation beyond the model's native window while reducing discontinuities between adjacent clips. Compared with prior
diffusion-based approaches for portrait animation that rely on landmark-based conditioning or implicit motion latents, our method achieves more detailed and consistent expression and pose control while faithfully preserving identity and appearance. Detailed ablation studies validate the effectiveness of our design choices. Project page: https://fusheng-ji.github.io/ViDS/
Read the full abstract ▾
All authors Wenbo Ji, Davide Davoli, Zhe Chen, Liam Schoneveld, Matthias Nießner, Jiapeng Tang
Computer Vision
BeyondFusion: Self-Aligned Latent Diffusion for Calibration-Free Infrared Super-Resolution and Infrared-Visible Fusion
Minchong Chen, Xiaoyun Yuan, Minyu Cao et al. (+4 more)
Mobile infrared-visible imaging typically pairs a compact infrared sensor with a high-resolution visible camera for complementary perception. While cross-sensor misalignment caused by different optics, viewpoints, fields of view, and exposure timings hinders practical deployment. In this paper, we propose BeyondFusion, a unified latent
diffusion framework for calibration-free visible-guided infrared super-resolution and infrared-visible fusion tasks. The proposed framework supports both task-specific training and joint training where two tasks are optimized and executed as two readouts of the same generative process. Instead of relying on explicit registration or
geometric warping, BeyondFusion introduces a cross-modal self-aligning (CMSA) module into the denoising U-Net. CMSA reorganizes infrared and visible latent
tokens into a shared
attention space to learn content-adaptive cross-modal correspondence during the denoising process. Together with misalignment augmentation module, the model is facilitated to exploit visible structural and semantic cues while preserving thermal consistency, enabling high-frequency infrared reconstruction and informative fused-image generation under uncalibrated conditions. Extensive experiments on public
benchmarks and a mobile infrared-visible imaging system show strong performance across aligned inputs, low-resolution infrared observations, synthetic misalignments, and real mobile captures with unsynchronized sensors. Ablation studies, unified training analysis, and downstream pedestrian detection further validate the effectiveness of BeyondFusion for calibration-free
multimodal imaging.
Read the full abstract ▾
All authors Minchong Chen, Xiaoyun Yuan, Minyu Cao, Jianing Zhang, Jun Zhang, Shuyang Liu, Xiaokang Yang
Computer Vision
LU-500: A Logo Benchmark for Concept Unlearning
Keyu Li, Jin Gao, Jialing Zhang et al. (+1 more)
Concept unlearning is increasingly used to limit the reproduction of protected or unsafe visual concepts in text-to-image models. Existing evaluations, however, mostly study targets that dominate the whole image, such as styles, broad object categories, or portrait-like identities, leaving company logos comparatively underexamined. Logos create a different failure mode: a small localized mark can carry the entire protected concept, must be visually precise to remain recognizable, and can be triggered implicitly by products, storefronts, packaging, or advertisements even when the word “logo” is absent. We introduce LU-500, a logo-unlearning
benchmark built from Fortune Global 500 companies to study this localized and semantically entangled setting. LU-500 contains nearly 10,000 curated text-query and logo-image pairs, with an explicit track (LUex-500) and an implicit contextual track (LUim-500). To avoid reducing the task to a binary detector score, we define a multi-grained protocol that evaluates both local logo removal and global image preservation in pixel and latent spaces. Experiments on representative inference-time methods, including NP, SLD, and SEGA, and compatible fine-tuning-based methods such as ESD and Forget-Me-Not, show that the evaluated methods struggle to remove logo evidence without changing non-target content. We further analyze ProLU, a
prompt-space multi-
agent baseline: it improves local erasure by removing logo-inducing semantics, but also illustrates why
prompt filtering is not a substitute for weight-level disentanglement. Correlation analyses over logo area, location, and structural complexity suggest that future logo unlearning may need spatially aware controls, such as SSIM-guided constraints, rather than purely global concept suppression.
Read the full abstract ▾
All authors Keyu Li, Jin Gao, Jialing Zhang, Dequan Wang
Computer Vision
ReflexTrack: A Feedback-Driven Agent for Training-Free Referring Video Object Segmentation
Yuanjia Li, Tianyang Xu, Tao Zhou et al. (+3 more)
Referring
video object
segmentation (RVOS) requires segmenting a target specified by natural
language throughout a
video. Recent agentic approaches combine
multimodal large
language models with promptable
segmentation models to perform RVOS without task-specific training. However, most pipelines rely on one-shot spatial grounding followed by mask propagation, leaving both the initial
prompts and temporal predictions largely unverified. We introduce ReflexTrack, a training-free, feedback-driven
agent that closes this loop at both spatial and temporal levels. Mask-guided Spatial Refinement evaluates the mask induced by the current keyframe
prompt and iteratively updates the bounding box together with positive and negative points, yielding a more reliable initialization.
Video-level Mask Reflection assesses the complete mask sequence, localizes unreliable intervals, selects complementary repair keyframes, and generates candidate predictions through mask-guided re-propagation. Only candidates that provide a verified improvement are used to update the affected intervals, preserving reliable predictions elsewhere. All components remain frozen during inference. ReflexTrack achieves an overall $\mathcal{Q}$ score of $69.7$ on Ref-VPS and a $\mathcal{J}\&\mathcal{F}$ score of $67.2$ on ReasonVOS. These results demonstrate that prediction-level feedback substantially improves the reliability of training-free RVOS.
Read the full abstract ▾
All authors Yuanjia Li, Tianyang Xu, Tao Zhou, Zhangyong Tang, Xiao-Jun Wu, Josef Kittler
Computer Vision
Cascade Forgery Mining Network for Fingerprint Presentation Attack Detection
Hongyan Fei, Chuanwei Huang, Zheng Wang et al. (+3 more)
Fingerprint Presentation Attack Detection (PAD) is a critical component of fingerprint identification systems, serving as a protective measure against unauthorized access. In this paper, we observe that different regions of a fingerprint image can exhibit varying Artifact Extraction Difficulty (AED), with high-AED regions requiring more sophisticated extraction mechanisms to capture more subtle discriminative evidence. To address this issue, we propose to quantify AED using local Gabor feature certainty and partition fingerprint images into multiple regions based on their respective AED values. We then propose an AED guided Cascade Forgery Mining Network (CFM-Net) that employs an adaptive-depth feature extraction architecture to detect more precise and comprehensive artifact evidence across regions with heterogeneous AED values. Furthermore, we introduce an Orientation Guided
Adversarial Training (OGAT) module to filter out identity information from PAD features while preserving the integrity of original artifact evidence. Experimental evaluations on LivDet
datasets demonstrate the superior performance of our approach compared to state-of-the-art methods and achieve significant improvement in the classification ability of high AED fingerprints.
Read the full abstract ▾
All authors Hongyan Fei, Chuanwei Huang, Zheng Wang, Pengcheng Luo, Jingwei Li, Jufu Feng
Computer Vision
When Low CER is Not Enough: An Analysis of Hallucinations in Vision-Language OCR Systems on Historical Uruguayan Documents
Marina Gardella, Camilo Mariño, Diego Belzarena et al. (+3 more)
Optical Character Recognition (OCR) is a key component in the digitization of historical archives. Recently,
Vision-
Language Models (VLMs) have emerged as strong alternatives to traditional OCR systems, achieving state-of-the-art performance on standard
benchmarks. However, their suitability for archival transcription remains insufficiently understood. In this work, we
benchmark traditional OCR systems and VLM-based approaches on the Berrutti
dataset, a challenging collection of Uruguayan dictatorship-era documents derived from microfilm scans. While VLMs consistently outperform traditional methods in terms of Character Error Rate (CER) and Word Error Rate (WER), we show that these improvements hide a more complex picture. Through a detailed qualitative analysis, we uncover systematic failure modes that are invisible to standard metrics, including orthographic normalization, spurious content generation, and semantic substitutions that preserve fluency while altering meaning. Errors affecting named entities are particularly critical, as they can introduce substantial semantic distortions with minimal impact on CER and WER. These findings reveal a critical gap between quantitative OCR performance and transcription fidelity in real-world archival settings, and highlight the need for evaluation frameworks that go beyond character-level accuracy to capture the semantic reliability of generated transcriptions.
Read the full abstract ▾
All authors Marina Gardella, Camilo Mariño, Diego Belzarena, Ignacio Ramírez, Gregory Randall, Jean-Michel Morel
Computer Vision
MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning
Tuan-An To, Yuk-Kwan Wong, Tuan-Anh Vu et al. (+2 more)
Recent
Vision-
Language Models (VLMs) have achieved remarkable success in visual understanding, driven by the growing availability of high-quality image-text pairs. However, the performance of VLMs often degrades in the
video domain due to the essential need for temporal understanding and the scarcity of large-scale annotated
video data. In this work, we focus on marine
video understanding, which brings further challenges: first, it requires substantial domain expertise; and
video VLMs usually struggle with localizing and interpreting critical information from marine
videos, as the informative events are typically sparse, unpredictable, and unevenly distributed. To address these challenges, we carefully curate the first event-centric marine
video understanding
dataset called MarineEVT, which features 20K multi-task,
video-level visual question-answering pairs spanning multiple dimensions of marine understanding and analysis. Meanwhile, based on MarineEVT, we decompose marine
video understanding as an Event-centric Visual Tool-integrated
Reasoning process EVT-R1 for short, where we leverage powerful visual tools to drive the model to localize and interpret critical information aligned with visual questions and human intent. To demonstrate its effectiveness, we compare EVT-R1 against 11 SOTA VLMs in different settings. EVT-R1 outperforms the top open-source and top commercial models by 5.22 and 11.09, respectively. MarineEVT and EVT-R1 lay the foundation for ecological discovery and marine education, fostering the development of VLMs capable of interpreting marine dynamics,
reasoning about ecological interactions, and supporting sustainable ocean
video understanding and analysis.
Read the full abstract ▾
All authors Tuan-An To, Yuk-Kwan Wong, Tuan-Anh Vu, Ziqiang Zheng, Sai-Kit Yeung
Computer Vision
PointCHR: Point Cloud Analysis via Curvature-Aware Hyperbolic Rectification
Xinxing Yu, Liying Yang, Hao Mo et al. (+4 more)
High-
curvature regions in 3D point clouds encapsulate critical fine-grained
geometric semantics yet exhibit a distinct long-tail sparsity in their spatial distribution. The inherent limitations of
polynomial volume growth in Euclidean space frequently render these intricate
geometric features challenging to adequately resolve within a uniform-scale feature space. Consequently, these regions are frequently overshadowed by smooth global features dominated by low-
curvature regions, thereby limiting the discriminative capacity of the network. To address this issue, we propose PointCHR, a
curvature-aware
hyperbolic rectification (CHR) for point cloud analysis. Utilising the property of exponential volume expansion in the vicinity of
hyperbolic manifolds, CHR presents a learnable
curvature-guided radial rectification mechanism. By adaptively projecting high-
curvature points towards boundary regions endowed with larger effective
embedding capacities, PointCHR effectively mitigates the representation crowding problem inherent in Euclidean settings. Extensive experimentation has demonstrated that PointCHR significantly enhances the ability of backbone to capture fine-grained
geometric details, achieving state-of-the-art performance across multiple
benchmarks.
Read the full abstract ▾
All authors Xinxing Yu, Liying Yang, Hao Mo, Hui Ma, Fang Kai, Ajian Liu, Yanyan Liang
Computer Vision
Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification
Haopeng Li, Yitong Li, Junsong Chen et al. (+8 more)
Diffusion transformers are essential for high-fidelity
video generation, but long
token sequences make
attention a dominant inference bottleneck. Training-free dynamic sparse
attention alleviates this bottleneck by computing only selected key-value blocks, yet existing methods struggle to sparsify
attention both efficiently and accurately for two reasons: (1) Rigid, unpredictable, and costly routing: selecting a fixed fraction of top-ranked blocks by proxy score imposes fixed budgets, whereas retaining blocks to reach a target cumulative proxy probability mass yields dynamic but potentially imbalanced budgets; both incur non-negligible overhead from computing and materializing proxy scores. (2) Lossy keep-or-drop sparsification: unselected blocks are discarded entirely, degrading accuracy under aggressive sparsity. These limitations motivate cheaper dynamic-budget routing while limiting accuracy degradation. In this paper, we introduce training-free Sol-Attn (Sparsifying online
attention), which unifies dynamic routing, sparse computation, and approximation correction in a single online-softmax pass, achieving a better accuracy-efficiency trade-off in sparse
attention. The core of Sol-Attn is on-the-fly block thresholding with proxy-score reuse, which selects critical blocks by comparing block proxy scores against a threshold during online softmax. This design enables dynamic yet controllable block budgets without materializing the proxy map, while directly reusing the proxy scores of unselected blocks to approximate their contribution. Experiments across image and
video generation tasks show that Sol-Attn advances the quality-efficiency frontier of training-free sparse
attention, delivering 2.1 times and 2.3 times end-to-end speedups for
video generation and editing, respectively, while preserving visual quality.
Read the full abstract ▾
All authors Haopeng Li, Yitong Li, Junsong Chen, Tian Ye, Haozhe Liu, Jincheng Yu, Duomin Wang, Ruihua Zhang, Zeke Xie, Enze Xie, Song Han
Computer Vision
GeoStereo: A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal
Qizhe Wei, Xianda Guo, Shaocong Xu et al. (+3 more)
Stereo matching and surface normal estimation are fundamental tasks in 3D
vision. However, existing feed-forward stereo methods still struggle to produce reliable predictions in challenging regions, mainly due to the lack of strong
geometric priors. In this paper, we propose $\textbf{GeoStereo}$, a unified stereo geometry estimation framework that leverages powerful
diffusion priors to jointly predict disparity and surface normals. Specifically, GeoStereo couples a feed-forward stereo matching pipeline with a
diffusion-based normal estimation branch. To enable effective interaction between the two tasks, we introduce a disparity to normal initialization strategy and construct a warp to left-view condition for the
diffusion process. This coupled design allows the
diffusion branch to provide strong structural priors that enhance disparity estimation in ill-posed regions, while the feed-forward branch offers reliable
geometric guidance for accurate normal prediction. Extensive experiments show that GeoStereo performs reliably in challenging scenarios, including low-light environments, highly reflective surfaces, and transparent objects. Under zero-shot settings, it achieves Rank-1 disparity estimation on multiple
benchmarks, including KITTI and NYUv2, and delivers the best normal estimation accuracy on many real indoor
benchmarks, such as iBims-1 and ScanNet. Project page: https://qz-wei.github.io/GeoStereo.github.io/
Read the full abstract ▾
All authors Qizhe Wei, Xianda Guo, Shaocong Xu, Hong Li, Runyi Yang, Hao Zhao
Computer Vision
Disentangling Semantic Attention from Structural Bias in the Attention Manifold
Pengkun Jiao, Bin Zhu, Jingjing Chen et al. (+1 more)
The empirical success of
attention mechanism in
Multimodal Large
Language Models (MLLMs) often obscures its inherent, subtle flaws. Specifically, MLLMs consistently exhibit disproportionate
attention toward certain semantically uninformative visual
tokens, a phenomenon termed "register" or "Visual
Attention Sinks." While existing inference intervention methods attempt to identify these sink
tokens and redistribute their
attention weights, such approaches typically treat these
tokens in isolation and suffer from computational inefficiency. Instead, we reframe this phenomenon as a generalized textual bias exerted over visual features that extends beyond isolated sink
tokens. From this perspective, a pervasive structural bias leads to the dilution of the semantic visual signal, precipitating
multimodal hallucinations as the model prioritizes linguistic priors over valid visual evidence. To address this limitation, we introduce Saliency-guided Purification and Adaptive Redistribution (SPAR), a training-free, plug-and-play intervention. SPAR mitigates this generalized textual bias by purifying structural noise and subsequently redistributing the reclaimed
attention budget to the most informative visual regions. Comprehensive evaluations across a diverse spectrum of
hallucination benchmarks demonstrate that SPAR effectively restores authentic visual grounding with negligible computational overhead.
Read the full abstract ▾
All authors Pengkun Jiao, Bin Zhu, Jingjing Chen, Yu-gang Jiang
Computer Vision
DailyBench: A Unified Benchmark for AI-Generated and Manipulated Images from Modern Generative Models
Xin Jiang, Hao Tang, Junyao Gao et al. (+6 more)
Recent advances in generative models have shifted AI-generated image detection from identifying easily distinguishable, fully synthetic images to identifying highly realistic content generated by both modern generation and manipulation pipelines. However, existing detection
benchmarks are often built with outdated generative models and primarily emphasize full-image synthesis, creating a growing mismatch between
benchmark data and the images encountered in real-world generation and editing scenarios. To bridge this gap, we introduce DailyBench, a high-quality unified
benchmark for evaluating whether AI-generated image detectors can generalize across both modern full-image synthesis and object-level manipulation. DailyBench contains two complementary subsets: FakeBench, which includes high-quality images synthesized by recent open-source and commercial generative models, and ManipulationBench, which introduces challenging object-level edits applied to real images using advanced image-conditional models. This design makes DailyBench a realistic testbed for studying both generator-level generalization and manipulation-aware detection under subtle local edits. Experiments on DailyBench reveal substantial robustness gaps in current detectors: methods reporting 91-96% balanced accuracy on GenImage drop to 52-79% on FakeBench and 43-67% on ManipulationBench. These results show that existing detectors remain poorly generalized to realistic synthesis and manipulation, highlighting DailyBench as a rigorous testbed for developing robust and manipulation-aware AI-generated image detection methods. The project is available at https://dailybench.github.io/
Read the full abstract ▾
All authors Xin Jiang, Hao Tang, Junyao Gao, Zijie Yang, Meiqi Cao, Fei Shen, Dongming Zhang, Zechao Li, Yongdong Zhang
Computer Vision
AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars
Hengyuan Zhang, Jingna Sun, Meiguang Jin et al. (+1 more)
Production-ready audio-driven avatar generation requires efficient inference without sacrificing fidelity or motion expressiveness. However, existing acceleration methods often compromise quality through restrictive architectural choices, such as causal
attention and short temporal horizons, or by reducing model capacity and resolution. Without such compromises, we propose AptAvatar, a 14B-parameter long-form audio-driven avatar generation framework that delivers fast and expressive inference. For efficiency in production-level applications, AptAvatar addresses the extreme two-step generation challenge. To bridge the gap between the multi-step teacher model and the two-step student model, we introduce Endpoint-Anchored Distribution
Distillation. It augments vanilla distribution matching with a dedicated Anchor Score
Estimator trained on the trajectory-endpoint distribution defined from a frozen pretrained 4-step bridge generator. This provides an attainable endpoint-level anchor for the evolving two-step student. To improve long-horizon consistency, we further introduce Self-Generated History Replay, which reuses cached outputs from earlier generator checkpoints as history conditions during chunk-wise training. This approximates inference-time conditioning on self-generated histories without costly online rollouts, mitigating quality degradation from accumulated history errors. Extensive experiments demonstrate that AptAvatar generates vivid 720p long-form avatar
videos with only 2 NFEs, achieving a 60x speedup while preserving visual fidelity and long-horizon identity. Code is available at https://github.com/TaoLiveAIGC/AptAvatar
Read the full abstract ▾
All authors Hengyuan Zhang, Jingna Sun, Meiguang Jin, Junfeng Ma
Computer Vision
Leakage and Multiway Interaction Loss: What Matricization and Single-Mode Truncation Discard from Multiway Signals
Hiroki Hasegawa
Tensor methods are said to preserve structure that
matrix methods destroy, but what is lost, and why, is unclear. Relative to a multilinear reference, we split the squared error of any reconstruction exactly into coefficient distortion, multiway interaction loss, directional loss, and out-of-model leakage. Di rectional loss removes components in several modes; interaction loss removes a direction in one mode while its partners in other modes remain. Optimal truncation of a
matrix or of an orthogonally decomposable tensor incurs only directional loss, whereas for generic tensors of order three or more with balanced ranks, optimally truncating one unfolding below its rank incurs only interaction loss. Matricization is the extreme case: when the other modes have low rank, it must under resolve the processed mode at budgets at which multilinear truncation is lossless, and under noise it adds leakage scaling with the product of the unprocessed dimensions. For sources with orthonormal signatures in all modes but one, interaction loss is the energy-weighted signature distortion of the surviving sources; without noise, a distortion below a threshold keeps the periodogram estimate within its main lobe and bounds its bias. To first order, multidimensional rotational-invariance estimation sees only in-model noise and single-mode leakage, so the leakage energy removed by tensor-based subspace estimation does not predict its gain. A calibrated resolution makes the split usable under noise. On a measured hyperspectral image with nearly full-rank spatial modes, spectral principal component analysis was as storage-efficient as multilinear truncation.
Read the full abstract ▾
Computer Vision
Low-light Image Enhancement via Multi-scale Attention combined with Fourier Transform
Wenbin Du, Jian Long and Zhu Cao
Low-light image enhancement (LLIE) aims to improve image quality and clarity in diverse and demanding low-illumination environments. However, existing deep learning-based LLIE methods struggle to accurately capture real-world illumination and restore texture details, largely because their algorithmic strengths remain underutilized. To address these issues, we present a supervised frequency domain deep learning network for LLIE, named multi-scale
attention combined with the
Fourier transform (MSFT) which adopts a U-shaped, one-stage architecture that infuses guidance from low-light images into the network by channeling it through multi-scale
attention. We further fuse the amplitude information from priori channels with that of the low-light image in MSFT's self-created module, and carry out multi-scale guidance along with the network. Subsequently, to better enhance the faint feature, such as fine content and textures, and to better fuse global context confidence in the decoding stage, we separately introduce a multi-shape synergistic
attention and a lightweight network that effectively integrate information in high-dimensional space to embed into the superlative feature space channel containing rich texture information. Extensive experiments conducted on LOL, SID, SMID, and SDSD
datasets demonstrate that MSFT significantly outperforms state-of-the-art competitors. For example, compared with Retinexformer, our method achieves a peak signal-to-noise ratio of up to 41.76 decibels on the SDSD-outdoor
dataset with an increase of 11.92 decibels and a structural similarity index of 0.988 with a 13.80% improvement.
Read the full abstract ▾
Computer Vision
Effective Receptive Field Ordering Matters for Infrared Small Target Detection
Guoyi Zhang, Yanjin Du, Zhengyao Zhao et al. (+6 more)
In this work, we investigate a previously unexplored architectural dimension for infrared small target detection: the organization of effective receptive fields (ERFs) during feature refinement. Unlike existing approaches that primarily improve individual feature operators, we argue that ERF organization constitutes an architectural dimension independent of receptive field design itself, and formulate deep feature transformation as a progressive residual correction process, from which a theoretical framework for ERF scheduling is established. Specifically, we reveal that ERF refinement is governed by two fundamental properties: scale-frequency correspondence, which aligns different ERF scales with distinct residual frequency characteristics, and nonlinear non-commutativity, which makes different ERF orderings produce fundamentally different refinement trajectories. Together, these properties show that ERF organization, rather than ERF scale alone, governs refinement dynamics. Guided by these principles, we propose Receptive Field Ordering Network (RFONet), which realizes hierarchical ERF scheduling through a multigrid-inspired V-cycle strategy using only standard $3\times3$ convolutions. RFONet achieves state-of-the-art performance on multiple
benchmarks with only 1.16M parameters and over 157 FPS inference speed. Beyond empirical performance, our theoretical analysis provides theoretical guarantees for stable residual refinement under perturbations, frequency shifts, and partial occlusions, which are consistently reflected in superior noise robustness and cross-
dataset generalization. Finally, our framework reformulates ERF organization as a task-dependent optimization objective, providing a principled foundation for future adaptive receptive field scheduling.
Read the full abstract ▾
All authors Guoyi Zhang, Yanjin Du, Zhengyao Zhao, Tongsu Zhang, Guangsheng Xu, Siyang Chen, Xiangpeng Xu, Han Wang, Xiaohu Zhang
Computer Vision
Multimodal Semantic-Probabilistic Objectness for Open World Object Detection
Weijun Tian and Rui Liu
Open-world object detection (OWOD) requires a detector to recognize known categories, discover unnamed objects from unseen categories, and incrementally learn newly annotated classes. PROB improves unknown discovery by modeling class-agnostic probabilistic objectness in the decoder-query space. However, visual objectness alone cannot determine whether an object-like query corresponds to a hard known instance, an unseen-category object, or background clutter, resulting in an ambiguous known-unknown decision boundary. We propose MSPO, a lightweight semantic calibration framework that augments PROB with task-aware known-category
language priors while preserving its detector architecture and incremental learning protocol. For each currently known category, MSPO constructs an extended text description covering category attributes, visual appearance, typical scenes, and functional usage, and encodes it using a frozen CLIP text encoder. Decoder query features are projected into the same semantic space to estimate their support from the current known-category semantics. This semantic evidence is fused with PROB's visual objectness to calibrate known and unknown predictions without turning OWOD into open-vocabulary classification. Importantly, MSPO never uses future-category names, and all unseen categories remain unnamed during evaluation. Experiments on M-OWODB and S-OWODB show that MSPO improves the strong PROB baseline on the main aggregate metrics while retaining competitive unknown recall. It also improves early unknown-confusion metrics and raises PASCAL VOC final mAP by up to 2.7 points. These results demonstrate that known-category
language semantics provide an effective calibration signal for probabilistic objectness under the standard OWOD setting.
Read the full abstract ▾
Computer Vision
Mutual Modality Trust with Lightweight Reconstruction Regularization for Fine-grained Tire Pattern Recognition
Jianning Yang, Jie Fang, Xinda Ma et al. (+3 more)
Visual tire recognition serves as a core supporting technique for vehicle safety monitoring, autonomous driving perception and automated automotive maintenance. Existing fine-grained tire recognition techniques suffer from three prominent limitations. They tend to depend on only one visual source, lack the capacity to jointly model spatial and frequency cues for minute tread texture extraction, and suffer severe overfitting given limited annotated tire imagery. This paper proposes a lightweight fine-grained tire pattern recognition method incorporating dual-branch independent inference and enhanced feature fusion to boost recognition performance. The framework employs two task-specialized branches dedicated to tire surface and tread indentation, respectively, to extract modality-specific discriminative features. Each branch conducts independent prediction, while cross-branch feature fusion exploits Mutual Modality Trust (M$^2$T) to realize complementary feature enhancement across two modalities. Besides, a frequency-domain hierarchical guidance module is devised, which leverages bandpass filters to decompose feature maps into high- and low-frequency components and enables fine-grained cross-layer feature modulation. Furthermore, a Lightweight Reconstruction Regularization (LR$^2$) is introduced to retain abundant intrinsic information within feature
embeddings, substantially improving feature stability and recognition robustness under limited labeled training data. In addition, we establish a surface-indentation multi-source
dataset namely MTire299 for fine-grained tire tread recognition, which covers 299 categories with a total of 14795 paired image samples. Extensive experiments conducted on two public tire
datasets validate the superiority and efficacy of the proposed
algorithm.
Read the full abstract ▾
All authors Jianning Yang, Jie Fang, Xinda Ma, Zirui Song, Dianwei Wang, Nan Wang
Computer Vision
Color Fundus Photography Analysis: Co-evolution of Data, Preprocessing, and Modeling toward Multimodal AI
Yu Li, Wengan He, Wenhui Xu et al. (+7 more)
Color Fundus Photography (CFP) is a primary non-invasive imaging modality for large-scale screening of ophthalmic and systemic diseases. Existing surveys mainly summarize task-specific
algorithms,
datasets, or preprocessing techniques independently, lacking a unified perspective on their co-
evolution with modern artificial intelligence. This review provides an integrated overview of CFP AI through the interplay of
dataset evolution, preprocessing paradigms, and modeling frameworks. We show that CFP
datasets have evolved from small single-center collections with task-specific labels to large multi-center resources featuring
multimodal pairings and longitudinal clinical records. Preprocessing has progressed from conventional image enhancement to neural data-engineering pipelines, hardware-aware
token optimization, and self-supervised imputation for incomplete electronic health records (EHRs). Meanwhile, modeling has advanced from convolutional neural networks (CNNs) to
vision foundation models, state space models (SSMs), and
multimodal expert architectures. At the
multimodal frontier, CFP is increasingly integrated with EHRs and longitudinal patient information, enabling more comprehensive clinical
reasoning beyond isolated image analysis. We conclude that future progress depends on the collaborative optimization of
datasets, preprocessing, and
multimodal modeling, providing a roadmap toward robust clinical deployment, improved cross-domain generalization, and resource-efficient edge intelligence.
Read the full abstract ▾
All authors Yu Li, Wengan He, Wenhui Xu, Lihong Jiang, Fan Xiao, Zhuohang Huang, Yuanzhu Liang, Jiayi Liu, Yuxi Chen, Yongsheng Luo
Computer Vision
Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigation
Mohammed Aldeen, Muhammad Sami Irfan, Sagar Dasgupta et al. (+3 more)
Autonomous vehicles (AVs) depend on Global
Navigation Satellite Systems (GNSS) for localization and
navigation, making them vulnerable to spoofing attacks that can covertly redirect vehicles or induce unsafe maneuvers. In this paper, we develop the first
Vision-
Language Model (VLM)-based framework for GNSS spoofing detection for autonomous vehicles by fusing front-camera visual data with in-vehicle sensor readings (e.g., speed, acceleration, yaw rate) against GNSS-derived maneuvers. Our approach introduces a three-stage fine-tuning process that first grounds visual cues, and then calibrates sensor data within a shared semantic space to detect discrepancies between predicted and GNSS-derived maneuvers across three attack scenarios. We also generated an independent real-world
dataset by driving an instrumented vehicle on public roads in Tuscaloosa, Alabama, equipped with time-synchronized GNSS, IMU, and camera logs to validate cross-regional generalization of our fine-tuned model on unseen data from training data. On this
dataset, we then generated intelligent spoofing attacks, including trajectory mirroring with road-network snapping for wrong-turn attacks, position freezing for overshoot scenarios, and drift generation for stop attacks. On this validation
dataset, the zero-shot VLMs baseline F1-score ranges from 23% to 32%, whereas our fine-tuned model achieves an F1-score ranging from 94% to 95%. Results show that our VLM-based approach correctly classified every wrong-turn and stop attacks, and attains 88%-93% accuracy for overshoot attacks. Furthermore, we introduce an adaptive inference policy that reduces VLM invocations to 14% (~86% computational reduction) and yields 65ms-73ms per 4s window. These results point to a practical, on-road layer of defense that complements signal-level integrity checks with the use of VLMs.
Read the full abstract ▾
All authors Mohammed Aldeen, Muhammad Sami Irfan, Sagar Dasgupta, Long Cheng, Mizanur Rahman, Mashrur Chowdhury
Computer Vision
RODR: Riemannian Orthogonally Decoupled Regularization for Disentangled Manifold Representation
Jiayu Zhu and Wenlai Zhao
Point cloud denoising is essentially a
geometric recovery task that aims to reconstruct the intrinsic structure of a smooth 2D Riemannian
manifold embedded in R^3 from noisy, discrete ambient-space samples. Despite the remarkable progress of modern
manifold-aware encoders and generative transport models in
geometric representation learning, a fundamental objective-geometry mismatch remains underexplored. Theoretically, we identified that this mismatched coupling leads to
geometric gradient interference, where conflicting optimization objectives result in structural degradation and point clustering. We introduce Riemannian Orthogonally Decoupled Regularization (RODR) to reformulate the optimization trajectory by disentangling the normal (fitting) and tangential (distribution) components. Guided by a vector-
attention and
entropy-aware adaptive strategy, RODR effectively preserves high-fidelity
geometric details while maintaining sampling uniformity. Experiments demonstrate that RODR reaches performance comparable to state-of-the-art baselines and suggests improved distribution regularity and reduced local aggregation effectively. Our work establishes a generic and interpretable framework for disentangled
geometric optimization in point cloud processing.
Read the full abstract ▾
Computer Vision
TimePLE: Rethinking Temporal Representation for Video Temporal Grounding
Yuhui Zeng, Xinyu Mao, Xiaokun Liu et al. (+4 more)
Video temporal grounding (VTG) aims to localize the continuous
video interval described by a natural-
language query. However, current VLM-based methods typically produce this interval indirectly through two endpoint outputs, represented either as discrete timestamp
tokens or continuous boundary coordinates. These formulations differ in how endpoints are encoded, but not in what is predicted: the event interval remains a derived object, while interval validity, duration, and interval-level similarity are handled only implicitly. We propose TimePLE, which reformulates VTG from endpoint prediction to interval-native grounding by predicting a single joint distribution over valid temporal intervals. TimePLE maps each interval to a point in a canonical position-duration square, where every support point corresponds to a valid span and neighboring points represent geometrically similar intervals. Given a
video and query, the VLM generates a single latent <|TIMESPAN|>
token whose hidden state is decoded into a joint interval distribution, refined through duration-aware coordinate correction, and converted into continuous boundaries. The same interval representation is used to encode input temporal anchors, aligning
video-side temporal evidence with output-side span prediction. To reliably align the latent span representation with complete event intervals, we curate 90K-scale grounded samples and human-verify 3K-scale
benchmark annotations. Experiments across four VTG
benchmarks show that TimePLE consistently outperforms endpoint prediction baselines, achieving an average mIoU of 58.9, with clear gains on short-duration and medium-duration events.
Read the full abstract ▾
All authors Yuhui Zeng, Xinyu Mao, Xiaokun Liu, Xin Tao, Jinfa Huang, Jiayi Ji, Xiawu Zheng
Computer Vision
Manifold-Constrained Noise Optimization for Diverse Diffusion Sampling
Qitan Shi, Cheng Jin, Ziyuan Liu et al. (+1 more)
Few-step distilled
diffusion models generate high-quality images quickly, but often lose per-
prompt diversity, producing near-identical samples across random seeds. Optimizing the initial noise at inference time offers an appealing way to recover this diversity, yet existing methods directly update the initial noise in an unconstrained Euclidean space, ignoring both the geometry of the Gaussian prior and the model's sensitivity to noise frequencies. They therefore introduce auxiliary quality-control objectives to maintain generation fidelity, adding compute and weighting hyperparameters while still requiring conservative updates to prevent degradation. In this work, we propose MoNO, a training-free method that performs
Manifold-constrained Noise Optimization on a low-dimensional, quality-stabilizing noise
manifold. MoNO sequentially optimizes each new initial noise so that its predicted visual feature complements previous generations, while Riemannian updates on an affine low-frequency sphere preserve prior likelihood and fix unstable high-frequency components by construction. This enables large geodesic steps, removes the need for auxiliary quality-control objectives, and converges in far fewer iterations than prior noise-optimization methods. Experiments with multiple distilled text-to-image
diffusion models show that MoNO consistently improves per-
prompt diversity while maintaining image quality.
Read the full abstract ▾
All authors Qitan Shi, Cheng Jin, Ziyuan Liu, Yuantao Gu
Computer Vision
DuoAD: Leveraging [CLS] Dual Characteristics for Training-Free Few-Shot Anomaly Detection
Jyun-Ze Tang, Po-Han Huang, Ming-Ching Chang et al. (+2 more)
Vision foundation models have enabled strong training-free anomaly detection (AD). However, most existing approaches rely primarily on independent local patch features, leaving the global contextual information encoded by
Vision Transformers (ViTs) underexploited. In this work, we identify the dual characteristics of the ViT [CLS]
token: its
embedding provides anomaly-invariant global semantic representation, while its
attention maps implicitly highlight spatially abnormal regions. Building on this observation, we propose a fully automated AD framework leveraging global context to remove manual tunings. Our framework introduces (1) an automatic augmentation selection strategy driven by [CLS]-level semantic consistency, and (2) an
attention-guided feature reweighting mechanism that dynamically adjusts patch contributions according to [CLS]
attention saliency. By integrating these components over multi-level features, our method achieves stable anomaly scoring and precise localization without training or parameter tuning. Under the one-shot setting, it achieves Image-AUC scores of 97.7%, 93.2%, and 84.5% on MVTec-AD, VisA, and Real-IAD. Using a single fixed configuration across categories, backbones, and
datasets, the method establishes a new state-of-the-art for plug-and-play, training-free anomaly detection while maintaining strong robustness and practical scalability.
Read the full abstract ▾
All authors Jyun-Ze Tang, Po-Han Huang, Ming-Ching Chang, Chih-Fan Hsu, Jeng-Lin Li
Computer Vision
DDVT: Dynamic Dual-level Vision Transformer Fusion Network for Answer Grounding in Visual Question Answering
Yue Zhang, Xiangyu Li, Wanshu Fan et al. (+2 more)
Answer grounding in visual question answering aims to locate the region from a given natural
language question associated with the visual content of an image, which has garnered significant
attention due to its practical applications. In this paper, we introduce the Dynamic Dual-level
Vision Transformer Fusion Network (DDVT) for answer grounding in visual question answering. Specifically, we propose a question-guided dynamic regional-level module (QGDR) that combines complementary image context through ROI Align and text content, enabling precise localization of text-related visual content. Moreover, we present a cross-modal multi-scale aggregation module (CMA) that enhances feature fusion between pixel-level and region-level features, facilitating the effective localization of visual content associated with grounded answers. Furthermore, we fuse the located visual content with text features to locate the region and provide answers to questions posed about the image. Experimental results demonstrate that our DDVT outperforms state-of-the-art methods on several widely-used
benchmarks.
Read the full abstract ▾
All authors Yue Zhang, Xiangyu Li, Wanshu Fan, Xin Yang, Dongsheng Zhou
Computer Vision
What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape
Qing Li, Zeyu Dong, Yin Cui et al. (+2 more)
Emotional image editing requires more than applying affective filters or modifying predefined visual factors: an effective edit must identify what a particular image can afford for a target
emotion. Existing affective image manipulation methods, including recent agentic variants, largely operate within bounded strategy spaces based on predefined factor taxonomies, knowledge libraries, or conventional editing templates, and therefore often miss image-specific, context-grounded strategies. We introduce EmoScope, a multi-
agent framework that reframes the task from "how should I edit?" to "what can I edit?" EmoScope first discovers an image-specific editable space through
emotion-conditioned affordance
reasoning, then uses a semantic hierarchy of anchors, variables, and context to balance content consistency and emotional expressiveness before executing and verifying the edit. Because its plans are expressed as image-specific affordances rather than retrieved templates, EmoScope also exposes the editing strategy as an interactive surface for user refinement at the plan level. In a large-scale human evaluation covering all eight Mikels
emotion categories, with 4,693 valid responses across 1,824 pairwise questions, participants preferred EmoScope over two competitive baselines by 88.1% on average. Attribution analysis further shows that EmoScope selects target-
emotion-adaptive strategies rather than applying a uniform template. The same affordance-level plan also supports lightweight user refinement in an interactive pilot. Finally, we show that classifier-based metrics exhibit
emotion-conditional blind spots toward non-stereotypical, context-grounded edits, and present a relative content-
emotion preference-affinity landscape showing that EmoScope's advantage varies systematically across image-
emotion combinations.
Read the full abstract ▾
All authors Qing Li, Zeyu Dong, Yin Cui, Chuan Yan, Xiaojiang Peng