Computer Vision
Lightweight Unpaired Smartphone ISP Transfer with Semantic Pseudo-Pairing
Yujin Cho, Flavien Armangeon and Yanhao Li
Unpaired smartphone ISP is a challenging problem due to the lack of scene and color
alignment between RAW and target RGB images. Many existing methods either require paired data or rely heavily on
adversarial training, which can become unstable in the unpaired setting. In this work, we present a simple and effective approach developed for the NTIRE 2026 Learned Smartphone ISP Challenge with Unpaired Data. Our method first reconstructs larger images from training patches to recover global context. Then, we extract semantic
embeddings with DINOv2, and use fused Gromov-Wasserstein (FGW) optimal transport to build pseudo pairs between RAW and RGB images at both image and patch levels. This semantic matching allows us to partially alleviate the unpairedness of the data and build these pseudo input-target pairs. Based on these pseudo pairs, we train a lightweight CNN with only 7K parameters for color rendering. The network is designed to be compact and focus on color transformation rather than structural change, which helps reduce artifacts and improve training stability. Our challenge submission achieves 22.569 PSNR, 0.675 SSIM, and 8.067 $\Delta E$ on the final hidden test set, significantly improving over the baseline and achieving the 3rd best SSIM and $\Delta E$ among all challenge entries. Our code is available at github.com/nuniniyujin/Unpaired-ISP .
Read the full abstract ▾
Computer Vision
DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
Mengxin Qin, Xiang Zhang, Xi Wang et al. (+3 more)
Continual learning enables
vision-
language models to accumulate knowledge and adapt to evolving tasks without retraining from scratch. However, in multi-domain task-incremental learning, large domain shifts intensify the stability-plasticity dilemma. Most existing methods rely on fixed architectures with statically allocated parameters, which limits adaptation to new domains and aggravates catastrophic forgetting. To address these challenges, we propose DIMoE-Adapters, a Dynamic Incremental Mixture-of-Experts Adapters framework that introduces a dynamic expert
evolution paradigm to balance stability and plasticity. This paradigm is implemented through two collaborative components: Self-Calibrated Expert
Evolution (SCEE) and Prototype-Guided Expert Selection (PGES). SCEE constructs and evolves a sparse expert pool through expert optimization dynamics, improving plasticity while reducing redundant capacity. PGES controls expert utilization based on the pool shaped by SCEE, improving stability across both previously encountered and unseen tasks. Extensive experiments show that DIMoE-Adapters outperforms previous state-of-the-art methods across various settings.
Read the full abstract ▾
All authors Mengxin Qin, Xiang Zhang, Xi Wang, Kun Wei, Xu Yang, Cheng Deng
Computer Vision
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings
Zhiheng Li, Zongyang Ma, Jiaxian Chen et al. (+12 more)
Document parsing models are improving rapidly, but existing
benchmarks are still insufficient for a comprehensive evaluation of their capabilities. We introduce PureDocBench, a
benchmark with 10 domains, 66 subcategories, and 1,475 pages. Each page has matched Clean, Digital, and Real versions, covering clean renders, simulated degradation, and real capture or sharing conditions. The resulting 4,425 images have source-derived annotations for text, formulas, tables, and reading order, checked through automated validation and human review. Keeping page content and annotations fixed allows direct comparisons across image conditions. We evaluate 58 models, including pipeline-based parsers, end-to-end parsers, and general
vision-
language models. The best three-track average score is 79.69 out of 100, and the mean across models is 63.84. Scores drop by 3.34 points on Digital and 11.61 points on Real on average, relative to Clean. TeleOCR leads on Clean, GLM-5.3-Flash on Digital, and Gemini 3.6 Flash on Real. The
benchmark dataset, source documents, generation tools, and evaluation code are publicly available at https://github.com/zhihengli-casia/PureDocBench.
Read the full abstract ▾
All authors Zhiheng Li, Zongyang Ma, Jiaxian Chen, Jianing Zhang, Zhaolong Su, Yutong Zhang, Zhiyin Yu, Ruiqi Liu, Xiaolei Lv, Bo Li, Jun Gao, Ziqi Zhang, Chunfeng Yuan, Bing Li, Weiming Hu
Computer Vision
Implicit Multi-Camera System Calibration Using Gaussian Processes
Ivan De Boi, Bart Ribbens, Veronika Golanova et al. (+2 more)
This paper proposes a novel framework for implicit multi-camera system calibration utilizing Gaussian Process (GP) regression. Conventional explicit calibration methods are constrained by rigid mathematical models and struggle with complex, non-linear distortions from unconventional optics, while existing neural network-based implicit approaches are typically data-hungry and lack inherent uncertainty quantification (UQ). Our GP-based model directly learns the complex, non-linear mapping from 2D image coordinates across all cameras to a 3D world coordinate, completely bypassing time-consuming estimation of explicit intrinsic and extrinsic parameters. Moreover, the inherent UQ is critical for transforming a simple 3D point prediction into a verifiable 3D measurement, complete with statistically-sound confidence bounds. To further enhance data efficiency and practical deployment, we integrate Active Learning (AL), which intelligently leverages the GP's predictive uncertainty to strategically guide the acquisition of new calibration data. This approach results in a robust, data-efficient, and reliable calibration solution, proving particularly effective in practical scenarios where collecting extensive calibration data is a dominant constraint. Our experiments show that the uncertainty for the 3D predictions is higher closer to the cameras. The data points in $uv$-coordinate space are more sparse in that region, even though they are not in 3D space. This work is relevant for anyone who is tasked with the calibration of complex multi-camera systems.
Read the full abstract ▾
All authors Ivan De Boi, Bart Ribbens, Veronika Golanova, Ursula Kapov, Simon Verspeek
Computer Vision
Kai Zheng, Zejian Kang, Rui Mao et al. (+4 more)
Speech-driven facial animation requires accurate correspondence between acoustic signals and facial motion, especially for articulation-related mouth movements. However, directly mapping
speech audio to facial coefficients often overlooks the linguistic and phonetic structure underlying
speech production. In this paper, we propose AudioFace, a
language-assisted framework for
speech-driven blendshape generation that treats mouth-related facial coefficient prediction as a structured generation problem guided by linguistic and articulatory information. Instead of relying solely on acoustic features, our method leverages the prior knowledge of
multimodal large
language models and introduces transcript- and phoneme-level cues to bridge
speech signals with interpretable facial actions. Extensive experiments show that AudioFace achieves superior performance across multiple evaluation metrics, validating the effectiveness of
language-assisted and
multimodal-prior-guided
speech-driven facial animation.
Read the full abstract ▾
All authors Kai Zheng, Zejian Kang, Rui Mao, Hongyuan Zou, Yuanchen Fei, Xuanyang Xu, Xiangru Huang
Computer Vision
ReasonEdit: Towards Interpretable Image Editing Evaluation via Reinforcement Learning
Honghua Chen, Zitong Xu, Huiyu Duan et al. (+3 more)
Recent text-guided image editing (TIE) models have achieved remarkable progress, however, many edited results still suffer from artifacts, unintended modifications, and suboptimal aesthetics. Although several
benchmarks and evaluation methods have been proposed, most existing approaches rely on scalar scores and lack interpretability. This limitation largely stems from the absence of high-quality interpretation
datasets for TIE and effective reward models to train interpretable evaluators. To address these challenges, we introduce ReasonEdit-22K, the first
dataset that combines 22K edited images with 113K Chain-of-Thought (CoT) samples, along with 1.3M human judgments assessing these interpretations in terms of logicality, accuracy, and usefulness. Building upon this
dataset, we propose RE-Reward, a
multimodal large
language model (MLLM)-based reward model designed to provide human-aligned feedback for evaluating interpretable
reasoning in image editing. Furthermore, we develop ReasonEdit, which is trained using reward signals derived from RE-Reward and the Group Relative Policy Optimization (GRPO)
algorithm to learn an interpretable evaluation model. Extensive experiments demonstrate that ReasonEdit achieves superior
alignment with human preferences and exhibits strong generalization across public
benchmarks. In addition, it is capable of generating high-quality interpretable evaluation text, enabling more transparent and trustworthy assessment for image editing. The code is available at https://github.com/IntMeGroup/ReasonEdit.
Read the full abstract ▾
All authors Honghua Chen, Zitong Xu, Huiyu Duan, Xinyun Zhang, Xiongkuo Min, Guangtao Zhai
Computer Vision
Yuhao Zhou, Yunpeng Zhu, Yang Zhou et al. (+7 more)
Vision-
Language-Action (VLA) models hold great promise for general-purpose robotic intelligence, yet scaling up such models is severely bottlenecked by the high cost of acquiring annotated training data. Fortunately,
vision-equipped
robots deployed across various domains already produce abundant
vision-action pairs that can be leveraged to scale up VLA training more efficiently. However, these raw data cannot be centrally aggregated due to various constraints and also exhibit severe heterogeneity. To address these challenges, in this paper, we propose ForgeVLA, a
federated VLA training framework that learns VLA models from distributed
vision-action pairs without centralizing raw data or requiring manual annotations. Specifically, each client in ForgeVLA is equipped with an
embodied instruction classifier that maps
vision-action pairs to a predefined instruction set, recovering the missing
language modality and forming complete
vision-
language-action triplets. Beyond triplet construction, we also identify
vision-
language feature collapse as a critical challenge that has been largely overlooked in prior
federated VLA research. To mitigate this issue, ForgeVLA combines a client-side contrastive planning loss with a server-side adaptive aggregation strategy to learn task-discriminative representations efficiently. Extensive experiments across multiple
benchmarks show that ForgeVLA significantly outperforms other baselines, and ablation studies further validate the contribution of each component.
Read the full abstract ▾
All authors Yuhao Zhou, Yunpeng Zhu, Yang Zhou, Jindi Lyu, Jian Lan, Zhangyuan Wang, Dan Si, Thomas Seidl, Qing Ye, Jiancheng Lyu
Computer Vision
A Unified Framework for the Detection and Classification of Fatty Pancreas in Ultrasound Images
Ioan-Tudor-Alexandru Anghel, Ciprian-Mihai Ceausescu, Elena Dana Nedelcu et al. (+5 more)
Non-alcoholic fatty pancreas disease (NAFPD) is an underdiagnosed condition associated with metabolic syndrome, insulin resistance, and increased risk of pancreatic cancer. Diagnosis typically relies on subjective visual assessment of ultrasound images by clinicians. We propose an end-to-end framework for automatically classifying normal versus fatty pancreas from abdominal ultrasound images. Our method employs a TransUNet-based
segmentation architecture with a ResNet encoder and
transformer bottleneck to delineate the pancreas and the splenic vein, followed by anatomically-guided patch extraction and patient-level classification through pairwise texture comparison. The feature engineering mimics clinical
reasoning by comparing the echogenicity of peri-venous fat to the pancreatic parenchyma, providing an interpretable signal for classification. The
segmentation models are initialized via domain-specific transfer learning from a liver
segmentation task. We validate the full pipeline on a clinical
dataset of 214 abdominal ultrasound images with 107 expert-labeled cases using 5-fold cross-validation. SVM with RBF kernel achieves a mean cross-validated accuracy of 89.7%\,$\pm$\,1.8% and F1 of 0.898\,$\pm$\,0.019, while the unsupervised K-Means baseline reaches 87.8% accuracy, demonstrating that the proposed features capture the relevant clinical signal even without labeled training data. To our knowledge, this is the first end-to-end automated framework for fatty pancreas classification from ultrasound using
segmentation-guided texture analysis.
Read the full abstract ▾
All authors Ioan-Tudor-Alexandru Anghel, Ciprian-Mihai Ceausescu, Elena Dana Nedelcu, Elena Raluca Stirban, Camelia Croitoru, Despina Ungureanu, Ana Maria Palan, Gabriela Pop
Computer Vision
EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinement
Zitong Xu, Huiyu Duan, Yifei Nie et al. (+9 more)
Recent text-guided image editing (TIE) models have made remarkable progress, yet edited images still frequently suffer from fine-grained issues such as unnatural objects, lighting mismatch, and unexpected changes. Existing refinement approaches either rely on costly iterative regeneration or employ
vision-
language models (VLMs) with weak spatial grounding, often resulting in semantic drift and unreliable local corrections. To address these limitations, we first construct EditFHF-15K, a
dataset of fine-grained human feedback for edited images, comprising (1) 15K images from 12 TIE models spanning 43 editing tasks, (2) 60K annotated artifact regions and 80K editing failure regions, each accompanied by textual
reasoning, and (3) 45K mean opinion scores (MOSs) assessing perceptual quality, instruction following, and visual consistency. Based on EditFHF-15K, we propose EditRefiner, a hierarchical, interpretable, and human-aligned agentic framework that reformulates post-editing correction as a human-like perception-
reasoning-action-evaluation loop. Specifically, we introduce: (1) a perception
agent that detects contextual saliency maps of artifacts and editing failures, (2) a
reasoning agent that interprets these perceptual cues to perform human-aligned diagnostic inference, (3) an action
agent that uses the
reasoning output to plan and execute localized re-editing, and (4) an evaluation
agent that assesses the re-edited image and guides the action
agent on whether further refinements are required. Extensive experiments demonstrate that EditRefiner consistently outperforms state-of-the-art methods in distortion localization, diagnose accuracy and human perception
alignment, establishing a new paradigm for self-corrective and perceptually reliable image editing. The code is available at https://github.com/IntMeGroup/EditRefiner.
Read the full abstract ▾
All authors Zitong Xu, Huiyu Duan, Yifei Nie, Mingda Du, Sijing Wu, Xiongkuo Min, Tianyi Zheng, Jian Zhang, Shusong Xu, Jinwei Chen, Bo Li, Guangtao Zhai
Computer Vision
EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing
Lan Chen, Qi Mao, Yiren Song et al. (+2 more)
Visual-
prompt-guided edit transfer aims to learn image transformations directly from example pairs, offering more precise and controllable editing than purely text-driven approaches. However, existing
diffusion transformer-based methods often fail to faithfully reproduce the demonstrated edits due to structural mismatches between the task and the backbone, including a pretrained bias toward textual conditioning and inherent
stochastic instability during sampling. To bridge this gap, we present EditTransfer++, a framework that combines progressively structured training with an efficient conditioning scheme to improve both visual
prompt faithfulness and inference efficiency. We first mitigate textual dominance with a text-decoupled training strategy that removes text conditioning during fine-tuning, compelling the model to infer transformations solely from visual evidence while still supporting optional text guidance at inference. On top of this visually grounded model, a best-worst contrastive refinement mechanism reshapes the denoising trajectories to suppress unfaithful generations and improve consistency across random seeds. To alleviate the computational bottleneck of high-resolution in-context editing, we further introduce a condition compression and reuse strategy that reduces
token redundancy and enables efficient generation of images with a 1024-pixel long edge. Extensive experiments on existing
benchmarks and the proposed EditTransfer-Bench show that EditTransfer++ achieves state-of-the-art visual
prompt faithfulness with substantially faster inference than prior methods, suggesting a promising direction for scalable
prompt-guided image editing and broader visual in-context learning.
Read the full abstract ▾
All authors Lan Chen, Qi Mao, Yiren Song, Yuchao Gu, Siwei Ma
Computer Vision
Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs
Hao Wang, Yiqun Sun, Pengfei Wei et al. (+2 more)
Vision-
language models (VLMs) have advanced rapidly and are increasingly deployed in real-world applications, especially with the rise of
agent-based systems. However, their safety has received relatively limited
attention. Even the latest proprietary and open-weight VLMs remain highly vulnerable to
adversarial attacks, leaving downstream applications exposed to significant risks. In this work, we propose a novel and lightweight
adversarial attack detection framework based on sparse autoencoders (SAEs), termed SAEgis. By inserting an SAE module into a pretrained VLM and training it with standard reconstruction objectives, we find that the learned sparse latent features naturally capture attack-relevant signals. These features enable reliable classification of whether an input image has been adversarially perturbed, even for previously unseen samples. Extensive experiments show that SAEgis achieves strong performance across in-domain, cross-domain, and cross-attack settings, with particularly large improvements in cross-domain generalization compared to existing baselines. In addition, combining signals from multiple layers further improves robustness and stability. To the best of our knowledge, this is the first work to explore SAE as a plug-and-play mechanism for
adversarial attack detection in VLMs. Our method requires no additional
adversarial training, introduces minimal overhead, and provides a practical approach for improving the safety of real-world VLM systems.
Read the full abstract ▾
All authors Hao Wang, Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh, Daisuke Kawahara
Computer Vision
Towards Photorealistic and Efficient Bokeh Rendering via Diffusion Framework
Linxiao Shi, Siming Zheng, Zerong Wang et al. (+5 more)
Existing mobile devices are constrained by compact optical designs, such as small apertures, which make it difficult to produce natural, optically realistic bokeh effects. Although recent learning-based methods have shown promising results, they still struggle with photos captured under high digital zoom levels, which often suffer from reduced resolution and loss of fine details. A naive solution is to enhance image quality before applying bokeh rendering, yet this two-stage pipeline reduces efficiency and introduces unnecessary error accumulation. To overcome these limitations, we propose MagicBokeh, a unified
diffusion-based framework designed for high-quality and efficient bokeh rendering. Through an alternative training strategy and a focus-aware masked
attention mechanism, our method jointly optimizes bokeh rendering and super-resolution, substantially improving both controllability and visual fidelity. Furthermore, we introduce degradation-aware depth module to enable more accurate depth estimation from low-quality inputs. Experimental results demonstrate that MagicBokeh efficiently produces photorealistic bokeh effects, particularly on real-world low-resolution images, paving the way for future advancements in bokeh rendering. Our code and models are available at https://github.com/vivoCameraResearch/MagicBokeh.
Read the full abstract ▾
All authors Linxiao Shi, Siming Zheng, Zerong Wang, Hao Zhang, Jinwei Chen, Bo Li, Shifeng Chen, Peng-Tao Jiang
Computer Vision · Cross-listed from cs.LG
SR$^2$-LoRA: Self-Rectifying Inter-layer Relations in Low-Rank Adaptation for Class-Incremental Learning
Fengqiang Wan, Yipeng Lin, Kan Lv et al. (+1 more)
Pre-trained models with parameter-efficient fine-tuning (PEFT) have demonstrated promising potential for class-incremental learning (CIL), yet catastrophic forgetting still persists when adapting models to new tasks. In this paper, we present a novel perspective on catastrophic forgetting through the analysis of inter-layer relation drift, i.e., the progressive disruption of relationships among layer-wise representations during the learning of new tasks. We theoretically show that the increase of such drift reduces the classification margins of previously learned tasks, thereby degrading overall model performance. To address this issue, we propose \underline{S}elf-\underline{R}ectifying inter-layer \underline{R}elation Low-Rank Adaptation~(SR$^2$-LoRA), a simple yet effective method that mitigates catastrophic forgetting by constraining inter-layer relation drift. Specifically, SR$^2$-LoRA constructs the relation
matrices induced by the previous and current models on current-task samples, and aligns the corresponding singular values. We further theoretically show that this
alignment exhibits greater robustness to estimation perturbations than direct entry-wise
alignment. Extensive experiments on standard CIL
benchmarks demonstrate that SR$^2$-LoRA effectively mitigates catastrophic forgetting, with its advantages becoming more pronounced as the number of tasks increases. Code is available in the \href{https://github.com/FqWan24/SR-2-LoRA}{repository}.
Read the full abstract ▾
All authors Fengqiang Wan, Yipeng Lin, Kan Lv, Yang Yang
Computer Vision
Learning Image-Adaptive Scale Fields for Metric Depth Recovery
Yuanyan Li and Matthias Althoff
Monocular depth estimation (MDE) typically produces depth estimations that are defined up to an unknown scale or shift. When only sparse metric anchors are available, recovering accurate metric depth becomes challenging yet necessary for practical applications. We address this problem by formulating metric depth recovery as image-adaptive scale field modeling. Instead of directly correcting the depth, we reformulate the correction as a low-dimensional linear combination of image-adaptive basis maps. These maps are derived from semantic and
geometric cues encoded in the MDE estimations and intermediate representations. The weights of basis maps are efficiently determined from sparse metric anchors via a least-squares problem. This formulation yields improved metric depth accuracy, strong robustness under extreme anchor sparsity, and an interpretable decomposition of spatial scale variations. Extensive experiments across multiple
datasets and representative MDE models demonstrate the effectiveness and general applicability of our approach.
Read the full abstract ▾
Computer Vision
ChartREG++: Towards Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referring
Tianhao Niu, Ziyu Han, Xuan Dong et al. (+2 more)
Referring expression grounding is a core problem in visual grounding and is widely used as a diagnostic of spatial grounding and
reasoning in
vision and
language models, yet most prior work focuses on natural images. In contrast, existing chart referring expression grounding-related
benchmarks remain limited: (1) they largely adopt bounding boxes, constraining localization precision for fine chart elements (2) they mostly assume a single and two referred target instances, failing to handle multi-instance target references; (3) the
language expressions over-rely on textual cues or data-rank clues (4) they cover only a narrow range of chart types. To address these issues, we introduce a chart referring expression grounding
benchmark that systematically supports multiple localization forms, multiple referred targets, diverse grounding cues and diverse chart types. Results across representative
multimodal large models reveal a significant performance gap. We further introduce a code-driven synthesis pipeline that exploits the inherent
alignment between plotting programs and rendered chart primitives to derive pixel accurate instance masks across chart element types and granularities. We train an instance
segmentation model with the synthesized masks and integrate it into a general-purpose
multimodal grounding framework. The resulting system consistently outperforms baselines on our
benchmark and generalizes well to a ChartQA-derived real-chart grounding
benchmark.
Read the full abstract ▾
All authors Tianhao Niu, Ziyu Han, Xuan Dong, Qingfu Zhu, Wanxiang Che
Computer Vision
InsHuman: Towards Natural and Identity-Preserving Human Insertion
Jie Li, Shulian Zhang, Wenbo Li et al. (+2 more)
Human insertion aims to naturally place specific individuals into a target background. Although existing image editing models may have such ability, they often produce failure cases, including inappropriate human pose in new background, inconsistent number of people, and modified facial identity. Moreover, publicly available human
datasets often lack full-body portraits and realistic physical interaction between humans and their background. To address these challenges, we propose InsHuman for natural and identity-preserving human insertion. Specifically, we propose Human-Background Adaptive Fusion (HBAF), which detects foreground humans to obtain a binary mask and applies region-aware weighting to align the human regions between predicted and ground-truth latents, ensuring the person's pose, count, and overall appearance are coherently adapted to the target background.We further propose Face-to-Face ID-Preserving (FFIP), which detects and matches faces between the generated image and the source image in terms of face recognition features to enforce identity consistency for each face.In addition, we propose Bidirectional Data Pairing (BDP) strategy to construct BDP-InsHuman, a high-quality
dataset with realistic human-background interactions. Experiments demonstrate that InsHuman achieves significant improvements in generating plausible images while keeping human identity unchanged.
Read the full abstract ▾
All authors Jie Li, Shulian Zhang, Wenbo Li, Jian Chen, Yong Guo
Computer Vision
GPO-V: Jailbreak Diffusion Vision Language Model by Global Probability Optimization
Yu Pan, Andi Zhang, Yi Wang et al. (+2 more)
Diffusion Vision-
Language Models (dVLMs), built upon the non-causal foundations of
Diffusion Large
Language Models (dLLMs), have demonstrated remarkable efficacy in
multimodal tasks by departing from the traditional autoregressive generation paradigm. While dVLMs appear inherently robust against conventional jailbreak tactics, which we categorize as Fixed Prefix Optimization (FPO) (e.g., anchoring responses with "Sure, here is"), this perceived resilience is deceptive. Our investigation into the safety landscape of dVLMs reveals a unique refusal pattern: Immediate Refusal and Progressive Refusal. We find that while FPO-based attacks often fail by triggering the latter, the progressive refinement process itself uncovers a novel, latent attack surface. To exploit this vulnerability, we propose Global Probability Optimization (GPO), a general jailbreak paradigm designed specifically for the denoising trajectory of masked
diffusion models. Unlike prefix-based methods, GPO manipulates the global generative dynamics to bypass guardrails in
diffusion language models. Building on this, we introduce GPO-V, the first visual-modality jailbreak framework tailored for dVLMs. Empirical results demonstrate that GPO-V produces stealthy perturbations with exceptional cross-model transferability, revealing a critical security gap in non-sequential generative architectures. Our findings underscore the critical urgency of addressing safety
alignment in dVLMs. These results necessitate an immediate and fundamental re-evaluation of current defense paradigms to mitigate the unique risks of
diffusion-based generation. Our code is available at: https://anonymous.4open.science/r/GPO-V-0250.
Read the full abstract ▾
All authors Yu Pan, Andi Zhang, Yi Wang, Sibei Yang, Wenjie Wang
Computer Vision
Exposing and Mitigating Temporal Attack in Deepfake Video Detection
Zheyuan Gu, Minghao Shao, Zhen Wang et al. (+4 more)
While spatiotemporal deepfake detectors achieve high AUC, our experiments reveal their susceptibility to evasion attacks. These models tend to overfit on fragile temporal spectrum cues, rather than learning robust semantic causality. To mitigate this vulnerability, we propose SpInShield, a temporal spectral-invariant defense framework explicitly designed to decouple semantic motion from manipulatable spectral artifacts. We propose a learnable spectral adversary that dynamically synthesizes severe spectral deformations, simulating extreme attack scenarios. By employing a shortcut suppression optimization strategy, SpInShield compels the encoder to extract reliable forensic cues while purging unstable spectral statistics from the latent space. Experiments show that SpInShield obtains competitive performance on widely used
datasets and outperforms the strongest baseline by 21.30 percentage points in AUC under simulated amplitude spectral attacks.
Read the full abstract ▾
All authors Zheyuan Gu, Minghao Shao, Zhen Wang, Yusong Wang, Mingkun Xu, Shijie Zhang, Hao Jiang
Computer Vision
BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
Shaokai Ye, Vasileios Saveris, Yihao Qian et al. (+3 more)
Image captioning is one of the most fundamental tasks in computer
vision. Owing to its open-ended nature, it has received significant
attention in the era of
multimodal large
language models (MLLMs). In pursuit of ever more detailed and accurate captions, recent work has increasingly turned to reinforcement learning (RL). However, existing captioning-RL methods and evaluation metrics often emphasize a narrow notion of caption quality, inducing trade-offs across core dimensions of captioning. For example, utility-oriented objectives can encourage noisy, hallucinated, or overlong captions that improve downstream question answering while harming fluency, whereas arena-style objectives can favor fluent but generic descriptions with limited usefulness. To address this, we propose a more balanced RL framework that jointly optimizes utility-aware correctness, reference coverage, and linguistic quality. In order to effectively optimize the resulting continuous multi-objective reward formulation, we apply GDPO-style reward-decoupled normalization to continuous-valued captioning rewards and show that it improves performance over vanilla GRPO. Additionally, we introduce length-conditional reward masking, yielding a more suitable length penalty for captioning. Across LLaVA-1.5-7B and Qwen2.5-VL 3B and 7B base models, our method consistently improves caption quality, with peak gains of +13.6 DCScore, +9.0 CaptionQA, and +29.0 CapArena across different models.
Read the full abstract ▾
All authors Shaokai Ye, Vasileios Saveris, Yihao Qian, Jiaming Hu, Elmira Amirloo, Peter Grasch
Computer Vision
ST-Gen4D: Embedding 4D Spatiotemporal Cognition into World Model for 4D Generation
Haonan Wang, Hanyu Zhou, Tao Gu et al. (+1 more)
Generative models have achieved success in producing apparently coherent 2D
videos, but remain challenging in the physical world due to lack of 4D spatiotemporal scale. Typically, existing 4D generative models directly embed macro scale constraints to enhance overall spatiotemporal consistency. However, these methods only ensure global appearance coherence and fail to reveal the local dynamics of the physical world. Our insight is that global appearance structure and local dynamic
topology empower 4D spatiotemporal cognition, thereby enabling 4D generation with spatiotemporal regularities. In this work, we propose ST-Gen4D, a 4D generation framework with 4D spatiotemporal cognition-based world model. Our model is guided by four key designs: 1) Spatiotemporal representation. We encode various modalities into multiple representations as a feature basis. 2) Spatiotemporal cognition. We sculpture these representations into global appearance graph and local dynamic graph, and fuse them via semantic-bridged spatiotemporal fusion to obtain a 4D cognition graph. 3) Spatiotemporal
reasoning. We utilize a world model to derive future state based on the 4D cognition. 4) Spatiotemporal generation. We leverage the derived cognition as condition to guide latent
diffusion for 4D Gaussian generation. By deeply integrating 4D intrinsic cognition with generative priors, our model guarantees the structural rationality and topological consistency of 4D generation. Moreover, we propose ST-4D
datasets by aggregating public 4D
datasets and self-built subset. Extensive experiments demonstrate the superiority of our ST-Gen4D across 3D and 4D generation tasks.
Read the full abstract ▾
All authors Haonan Wang, Hanyu Zhou, Tao Gu, Luxin Yan
Computer Vision
A Marine Debris Detection Framework for Ocean Robots via Self-Attention Enhancement and Feature Interaction Optimization
Yuyang Li, Jiashu Han, Yinyi Lai et al. (+2 more)
Marine debris detection for ocean
robot is crucial for ecological protection, yet performance is often degraded by low-quality images with blur, complex backgrounds, and small targets. To address these challenges, we propose YOLO-MD, an enhanced YOLO-based detection framework. A Dual-Branch Convolutional Enhanced Self-
Attention (DB-CASA) module is designed to strengthen spatial-channel interactions, improving feature representation in degraded images. Additionally, a lightweight shift-based operation is introduced to enhance fine-grained feature extraction for objects of varying scales while maintaining parameter efficiency. We further propose SFG-Loss to mitigate class imbalance and optimization instability via dynamic sample reweighting. Experiments on the UODM
dataset demonstrate that YOLO-MD achieves 0.875 precision, 0.822 F1-score, and 0.849 mAP50, outperforming the latest state-of-the-art methods. The effectiveness of this method has also been verified through real-world robotic edge deployment experiments.
Read the full abstract ▾
All authors Yuyang Li, Jiashu Han, Yinyi Lai, Wenbin Kang, Zenghui Liu
Computer Vision · Cross-listed from cs.GR
Velocity-Space 3D Asset Editing
Hao Liu, Yuxuan Lin, Jingfeng Guo et al. (+4 more)
Local 3D asset editing requires modifying a target region while preserving the rest. Existing methods enforce locality through mechanisms external to the generator, such as manual 3D masks, post-hoc voxel merging, or 2D multi-view lifting. None intervene where the corruption originates: inside the ODE sampler. For faithful local editing, a rectified-flow generator's velocity field should be strong over the target region while vanishing on preserved content. Yet a single velocity field can hardly satisfy both requirements, leading to three problems: (i) identity leakage that keeps the edit signal non-zero on preserved regions; (ii) no dedicated edit-amplification channel, so strengthening the edit perturbs identity; and (iii) identity drag at the geometry and material stages, where a global condition pulls every
token toward the target. We propose VS3D (Velocity-Space 3D Asset editing), an inversion-free, training-free, and mask-free framework that derives edit and preserve signals by comparing the frozen model's source- and target-conditioned responses and realizes this principle through three stage-specific modules. Reconstruction-Anchored Source Injection (RASI) absorbs identity leakage by turning the unconditional
embedding into a per-step, asset-specific anchor calibrated through source reconstruction. Partial-Mean Guidance (PMG) amplifies the edit signal by contrasting high- and low-quality subsample estimates of the velocity difference, active only where a consistent edit exists. Twin-Agreement Residual Injection (TAR) lets the sampler decide
token by
token what to preserve at the geometry and material stages. Experiments on diverse 3D assets show that VS3D outperforms state-of-the-art native-3D and 2D-lifted editors, demonstrating the effectiveness of velocity-space editing for training-free 3D asset editing. Code is publicly available at https://github.com/LHaoooo/VS3D.
Read the full abstract ▾
All authors Hao Liu, Yuxuan Lin, Jingfeng Guo, Ruihang Chu, Junjie Wang, Ruotong Li, Yujiu Yang
Computer Vision
RELO: Reinforcement Learning to Localize for Visual Object Tracking
Xin Chen, Chuanyu Sun, Jiao Xu et al. (+4 more)
Conventional visual object trackers localize targets using handcrafted spatial priors, often in the form of heatmaps. Such priors provide only surrogate supervision and are poorly aligned with tracking optimization and evaluation metrics, such as intersection over union (IoU) and area under the success curve (AUC). Here, we introduce RELO, a REinforcement-learning-to-LOcalize method for visual object tracking that formulates target localization as a Markov decision process. Specifically, RELO replaces handcrafted spatial priors with a localization policy learned over spatial positions via reinforcement learning, with rewards combining frame-level IoU and sequence-level AUC. We additionally introduce layer-aligned temporal
token propagation to improve semantic consistency across frames, with negligible computational overhead. Across multiple
benchmarks, RELO achieves superior results, attaining 57.5% AUC on LaSOText without template updates. This confirms that reward-driven localization provides an effective alternative to prior-driven localization for visual object tracking.
Read the full abstract ▾
All authors Xin Chen, Chuanyu Sun, Jiao Xu, Houwen Peng, Dong Wang, Huchuan Lu, Kede Ma
Computer Vision · Cross-listed from cs.RO
Weather-Robust Scene Semantics with Vision-Aligned 4D Radar
Kali Hamilton and Christoffer Heckman
Cameras and
LiDAR degrade in rain, fog, and snow, while millimeter-wave radar remains largely unaffected. We align a radar encoder to frozen SigLIP
vision embeddings and decode structured scene captions through a frozen
vision-
language model (VLM) with approximately 7M trainable parameters. On K-RADAR with held-out fog, light snow, and heavy snow sequences, all radar configurations outperform a camera baseline that collapses to over 90%
hallucination. We identify a
token-norm mismatch as the dominant failure mode when bridging radar to a frozen VLM and show that projector-output LayerNorm resolves it. Analysis of encoder complexity, caption format, and pooling strategy reveals tradeoffs that inform future radar-VLM pipeline design.
Read the full abstract ▾
Computer Vision
UniISP: A Unified ISP Framework for Both Human and Machine Vision
Hanxi Li, Yao Cheng, Bo Zhang et al. (+1 more)
Compared to RGB images, raw sensor data provides a richer representation of information, which is crucial for accurate recognition, particularly under challenging conditions such as low-light environments. The traditional Image Signal Processing (ISP) pipeline generates visually pleasing RGB images for human perception through a series of steps, but some of these operations may adversely impact the information integrity by introducing compression and loss. Furthermore, in computer
vision tasks that directly utilize raw camera data, most existing methods integrate minimal ISP processing with downstream networks, yet the resulting images are often difficult to visualize or do not align with human aesthetic preferences. This paper proposes UniISP, a novel ISP framework designed to simultaneously meet the requirements of both human visual perception and computer
vision applications. By incorporating a carefully designed Hybrid
Attention Module (HAM) and employing supervised learning, the proposed method ensures that the generated images are visually appealing. Additionally, a Feature Adapter module is introduced to effectively propagate informative features from the ISP stage to subsequent downstream networks. Extensive experiments demonstrate that our approach achieves state-of-the-art performance across various scenarios and multiple
datasets, proving its generalizability and effectiveness.
Read the full abstract ▾
All authors Hanxi Li, Yao Cheng, Bo Zhang, Li Zeng
Computer Vision
UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition
Shuai Zhang, Zhecheng Shi, Zhuxiao Li et al. (+4 more)
Semantic
segmentation of large-scale 3D point clouds is crucial for applications such as autonomous driving and urban digital twins. However, the sparse sampling pattern of
LiDAR and the view-dependent
geometric distortion in image observations complicate cross-modal
alignment and hinder stable fusion. Inspired by the fact that 2D images captured by cameras are representations of the 3D world, we recognize that the features learned from 2D and 3D
segmentation share some common semantics, while other aspects remain modality-specific. This insight motivates a unified
multimodal framework for joint 2D-3D semantic
segmentation. We combine a SAM-based
vision encoder with a SPTNet-based
geometric encoder to extract complementary semantic and
geometric representations. The resulting features from both modalities are explicitly decomposed into shared and private subspaces, where the shared components summarize semantic factors common to both domains, and the private components preserve properties that are unique to each modality. A lightweight
attention-based fusion module aggregates the shared features into a consistent cross-modal representation, and a regularized training objective ensures both semantic
alignment and subspace independence. Experiments on the SemanticKITTI and nuScenes
benchmarks demonstrate consistent improvements in
segmentation accuracy over representative
multimodal baselines, accompanied by competitive computational efficiency. Cross-domain evaluation on nuScenes USA-Singapore shows stable performance under distribution shifts, demonstrating strong generalization. The implementation code is publicly available at: https://github.com/shuaizhang69/UniD-Shift.
Read the full abstract ▾
All authors Shuai Zhang, Zhecheng Shi, Zhuxiao Li, Jing Ou, Tengxi Wang, Yuan Liu, Wufan Zhao
Computer Vision
TTF: Temporal Token Fusion for Efficient Video-Language Model
Simin Huo and Ning LI
Video-
language models (VLMs) face rapid inference costs as visual
token counts scale with
video length. For example, 32 frames at $448{\times}448$ resolution already yield >8,000 visual
tokens in Qwen3-VL, making LLM prefill the dominant throughput bottleneck. Existing methods often rely on global similarity or
attention-guided compression, incurring offsets to their gains. We propose Temporal
Token Fusion (TTF), a training-free, plug-and-play pre-LLM
token compression framework that exploits structured temporal redundancy in
video. TTF automatically selects an anchor frame, then for each subsequent frame, performs a local window similarity search (e.g.,$3\times 3$), fusing
tokens that exceed a threshold. The compressed sequence maintains positional consistency across both prefill and decoding through coordinate realignment, enabling seamless integration with existing VLM pipelines. On Qwen3-VL-8B with threshold t=0.70, TTF removes about 67% of visual
tokens while retaining 99.5% of the baseline accuracy and introducing only ${\approx}0.16$\,GFLOPs of matching overhead. Overall, TTF offers a practical, efficient solution for
video understanding. The code is available at \href{https://github.com/Cominder/ttf}{https://github.com/Cominder/ttf}
Read the full abstract ▾
Computer Vision · Cross-listed from eess.SP
Task-Oriented Communication for Human Action Understanding via Edge-Cloud Co-Inference
Jingyi Liu, Cheng Yuan, Lijun He et al. (+2 more)
The expanding application of smart sensing has created a growing demand for the accurate understanding of human action at the network edge. Traditional approaches require massive
video data to be transmitted from resource-constrained edge devices to powerful cloud servers, incurring prohibitive uplink bandwidth consumption and unacceptable latency while raising
privacy concerns. To overcome these bottlenecks, we propose a task-oriented communication framework for human action understanding (TOAU) through edge-cloud collaboration. Our framework utilizes a
monocular pose
estimator to extract continuous joint coordinates from raw
videos, followed by a vector quantized variational autoencoder (VQ-VAE) to convert these coordinates into discrete motion
tokens. Consequently, only a compact sequence of codebook indices is transmitted over the network, consuming as few as 9 bits per frame and avoiding
privacy leakages. At the cloud server, a lightweight projector aligns these motion
tokens with the
embedding space of a large
vision-
language model (VLM) to facilitate complex action understanding, which is trained with an efficient instruction tuning paradigm. Comprehensive evaluations on three
benchmarks demonstrate that our TOAU system reduces the transmission payload to approximately 1% and the system latency to around 20% compared to
video codec-based solutions, while delivering comparable action understanding accuracy.
Read the full abstract ▾
All authors Jingyi Liu, Cheng Yuan, Lijun He, Jun Zhang, Jiawei Shao
Computer Vision
Disambiguating 2D-3D Correspondences in Gaussian Splatting-based Feature Fields for Visual Localization
Miso Lee, Sangeek Hyun, Yerim Jeon et al. (+1 more)
While Gaussian Splatting-based Feature Fields (GSFFs) have shown promise for visual localization, this paper highlights that photometrically optimized GSFFs are inherently not suitable for 2D-3D matching. The volumetric extent of each Gaussian induces many-to-one pixel-to-point mappings that destabilize PnP-based pose estimation, while photometric optimization gives rise to superfluous Gaussians devoid of multi-view consistency. To address these issues, we propose SplitGS-Loc, a GSFFs construction framework for localization, which disambiguates 2D-3D correspondences by exploiting Gaussian attributes. Our key design, Mixture-of-Gaussians-based splitting, decomposes each Gaussian into smaller Gaussians, replacing ambiguous many-to-one with precise one-to-one correspondences. In parallel, we exploit composition weights from GS rasterization to select Gaussians that significantly and consistently contribute across multiple views and aggregate discriminative features through strong pixel-Gaussian associations. The resulting compact yet discriminative feature fields enable stable PnP convergence, achieving competitive performance on localization
benchmarks. Extensive experiments validate that SplitGS-Loc extends the utility of photometric GSFFs to accurate and efficient localization by exploiting Gaussian attributes, without per-scene training for feature fields or iterative pose refinement. The code is available at: https://github.com/misoleekr/SplitGS-Loc.
Read the full abstract ▾
All authors Miso Lee, Sangeek Hyun, Yerim Jeon, Jae-Pil Heo
Computer Vision
SoLAR: Error-Resilient Streamable Long-Horizon Free-Viewpoint Video Reconstruction with Anchor Activation and Latent Recalibration
Haotian Zhang, Xu Mo, Yixin Yu et al. (+7 more)
Free-Viewpoint
Video (FVV) has emerged as a cornerstone of next-generation immersive media systems and attracted widespread
attention. Previous methods primarily focus on short
video sequences and suffer from significant performance degradation when processing long-horizon free-viewpoint
video (LFVV). Motivated by bit allocation theory, we analyze dynamic-anchor-based volumetric
video representation within a rate-distortion optimization framework and propose
SoLAR, which is the first error-resilient streamable FVV framework that maintains stable reconstruction quality on long sequences without requiring group-of-pictures partitioning. We propose the Anchor Activation Dynamics (AAD), which enables dynamic anchors to model non-rigid transformations by dynamically activating informative anchors and suppressing redundant ones. Furthermore, we introduce Latent Discrepancy Aware Recalibration (LaDAR), which is a mechanism to identify discrepancies between latent representations and recalibrate the correspondences encoded in the network, effectively mitigating error propagation in LFVV without compromising real-time performance or storage compactness. Extensive experiments demonstrate that
SoLAR achieves state-of-the-art reconstruction performance while maintaining minimum storage overhead, which provides a new direction for LFVV reconstruction and advances the practical deployment of immersive systems. Demo free-viewpoint
videos are provided in the supplementary material.
Read the full abstract ▾
All authors Haotian Zhang, Xu Mo, Yixin Yu, Guanhua Zhu, Jian Xue, Tongda Xu, Yan Wang, Jiaqi Zhang, Siwei Ma, Wen Gao