Computer Vision
ShellfishNet: A Domain-Specific Benchmark for Visual Recognition of Marine Molluscs
Ziheng Zhou, Yang Wang, Nan Wang et al. (+2 more)
Reliable visual recognition of shellfish is important for ecological monitoring and aquaculture, but real-world imagery exhibits substantial variation in acquisition conditions, appearance, and posture. Existing marine
vision resources cover diverse taxa and imaging settings, but fine-grained mollusc recognition remains underexplored in a group-disjoint
benchmark that jointly evaluates class prediction, representation-space
retrieval, and condition sensitivity. To address this gap, we introduce ShellfishNet, a provenance-aware, group-disjoint
benchmark of 40,130 images from 100 marine mollusc classes, organized into 30,879 acquisition or observation groups. Rather than treating recognition as a single clean-accuracy leaderboard, ShellfishNet evaluates model configurations across classification, image-to-image
retrieval, taxonomic error structure, and controlled condition sensitivity. We
benchmark representative convolutional,
transformer-based, hybrid and efficient, self-supervised, and
vision-
language models on classification and
retrieval. Separately, we evaluate open image-captioning and
vision-
language models on the captioning subset and assess selected fixed checkpoints under controlled generic and marine-inspired synthetic transformations. Across the
benchmark, image-level and class-aware classification largely agree, whereas
retrieval provides complementary model rankings and taxonomy projection reveals that a non-negligible subset of exact-class errors remains correct at higher taxonomic ranks. Selected fixed checkpoints with similar clean accuracy can also exhibit distinct sensitivity profiles under controlled transformations. These results show that model comparison on shellfish imagery should account for both task and image condition, rather than rely on a single clean classification score. The
dataset and evaluation protocols will be publicly released.
Read the full abstract ▾
All authors Ziheng Zhou, Yang Wang, Nan Wang, Chengliang Wu, Jun Yan
Computer Vision
Junwei Wen, Deshui Miao, Guangming Lu et al. (+2 more)
Video Reasoning Segmentation (VRS) aims to segment target objects in
videos based on implicit instructions that convey human intent and temporal logic. Existing MLLM-based methods predict masks with a [SEG]
token after selecting frames via simple sampling or an auxiliary MLLM, where limited supervision and frame-
language similarity rules often yield narrow-scope keyframe choices that weaken holistic temporal understanding and lead to brittle localization in complex multi-object scenes. To address these issues, we introduce RCoT-Seg, a
video-of-thought framework that factorizes VRS into temporal
video reasoning (TVR) and keyframe target perception (KTP), explicitly separating temporal
reasoning from spatial perception. Specifically, in the TVR stage, an agentic keyframe selection module, initialized with a curated CoT-start corpus and refined by GRPO under task-aligned rewards, is proposed to generate and reselect the keyframe through self-evaluation, strengthening moment localization and temporal
reasoning. In the KTP stage, RCoT-Seg performs high-resolution
segmentation on the selected frame and propagates masks with SAM2-based methods across the sequence, replacing heuristic sampling and external selectors while improving spatial precision and inter-frame consistency. Extensive experimental results demonstrate that the proposed RCoT-Seg achieves favorable performance against the state-of-the-art methods. The code and models will be publicly released at https://github.com/Victor-wjw/RCoT-Seg.
Read the full abstract ▾
All authors Junwei Wen, Deshui Miao, Guangming Lu, Xin Li, Wenjie Pei
Computer Vision
GC-ART: Global Learnable Second-Order Rational Tone Curves for Illumination Robustness
Wei Huang and Joyce Huang
We introduce GC-ART (Global Curve Adaptive Rational Tone-mapping), a lightweight differentiable pre-processing module for robust image classification. GC-ART predicts an endpoint-pinned rational tone curve from per-channel soft histograms using a 643-parameter MLP, then applies the curve pointwise before the classifier. The module is trained end-to-end with cross-
entropy and a soft monotonicity penalty. On CIFAR-10 with a CIFAR-style ResNet-18, GC-ART matches clean accuracy with the unenhanced baseline and other learned enhancers, improves over the baseline on multiplicative darkening, and achieves the best learned-method result on contrast corruption (48.45% vs. 46.27% for the baseline and 47.13% for Zero-DCE++). These results suggest that histogram-conditioned rational curves can learn useful global tone corrections, including contrast-expanding behavior, while preserving edge locations by construction through pointwise mapping. GC-ART also uses substantially fewer FLOPs than convolutional learned enhancers at 32 x 32. The current hyperparameters are untuned, leaving room for systematic improvement.
Read the full abstract ▾
Computer Vision
Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representations
Yuan Zhang, Chenyi Li, Haodong Yu et al. (+8 more)
Sampling from pretrained
diffusion and flow-matching models typically requires many forward passes to generate diverse and high-fidelity images. Existing
distillation methods often rely on multiple auxiliary networks, carefully designed training stages, or complex optimization pipelines. In this work, we revisit the recently proposed Drifting Model objective and show that a single drifting loss can be directly used to simplify one step
distillation. A key observation is that the pretrained
diffusion teacher itself already provides a strong representation space. Unlike the original Drifting Model, which relies on an additional pretrained feature extractor, we use intermediate hidden states of the pretrained teacher model as the feature representation. This removes the need for training or introducing an extra representation network while preserving a semantically meaningful feature geometry for drifting. Furthermore, we introduce a lightweight mode coverage loss to mitigate mode collapse during
distillation and encourage the student generator to cover diverse teacher-supported regions. Extensive experiments on ImageNet and SDXL demonstrate that our method achieves efficient one step generation with competitive image quality and diversity, achieving FID scores of 1.58 on ImageNet-64$\times$64 and 18.4 on SDXL, while substantially simplifying the overall
distillation framework.
Read the full abstract ▾
All authors Yuan Zhang, Chenyi Li, Haodong Yu, Guoqing Ma, Jiajun Zha, Yuanming Yang, Bo Wang, Wei Tang, Wenbo Li, Haoyang Huang, Nan Duan
Computer Vision
GEM: Generating LiDAR World Model via Deformable Mamba
Yang Wu, Zhaojiang Liu, Qiang Meng et al. (+5 more)
World models, which simulate environmental dynamics and generate sensor observations, are gaining increasing
attention in autonomous driving. However, progress in
LiDAR-based world models has lagged behind those built on camera
videos or occupancy data, primarily due to two core challenges: the inherent disorder of
LiDAR point clouds and the difficulty of distinguishing dynamic objects from static structures. To address these issues, we propose GEM: a Generative
LiDAR world model that leverages deformable mamba architecture, significantly improving fidelity and imaginative capability. Specifically, leveraging the structural similarity between sequential
laser scanning and Mamba's processing mechanism, we first tokenize
LiDAR sweeps into compact representations via a custom
LiDAR scene tokenizer. After unsupervised disentanglement of tokenized features via a dynamic-static separator, a tri-path deformable Mamba is introduced to perform selective scanning and adaptive gating fusion over the disentangled features, leading to enhanced spatial-temporal understanding of the world
evolution. Optionally, a planner and a BEV layout controller can be integrated to explore the model's capability for autonomous rollout and its potential to generate “what-if" scenarios. Extensive experiments show that GEM achieves state-of-the-art performances across diverse
benchmarks and evaluation settings, demonstrating its superiority and effectiveness. Project page: https://github.com/wuyang98/GEM.
Read the full abstract ▾
All authors Yang Wu, Zhaojiang Liu, Qiang Meng, Youquan Liu, Renliang Weng, Jianjun Qian, Jian Yang, Jin Xie
Computer Vision
Amortized-Precision Quantization for Early-Exit Vision Transformers
Rui Fang, Hsi-Wen Chen and Ming-Syan Chen
Vision Transformers (ViTs) achieve strong performance across
vision tasks, yet their deployment with low-precision early exiting remains fragile. Existing quantization methods assume static full-depth execution, making them unstable when exit decisions are perturbed by quantization noise, which can amplify errors along dynamic inference paths. In this paper, we introduce Amortized-Precision Quantization (APQ), a utilization-aware formulation that accounts for layer-wise
stochastic exposure to quantization noise and reveals depth-precision trade-offs. Building on APQ, we propose Mutual Adaptive Quantization with Early Exiting (MAQEE), a bi-level framework that jointly optimizes exit thresholds and bit-widths under explicit risk control to improve inference stability. MAQEE establishes a superior Pareto frontier in the accuracy-efficiency trade-off, reducing BOPs by up to 95% while maintaining accuracy and outperforming strong baselines by up to 20% across classification, detection, and
segmentation tasks.
Read the full abstract ▾
Computer Vision
EgoPro-Bench: Benchmarking Personalized Proactive Interaction in Egocentric Video Streams
Dongchuan Ran, Linyu Ou, Xueheng Li et al. (+5 more)
Existing
Multimodal Large
Language Models (MLLMs) remain primarily reactive, failing to continuously perceive environments or proactively assist users. While emerging
benchmarks address proactivity, they are largely confined to alert scenarios, neglect personalized context, and fail to evaluate the precise timing of human-machine interactions (HMI).In this paper, we introduce EgoPro-Bench, a novel
benchmark for training and evaluating proactive interaction capabilities based on streaming
egocentric videos; it comprises 2,400
videos in the evaluation set and over 12,000
videos in the training set.Unlike previous works, EgoPro-Bench leverages simulated user profiles to generate diverse user intentions and to construct high-fidelity HMI data across 12 distinct domains.Subsequently, we propose a specialized evaluation protocol and metrics, train proactive interaction models designed for efficient
reasoning and low-latency interaction on streaming
video data, and conduct comprehensive evaluations.Furthermore, we introduce an interaction principle termed "short thinking, better interaction", which allocates a limited
token budget prior to intent recognition, thereby enhancing interaction performance.The experiments demonstrate that EgoPro-Bench substantially enhances the intention understanding capabilities of MLLMs and enables accurate identification of appropriate timings for HMI, thereby laying a solid foundation for next-generation user-centric proactive interactive
agents.
Read the full abstract ▾
All authors Dongchuan Ran, Linyu Ou, Xueheng Li, Wenwen Tong, Chenxu Guo, Hewei Guo, Kaibing Wang, Lewei Lu
Computer Vision
Sword: Style-Robust World Models as Simulators via Dynamic Latent Bootstrapping for VLA Policy Post-Training
Jiaxuan Gao, Yongjian Guo, Zhong Guan et al. (+5 more)
The integration of
Vision-
Language-Action (VLA) models with World Models has gained increasing
attention. One representative approach treats learned World Models as generative simulators, enabling policy optimization entirely within "imagination." However, when deployed as simulators for specific environments such as the LIBERO
benchmark, existing World Models often suffer from poor generalization and long-horizon error accumulation. During closed-loop rollouts, these models are highly sensitive to initial-state perturbations; minor changes in color, illumination, and other visual factors can trigger cascading
hallucinations, leading to severe blurriness or overexposure. Moreover, long-horizon error accumulation further degrades the quality and fidelity of predicted future states. These issues limit the reliability of World Models as simulators. To mitigate these problems, we propose Sword, a robust World Model framework. Our method introduces Structure-Guided Style Augmentation to disentangle the visual textures of interactive environments from task-relevant dynamics, thereby improving generalization. We further propose Dynamic Latent Bootstrapping, which maintains consistency between training and inference while keeping
memory consumption low. Extensive experiments on the LIBERO
benchmark show that our method significantly outperforms the baseline WoVR in terms of generalization, generation quality, robustness, fidelity, and the success rate of reinforcement-learning post-training for VLA models.
Read the full abstract ▾
All authors Jiaxuan Gao, Yongjian Guo, Zhong Guan, Wen Huang, Wanlun Ma, Xi Xiao, Junwu Xiong, Sheng Wen
Computer Vision
SplatWeaver: Learning to Allocate Gaussian Primitives for Generalizable Novel View Synthesis
Yecong Wan, Fan Li, Mingwen Shao et al. (+1 more)
Generalizable novel view synthesis aims to render unseen views from uncalibrated input images without requiring per-scene optimization. Recent feed-forward approaches based on 3D Gaussian Splatting have achieved promising efficiency and rendering quality. However, most of them assign a fixed number of Gaussians to each pixel or voxel, ignoring the spatially varying complexity of real-world scenes. Such uniform allocation often wastes Gaussian primitives in smooth regions while providing insufficient capacity for fine structures, complex geometry, and high-frequency details. This motivates us to predict region-dependent primitive cardinalities rather than impose a fixed primitive budget everywhere, enabling a more expressive 3D scene representation. Therefore, we propose SplatWeaver, a generalizable novel view synthesis framework that is able to dynamically allocate Gaussian primitives over different regions in a feed-forward manner. Specifically, SplatWeaver introduces cardinality Gaussian experts and a pixel-level routing scheme, wherein each expert specializes in producing a specific number of primitives from 0 to M, and the routing scheme coordinates these experts to adaptively determine how many Gaussian primitives should be allocated to each spatial location. Moreover, SplatWeaver incorporates a high-frequency prior with attendant guidance module and routing regularization to stabilize expert selection and promote complexity-aware allocation. By leveraging high-frequency cues, the routing process is encouraged to assign more Gaussian primitives to fine structures and textured regions, while suppressing redundancy in smooth areas. Extensive experiments across diverse scenarios show that SplatWeaver consistently outperforms state-of-the-art methods, delivering more faithful novel-view renderings with fewer Gaussian primitives. Project Page: https://yecongwan.github.io/SplatWeaver/
Read the full abstract ▾
All authors Yecong Wan, Fan Li, Mingwen Shao, Wangmeng Zuo
Computer Vision · Cross-listed from cs.LG
Predictive but Not Plannable: RC-aux for Latent World Models
Wenyuan Li, Guang Li, Keisuke Maeda et al. (+2 more)
A latent world model may achieve accurate short-horizon prediction while still inducing a latent space that is poorly aligned with planning. A key issue is spatiotemporal mismatch: these models are often trained with local predictive supervision, but deployed for long-horizon goal-directed search in latent spaces where Euclidean distance may not reflect what is reachable within a finite action budget. We present the Reachability-Correction auxiliary objective (RC-aux), a lightweight correction for this mismatch in reconstruction-free latent world models. RC-aux keeps the world-model backbone unchanged and adds planning-aligned supervision along two axes. Along the time axis, multi-horizon open-loop prediction trains the model beyond one-step consistency. Along the space axis, budget-conditioned reachability supervision, together with temporal hard negatives, encourages the latent space to distinguish states that are eventually reachable from those reachable within the current planning horizon. At test time, the learned reachability signal can also be used by a reachability-aware planner to favor trajectories that are both goal-directed and attainable under the available budget. We instantiate RC-aux on LeWorldModel and evaluate it under both continuation-training and matched-from-scratch settings. Across goal-conditioned pixel-control tasks and a LIBERO-Goal extension, RC-aux improves LeWM-style planning with modest additional cost. These results suggest that planning with latent world models depends not only on predictive accuracy, but also on whether the learned representation encodes the temporal and
geometric structure required by downstream search. The code is available at https://github.com/Guang000/RC-aux.
Read the full abstract ▾
All authors Wenyuan Li, Guang Li, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Computer Vision
Jiaju Han, Chao Li, Chengyin Hu et al. (+8 more)
Multimodal RAG systems increasingly rely on
vision-
language retrievers to ground visual queries in external textual evidence. Existing
adversarial studies on RAG mainly manipulate the
retrieval corpus or
memory, while attacks on
vision-
language and remote sensing models typically target end-task predictions. Input-space threats to the evidence
retrieval stage of remote sensing
multimodal RAG remain underexplored. To address this gap, we introduce CloudWeb, an atmospheric
retrieval hijacking attack that modifies only the input image while keeping the retriever, generator, and knowledge base fixed at deployment. CloudWeb overlays parameterized cloud- and haze-like patterns on remote sensing images and optimizes them with a
retrieval-oriented objective that pulls
adversarial image
embeddings toward target atmospheric evidence, suppresses source-scene evidence, enforces rank separation, and regularizes naturalness and coverage. To the best of our knowledge, this is the first study of
retrieval-stage atmospheric evidence hijacking in remote sensing
multimodal RAG. We evaluate CloudWeb on a seven-
dataset remote sensing RAG
benchmark with five CLIP-style retrievers, including GeoRSCLIP, RemoteCLIP, OpenAI CLIP, and OpenCLIP, together with downstream
vision-
language generators. Across retrievers, CloudWeb consistently outperforms clean
retrieval, handcrafted atmospheric baselines, random cloud perturbations, and fixed variants in injecting weather-related evidence into top-ranked results. On GeoRSCLIP ViT-B/32, Weather@5 increases from 0.71% to 43.29%. Downstream generation further shows measurable weather
hallucination and semantic shift, indicating that
retrieval-stage hijacking can propagate to the final RAG response. These findings reveal a practical failure mode: natural-looking atmospheric changes can compromise evidence
retrieval before generation begins.
Read the full abstract ▾
All authors Jiaju Han, Chao Li, Chengyin Hu, Qike Zhang, Xuemeng Sun, Xin Wang, Fengyu Zhang, Xiang Chen, Yiwei Wei, Jiahuan Long, Jiujiang Guo
Computer Vision
Sat3R: Satellite DSM Reconstruction via RPC-Aware Depth Fine-tuning
Qiaoyi Yang, Chaoyi Zhou, Xi Liu et al. (+9 more)
Accurate Digital Surface Model (DSM) reconstruction from satellite imagery is critical for applications such as disaster response, urban planning, and large-scale geographic mapping. Existing approaches face a fundamental trade-off: optimization-based methods achieve strong accuracy but require hours of per-scene computation, while generalizable geometry foundation models offer near-instant inference but fail to generalize to satellite imagery due to the domain gap introduced by the Rational
Polynomial Camera (RPC) model and mismatched depth scale distributions. We present Sat3R, a feed-forward framework that bridges this gap via RPC-aware metric depth fine-tuning of Depth Anything V2 using the Scale-Invariant Logarithmic (SiLog) loss. By constructing physically consistent pseudo depth supervision from RPC geometry, Sat3R adapts a
monocular depth foundation model to the satellite domain without per-scene optimization. Experiments on the DFC2019
benchmark demonstrate that Sat3R reduces MAE by 38% over zero-shot feed-forward baselines and achieves competitive accuracy against optimization-based methods, while delivering over 300x speedup. Sat3R demonstrates that feed-forward models, when properly adapted to the satellite domain, can match optimization-based accuracy at a fraction of the computational cost, paving the way for practical large-scale satellite DSM reconstruction.
Read the full abstract ▾
All authors Qiaoyi Yang, Chaoyi Zhou, Xi Liu, Run Wang, Minghui Xu, Mert D. Pesé, Feng Luo, Yuhao Xu, Zhi-Qi Cheng, Qiushi Chen, Hairong Qi, Siyu Huang
Computer Vision
Adaptive Subspace Projection for Generative Personalization
Van-Anh Nguyen, Anh Tuan Bui, Tamas Abraham et al. (+5 more)
Generative personalization often suffers from the semantic collapsing problem (SCP), where a learned personalized concept overpowers the rest of the text
prompt, causing the model to ignore important contextual details. To address this, we first analyze the underlying cause, revealing that the semantic drift responsible for SCP is not random but is concentrated within a specific low-dimensional subspace. We also discover that the personalization process perturbs the
embedding of the original base concept, making it an unstable reference point. Based on these insights, we introduce Test-time
Embedding Adjustment with Adaptive Subspace Projection (AdaptSP), a training-free method that uses the stable, pre-trained
embedding as an anchor. AdaptSP isolates the semantic drift and projects it onto the identified subspace, performing a precise adjustment that mitigates SCP while maintaining the subject identity. Our experiments show that this targeted approach significantly improves
prompt fidelity and contextual
alignment.
Read the full abstract ▾
All authors Van-Anh Nguyen, Anh Tuan Bui, Tamas Abraham, Junae Kim, Amardeep Kaur, Rollin Omari, Thuy-Trang Vu, Dinh Phung
Computer Vision
TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts
Jeimin Jeon, Hyunju Lee and Bumsub Ham
Transformer architecture search (TAS) discovers optimal
vision transformer (ViT) architectures automatically, reducing human effort to manually design ViTs. However, existing TAS methods suffer from the feature collapse problem, where subnets within a supernet fail to learn subnet-specific features, mainly due to the shared weights in a supernet, limiting the performance of individual subnets. To address this, we propose TAS-LoRA, a novel method that introduces parameter-efficient low-rank adaptation (LoRA) to enable subnet-specific feature learning, while maintaining computational efficiency. TAS-LoRA incorporates a Mixture-of-LoRAExperts (MoLE) strategy, where a lightweight router dynamically assigns LoRA experts based on subnet architectures, and introduces a group-wise router initialization technique to encourage diverse feature learning across experts early in training. Extensive experiments on ImageNet and several transfer learning
benchmarks, including CIFAR-10/100, Flowers, CARS, and INAT-19, demonstrate that TAS-LoRA mitigates feature collapse effectively, improving performance over state-of-the-art TAS methods significantly.
Read the full abstract ▾
Computer Vision
High-Fidelity Surface Splatting-Based 3D Reconstruction from Multi-View Images
Nandhana Sunil, Abhirami R Iyer and Avirup Mandal
Multi-view mesh reconstruction remains a core challenge in computer graphics and
vision, especially for recovering high-frequency geometry from sparse observations. Recent methods such as 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF) rely on post-processing for mesh extraction, thereby limiting joint optimization of geometry and appearance. Implicit Moving Least Squares (IMLS) instead enables direct conversion of point clouds into signed distance and texture fields, supporting end-to-end reconstruction and rendering. However, existing IMLS formulations use exponential kernels that struggle with high-frequency detail. We introduce a compact
polynomial kernel with local support and greater flexibility, allowing better control over frequency content and improved
geometric fidelity. To further enhance fine details, we incorporate
stochastic regularization with
Laplacian filtering. Together, these improve the preservation of high-frequency structure while maintaining stable optimization. Experiments show state-of-the-art performance in both surface reconstruction and rendering, yielding more accurate geometry and sharper visuals from multi-view data.
Read the full abstract ▾
Computer Vision
LENS: Low-Frequency Eigen Noise Shaping for Efficient Diffusion Sampling
Haewon Jeon and Si-Hyeon Lee
Distilled
diffusion models accelerate image generation by reducing the number of denoising steps, but often suffer from degraded image quality. To mitigate this trade-off, test-time optimization methods improve quality, yet their iterative nature incurs substantial computational overhead and leads to slow inference, limiting practical usability. Recent hypernetwork-based approaches amortize this process during training, but still require costly noise modulation in high-dimensional latent spaces. In this work, we propose LENS (Low-frequency Eigen Noise Shaping), an efficient noise modulation framework that operates in a low-dimensional subspace. Our approach is motivated by the observation that low-frequency components of the noise largely determine the global structure and visual fidelity of generated images. Based on this observation, we provide a theoretical justification for restricting modulation to the low-frequency subspace and derive a principled training objective. Building on this, LENS employs a lightweight, standalone network to selectively modulate these components, enabling efficient and targeted noise modulation. Extensive experiments demonstrate that LENS achieves competitive image quality while reducing FLOPs by 400-700$\times$, model parameters by 25-75$\times$, and inference-time overhead by 10-20$\times$ compared to prior methods.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
PersonaGest: Personalized Co-Speech Gesture Generation with Semantic-Guided Hierarchical Motion Representation
Junchuan Zhao, Qifan Liang and Ye Wang
Co-
speech gesture generation aims to synthesize realistic body movements that are semantically coherent with
speech and faithful to a user-specified gestural style. Existing VQ-VAE based co-
speech gesture generation methods improve generation quality but fail to encode semantic structure into the motion representation or explicitly disentangle content from style, limiting both semantic coherence and personalization fidelity. We present PersonaGest, a two-stage framework addressing both limitations. In the first stage, a semantic-guided RVQ-VAE disentangles motion content and gestural style within the residual quantization structure, where a Semantic-Aware Motion Codebook (SMoC) organizes the content codebook by gesture semantics and contrastive learning further enforces content-style separation. In the second stage, a Masked Generative
Transformer generates content
tokens via a semantic-aware re-masking strategy, followed by a cascade of Style Residual
Transformers conditioned on a reference motion
prompt for style control. Extensive experiments demonstrate state-of-the-art performance on objective metrics and perceptual user studies, with strong style consistency to the reference
prompt. Our project page with demo
videos is available at https://danny-nus.github.io/PersonaGest/
Read the full abstract ▾
Computer Vision
Hard to Read, Easy to Jailbreak: How Visual Degradation Bypasses MLLM Safety Alignment
Zhixue Song, Boyan Han, Yiwei Wang et al. (+1 more)
Recent advancements in visual context compression enable MLLMs to process ultra-long contexts efficiently by rendering text into images. However, we identify a critical vulnerability inherent to this paradigm: lowering image resolution inadvertently catalyzes jailbreaking. Our experiments reveal that the safety defenses of SOTA models deteriorate sharply as resolution degrades, surprisingly persisting even when text remains legible. We attribute this to “Cognitive Overload”, hypothesizing that the effort required to decipher degraded inputs diverts attentional resources from safety
auditing. This phenomenon is consistent across various visual perturbations, including noise and
geometric distortion. To address this, we propose a simple “Structured Cognitive Offloading” strategy that mitigates these risks by enforcing a serialized pipeline to decouple visual transcription from safety assessment. Our work exposes a significant risk in
vision-based compression and provides critical insights for the secure design of future MLLMs.
Read the full abstract ▾
All authors Zhixue Song, Boyan Han, Yiwei Wang, Chi Zhang
Computer Vision
Towards multi-modal forgery representation learning for AI-generated video detection and localization
Dat Le, Khoa Nguyen, Xin Wang et al. (+1 more)
Recent advances in generative AI have democratized
video creation at scale. AI-generated
videos, including partially manipulated clips across visual and audio channels, pose escalating risks of semantic distortion and misuse, which motivates the need for reliable detection tools. Most existing AI-generated
video detectors remain limited by single- or partial-modality of data modeling and the lack of fine-grained temporal forgery localization. To address these challenges, our primary novelty introduces a core architecture that jointly integrates an LMM semantic branch with a spatio-temporal (ST) visual branch and a multi-scale partial-spoof (PS) audio branch. This multi-modal approach enables simultaneous detection and fine-grained temporal localization of partially manipulated AI-generated
video forgeries. Extensive experiments show that this approach outperforms existing state-of-the-art methods.
Read the full abstract ▾
All authors Dat Le, Khoa Nguyen, Xin Wang, Shu Hu
Computer Vision
CASCADE: Context-Aware Relaxation for Speculative Image Decoding
Selin Yildirim, Subhajit Dutta Chowdhury, Mohammad Mahdi Kamani et al. (+2 more)
Autoregressive generation is a powerful approach for high-fidelity image synthesis, but it remains computationally demanding and slow even on the most advanced accelerators. While speculative decoding has been explored to mitigate this bottleneck, existing approaches fail to achieve efficiency gains comparable to those observed in text generation. A key limitation is the target model's high uncertainty during image generation, which leads to high draft
token rejection rates. In this work, we identify previously overlooked patterns in the target model's behavior that emerge naturally in tree-based speculative decoding. Specifically, we formalize two properties, semantic interchangeability and convergence, arising from the redundancies in the target model's hidden state representations. By capturing these redundancies across the depth and breadth of the predicted
token tree, our method identifies principled opportunities for acceptance relaxation without requiring additional training. Additionally, we enhance standalone drafter performance by injecting the redundancy signals from the target model into drafter training with minimal modification. We evaluate our approach across multiple text-to-image models and drafter architectures. Results show that CASCADE achieves state-of-the-art speedups for drafter-based speculative decoding, with up to 3.6x acceleration, while maintaining image quality and text-
prompt fidelity.
Read the full abstract ▾
All authors Selin Yildirim, Subhajit Dutta Chowdhury, Mohammad Mahdi Kamani, Vikram Appia, Deming Chen
Computer Vision
DINO-MVR: Multi-View Readout of Frozen DINOv3 for Annotation-Efficient Medical Segmentation
Wei Jiang, Feng Liu, Nan Ye et al. (+1 more)
Adapting foundation models to medical
segmentation typically requires either backbone fine-tuning or high-capacity task-specific decoders, both of which are difficult to fit reliably when annotations are scarce. We show that frozen DINOv3 features already contain useful structural and boundary cues for medical
segmentation, and that the main bottleneck lies in how these features are read out. We propose DINO-MVR, a Multi-View Readout framework for annotation-efficient medical
segmentation. DINO-MVR trains only lightweight MLP probes on features from the final three
transformer blocks of a frozen DINOv3 backbone, without updating the backbone itself. At inference, each input is interpreted through complementary resolutions and test-time augmentations, whose probability maps are combined by
entropy-weighted fusion and refined with simple spatial regularization. For volumetric inputs, Gaussian z-axis smoothing further improves inter-slice consistency. Under fixed evaluation protocols on endoscopy, dermoscopy, and MRI
benchmarks, DINO-MVR achieves strong readout-only performance, including 0.895 Dice on Kvasir-SEG, 0.897 Dice on ISIC 2018, and 0.908 Dice on BraTS FLAIR whole-tumor
segmentation. With only five annotated BraTS patients, it recovers 98.4% of the performance obtained by the 40-patient BraTS reference run. These results suggest that frozen self-supervised
vision backbones can support accurate medical
segmentation when paired with an effective multi-view readout.
Read the full abstract ▾
All authors Wei Jiang, Feng Liu, Nan Ye, Hongfu Sun
Computer Vision
LoHGNet: Infrared Small Target Detection through Lorentz Geometric Encoding with High-Order Relation Learning
Qianwen Ma, Yang Xu, Shangwei Deng et al. (+2 more)
Infrared small target detection (IRSTD) remains challenging due to the scarcity of useful target cues and the presence of severe background clutter. Most current methods rely on conventional feature learning and local interaction modeling, where features are represented in Euclidean space. However, such designs may still be limited in describing the subtle differences of weak targets and the contextual relations between targets and backgrounds. To address these limitations, we propose LoHGNet, an IRSTD network that integrates Lorentz
geometric encoding with high-order relation learning. By introducing Lorentz
manifold based feature learning, LoHGNet offers a different feature representation from conventional IRSTD methods and provides new discriminative cues for IRSTD. Specifically, a Lorentz encoding branch is constructed with the
Geometric Attention Guided Lorentz Residual Convolution Module (GA-LRCM) to perform feature modeling under
hyperbolic geometric constraints and enhance the hierarchical
geometric representation capability of weak targets. Subsequently, the
hyperbolic features are mapped into the Euclidean tangent space through logarithmic mapping, and a High-Order Relation Learning Module (HORL) is designed to model the high-order contextual dependencies between targets and backgrounds via hypergraph construction, thereby improving target discrimination in complex backgrounds. Experimental results on three
datasets demonstrate that the proposed LoHGNet achieves competitive performance in both detection accuracy and adaptability to complex scenes. The code will be available at https://github.com/Kingwin97.
Read the full abstract ▾
All authors Qianwen Ma, Yang Xu, Shangwei Deng, Xiaobo Li, Haofeng Hu
Computer Vision
From Pixels to Primitives: Scene Change Detection in 3D Gaussian Splatting
Chamuditha Jayanga Galappaththige, Jason Lai, Timothy Patten et al. (+3 more)
Scene change detection methods built on Gaussian splatting universally follow a render-then-compare paradigm: the pre-change scene is rendered into 2D and compared against post-change images via pixel or feature residuals. This change detection problem with Gaussian Splatting has been treated as a question about pixels; we treat it as a question about primitives. We provide direct evidence that native primitive attributes alone – position, anisotropic covariance, and color – carry sufficient signal for scene change detection. What makes primitive-space comparison hard is the under-constrained nature of Gaussian splatting representation: independent optimizations yield primitive solutions whose count, positions, shapes, and colors differ even where nothing has changed. We address this challenge with anisotropic models of
geometric and photometric drift, complemented by a per-primitive observability term that reflects the extent to which each Gaussian is constrained by the camera geometry. Operating directly on primitives gives our method, GD-DIFF, two properties that distinguish it from render-then-compare methods. First, change maps are multi-view consistent by construction, where prior work had to learn this through an additional optimization objective. Second,
geometric and appearance changes are scored separately, identifying not just where but what kind of change occurred, distinguishing structural changes (e.g., an added object) from surface-level ones (e.g., a color change) without supervision or external model dependencies. On real-world
benchmarks, GS-DIFF surpasses the prior state-of-the-art approach by $\sim$17% in mean Intersection over Union.
Read the full abstract ▾
All authors Chamuditha Jayanga Galappaththige, Jason Lai, Timothy Patten, Donald Dansereau, Niko Suenderhauf, Dimity Miller
Computer Vision
See Tomorrow, Act Today: Foresight-Driven Autonomous Driving
Bozhou Zhang, Nan Song, Yuang Wang et al. (+3 more)
Current end-to-end autonomous driving planners are fundamentally reactive: they condition on historical and present observations to predict future actions. We argue that autonomous
agents should instead imagine future scenes before deciding, just as human drivers mentally simulate “what will happen next" before acting. We introduce ForeSight, a foundation world model centric planning framework that reframes autonomous driving as anticipatory decision-making. Rather than treating world models as auxiliary components, ForeSight makes future scene imagination the primary driver of action prediction. Our approach operates in two stages: (1) generating plausible future visual worlds via a pretrained world model, and (2) planning actions conditioned on these imagined futures. This paradigm shift from “what should I do now?" to “what will happen, and how should I respond?" enables genuinely anticipatory rather than reactive planning. By grounding decisions in anticipated contexts rather than present observations alone, ForeSight navigates dynamic, interactive scenarios more effectively. Extensive experiments on NAVSIM and nuScenes demonstrate that explicit future imagination significantly outperforms previous state-of-the-art alternatives, validating our foresight-driven approach.
Read the full abstract ▾
All authors Bozhou Zhang, Nan Song, Yuang Wang, Jiankang Deng, Xiatian Zhu, Li Zhang
Computer Vision
Bincheng Peng, Guang Li, Ping Liu et al. (+2 more)
Dataset distillation compresses a large training set into a small synthetic set that preserves downstream training utility. While most existing methods target training networks from scratch, modern visual transfer learning often uses frozen pre-trained encoders followed by lightweight linear probing. Existing
distillation methods for this setting either unroll iterative linear-probe updates with trajectory-based gradient matching, or rely on closed-form formulations originally designed for from-scratch training with neural-tangent-kernel (NTK) approximations. Neither route exploits the fact that frozen-feature linear probing admits a closed-form solution determined directly by the pre-trained features themselves, with no infinite-width approximation and no inner-loop trajectory. We propose Closed-Form Linear-Probe
Dataset Distillation (CLP-DD), a bilevel formulation that computes the linear probe induced by the synthetic set with a sample-space kernel ridge solver. The synthetic images are then updated by evaluating this induced classifier on real features through a temperature-scaled softmax cross-
entropy, where the classifier columns act as learned class anchors in feature space. We further show that the choice of outer objective is decisive: pairing the closed-form inner solver with a standard MSE outer loss substantially underperforms trajectory-based methods, while the discriminative outer loss closes most of the gap. On ImageNet-100 with four pre-trained backbones, CLP-DD substantially improves over LGM without DSA and approaches LGM with DSA at a fraction of the computational cost. On ImageNet-1K, CLP-DD matches or surpasses LGM with DSA on three of four backbones while running roughly $14\times$ faster and using less than one-eighth of the GPU
memory.
Read the full abstract ▾
All authors Bincheng Peng, Guang Li, Ping Liu, Takahiro Ogawa, Miki Haseyama
Computer Vision
AsyncEvGS: Asynchronous Event-Assisted Gaussian Splatting for Handheld Motion-Blurred Scenes
Jun Dai, Renbiao Jin, Bo Xu et al. (+5 more)
3D reconstruction methods such as 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF) achieve impressive photorealism but fail when input images suffer from severe motion blur. While event cameras provide high-temporal-resolution motion cues, existing event-assisted approaches rely on low-resolution sensors and strict synchronization, limiting their practicality for handheld 3D capture on common devices, such as smartphones. We introduce a flexible, high-resolution asynchronous RGB-Event dual-camera system and a corresponding reconstruction framework. Our approach first reconstructs sharp images from the event data and then employs a cross-domain pose estimation module based on the Visual Geometry
Transformer (VGGT) to obtain robust initialization for 3DGS. During optimization, we employ a structure-driven event loss and view-specific consistency regularizers to mitigate the ill-posed behavior of traditional event losses and deblurring losses, ensuring both stable and high-fidelity reconstruction. We further contribute AsyncEv-Deblur, a new high-resolution RGB-Event
dataset captured with our asynchronous system. Experiments demonstrate that our method achieves state-of-the-art performance on both our challenging
dataset and existing
benchmarks, substantially improving reconstruction robustness under severe motion blur. Project page: https://openimaginglab.github.io/AsyncEvGS/
Read the full abstract ▾
All authors Jun Dai, Renbiao Jin, Bo Xu, Yutian Chen, Linning Xu, Mulin Yu, Tianfan Xue, Shi Guo
Computer Vision
Huaiyuan Qin, Muli Yang, Gabriel James Goenawan et al. (+4 more)
A recent work shows that
Attention Transfer, which transfers only the
attention patterns from a pre-trained teacher
Vision Transformer (ViT) to a randomly initialized standard student ViT, is sufficient to recover the full benefit of the teacher's pre-trained weights. We revisit this finding on a comprehensive
benchmark of 20 teachers from 11 well-known ViT families and reveal that
Attention Transfer is not universally effective. While 7 families transfer successfully, 4 consistently fail, falling up to 5.1% below the from-scratch no-transfer baseline. Further results demonstrate that this failure is family-consistent across model sizes, and persists under extended training durations, different transfer
datasets, and out-of-distribution evaluations. Controlled analyses then consistently localize the problem to the
attention-routing channel, indicating that the key issue is not whether the student can match the teacher's
attention patterns, but whether the matched patterns remain functional for the student. Crucially, we identify architectural mismatch between the pre-trained teacher and the standard student as the primary mechanism. By adding only the teacher's native architectural components to the student in a randomly initialized state, we completely reverse the failure for all 4 families. Notably, these components alone do not improve from-scratch training, confirming that they specifically unlock the usability of the teacher's
attention. We further show that this failure is not explained by the inadequate choice of transfer loss, differences in pre-training recipes, or the intrinsic quality of
attention maps. Our findings refine the prevailing understanding of
attention in ViT representations:
attention is sufficient only when the student architecture matches the teacher.
Read the full abstract ▾
All authors Huaiyuan Qin, Muli Yang, Gabriel James Goenawan, Peng Hu, Chen Gong, Xi Peng, Hongyuan Zhu
Computer Vision
PicoEyes: Unified Gaze Estimation Framework for Mixed Reality with a Large-Scale Multi-View Dataset
Fuxin Duan and Hui Wang
We present PicoEyes, a unified gaze estimation framework that directly predicts all key attributes of gaze, including 3D eye parameters, eye-region
segmentation, optical axis, visual axis, and depth maps, from either
monocular or binocular inputs. The framework simultaneously addresses calibration, gaze
forecasting, and varying device postures, while also supporting 3D eye reconstruction via joint estimation of eye parameters and depth maps in an end-to-end manner. In addition, we introduce a large-scale multi-view near-eye
dataset containing comprehensive 2D and 3D annotations under diverse conditions, including train, test, rewear-test, and calibration sessions. Extensive experiments demonstrate that PicoEyes achieves state-ofthe-art performance, consistently outperforming both academic and industrial gaze tracking methods across nocalibration, calibration, rewear-after-calibration, and
forecasting settings. This work establishes a practical, end-toend paradigm for robust and generalizable gaze estimation in mixed reality (MR) applications.
Read the full abstract ▾
Computer Vision
SatSurfGS: Generalizable 2D Gaussian Splatting for Sparse-View Satellite Surface Reconstruction
Min Chen, Wei Guo, Bin Wang et al. (+9 more)
Sparse-view satellite image surface reconstruction remains highly challenging, fundamentally because the reliability of multi-view matching under satellite imaging conditions is strongly spatially heterogeneous. Affected by large photometric differences, weak textures, and repetitive textures, multi-view
geometric constraints are often sparse, unevenly distributed, and locally unreliable. Although 2D Gaussian Splatting (2DGS) is more suitable than 3D Gaussian Splatting (3DGS) for the explicit representation of continuous surfaces, research on generalizable feed-forward 2DGS frameworks for sparse-view satellite surface reconstruction is still lacking. To address this issue, we propose SatSurfGS, a generalizable sparse-view surface reconstruction method for satellite imagery based on 2DGS. The proposed method builds a coarse-to-fine Gaussian attribute prediction framework and explicitly models local
geometric reliability at three levels: feature learning, Gaussian parameter estimation, and training optimization. Specifically, we propose a confidence-aware
monocular multi-view feature fusion module to adaptively integrate
monocular priors and multi-view matching features according to local confidence; a cross-stage self-consistency residual guidance module to stabilize stage-wise Gaussian parameter refinement using the residual between the rendered height map from the previous stage and the current-stage MVS height map, together with confidence information; and a confidence bidirectional routing loss to achieve differentiated allocation of
geometric and appearance supervision. Experiments on satellite
datasets show that the proposed method achieves improved rendering quality, surface reconstruction accuracy, cross-
dataset generalization, and inference efficiency compared with representative generalizable baselines and competitive per-scene optimization methods.
Read the full abstract ▾
All authors Min Chen, Wei Guo, Bin Wang, Wen Li, Tong Fang, Jinbo Zhang, Junqi Zhao, Hong Kuang, Han Hu, Xuming Ge, Qing Zhu, Bo Xu
Computer Vision
Masks Can Talk: Extracting Structured Text Information from Single-Modal Images for Remote Sensing Change Detection
Kai Zheng, Hang-Cheng Dong, Jiatong Pan et al. (+3 more)
Remote sensing change detection is pivotal for urban monitoring, disaster assessment, and environmental resource management. Yet, unimodal deep learning methods frequently confuse genuine semantic changes with visually similar but irrelevant variations. Recent
multimodal approaches incorporate text as auxiliary supervision, but their descriptions are either semantically coarse and unstructured or model-generated and thus noisy. Critically, all of them overlook a simple fact: fine-grained change semantics are already implicitly encoded in the ground-truth mask labels that come standard with every change detection
dataset. These masks know where the change happened, what the land-cover types were before and after, how the transition occurred, and how many objects were involved. In this paper, we propose S2M, a framework that obtains structured textual features directly from change labels at zero additional annotation cost. Specifically, each change region is automatically transcribed into a semantic quadruple (where, what, how, how many) and converted into several fixed-template text descriptions, providing precise, dense, and noise-free
multimodal supervision. We adopts a two-stage training strategy to fine-tune on remote sensing imagery firstly for robust domain-specific representation, after which a
multimodal decoder with a bi-directional contrastive loss is introduced to achieve deep
alignment between visual features and structured textual
embeddings. To validate our method, we construct Gaza-Change-v2, a new multi-class change detection (MCD)
dataset about the Gaza Strip. On this MCD
dataset, S2M achieves a Sek of 17.80% and an F$_{\text{scd}}$ of 66.14%, notably surpassing even
multimodal methods that leverage large
language models. Our work demonstrates that masks can indeed talk. They tell us exactly what, where, how, and how many changes have occurred.
Read the full abstract ▾
All authors Kai Zheng, Hang-Cheng Dong, Jiatong Pan, Zhenkai Wu, Fupeng Wei, Wei Zhang