Computer Vision · Cross-listed from physics.med-ph
Dose-PlanNet: Physics Based Radiotherapy Dose Prediction with Deep Learning
Ankit Bhattacharjee, Sougata Maity, Santam Chakraborty et al. (+1 more)
Automating prostate radiotherapy treatment planning is dosimetrically complex, particularly for extreme hypofractionated regimens. In this study, we introduce Dose-PlanNet, a physics-guided 3D deep learning architecture designed to predict dose distributions. This model's performance was evaluated on a cohort of patients treated in a prospective trial where two different dose fractionation regimens were employed. Dose-PlanNet achieved comparable target coverage ($D_{95}$), though statistical analysis revealed a marginal reduction in target homogeneity ($p<0.001$) offset. However the model achieved statistically significant improvements in high-dose organ-at-risk sparing ($p<0.001$). When evaluated against strict Prospective Randomized protocol volumetric constraints, automated plans met prespecified clinical acceptance criteria in $11$ out of $14$ Moderate Hypofraction Arm plans and $9$ out of $12$ Stereotactic Body Radiation Therapy Arm plans. This pipeline demonstrates that physics-informed deep learning can accelerate radiotherapy workflows while safely maintaining the stringent dosimetric quality required for high-precision clinical deployment.
Read the full abstract ▾
All authors Ankit Bhattacharjee, Sougata Maity, Santam Chakraborty, Indranil Mallick
Computer Vision
Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
Shiyi Zhang, Mushui Liu, Yunze Tong et al. (+8 more)
On-policy
distillation (OPD), which leverages a pre-trained, specialized teacher model to provide dense supervisory signals, has achieved significant success in Large
Language Models (LLMs) and has recently been adapted to flow matching models. However, this paradigm suffers from two major issues: First, training a separate, task-specific teacher for every new objective incurs high computational costs. Second, the discrepancy between teacher and student distributions often leads to compounding errors along the generation trajectory. In this paper, we introduce Self-OPD, a teacher-free OPD framework for flow matching models that turns the student's own self-exploration into step-wise supervision. At each timestep, Self-OPD branches the deterministic next-state prediction into $K$
stochastic SDE candidates, rolls them out with the ODE sampler, and compares their rewards against a deterministic self-reference baseline to obtain normalized advantages. The velocity field is optimized with an all-branch pull-push objective, where high-advantage branches attract the student and low-advantage branches repel it under direction-aware attenuation and SDE-variance normalization. For multi-objective
alignment, Self-OPD fuses normalized scores at the reward level, avoiding direct gradient conflict. Experiments on single and mixed reward
benchmarks show that Self-OPD outperforms prior RL and OPD methods without task-specific teachers.
Read the full abstract ▾
All authors Shiyi Zhang, Mushui Liu, Yunze Tong, Wanggui He, Siyu Zou, Jinlong Liu, Yunlong Yu, Jian Song, Hao Jiang, Pipei Huang, Bo Zheng
Computer Vision
CGS-SLAM: Collaborative Gaussian Splatting based SLAM for Multi-Agent Reconstruction
Jean-Daniel de Ambrogi, Aladine Chetouani, Vincent Nguyen et al. (+1 more)
Recent advances in SLAM have leveraged 3DGS for photorealistic reconstruction and novel view synthesis. However, most methods rely on RGB-D input, which is unavailable on consumer-grade smartphones, and few integrate 3DGS within a collaborative framework. Therefore, we present CGS-SLAM, a hybrid decentralized/centralized system enabling multi-
agent 3DGS SLAM using only RGB and inertial data. Each
agent performs local tracking with inertial data as a motion prior and reconstructs a scaled map using a metric
monocular depth
estimator (Depth Pro). Keyframe encodings are shared among
agents, enabling dynamic keyframing in regions of spatial overlaps with other
agents, enhancing submap
alignment. Afterwards, a central server aligns submaps using VGGT as a view
alignment model. This bidirectional communication keeps communication cost low during mapping and global reconstruction in difficult GNSS-denied environments. Experiments on multiple
datasets demonstrate competitive tracking performance, improved rendering quality over state-of-the-art methods, and accurate submap
alignment.
Read the full abstract ▾
All authors Jean-Daniel de Ambrogi, Aladine Chetouani, Vincent Nguyen, Aurélien Chateigner
Computer Vision
Haihan Li, Haihao Li, Zhenfei Xu et al. (+2 more)
Many
multimodal tasks depend on how visual elements are ordered and composed, not only on recognizing them in isolation. Internet memes are a compact case of this problem: their punchline often depends on a constrained reading order and cross-panel visual–textual cues. While large
vision-
language models (LVLMs) show strong performance on single-image understanding, it remains unclear whether they can perform sequence-aware
reasoning over structured meme layouts, especially in Chinese social media. We introduce CMPM, a Chinese Multi-Panel Meme
benchmark with 1,214 annotated samples covering five structural types, ordering dependency, panel-order constraints, and optional comment context. We formulate a two-layer evaluation: Task1 probes structure typing and order-sensitive panel sequencing (with a context ablation setting), and Task2 evaluates Chinese meme explanation generation with human ratings on five 1-3 Likert dimensions (visual, panel, humor, context, and faithfulness). We
benchmark five representative LVLMs under a unified protocol. Results indicate that canonical-display accuracy is not by itself evidence of order understanding: the primary shuffled condition produces a sharp accuracy drop, revealing a persistent gap in order-sensitive
multimodal reasoning. Task2 preferences place Gemini 3.1 Pro and GPT-5.5 above the open models, while comment context yields only a small and mixed Core4 gain. Code and data will be released upon acceptance.
Read the full abstract ▾
All authors Haihan Li, Haihao Li, Zhenfei Xu, Jize Qian, Yubo Xie
Computer Vision
FIDA: Feature Instability-Driven Attack on Self-Supervised Facial Representation
Zhiyang Chen, Changchun Yin, Huiqin Yang et al. (+1 more)
Self-supervised learning (SSL) models are vulnerable to
backdoor attacks. However, the systemic risks they pose in face representation have received little
attention. The
entanglement of identity features in self-supervised face learning presents unique challenges for attack stealthiness. To address this gap, we propose FIDA (Feature Instability-Driven Attack), a novel
backdoor attack framework. FIDA uses subtle semantic triggers for injection, but its key innovation is a novel objective called Feature Instability Loss. It trains the encoder to increase the sensitivity of triggered features along perturbation directions sampled during attack optimization . By preventing the
backdoor from exhibiting the rigid feature patterns typical of previous attacks, FIDA effectively evades the evaluated perturbation-based defenses. Experiments show that FIDA achieves a high attack success rate and generally preserves benign utility across the evaluated settings , posing a significant threat to real-world multimedia applications relying on facial analysis.
Read the full abstract ▾
All authors Zhiyang Chen, Changchun Yin, Huiqin Yang, Liming Fang
Computer Vision
A Geometry-Driven, Framework-Agnostic Optimization for Object Pose Estimation
Wei Chen, Tao Zhen, Zhongchen Shi et al. (+3 more)
Current object pose estimation research remains predominantly model-centric, focusing on architectural innovations and post-processing refinements. This paper introduces a data-centric optimization by proposing a novel, physically grounded rotation representation through principal axes
alignment. Our method aligns the object's coordinate system with its inherent
geometric axes, derived from inertial properties, yielding three key advantages: Inherent Stability-leveraging the energy-minimizing property of principal axes provides a robust representation that is less sensitive to noise and occlusions;
Symmetry-Aware Canonicalization-explicitly resolving rotational ambiguities for symmetric objects at the data level, which fundamentally eliminates label confusion during network training; and Framework Agnosticism-the optimization is applied purely at the
dataset level, ensuring plug-and-play compatibility with existing networks without any architectural modification. We validate the framework across diverse category-level and instance-level models. Extensive experiments demonstrate consistent and significant accuracy improvements, while preserving the integrity of the baseline network. This work establishes a new, geometry-driven direction for enhancing pose estimation, circumventing the need for complex network redesign.
Read the full abstract ▾
All authors Wei Chen, Tao Zhen, Zhongchen Shi, Jing Zhang, Liang Xie, Erwei Yin
Computer Vision
Haowen Gu, Gensheng Pei, Junzhu Mao et al. (+3 more)
Although
Multimodal Large
Language Models (MLLMs) have demonstrated impressive performance in Medical Visual Question Answering (Med-VQA), their reliance on global image features often lacks precise pixel-level grounding, thereby limiting clinical trustworthiness. To bridge the semantic gap between high-level clinical
reasoning and spatial localization, we propose \textsc{MedREAL} (Medical
REasoning-driven Answering and Localization), a unified framework that seamlessly aligns linguistic
reasoning with spatial grounding. Specifically, MedREAL introduces Seg Anchored
Reasoning Pooling (SARP) to distill task-relevant semantic evidence directly from [SEG]
tokens within the MLLM's hidden states. Furthermore, a
Reasoning-to-Visual (R2V) fusion mechanism is proposed to effectively inject these
reasoning-aware features into a
segmentation pipeline for accurate mask decoding. To facilitate this paradigm, we construct MedRAVS-13K, a comprehensive
dataset comprising 13,824 expertly validated samples across four diverse imaging modalities. Extensive experiments demonstrate that MedREAL significantly outperforms state-of-the-arts, achieving 68.49% gIoU and 70.47% cIoU on
benchmark evaluations. By generating evidence masks that are strictly consistent with textual diagnoses, MedREAL provides a robust, interpretable framework for
reasoning-driven medical image analysis.
Read the full abstract ▾
All authors Haowen Gu, Gensheng Pei, Junzhu Mao, Qiong Wang, Mingwu Ren, Yazhou Yao
Computer Vision
MedFG-VQA: Low-Frequency Memory and Graph Attention for Lightweight Medical VQA
Haowen Gu, Gensheng Pei, Zeren Sun et al. (+4 more)
Medical Visual Question Answering (Med-VQA) holds significant promise for clinical decision support, yet faces challenges due to limited annotated data and the high computational demands of existing large
vision-
language models. We propose MedFG-VQA, a lightweight framework that leverages a
memory bank to augment DCT-based low-frequency features and employs graph-enhanced cross-
attention for effective visual-textual
alignment. Specifically, our approach features two key components: Frequency-
Memory Fusion (FMF), which enhances low-frequency features by retrieving from a learnable
memory bank built on DCT decomposition, and Graph-Aware Cross-
Attention (GACA), which aligns visual-textual features via cross-
attention and refines them through graph-convolutional aggregation. To address data scarcity, we construct SynMed-VQA, a large-scale synthetic
dataset comprising over 2 million question-answer pairs across 9 imaging modalities and 10 major organs, generated with GPT-4o. Extensive experiments on SynMed-VQA and three other standard biomedical VQA
benchmarks demonstrate that MedFG-VQA achieves competitive or superior performance compared to much larger models while maintaining significantly lower computational costs, highlighting its efficiency and potential for clinical deployment.
Read the full abstract ▾
All authors Haowen Gu, Gensheng Pei, Zeren Sun, Mingwu Ren, Xiangbo Shu, Yazhou Yao, Fumin Shen
Computer Vision
Rethinking Image Processing for the Age of AI: A Problem-First Framework for Scientific Progress
Guoping Qiu
Modern AI has greatly expanded the capabilities of image processing. However, the ready availability of powerful models, public
datasets, and
benchmark leaderboards has also en- couraged a model-first research pattern: researchers increasingly begin with an available architecture and optimize it on a public
benchmark, rather than beginning with the underlying real-world imaging problem. This can produce impressive
benchmark results without necessarily improving our understanding or solution of the real problem. This paper argues for a problem-first approach that distinguishes the physical imaging problem, solution principle, statistical
estimator, and computational implementation, while clarifying what modern AI can achieve and which fundamental problems remain unsolved. Through case studies of super- resolution and low-light enhancement, we show how
benchmark datasets may define tasks that differ substantially from the real-world problems they are intended to represent, and why performance improvements must be interpreted within the conditions under which they are obtained. We propose a six-stage workflow that places problem formulation, image acquisition, information-loss analysis, assumptions, ambiguity, and evaluation before model and
dataset selection. The paper also proposes clearer standards for evidence, reproducibility, uncertainty, and claims of state-of-the-art performance. More fundamentally, it calls for a change in research culture and education so that future researchers learn to understand imaging problems deeply and use modern AI to achieve genuine scientific and technical advancement.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
SAGE: Variate-Wise Semantic Augmentation for Vision-Language Time Series Forecasting
Haizhao Fan and Xinyi Le
Time series
forecasting models operate on raw numerical sequences, lacking the semantic knowledge that domain experts implicitly leverage, such as the physical meaning of each variable, its statistical behavior, and its temporal dynamics. Recent efforts to bridge this gap fall into two camps. Some rely on large
language models at inference time, which is computationally expensive. Others apply uniform textual
prompts at the
dataset level, ignoring the heterogeneous semantics across individual variates. We propose SAGE (Seeing and Augmenting with Grounded Encoding), an end-to-end CLIP-based framework that jointly models temporal, cross-variable, textual, and visual information. The CLIP text encoder processes frequency-enhanced patches and variable
tokens, while gated residual paths inject variable-specific descriptions and statistical descriptors. In parallel, the frozen CLIP
vision encoder aligns rendered series with temporal representations through a training-only contrastive objective. This dual use of CLIP adds complementary semantic and visual supervision without placing an LLM in the
forecasting loop. Across eight long-term
benchmarks and M4, SAGE achieves state-of-the-art accuracy. Ablations confirm complementary gains from
multimodal alignment and variable-level knowledge.
Read the full abstract ▾
Computer Vision
Evaluator-Dependent Patient-Adaptive ECG Lead-Channel Allocation
Xiaoyang Li and Zeyan Tao
The diagnostic value of an ECG channel depends on the model that interprets it. We test whether adaptive acquisition retains its advantage over fixed protocols when that model changes. Two policies developed with a logistic evaluator are frozen and assessed with a masked waveform ResNet1D, using exhaustive, metric-matched fixed comparators. On PTB-XL, replacing the evaluator reverses the mean adaptive advantage in negative log-likelihood and Brier score across the tested budgets, while calibration responds less uniformly. The shift persists with common fixed references and broader training-mask exposure. Training a policy against the stronger evaluator partly recovers the lost advantage. These retrospective exploratory results show that acquisition quality cannot be assessed independently of the downstream evaluator and motivate validating adaptive policies with the model intended for diagnosis.
Read the full abstract ▾
Computer Vision
LLaVAFlow: Preserving Latent Alignment Flow for Parameter-Efficient Multimodal Fine-Tuning
Muyao Yuan, Muyan Jiao, Jiangyong Ying et al. (+5 more)
While
Multimodal Large
Language Models (MLLMs) exhibit strong generalization, visual instruction tuning for downstream tasks inevitably causes catastrophic forgetting, impairing overall generalization. While existing methods regulate weight updates to reduce forgetting, they overlook the fundamental cross-modal
alignment in MLLMs. Based on prior work and our observations, we argue that cross-modal
alignment is implicitly captured in the information-compression trajectory. To preserve the
alignment flow embedded in the trajectory, we propose LLaVAFlow, an information-theoretic
distillation framework. First, we compress the mutual information between the extracted relations and MLLM
embeddings, encouraging a learnable module to produce a refined
alignment flow that benefits downstream tasks. Second, we maximize the mutual information between the extracted
alignment flows of the pretrained and fine-tuned MLLMs, enabling the transfer of compact
alignment information. Extensive experiments show that LLaVAFlow is an effective plug-and-play framework that preserves
alignment flow and enhances both downstream performance and generalization.
Read the full abstract ▾
All authors Muyao Yuan, Muyan Jiao, Jiangyong Ying, Weizhan Zhang, Yuanhong Zhang, Lan Ma, Yuan Gao, Haipeng Du
Computer Vision
Hyperspectral Diffusion Equivariant Imaging (HyDiff-EI): A Self-supervised Framework for Hyperspectral Image Inpainting
Shuo Li, Mike Davies and Mehrdad Yaghoobi
A novel Hyperspectral
diffusion Equivariant Imaging (HyDiff-EI) framework for solving the hyperspectral image (HSI) inpainting problem has been presented here. Unlike conventional
diffusion-based methods that rely on large-scale pretraining, HyDiff-EI is a test-time optimization framework that learns directly from a single corrupted HSI acquisition. This makes it flexible for different sensor configurations and particularly well-suited for practical remote sensing scenarios where large annotated hyperspectral
datasets are limited. To address the ill-posed nature of unsupervised inpainting, we embed equivariant consistency constraints within the
diffusion process. By leveraging the inherent
geometric symmetries and intrinsic characteristics of HSIs, HyDiff-EI bridges the gap between generative
diffusion modeling and self-consistent physical priors. We empirically show that coupling
diffusion modeling with equivariant priors substantially enhances noise robustness and generalizability. Extensive experiments on real-world
datasets including Chikusei, Botswana, and EMIT demonstrate that HyDiff-EI offers remarkable inpainting quality over existing self-supervised and
diffusion-based
algorithms in both noiseless and noisy cases.
Read the full abstract ▾
Computer Vision
Thinking on Shots: Consistent Multi-Shot Video Editing with Agentic Reasoning
Chenyang Wu, Fuchen Long, Binyuan Huang et al. (+4 more)
While generative AI has significantly advanced
video editing, existing methods primarily focus on single-shot or short
video clips. Editing long
videos with multiple instructions remains a formidable challenge. Naive chunking strategies, e.g., fixed-duration
segmentation, often lead to entity fragmentation, severe editing
hallucinations, and disrupted temporal continuity. To bridge this gap, we introduce the Multi-Instruction Multi-Shot Long-
Video Editing (MMLVE) task, which is structured around three core objectives: Cross-Shot Editing Consistency (CSEC), Multi-Instruction Decoupling (MID), and Zero-Destruction on Spatiotemporal Structure (ZDSS). To tackle these three unique challenges, we introduce an agentic editing framework that leverages the synergy of Large
Language Models (LLMs) and
Vision-
Language Models (VLMs) to achieve shot-level
video decoupling and precise instruction parsing. Furthermore, to comprehensively evaluate this task, we construct MMLVE-Bench, which is an MMLVE-focused
dataset characterized by complex real-world spatiotemporal dynamics, high-density heterogeneous instructions, and sparse, random entity distributions. Three MMLVE-focused evaluation metrics are further exploited to assess the quality of the editing results. Extensive experiments demonstrate that our MMLVE-
Agent outperforms existing closed-source SOTA approaches (e.g., Seedance 2.0), successfully eliminating editing
hallucinations, preserving cross-shot editing consistency, and attaining seamless spatiotemporal transitions.
Read the full abstract ▾
All authors Chenyang Wu, Fuchen Long, Binyuan Huang, Xinlong Sun, Xi Chen, Chun-Le Guo, Chongyi Li
Computer Vision
Multi-Image Visual Token Pruning in Large Visual Language Models
Rongyang Zhang, Chengqiang Lu, Cong Li et al. (+9 more)
With the growing demand for processing multiple image sequences in real-world applications, various visual
token pruning methods have emerged to mitigate the computational and context length constraints faced by Large
Vision Language Models (LVLMs). However, most existing
pruning approaches rely on static strategies that struggle to adapt across different architectural LVLMs and multi-image scenarios, and are additionally constrained by their dependence on
attention computations that are incompatible with efficient techniques like FlashAttention. To address these limitations, we propose a training-free, Adaptive Visual
Token Pruning (AVTP) framework, applicable to diverse LVLM architectures. We strategically determine
pruning layers based on empirical analysis of visual
attention distributions across various LVLMs, and implement adaptive
pruning ratios in multi-image contexts where images of higher importance retain proportionally more
tokens. We conduct extensive experiments across different LVLMs to demonstrate the effectiveness and robustness of AVTP. Specifically, Qwen3VL-8B achieves 2 times inference speedup while maintaining 96.1% of its original accuracy on multiple multi-image
benchmarks, InternVL3.5-8B retains 94.1% accuracy, and LLaVA-OV-7B even exceeds its original baseline performance. Our code is available at \href{https://github.com/zry13/AVTP}{this link}.
Read the full abstract ▾
All authors Rongyang Zhang, Chengqiang Lu, Cong Li, Hongchao Gu, Tingjia Shen, Xuyang Zhi, Qimeng Wang, Yan Gao, Yi Wu, Yao Hu, Hao Wang, Enhong Chen
Computer Vision
Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion
Bowen Xue, Brandon Y. Feng, Chenguo Lin et al. (+6 more)
Scaling
video generation to long durations reveals a critical bottleneck: current models lack robust long-term
memory. This deficiency can be studied along two critical aspects: object permanence, the ability to precisely reproduce the appearance of objects upon re-entry; and
memory capacity, the ability to process ultra-long context and use information from distant history. Robust long-term
memory requires both: object permanence without sufficient context handling limits the temporal scope, while long context length without permanence fails to maintain identity. To address this, we present Ring Forcing, an autoregressive
video diffusion framework designed to robustly construct and precisely utilize long-term
memory. Our ring-structured training strategy enforces
retrieval from distant history, effectively reconciling the trade-off between strict historical adherence and generative diversity. To expand
memory capacity, we introduce a compression and timestep composition strategy. Under fixed sequence length constraints, this method extends the effective historical span to minutes-long durations and achieves a comprehensive receptive field over the entire history. Furthermore, we present a sparse RoPE mechanism to enable flexible, scalable
memory adaptation while fully exploiting pre-trained priors. Extensive experiments demonstrate that Ring Forcing achieves superior minutes-long coherence and object permanence, significantly outperforming state-of-the-art methods.
Read the full abstract ▾
All authors Bowen Xue, Brandon Y. Feng, Chenguo Lin, Yuchen Lin, Yujia Zeng, Lvmin Zhang, Maneesh Agrawala, Honglei Yan, Panwang Pan
Computer Vision
Cross-Architecture Knowledge Distillation from a Vision Foundation Model to a Lightweight Visual State Space Model for Tea Leaf Disease Classification
Zibo Zhou, Zongsen Qiu, Rui Chen et al. (+3 more)
Automated tea leaf disease classification supports precision agriculture, yet deploying accurate models on edge devices remains challenging under tight compute budgets. Self-supervised
vision foundation models such as DINOv2 provide strong features but are too large for field deployment, while lightweight models trained from scratch on small agricultural
datasets often underfit. We study cross-architecture knowledge
distillation (KD) from a fine-tuned DINOv2 teacher (
Vision Transformer) to a compact bidirectional Visual State Space Model (LVSSM) student, an underexplored direction because the architectures use fundamentally different
token-mixing mechanisms. We identify and fix two training-stability problems that prevent the from-scratch SSM student from learning on limited data: a single large patch-
embedding convolution and a fusion layer that severs the residual path. With a progressive convolutional stem and gated bidirectional selective-scan block, the 4.45M-parameter student trains stably. Across three seeds, temperature-scaled logit
distillation raises test accuracy from 92.32+/-2.14% to 95.41+/-1.17% (best single run: 96.20%; macro-F1: 94.45%), a +3.09 percentage-point mean gain. The student uses 5.0 times fewer parameters than the 22M-parameter teacher while retaining 98.3% of its accuracy. Ablations show that intermediate feature-
alignment losses reduce accuracy, making simple logit-level KD the strongest configuration. A fair from-scratch comparison shows the gain is specific to students that start below the teacher. We report per-class metrics, confusion
matrices, bootstrap confidence intervals, and FLOPs/latency measurements, and discuss limitations including the single-
dataset scope and simplified non-official SSM implementation.
Read the full abstract ▾
All authors Zibo Zhou, Zongsen Qiu, Rui Chen, Yujie Yao, Yue Zhou, Jianjun Wang
Computer Vision
Glass Surface Detection Grounded in 3D Visual Geometry
Yiwei Lu, Ke Xu, Tao Yan et al. (+3 more)
Glass surface detection (GSD) is critical for scene understanding and reconstruction, and yet remains challenging due to the transparency and reflectivity of glass surfaces. Existing GSD methods typically rely on 2D appearance cues, which may fail in geometrically ambiguous scenes. In this paper, we propose a paradigm shift: grounding GSD in 3D visual geometry to explicitly model the physical existence of glass surfaces. Our method first distills rich 3D priors from the visual geometry grounded
transformer (VGGT) and generates glass-aware 3D representations. It then exploits multi-tasking learning with a novel glass detection head, consisting of two core modules: a Frequency Self-
Attention Module (FSAM) that identifies glass-specific spectral features for glass surface localization, and a Geometry Grounding Block (GeGB) that selectively grounds 2D features in 3D geometry for glass surface
segmentation. Extensive experiments demonstrate that our method achieves state-of-the-art performance across seven standard GSD
benchmarks, generalizes well to
video/multi-modal data, and substantially improves reconstruction in glass scenes. Code is available in https://github.com/YT3DVision/VGGT_GLASS.
Read the full abstract ▾
All authors Yiwei Lu, Ke Xu, Tao Yan, Xiaojun Chang, Radu Timofte, Rynson W. H. Lau
Computer Vision
G2D: Generative-to-Discriminative Collaborative Inference for Zero-Shot Image Classification
Zehua Hao, Fang Liu, Qinliang Wang et al. (+3 more)
Zero-shot classification needs efficient label
retrieval and fine-grained visual
reasoning, yet discriminative and generative
vision-
language models fail in complementary ways.When CLIP's top-1 prediction is wrong, the correct label often remains in its top-$K$ shortlist, making disambiguation rather than recall the key challenge.Standalone generative models, however, are hindered by large label spaces and unconstrained outputs.This complementarity motivates separating broad candidate
retrieval from fine-grained, image-grounded verification.We propose G2D, a training-free framework that uses a generative VLM to verify CLIP-retrieved candidates against the image.Candidate names and CLIP probabilities provide a structured prior for resolving visually similar classes.Fixed confidence routing,
entropy-adaptive candidate sizing, and trie-constrained decoding focus generative
reasoning on uncertain samples and ensure one valid output for each input at test time.Across eight
benchmarks, G2D achieves 68.85% average accuracy, versus 59.35% for CLIP and 63.11% for the standalone VLM.Across seven generator configurations, candidate-set verification improves average accuracy by 1.08–27.42 percentage points.G2D also transfers to DCLIP, WaffleCLIP, and CuPL, supporting a practical interface between discriminative proposal and generative visual
reasoning. Code: https://github.com/Harzva/G2D
Read the full abstract ▾
All authors Zehua Hao, Fang Liu, Qinliang Wang, Yaoyang Du, Xinyan Huang, Puhua Chen
Computer Vision
Generative Semantic Scene Completion
Shi Chen and Weifeng Ge
Outdoor
LiDAR semantic scene completion (SSC) recovers a dense semantic voxel grid from a scan observing 1% of the target volume, under class imbalance beyond 7,000x. We recast SSC as generative semantic scene completion (GSSC): a single discrete-
diffusion formulation in three roles. First, paired sparse-dense scene synthesis (PS$^3$) generates matched sparse
LiDAR observations with their dense semantic completions, addressing the long tail at its source and yielding the PS$^3$-SemanticKITTI corpus we train on alongside SemanticKITTI. Second, semantic-guided generative scene completion (SGSC) generates the scene from noise with multinomial discrete
diffusion, conditioned on the sparse scan through a bird's-eye-view semantic map and a sparse 3D feature stream. Third, the same framework instead refines an existing completion in one flow-matching step: structured source discrete
diffusion (S$^2$D$^2$). S$^2$D$^2$ improves the mIoU of SGSC's own output and every external SSC base tested, without base retraining or test-time adaptation. On the strongest base, one step without test-time augmentation reaches 38.8% mIoU on the SemanticKITTI hidden test. To our knowledge that is the best causal, single-sweep, single-sample result on that leaderboard, +2.1 pp over the previous best published score under the same restriction. Four correction steps with eight-view test-time augmentation reach 39.2%, outside that restriction.
Read the full abstract ▾
Computer Vision
GeoMAD: Geometry-Aware Multi-View Anomaly Detection via Deformable Fusion and Distributional Alignment
Shang-Fu Chen, Jhih-Ciang Wu, Kuan-Chuan Peng et al. (+2 more)
Multi-view anomaly detection (MvAD) detects defects by exploiting complementary observations from multiple camera viewpoints. The central challenge is to fuse views with sufficient
geometric awareness while remaining scalable to multi-class industrial settings. Existing methods typically fall into two extremes: voxel-based fusion provides explicit
geometric alignment but requires costly 3D construction and class-specific assumptions, whereas lightweight patch-based fusion is efficient but relies on discrete candidate matching and lacks continuous cross-view correspondence. In this paper, we propose GeoMAD, a unified multi-view, multi-class AD framework that addresses both
geometric correspondence deficiency and distributional inconsistency. Our Cross-view Deformable Fusion Module (CDFM) learns content-adaptive, view-pair-specific sampling offsets directly on 2D feature maps and arranges them across a multi-scale window pyramid with image-global reference sampling, enabling hierarchical cross-view correspondence without camera calibration, voxel construction, or class-specific 3D supervision. We further introduce Distributional View
Alignment (DVA), a self-supervised cross-view regularization loss that aligns each view's bottleneck distribution against a per-instance view-centric target, enforcing global consistency without pixel-level correspondence. Together, CDFM and DVA bridge local
geometric correspondence and global distributional consistency, providing geometry-aware and distribution-consistent fusion while preserving the efficiency of 2D feature-space learning. Extensive experiments on Real-IAD and MANTA-Tiny show that GeoMAD achieves strong detection and localization performance in unified MvAD.
Read the full abstract ▾
All authors Shang-Fu Chen, Jhih-Ciang Wu, Kuan-Chuan Peng, Wen-Huang Cheng, Kai-Lung Hua
Computer Vision
UniGeo: A Multi-modal Large Language Model for Text-Guided Cross-View Geo-Localization
Jiahao Wen, Hang Yu and Zhedong Zheng
Text-guided drone geo-localization aims to identify a target region in a large-scale image gallery from a natural-
language description. Existing methods mainly formulate this task as direct matching between an open-ended text query and candidate images. However, incomplete queries and highly similar candidates often make global cross-modal matching insufficient for reliable fine-grained localization. We propose UniGeo, a unified
multimodal large
language model (MLLM) for text-guided drone geo-localization. Built on a shared
vision-
language framework, UniGeo jointly supports geo-semantic understanding, cross-view semantic generation, and candidate-level verification. Specifically, it establishes stable correspondences among local scene elements, spatial relations, and
language descriptions through geo-semantic learning, and further models semantic mappings between drone and satellite views through cross-view generation. Based on these capabilities, a plug-and-play verification module performs fine-grained discrimination among highly confusable candidates. We further introduce a multi-stage training strategy that progressively learns geo-semantic understanding, cross-view generation, and candidate verification, improving adaptation to text-guided geo-localization. Experiments demonstrate consistent improvements across multiple
retrieval backbones. On GeoText-1652, UniGeo improves R@10 and mAP by 13.59 and 2.83 percentage points, respectively, validating its effectiveness for fine-grained text-guided drone geo-localization.
Read the full abstract ▾
Computer Vision
Parameter Efficient Continual Learning for Sparse Event-Based Transformers
Vaishnavi Nagabhushana, Kartikay Agrawal and Ayon Borthakur
Robotic and edge intelligence systems operate in dynamic environments where data arrives continuously, requiring models to adapt while preserving previously learned knowledge under strict
memory and energy constraints. While parameter-efficient fine-tuning has shown promise for continual learning with
vision transformers, conventional architectures rely on dense computation and remain costly for real-world deployment. Sparse event-based
vision transformers provide energy-efficient event-driven computation, yet their continual learning capabilities remain largely unexplored. We here introduce sLoTh, a parameter-efficient continual learning framework for pretrained sparse event-based (spiking)
vision transformers. sLoTh freezes the backbone and restricts plasticity to scalable-efficient low-rank
attention updates (seLoRA) and shared neuronal threshold modulation, enabling adaptation without replay buffers by updating less than 1% of model parameters. Experiments across CIFAR-100, Tiny-ImageNet, ImageNet-100, and ImageNet-R with up to 100 tasks demonstrate competitive rehearsal-free performance in class-incremental learning and online continual learning, while enabling approximately 6.5x lower energy consumption than conventional dense
vision transformers.
Read the full abstract ▾
Computer Vision
Beyond Atomic Layouts: Compositional Design Understanding with Vision-Language Models
Yiyang Huang, Zhaowen Wang, Simon Jenni et al. (+4 more)
Layout understanding, or the interpretation of element organization, is essential for document analysis, user interface (UI) creation, and graphic design. While recent
vision-
language models (VLMs) excel at interpreting atomic layouts composed of independent elements, they struggle with compositional layouts that require
reasoning over visually entangled elements within hierarchical multi-layer structures. In this paper, we introduce a new task, compositional layout understanding, and present CoDeLayout, a VQA
dataset of ~20K real-world multi-layer layouts annotated with compositional element pairs and design intent. Through empirical analysis on CoDeLayout, we identify two key challenges for existing VLMs: semantic drift between textual metadata and visual content, and structural ambiguity in hierarchical inter-element relationships. To address these challenges, we propose MASON, a post-training paradigm that integrates
multimodal alignment (MA) and structural perception (SP). MA enhances element interpretation by grounding metadata-defined elements to their visual counterparts, mitigating semantic drift, while SP models layer-aware inter-element spatial relationships to improve hierarchical understanding and reduce structural ambiguity. Experiments reveal substantial gaps in existing VLMs: even the strongest baseline, GPT-o3, achieves only 79.68% accuracy, whereas Qwen2.5-VL 7B with MASON reaches 91.66%. Notably, MASON surpasses full-data Direct Finetune using only 30% of the training data and scales better with additional data.
Read the full abstract ▾
All authors Yiyang Huang, Zhaowen Wang, Simon Jenni, Jing Shi, Yitian Zhang, Yizhou Wang, Yun Fu
Computer Vision
LiveVVT: High-Fidelity Video Virtual Try-On in Real Time
Yushe Cao, Shikun Feng, Ruxiang Duan et al. (+4 more)
Diffusion-based
Video Virtual Try-On (VVT) achieves high visual fidelity through bidirectional spatio-temporal modeling, but complete-clip dependence incurs prohibitive latency and computational overhead in practical continuous deployment. Naively enforcing causality disrupts pretrained bidirectional priors and substantially degrades synthesis quality. We introduce LiveVVT, a rolling streaming
diffusion framework that preserves bounded bidirectional modeling within causal recurrent generation. Within a fixed-size window, LiveVVT jointly denoises multiple
video chunks under bounded look-ahead, preserving local bidirectional interactions while emitting one clean chunk per iteration. Beyond the window, two complementary
memories sustain long-term consistency: a bounded temporal
memory propagates recent dynamics and occlusion context, whereas a persistent global appearance
memory, constructed once from the target garment and a frontal try-on keyframe, anchors garment details and dressed appearance throughout the stream. We further introduce a progressive
distillation framework integrating bidirectional VVT learning, teacher-trajectory regression for causal few-step adaptation, and Collaborative Matching
Distillation, which couples teacher-distribution matching with rolling flow matching on real
videos to align optimization with recurrent inference. Experiments on paired and unpaired long-sequence
benchmarks demonstrate superior generation quality over similarly sized models, with $26\times$ lower latency and $11\times$ higher throughput, enabling high-fidelity real-time streaming VVT.
Read the full abstract ▾
All authors Yushe Cao, Shikun Feng, Ruxiang Duan, Liyong Wang, Dianxi Shi, Chun Yu, Junliang Xing
Computer Vision
AesCanvas: A Large-Scale Dataset and Benchmark for Aesthetic Critique and Contextual Suitability
Xuanwei Hu, Haoyu Dong, Kejun Wu et al. (+2 more)
Recent advances in
Multimodal Large
Language Models (MLLMs) have extended Image Aesthetic Assessment (IAA) beyond scalar scores toward interpretable critique and guidance. Yet existing
benchmarks mainly assess intrinsic visual quality or fixed domain criteria, leaving open whether an appealing image is appropriate for a specific purpose, audience, cultural setting, or domain convention. We introduce AesCanvas, a unified suite with two complementary components: CritiqueCanvas with 519,136 instruction-response pairs from 54,300 images supports long-form, multi-dimensional critique across photography, painting, and virtual imagery, whereas ContextCanvas with 301 expert-reviewed use scenarios evaluates contextual aesthetic suitability in realistic use scenarios. Under a unified protocol, we evaluate closed-source frontier, open-weight general, and aesthetic-specific MLLMs. Results reveal a clear separation between critique generation and context-sensitive judgment: reference-based lexical and semantic metrics only partially capture critique quality, while aesthetic specialists remain competitive on selected critique metrics yet substantially lag strong general-purpose MLLMs on ContextCanvas. Further analyses show that aesthetic specialization does not reliably transfer to contextual suitability and that model decisions may fail to track or ground themselves in decisive contextual visual cues. These findings establish culturally situated, evidence-grounded suitability as a distinct objective for aesthetic modeling.
Read the full abstract ▾
All authors Xuanwei Hu, Haoyu Dong, Kejun Wu, Tianyi Liu, Jianjun Gao
Computer Vision
Domain-Specific Self-Supervised Representation Learning for Retinal Fundus Classification
Bekzat Nurlanbekova and Fung Fung Ting
Despite the growing number of public
datasets, annotated medical images remain scarce. Supervised learning methods achieve strong performance on many
benchmarks, however require large amounts of labeled data, which are costly and time-consuming to obtain in the medical domain. To address this limitation, contrastive self-supervised learning (SSL) has emerged as a promising alternative for learning useful representations from unlabeled data. In this work, we investigate two SSL frameworks, SimSiam and SimCLR, for retinal disease classification from fundus images. We focus on understanding how augmentation strategies and training parameters influence representation learning under resource-constrained settings. Given limited data and computational capacity, we explore the feasibility of training SSL models with small batch sizes incorporated with retinal-specific augmentation techniques. Through a series of experiments, we assess the quality of learned representations via linear evaluation and fine-tuning across downstream tasks, including multi-disease classification and diabetic retinopathy grading. Our results show that tailoring augmentation strategies to the characteristics of retinal images plays a critical role in improving performance. Even under constrained settings, lightweight SSL frameworks can learn transferable representations that reduce dependence on large annotated
datasets and achieve competitive results.
Read the full abstract ▾
Computer Vision
Reason in the Words You Speak: Idiolectal Paraphrasing Off-Policy Traces for Reasoning Distillation in VideoLLMs
Ji Soo Lee, Jinyoung Park, Seohyun Lee et al. (+4 more)
Recent large
language models achieve strong performance on complex
reasoning tasks, where reinforcement learning with Group Relative Policy Optimization (GRPO) has emerged as a leading paradigm for optimizing models on self-generated trajectories. However, the on-policy nature of GRPO bounds the model to the
reasoning skills it can already produce, restricting to learn more advanced capabilities. Prior works inject privileged
reasoning traces from a stronger teacher policy to guide training, yet these traces are inherently out of distribution with respect to the student policy. We observe that this mismatch between on-policy and off-policy causes gradient clipping on semantically critical
reasoning tokens, ultimately rewarding correct answers while leaving the
reasoning that justifies them unlearned. Hence, we propose Echo-GRPO, a framework that lets the model reason in the words it speaks. Rather than imitating low-probability privileged traces from the teacher model, Echo-GRPO rewrites them into the student policy's own idiolect, that is, its own characteristic vocabulary and expression patterns, while preserving their semantics via Dual-Reference Decoding. We instantiate this framework as VideoEcho-R1 for
video reasoning distillation, achieving consistent improvements across three
multimodal LLM backbones and five
benchmarks. Finally, we show that our idiolectal paraphrasing is a plug-in module that consistently improves both RL and supervised fine-tuning frameworks for
reasoning distillation, demonstrating that policy-aligned supervision extends beyond GRPO.
Read the full abstract ▾
All authors Ji Soo Lee, Jinyoung Park, Seohyun Lee, Jongha Kim, Joonmyung Choi, Jinsung Yoon, Hyunwoo J. Kim
Computer Vision
RECAP-Forcing: Retaining Content Appearances for Long Video Generation
Haiyang Xu, Zheng Ding and Zhuowen Tu
Long autoregressive
video generation faces a fundamental
memory challenge: with a finite
attention window, a model must decide which information from an ever-expanding history to retain. Existing methods organize
memory temporally, preserving recent frames while compressing or discarding older ones. We instead propose RECAP-Forcing, organizing
memory by appearance novelty. A long
video is not merely a sequence of frames, but an evolving cast of subjects, objects, and scenes whose identities must remain consistent over time. We organize
memory by retaining the KV cache associated with newly appearing content–such as entering subjects, disoccluded regions, and newly introduced scenes–at the moment it first becomes visible, prioritizing novelty over recency.
Memory should scale with the amount of newly introduced content, rather than with
video length. This appearance-indexed
memory makes long-range consistency an explicit property of the
memory structure. Our framework unifies two mechanisms under this single principle. At the beginning of a
video, when all visible content is novel, an
attention sink preserves the initial scene. As the
video evolves, an optical-flow-based novelty bank extends the same principle by selectively retaining newly revealed content. As a training-free inference method with no additional learnable parameters, RECAP-Forcing consistently improves visual quality and semantic fidelity across multiple strong baselines and outperforms existing
memory methods.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Beyond the Proving Ground: Independent Public-Road Testing of Assisted Lane Change Systems using LiDAR
Marcello Cellina, Akos Kriston, Antonio Migneco et al. (+7 more)
Testing of commercial Advanced Driver Assistance Systems is essential to ensure safety and compliance during type approval and in service operation. However, proving ground scenarios may not reflect real world driving complexity, while geo fencing can require manufacturer collaboration and limit assessment independence. This work presents a methodology for independently testing Assisted Lane Change systems on public roads. A campaign on the A31 French motorway used a test vehicle equipped with a
LiDAR based vehicle detection and tracking system. Tests covered combinations of inter vehicle distance and speed between the test vehicle and the take over vehicle. Real time kinematic global
navigation satellite system receivers assessed detection and tracking performance. Recorded lane change trajectories were compared with the lane change suppression requirements of UNECE Regulation Number 79. Of 27 predefined lane change manoeuvres, 18 were completed and 9 suppressed. In 6 cases, the system allowed manoeuvres that did not meet regulatory minimum distance requirements. In 3 cases, the deviation remained statistically significant after accounting for measurement uncertainty. To the authors knowledge, this is the first public road campaign designed to assess Assisted Lane Change compliance with Regulation Number 79 safety distance requirements. The results demonstrate the suitability of
LiDAR based sensing for this purpose. The methodology can support
market surveillance and future regulatory revisions by revealing real world behaviours not covered by approval procedures.
Read the full abstract ▾
All authors Marcello Cellina, Akos Kriston, Antonio Migneco, Davide Maggi, Stefano Favelli, Fabrizio Re, Fabrizio Minarini, Andrea Nuovo, Riccardo Dona, Biagio Ciuffo