Computer Vision · Cross-listed from eess.IV
Patient-Agnostic Synthetic Pretraining for Efficient Patient-Specific Intraoperative 2D/3D Registration
Minheng Chen and Youyong Kong
Intraoperative 2D/3D registration aligns preoperative CT volumes with intraoperative X-ray or fluoroscopic images and is essential for image-guided interventions. Recent learning-based and differentiable registration methods have shown promising accuracy, especially in patient-specific settings where abundant digitally reconstructed radiographs (DRRs) can be synthesized from the target CT. However, training a separate patient-specific model from scratch for every new patient is computationally inefficient and limits practical deployment. In this work, we propose an efficient patient-specific 2D/3D registration framework based on patient-agnostic synthetic pretraining and spherical similarity learning. The model is first pretrained on synthetic DRRs generated from multiple CT volumes to learn transferable pose-sensitive representations, and is then adapted to a new patient using only a limited number of synthetic projections from the target CT. To improve synthetic-to-real robustness without requiring anatomical labels, we introduce a
segmentation-free domain randomization strategy that perturbs image intensity, projection physics, field-of-view, occlusion, and fluoroscopic artifacts. The adapted model provides an initial pose estimate, which is further refined using spherical similarity learning and differentiable Levenberg-Marquardt optimization. Experiments on multiple anatomical
datasets evaluate whether patient-agnostic synthetic pretraining can improve the efficiency of patient-specific registration, with particular focus on the trade-off between adaptation cost and registration accuracy. The results demonstrate that patient-agnostic synthetic pretraining can significantly reduce patient-specific training requirements while preserving accurate intraoperative 2D/3D registration.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Stabilizing Deep Reconstruction Operators with Contractive Anchoring
Arghya Sinha, Trishit Mukherjee and Kunal N. Chaudhury
Pretrained deep denoisers can be used to solve a wide range of model-based image reconstruction tasks via Plug-and-Play (PnP) and Regularization-by-Denoising (RED)
algorithms, without retraining per task. These denoisers are trained only for single-step denoising. Using them as Image Reconstruction (IR) regularizers in an iterative process can destabilize reconstruction. A common failure mode is the peak-and-collapse behaviour: metrics such as PSNR improve for early iterations and then abruptly degrade, making these
algorithms unreliable in practice. We propose a data-driven stabilization framework that (i) formalizes this instability of any IR operator through a local quantity and (ii) prevents collapse by regularizing this quantity adaptively, requiring no retraining or modification of the given pretrained network. Our key idea is to control the potentially unstable IR operator with a contractive operator whose stable iterates act as an anchor and prevent collapse. We further introduce an efficient family of trainable contractive operators that serve as strong anchors while remaining lightweight. Extensive experiments across proximal
algorithms, denoiser architectures, noise levels, and imaging tasks show consistent, collapse-free performance and improved reliability of PnP and RED reconstruction.
Read the full abstract ▾
Computer Vision
The Gate Always Closes: On Injecting Auxiliary Signals into Frozen Vision-Language Models
Moshiur Farazi, Sameera Ramasinghe, Bekir Sait Ciftler et al. (+2 more)
Auxiliary signal pathways in VLMs are routinely fitted with learnable gates so the optimiser can decide how much of the signal to admit. We find that the optimiser almost always decides on zero: across five injection designs, every gated pathway becomes behaviourally closed, with accuracy invariant to ablating the pathway at inference even when the gate parameter would nominally pass 30-45% of the signal. We attribute this suppression phenomenon to two regimes, a dead-gradient regime formalised through the caption-invariance of image-derived signals, and a negative-utility regime in which the auxiliary signal actively hurts the loss. Rather than fight suppression, we exploit it: we regularise LoRA fine-tuning with
geometric auxiliary losses from
hyperbolic visual relational graphs (IoA-driven entailment cones and angular repulsion on the Lorentz
manifold), coupled only through the forward pass at training time and dropped at inference. Disaggregating GQA by question type exposes a clean dissociation. Three configurations without
geometric losses at inference lose 2.85-3.39pp on relational questions while gaining ~1pp on attribute questions; a fourth that trains with the losses but infers through a soft
prompt loses 5.14pp on rel for only +0.23pp on attr, so training-time regularisation alone does not protect relational accuracy without a
geometric inference pathway. Configurations that keep the
geometric pathway at inference preserve vanilla-level relational accuracy and match the attribute gain. Out of distribution on VSR, the RMS-prefix recipe preserves the spatial signal; stripping the
geometric losses (G2) collapses VSR by 4.6pp, isolating them as the OOD source. A secondary result:
embedding-norm
alignment is necessary for generation-safe prefix injection, and learnable gates should be replaced with fixed, non-optional injection at matched scales.
Read the full abstract ▾
All authors Moshiur Farazi, Sameera Ramasinghe, Bekir Sait Ciftler, Mahbub Ahmed Turza, Shafin Rahman
Computer Vision · Cross-listed from eess.IV
A Reference-Free Framework for Evaluating Single-Frame ISP Pipelines
Yujin Cho, Sira Ferradans, Jean-Michel Morel et al. (+2 more)
Evaluating camera image signal processing (ISP) pipelines requires measuring low-level artifacts introduced by operations such as denoising, demosaicing, tone mapping, and compression. Blind image quality assessment (IQA) techniques can grade visual quality without a reference, but they typically focus on semantic and high-level visual cues or human perceptual scores rather than the low-level image-processing artifacts introduced by camera pipelines. In contrast, full-reference metrics such as PSNR and SSIM measure pixel-level differences and structural similarity, while LPIPS measures perceptual similarity in deep feature space. However, these metrics require perfectly aligned image pairs, which are difficult to collect in practical settings. We propose a reference-free learning framework that estimates full-reference image quality metrics from a processed sRGB image and its ISO metadata. Our method predicts a proxy sRGB reference, which is then compared with the processed image to compute PSNR, SSIM, and LPIPS in their standard full-reference form. Our experiments show that the proxy-reference model can be learned from synthetic data and applied to real camera data. We further show that lightweight LoRA fine-tuning enables efficient adaptation when ISP components or pipeline configurations are changed. The proposed method outperforms direct metric regression in estimating metric values and achieves higher agreement with full-reference rankings than conventional blind IQA methods. These results demonstrate the feasibility of reference-free estimation of full-reference metrics for practical camera-pipeline evaluation.
Read the full abstract ▾
All authors Yujin Cho, Sira Ferradans, Jean-Michel Morel, Gabriele Facciolo, Thomas Eboli
Computer Vision · Cross-listed from cs.CR
Hiding in Plain Sight: An Effective Physical Adversarial Patch Attack against Visual-Infrared Fused Face Detection
Qiucheng Yu, Tao Ni, Yihe Zhou et al. (+2 more)
Deep learning-based visual-infrared fused face detection models are increasingly deployed across a wide range of applications, yet they remain susceptible to
adversarial patch attacks. Most prior attacks target either the visual or the infrared image alone in the digital domain, which renders them ineffective against fused models in the physical world. Moreover, many of these methods are readily noticeable, as their patch patterns deviate substantially from those seen in the real world. In this paper, we introduce VIPatch (Visual-Infrared Patch), a novel physical
adversarial patch attack that produces inconspicuous, realistic, and natural-looking patches for facial images. Specifically, VIPatch crafts a gradient-color mask together with a band-aid sticker across both the visual and infrared images, and jointly optimizes these two elements; the resulting digital patches further guide the fabrication of their physical counterparts. Experimental results show that VIPatch achieves competitive attack success rates (over 90%) in both the digital and physical domains, while keeping the patches unobtrusive to human observers.
Read the full abstract ▾
All authors Qiucheng Yu, Tao Ni, Yihe Zhou, Jiayimei Wang, Qingchuan Zhao
Computer Vision
What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features
Chen-Yi Lu, Yueh-Shao Chen and Somali Chaterji
Contrastive
vision-
language models such as CLIP map semantically opposite phrases (e.g., "a dog" vs. "not a dog") to nearly identical
embeddings, rendering them insensitive to negation. We attribute this failure to a phenomenon we call Representational Collapse: by tracking compositional divergence and visual
alignment across the CLIP text encoder, we show that middle layers build compositional syntax, but the final layers collapse this structure as visual
alignment rises, producing a syntax-blind final representation. To recover the lost negation signal without altering pretrained weights, we propose PeakPatch, a lightweight post-hoc correction system that intercepts the encoder at its compositional peak. An
Embedding Correction Network (ECN) uses cross-
attention to extract a negation-specific signal from the peak layer, anchored to a stable baseline, and predicts a deviation vector that re-injects the lost syntax into the final-layer
embedding space. A complementary Score Correction Network (SCN) predicts bounded scalar score offsets for discriminative tasks. Both modules are trained jointly end-to-end while all CLIP parameters remain frozen, adding only 5.2M parameters (3.5% of the backbone) and preserving the standard cosine similarity interface. On NegBench, PeakPatch achieves 74.3% on COCO MCQ (+35.1 over CLIP, +17.8 over the best encoder fine-tuning method) and 65.5% on VOC MCQ, while outperforming all fine-tuning baselines on fully out-of-distribution negation
retrieval despite training only 3.5% of the parameters. The corrected
embeddings also transfer to text-to-image generation (+18.4 negation score) and generalize across ViT-B/32, ViT-L/14, and SigLIP backbones. Project URL: https://stevencylu.github.io/PeakPatch/.
Read the full abstract ▾
Computer Vision
WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation
Yuhui Zeng, Wang Chen, Jinfa Huang et al. (+5 more)
Existing Large
Vision-
Language Models (LVLMs) struggle with long-form
video understanding due to the quadratic computational cost of visual
tokens. While recent efficient methods attempt to compress
tokens via hard
pruning or uniform merging, they operate strictly in the spatial feature domain, where robust structural context and discriminative semantic details are inherently entangled. In this work, we propose WaveZip, a joint signal-frequency-domain framework for efficient
video inference. Driven by the insight that temporal redundancy resides in low-pass approximation scales while spatial saliency strongly correlates with high-frequency components, WaveZip leverages Discrete Wavelet Transforms (DWT) to disentangle these signals. Temporally, it employs 1D DWT to analyze query-frame relevance, and the resulting high-frequency coefficients are further gated by inter-frame differences, with both signals jointly driving the dynamic allocation of a precise frame-level
token budget. Spatially, a 2D DWT decomposes features into low-frequency approximations and high-frequency detail components, where the high-frequency coefficients are modulated within query-salient regions to regulate spatial reconstruction. Importantly, WaveZip requires no task-specific training and can be seamlessly integrated into off-the-shelf LVLMs to boost inference efficiency. Extensive experiments on long
video understanding
benchmarks demonstrate that WaveZip retains 99.6% of the full performance under an extreme 10x compression ratio, consistently outperforming state-of-the-art methods.
Read the full abstract ▾
All authors Yuhui Zeng, Wang Chen, Jinfa Huang, Tianyu Xie, Yongdong Luo, Jiayi Ji, Xiawu Zheng, jiebo Luo
Computer Vision
SARATR-X-v2: Scale-Aware Structural Pre-Training for SAR Foundation Models
Weijie Li, Yafei Song, Yongxiang Liu et al. (+7 more)
Masked image modeling has become a dominant paradigm for SAR pre-training, yet the design of the reconstruction target remains fundamentally unsettled. This article argues that a SAR pre-training target should satisfy two conditions to produce transferable representations: (i) physics-grounded stability, i.e., approximate invariance of the target operator to multiplicative speckle inherent in coherent imaging; and (ii) semantic scale compatibility, i.e., coverage of the heterogeneous spatial scales that downstream tasks demand. These two conditions are individually achievable but jointly difficult: physics-grounded stability favors fixed operators, while semantic scale compatibility favors data-driven composition. To this end, SARATR-X-v2 reconciles both within a single design. The target is constructed through fixed structural extractors spanning six receptive fields, from blind-spot local aggregation to directional log-ratio region contrast, and fused via learnable weights into one unified supervision signal for masked reconstruction. On twelve SAR
benchmarks across classification, detection, and
segmentation, SARATR-X-v2 achieves state-of-the-art transfer performance. Under synthetic speckle variation, the proposed target reduces perturbation drift in the learned supervision by nearly two orders of magnitude relative to pixel-space supervision. Taken together, these results support physics-grounded stability and semantic scale compatibility as a principled framework for pre-training target design under coherent imaging, and suggest that effective SAR pre-training is not about reconstructing more signal, but about reconstructing the right structural target.
Read the full abstract ▾
All authors Weijie Li, Yafei Song, Yongxiang Liu, Bowen Peng, Jie Zhou, Jingyuan Xia, Wei Yang, Tianpeng Liu, Zhen Liu, Li Liu
Computer Vision
A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions
Zhijiang Tang, Jiaxin Qi, Kaihua Tang et al. (+2 more)
Image captioning is a primary task in
vision–
language research, yet assessing how faithfully a caption preserves image semantics without relying on reference captions remains unsettled. Prevailing evaluations rely on human-annotated references, whose content reflects annotator intent and captioning proficiency. In this paper, we study a reconstruction-based principle for caption evaluation: a caption is as good as its capacity to enable reconstruction of the original image. However, because captioning inherently compresses visual information, it is impossible to recover all details, and pixel-wise comparison between reconstructed and source images is neither feasible nor meaningful. Through our in-depth analysis of the nature of captions, whose fundamental purpose is to transmit the semantic content of an image, we propose a revised principle: a caption is as good as its capacity to enable a reconstruction that is semantically equivalent to the original. To assess semantic equivalence, we test whether the reconstruction matches the original image across a suite of downstream
vision–
language tasks, yielding a reference-free, task-conditioned caption score. We characterize component-dependent limitations and introduce the lower-cost Captioning Turing Test
Dataset (CTTD) surrogate.
Read the full abstract ▾
All authors Zhijiang Tang, Jiaxin Qi, Kaihua Tang, Yuhua Zheng, Jianqiang Huang
Computer Vision
BoneAgeTW2: Automated Skeletal Maturation Assessment via the Tanner-Whitehouse 2 Method, Deep Learning, and Clinical Report Generation with Distribution Curves
Juan Manuel Castillo Pinto
We present BoneAgeTW2, the first fully open-source system to automate the complete Tanner-Whitehouse 2 (TW2) clinical protocol for skeletal maturity assessment end-to-end. The system employs YOLOv8 for precise detection and localization of the 20 TW2 hand bones from radiographic images, and an EfficientNet-B3 backbone with 20 independent classification heads to assign maturation stages (A-I) to each bone simultaneously. From these predictions, the system automatically generates clinical PDF reports including interactive Gaussian distribution curves for all 20 bones, enabling direct comparison with population norms. The model is trained on the public RSNA Pediatric Bone Age Challenge
dataset (12,611 hand radiographs) using a pseudo-labeling strategy to derive per-bone stage labels from global bone age annotations. The full codebase is publicly available at https://github.com/jmmana/BoneAgeTW2.
Read the full abstract ▾
Computer Vision
Counterfactual Motion Reliability Learning for Robust UAV Tracking
Yuehai Chen
Infrared unmanned aerial vehicle (
UAV) tracking is challenging because the target is often small, low-contrast, and easily confused with thermal distractors or cluttered backgrounds. Recent
Transformer-based trackers have achieved promising performance by learning strong appearance representations, but their responses can still be dominated by background structures when the target appearance is weak or ambiguous. A natural solution is to introduce temporal motion cues. However, in infrared
UAV tracking, motion cues are not always reliable: camera jitter, dynamic backgrounds, sensor noise, and target disappearance may produce temporal variations that are stronger than the true target motion. Therefore, the key challenge is not simply how to use motion, but how to distinguish target-consistent motion from background-induced pseudo motion. To this end, we propose CMRTrack, a counterfactual motion reliability learning framework for robust infrared
UAV tracking. CMRTrack first extracts temporal evidence from adjacent search regions using a lightweight motion evidence encoder. During training, a counterfactual target-erased history branch is introduced to construct hard motion references, encouraging the motion encoder to learn reliable target-consistent motion rather than arbitrary temporal changes. The learned motion evidence is then incorporated into a one-stream tracking framework through motion-guided
token modulation and reliability-aware score fusion, enabling adaptive feature enhancement and response refinement. Extensive experiments on Anti-
UAV410 demonstrate that CMRTrack consistently outperforms representative state-of-the-art trackers and significantly improves the OSTrack baseline, with ablation studies and qualitative analysis verifying the effectiveness of the proposed counterfactual motion reliability learning.
Read the full abstract ▾
Computer Vision
Out-of-Length Scene Text Recognition: A Two-Axis Diagnosis and a Training-Free Fix
Zobeir Raisi and John Zelek
Scene Text Recognition (STR) models are trained almost exclusively on word crops of at most 25 characters, yet real deployments (signage, product labels, dense captions) require reading much longer text. This paper diagnoses that failure and then closes it. The diagnosis separates out-of-length failure into two simultaneously extrapolating axes (the encoder's width axis and the decoder's time axis) and shows that encoder width, not decoder length, is the dominant failure mode. Representation-side fixes bring only partial relief: training-free rotary rescalings recover at most 2-4 points of character error rate (CER), and a weighted fine-tuning recipe recovers 6-8 points while improving standard-
benchmark accuracy, yet word accuracy on the Long Text
Benchmark (LTB) stays near zero, because the residual gap lies in the decoding mechanism rather than the representation. We then close that gap at inference time, on an unmodified word-level checkpoint: the long image is sliced into overlapping crops at the model's training width, each decoded independently and in-distribution, and the reads stitched by geometry-anchored edit-distance
alignment. This procedure reaches 42.79-43.05% bucket-average word accuracy on LTB across two base checkpoints, matching the published state of the art (41.57%) and beating it by 11-12 points on the hardest bucket, at wall-clock parity with plain decoding; applied unchanged to the public PARSeq checkpoint it reaches 47.11%. Once chunking is applied fine-tuning no longer helps: the decoding-side fix alone matches purpose-built architectures. We release the diagnosis harness and implementation.
Read the full abstract ▾
Computer Vision
OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
Jinsen Su, Yongdong Luo, Yuexiao Ma et al. (+3 more)
Existing
token compression methods for omnimodal large
language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and
video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free
token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and
video. OmniScope allocates modality-specific
token budgets, prunes visual
tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio
tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio-
video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall
token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU
memory reduction, with only a 0.35-point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC-AutoML/OmniScope.
Read the full abstract ▾
All authors Jinsen Su, Yongdong Luo, Yuexiao Ma, Yibo Hu, Meiguang Jin, Xiawu Zheng
Computer Vision
Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design
Shenghao Yang, Hongtao Zhang, Yuhan Yi et al. (+6 more)
AI-generated content (AIGC) has made significant progress, with 2D generative models becoming ready-to-use tools for the digital fashion industry. However, 3D garment generation remains in its nascent stage, where in the realm of fashion, the semantic information of diverse design elements exhibits intricate coupling relationships in 3D representations, posing substantial challenges for generating diverse 3D garments. In this work, to handle the above problem, We introduce Fashion-3DLR, a novel 3D garment generation framework that utilizes diverse design elements to create high-quality, versatile 3D garment assets. Specifically, to bridge the semantic gaps between different fashion elements, we propose a Garment Feature Fusion
Diffusion Transformer (GFF-DiT) module to integrate 2D fashion design elements, e.g., sketch and texture, into latent space. Within the latent space, we then employ a rectified flow
transformer to generate geometry latents, which can be decoded into various 3D garment representations, including 3D Gaussians and meshes. Furthermore, we integrate Fashion-3DLR into downstream tasks, achieving the 3D Gaussian Splatting (3DGS)-driven cloth physical simulation and mesh-based virtual try-on. Experimental results indicate that Fashion-3DLR surpass the previous state-of-the-art methods, which verify that the proposed work can generate well-structured, non-watertight garments capable of physical simulation and virtual try-on, underscoring its potential as a versatile 3D garment design tool.
Read the full abstract ▾
All authors Shenghao Yang, Hongtao Zhang, Yuhan Yi, Zhihao Tang, Zihao Cui, Lian Wen, Han Yan, Yuan Gao, Mingbo Zhao
Computer Vision · Cross-listed from cs.CE
A Scale-adaptive Vision Model Links C. elegans Neuronal Morphology to Behavior for Neurotoxicity Assessment
Haochao Ying, Shenchong Lv, Yutao Sun et al. (+8 more)
Neurological disorders are a leading cause of global disability and are increasingly linked to environmental chemical exposures. Yet neurotoxicity assessment still relies on hand-scored morphological readouts that are subjective and poorly predictive of behavioral outcomes. Caenorhabditis elegans provides a genetically tractable, 3R-compliant alternative, but quantifying neuronal phenotypes from confocal microscopy at scale remains computationally challenging: existing
vision foundation models, trained on natural or radiological images, cannot resolve the sparse signals and multi-scale lesions of neuronal imaging. Here, we introduce a dedicated self-supervised
vision model for C. elegans dopaminergic
neurons, together with CeNeuMorph, a multi-grained confocal
benchmark of 27,117 annotated images. Specifically, moving beyond standard Masked Autoencoders, we propose a scale-adaptive masked image modeling strategy that jointly learns representations across resolutions and patch sizes under a fixed
token budget. By decoupling structural semantic learning from rigid grid constraints, the model effectively resolves the full spectrum of neurodegenerative lesions - ranging from fine dendritic beading to gross soma shrinkage - within a tractable computational framework. Finally, our model surpasses both generalist and biomedical foundation models across classification,
segmentation and detection tasks. Fusing visual features with morphological descriptors enables prediction of dopamine-dependent behavioral deficits ($R^2=0.498$). Screening 180 agrochemicals, we identify the benzimidazole moiety as a previously unrecognized determinant of dopaminergic neurotoxicity. Together, the work demonstrates how scale-adaptive self-supervised learning can connect morphology to function for a scalable alternative to mammalian in vivo models for neurotoxicity assessment and drug discovery.
Read the full abstract ▾
All authors Haochao Ying, Shenchong Lv, Yutao Sun, Zijian Tu, Xufeng Jin, Yuyang Xu, Yizhe Wang, Wei Yang, Xiaomin Yue, Jian Wu, Peilin Yu
Computer Vision
Yihao Wu, Chenyi Xu, Liqi Yan et al. (+6 more)
Vision-and-
Language Navigation in continuous environments (VLN-CE) requires an
agent to ground
language in
egocentric observations and plan in unseen scenes. Although recent
multimodal large models and world-model-based methods have improved
navigation, they often preserve excessive task-irrelevant detail, weakening generalization and increasing computational burden. We propose BrainNav, a
navigation framework grounded in the Principle of Minimal Sufficiency. BrainNav consists of three components: a Logical Anchor Model that implements instruction-aware selective perception to suppress environmental noise, a Minimalist Constraint
Alignment module that serves as a compact cross-modal bottleneck, efficiently synchronizing discrete linguistic intent with continuous latent dynamics while filtering out redundant information, and a Compression World Model that predicts action-conditioned states within a condensed, low-rank latent space. These modules align semantic intent with spatial perception, enhancing the
agent's robustness and efficiency in complex tasks. Experiments show that BrainNav improves over prior SOTA by 2.0 % / 1.0 in SR/SPL on R2R-CE val-unseen and 0.94 % / 0.78 on RxR-CE val-unseen. These results indicate that minimally sufficient world representations provide an effective foundation for robust VLN.
Read the full abstract ▾
All authors Yihao Wu, Chenyi Xu, Liqi Yan, Chenhuan Cai, Geyong Min, Bin Lin, Fangli Guan, Jianhui Zhang, Pan Li
Computer Vision · Cross-listed from cs.AI
CachedSearch: Training-Free Cached Exploration for Test-Time Search in Video Diffusion
Shreshth Saini, Neil Birkbeck, Yilin Wang et al. (+2 more)
Test-time search lets small
video diffusion models rival larger ones, but costs 2-10x more. All candidates are fully denoised, although most are discarded. Training-free caching makes each rollout 2-3x faster at near-lossless quality. Composition is safe only if lossy caching preserves verifier rankings. We present the first study of whether caching corrupts candidate ranking in
video test-time search. On Wan2.1-T2V-1.3B with an adaptive caching wrapper (~2x per-candidate speedup), ImageReward scores seed-matched cached and full rollouts. Median per-
prompt Spearman rank correlation is 0.905, with 72% top-1 agreement on the VBench suite. VBench-2.0 replicates this result on a harder suite. Recomputing the cached winner at full compute retains 90-94% of the full-search gain. Errors cluster among near-tied candidates, making corruption self-limiting. This finding leads to CachedSearch. It explores every candidate with aggressive caching, then re-generates only the winner at full compute. At N=8, it captures 94.7% of best-of-N's gain at 63% of the cost. Capture rises with width. At matched budget, it searches twice as wide for 38% more gain. The result holds from 1.3B-14B across six models and four families: Wan, LTX, CogVideoX, and Hunyuan. Wan2.1-14B matches the 1.3B model's fidelity. Mid-trajectory
pruning multiplies the exploration saving to 3.11x at 88.6% capture. Ports to other model families require recalibrating a single parameter, showing that fidelity tracks architecture rather than parameter count. CachedSearch is training-free, verifier-agnostic, and orthogonal to the search
algorithm, making it a plug-in multiplier for test-time scaling.
Read the full abstract ▾
All authors Shreshth Saini, Neil Birkbeck, Yilin Wang, Balu Adsumilli, Alan C. Bovik
Computer Vision
DispatchRAG: Grounding Emergency Dispatch Decisions in Real-World Protocols from Traffic Accident Video
Muhammad Sulthan Adhipradhana, Ehsan Javanmardi, Naren Bao et al. (+1 more)
Assessing the severity of a traffic accident scenario is important to decide which emergency service to dispatch. Missing an ambulance dispatch on a pedestrian accident is a fatal issue that can lead to death. Recently,
Vision-
Language Models (VLMs) have been a promising tool for accident
reasoning, yet many VLMs are not grounded in real-life accident response protocols, making them not usable in accident severity assessment off-the-shelf. We introduced DispatchRAG, an accident assessor and dispatcher framework grounded in real-life Japanese traffic-accident response protocols, designed to enhance VLMs to generate an appropriate emergency response during an emergency scenario. Utilizing a RAG-based
retrieval mechanism to retrieve the most relevant accident protocol and an LLM-powered reasoner to suggest the most proper response. To support evaluation, we introduce Accident Dispatch
Dataset, a comprehensive
dataset of accident assessment and emergency response according to Japanese accident response protocols adapted from the MM-AU
dataset. We validate our framework on the Accident Dispatch
Dataset, showing strong performance across various accident scenarios compared to the baseline VLM, pointing toward integration in autonomous vehicles that can automatically report both their own and nearby accidents.
Read the full abstract ▾
All authors Muhammad Sulthan Adhipradhana, Ehsan Javanmardi, Naren Bao, Manabu Tsukada
Computer Vision · Cross-listed from cs.GR
Occlusion-Point Reuse for Ray-Traced Ambient Occlusion and Shadow
Haojie Jin, Fujia Su, Zehui Lin et al. (+9 more)
Ambient occlusion (AO) and soft shadows are critical visibility cues for spatial perception in real-time rendering. Hardware ray tracing provides a direct way to evaluate these effects, enabling ray-traced AO and area-light shadows that avoid many limitations of screen-space AO and shadow mapping. However, real-time budgets allow only a few rays per pixel, leaving raw ray-traced estimates noisy and expensive. We present an occlusion-point reuse framework that reuses traced samples in the domain of first-hit occlusion points instead of directly reusing final shading values or light samples. This provides a ray-reuse formulation for AO, rather than merely filtering or reusing completed AO values. The key idea is to transform AO and area-light shadow
estimators into occluder-domain integrals, then combine neighboring occluder samples with a multiple-importance-sampling (MIS) formulation. For both AO and shadows, we derive unbiased
estimators that validate convergence to the transformed integrals, as well as biased
estimators designed for practical real-time execution. The biased variants assume local first-hit occluder consistency; for shadows, this occluder-based assumption better matches local visibility geometry than the visibility-consistency assumption commonly used when reusing light samples. Experiments show higher AO and shadow quality than non-reuse ray-traced baselines, and better shadow quality than light-sample reuse at comparable cost.
Read the full abstract ▾
All authors Haojie Jin, Fujia Su, Zehui Lin, Chenxiao Hu, Jierui Ren, Yuqing Yuan, Yanchen Zhang, Zhongtao Wang, Yisong Chen, Kangying Cai, Guoping Wang, Sheng Li
Computer Vision
SHReg: Strictly Rotation-Equivariant Point Cloud Registration via Spherical Harmonics
Chongjian Wang and Junjie Gao
Point cloud registration critically depends on local features that are both distinctive and robust to arbitrary 3D rotations. Existing learning-based methods typically approximate rotation invariance via fragile local reference frames or extensive data augmentation, providing only empirical invariance and often degrading under unseen rotational transformations. In this paper, we propose SHReg, a strictly rotation-equivariant point cloud registration framework grounded in the representation theory of $SO(3)$. By representing local
geometric features as irreducible representations of $SO(3)$, SHReg guarantees exact equivariance under arbitrary rotations without relying on local reference frames. Built upon a spherical-harmonics-based equivariant backbone, SHReg jointly learns rotation-invariant descriptors for robust correspondence matching and rotation-equivariant features that preserve fine-grained orientation information. The preserved equivariant structure enables each correspondence to directly hypothesize a rigid transformation, reducing reliance on large-scale hypothesis sampling in conventional RANSAC-based pipelines and leading to improved robustness under challenging rotational variations. Extensive experiments on 3DMatch, 3DLoMatch, and KITTI demonstrate that SHReg consistently outperforms state-of-the-art methods in registration accuracy, particularly under large rotational perturbations.
Read the full abstract ▾
Computer Vision
Inverse Bayesian Inference for Extracting Lesion Dynamics from Longitudinal Spectral CT
Lukas Förner, Melina Wördehoff, Julian Steffens et al. (+6 more)
Longitudinal medical imaging captures temporal
evolution of lesions, yet extracting the underlying dynamical parameters governing this
evolution remains challenging. We propose an inverse
Bayesian framework for inferring lesion dynamics from longitudinal spectral CT. We decompose spectral feature ($x$)
evolution into three components: \begin{equation*} \frac{dx_i}{dt} = A_i x_i + B \cdot n + C \cdot \Delta x_{\text{sat}} \end{equation*} where $A_i$ captures intrinsic dynamics (lesion-autonomous
evolution), $B$ captures local environment tumour burden (organ tumour burden through satellite count coupling), and $C$ captures environment/satellite state change (i.e., whether surrounding lesions move similarly or not). We demonstrate the framework on
photon-counting NSCLC CT data from metastases, recovering distinct dynamical regimes: lung lesions exhibit significant satellite count coupling ($B=-0.34$, $p<0.05$) suggesting competitive dynamics, while liver lesions show synergistic satellite behaviour coupling ($C\approx+1.0$, $p<0.05$). Synthetic validation confirms parameter recovery, and cross-coupling analysis validates that our method detects non-zero coupling when present. This work establishes inverse dynamical inference as a principled methodology for extracting interpretable parameters from longitudinal imaging, moving beyond static feature extraction toward mechanistic characterisation of lesion behaviour. The code and data are available at: https://github.com/lukasf98/inverse-
bayesian-inference
Read the full abstract ▾
All authors Lukas Förner, Melina Wördehoff, Julian Steffens, Maximilian Schmutz, Rainer Claus, Josua Decker, Thomas Kröncke, Kartikay Tehlan, Thomas Wendler
Computer Vision
Yilin Wang, Haochen Shi, Guanyu Chen et al. (+4 more)
Food
segmentation is essential for applications such as intelligent catering, dietary assessment, and recommendation. However, existing
benchmarks fail to capture the complexity of real-world dining scenes. The challenges of dense inter-dish overlap, fine-grained class similarity, and extreme long-tail class distributions exceed the fidelity of current
datasets. To fill this gap, we introduce DishSeg24k, a large-scale dish-level
segmentation benchmark with 24,096 images, 112,281 instances, and 278 fine-grained categories in real-world dining environments. Based on DishSeg24k, we further propose Food Expert-Adaptive
Segmentation Transformers (FEAST) to address these challenges. FEAST models query-based decoding as a Markov Decision Process (MDP), where each decoder layer update is treated as a sequential decision step that explores uncertainty along dish boundaries. We further redesign the decoder with a reinforcement learning (RL)-guided Mixture-of-Experts (MoE) module, in which a dual-critic decoupled optimization scheme separates task-oriented query refinement from structure-aware expert routing. This design promotes expert specialization and prevents expert collapse under long-tail category distributions. Finally, extensive experiments on DishSeg24k demonstrate the state-of-the-art performance of FEAST, which outperforms previous methods by {+3.21%} mIoU, {+3.68%} mDice, and {+4.00%} mAcc, respectively. We further validate the effectiveness of FEAST on FoodSeg103. The
dataset and code will be publicly released.
Read the full abstract ▾
All authors Yilin Wang, Haochen Shi, Guanyu Chen, Weiqing Min, Jinkai Zheng, Chenggang Yan, Shuqiang Jiang
Computer Vision
Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models
Sultan Alshehri, Zhantao Yang, Han Zhang et al. (+1 more)
Dual-encoder
vision-
language models (VLMs) expose a similarity interface that enables zero-shot
retrieval but fails compositional constraints: queries like "umbrella and no person" retrieve images containing both, even when concept detection is reliable. We trace this to an interface-level Bag-of-Concepts effect, where similarity scores approximate mean pooling of concept evidence regardless of operators. Although operator-dependent signals exist in text
embeddings, they are too weak or misaligned to affect rankings. Fine-tuning does not reliably resolve this failure because the dominant bottleneck is how similarity aggregates evidence rather than what encoders represent. We propose factored inference, which separates evidence extraction from constraint execution, and introduce LCSE (Logic-Constrained Score Editing), a training-free method that executes constraints externally using concept scores from frozen encoders. We also introduce FACTOR-Bench, where LCSE achieves 85.5% accuracy versus 73.2% for the best fine-tuned baseline, 90.7% when applied to SigLIP 2, and improves NegBench COCO MCQ accuracy from 27.2% to 65.2% while preserving
retrieval performance.
Read the full abstract ▾
All authors Sultan Alshehri, Zhantao Yang, Han Zhang, Marios Savvides
Computer Vision
Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs
Jouwon Song, Woohyeong Kim and Kyeongbo Kong
Recent high-resolution
Multimodal Large
Language Models (MLLMs) generate thousands of visual
tokens per input, leading to a visual
token explosion that introduces severe latency bottlenecks. While
token pruning mitigates this issue, state-of-the-art subset-optimization methods typically rely on iterative subset construction to jointly capture visual diversity and instruction relevance. As visual
token counts scale, this sequential dependency introduces significant selection overhead, severely limiting the translation of theoretical FLOPs reductions into actual wall-clock speedups. To address this limitation, we propose Single-Forward Pruner (SFPruner), a structural reformulation of visual
token pruning that embeds redundancy control directly into the scoring space, bypassing the need for iterative combinatorial optimization. Our non-iterative framework achieves redundancy-aware importance selection in a single forward pass through two complementary mechanisms. First, to attenuate redundancy at the covariance level, we introduce a semantics-guided ridge leverage scheme. By integrating instruction relevance and visual saliency, this mechanism suppresses dominant covariance directions and mitigates representation bias. Second, ranking-based directional masking resolves residual overlap through asymmetric similarity competition, where higher-scoring
tokens explicitly suppress redundant lower-scoring alternatives via parallel tensor operations. Extensive evaluations demonstrate that our approach maintains stable selection costs, reducing the
token selection process by up to 110 ms, from 112.4 ms to just 2.5 ms at 512
tokens in Qwen2.5-VL. This structural efficiency successfully translates theoretical
token reductions into tangible inference speedups while preserving highly competitive performance against state-of-the-art techniques under aggressive compression.
Read the full abstract ▾
Computer Vision
When Less Is More: A Controlled Benchmark of Lightweight CNNs for Satellite Land-Cover Segmentation on DeepGlobe
Atiq Ur Rehman and Joseph Michael Donovan
High-resolution satellite imagery is the backbone of good land-cover classification, and without that, environmental monitoring, urban planning, and sustainable resource management all fall short. Deep learning architectures perform well in semantic
segmentation, but the efficiency-accuracy trade-off across classical convolutional encoders is not well quantified under controlled, reproducible conditions. This study compares five architectures VGG16, MobileNetV2, InceptionV3, AlexNet, and CNN on the DeepGlobe Land Cover Classification
dataset using three progressively optimized iterations to isolate regularisation, transfer learning, and architectural depth. To ensure performance differentials reflect architectural properties, all experiments used identical preprocessing, hyperparameter, and training protocols without data augmentation or class-imbalance correction. At 24.98 MB, MobileNetV2_v1 had the highest overall accuracy (0.7906) and mean Intersection over Union (0.4625), outperforming deeper alternatives like InceptionV3_v2 (125.17 MB, accuracy 0.7610) and VGG16_v2 (71.13 MB, accuracy 0.7653). Class-wise analysis showed strength in urban, agricultural, and water categories, but rangeland-barren confusion showed that architectural optimization alone cannot optimize spectrally similar minority classes. Strong spatial generalization and crisp boundary delineation were confirmed on held-out test imagery, validating operational applicability. These results show that lightweight, transfer-learned models can match or outperform deeper models in resource-constrained remote-sensing environments, enabling scalable land-cover mapping.
Read the full abstract ▾
Computer Vision
OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars
Quanyue Song, Yishan He, Yanbo Ding et al. (+4 more)
Recent advances in
diffusion-based generative models have enabled real-time audio-driven avatar generation and unified audio-visual synthesis, providing a promising foundation for interactive avatar systems. However, extending unified audio-visual synthesis to real-time interactive streaming remains challenging, as the generation horizon is unknown in advance and the generated identity may drift over long-term generation. To address these challenges, we propose OmniMate, a unified framework for open-ended real-time interactive audio-visual avatar generation. OmniMate jointly synthesizes visual content,
speech, and sound effects in real time, enabling natural and immersive multi-turn interactions. To achieve adaptive response progression, we introduce a Generation Progress Controller (GPC) that explicitly models the generation progress of each streaming chunk, allowing the model to complete responses according to the desired progress and achieve seamless transitions between execution and listening states. To preserve long-term cross-modal identity consistency, we propose a Multi-Reference Conditioning Module (MRCM), which leverages multiple reference images and a reference
speech segment to provide persistent visual and speaker identity cues throughout long-duration streaming interactions. Extensive experiments on an interaction-oriented adaptation of VerseBench demonstrate that OmniMate achieves high-quality, low-latency streaming generation while maintaining strong long-term audio-visual consistency. The results further show that OmniMate supports realistic, coherent, and responsive interactive avatar experiences over extended multi-turn conversations.
Read the full abstract ▾
All authors Quanyue Song, Yishan He, Yanbo Ding, Zhixiang He, Yongxiang Li, Caigui Jiang, Zhi Zhi Guo
Computer Vision
Breaking the Synthetic-Real Domain Shortcut for Training-Free Generative Replay-based Class Incremental Learning
Tao Zhang, Qixuan Fan, Yiyuan Liang et al. (+7 more)
Class-incremental learning (CIL) requires models to continuously acquire new knowledge while avoiding catastrophic forgetting. While exemplar replay is effective, it raises concerns regarding
privacy and storage. Thus, generative replay has emerged as a viable alternative, synthesizing old data using frozen pretrained text-to-image (T2I) models without any extra training. However, we observe that directly mixing synthetic old-class data with real new-class data during incremental training leads to significant performance degradation. This issue stems from a "domain shortcut", where models rely on domain-discriminative features instead of semantic class cues. To address this, we propose DREAM ($\underline{\mathbf{D}}$omain-$\underline{\mathbf{R}}$egularized $\underline{\mathbf{E}}$xemplar-free $\underline{\mathbf{A}}$lignment $\underline{\mathbf{M}}$odel), which uses a training-free generator to synthesize old-class data and eliminates domain shortcut via subspace rectification and orthogonal projection, while reinforcing semantic
alignment through real-anchored prototype regularization. Extensive experiments on 4
datasets demonstrate that DREAM outperforms existing exemplar-free CIL methods and achieves state-of-the-art performance. Our source code is available at https://github.com/Light-ZhangTao/DREAM.
Read the full abstract ▾
All authors Tao Zhang, Qixuan Fan, Yiyuan Liang, Yanjie Wang, Song Yan, Tian Tian, Jiahuan Zhou, Luxin Yan, Sheng Zhong, Xu Zou
Computer Vision
mmSimPrior: Learning Simulation Priors for Data-Efficient and Generalizable Real-World Radar-based Human Motion Reconstruction
Cheng Guo, Qiming Cao, Shengkai Xu et al. (+4 more)
Millimeter-wave (mmWave) radar enables
privacy-preserving and illumination-robust human motion reconstruction, but training generalizable models typically requires costly paired radar-motion recordings. Simulation can scale such supervision, yet even physics-based simulators cannot fully reproduce real-world multipath, clutter, hardware-specific response statistics, or distance-dependent resolution degradation, leaving a sim-to-real gap. We present mmSimPrior, a simulation-pretrained framework that factorizes transferable knowledge into signal, motion, and radar-to-motion mapping priors. To learn transferable signal and motion priors, we pretrain a
multimodal radar encoder with a physics-informed domain-randomization curriculum designed to mitigate the sim-to-real gap by approximating real-world propagation- and acquisition-level variations, while a joint-temporal tokenizer learns a discrete prior over plausible human motion. A dual-mode mapping module predicts either motion-code distributions for structurally constrained zero-shot reconstruction or continuous motion parameters for flexible adaptation from limited real data. We further construct a 4.2M-frame, 31K-sequence
dataset suite and introduce a No-Overlap Setting that prevents any exact subject-environment-location-motion tuple from appearing in both the adaptation and test sets. Experiments on mmSimPrior-Real and RT-Pose demonstrate consistent gains: with only 24 paired real sequences, mmSimPrior-Reg reduces MPJPE by 24.7-39.0% over the strongest baseline across the three environments, while mmSimPrior-Cls reduces zero-shot MPJPE by 8.5% without fine-tuning.
Read the full abstract ▾
All authors Cheng Guo, Qiming Cao, Shengkai Xu, Haoyu Xie, Kaixiang Su, Pu Wang, Hongfei Xue
Computer Vision · Cross-listed from eess.IV
PriSAR: 3D Geometric-Prior-Guided Diffusion for Parameter-Controlled SAR Image Generation
Fan Zhang, Xuanting Wu, Fei Ma et al. (+2 more)
Synthetic aperture radar (SAR) image generation can mitigate data scarcity, but controllablegeneration under sparse observation angles remains difficult. Recent SAR generative studies im-prove texture realism, yet explicit geometry-aware control is still limited. This paper studiesthe focused and verifiable setting of intermediate-azimuth completion: 3D-model-derived geo-metric priors guide a
diffusion model to synthesize the views missing from sparse-angle trainingdata. GeoDiff-SAR constructs a lightweight multi-bounce ray-tracing prior, encodes the result-ing point cloud, and fuses it with text conditioning while adapting Stable
Diffusion 3.5 Mediumthrough low-rank adaptation. On a real four-category aircraft
dataset, GeoDiff-SAR reaches anSSIM of 0.812 and azimuth consistency of 0.940, compared with 0.738 and 0.782 for the text-conditioned SD3.5 Medium baseline. The same sparse-angle protocol on five MSTAR vehicleclasses yields an SSIM of 0.878 and azimuth consistency of 0.917. These results support theconclusion that a lightweight 3D
geometric prior improves viewpoint adherence for controllableSAR generation; it is intended as generation guidance rather than high-fidelity electromagneticreconstruction.
Read the full abstract ▾
All authors Fan Zhang, Xuanting Wu, Fei Ma, Qiang Yin, Yuxin Hu
Computer Vision
HALLELUAI: A Hallucination-Aware AI System for Ultra-Realistic Image-to-Video Generation at Scale
Aniket Sakpal, Yang Jiang, Rouzbeh Davoudi et al. (+2 more)
AI-generated
video is increasingly used across marketing, product storytelling, and creative workflows, yet automated; high-precision quality control remains a major constraint to scaling production. We present HALLELUAI, an end-to-end system that moderates and regenerates image-to-
video outputs to meet expert-level creative standards and deliver ultra-realistic
videos with consistent end-user quality of experience (QoE) at scale. The system integrates a
video moderation module that evaluates frame-level aesthetics, temporal motion fidelity, and fine-grained
hallucination risks relative to the source image, with an agentic regeneration module that iteratively fixes failures through
prompt refinement, controlled camera adjustments, targeted model or image switching, and structured retry strategies. The moderation logic is aligned with domain-specific creative guidelines and produces granular, machine-actionable feedback that directly drives regeneration. In human-in-the-loop evaluations with creative experts, HALLELUAI shows strong
alignment and reliably outputs ultra-realistic, production-grade
videos suitable for product and marketing placements at scale. This framework advances trustworthy AI generated
video content by enforcing visual realism, brand safety, and strict input-image fidelity while enabling image-to-
video generation at scale.
Read the full abstract ▾
All authors Aniket Sakpal, Yang Jiang, Rouzbeh Davoudi, Shayan Hassantabar, Mani Najmabadi