Computer Vision
Attention-Guided Flow-Matching for Sparse 3D Geological Generation
Zhixiang Lu, Mengqi Han, Peixin Guo et al. (+4 more)
Constructing high-resolution 3D geological models from sparse 1D borehole and 2D surface data is a highly ill-posed inverse problem. Traditional heuristic and implicit modeling methods fundamentally fail to capture non-linear topological discontinuities under extreme sparsity, often yielding unrealistic artifacts. Furthermore, while deep generative architectures like
Diffusion Models have revolutionized continuous domains, they suffer from severe representation collapse when conditioned on sparse categorical grids. To bridge this gap, we propose 3D-GeoFlow, the first
Attention-Guided Continuous Flow Matching framework tailored for sparse
multimodal geological modeling. By reformulating discrete categorical generation as a simulation-free, continuous vector field regression optimized via Mean Squared Error, our model establishes stable, deterministic optimal transport paths. Crucially, we integrate 3D
Attention Gates to dynamically propagate localized borehole features across the volumetric latent space, ensuring macroscopic structural coherence. To validate our framework, we curated a large-scale
multimodal dataset comprising 2,200 procedurally generated 3D geological cases. Extensive out-of-distribution (OOD) evaluations demonstrate that 3D-GeoFlow achieves a paradigm shift, significantly outperforming heuristic interpolations and standard
diffusion baselines.
Read the full abstract ▾
All authors Zhixiang Lu, Mengqi Han, Peixin Guo, Tianming Bai, Jionglong Su, Fei Fang, Sifan Song
Computer Vision · Cross-listed from cs.IR
Pretrain-then-Adapt: Uncertainty-Aware Test-Time Adaptation for Text-based Person Search
Jiahao Zhang, Shaofei Huang, Yaxiong Wang et al. (+1 more)
Text-based person search faces inherent limitations due to data scarcity, driven by stringent
privacy constraints and the high cost of manual annotation. To mitigate this, existing methods usually rely on a Pretrain-then-Finetune paradigm, where models are first pretrained on synthetic person-caption data to establish cross-modal
alignment, followed by fine-tuning on labeled real-world
datasets. However, this paradigm lacks practicality in real-world deployment scenarios, where large-scale annotated target-domain data is typically inaccessible. In this work, we propose a new Pretrain-then-Adapt paradigm that eliminates reliance on extensive target-domain supervision through an offline test-time adaptation manner, enabling dynamic model adaptation using only unlabeled test data with minimal post-train time cost. To mitigate overconfidence with false positives of previous
entropy-based test-time adaptation, we propose an Uncertainty-Aware Test-Time Adaptation (UATTA) framework, which introduces a bidirectional
retrieval disagreement mechanism to estimate uncertainty, i.e., low uncertainty is assigned when an image-text pair ranks highly in both image-to-text and text-to-image
retrieval, indicating high
alignment; otherwise, high uncertainty is detected. This indicator drives offline test-time model recalibration without labels, effectively mitigating domain shift. We validate UATTA on four
benchmarks, i.e., CUHK-PEDES, ICFG-PEDES, RSTPReid, and PAB, showing consistent improvements across both CLIP-based (one-stage) and XVLM-based (two-stage) frameworks. Ablation studies confirm that UATTA outperforms existing offline test-time adaptation strategies, establishing a new
benchmark for label-efficient, deployable person search systems. Our code is available at https://github.com/nkuzjh/UATTA.
Read the full abstract ▾
All authors Jiahao Zhang, Shaofei Huang, Yaxiong Wang, Zhedong Zheng
Computer Vision · Cross-listed from eess.IV
ADP-FL-MedSeg: Adaptive Differential Privacy for Federated Medical Segmentation Across Diverse Modalities
Puja Saha and Eranga Ukwatta
Large volumes of medical data remain underutilized because centralizing distributed data is often infeasible due to strict
privacy regulations and institutional constraints. In addition, models trained in centralized settings frequently fail to generalize across clinical sites because of heterogeneity in imaging protocols and continuously evolving data distributions arising from differences in scanners, acquisition parameters, and patient populations.
Federated learning offers a promising solution by enabling collaborative model training without sharing raw data. However, incorporating differential
privacy into
federated learning, while essential for
privacy guarantees, often leads to degraded accuracy, unstable convergence, and reduced generalization. In this work, we propose an adaptive differentially private
federated learning (ADP-FL) framework for medical image
segmentation that dynamically adjusts
privacy mechanisms to better balance the
privacy-utility trade-off. The proposed approach stabilizes training, significantly improves Dice scores and
segmentation boundary quality, and maintains rigorous
privacy guarantees. We evaluated ADP-FL across diverse imaging modalities and
segmentation tasks, including skin lesion
segmentation in dermoscopic images, kidney tumor
segmentation in 3D CT scans, and
brain tumor
segmentation in multi-parametric MRI. Compared with conventional
federated learning and standard differentially private
federated learning, ADP-FL consistently achieves higher accuracy, improved boundary delineation, faster convergence, and greater training stability, with performance approaching that of non-private
federated learning under the same
privacy budgets. These results demonstrate the practical viability of ADP-FL for high-performance,
privacy-preserving medical image
segmentation in real-world
federated settings.
Read the full abstract ▾
Computer Vision
DesigNet: Learning to Draw Vector Graphics as Designers Do
Tomas Guija-Valiente and Iago Suárez
AI-driven content generation has made remarkable progress in recent years. However, neural networks and human designers operate in fundamentally different ways, making collaboration between them challenging. We address this gap for Scalable Vector Graphics (SVG) by equipping neural networks with tools commonly used by designers, such as axis
alignment and explicit continuity control at command junctions. We introduce DesigNet, a hierarchical
Transformer-VAE that operates directly on SVG sequences with a continuous command parameterization. Our main contributions are two differentiable modules: a continuity self-refinement module that predicts $C^0$, $G^1$, and $C^1$ continuity for each curve point and enforces it by modifying Bézier control points, and an
alignment self-refinement module with snapping capabilities for horizontal or vertical lines. DesigNet produces editable outlines and achieves competitive results against state-of-the-art methods, with notably higher accuracy in continuity and
alignment. These properties ensure the outputs are easier to refine and integrate into professional design workflows. Source Code: https://github.com/TomasGuija/DesigNet.
Read the full abstract ▾
Computer Vision
Hybrid ResNet-1D-BiGRU with Multi-Head Attention for Cyberattack Detection in Industrial IoT Environments
Afrah Gueriani, Hamza Kheddar and Ahmed Cherif Mazari
This study introduces a hybrid deep learning model for intrusion detection in Industrial IoT (IIoT) systems, combining ResNet-1D, BiGRU, and Multi-Head
Attention (MHA) for effective spatial-temporal feature extraction and
attention-based feature weighting. To address class imbalance, SMOTE was applied during training on the EdgeHoTset
dataset. The model achieved 98.71% accuracy, a loss of 0.0417%, and low inference latency (0.0001 sec /instance), demonstrating strong real-time capability. To assess generalizability, the model was also tested on the CICIoV2024
dataset, where it reached 99.99% accuracy and F1-score, with a loss of 0.0028, 0 % FPR, and 0.00014 sec/instance inference time. Across all metrics and
datasets, the proposed model outperformed existing methods, confirming its robustness and effectiveness for real-time IoT intrusion detection.
Read the full abstract ▾
Computer Vision
Predicting Alzheimer's disease progression using rs-fMRI and a history-aware graph neural network
Mahdi Moghaddami, Mohammad-Reza Siadat, Austin Toma et al. (+2 more)
Alzheimer's disease (AD) is a neurodegenerative disorder that affects more than seven million people in the United States alone. AD currently has no cure, but there are ways to potentially slow its progression if caught early enough. In this study, we propose a graph neural network (GNN)-based model for predicting whether a subject will transition to a more severe stage of cognitive impairment at their next clinical visit. We consider three stages of cognitive impairment in order of severity: cognitively normal (CN), mild cognitive impairment (MCI), and AD. We use functional connectivity graphs derived from resting-state functional
magnetic resonance imaging (rs-fMRI) scans of 303 subjects, each with a different number of visits. Our GNN-based model incorporates a recurrent neural network (RNN) block, enabling it to process data from the subject's entire visit history. It can also work with irregular time gaps between visits by incorporating visit distance information into our input features. Our model demonstrates robust predictive performance, even with missing visits in the subjects' visit histories. It achieves an accuracy of 82.9%, with an especially impressive accuracy of 68.8% on CN to MCI conversions - a task that poses a substantial challenge in the field. Our results highlight the effectiveness of rs-fMRI in predicting the onset of MCI or AD and, in conjunction with other modalities, could offer a viable method for enabling timely interventions to slow the progression of cognitive impairment.
Read the full abstract ▾
All authors Mahdi Moghaddami, Mohammad-Reza Siadat, Austin Toma, Connor Laming, Huirong Fu
Computer Vision
PhysHead: Simulation-Ready Gaussian Head Avatars
Berna Kabadayi, Vanessa Sklyarova, Wojciech Zielonka et al. (+2 more)
Realistic digital avatars require expressive and dynamic hair motion; however, most existing head avatar methods assume rigid hair movement. These methods often fail to disentangle hair from the head, representing it as a simple outer shell and failing to capture its natural volumetric behavior. In this paper, we address these limitations by introducing PhysHead, a hybrid representation for animatable head avatars with realistic hair dynamics learned from multi-view
video. At the core is a 3D Gaussian-based layered representation of the head. Our approach combines a 3D parametric mesh for the head with strand-based hair, which can be directly simulated using physics engines. For the appearance model, we employ Gaussian primitives attached to both the head mesh and hair segments. This representation enables the creation of photorealistic head avatars with dynamic hair behavior, such as wind-blown motion, overcoming the constraints of rigid hair in existing methods. However, these animation capabilities also require new training schemes. In particular, we propose the use of VLM-based models to generate appearance of regions that are occluded in the dynamic training sequences. In quantitative and qualitative studies, we demonstrate the capabilities of the proposed model and compare it with existing baselines. We show that our method can synthesize physically plausible hair motion besides expression and camera control.
Read the full abstract ▾
All authors Berna Kabadayi, Vanessa Sklyarova, Wojciech Zielonka, Justus Thies, Gerard Pons-Moll
Computer Vision
Visual prompting reimagined: The power of the Activation Prompts
Yihua Zhang, Hongkang Li, Yuguang Yao et al. (+5 more)
Visual prompting (VP) has emerged as a popular method to repurpose pretrained
vision models for adaptation to downstream tasks. Unlike conventional model fine-tuning techniques, VP introduces a universal perturbation directly into the input data to facilitate task-specific fine-tuning rather than modifying model parameters. However, there exists a noticeable performance gap between VP and conventional fine-tuning methods, highlighting an unexplored realm in theory and practice to understand and advance the input-level VP to reduce its current performance gap. Towards this end, we introduce a generalized concept, termed activation
prompt (AP), which extends the scope of the input-level VP by enabling universal perturbations to be applied to activation maps within the intermediate layers of the model. By using AP to revisit the problem of VP and employing it as an analytical tool, we demonstrate the intrinsic limitations of VP in both performance and efficiency, revealing why input-level prompting may lack effectiveness compared to AP, which exhibits a model-dependent layer preference. We show that AP is closely related to normalization tuning in convolutional neural networks and
vision transformers, although each model type has distinct layer preferences for prompting. We also theoretically elucidate the rationale behind such a preference by analyzing global features across layers. Through extensive experiments across 29
datasets and various model architectures, we provide a comprehensive performance analysis of AP, comparing it with VP and parameter-efficient fine-tuning baselines. Our results demonstrate AP's superiority in both accuracy and efficiency, considering factors such as time, parameters,
memory usage, and throughput.
Read the full abstract ▾
All authors Yihua Zhang, Hongkang Li, Yuguang Yao, Aochuan Chen, Shuai Zhang, Pin-Yu Chen, Meng Wang, Sijia Liu
Computer Vision
Continual Visual Anomaly Detection on the Edge: Benchmark and Efficient Solutions
Manuel Barusco, Francesco Borsatti, David Petrovic et al. (+2 more)
Visual Anomaly Detection (VAD) is a critical task for many applications including industrial inspection and healthcare. While VAD has been extensively studied, two key challenges remain largely unaddressed in conjunction: edge deployment, where computational resources are severely constrained, and continual learning, where models must adapt to evolving data distributions without forgetting previously acquired knowledge. Our
benchmark provides guidance for the selection of the optimal backbone and VAD method under joint efficiency and adaptability constraints, characterizing the trade-offs between
memory footprint, inference cost, and detection performance. Studying these challenges in isolation is insufficient, as methods designed for one setting make assumptions that break down when the other constraint is simultaneously imposed. In this work, we propose the first comprehensive
benchmark for VAD on the edge in the continual learning scenario, evaluating seven VAD models across three lightweight backbone architectures. Furthermore, we propose Tiny-Dinomaly, a lightweight adaptation of the Dinomaly model built on the DINO foundation model that achieves a 11$\times$ smaller
memory footprint and 20$\times$ lower computational cost while improving localization (Pixel F1) by 5 percentage points. Finally, we introduce targeted modifications to PatchCore and PaDiM to improve their efficiency in the continual learning setting.
Read the full abstract ▾
All authors Manuel Barusco, Francesco Borsatti, David Petrovic, Davide Dalle Pezze, Gian Antonio Susto
Computer Vision · Cross-listed from cs.CL
When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't
Jonathan Nemitz, Carsten Eickhoff, Junyi Jessy Li et al. (+3 more)
Understanding when
Vision-
Language Models (VLMs) will behave unexpectedly, whether models can reliably predict their own behavior, and if models adhere to their introspective
reasoning are central challenges for trustworthy deployment. To study this, we introduce the Graded Color Attribution (GCA)
dataset, a controlled
benchmark designed to elicit decision rules and evaluate participant faithfulness to these rules. GCA consists of line drawings that vary pixel-level color coverage across three conditions: world-knowledge recolorings, counterfactual recolorings, and shapes with no color priors. Using GCA, we ask both VLMs and human participants to state a threshold rule: the share of an object's pixels that must be a given color for the object to receive that color label. We then compare these rules with their subsequent color attribution decisions. Our findings reveal that models systematically violate their own introspective rules. For example, GPT-5-mini violates its stated introspection rules in nearly 60% of cases on objects with strong color priors. Human participants remain faithful to their stated rules, with any apparent violations being explained by a well-documented tendency to overestimate color coverage. In contrast, we find that VLMs can accurately estimate color coverage, yet directly contradict their own
reasoning in their final responses. Across all models and strategies for eliciting introspective rules, world-knowledge priors systematically degrade faithfulness in ways that do not mirror human cognition. Our findings challenge the view that VLM
reasoning failures are difficulty-driven and suggest that VLM introspective self-knowledge is miscalibrated, with direct implications for high-stakes deployment.
Read the full abstract ▾
All authors Jonathan Nemitz, Carsten Eickhoff, Junyi Jessy Li, Kyle Mahowald, Michal Golovanevsky, William Rudman
Computer Vision · Cross-listed from cs.AI
ProofSketcher: Hybrid LLM + Lightweight Proof Checker for Reliable Math/Logic Reasoning
Kranthi Kommuru, Kunal Khanvilkar and Gaurav Parekh
The large
language models (LLMs) might produce a persuasive argument within mathematical and logical fields, although such argument often includes some minor missteps, including the entire omission of side conditions, invalid inference patterns, or appeals to a lemma that cannot be derived logically out of the context being discussed. These omissions are infamously hard to notice solely out of the text, as even the misconstrued construction still may seem mostly accurate. Conversely, interactive
theorem provers like Lean and Coq have rigorous reliability by ensuring that syntactic and semantic statements only accept statements that can pass all the syntactic and semantic steps in the program which is a small trusted kernel of the
language type-checks with. Despite the fact that this technique provides strong guarantees, it comes at quite a heavy price: the evidence must be completely formalized, and the evidence user or a auxiliary search program must provide an avalanche of low-level information. This paper presents a hybrid pipeline where an LLM generates a typed proof sketch in a compact DSL and a lightweight trusted kernel expands the sketch into explicit proof obligations.
Read the full abstract ▾
Computer Vision
MorphDistill: Distilling Unified Morphological Knowledge from Pathology Foundation Models for Colorectal Cancer Survival Prediction
Hikmat Khan, Usama Sajjad, Metin N. Gurcan et al. (+4 more)
Background: Colorectal cancer (CRC) remains a leading cause of cancer-related mortality worldwide. Accurate survival prediction is essential for treatment stratification, yet existing pathology foundation models often overlook organ-specific features critical for CRC prognostication. Methods: We propose MorphDistill, a two-stage framework that distills complementary knowledge from multiple pathology foundation models into a compact CRC-specific encoder. In Stage I, a student encoder is trained using dimension-agnostic multi-teacher relational
distillation with supervised contrastive regularization on large-scale colorectal
datasets. This preserves inter-sample relationships from ten foundation models without explicit feature
alignment. In Stage II, the encoder extracts patch-level features from whole-slide images, which are aggregated via
attention-based multiple instance learning to predict five-year survival. Results: On the Alliance/CALGB 89803 cohort (n=424, stage III CRC), MorphDistill achieves an AUC of 0.68 (SD 0.08), an approximately 8% relative improvement over the strongest baseline (AUC 0.63). It also attains a C-index of 0.661 and a hazard ratio of 2.52 (95% CI: 1.73-3.65), outperforming all baselines. On an external TCGA cohort (n=562), it achieves a C-index of 0.628, demonstrating strong generalization across
datasets and robustness across clinical subgroups. Conclusion: MorphDistill enables task-specific representation learning by integrating knowledge from multiple foundation models into a unified encoder. This approach provides an efficient strategy for prognostic modeling in computational pathology, with potential for broader oncology applications. Further validation across additional cohorts and disease stages is warranted.
Read the full abstract ▾
All authors Hikmat Khan, Usama Sajjad, Metin N. Gurcan, Anil Parwani, Wendy L. Frankel, Wei Chen, Muhammad Khalid Khan Niazi
Computer Vision
Xiangyu Peng, Can Qin, An Yan et al. (+4 more)
Multimodal large
language models (MLLMs) have demonstrated strong capabilities in visual understanding, yet they remain limited in complex, multi-step
reasoning that requires deep searching and integrating visual evidence with external knowledge. In this work, we address this challenge by constructing high-quality, verified multi-hop
vision-
language training data for
multimodal deep-search
agents. We propose a Multi-hop Tool-Augmented
Agent for Evidence-based QA Synthesis (MTA-
Agent), which automatically selects tools and their parameters to retrieve and validate evidence from both visual and textual sources and generates structured multi-hop question-answer trajectories. Starting from diverse VQA seed
datasets, our pipeline produces a large-scale training
dataset, MTA-
Vision-DeepSearch, containing 21K high-quality multi-hop examples. The data is filtered through a multi-stage verification process to ensure factual consistency and answer uniqueness. Using MTA-
Vision-DeepSearch, a 32B open-source
multimodal search
agent achieves state-of-the-art performance, reaching an average of 54.63% across six challenging
benchmarks, outperforming GPT-5 (51.86%), Gemini-2.5-Pro (50.98%), and Gemini-3-Pro (54.46%) under the same tool settings. We further show that training on our data improves both
reasoning depth and tool-use behavior, increasing the average number of steps from 2.27 to 4.28, and leading to more systematic and persistent search strategies. Additionally, we demonstrate that training can be performed without real-time tool calls by replaying cached interactions, significantly reducing training cost. Importantly, we present MTA-
Agent as a fully open recipe for
multimodal deep search: we release the entire
dataset, training trajectories, and implementation details to enable reproducibility and future research on open
multimodal search
agents.
Read the full abstract ▾
All authors Xiangyu Peng, Can Qin, An Yan, Xinyi Yang, Zeyuan Chen, Ran Xu, Chien-Sheng Wu
Computer Vision
DietDelta: A Vision-Language Approach for Dietary Assessment via Before-and-After Images
Gautham Vinod, Siddeshwar Raghavan, Bruce Coburn et al. (+1 more)
Accurate dietary assessment is critical for precision nutrition, yet most image-based methods rely on a single pre-consumption image and provide only coarse, meal-level estimates. These approaches cannot determine what was actually consumed and often require restrictive inputs such as depth sensing, multi-view imagery, or explicit
segmentation. In this paper, we propose a simple
vision-
language framework for food-item-level nutritional analysis using paired before-and-after eating images. Instead of relying on rigid
segmentation masks, our method leverages natural
language prompts to localize specific food items and estimate their weight directly from a single RGB image. We further estimate food consumption by predicting weight differences between paired images using a two-stage training strategy. We evaluate our method on three publicly available
datasets and demonstrate consistent improvements over existing approaches, establishing a strong baseline for before-and-after dietary image analysis.
Read the full abstract ▾
All authors Gautham Vinod, Siddeshwar Raghavan, Bruce Coburn, Fengqing Zhu
Computer Vision · Cross-listed from cs.LG
Bi-Level Optimization for Single Domain Generalization
Marzi Heidari, Hanping Zhang, Hao Yan et al. (+1 more)
Generalizing from a single labeled source domain to unseen target domains, without access to any target data during training, remains a fundamental challenge in robust machine learning. We address this underexplored setting, known as Single Domain Generalization (SDG), by proposing BiSDG, a bi-level optimization framework that explicitly decouples task learning from domain modeling. BiSDG simulates distribution shifts through surrogate domains constructed via label-preserving transformations of the source data. To capture domain-specific context, we propose a domain
prompt encoder that generates lightweight modulation signals to produce augmenting features via feature-wise linear modulation. The learning process is formulated as a bi-level optimization problem: the inner objective optimizes task performance under fixed
prompts, while the outer objective maximizes generalization across the surrogate domains by updating the domain
prompt encoder. We further develop a practical gradient approximation scheme that enables efficient bi-level training without second-order derivatives. Extensive experiments on various SGD
benchmarks demonstrate that BiSDG consistently outperforms prior methods, setting new state-of-the-art performance in the SDG setting.
Read the full abstract ▾
All authors Marzi Heidari, Hanping Zhang, Hao Yan, Yuhong Guo
Computer Vision
Evidence-Based Actor-Verifier Reasoning for Echocardiographic Agents
Peng Huang, Yiming Wang, Yineng Chen et al. (+9 more)
Echocardiography plays an important role in the screening and diagnosis of cardiovascular diseases. However, automated intelligent analysis of echocardiographic data remains challenging due to complex cardiac dynamics and strong view heterogeneity. In recent years, visual
language models (VLM) have opened a new avenue for building ultrasound understanding systems for clinical decision support. Nevertheless, most existing methods formulate this task as a direct mapping from
video and question to answer, making them vulnerable to template shortcuts and spurious explanations. To address these issues, we propose EchoTrust, an evidence-driven Actor-Verifier framework for trustworthy
reasoning in echocardiography VLM-based
agents. EchoTrust produces a structured intermediate representation that is subsequently analyzed by distinct roles, enabling more reliable and interpretable decision-making for high-stakes clinical applications.
Read the full abstract ▾
All authors Peng Huang, Yiming Wang, Yineng Chen, Liangqiao Gui, Hui Guo, Bo Peng, Shu Hu, Xi Wu, Tsao Connie, Hongtu Zhu, Balakrishnan Prabhakaran, Xin Wang
Computer Vision
Teng Hu, Jiangning Zhang, Hongrui Huang et al. (+7 more)
The rapid advancement of Artificial Intelligence Generated Content (AIGC) has revolutionized
video generation, enabling systems ranging from proprietary pioneers like OpenAI's Sora, Google's Veo3, and Bytedance's Seedance to powerful open-source contenders like Wan and HunyuanVideo to synthesize temporally coherent and semantically rich
videos. These advancements pave the way for building "world models" that simulate real-world dynamics, with applications spanning entertainment, education, and virtual reality. However, existing reviews on
video generation often focus on narrow technical fields, e.g., Generative
Adversarial Networks (GAN) and
diffusion models, or specific tasks (e. g.,
video editing), lacking a comprehensive perspective on the field's
evolution, especially regarding Auto-Regressive (AR) models and integration of
multimodal information. To address these gaps, this survey firstly provides a systematic review of the development of
video generation technology, tracing its
evolution from early GANs to dominant
diffusion models, and further to emerging AR-based and
multimodal techniques. We conduct an in-depth analysis of the foundational principles, key advancements, and comparative strengths/limitations. Then, we explore emerging trends in
multimodal video generation, emphasizing the integration of diverse data types to enhance contextual awareness. Finally, by bridging historical developments and contemporary innovations, this survey offers insights to guide future research in
video generation and its applications, including virtual/augmented reality, personalized education, autonomous driving simulations, digital entertainment, and advanced world models, in this rapidly evolving field. For more details, please refer to the project at https://github.com/sjtuplayer/Awesome-
Video-Foundations.
Read the full abstract ▾
All authors Teng Hu, Jiangning Zhang, Hongrui Huang, Ran Yi, Zihan Su, Jieyu Weng, Zhucun Xue, Lizhuang Ma, Ming-Hsuan Yang, Dacheng Tao
Computer Vision · Cross-listed from cs.LG
Drifting Fields are not Conservative
Leonard T. Franz, Sebastian Hoffmann, Tim Weiland et al. (+2 more)
Drifting models have recently gained
attention for generating high-quality samples in a single forward pass. During training, they learn a push-forward map by following a vector-valued field, the drift field. We ask whether this procedure is equivalent to optimizing a scalar loss and find that, in general, it is not: drift fields are not conservative and cannot be written as the gradient of any scalar potential. We identify the position-dependent normalization as the source of non-conservatism, with the Gaussian kernel as the unique radial exception. Guided by this, we introduce the sharp kernel $k^\#$ and a sharp-normalized drift field that is conservative for general radial kernels. The resulting vector field is the gradient of a scalar potential that can be optimized directly using
stochastic gradient descent. Moreover, the field has the form of a score difference of kernel density estimates, and gives exact
equilibrium identifiability. Thus, sharp normalization closes the gap to related literature, such as Wasserstein gradient-flows and denoising score matching, also for non-Gaussian kernels. Empirically, sharp normalization preserves the performance of the original drifting objective, suggesting that the non-conservative flexibility is not required for high-quality generation.
Read the full abstract ▾
All authors Leonard T. Franz, Sebastian Hoffmann, Tim Weiland, Bernhard Schölkopf, Georg Martius
Computer Vision
Telescope: Learnable Hyperbolic Foveation for Ultra-Long-Range Object Detection
Parker Ewen, Dmitriy Rivkin, Mario Bijelic et al. (+1 more)
Autonomous highway driving, especially for long-haul heavy trucks, requires detecting objects at long ranges beyond 500 meters to satisfy braking distance requirements at high speeds. At long distances, vehicles and other critical objects occupy only a few pixels in high-resolution images, causing state-of-the-art object detectors to fail. This challenge is compounded by the limited effective range of commercially available
LiDAR sensors, which fall short of ultra-long range thresholds because of quadratic loss of resolution with distance, making image-based detection the most practically scalable solution given commercially available sensor constraints. We introduce
Telescope, a two-stage detection model designed for ultra-long range autonomous driving. Alongside a powerful detection backbone, this model contains a novel re-sampling layer and image transformation to address the fundamental challenges of detecting small, distant objects.
Telescope achieves $76\%$ relative improvement in mAP in ultra-long range detection compared to state-of-the-art methods (improving from an absolute mAP of 0.185 to 0.326 at distances beyond 250 meters), requires minimal computational overhead, and maintains strong performance across all detection ranges.
Read the full abstract ▾
All authors Parker Ewen, Dmitriy Rivkin, Mario Bijelic, Felix Heide
Computer Vision · Cross-listed from cs.CR
Harnessing Hyperbolic Geometry for Harmful Prompt Detection and Sanitization
Igor Maljkovic, Maria Rosaria Briglia, Iacopo Masi et al. (+2 more)
Vision-
Language Models (VLMs) have become essential for tasks such as image synthesis, captioning, and
retrieval by aligning textual and visual information in a shared
embedding space. Yet, this flexibility also makes them vulnerable to malicious
prompts designed to produce unsafe content, raising critical safety concerns. Existing defenses either rely on blacklist filters, which are easily circumvented, or on heavy classifier-based systems, both of which are costly and fragile under
embedding-level attacks. We address these challenges with two complementary components:
Hyperbolic Prompt Espial (HyPE) and
Hyperbolic Prompt Sanitization (HyPS). HyPE is a lightweight anomaly detector that leverages the structured geometry of
hyperbolic space to model benign
prompts and detect harmful ones as outliers. HyPS builds on this detection by applying explainable attribution methods to identify and selectively modify harmful words, neutralizing unsafe intent while preserving the original semantics of user
prompts. Through extensive experiments across multiple
datasets and
adversarial scenarios, we prove that our framework consistently outperforms prior defenses in both detection accuracy and robustness. Together, HyPE and HyPS offer an efficient, interpretable, and resilient approach to safeguarding VLMs against malicious
prompt misuse.
Read the full abstract ▾
All authors Igor Maljkovic, Maria Rosaria Briglia, Iacopo Masi, Antonio Emanuele Cinà, Fabio Roli
Computer Vision · Cross-listed from eess.IV
Structural Regularities of Cinema SDR-to-HDR Mapping in a Controlled Mastering Workflow: A Pixel-wise Case Study on ASC StEM2
Xin Zhang and Xiaoyi Chen
We present an empirical case study of cinema SDR-to-HDR mapping using ASC StEM2, a rare common-source
dataset containing EXR scene-referred images and matched SDR/HDR cinema release masters from the same ACES-based mastering workflow. Based on pixel-wise statistics over all 18,580 frames of the test film, we construct a three-domain comparison involving EXR source data, SDR release masters, and HDR release masters to characterize their luminance and color structural relationships within this controlled workflow. In the luminance dimension, SDR and HDR masters exhibit a highly stable global monotonic correspondence, with
geometric structure remaining largely consistent overall; sparse and structured deviations appear in self-luminous highlights and specific material regions. In the color dimension, the two masters remain largely consistent in hue, with saturation exhibiting a redistribution pattern of shadow suppression, midtone expansion, and highlight convergence. Using EXR as a scene-referred anchor, we further define a pixel-level decision map that operationally separates EXR-closer recovery regions from content-adaptive adjustment regions. Under this operational definition, 82.4% of sampled image regions are classified as EXR-closer recovery, while the remainder require localized adaptive adjustment. Rather than claiming a universal law for all cinema mastering pipelines, the study provides an interpretable quantitative baseline for structure-aware SDR-to-HDR analysis and for designing learning-based models under shared-source mastering conditions.
Read the full abstract ▾
Computer Vision
Action Images: End-to-End Policy Learning via Multiview Video Generation
Haoyu Zhen, Zixian Gao, Qiao Sun et al. (+7 more)
World action models (WAMs) have emerged as a promising direction for
robot policy learning, as they can leverage powerful
video backbones to model the future states. However, existing approaches often rely on separate action modules, or use action representations that are not pixel-grounded, making it difficult to fully exploit the pretrained knowledge of
video models and limiting transfer across viewpoints and environments. In this work, we present Action Images, a unified world action model that formulates policy learning as multiview
video generation. Instead of encoding control as low-dimensional
tokens, we translate 7-DoF
robot actions into interpretable action images: multi-view action
videos that are grounded in 2D pixels and explicitly track
robot-arm motion. This pixel-grounded action representation allows the
video backbone itself to act as a zero-shot policy, without a separate policy head or action module. Beyond control, the same unified model supports
video-action joint generation, action-conditioned
video generation, and action labeling under a shared representation. On RLBench and real-world evaluations, our model achieves the strongest zero-shot success rates and improves
video-action joint generation quality over prior
video-space world models, suggesting that interpretable action images are a promising route to policy learning.
Read the full abstract ▾
All authors Haoyu Zhen, Zixian Gao, Qiao Sun, Yilin Zhao, Yuncong Yang, Yilun Du, Pengsheng Guo, Tsun-Hsuan Wang, Yi-Ling Qiao, Chuang Gan
Computer Vision
Reihaneh Zohrabi, Hosein Hasani, Akshita Gupta et al. (+3 more)
Large
vision-
language models can produce object
hallucinations in image descriptions, highlighting the need for effective detection and mitigation strategies. Prior work commonly relies on the model's
attention weights on visual
tokens as a detection signal. We reveal that coarse-grained
attention-based analysis is unreliable due to hidden confounders, specifically
token position and object repetition in a description. This leads to Simpson's paradox: the
attention trends reverse or disappear when statistics are aggregated. Based on this observation, we introduce HaloProbe, a
Bayesian framework that factorizes external description statistics and internal decoding signals to estimate
token-level
hallucination probabilities. HaloProbe uses balanced training to encourage learning from internal evidence and combines it with a learned prior over external features to recover the true posterior. While intervention-based mitigation methods often degrade utility or fluency by modifying models' internals, we use HaloProbe as an external scoring signal for non-invasive mitigation. Our experiments show that HaloProbe-guided decoding reduces
hallucinations more effectively than state-of-the-art intervention-based methods while preserving caption utility and linguistic fluency.
Read the full abstract ▾
All authors Reihaneh Zohrabi, Hosein Hasani, Akshita Gupta, Mahdieh Soleymani Baghshah, Anna Rohrbach, Marcus Rohrbach
Computer Vision
Zhengming Yu, Li Ma, Mingming He et al. (+11 more)
Most digital
videos are stored in 8-bit low dynamic range (LDR) formats, where much of the original high dynamic range (HDR) scene radiance is lost due to saturation and quantization. This loss of highlight and shadow detail precludes mapping accurate luminance to HDR displays and limits meaningful re-exposure in post-production workflows. Although techniques have been proposed to convert LDR images to HDR through dynamic range expansion, they struggle to restore realistic detail in the over- and underexposed regions. To address this, we present DiffHDR, a framework that formulates LDR-to-HDR conversion as a generative radiance inpainting task within the latent space of a
video diffusion model. By operating in Log-Gamma color space, DiffHDR leverages spatio-temporal generative priors from a pretrained
video diffusion model to synthesize plausible HDR radiance in over- and underexposed regions while recovering the continuous scene radiance of the quantized pixels. Our framework further enables controllable LDR-to-HDR
video conversion guided by text
prompts or reference images. To address the scarcity of paired HDR
video data, we develop a pipeline that synthesizes high-quality HDR
video training data from static HDRI maps. Extensive experiments demonstrate that DiffHDR significantly outperforms state-of-the-art approaches in radiance fidelity and temporal stability, producing realistic HDR
videos with considerable latitude for re-exposure.
Read the full abstract ▾
All authors Zhengming Yu, Li Ma, Mingming He, Leo Isikdogan, Yuancheng Xu, Dmitriy Smirnov, Pablo Salamanca, Dao Mi, Pablo Delgado, Ning Yu, Julien Philip, Xin Li, Wenping Wang, Paul Debevec
Computer Vision
The Character Error Vector: Decomposable errors for page-level OCR evaluation
Jonathan Bourne, Mwiza Simbeye and Joseph Nockels
The Character Error Rate (CER) is a key metric for evaluating the quality of Optical Character Recognition (OCR). However, this metric assumes that text has been perfectly parsed, which is often not the case. Under page-parsing errors, CER becomes undefined, limiting its use as a metric and making evaluating page-level OCR challenging, particularly when using data that do not share a labelling schema. We introduce the Character Error Vector (CEV), a bag-of-characters evaluator for OCR. The CEV can be decomposed into parsing and OCR, and interaction error components. This decomposability allows practitioners to focus on the part of the Document Understanding pipeline that will have the greatest impact on overall text extraction quality. The CEV can be implemented using a variety of methods, of which we demonstrate SpACER (Spatially Aware Character Error Rate) and a Character distribution method using the Jensen-Shannon Distance. We validate the CEV's performance against other metrics: first, the relationship with CER; then, parse quality; and finally, as a direct measure of page-level OCR quality. The validation process shows that the CEV is a valuable bridge between parsing metrics and local metrics like CER. We analyse a
dataset of archival newspapers made of degraded images with complex layouts and find that state-of-the-art end-to-end models are outperformed by more traditional pipeline approaches. Whilst the CEV requires character-level positioning for optimal triage, thresholding on easily available values can predict the main error source with an F1 of 0.91. We provide the CEV as part of a Python library to support Document understanding research.
Read the full abstract ▾
Computer Vision
MMEmb-R1: Reasoning-Enhanced Multimodal Embedding with Pair-Aware Selection and Adaptive Control
Yuchi Wang, Dingkang Yang, Haiyang Yu et al. (+5 more)
MLLMs have been successfully applied to
multimodal embedding tasks, yet their generative
reasoning capabilities remain underutilized. Directly incorporating chain-of-thought
reasoning into
embedding learning introduces two fundamental challenges. First, structural misalignment between instance-level
reasoning and pairwise contrastive supervision may lead to shortcut behavior, where the model merely learns the superficial format of
reasoning. Second,
reasoning is not universally beneficial for
embedding tasks. Enforcing
reasoning for all inputs may introduce unnecessary computation and latency, and can even obscure salient semantic signals for simple cases. To address these issues, we propose MMEmb-R1, an adaptive
reasoning-based
multimodal embedding framework. We formulate
reasoning as a latent variable and introduce pair-aware
reasoning selection that employs counterfactual intervention to identify
reasoning paths beneficial for query-target
alignment. Furthermore, we adopt reinforcement learning to selectively invoke
reasoning only when necessary. Experiments on the MMEB-V2
benchmark demonstrate that our model achieves a score of 71.2 with only 4B parameters, establishing a new state-of-the-art while significantly reducing
reasoning overhead and inference latency.
Read the full abstract ▾
All authors Yuchi Wang, Dingkang Yang, Haiyang Yu, Weikang Bian, Jiefeng Long, Xiao Liang, Chao Feng, Hongsheng Li
Computer Vision
PoM: A Linear-Time Replacement for Attention with the Polynomial Mixer
David Picard, Nicolas Dufour, Lucas Degeorge et al. (+14 more)
This paper introduces the
Polynomial Mixer (PoM), a novel
token mixing mechanism with linear complexity that serves as a drop-in replacement for self-
attention. PoM aggregates input
tokens into a compact representation through a learned
polynomial function, from which each
token retrieves contextual information. We prove that PoM satisfies the contextual mapping property, ensuring that
transformers equipped with PoM remain universal sequence-to-sequence approximators. We replace standard self-
attention with PoM across five diverse domains: text generation, handwritten text recognition, image generation, 3D modeling, and Earth observation. PoM matches the performance of
attention-based models while drastically reducing computational cost when working with long sequences. The code is available at https://github.com/davidpicard/pom.
Read the full abstract ▾
All authors David Picard, Nicolas Dufour, Lucas Degeorge, Arijit Ghosh, Davide Allegro, Tom Ravaud, Yohann Perron, Corentin Sautier, Zeynep Sonat Baltaci, Fei Meng, Syrine Kalleli, Marta López-Rauhut, Thibaut Loiseau, Ségolène Albouy, Raphael Baena, Elliot Vincent, Loic Landrieu
Computer Vision
Lightweight Multimodal Adaptation of Vision Language Models for Species Recognition and Habitat Context Interpretation in Drone Thermal Imagery
Hao Chen, Fang Qiu, Fangchao Dong et al. (+3 more)
This study proposes a lightweight
multimodal adaptation framework to bridge the representation gap between RGB-pretrained VLMs and thermal infrared imagery, and demonstrates its practical utility using a real drone-collected
dataset. A thermal
dataset was developed from drone-collected imagery and was used to fine-tune VLMs through
multimodal projector
alignment, enabling the transfer of information from RGB-based visual representations to thermal radiometric inputs. Three representative models, including InternVL3-8B-Instruct, Qwen2.5-VL-7B-Instruct, and Qwen3-VL-8B-Instruct, were benchmarked under both closed-set and open-set prompting conditions for species recognition and instance enumeration. Among the tested models, Qwen3-VL-8B-Instruct with open-set prompting achieved the best overall performance, with F1 scores of 0.935 for deer, 0.915 for rhino, and 0.968 for elephant, and within-1 enumeration accuracies of 0.779, 0.982, and 1.000, respectively. In addition, combining thermal imagery with simultaneously collected RGB imagery enabled the model to generate habitat-context information, including land-cover characteristics, key landscape features, and visible human disturbance. Overall, the findings demonstrate that lightweight projector-based adaptation provides an effective and practical route for transferring RGB-pretrained VLMs to thermal drone imagery, expanding their utility from object-level recognition to habitat-context interpretation in ecological monitoring.
Read the full abstract ▾
All authors Hao Chen, Fang Qiu, Fangchao Dong, Defei Yang, Eve Bohnett, Li An
Computer Vision
SEM-ROVER: Semantic Voxel-Guided Diffusion for Large-Scale Driving Scene Generation
Hiba Dahmani, Nathan Piasco, Moussab Bennehar et al. (+5 more)
Scalable generation of outdoor driving scenes requires 3D representations that remain consistent across multiple viewpoints and scale to large areas. Existing solutions either rely on image or
video generative models distilled to 3D space, harming the
geometric coherence and restricting the rendering to training views, or are limited to small-scale 3D scene or object-centric generation. In this work, we propose a 3D generative framework based on $\Sigma$-Voxfield grid, a discrete representation where each occupied voxel stores a fixed number of colorized surface samples. To generate this representation, we train a semantic-conditioned
diffusion model that operates on local voxel neighborhoods and uses 3D positional encodings to capture spatial structure. We scale to large scenes via progressive spatial outpainting over overlapping regions. Finally, we render the generated $\Sigma$-Voxfield grid with a deferred rendering module to obtain photorealistic images, enabling large-scale multiview-consistent 3D scene generation without per-scene optimization. Extensive experiments show that our approach can generate diverse large-scale urban outdoor scenes, renderable into photorealistic images with various sensor configurations and camera trajectories while maintaining moderate computation cost compared to existing approaches.
Read the full abstract ▾
All authors Hiba Dahmani, Nathan Piasco, Moussab Bennehar, Luis Roldão, Dzmitry Tsishkou, Laurent Caraffa, Jean-Philippe Tarel, Roland Brémond
Computer Vision
Extending ZACH-ViT to Robust Medical Imaging: Corruption and Adversarial Stress Testing in Low-Data Regimes
Athanasios Angelakis and Marta Gomez-Barrero
The recently introduced ZACH-ViT (Zero-
token Adaptive Compact Hierarchical
Vision Transformer) formalized a compact permutation-invariant
Vision Transformer for medical imaging and argued that architectural
alignment with spatial structure can matter more than universal
benchmark dominance. Its design was motivated by the observation that positional
embeddings and a dedicated class
token encode fixed spatial assumptions that may be suboptimal when spatial organization is weakly informative, locally distributed, or variable across biomedical images. The foundational study established a regime-dependent clean performance profile across MedMNIST, but did not examine robustness in detail. In this work, we present the first robustness-focused extension of ZACH-ViT by evaluating its behavior under common image corruptions and
adversarial perturbations in the same low-data setting. We compare ZACH-ViT with three scratch-trained compact baselines, ABMIL, Minimal-ViT, and TransMIL, on seven MedMNIST
datasets using 50 samples per class, fixed hyperparameters, and five random seeds. Across the
benchmark, ZACH-ViT achieves the best overall mean rank on clean data (1.57) and under common corruptions (1.57), indicating a favorable balance between baseline predictive performance and robustness to realistic image degradation. Under
adversarial stress, all models deteriorate substantially; nevertheless, ZACH-ViT remains competitive, ranking first under FGSM (2.00) and second under PGD (2.29), where ABMIL performs best overall. These results extend the original ZACH-ViT narrative: the advantages of compact permutation-invariant
transformers are not limited to clean evaluation, but can persist under realistic perturbation stress in low-data medical imaging, while
adversarial robustness remains an open challenge for all evaluated models.
Read the full abstract ▾