Computer Vision
PRIMA: Boosting Animal Mesh Recovery with Taxonomic Priors and Adaptation-Driven Data Bootstrapping
Xiaohang Yu, Ti Wang and Mackenzie Weygandt Mathis
Recovering articulated 3D animal meshes from
monocular images is challenging due to the long-tailed distribution of species and limited large-scale 3D supervision. Existing methods rely on small annotated
datasets and thus regress to average shapes and poses, leading to poor generalization to rare species and articulations. We present PRIMA (PRIors for Mesh Adaptation), a framework that tackles these limitations with two components: taxonomic shape priors and adaptation-driven data bootstrapping. First, we derive taxonomic shape priors from BioCLIP image features, which are learned under taxonomic supervision, to inject species-level knowledge into mesh reconstruction, improving shape estimation across diverse quadrupeds. Second, we propose an adaptation-driven bootstrapping scheme that turns abundant 2D animal annotations into reliable pseudo-3D supervision via lightweight instance-level mesh refinement and quality-aware filtering. This yields Quadruped3D, a large-scale pseudo-3D
dataset constructed from real-world animal images, containing approximately 9K pseudo-3D annotations. Together, these components alleviate the long-tail problem: extensive experiments on Animal3D, CtrlAni3D, Quadruped2D, and Animal Kingdom demonstrate state-of-the-art performance for mesh reconstruction, with particularly strong gains on challenging poses and underrepresented species. The code is available at https://github.com/AdaptiveMotorControlLab/PRIMA.
Read the full abstract ▾
Computer Vision
Do Multimodal Agents Really Benefit from Tool Use? A Systematic Study of Capability Gains
Jiawei Guo, Donglei Yu, Yu Chen et al. (+6 more)
Tool-augmented
multimodal agents show strong
benchmark gains, often taken as evidence that
agents have learned to use tools. We argue that this interpretation can be premature: a tool-call trace alone does not show whether the tool supplied answer-critical information. We study two representative “thinking with images”
agents, Thyme and DeepEyesV2, across real-world understanding, OCR, chart understanding, and mathematical
reasoning. Each
agent is compared with its Tool-Free counterpart and with a Pure-Text Reasoner trained from the same source pool without tool-calling trajectories. Tool access yields little consistent aggregate improvement, does not reliably reduce generated-
token cost, and leaves only a small tool-only solved set: 93% of DeepEyesV2's tool-solved problems and 96% of Thyme's are also solved by at least one non-tool setting. Mechanism ablations further show that the full tool-use loop does not consistently outperform either the tool-call format or the returned execution result alone. In the settings we study, the analyzed
agents appear to learn tool-calling patterns more reliably than tool-contributed capabilities, suggesting that evaluation should distinguish tool availability from whether tools actually expand what
agents can solve.
Read the full abstract ▾
All authors Jiawei Guo, Donglei Yu, Yu Chen, Xiang Wang, Shuai Li, Xinpei Zhao, Huaxing Liu, Qinghao Wang, Minpeng Liao
Computer Vision
Multi-modal Video Representation Alignment for Robust Self-supervised Driver Distraction Detection
David J. Lerch, Livien Majer, Zeyun Zhong et al. (+3 more)
Robust self-supervised learning of multi-modal
video representations is critical for real-world applications such as driver distraction detection, where multiple sensors provide complementary but noisy signals. Conventional contrastive objectives, such as InfoNCE, assume all negatives are equally informative and all positives are reliable. However, this assumption is frequently violated in multi-modal data due to viewpoint changes, occlusions, or semantic overlap across modalities. In this work, we propose a novel framework for multi-modal global
alignment that addresses these challenges by jointly modeling faulty negatives and unreliable or faulty positives. We introduce soft targets derived from cycle-consistency scores to relax the hard-negative assumption, and a weighting mechanism based on similarity distributions to mitigate the impact of noisy or faulty positives. Our approach extends traditional pairwise
alignment to a principled global multi-modal setting, aggregating
alignment information across all modality pairs. We evaluate our method on the Drive&Act
dataset, demonstrating that it consistently outperforms both pairwise and existing global
alignment baselines across RGB, IR, Depth, and Skeleton modalities. Cross-view ablation studies further show strong generalization to unseen camera perspectives, highlighting the robustness of our representations. Overall, our framework provides a scalable and effective solution for self-supervised global multi-modal representation learning, enabling reliable driver distraction detection and pioneering in real-world multi-modal
video understanding. Our code will be published on GitHub.
Read the full abstract ▾
All authors David J. Lerch, Livien Majer, Zeyun Zhong, Manuel Martin, Frederik Diederichs, Rainer Stiefelhagen
Computer Vision
TROPHIES: Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videos
Jinpeng Liu, Yukang Xu, Yutong Li et al. (+1 more)
Reconstructing humans and their surrounding environments in a globally consistent 4D space is essential for comprehensive perception. However, prior works typically assume single-view inputs or decouple humans, scenes, and cameras, making them unable to recover coherent geometry, stable motion, and physically aligned trajectories. These limitations motivate us to introduce a new task: unified human-scene-camera reconstruction from multi-view
videos, which aims to jointly estimate dynamic humans, static scenes, and camera poses in one global coordinate frame. We propose TROPHIES–Temporal Reconstruction of Places, Humans, and Cameras from Multi-view
Videos-a unified framework tailored for this task. TROPHIES features a Human Branch that models humans through temporal and spatial
reasoning, and a Scene Branch that reconstructs static geometry with human-aware
attention. A global
alignment and optimization module couples both branches by enforcing scale consistency, contact priors, and cross-view temporal coherence. Experiments on EgoHuman and EgoExo4D demonstrate that TROPHIES achieves globally aligned, physically plausible 4D reconstructions and consistently outperforms existing paradigms in both global fidelity and human-scene consistency.
Read the full abstract ▾
All authors Jinpeng Liu, Yukang Xu, Yutong Li, Xingyu Liu
Computer Vision
VEDAL: Variational Error-Driven Asynchronous Learning for 3D Gaussian Splatting Pruning
Aoduo Li, Jiancheng Li, Huan Ye et al. (+5 more)
3D Gaussian Splatting (3DGS) achieves remarkable novel view synthesis quality with real-time rendering, yet suffers from excessive
memory consumption due to millions of Gaussian primitives. Existing
pruning methods rely on heuristic importance scores or synchronous batch updates, leading to suboptimal compression and training instability. We propose VEDAL, a principled framework that formulates Gaussian
pruning as variational free energy minimization. Our approach introduces (1) a prediction-error gating mechanism that asynchronously activates
pruning based on per-Gaussian reconstruction uncertainty, and (2) a variational uncertainty head that models
pruning decisions as latent variables with learnable priors. The free energy objective naturally balances reconstruction fidelity against model complexity through an information-theoretic lens. Extensive experiments on Mip-NeRF 360, Tanks&Temples, and Deep Blending demonstrate that VEDAL achieves 5.2x compression with only 0.31 dB PSNR drop, outperforming PUP 3D-GS by +0.05 dB at a higher compression ratio and LightGaussian by +0.35 dB at comparable quality, while maintaining real-time rendering at 185 FPS.
Read the full abstract ▾
All authors Aoduo Li, Jiancheng Li, Huan Ye, Hongjian Xu, Shiting Wu, Xiujun Zhang, Zimeng Li, Xuhang Chen
Computer Vision
Detecting Pen-In-Air States from Video: A Proof-of-Concept Toward Complementary Handwriting Analysis
Lauren Sismeiro, Remy Plastre, Binbin Xu et al. (+2 more)
Dynamic aspects of handwriting are critical for assessing developmental disorders such as dysgraphia and are typically captured using digitizing tablets. However, tablet-based sensing restricts analysis of Pen-Up behavior to a short proximity range above the writing surface, potentially missing high-lift in-air movements. As a proof of concept, we investigate whether top-view
video can provide a complementary source of information for inferring pen-contact states without relying on tablet proximity sensing. We propose an interpretable hybrid pipeline combining pen-tip tracking using a YOLO-based detector with kinematic feature extraction and machine learning classification. A pilot
dataset of diverse handwriting
videos was manually annotated at the frame level and evaluation used a Leave-One-
Video-Out (LOVO) protocol. The method achieved reliable event-level detection of Pen-Up segments, with an F_2 score up to 0.805, consistent with the emphasis on recall in a screening-oriented setting. These results support the feasibility of
video-based Pen-Up detection as a low-cost and non-intrusive complement to digitizing tablets, and provide a foundation for future large-scale studies.
Read the full abstract ▾
All authors Lauren Sismeiro, Remy Plastre, Binbin Xu, Frederic Puyjarinet, Gerard Dray
Computer Vision · Cross-listed from cs.LG
Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging
Tim Nielen, Sameer Ambekar, Johannes Kiechle et al. (+2 more)
Entropy minimization (EM) is the dominant objective for test-time adaptation, yet its failure mode, model collapse, remains poorly understood. In this work, we show that distribution shifts can cause feature clusters corresponding to distinct classes in the model's representation space to merge, while the decision boundary remains fixed. This induces a systematic skew in the predicted class distribution, referred to as prediction bias. Prediction bias refers to a shift in the predicted class distribution, with some classes overrepresented and others suppressed. We show that
entropy minimization amplifies this prediction bias by tightening the existing clusters, reinforcing the incorrect groupings until all predictions collapse to a trivial solution. Next, to demonstrate the significance of prediction bias and mitigate it, we further propose Distribution Shift Bias Reduction (DSBR), a bias-correcting objective that specifically targets this failure mode by equalizing the contribution of each predicted class to the unsupervised
entropy minimization loss. To study this failure mode, we design suitable adaptation settings using four medical-imaging
datasets and additionally evaluate on ImageNet-C. We find that DSBR consistently stabilizes test-time adaptation, prevents model collapse, and matches or outperforms state-of-the-art methods. Moreover, DSBR operates solely at test-time.
Read the full abstract ▾
All authors Tim Nielen, Sameer Ambekar, Johannes Kiechle, Daniel M. Lang, Julia A. Schnabel
Computer Vision
Hallucination-Aware Diffusion Sampling for Inverse Problems via Robust Prior Updates
Pengfei Jin, Yiqi Tian, Kailong Fan et al. (+2 more)
Diffusion-based inverse problem solvers can produce realistic reconstructions, but realism alone does not ensure that the recovered details are supported by the measurement. We study this failure as measurement-conditioned
hallucination: visually meaningful content that is either implausible or inconsistent with the measured instance. Our analysis separates Bayes-rule-based
diffusion inverse solvers into a prior update and a measurement-conditioning step, showing that hallucinated content can enter through the prior-side proposal before the measurement correction is applied. Motivated by this view, we propose Robust Prior Update (RPU), a solver-level module that probes the local stability of the
diffusion prior update, re-anchors the resulting displacement at the current iterate, and leaves the measurement update unchanged. We instantiate RPU in DPS and evaluate it on FFHQ and ImageNet inverse problems using automatic metrics and human faithfulness studies. On FFHQ, RPU improves PSNR and LPIPS over DPS across box inpainting, Gaussian deblurring, and motion deblurring. In human judgments, RPU receives 91.9% of blind non-tie majority preferences and 91.1% of ground-truth-assisted non-tie preferences on FFHQ box inpainting, while the ImageNet Gaussian reader study is tie-heavy but favors RPU among non-tie cases. These results support a targeted claim: robustifying the prior update can improve instance faithfulness in
diffusion inverse solvers, especially when the prior shapes weakly constrained content.
Read the full abstract ▾
All authors Pengfei Jin, Yiqi Tian, Kailong Fan, Bingjie Qi, Quanzheng Li
Computer Vision
Training-Free Composed Video Retrieval via Visual Representation-Guided Video-LLM Reasoning
Yang Liu, Qianqian Xu, Peisong Wen et al. (+2 more)
Recent advances in large
vision-
language models have expanded
video retrieval from simple text-based search to more flexible scenarios, where users may specify the desired result through both visual examples and textual instructions. In the CVPR 2026 Reason-Aware Composed
Video Retrieval Challenge, the system is required to retrieve a target
video according to a reference
video and a modification instruction. To address this task, we develop Visual Representation-Guided
Video-LLM
Reasoning for Training-Free Composed
Video Retrieval. Our framework first uses frozen DINOv3 models to obtain a compact set of visually relevant candidates, and then applies large
vision-
language models to evaluate whether each candidate satisfies the modification instruction. A final
reasoning-based refinement is further performed on the top candidates to improve the first-ranked prediction. Without training, our system achieves 48.78 Recall@1 and 51.48 Recall@5 on the test set. Future work may further improve
retrieval accuracy through stronger
video-LLMs and detailed integration between visual representations and
language reasoning.
Read the full abstract ▾
All authors Yang Liu, Qianqian Xu, Peisong Wen, Siran Dai, Qingming Huang
Computer Vision
Deep Learning for Remote Sensing to Improve Flood Inundation Mapping
Yogesh Bhattarai, Vijay Chaudhary, Wai Lim Ku et al. (+1 more)
Flooding is the most pervasive natural disaster worldwide. Timely and accurate flood inundation mapping are essential for informing disaster risk management. Optical satellite missions provide high-resolution, multispectral observations critical for flood detection and inundation mapping. However, their operational utility is severely constrained by cloud cover during extreme precipitation events. Conventional cloud-removal techniques based on temporal compositing or interpolation often fail to capture inundation dynamics. In this study, we introduce a cloud-removal framework for flood imagery based on Denoising
Diffusion Probabilistic Models, leveraging the Masked
Diffusion Transformer architecture. The proposed approach exploits self-
attention mechanisms to capture wider spatial context and employs masked
token modeling to explicitly learn the reconstruction of cloud-obscured regions. Trained on multispectral Sentinel-2B flood scenes with realistic cloud patterns, the model generates cloud-free image realizations that preserve both visual fidelity and hydrological consistency. Reconstruction performance is evaluated using standard image quality metrics alongside flood-specific hydrological measures, demonstrating improved continuity of water bodies and preservation of spectral signatures critical for water detection indices. The results indicate that
diffusion-based generative modeling offers a robust and physically consistent alternative for cloud removal in optical flood monitoring, enabling more reliable, continuous observations to support disaster risk management and flood-related decision making.
Read the full abstract ▾
All authors Yogesh Bhattarai, Vijay Chaudhary, Wai Lim Ku, Sanjib Sharma
Computer Vision · Cross-listed from cs.LG
Measurement Geometry and Design for Trustworthy Generative Inverse Problems
Pengfei Jin, Na Li and Quanzheng Li
Generative models are increasingly used as priors for inverse problems, but their ability to produce realistic images creates a basic trust problem: a plausible reconstruction may be supported by the measurements, or it may be filled in by the prior along unobserved directions. This distinction is especially important in medical imaging, where acquisition operators are designed under scan-time, dose, and calibration constraints. We study generative inverse problems from a measurement-geometry perspective. The central question is whether a fixed measurement operator can distinguish nearby images that are plausible under the generative prior, and whether this relationship can guide better measurements. We introduce a local measurement-
manifold compatibility measure that quantifies how well the operator observes prior-relevant tangent directions. Under local regularity assumptions, we prove that this quantity controls the stable part of the reconstruction error, while the generative prior controls off-
manifold drift. This worst-direction certificate motivates practical fixed and sequential acquisition rules based on overall local volume preservation, including a posterior-cloud design that adapts measurements at test time without training a sampling policy. Across row-sampling, tomographic, and MR acquisition settings, the proposed scores predict failure modes, explain measurement-induced
hallucinations, and guide better sampling. In fastMRI Cartesian sampling, posterior-cloud measurement design improves over strong non-learned ACS-preserving baselines, including variable-density and Poisson-like masks.
Read the full abstract ▾
Computer Vision
Cross-Domain Dead Tree Detection via Knowledge Distillation in Aerial Imagery
Anis Ur Rahman, Mete Ahishali, Einari Heinaro et al. (+1 more)
Detecting dead trees in aerial imagery is vital for assessing forest health, especially as tree mortality increases globally due to
climate change, but domain variability and scarce labeled data often limit model generalization. This study advances the TreeMort-1T-UNet (Tree Mortality 1-Task U-Net) model, initially trained on Finnish aerial imagery (source domain), by applying knowledge
distillation (KD) to adapt it to various target domains, including Polish, German, and Estonian
datasets representing diverse forest types. We assess four KD variants: Basic, Self, Feature-level, and Ensemble, against a fine-tuning baseline, using Mean Tree IoU, Instance F1-score, Instance Precision, and Mean Centroid Error as key metrics, alongside representational analyses (e.g., cosine similarity, CKA, SSIM, t-SNE, and linear probing) for domain invariance. Feature-level KD outperforms others, yielding a Mean Tree IoU of 0.106, Instance F1-score of 0.63, Instance Precision of 0.55, and Mean Centroid Error of 3.039 on the Polish
dataset, with robust precision across other target domains (e.g., 0.15 on Finnish, 0.67 on Polish, 0.60 on German, 0.59 on Estonian). It excels in low-data scenarios with fewer false positives and shows superior representational invariance (e.g., higher deep-layer CKA/SSIM, better domain mixing in t-SNE, and linear probing AUC of 0.95), making it ideal for precision-critical forestry applications. Additional ablation studies confirm that key components like feature
alignment enhance its performance balance across metrics. Our findings demonstrate KD's potential to enhance transfer learning in remote sensing, offering a scalable, domain-robust tool for ecological monitoring and sustainable forest management.
Read the full abstract ▾
All authors Anis Ur Rahman, Mete Ahishali, Einari Heinaro, Samuli Junttila
Computer Vision · Cross-listed from cs.HC
Quantitative Movement Testing: Measuring Chronic Pain Patient Movements from a Single Smartphone Video
Pranav Mahajan, Amanda Wall, Eleonora Maria Camerone et al. (+7 more)
Chronic pain diminishes quality of life by decreasing functional ability, yet objectively measuring this functional impact remains challenging in real-world settings. While optical motion capture provides high precision for assessing altered movement quality, it is costly and restricted to laboratory environments. We aimed to develop and validate Quantitative Movement Testing (QMT), a computer
vision pipeline extracting 3D kinematic biomarkers from standard
monocular smartphone
video, balancing clinical accessibility with biomechanical accuracy. We validated the QMT pipeline, utilising deep learning-based 3D pose-estimation, against gold-standard optical motion capture in healthy controls (N=13). Following leave-one-subject-out calibration to correct systematic bias, we deployed QMT in two prospective clinical cohorts to assess real-world utility: a pre- and post-intervention trial for fibromyalgia patients, and a 30-day longitudinal at-home monitoring study of chronic sciatica patients and healthy controls. In laboratory validation, QMT extracted clinical kinematic metrics with high agreement to optical motion capture, yielding strong correlations (r > 0.85) and low mean absolute errors. QMT demonstrated high test-retest reliability (r > 0.86) in fibromyalgia patients and successfully tracked day-to-day movement fluctuations in chronic sciatica. While real-world home settings introduced higher measurement variance than lab settings, QMT found group-level differences between healthy controls and sciatica patients based entirely on remote recordings.
Monocular 3D pose estimation offers a scalable alternative to traditional assessments. QMT provides an objective, accessible biomarker for tracking disease progression and treatment response in clinical trials, though further research is needed to optimise reliability in home environments.
Read the full abstract ▾
All authors Pranav Mahajan, Amanda Wall, Eleonora Maria Camerone, Julie Stebbins, Eoin Kelleher, Shuangyi Tong, Annina Schmid, Katja Wiech, Anushka Irani, Ben Seymour
Computer Vision
Neural Acquisition & Representation of Subsurface Scattering
Arjun Majumdar, Raphael Braun and Hendrik Lensch
We present a method to acquire and estimate the sub-surface scattering properties of light transport at a highly detailed level by learning the pixel footprint response at each point on the object surface. The reconstruction leverages 3D scanning techniques as input to a U-Net CNN. A stereo projector-camera setup using phase-shifted profilometry (PSP) patterns efficiently captures the data for a variety of scattering objects. Reconstructing dense pixel footprints allows for relighting with arbitrary high-resolution projector patterns. The final output is a relit color image. Qualitative and quantitative comparison against illuminated real-world captured images demonstrate that the predicted footprints are almost identical to the actual responses. The same model is trained for multiple views across multiple objects such that the learned representations can be used to generalize to unseen sub-surface scattering materials as well.
Read the full abstract ▾
Computer Vision
Selective Linkage Precision for Released Medical Vision-Language Models
Soroosh Tayebi Arasteh, Mahshad Lotfinia, Sven Nebelung et al. (+1 more)
Hospitals may release de-identified chest radiographs while restricting access to their reports. An attacker who already has reports from the same population and their metadata can use a
vision-
language model (VLM) to match images to reports. We introduce selective linkage precision, which measures correct matches among the highest-confidence queries after filtering candidates by the available attributes. Across 8 public VLMs and 3 chest radiograph
datasets, the maximum whole-pool Recall@1 is 0.40%. However, this metric averages over every query without attribute filtering. On ReXGradient-160K, MedSigLIP-448 reaches 67.68% precision at a budget of 100 after filtering by sex, age band, and projection, with a median of 327 candidate reports. On the same eligible query set, an attribute-only policy reaches 24.99% and the model alone reaches 1.98%. Adding acquisition year, available in only 1 of the 3
datasets, yields 99.05% precision and a median of 24 candidates. Uniform guessing on the observed selected queries has an expected precision of 19.91%. Expanding the primary-setting collection to all 123,100 ReXGradient studies lowers precision to 37.23%. On BioViL-T, the tested head finetuning procedures do not lower linkage below the public checkpoint. Selective linkage precision requires inference without retraining and could help hospitals evaluate which attributes to publish.
Read the full abstract ▾
All authors Soroosh Tayebi Arasteh, Mahshad Lotfinia, Sven Nebelung, Daniel Truhn
Computer Vision
Vision-language Models for Driver Monitoring Systems: A Driver Activity Description Dataset
David J. Lerch, Sarath Mulugurthi, Manuel Martin et al. (+2 more)
Understanding subtle driver actions is essential for building reliable driver monitoring systems. Existing visionlanguage models (VLMs) are trained on general
datasets and struggle to recognize fine distinctions in driver behaviors. This paper addresses this limitation by creating a detailed natural
language version of the Drive&Act
dataset. We evaluate three VLMs on our new
benchmark using LLM-based scoring methods. Their performance on the new
benchmark shows that they cannot reliably generate accurate fine-grained driver activity descriptions. Based on the labeled Drive&Act
dataset we create a new Drive&Act description
dataset containing finegrained descriptions to train VLMs on driver activity understanding. Cross
dataset evaluation on the Driver Monitoring
Dataset (DMD) shows that the VLM fine-tuned on our new Drive&Act description
dataset generalizes well to actions in the DMD
dataset. The VLM fine-tuned on our Drive&Act description
dataset achieves an ACCR score of 76 outperforming the zero-shot VLM baseline with an ACCR score of 66. These findings demonstrate that adapting VLMs with richly described driver actions can significantly improve their ability to interpret driver behavior while also highlighting the need for more diverse
datasets to support broader generalization in future applications. Our Drive&Act description
dataset and code will be publicly available on GitHub.
Read the full abstract ▾
All authors David J. Lerch, Sarath Mulugurthi, Manuel Martin, Frederik Diederichs, Rainer Stiefelhagen
Computer Vision
From Extrinsic to Intrinsic: Geodesic-Guided Representation Learning for 3D Geometric Data
Yuming Zhao, Junhui Hou, Qijian Zhang et al. (+2 more)
Geometric analysis fundamentally distinguishes between extrinsic and intrinsic perspectives. The dominant paradigm in current 3D representation learning relies on either extrinsic spatial structures or high-level semantics, struggling to capture the essence of shape identity and underlying
manifold topology. To bridge this gap, we introduce a novel 3D representation learning paradigm, namely PRISM, for Pre-training, which learns isometric
embeddings by Recovering the Intrinsic Surface geodesic Metric. PRISM incorporates a
topology-enforcing objective that explicitly constrains the structure of latent space, alongside a specialized two-stage training recipe mitigating sample imbalance inherent in the distribution of geodesic distances. Experiments demonstrate that our approach shows satisfactory accuracy, robustness, and high efficiency in geodesic distance prediction and achieves superior performance across diverse downstream tasks, including shape recognition, surface parameterization, and non-rigid correspondence. The code will be publicly available at https://github.com/AidenZhao/PRISM.
Read the full abstract ▾
All authors Yuming Zhao, Junhui Hou, Qijian Zhang, Jia Qin, Ying He
Computer Vision · Cross-listed from cs.LG
A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs
Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati et al. (+1 more)
The vulnerability of deep neural networks to
adversarial examples poses a significant challenge for real-world deployment. Existing techniques to enhance deep network robustness rely on
adversarial training, an approach that is powerful but computationally intensive and typically tailored to specific attack types. To address these limitations, existing works have explored techniques such as adding gaussian noise or filtering images, both of which can boost the network robustness to various
adversarial attacks, albeit modestly. Here, we theoretically demonstrate that these two approaches enhance robustness against
adversarial attacks through complementary mechanisms, resulting in supralinear robustness when combined. Building on this insight, we experimentally show that a simple preprocessor combining Gaussian noise and bilateral filtering yields supralinear improvements in
adversarial robustness with minimal computational cost. Next, we combine our preprocessor with
adversarial training and test on RobustBench to assess its supralinear improvement over state-of-the-art defenses. First, this combination ranks second on AutoAttack and third overall, while using only $\sim$35% of the training FLOPs, using a model with $\sim$50% less parametets, trained with $\sim$33% of the epochs and $\sim$15% the data compared to state-of-the-art defenses. Second, our method scales efficiently, matching the accuracy of competing models with roughly 2-8x less total compute across 3 orders of magnitude. Overall, our approach provides a principled and easily integrable framework for enhancing
adversarial robustness, offering negligible computational overhead and a simple yet theoretically grounded design.
Read the full abstract ▾
All authors Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati, Pau Vilimelis Aceituno
Computer Vision
Analyzing the Optimization Conflict Between Image- and Text-Based Person Re-Identification
Karina Kvanchiani and Timur Mamedov
Unifying image-based (I2I) and text-based (T2I) person re-identification (ReID) within a single model is complicated by a fundamental conflict between their training objectives. While I2I ReID focuses on identity-level invariance across images of the same person captured by multiple cameras, T2I ReID is driven by instance-specific textual descriptions tied to unique visual traits. When both objectives are optimized on a shared
vision encoder, they impose opposing requirements on the same
embeddings - I2I contracts intra-identity distances while T2I requires them to persist - so improving one task's representation actively degrades the other's. This paper analyzes this optimization conflict and shows empirically that the two objectives are not jointly satisfiable by a single unconstrained encoder. Motivated by this finding, we investigate the conflict through a controlled two-stage pipeline: a shared
vision encoder is first trained for I2I invariance, then frozen while only the text encoder is aligned to the fixed visual space, preventing the T2I objective from unwinding the identity-level structure already learned. We conduct extensive experiments across multiple configurations, varying domain-mixing procedures, learning strategies, and task objectives. We observe that I2I pre-training improves subsequent T2I performance, while textual supervision during this pre-training further improves T2I performance, although the
vision encoder is frozen during subsequent cross-modal training. We believe these insights provide a meaningful step toward unified ReID systems that reconcile identity-level and instance-level supervision within a single representation.
Read the full abstract ▾
Computer Vision · Cross-listed from stat.ML
Bayesian meta-learning for modeling Alzheimer's disease progression
Clara Hoffmann and Nadja Klein
Predicting whether an individual with Alzheimer's disease will experience mild or severe disease progression is essential for personalized treatment. Typically, practitioners seek to predict the distribution of a discrete disease score, conditional on an individual's current MRI volume and their historical disease trajectory. Classical statistical regression models and single-task neural networks are not well-suited for this purpose because fitting separate models is infeasible (since each individual typically has few observations), while ignoring individual-level correlation leads to poor generalization. Meta-learning, in contrast, provides a natural avenue to dynamically predict distributions without retraining and model nonlinear relationships between the outcome and
covariates. Motivated by this, we propose a
Bayesian meta-learner that is trained on multiple individuals but tailors the predictive disease score distribution to each individual's historical data. Our model predicts on unseen individuals without retraining, scales linearly with the number of historical observations, and is guaranteed to be less overconfident when predicting long-term disease scores compared to its deterministic counterpart. On real-world data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, our model achieves performance competitive with both single-task models and deterministic meta-learners, while substantially improving performance when predicting long-term disease progression.
Read the full abstract ▾
Computer Vision
Chroma Clues: Leveraging Color Statistics to Detect Synthetic Images
Lea Uhlenbrock, Davide Cozzolino and Christian Riess
The
evolution and dissemination of AI-synthesized images is occurring at an unprecedented rate. Image generators are making rapid progress in their goal of perfectly imitating natural images, which also challenges image forensics. In this work, we exploit an underexplored cue in current generative models, namely their weakness to imitate color statistics of natural images. We first show that the LPIPS loss used for training image generators is less sensitive to chrominance than to luminance, which may lead to statistical discrepancies in the colors of synthetic images. Building on this observation, we then introduce six hand-crafted color transformations and a method to learn a task-optimized color transform to statistically expose generated images. These transformations can be used in various ways. First, we define color-sensitive features at pixel-level or patch-level. A simple, interpretable classifier achieves with these features an average generalization accuracy of 93.27% and strong robustness against six types of post-processing. Second, we demonstrate that the transformations exhibit characteristic visual noise patterns in natural and synthetic image areas, which enables an intuitive visual image evaluation. Third, we demonstrate that the transforms can enhance color patterns in generated images for improved multiclass attribution.
Read the full abstract ▾
Computer Vision
CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representations
Chengfeng Wu, Tao Zou, Yanru Wu et al. (+1 more)
Multi-task learning (MTL) aims to construct a joint model for multiple tasks by sharing a common representation across domains. To achieve this goal, existing optimization-centric methods either balance task gradients or modify the shared architecture. However, as these approaches remain agnostic to the content of the shared representation, they fail to disentangle task-relevant structure from spurious context, leading to negative transfer and poor generalization. To overcome this limitation, we propose Causal Orthogonal Representations for Multi-Task Learning (CORE-MTL), a causally motivated representation-centric framework that encourages a structured semantic-residual factorization of the shared representation, concentrating task-relevant structure in the semantic stream while relegating nuisance variation to the residual stream. We instantiate this framework in the visual domain by leveraging physical priors for structured scenes and statistical constraints for attributes. Theoretically, our method enjoys a tighter out-of-distribution generalization bound than optimization-centric methods and reduces task gradient interference without explicit gradient projection or reweighting. Empirically, CORE-MTL consistently outperforms existing methods on visual multi-task
benchmarks in both in-distribution and out-of-distribution settings. Code is publicly available at https://github.com/Hope-Rita/CORE-MTL.
Read the full abstract ▾
All authors Chengfeng Wu, Tao Zou, Yanru Wu, Jingge Wang
Computer Vision
Symmetry-Aware 9D Pose Estimation with Sim(3)-Consistent Feature and Spherical Inception Convolution
Panfei Cheng, Hongshan Yu, Wenrui Chen et al. (+3 more)
Object pose estimation is a fundamental problem for an
agent system to perceive or manipulate objects in images or
videos. However, current instance-level methods struggle with generalization to unseen objects. Category-level methods seek to address this, but remain constrained by the complexities of learning in the non-linear Sim(3) space and intra-class variations. To address these challenges, We propose an effective method for category-level object pose estimation with two key innovations: (1) A translation/size
estimator, featuring a semantic-guided
symmetry-aware module that leverages robust generalization capabilities of a large
vision model (LVM) to infer
symmetry points, resulting in accurate translation and size without shape priors. This result serves as a precomputed cue for rotation estimation, thereby reducing the difficulty of learning in the non-linear Sim(3) space and laying a robust foundation for tackling the inherently more challenging rotation estimation. (2) A feature fusion module, based on our proposed spherical large-kernel inception convolution, fuses semantic features from the LVM with systematically computed
geometric features to extract essential pose features from intra-class variations by modeling long-range dependencies without excessive computational cost. Built on these innovations, we achieve SOTA on
benchmarks and real-world scenes, while developing a robust robotic picking system capable of handling diverse objects. Our code will be available at the project page: {\hypersetup{urlcolor=blue}https://panfei-cheng.github.io/SSH-Pose}.
Read the full abstract ▾
All authors Panfei Cheng, Hongshan Yu, Wenrui Chen, Xiaojun Tang, Jian Liu, Naveed Akhtar
Computer Vision
Order within Chaos: Capturing Intrinsic Energy Anomalies for AI-Manipulated Image Forgery Localization
Yiming Wang, Baiqi Wu, Qingming Li et al. (+3 more)
Recent advancements in generative AI have led to image editing models capable of producing realistic forgeries that evade traditional image forgery localization methods, as these approaches depend on physical noise absent in synthetic data. To address this challenge, we theoretically demonstrate that the
diffusion process inherently suppresses local high-frequency variance, creating a statistical energy gap that is distinguishable from the natural
entropy of optical imaging. Guided by this insight, we propose FLAME, a unified framework that utilizes a LAD map to capture these intrinsic anomalies, coupled with a parameter-efficient adapter for SAM to achieve precise, pixel-level forgery localization. Furthermore, to bridge the lag between forensic
benchmarks and evolving generative models, we introduce EditStream, an automated pipeline for continuous, instruction-based training data synthesis. Extensive experiments demonstrate that FLAME establishes a new state-of-the-art, significantly outperforming previous methods on AI-generated forgery
datasets while effectively generalizing to unseen generative architectures. Our code is available at https://github.com/phoenixnir/FLAME.
Read the full abstract ▾
All authors Yiming Wang, Baiqi Wu, Qingming Li, Jiahao Chen, Tong Zhang, Shouling Ji
Computer Vision · Cross-listed from cs.LG
Closing the Alignment-Maturity Gap in Federated Prototype Learning
Mario Casado-Diez, Alejandro Dopico-Castro, Verónica Bolón-Canedo et al. (+1 more)
Learning discriminative visual representations from distributed, heterogeneous data is a fundamental challenge in
Federated Learning (FL). Prototype-based methods address statistical heterogeneity by sharing class-level representations across clients but create a distance-dependent gradient pressure that is particularly severe during early training rounds:
alignment pressure applied to immature global prototypes, aggregated from noisy local representations, generates large gradients that suppress the emergence of local discriminative structure. The result is a poorly organized
embedding space and degraded recognition performance, particularly under severe non-IID conditions. We propose FedSAP, a framework that stabilises
federated representation learning through two complementary mechanisms: a deterministic
alignment curriculum that delays global
alignment until local representations become stable and a geometry-driven proxy separation loss that enforces inter-class structure on the unit hypersphere using the existing prototype bank without introducing additional parameters or communication overhead. Together, these mechanisms produce compact, well-separated class clusters without altering the underlying communication protocol between federation's participants. Experiments across three
benchmarks and varying degrees of heterogeneity show gains of up to 4 percentage points over the prototype-based baselines evaluated, with improvements most pronounced under high heterogeneity. The representational nature of our framework further enables a straightforward extension to semi-supervised settings, where unlabelled data is incorporated with minimal modification, underscoring the generality of scheduled
alignment as a design principle.
Read the full abstract ▾
All authors Mario Casado-Diez, Alejandro Dopico-Castro, Verónica Bolón-Canedo, Bertha Guijarro-Berdiñas
Computer Vision
InsightVQA: High-Dimensional Emotion-Cognitive Visual Question Answering Benchmark
Shiyu Wang, Ziyu Liu, Chaoyi Yu et al. (+6 more)
Visual
emotion understanding requires models not only to recognize emotional states, but also to why they arise and perform higher-level cognitive
reasoning. However, existing
benchmarks mainly focus on
emotion recognition, offering limited support for grounded understanding and response-oriented analysis. To address this gap, we introduce InsightVQA, a large-scale
dataset for hierarchical visual question answering on
emotion understanding and cognitive
reasoning. Building from 351K images collected from six public sources, we apply a rigorous multi-stage filtering pipeline to curate 138K high-confidence images. Each image is annotated at three hierarchical levels: perception QA for
emotion and valence recognition, grounded understanding QA constructed from visual trigger extraction through constraint-guided generation, and cognition QA centered on response intent prediction and sequential insight
reasoning. In total, InsightVQA contains 725K QA pairs. We further present InsightVQA-Bench, a high-quality evaluation
benchmark comprising 30K samples for fine-grained evaluation. To support evaluation, we introduce InsightNet, an
emotion-tuned baseline for MLLMs. Results demonstrate that InsightVQA poses significant challenges for grounded
emotion understanding and
reasoning.
Read the full abstract ▾
All authors Shiyu Wang, Ziyu Liu, Chaoyi Yu, Yujie Yin, Zhongqian Mao, Jing Chen, Jiaqi Song, Yunshi Lan, Yan Wang
Computer Vision
Disentanglement-Based Equivariant Learning for Compositional VQA
Zhou Du, Zhaoquan Yuan, Xiao Wu et al. (+1 more)
Compositional visual question answering (VQA) represents a challenging yet fundamental task that requires models to comprehend novel combinations of previously learned concepts. The current methods often overlook the disentanglement of underlying concepts and are restricted in terms of their ability to effectively capture the compositional variation mechanism. Moreover, the state-of-the-art techniques depend on additional clues for training, which is not feasible in real-world VQA scenarios. To address these issues, in this paper, we introduce a novel Disentanglement-based EquivAriant Learning (DEAL) framework for compositional VQA, which is guided exclusively by ground-truth answers. In DEAL, we employ causality-inspired interventions to disentangle concepts derived from visual and textual inputs within a re-encoding framework. Based on the principle of equivariance, we subsequently perform a compositional transformation on the inference input and impose the equivariant constraint on the output to augment the compositional
reasoning capacity of the model. Comprehensive experiments conducted on the
benchmark CLEVR-CoGenT and GQA-SGL
datasets validate the superiority of our proposed DEAL approach over the existing state-of-the-art methods for compositional VQA tasks in both visual and linguistic generalization settings.
Read the full abstract ▾
All authors Zhou Du, Zhaoquan Yuan, Xiao Wu, Changsheng Xu
Computer Vision
Multimodal Approaches for Visually-Rich Document Type Classification: A Comparative Analysis
Catyana Heyne, Jürgen Frikel and Filippo Riccio
Document type classification in visually rich documents remains challenging, as relevant information is distributed across textual, visual, and layout modalities. To capture this complexity, current approaches rely on diverse
multimodal modeling strategies, resulting in heterogeneous architectures that complicate systematic comparison. This variability is also reflected in existing comparative studies, which often rely on heterogeneous evaluation setups, further complicating systematic comparison and making it difficult to assess progress. To address these limitations, this work provides a structured analysis of
multimodal design strategies across
transformer- and LLM-based architectures, combined with a controlled empirical comparison within a unified experimental framework. Specifically, four representative models (LayoutLMv3, Donut, Qwen3-VL-32B-Instruct, and Qwen3-32B) are evaluated on the RVL-CDIP
benchmark to systematically analyze the contributions of text, image, and layout information for document type classification, with a particular focus on contrasting OCR-dependent and OCR-free approaches. The results show that specialized
multimodal Transformers outperform LLM-based approaches on visually rich and layout-intensive documents. Image information contributes most strongly to reliable classification, while OCR-derived text provides useful but secondary support. These findings highlight that
multimodal processing remains essential for documents with pronounced layout structure. Overall, the study provides a systematic basis for comparing
multimodal architectures and offers practical guidance for selecting effective feature combinations and model designs for document type classification.
Read the full abstract ▾
Computer Vision
InfoMerge: Information-aware Token Compression for Efficient Video Large Language Models
Xinxin Liu, Shiwei Gan, Xiao Liu et al. (+3 more)
Video Large
Language Models (
Video-LLMs) achieve strong performance in
video understanding, but their excessive visual
tokens bring substantial computational overhead. Existing training-free compression methods improve inference efficiency by reducing visual
tokens, yet they often rely on local adjacent-frame similarity for temporal redundancy estimation or allocate
token budgets mainly according to segment length. Such designs are sensitive to frame-level noise and fail to capture the non-uniform information distribution of real-world
videos. To address these challenges, we propose InfoMerge, a training-free visual
token compression method that improves
token utilization through robust redundancy estimation and content-aware budget allocation. Specifically, we propose the Temporal Fingerprint Difference: a segment-level second-order temporal redundancy estimation strategy, which models the temporal similarity structure of
tokens at the same spatial positions within each segment. We further introduce Content-Aware Budget Allocation (CABA), which dynamically allocates segment-level
token budgets based on segment uniqueness and spectral-
entropy-based representational richness. By reducing repeated preservation of redundant static regions and allocating more
tokens to informative segments, InfoMerge makes better use of the limited
token budget while maintaining strong performance. Extensive experiments show that InfoMerge achieves strong efficiency–accuracy trade-offs across multiple
benchmarks and backbones, with more pronounced advantages under aggressive compression. On LLaVA-OneVision-7B, InfoMerge retains 98.8% of the original average performance while reducing 85% of visual
tokens and achieving a 4.24-fold speedup in the prefill stage.
Read the full abstract ▾
All authors Xinxin Liu, Shiwei Gan, Xiao Liu, Yafeng Yin, Lei Xie, Sanglu Lu
Computer Vision · Cross-listed from eess.IV
Predicting the risk of colorectal anastomotic leak based on preoperative mapping of the blood supply of the bowel
Zahra Tabatabaei, Jon Sporring, Mark Bremholm Ellebæk et al. (+1 more)
Anastomotic leak remains one of the most serious complications following colorectal cancer surgery, substantially affecting patient outcomes, recovery trajectories, and healthcare costs. Despite advances in imaging technology, current preoperative assessment relies only on clinical assessment, a process that is subjective, error-prone, and highly dependent on individual expertise. To date, no validated CT-based method exists to predict anastomotic leak risk prior to surgery. This protocol paper outlines a comprehensive framework for developing and validating an AI-driven system for preoperative risk assessment using pre- and post-contrast CT imaging. The study describes the stages of data collection, ethical handling, and preprocessing of patient data in accordance with GDPR, image preprocessing, and the exploration of deep learning architectures designed to generate clinically interpretable outputs. Two integrated tools constitute the main deliverables of this workflow: 1) a risk assessment module, which quantifies the likelihood of leakage by analyzing vascular and tissue features in CT scans, and 2) a Content-Based Medical Image
Retrieval (CBMIR) module, which identifies and displays similar historical cases to support evidence-based
surgical decision making. The protocol paper requires close collaboration between hospitals and universities; this protocol demonstrates that such a system is technically feasible and clinically implementable within existing healthcare infrastructures. By following the proposed methodological stages and regulatory principles, other institutions can reproduce this workflow to develop analogous decision-support tools. Ultimately, this interdisciplinary framework aims to enhance
surgical planning, reduce leak incidence, and contribute to a broader paradigm shift toward explainable, data-driven precision surgery.
Read the full abstract ▾
All authors Zahra Tabatabaei, Jon Sporring, Mark Bremholm Ellebæk, Alaa El-Hussuna