Computer Vision
Safety-Guided Flow (SGF): A Unified Framework for Negative Guidance in Safe Generation
Mingyu Kim, Young-Heon Kim and Mijung Park
Safety mechanisms for
diffusion and flow models have recently been developed along two distinct paths. In
robot planning, control barrier functions are employed to guide generative trajectories away from obstacles at every denoising step by explicitly imposing
geometric constraints. In parallel, recent data-driven, negative guidance approaches have been shown to suppress harmful content and promote diversity in generated samples. However, they rely on heuristics without clearly stating when safety guidance is actually necessary. In this paper, we first introduce a unified probabilistic framework using a Maximum Mean Discrepancy (MMD) potential for image generation tasks that recasts both Shielded
Diffusion and Safe Denoiser as instances of our energy-based negative guidance against unsafe data samples. Furthermore, we leverage control-barrier functions analysis to justify the existence of a critical time window in which negative guidance must be strong; outside of this window, the guidance should decay to zero to ensure safe and high-quality generation. We evaluate our unified framework on several realistic safe generation scenarios, confirming that negative guidance should be applied in the early stages of the denoising process for successful safe generation.
Read the full abstract ▾
Computer Vision
SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation
Zhehao Yu, Baoquan Zhang, Bingqi Shan et al. (+5 more)
Autoregressive (AR) image models have recently demonstrated remarkable generative capability, but their sequential nature results in significant inference latency. Existing training-free acceleration methods typically verify
tokens independently, overlooking the strong co-occurrence patterns between adjacent visual
tokens. This independence assumption often leads to contextual inconsistency and limits decoding efficiency. In this work, we introduce a novel training-free acceleration framework that performs phrase-level speculative verification, enabling the model to jointly validate multiple correlated
tokens within each decoding window. To construct such phrase units, we analyze
token co-occurrence statistics from the training corpus and group frequently co-occurring
tokens into semantically coherent visual phrases. During inference, the proposed phrase-level verification evaluates aggregated likelihood ratios over each phrase, allowing simultaneous acceptance of multiple
tokens while preserving generation quality. Extensive experiments on autoregressive text-to-image generation show that our method significantly reduces the number of function evaluations (NFE) and achieves up to 30% faster decoding without compromising visual fidelity. Our findings reveal that modeling short-range
token co-occurrence provides an effective and general principle for accelerating autoregressive inference.
Read the full abstract ▾
All authors Zhehao Yu, Baoquan Zhang, Bingqi Shan, Xinhao Liu, Dongliang Zhou, Guotao Liang, Guangming Ye, Yunming Ye
Computer Vision
Better Eyes, Better Thoughts: Why Vision Chain-of-Thought Fails in Medicine
Yuan Wu, Zongxian Yang, Jiayu Qian et al. (+5 more)
Large
vision-
language models (VLMs) often benefit from chain-of-thought (CoT) prompting in general domains, yet its efficacy in medical
vision-
language tasks remains underexplored. We report a counter-intuitive trend: on medical visual question answering, CoT frequently underperforms direct answering (DirA) across general-purpose and medical-specific models. We attribute this to a medical perception bottleneck: subtle, domain-specific cues can weaken visual grounding, and CoT may compound early perceptual uncertainty rather than correct it. To probe this hypothesis, we introduce two training-free, inference-time grounding interventions: (i) perception anchoring via region-of-interest cues and (ii) description grounding via high-quality textual guidance. Across multiple
benchmarks and model families, these interventions improve accuracy, mitigate CoT degradation, and in several settings reverse the CoT–DirA inversion. Our findings suggest that reliable clinical VLMs require robust visual grounding and cross-modal
alignment, beyond extending text-driven
reasoning chains. Code is available \href{https://github.com/TianYin123/Better_Eyes_Better_Thoughts}{here}.
Read the full abstract ▾
All authors Yuan Wu, Zongxian Yang, Jiayu Qian, Songpan Gao, Guanxing Chen, Qiankun Li, Yu-An Huang, Zhi-An Huang
Computer Vision
Accelerating Video Generation Inference with Sequential-Parallel 3D Positional Encoding Using a Global Time Index
Chao Yuan and Pan Li
Diffusion Transformer (DiT)-based
video generation models inherently suffer from bottlenecks in long
video synthesis and real-time inference, which can be attributed to the use of full spatiotemporal
attention. Specifically, this mechanism leads to explosive O(N^2)
memory consumption and high first-frame latency. To address these issues, we implement system-level inference optimizations for a causal autoregressive
video generation pipeline. We adapt the Self-Forcing causal autoregressive framework to sequence parallel inference and implement a sequence-parallel variant of the causal rotary position
embedding which we refer to as Causal-RoPE SP. This adaptation enables localized computation and reduces cross-rank communication in sequence parallel execution. In addition, computation and communication pipelines are optimized through operator fusion and RoPE precomputation. Experiments conducted on an eight GPU A800 cluster show that the optimized system achieves comparable generation quality, sub-second first-frame latency, and near real-time inference speed. For generating five second 480P
videos, a 1.58x speedup is achieved, thereby providing effective support for real-time interactive applications.
Read the full abstract ▾
Computer Vision
Graph-of-Mark: Promote Spatial Reasoning in Multimodal Language Models with Graph-Based Visual Prompting
Giacomo Frisoni, Lorenzo Molfetta, Mattia Buzzoni et al. (+1 more)
Recent advances in training-free visual prompting, such as Set-of-Mark, have emerged as a promising direction for enhancing the grounding capabilities of
multimodal language models (MLMs). These techniques operate by partitioning the input image into object regions and annotating them with marks, predominantly boxes with numeric identifiers, before feeding the augmented image to the MLM. However, these approaches treat marked objects as isolated entities, failing to capture the relationships between them. On these premises, we propose Graph-of-Mark (GoM), the first pixel-level visual prompting technique that overlays scene graphs onto the input image for spatial
reasoning tasks. We evaluate GoM across 3 open-source MLMs and 4 different
datasets, conducting extensive ablations on drawn components and investigating the impact of auxiliary graph descriptions in the text
prompt. Our results demonstrate that GoM consistently improves the zero-shot capability of MLMs in interpreting object positions and relative directions, improving base accuracy in visual question answering and localization up to 11 percentage points.
Read the full abstract ▾
All authors Giacomo Frisoni, Lorenzo Molfetta, Mattia Buzzoni, Gianluca Moro
Computer Vision
DynaTokens: Controlling Token Dynamics for Continual Video-Language Understanding
Toan Nguyen, Yang Liu, Celso De Melo et al. (+1 more)
Continual VideoQA with
multimodal LLMs remains challenging because sequential adaptation induces task interference, while storing task-specific
prompts becomes impractical as task sequences grow. We introduce DynaTokens, a
transformer-based
token generator that dynamically produces fine-tuning
tokens on demand, enabling task-adaptive
prompt updates through shared generation weights. To mitigate forgetting, we introduce meta-learning-inspired regularisers that look ahead to avoid task-specific sharp update directions while anchoring the evolving generator to prior-task behaviours. We theoretically connect this objective to sharpness-aware optimisation, showing how it favours flatter cross-task minima and improves retention. DynaTokens combines gradient-free routing based on robust pretrained
token and visual
embeddings with lightweight auxiliary
multimodal supervision, reducing router drift during continual adaptation. Across standard continual VideoQA
benchmarks, DynaTokens achieves higher average accuracy and substantially lower forgetting than strong baselines. It also improves zero-shot generalisation and remains effective in longer domain-incremental sequences with extended task shifts. Finally, we introduce a challenging ImageQA->VideoQA protocol and show that DynaTokens enables robust cross-modal continual transfer.
Read the full abstract ▾
All authors Toan Nguyen, Yang Liu, Celso De Melo, Flora D. Salim
Computer Vision
EnsAug: Augmentation-Driven Ensembles for Human Motion Sequence Analysis
Bikram De, Habib Irani and Vangelis Metsis
Data augmentation is a crucial technique for training robust deep learning models for human motion, where annotated
datasets are often scarce. However, generic augmentation methods often ignore the underlying
geometric and kinematic constraints of the human body, risking the generation of unrealistic motion patterns that can degrade model performance. Furthermore, the conventional approach of training a single generalist model on a
dataset expanded with a mixture of all available transformations does not fully exploit the unique learning signals provided by each distinct augmentation type. We challenge this convention by introducing a novel training paradigm, EnsAug, that strategically uses augmentation to foster model diversity within an ensemble. Our method involves training an ensemble of specialists, where each model learns from the original
dataset augmented by only a single, distinct
geometric transformation. Experiments on sign
language and human activity recognition
benchmarks demonstrate that our diversified ensemble methodology significantly outperforms the standard practice of training one model on a combined augmented
dataset and achieves state-of-the-art accuracy on two sign
language and one human activity recognition
dataset while offering greater modularity and efficiency. Our primary contribution is the empirical validation of this training strategy, establishing an effective baseline for leveraging data augmentation in skeletal motion analysis.
Read the full abstract ▾
Computer Vision
Skin Tone in the Wild: A Dataset and Benchmark for 10-tone MST Classification
Vitor Matias, Márcus Vinícius Lobo Costa, Tiago Novello et al. (+1 more)
Skin tone classification is a fundamental task in computer
vision, with applications in quantifying medical features, assessing demographic representativeness, and mitigating algorithmic bias. However, existing
datasets either rely on a restrictive 6-tone Fitzpatrick scale, prohibit training through restrictive licenses, or lack representation of extreme skin tones, while current methods often fail to disentangle true skin tone from in-the-wild imaging conditions, degrading to random-classifier performance. We introduce skin tone in the wild (STW), the first open-access
dataset annotated with the 10-tone Monk Skin Tone (MST) scale, containing over 42K images and specifically curated to include underrepresented tones. Alongside a rigorous evaluation protocol, we propose SkinToneNet, a ViT-based model that leverages global image context for robust in-the-wild generalization. Experiments show that SkinToneNet achieves 90% weighted off-by-one accuracy, substantially outperforming prior baselines. We further use SkinToneNet to audit widely used face
datasets, including CelebA and VGGFace2, revealing substantial representation imbalances across MST tones.
Read the full abstract ▾
All authors Vitor Matias, Márcus Vinícius Lobo Costa, Tiago Novello, João Batista Neto
Computer Vision
Toward Generalizable Deep Learning Based Peatland Fire Detection via Walsh Hadamard Transform and Domain Adaptation
Emadeldeen Hamdan, Ahmad Faiz Tharima, Mohd Zahirasri Mohd Tohir et al. (+4 more)
Machine learning-based wildfire detection has advanced significantly using deep learning models trained on large wildfire image and
video datasets. However, peatland fires exhibit distinct characteristics, including smoldering combustion, low flame intensity, persistent smoke, and subsurface burning, limiting the effectiveness of conventional wildfire detectors. To address these challenges, we propose an efficient deep learning framework for peatland fire detection based on a Walsh–Hadamard Transform enhanced ResNet-50 (WHT-ResNet-50), which improves feature representation while reducing model complexity. To enable efficient deployment, the training-time architecture is structurally reparameterized into an equivalent inference model without sacrificing detection performance. Furthermore, the proposed framework leverages wildfire-to-peatland domain adaptation through transfer learning and introduces a mixed-domain training strategy that produces a unified detector capable of recognizing both wildfire and peatland fire events. Experimental results demonstrate that transfer learning substantially improves peatland fire detection under limited-data conditions, while the proposed WHT-ResNet-50 achieves higher accuracy and F1-score than conventional architectures with fewer parameters. The structurally reparameterized model further reduces inference cost while preserving detection accuracy.
Video-based evaluation demonstrates robust performance with low false alarm rates, achieving a 100% event detection rate across all positive test
videos. Overall, the proposed framework provides an accurate, efficient, and practical solution for early peatland fire detection.
Read the full abstract ▾
All authors Emadeldeen Hamdan, Ahmad Faiz Tharima, Mohd Zahirasri Mohd Tohir, Dayang Nur Sakinah Musa, Erdem Koyuncu, Adam J. Watts, Ahmet Enis Cetin
Computer Vision
ORCA: Orchestrated Reasoning with Collaborative Agents for Document Visual Question Answering
Aymen Lassoued, Mohamed Ali Souibgui and Yousri Kessentini
Document Visual Question Answering (DocVQA) remains challenging for existing
Vision-
Language Models (VLMs), especially under complex
reasoning and multi-step workflows. Current approaches struggle to decompose intricate questions into manageable sub-tasks and often fail to leverage specialized processing paths for different document elements. We present ORCA: Orchestrated
Reasoning with Collaborative
Agents for Document Visual Question Answering, a novel multi-
agent framework that addresses these limitations through strategic
agent coordination and iterative refinement. ORCA begins with a
reasoning agent that decomposes queries into logical steps, followed by a routing mechanism that activates task-specific
agents from a specialized
agent dock. Our framework leverages a set of specialized AI
agents, each dedicated to a distinct modality, enabling fine-grained understanding and collaborative
reasoning across diverse document components. To ensure answer reliability, ORCA employs a debate mechanism with stress-testing, and when necessary, a thesis-antithesis adjudication process. This is followed by a sanity checker to ensure format consistency. Extensive experiments on three
benchmarks demonstrate that our approach achieves significant improvements over state-of-the-art methods, establishing a new paradigm for collaborative
agent systems in
vision-
language reasoning.
Read the full abstract ▾
Computer Vision
MIRAGE: Knowledge Graph-Guided Cross-Cohort MRI Synthesis for Alzheimer's Disease Prediction
Guanchen Wu, Zhe Huang, Yuzhang Xie et al. (+6 more)
Reliable Alzheimer's disease (AD) diagnosis increasingly relies on
multimodal assessments combining structural
Magnetic Resonance Imaging (MRI) and Electronic Health Records (EHR). However, deploying these models is bottlenecked by modality missingness, as MRI scans are expensive and frequently unavailable in many patient cohorts. Furthermore, synthesizing de novo 3D anatomical scans from sparse, high-dimensional tabular records is technically challenging and poses severe clinical risks. To address this, we introduce MIRAGE, a novel framework that reframes the missing-MRI problem as an anatomy-guided cross-modal latent
distillation task. First, MIRAGE leverages a Biomedical Knowledge Graph (KG) and Graph
Attention Networks to map heterogeneous EHR variables into a unified
embedding space that can be propagated from cohorts with real MRIs to cohorts without them. To bridge the semantic gap and enforce physical spatial awareness, we employ a frozen pre-trained 3D U-Net decoder strictly as an auxiliary regularization engine. Supported by a novel cohort-aggregated skip feature compensation strategy, this decoder acts as a rigorous structural penalty, forcing 1D latent representations to encode biologically plausible, macro-level pathological semantics. By exclusively utilizing this distilled "diagnostic-surrogate" representation during inference, MIRAGE completely bypasses computationally expensive 3D voxel reconstruction. Experiments demonstrate that our framework successfully bridges the missing-modality gap, improving the AD classification rate by 13% compared to unimodal baselines in cohorts without real MRIs.
Read the full abstract ▾
All authors Guanchen Wu, Zhe Huang, Yuzhang Xie, Runze Yan, Akul Chopra, Deqiang Qiu, Xiao Hu, Fei Wang, Carl Yang
Computer Vision · Cross-listed from cs.LG
A Unified Revisit of Temperature in Classification-Based Knowledge Distillation
Logan Frank and Jim Davis
A central idea of knowledge
distillation is to expose relational structure embedded in the teacher's weights for the student to learn, which is often facilitated using a temperature parameter. Despite its widespread use, there remains limited understanding on how to select an appropriate temperature value, or how this value depends on other training elements such as optimizer, teacher pretraining/finetuning, etc. In practice, temperature is commonly chosen via grid search or by adopting values from prior work, which can be time-consuming or may lead to suboptimal student performance when training setups differ. In this work, we posit that temperature is closely linked to these training components and present a unified study that systematically examines such interactions. From analyzing these cross-connections, we identify and present common situations that have a pronounced impact on temperature selection, providing valuable guidance for practitioners employing knowledge
distillation in their work.
Read the full abstract ▾
Computer Vision
DINOv3 Visual Representations for Blueberry Perception Toward Robotic Harvesting
Rui-Feng Wang, Daniel Petti, Yue Chen et al. (+1 more)
Vision Foundation Models trained via large-scale self-supervised learning have demonstrated strong generalization in visual perception; however, their practical role and performance limits in agricultural settings remain insufficiently understood. This work evaluates DINOv3 as a frozen backbone for blueberry robotic harvesting-related visual tasks, including fruit and bruise
segmentation, as well as fruit and cluster detection. Under a unified protocol with lightweight decoders,
segmentation benefits consistently from stable patch-level representations and scales with backbone size. In contrast, detection is constrained by target scale variation, patch discretization, and localization compatibility. The failure of cluster detection highlights limitations in modeling relational targets defined by spatial aggregation. Overall, DINOv3 is best viewed not as an end-to-end task model, but as a semantic backbone whose effectiveness depends on downstream spatial modeling aligned with fruit-scale and aggregation structures, providing guidance for blueberry robotic harvesting. Code and
dataset will be available upon acceptance.
Read the full abstract ▾
All authors Rui-Feng Wang, Daniel Petti, Yue Chen, Changying Li
Computer Vision
TruckDrive: Long-Range Autonomous Highway Driving Dataset
Filippo Ghilotti, Edoardo Palladin, Samuel Brucker et al. (+3 more)
Safe highway autonomy for heavy trucks remains an open and unsolved challenge: due to long braking distances, scene understanding of hundreds of meters is required for anticipatory planning and to allow safe braking margins. However, existing driving
datasets primarily cover urban scenes, with perception effectively limited to short ranges of only up to 100 meters. To address this gap, we introduce TruckDrive, a highway-scale
multimodal driving
dataset, captured with a sensor suite purpose-built for long range sensing: seven long-range FMCW
LiDARs measuring range and radial velocity, three high-resolution short-range
LiDARs, eleven 8MP surround cameras with varying focal lengths and ten 4D FMCW radars. The
dataset offers 475 thousands samples with 165 thousands densely annotated frames for driving perception benchmarking up to 1,000 meters for 2D detection and 400 meters for 3D detection, depth estimation, tracking, planning and end to end driving over 20 seconds sequences at highway speeds. We find that state-of-the-art autonomous driving models do not generalize to ranges beyond 150 meters, with drops between 31% and 99% in 3D perception tasks, exposing a systematic long-range gap that current architectures and training signals cannot close.
Read the full abstract ▾
All authors Filippo Ghilotti, Edoardo Palladin, Samuel Brucker, Adam Sigal, Mario Bijelic, Felix Heide
Computer Vision
From Fewer Samples to Fewer Bits: Reframing Dataset Distillation as Joint Optimization of Precision and Compactness
My H. Dinh, Aditya Sant, Akshay Malhotra et al. (+2 more)
Dataset Distillation (DD) compresses large
datasets into compact synthetic ones that maintain training performance. However, current methods mainly target sample reduction, with limited consideration of data precision and its impact on efficiency. We propose Quantization-aware
Dataset Distillation (QuADD), a unified framework that jointly optimizes
dataset compactness and precision under fixed bit budgets. QuADD integrates a differentiable quantization module within the
distillation loop, enabling end-to-end co-optimization of synthetic samples and quantization parameters. Guided by the rate-distortion perspective, we empirically analyze how bit allocation between sample count and precision influences learning performance. Our framework supports both uniform and adaptive non-uniform quantization, where the latter learns quantization levels from data to represent information-dense regions better. Experiments on image classification and 3GPP beam management tasks show that QuADD surpasses existing DD and post-quantized baselines in accuracy per bit, establishing a new standard for information-efficient
dataset distillation.
Read the full abstract ▾
All authors My H. Dinh, Aditya Sant, Akshay Malhotra, Keya Patani, Shahab Hamidi-Rad
Computer Vision
OpenMarcie: Dataset for Multimodal Action Recognition in Industrial Environments
Hymalai Bello, Lala Ray, Joanna Sorysz et al. (+2 more)
Smart factories use advanced technologies to optimize production and increase efficiency. To this end, the recognition of worker activity allows for accurate quantification of performance metrics, improving efficiency holistically while contributing to worker safety. OpenMarcie is, to the best of our knowledge, the biggest
multimodal dataset designed for human action monitoring in manufacturing environments. It includes data from wearables sensing modalities and cameras distributed in the surroundings. The
dataset is structured around two experimental settings, involving a total of 36 participants. In the first setting, twelve participants perform a bicycle assembly and disassembly task under semi-realistic conditions without a fixed protocol, promoting divergent and goal-oriented problem-solving. The second experiment involves twenty-five volunteers (24 valid data) engaged in a 3D printer assembly task, with the 3D printer manufacturer's instructions provided to guide the volunteers in acquiring procedural knowledge. This setting also includes sequential collaborative assembly, where participants assess and correct each other's progress, reflecting real-world manufacturing dynamics. OpenMarcie includes over 37 hours of
egocentric and exocentric,
multimodal, and multipositional data, featuring eight distinct data types and more than 200 independent information channels. The
dataset is benchmarked across three human activity recognition tasks: activity classification, open vocabulary captioning, and cross-modal
alignment.
Read the full abstract ▾
All authors Hymalai Bello, Lala Ray, Joanna Sorysz, Sungho Suh, Paul Lukowicz
Computer Vision
Advancing Earth Observation Through Machine Learning: A TorchGeo Tutorial
Caleb Robinson, Nils Lehmann, Adam J. Stewart et al. (+4 more)
Earth observation machine learning pipelines differ fundamentally from standard computer
vision workflows. Imagery is typically delivered as large, georeferenced scenes, labels may be raster masks or vector geometries in distinct coordinate reference systems, and both training and evaluation often require spatially aware sampling and splitting strategies. TorchGeo is a PyTorch-based domain library that provides
datasets, samplers, transforms and pre-trained models with the goal of making it easy to use geospatial data in machine learning pipelines. In this paper, we introduce a tutorial that demonstrates 1.) the core TorchGeo abstractions through code examples, and 2.) an end-to-end case study on multispectral water
segmentation from Sentinel-2 imagery using the Earth Surface Water
dataset. This demonstrates how to train a semantic
segmentation model using TorchGeo
datasets, apply the model to a Sentinel-2 scene over Rio de Janeiro, Brazil, and save the resulting predictions as a GeoTIFF for further geospatial analysis. The tutorial code itself is distributed as two Python notebooks: https://torchgeo.readthedocs.io/en/stable/tutorials/torchgeo.html and https://torchgeo.readthedocs.io/en/stable/tutorials/earth_surface_water.html.
Read the full abstract ▾
All authors Caleb Robinson, Nils Lehmann, Adam J. Stewart, Burak Ekim, Heng Fang, Isaac A. Corley, Mauricio Cordeiro
Computer Vision · Cross-listed from cs.CR
Authenticated Contradictions from Desynchronized Provenance and Watermarking
Alexander Nemecek, Hengzhi He, Guang Cheng et al. (+1 more)
Cryptographic provenance standards such as C2PA and invisible watermarking are positioned as complementary defenses for content authentication, yet the two verification layers are technically independent: neither conditions on the output of the other. This work formalizes and empirically demonstrates the $\textit{Integrity Clash}$, a condition in which a digital asset carries a cryptographically valid C2PA manifest asserting human authorship while its pixels simultaneously carry a watermark identifying it as AI-generated, with both signals passing their respective verification checks in isolation. We construct metadata washing workflows that produce these authenticated fakes through standard editing pipelines, requiring no cryptographic compromise, only the semantic omission of a single assertion field permitted by the current C2PA specification. To close this gap, we propose a cross-layer audit protocol that jointly evaluates provenance metadata and watermark detection status, achieving 100% classification accuracy across 3,500 test images spanning four conflict-
matrix states and three realistic perturbation conditions. Our results demonstrate that the gap between these verification layers is unnecessary and technically straightforward to close.
Read the full abstract ▾
All authors Alexander Nemecek, Hengzhi He, Guang Cheng, Erman Ayday
Computer Vision
Aligning Fetal Anatomy with Kinematic Tree Log-Euclidean PolyRigid Transforms
Yingcheng Liu, Athena Taymourtash, Yang Liu et al. (+5 more)
Automated analysis of articulated bodies is crucial in medical imaging. Existing surface-based models often ignore internal volumetric structures and rely on deformation methods that lack anatomical consistency guarantees. To address this problem, we introduce a differentiable volumetric body model based on the Skinned Multi-Person Linear (SMPL) formulation, driven by a new Kinematic Tree-based Log-Euclidean PolyRigid (KTPolyRigid) transform. KTPolyRigid resolves Lie
algebra ambiguities associated with large, non-local articulated motions, and encourages smooth, bijective volumetric mappings. Evaluated on 53 fetal MRI volumes, KTPolyRigid yields deformation fields with significantly fewer folding artifacts. Furthermore, our framework enables robust groupwise image registration and a label-efficient, template-based
segmentation of fetal organs. It provides a robust foundation for standardized volumetric analysis of articulated bodies in medical imaging.
Read the full abstract ▾
All authors Yingcheng Liu, Athena Taymourtash, Yang Liu, Esra Abaci Turk, William M. Wells, Leo Joskowicz, P. Ellen Grant, Polina Golland
Computer Vision
Cultural Counterfactuals: Evaluating Cultural Biases in Large Vision-Language Models with Counterfactual Examples
Phillip Howard, Xin Su and Kathleen C. Fraser
Large
Vision-
Language Models (LVLMs) have grown increasingly powerful in recent years, but can also exhibit harmful biases. Prior studies investigating such biases have primarily focused on demographic traits related to the visual characteristics of a person depicted in an image, such as their race or gender. This has left biases related to cultural differences (e.g., religion, socioeconomic status), which cannot be readily discerned from an individual's appearance alone, relatively understudied. A key challenge in measuring cultural biases is that determining which group an individual belongs to often depends upon cultural context cues in images, and
datasets annotated with cultural context cues are lacking. To address this gap, we introduce Cultural Counterfactuals: a high-quality synthetic
dataset containing nearly 60k counterfactual images for measuring cultural biases related to religion, nationality, and socioeconomic status. To ensure that cultural contexts are accurately depicted, we generate our
dataset using an image-editing model to place people of different demographics into real cultural context images. This enables the construction of counterfactual image sets which depict the same person in multiple different contexts, allowing for precise measurement of the impact that cultural context differences have on LVLM outputs. We demonstrate the utility of Cultural Counterfactuals for quantifying cultural biases in popular LVLMs.
Read the full abstract ▾
Computer Vision
Retrieving Patient-Specific Radiomic Feature Sets for Transparent Knee MRI Assessment
Yaxi Chen, Simin Ni, Jingjing Zhang et al. (+7 more)
Classical radiomic features are designed to quantify image appearance and intensity patterns. Compared with end-to-end deep learning (DL) models trained for disease classification, radiomics pipelines with low-dimensional parametric classifiers offer enhanced transparency and interpretability, yet often underperform because of the reliance on population-level predefined feature sets. Recent work on adaptive radiomics uses DL to predict feature weights over a radiomic pool, then thresholds these weights to retain the top-k features from large radiomic pool F (often ~10^3). However, such marginal ranking can over-admit redundant descriptors and overlook complementary feature interactions. We propose a patient-specific feature-set selection framework that predicts a single compact feature set per subject, targeting complementary and diverse evidence rather than marginal top-k features. To overcome the intractable combinatorial search space of F choose k features, our method utilizes a 2-stage
retrieval strategy: randomly sample diverse candidate feature sets, then rank these sets with a learned scoring function to select a high-performing feature set for the specific patient. The system consists of a feature-set scorer, and a classifier that performs the final diagnosis. We empirically show that the proposed two-stage
retrieval approximates the original exhaustive all k-feature selection. Validating on tasks including ACL tear detection and KL grading for osteoarthritis, the experimental results achieve diagnostic performance, outperforming the top-k approach with the same k values, and competitive with end-to-end DL models while maintaining high transparency. The model generates auditable feature sets that link clinical outcomes to specific anatomical regions and radiomic families, allowing clinicians to inspect which anatomical structures and quantitative descriptors drive the prediction.
Read the full abstract ▾
All authors Yaxi Chen, Simin Ni, Jingjing Zhang, Shaheer U. Saeed, Yipei Wang, Aleksandra Ivanova, Rikin Hargunani, Chaozong Liu, Jie Huang, Yipeng Hu
Computer Vision
Beyond Caption-Based Queries for Video Moment Retrieval
David Pujol-Perich, Albert Clapés, Dima Damen et al. (+2 more)
In this work, we investigate the degradation of existing VMR methods, particularly of DETR architectures, when trained on caption-based queries but evaluated on search queries. For this, we introduce three
benchmarks by modifying the textual queries in three public VMR
datasets – i.e., HD-EPIC, YouCook2 and ActivityNet-Captions. Our analysis reveals two key generalization challenges: (i) A
language gap, arising from the linguistic under-specification of search queries, and (ii) a multi-moment gap, caused by the shift from single-moment to multi-moment queries. We also identify a critical issue in these architectures – an active decoder-query collapse – as a primary cause of the poor generalization to multi-moment instances. We mitigate this issue with architectural modifications that effectively increase the number of active decoder queries. Extensive experiments demonstrate that our approach improves performance on search queries by up to 14.82% mAP_m, and up to 21.83% mAP_m on multi-moment search queries. The code, models and data are available in the project webpage: https://davidpujol.github.io/beyond-vmr/
Read the full abstract ▾
All authors David Pujol-Perich, Albert Clapés, Dima Damen, Sergio Escalera, Michael Wray
Computer Vision
MERG3R: A Divide-and-Conquer Approach to Large-Scale Neural Visual Geometry
Leo Kaixuan Cheng, Abdus Shaikh, Ruofan Liang et al. (+3 more)
Recent advancements in neural visual geometry, including
transformer-based models such as VGGT and Pi3, have achieved impressive accuracy on 3D reconstruction tasks. However, their reliance on full
attention makes them fundamentally limited by GPU
memory capacity, preventing them from scaling to large, unordered image collections. We introduce MERG3R, a training-free divide-and-conquer framework that enables
geometric foundation models to operate far beyond their native
memory limits. MERG3R first reorders and partitions unordered images into overlapping, geometrically diverse subsets that can be reconstructed independently. It then merges the resulting local reconstructions through an efficient global
alignment and confidence-weighted bundle adjustment procedure, producing a globally consistent 3D model. Our framework is model-agnostic and can be paired with existing neural geometry models. Across large-scale
datasets, including 7-Scenes, NRGBD, Tanks & Temples, and Cambridge Landmarks, MERG3R consistently improves reconstruction accuracy,
memory efficiency, and scalability, enabling high-quality reconstruction when the
dataset exceeds
memory capacity limits.
Read the full abstract ▾
All authors Leo Kaixuan Cheng, Abdus Shaikh, Ruofan Liang, Zhijie Wu, Yushi Guan, Nandita Vijaykumar
Computer Vision · Cross-listed from cs.LG
Preconditioned Flow Matching
Shadab Ahamed, Eshed Gal, Md Shahriar Rahim Siddiqui et al. (+3 more)
Flow matching (FM) learns vector fields by regressing
stochastic velocity targets along intermediate distributions $p_t$. We identify a
geometric optimization bottleneck in this regression problem: when the covariance $\Sigma_t$ of $p_t$ is ill-conditioned, gradient-based training rapidly fits high-variance directions while making slow progress along low-variance ones. In an exactly solvable Gaussian setting, we prove that the excess risk is weighted by $\Sigma_t$, and that both gradient descent and
stochastic gradient descent inherit condition-number-dependent convergence. We then extend the analysis to Gaussian mixtures, showing that multimodality does not average away this effect; instead, the slowest and worst-conditioned component can control optimization. Motivated by this analysis, we propose $\textit{preconditioned flow matching}$, a precondition-then-match framework that transforms the target distribution into a more isotropic representation, trains the main flow in the transformed space, and maps generated samples back through the inverse transformation. We show theoretically that preconditioning reshapes the intermediate FM path and improves its conditioning. Across controlled Gaussian and Gaussian-mixture experiments, latent MNIST and other high resolution image
datasets up to $512{\times}512$ resolution, preconditioning improves path-conditioning diagnostics, low-
eigenvalue recovery, FID, MMD, precision, and recall. Compute-matched baselines and preconditioner-quality ablations further show that the gains are not explained merely by additional preconditioner parameters, but by improved geometry of the downstream flow matching problem. GitHub: https://github.com/ahxmeds/preconditioned-flow-matching.git.
Read the full abstract ▾
All authors Shadab Ahamed, Eshed Gal, Md Shahriar Rahim Siddiqui, Simon Ghyselincks, Moshe Eliasof, Eldad Haber
Computer Vision
HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Grounding
Lei Yao, Yong Chen, Yuejiao Su et al. (+3 more)
Humans commonly identify 3D object affordance through observed interactions in images or
videos, and once formed, such knowledge can be generically generalized to novel objects. Inspired by this principle, we advocate for a novel framework that leverages emerging
multimodal large
language models (MLLMs) for interaction intention-driven 3D affordance grounding, namely HAMMER. Instead of generating explicit object attribute descriptions or relying on off-the-shelf 2D segmenters, we alternatively aggregate the interaction intention depicted in the image into a contact-aware
embedding and guide the model to infer textual affordance labels, ensuring it thoroughly excavates object semantics and contextual cues. We further devise a hierarchical cross-modal integration mechanism to fully exploit the complementary information from the MLLM for 3D representation refinement and introduce a multi-granular geometry lifting module that infuses spatial characteristics into the extracted intention
embedding, thus facilitating accurate 3D affordance localization. Extensive experiments on public
datasets and our newly constructed corrupted
benchmark demonstrate the superiority and robustness of HAMMER compared to existing approaches. All code and weights are publicly available.
Read the full abstract ▾
All authors Lei Yao, Yong Chen, Yuejiao Su, Yi Wang, Moyun Liu, Lap-Pui Chau
Computer Vision · Cross-listed from eess.IV
Loss Design and Architecture Selection for Long-Tailed Multi-Label Chest X-Ray Classification
Nikhileswara Rao Sulake
Long-tailed class distributions pose a significant challenge for multi-label chest X-ray (CXR) classification, where rare but clinically important findings are severely underrepresented. In this work, we present a systematic empirical evaluation of loss functions, CNN backbone architectures and post-training strategies on the CXR-LT 2026
benchmark, comprising approximately 143K images with 30 disease labels from PadChest. Our experiments demonstrate that LDAM with deferred re-weighting (LDAM-DRW) consistently outperforms standard BCE and asymmetric losses for rare class recognition. Amongst the architectures evaluated, ConvNeXt-Large achieves the best single-model performance with 0.5220 mAP and 0.3765 F1 on our development set, whilst classifier re-training and test-time augmentation further improve ranking metrics. On the official test leaderboard, our submission achieved 0.3950 mAP, ranking 5th amongst all 68 participating teams with total of 1528 submissions. We provide a candid analysis of the development-to-test performance gap and discuss practical insights for handling class imbalance in clinical imaging settings. Code is available at https://github.com/Nikhil-Rao20/Long_Tail.
Read the full abstract ▾
Computer Vision
AutoFFS: Adversarial Deformations for Facial Feminization Surgery Planning
Paul Friedrich, Florentin Bieder, Florian M. Thieringer et al. (+1 more)
Facial feminization surgery (FFS) is a key component of gender affirmation for transgender and gender diverse patients, aiming to reshape craniofacial structures toward a female morphology. Current
surgical planning procedures largely rely on subjective clinical assessment, lacking quantitative and reproducible anatomical guidance. We therefore propose AutoFFS, a novel data-driven framework that generates counterfactual skull morphologies through
adversarial free-form deformations. Our method performs a deformation-based targeted
adversarial attack on an ensemble of pre-trained binary sex classifiers that learned sexual dimorphism, effectively transforming individual skull shapes toward the target sex. The generated counterfactual skull morphologies provide a quantitative foundation for preoperative planning in FFS, driving advances in this largely overlooked patient group. We validate our approach through classifier-based evaluation, propose Morphological Fréchet Distance (MFD) and Morphological Kernel Distance (MKD) to evaluate distributional
alignment of generated and real populations, and perform a human perceptual study, confirming that the generated morphologies exhibit target sex characteristics.
Read the full abstract ▾
All authors Paul Friedrich, Florentin Bieder, Florian M. Thieringer, Philippe C. Cattin
Computer Vision
Beyond Prompt Degradation: Prototype-guided Dual-pool Prompting for Incremental Object Detection
Yaoteng Zhang, Zhou Qing, Junyu Gao et al. (+1 more)
Incremental Object Detection (IOD) aims to continuously learn new object categories without forgetting previously learned ones. Recently,
prompt-based methods have gained popularity for their replay-free design and parameter efficiency. However, due to
prompt coupling and
prompt drift, these methods often suffer from
prompt degradation during continual adaptation. To address these issues, we propose a novel
prompt-decoupled framework called PDP. PDP innovatively designs a dual-pool
prompt decoupling paradigm, which consists of a shared pool used to capture task-general knowledge for forward transfer, and a private pool used to learn task-specific discriminative features. This paradigm explicitly separates task-general and task-specific
prompts, preventing interference between
prompts and mitigating
prompt coupling. In addition, to counteract
prompt drift resulting from inconsistent supervision where old foreground objects are treated as background in subsequent tasks, PDP introduces a Prototypical Pseudo-Label Generation (PPG) module. PPG can dynamically update the class prototype space during training and use the class prototypes to further filter valuable pseudo-labels, maintaining supervisory signal consistency throughout the incremental process. PDP achieves state-of-the-art performance on MS-COCO (with a 9.2% AP improvement) and PASCAL VOC (with a 3.3% AP improvement)
benchmarks, highlighting its potential in balancing stability and plasticity. The code and
dataset are released at: https://github.com/zyt95579/PDP\_IOD/tree/main
Read the full abstract ▾
All authors Yaoteng Zhang, Zhou Qing, Junyu Gao, Qi Wang
Computer Vision
HiFi-Inpaint: Towards High-Fidelity Reference-Based Inpainting for Generating Detail-Preserving Human-Product Images
Yichen Liu, Donghao Zhou, Jie Wang et al. (+9 more)
Human-product images, which showcase the integration of humans and products, play a vital role in advertising, e-commerce, and digital marketing. The essential challenge of generating such images lies in ensuring the high-fidelity preservation of product details. Among existing paradigms, reference-based inpainting offers a targeted solution by leveraging product reference images to guide the inpainting process. However, limitations remain in three key aspects: the lack of diverse large-scale training data, the struggle of current models to focus on product detail preservation, and the inability of coarse supervision for achieving precise guidance. To address these issues, we propose HiFi-Inpaint, a novel high-fidelity reference-based inpainting framework tailored for generating human-product images. HiFi-Inpaint introduces Shared Enhancement
Attention (SEA) to refine fine-grained product features and Detail-Aware Loss (DAL) to enforce precise pixel-level supervision using high-frequency maps. Additionally, we construct a new
dataset, HP-Image-40K, with samples curated from self-synthesis data and processed with automatic filtering. Experimental results show that HiFi-Inpaint achieves state-of-the-art performance, delivering detail-preserving human-product images.
Read the full abstract ▾
All authors Yichen Liu, Donghao Zhou, Jie Wang, Xin Gao, Guisheng Liu, Jiatong Li, Quanwei Zhang, Qiang Lyu, Lanqing Guo, Shilei Wen, Weiqiang Wang, Pheng-Ann Heng
Computer Vision
Adaptive Confidence Regularization for Multimodal Failure Detection
Moru Liu, Hao Dong, Olga Fink et al. (+1 more)
The deployment of
multimodal models in high-stakes domains, such as self-driving vehicles and medical diagnostics, demands not only strong predictive performance but also reliable mechanisms for detecting failures. In this work, we address the largely unexplored problem of failure detection in
multimodal contexts. We propose Adaptive Confidence Regularization (ACR), a novel framework specifically designed to detect
multimodal failures. Our approach is driven by a key observation: in most failure cases, the confidence of the
multimodal prediction is significantly lower than that of at least one unimodal branch, a phenomenon we term confidence degradation. To mitigate this, we introduce an Adaptive Confidence Loss that penalizes such degradations during training. In addition, we propose
Multimodal Feature Swapping, a novel outlier synthesis technique that generates challenging, failure-aware training examples. By training with these synthetic failures, ACR learns to more effectively recognize and reject uncertain predictions, thereby improving overall reliability. Extensive experiments across four
datasets, three modalities, and multiple evaluation settings demonstrate that ACR achieves consistent and robust gains. The source code will be available at https://github.com/mona4399/ACR.
Read the full abstract ▾
All authors Moru Liu, Hao Dong, Olga Fink, Mario Trapp