Computer Vision
Arc2Morph: Identity-Preserving Facial Morphing with Arc2Face
Nicolò Di Domenico, Annalisa Franco, Matteo Ferrara et al. (+1 more)
Face morphing attacks are widely recognized as one of the most challenging threats to face recognition systems used in electronic identity documents. These attacks exploit a critical vulnerability in passport enrollment procedures adopted by many countries, where the facial image is often acquired without a supervised live capture process. In this paper, we propose a novel face morphing technique based on Arc2Face, an identity-conditioned face foundation model capable of synthesizing photorealistic facial images from compact identity representations. We demonstrate the effectiveness of the proposed approach by comparing the morphing attack potential metric on two large-scale sequestered face morphing attack detection
datasets against several state-of-the-art morphing methods, as well as on two novel morphed face
datasets derived from FEI and ONOT. Experimental results show that the proposed deep learning-based approach achieves a morphing attack potential comparable to that of landmark-based techniques, which have traditionally been regarded as the most challenging. These findings confirm the ability of the proposed method to effectively preserve and manage identity information during the morph generation process.
Read the full abstract ▾
All authors Nicolò Di Domenico, Annalisa Franco, Matteo Ferrara, Davide Maltoni
Computer Vision
Let's Split Up: Zero-Shot Classifier Edits for Fine-Grained Video Understanding
Kaiting Liu and Hazel Doughty
Video recognition models are typically trained on fixed taxonomies which are often too coarse, collapsing distinctions in object, manner or outcome under a single label. As tasks and definitions evolve, such models cannot accommodate emerging distinctions and collecting new annotations and retraining to accommodate such changes is costly. To address these challenges, we introduce category splitting, a new task where an existing classifier is edited to refine a coarse category into finer subcategories, while preserving accuracy elsewhere. We propose a zero-shot editing method that leverages the latent compositional structure of
video classifiers to expose fine-grained distinctions without additional data. We further show that low-shot fine-tuning, while simple, is highly effective and benefits from our zero-shot initialization. Experiments on our new
video benchmarks for category splitting demonstrate that our method substantially outperforms
vision-
language baselines, improving accuracy on the newly split categories without sacrificing performance on the rest. Project page: https://kaitingliu.github.io/Category-Splitting/.
Read the full abstract ▾
Computer Vision
DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images
Zeng Tao, Ying Jiang, Yunuo Chen et al. (+7 more)
Recent advances in garment pattern generation have shown promising progress. However, existing feed-forward methods struggle with diverse poses and viewpoints, while optimization-based approaches are computationally expensive and difficult to scale. This paper focuses on sewing pattern generation for garment modeling and fabrication applications that demand editable, separable, and simulation-ready garments. We propose DressWild, a novel feed-forward pipeline that reconstructs physics-consistent 2D sewing patterns and the corresponding 3D garments from a single in-the-wild image. Given an input image, our method leverages
vision-
language models (VLMs) to normalize pose variations at the image level, then extract pose-aware, 3D-informed garment features. These features are fused through a
transformer-based encoder and subsequently used to predict sewing pattern parameters, which can be directly applied to physical simulation, texture synthesis, and multi-layer virtual try-on. Extensive experiments demonstrate that our approach robustly recovers diverse sewing patterns and the corresponding 3D garments from in-the-wild images without requiring multi-view inputs or iterative optimization, offering an efficient and scalable solution for realistic garment simulation and animation.
Read the full abstract ▾
All authors Zeng Tao, Ying Jiang, Yunuo Chen, Tianyi Xie, Huamin Wang, Yingnian Wu, Yin Yang, Abishek Sampath Kumar, Kenji Tashiro, Chenfanfu Jiang
Computer Vision
Benchmarking Adversarial Robustness and Adversarial Training Strategies for Object Detection
Alexis Winter, Jean-Vincent Martini, Romaric Audigier et al. (+2 more)
Object detection models are critical components of automated systems, such as autonomous vehicles and perception-based
robots, but their sensitivity to
adversarial attacks poses a serious security risk. Progress in defending these models lags behind classification, hindered by a lack of standardized evaluation. It is nearly impossible to thoroughly compare attack or defense methods, as existing work uses different
datasets, inconsistent efficiency metrics, and varied measures of perturbation cost. This paper addresses this gap by investigating three key questions: (1) How can we create a fair
benchmark to impartially compare attacks? (2) How well do modern attacks transfer across different architectures, especially from Convolutional Neural Networks to
Vision Transformers? (3) What is the most effective
adversarial training strategy for robust defense? To answer these, we first propose a unified
benchmark framework focused on digital, non-patch-based attacks. This framework introduces specific metrics to disentangle localization and classification errors and evaluates attack cost using multiple perceptual metrics. Using this
benchmark, we conduct extensive experiments on state-of-the-art attacks and a wide range of detectors. Our findings reveal two major conclusions: first, modern
adversarial attacks against object detection models show a significant lack of transferability to
transformer-based architectures. Second, we demonstrate that the most robust
adversarial training strategy leverages a
dataset composed of a mix of high-perturbation attacks with different objectives (e.g., spatial and semantic), which outperforms training on any single attack.
Read the full abstract ▾
All authors Alexis Winter, Jean-Vincent Martini, Romaric Audigier, Angelique Loesch, Bertrand Luvison
Computer Vision
Yihao Lu, Wanru Cheng, Zeyu Zhang et al. (+1 more)
Long-horizon
multimodal agents depend on external
memory; however, similarity-based
retrieval often surfaces stale, low-credibility, or conflicting items, which can trigger overconfident errors. We propose
Multimodal Memory Agent (MMA), which assigns each retrieved
memory item a dynamic reliability score by combining source credibility, temporal decay, and conflict-aware network consensus, and uses this signal to reweight evidence and abstain when support is insufficient. We also introduce MMA-Bench, a programmatically generated
benchmark for belief dynamics with controlled speaker reliability and structured text-
vision contradictions. Using this framework, we uncover the "Visual Placebo Effect", revealing how RAG-based
agents inherit latent visual biases from foundation models. On FEVER, MMA matches baseline accuracy while reducing variance by 35.2% and improving selective utility; on LoCoMo, a safety-oriented configuration improves actionable accuracy and reduces wrong answers; on MMA-Bench, MMA reaches 41.18% Type-B accuracy in
Vision mode, while the baseline collapses to 0.0% under the same protocol. Code: https://github.com/AIGeeksGroup/MMA.
Read the full abstract ▾
All authors Yihao Lu, Wanru Cheng, Zeyu Zhang, Hao Tang
Computer Vision
Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing
Jinsong Li, Xiaoyi Dong, Yuhang Zang et al. (+3 more)
While Large
Vision-
Language Models (LVLMs) have demonstrated remarkable capabilities for
reasoning and self-correction at the textual level, these strengths provide minimal benefits for complex tasks centered on visual perception, such as Chart Parsing. Existing models often struggle with visually dense charts, leading to errors like data omission, misalignment, and
hallucination. Inspired by the human strategy of using a finger as a “visual anchor” to ensure accuracy when reading complex charts, we propose a new paradigm named Visual Self-Refine (VSR). The core idea of VSR is to enable a model to generate pixel-level localization outputs, visualize them, and then feed these visualizations back to itself, allowing it to intuitively inspect and correct its own potential visual perception errors. We instantiate the VSR paradigm in the domain of Chart Parsing by proposing ChartVSR. This model decomposes the parsing process into two stages: a Refine Stage, where it iteratively uses visual feedback to ensure the accuracy of all data points' Pixel-level Localizations, and a Decode Stage, where it uses these verified localizations as precise visual anchors to parse the final structured data. To address the limitations of existing
benchmarks, we also construct ChartP-Bench, a new and highly challenging
benchmark for chart parsing. Our work also highlights VSR as a general-purpose visual feedback mechanism, offering a promising new direction for enhancing accuracy on a wide range of
vision-centric tasks.
Read the full abstract ▾
All authors Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jiaqi Wang, Dahua Lin
Computer Vision
Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems
Ali Faraz, Raja Kolla, Ashish Kulkarni et al. (+1 more)
Designing Optical Character Recognition (OCR) systems for India requires balancing linguistic diversity, document heterogeneity, and deployment constraints. In this paper, we study two training strategies for building
multilingual OCR systems with
Vision-
Language Models through the Chitrapathak series. We first follow a popular
multimodal approach, pairing a generic
vision encoder with a strong
multilingual language model and training the system end-to-end for OCR. Alternatively, we explore fine-tuning an existing OCR model, despite not being trained for the target
languages. Through extensive evaluation on
multilingual Indic OCR
benchmarks and deployment-oriented metrics, we find that the second strategy consistently achieves better accuracy-latency trade-offs. Chitrapathak-2 achieves 3-6x speedup over its predecessor with being state-of-the-art (SOTA) in Telugu (6.69 char ANLS) and second best in the rest. In addition, we present Parichay, an independent OCR model series designed specifically for 9 Indian government documents to extract structured key fields, achieving 89.8% Exact Match score with a faster inference. Together, these systems achieve SOTA performance and provide practical guidance for building production-scale OCR pipelines in the Indian context.
Read the full abstract ▾
All authors Ali Faraz, Raja Kolla, Ashish Kulkarni, Shubham Agarwal
Computer Vision · Cross-listed from eess.IV
Automated Histopathology Report Generation via Pyramidal Feature Extraction and the UNI Foundation Model
Ahmet Halici, Ece Tugba Cebeci, Musa Balci et al. (+2 more)
Generating diagnostic text from histopathology whole slide images (WSIs) is challenging due to the gigapixel scale of the input and the requirement for precise, domain specific
language. We propose a hierarchical
vision language framework that combines a frozen pathology foundation model with a
Transformer decoder for report generation. To make WSI processing tractable, we perform multi resolution pyramidal patch selection (downsampling factors 2^3 to 2^6) and remove background and artifacts using
Laplacian variance and HSV based criteria. Patch features are extracted with the UNI
Vision Transformer and projected to a 6 layer
Transformer decoder that generates diagnostic text via cross
attention. To better represent biomedical terminology, we tokenize the output using BioGPT. Finally, we add a
retrieval based verification step that compares generated reports with a reference corpus using Sentence BERT
embeddings; if a high similarity match is found, the generated report is replaced with the retrieved ground truth reference to improve reliability.
Read the full abstract ▾
All authors Ahmet Halici, Ece Tugba Cebeci, Musa Balci, Mustafa Cini, Serkan Sokmen
Computer Vision
ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understanding
Daichi Yashima, Shuhei Kurita, Yusuke Oda et al. (+1 more)
While
multimodal large
language models (MLLMs) have shown remarkable success across a wide range of tasks, long-form
video understanding remains a significant challenge. In this study, we focus on
video understanding by MLLMs. This task is challenging because processing a full stream of RGB frames is computationally intractable and highly redundant, as self-
attention have quadratic complexity with sequence length. In this paper, we propose ReMoRa, a
video MLLM that processes
videos by operating directly on their compressed representations. A sparse set of RGB keyframes is retained for appearance, while temporal dynamics are encoded as a motion representation, removing the need for sequential RGB frames. These motion representations act as a compact proxy for optical flow, capturing temporal dynamics without full frame decoding. To refine the noise and low fidelity of block-based motions, we introduce a module to denoise and generate a fine-grained motion representation. Furthermore, our model compresses these features in a way that scales linearly with sequence length. We demonstrate the effectiveness of ReMoRa through extensive experiments across a comprehensive suite of long-
video understanding
benchmarks. ReMoRa outperformed baseline methods on multiple challenging
benchmarks, including LongVideoBench, NExT-QA, and MLVU.
Read the full abstract ▾
All authors Daichi Yashima, Shuhei Kurita, Yusuke Oda, Komei Sugiura
Computer Vision
Adaptive Multi-Scale Channel-Spatial Attention Aggregation Framework for 3D Indoor Semantic Scene Completion Toward Assisting Visually Impaired
Qi He, XiangXiang Wang, Jingtao Zhang et al. (+5 more)
Independent indoor mobility remains a critical challenge for individuals with visual impairments, largely due to the limited capability of existing assistive systems in detecting fine-grained hazardous objects such as chairs, tables, and small obstacles. These perceptual blind zones substantially increase the risk of collision in unfamiliar environments. To bridge the gap between
monocular 3D
vision research and practical assistive deployment, this paper proposes an Adaptive Multi-scale
Attention Aggregation (AMAA) framework for
monocular 3D semantic scene completion using only a wearable RGB camera. The proposed framework addresses two major limitations in 2D-to-3D feature lifting: noise
diffusion during back-projection and structural instability in multi-scale fusion. A parallel channel–spatial
attention mechanism is introduced to recalibrate lifted features along semantic and
geometric dimensions, while a hierarchical adaptive gating strategy regulates cross-scale information flow to preserve fine-grained structural details. Experiments on the NYUv2
benchmark demonstrate that AMAA achieves an overall mIoU of 27.88%. Crucially, it yields significant relative improvements of 16.9% for small objects and 10.4% for tables over the MonoScene baseline. Furthermore, a wearable prototype based on an NVIDIA Jetson Orin NX and a ZED~2i camera validates stable real-time performance in indoor environments, demonstrating the feasibility of deploying
monocular 3D scene completion for assistive
navigation.
Read the full abstract ▾
All authors Qi He, XiangXiang Wang, Jingtao Zhang, Yongbin Yu, Hongxiang Chu, Manping Fan, JingYe Cai, Zhenglin Yang
Computer Vision · Cross-listed from cs.RO
MarUco: A Markerless 6D Pose Estimation Framework for Closed-Loop Control of Surgical Continuum Manipulators
Junhyun Park, Chunggil An, Myeongbo Park et al. (+3 more)
Flexible endoscopic continuum manipulators offer high dexterity and access to complex anatomy, but nonlinear hysteresis limits feedforward control accuracy. Closed-loop control can compensate for these errors but requires accurate six-degree-of-freedom (6D) end-effector pose feedback. We present MarUco, a markerless 6D pose estimation framework for closed-loop control using only stereo
vision during operation. A photorealistic pseudo-rigid-body simulation pipeline generates large-scale annotated training data without manual labeling. A multifeature fusion network integrates masks, keypoints, heatmaps, and bounding boxes from both stereo views to estimate an initial pose, followed by a learned single-pass render-and-compare module that refines the pose without iterative optimization. Kinematics-free hand-eye calibration estimates camera-to-
robot-base extrinsics, and self-supervised adaptation uses unlabeled real stereo pairs to mitigate sim-to-real pose error. Across 1,000 real samples, MarUco achieves translation and rotation errors of 0.78 $\pm$ 0.50 mm and 3.07 $\pm$ 1.25{\deg}, respectively, with a total processing time of 52.4 ms per stereo pair. In closed-loop experiments over eight reference paths, MarUco achieves a mean terminal translation error of 1.8 mm, an 88% reduction relative to uncompensated open-loop control. To the best of our knowledge, this is the first markerless position-based visual servoing framework for continuum manipulators.
Read the full abstract ▾
All authors Junhyun Park, Chunggil An, Myeongbo Park, Ihsan Ullah, Sihyeong Park, Minho Hwang
Computer Vision · Cross-listed from cs.RO
Articulated 3D Scene Graphs for Open-World Mobile Manipulation
Martin Büchner, Adrian Röfer, Tim Engelbracht et al. (+5 more)
Semantic scene understanding has enabled
robots to reason and perform affordance-driven object interaction. Despite this, real-world manipulation remains fundamentally limited by the inability to predict how objects move and are articulated. Long-horizon mobile manipulation requires closing the gap between semantics, geometry, and kinematics. In this work, we present MoMa-SG, a novel framework for building semantic-kinematic 3D scene graphs of articulated scenes containing a myriad of interactable objects. Given RGB-D sequences containing multiple object articulations, we temporally segment object interactions and infer object motion using occlusion-robust point tracking. We lift point trajectories to 3D and estimate articulation models using a novel unified twist estimation formulation that robustly estimates revolute and prismatic joint parameters in a single optimization pass. Next, we associate objects with articulations and detect contained objects by
reasoning over parent-child relations at identified opening states. We also introduce the Arti4D-Semantic
dataset, which provides parent-child relation annotations between container and contained objects together with object axis annotations across 62 in-the-wild RGB-D sequences containing 600 object interactions and three distinct observation paradigms. We extensively evaluate the performance of MoMa-SG on two
datasets and ablate key design choices of our approach. In addition, real-world experiments on both a quadruped and a mobile manipulator demonstrate robust manipulation of articulated objects in everyday home environments. We provide code and data at: https://momasg.cs.uni-freiburg.de.
Read the full abstract ▾
All authors Martin Büchner, Adrian Röfer, Tim Engelbracht, Tim Welschehold, Zuria Bauer, Hermann Blum, Marc Pollefeys, Abhinav Valada
Computer Vision
SCAR: Satellite Imagery-Based Calibration for Aerial Recordings
Henry Hölzemann and Michael Schleiss
We introduce SCAR, a method for long-term auto-calibration refinement of aerial visual-inertial systems that exploits georeferenced satellite imagery as a persistent global reference. SCAR estimates both intrinsic and extrinsic parameters by aligning aerial images with 2D–3D correspondences derived from publicly available orthophotos and elevation models. In contrast to existing approaches that rely on dedicated calibration maneuvers or manually surveyed ground control points, our method leverages external geospatial data to detect and correct calibration degradation under field deployment conditions. We evaluate our approach on six large-scale aerial campaigns conducted over two years under diverse seasonal and environmental conditions. Across all sequences, SCAR consistently outperforms established baselines (Kalibr, COLMAP, VINS-Mono), reducing median reprojection error by a large margin, and translating these calibration gains into substantially lower visual localization rotation errors and higher pose accuracy. These results demonstrate that SCAR provides accurate, robust, and reproducible calibration over long-term aerial operations without the need for manual intervention.
Read the full abstract ▾
Computer Vision
Subtractive Modulative Network with Learnable Periodic Activations
Tiou Wang, Zhuoqian Yang, Markus Flierl et al. (+2 more)
We propose the Subtractive Modulative Network (SMN), a novel, parameter-efficient Implicit Neural Representation (INR) architecture inspired by classical subtractive synthesis. The SMN is designed as a principled signal processing pipeline, featuring a learnable periodic activation layer (Oscillator) that generates a multi-frequency basis, and a series of modulative mask modules (Filters) that actively generate high-order harmonics. We provide both theoretical analysis and empirical validation for our design. Our SMN achieves a PSNR of $40+$ dB on two image
datasets, comparing favorably against state-of-the-art methods in terms of both reconstruction accuracy and parameter efficiency. Furthermore, consistent advantage is observed on the challenging 3D NeRF novel view synthesis task. Supplementary materials are available at https://inrainbws.github.io/smn/.
Read the full abstract ▾
All authors Tiou Wang, Zhuoqian Yang, Markus Flierl, Mathieu Salzmann, Sabine Süsstrunk
Computer Vision · Cross-listed from cs.LG
Guide-Guard: Off-Target Predicting in CRISPR Applications
Joseph Bingham, Netanel Arussy and Saman Zonouz
With the introduction of cyber-physical genome sequencing and editing technologies, such as CRISPR, researchers can more easily access tools to investigate and create remedies for a variety of topics in genetics and health science (e.g. agriculture and medicine). As the field advances and grows, new concerns present themselves in the ability to predict the off-target behavior. In this work, we explore the underlying biological and chemical model from a data driven perspective. Additionally, we present a machine learning based solution named Guide-Guard to predict the behavior of the system given a gRNA in the CRISPR gene-editing process with 84% accuracy. This solution is able to be trained on multiple different genes at the same time while retaining accuracy.
Read the full abstract ▾
Computer Vision
A Self-Supervised Approach for Enhanced Feature Representations in Object Detection Tasks
Santiago C. Vilabella, Pablo Pérez-Núñez and Beatriz Remeseiro
In the fast-evolving field of artificial intelligence, where models are increasingly growing in complexity and size, the availability of labeled data for training deep learning models has become a significant challenge. Addressing complex problems like object detection demands considerable time and resources for data labeling to achieve meaningful results. For companies developing such applications, this entails extensive investment in highly skilled personnel or costly outsourcing. This research work aims to demonstrate that enhancing feature extractors can substantially alleviate this challenge, enabling models to learn more effective representations with less labeled data. Utilizing a self-supervised learning strategy, we present a model trained on unlabeled data that outperforms state-of-the-art feature extractors pre-trained on ImageNet and particularly designed for object detection tasks. Moreover, the results demonstrate that our approach encourages the model to focus on the most relevant aspects of an object, thus achieving better feature representations and, therefore, reinforcing its reliability and robustness.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
RefineFormer3D: Efficient 3D Medical Image Segmentation via Adaptive Multi-Scale Transformer with Cross Attention Fusion
Kavyansh Tyagi, Vishwas Rathi and Puneet Goyal
Accurate and computationally efficient 3D medical image
segmentation remains a critical challenge in clinical workflows.
Transformer-based architectures often demonstrate superior global contextual modeling but at the expense of excessive parameter counts and
memory demands, restricting their clinical deployment. We propose RefineFormer3D, a lightweight hierarchical
transformer architecture that balances
segmentation accuracy and computational efficiency for volumetric medical imaging. The architecture integrates three key components: (i) GhostConv3D-based patch
embedding for efficient feature extraction with minimal redundancy, (ii) MixFFN3D module with low-rank projections and depthwise convolutions for parameter-efficient feature extraction, and (iii) a cross-
attention fusion decoder enabling adaptive multi-scale skip connection integration. RefineFormer3D contains only 2.94M parameters, substantially fewer than contemporary
transformer-based methods. Extensive experiments on ACDC and BraTS
benchmarks demonstrate that RefineFormer3D achieves 93.44% and 85.9% average Dice scores respectively, outperforming or matching state-of-the-art methods while requiring significantly fewer parameters. Furthermore, the model achieves fast inference (8.35 ms per volume on GPU) with low
memory requirements, supporting deployment in resource-constrained clinical environments. These results establish RefineFormer3D as an effective and scalable solution for practical 3D medical image
segmentation.
Read the full abstract ▾
Computer Vision
Breaking the Sub-Millimeter Barrier: Eyeframe Acquisition from Color Images
Manel Guzmán and Antonio Agudo
Eyeframe lens tracing is an important process in the optical industry that requires sub-millimeter precision to ensure proper lens fitting and optimal
vision correction. Traditional frame tracers rely on mechanical tools that need precise positioning and calibration, which are time-consuming and require additional equipment, creating an inefficient workflow for opticians. This work presents a novel approach based on artificial
vision that utilizes multi-view information. The proposed
algorithm operates on images captured from an InVision system. The full pipeline includes image acquisition, frame
segmentation to isolate the eyeframe from background, depth estimation to obtain 3D spatial information, and multi-view processing that integrates segmented RGB images with depth data for precise frame contour measurement. To this end, different configurations and variants are proposed and analyzed on real data, providing competitive measurements from still color images with respect to other solutions, while eliminating the need for specialized tracing equipment and reducing workflow complexity for optical technicians.
Read the full abstract ▾
Computer Vision
AFFMAE: Scalable Vision Pre-Training for High-Resolution Microscopy Segmentation on Desktop Hardware
David Smerkous, Zian Wang and Behzad Najafian
Self-supervised pretraining has transformed computer
vision by enabling data-efficient fine-tuning, yet high-resolution pretraining typically requires server-scale infrastructure, limiting custom in-domain training for many research laboratories. Masked Autoencoders (MAE) reduce computation by encoding only visible
tokens, but combining MAE with hierarchical downsampling architectures has remained structurally challenging due to dense grid priors and mask-aware design compromises. We introduce AFFMAE, a masking-friendly hierarchical pretraining framework built on adaptive, off-grid
token merging. AFFMAE removes dense-grid assumptions while preserving hierarchical scalability during pre-training and fine-tuning. To support this architecture, we developed numerically stable mixed-precision Triton kernels and a lightweight, point-based decoder that can be directly repurposed as a
segmentation head. On high-resolution microscopy
segmentation, AFFMAE matches MAE finetuning performance on foot process width estimation with ViT backbone at equal parameter counts while being 2x faster during pre-training and halving peak
memory usage. Furthermore, AFFMAE achieves up to 5x throughput speedups fine-tuning at the 1024px resolution, providing high-resolution model training on desktop hardware. Code available at https://github.com/najafian-lab/affmae.
Read the full abstract ▾
Computer Vision
HyPCA-Net: Advancing Multimodal Fusion in Medical Image Analysis
J. Dhar, M. K. Pandey, D. Chakladar et al. (+4 more)
Multimodal fusion frameworks, which integrate diverse medical imaging modalities (e.g., MRI, CT), have shown great potential in applications such as skin cancer detection, dementia diagnosis, and
brain tumor prediction. However, existing
multimodal fusion methods face significant challenges. First, they often rely on computationally expensive models, limiting their applicability in low-resource environments. Second, they often employ cascaded
attention modules, which potentially increase risk of information loss during inter-module transitions and hinder their capacity to effectively capture robust shared representations across modalities. This restricts their generalization in multi-disease analysis tasks. To address these limitations, we propose a Hybrid Parallel-Fusion Cascaded
Attention Network (HyPCA-Net), composed of two core novel blocks: (a) a computationally efficient residual adaptive learning
attention block for capturing refined modality-specific representations, and (b) a dual-view cascaded
attention block aimed at learning robust shared representations across diverse modalities. Extensive experiments on ten publicly available
datasets exhibit that HyPCA-Net significantly outperforms existing leading methods, with improvements of up to 5.2% in performance and reductions of up to 73.1% in computational cost. Code: https://github.com/misti1203/HyPCA-Net.
Read the full abstract ▾
All authors J. Dhar, M. K. Pandey, D. Chakladar, M. Haghighat, A. Alavi, S. Mistry, N. Zaidi
Computer Vision
EasyControlEdge: A Foundation-Model Fine-Tuning for Edge Detection
Hiroki Nakamura, Hiroto Iino, Masashi Okada et al. (+1 more)
We propose EasyControlEdge, adapting an image-generation foundation model to edge detection. In real-world edge detection (e.g., floor-plan walls, satellite roads/buildings, and medical organ boundaries), crispness and data efficiency are crucial, yet producing crisp raw edge maps with limited training samples remains challenging. Although image-generation foundation models perform well on many downstream tasks, their pretrained priors for data-efficient transfer and iterative refinement for high-frequency detail preservation remain underexploited for edge detection. To enable crisp and data-efficient edge detection using these capabilities, we introduce an edge-specialized adaptation of image-generation foundation models. To better specialize the foundation model for edge detection, we incorporate an edge-oriented objective with an efficient pixel-space loss. At inference, we introduce guidance based on unconditional dynamics, enabling a single model to control the edge density through a guidance scale. Experiments on BSDS500, NYUDv2, BIPED, and CubiCasa compare against state-of-the-art methods and show consistent gains, particularly under no-post-processing crispness evaluation and with limited training data.
Read the full abstract ▾
All authors Hiroki Nakamura, Hiroto Iino, Masashi Okada, Tadahiro Taniguchi
Computer Vision
DataCube: A Video Retrieval Platform via Natural Language Semantic Profiling
Yiming Ju, Hanyu Zhao, Quanyue Ma et al. (+5 more)
Large-scale
video repositories are increasingly available for modern
video understanding and generation tasks. However, transforming raw
videos into high-quality, task-specific
datasets remains costly and inefficient. We present DataCube, an intelligent platform for automatic
video processing, multi-dimensional profiling, and query-driven
retrieval. DataCube constructs structured semantic representations of
video clips and supports hybrid
retrieval with neural re-ranking and deep semantic matching. Through an interactive web interface, users can efficiently construct customized
video subsets from massive repositories for training, analysis, and evaluation, and build searchable systems over their own private
video collections. The system is publicly accessible at https://datacube.baai.ac.cn/. Demo
Video: https://baai-data-cube.ks3-cn-beijing.ksyuncs.com/custom/Adobe%20Express%20-%202%E6%9C%8818%E6%97%A5%20%281%29%281%29%20%281%29.mp4
Read the full abstract ▾
All authors Yiming Ju, Hanyu Zhao, Quanyue Ma, Donglin Hao, Chengwei Wu, Ming Li, Songjing Wang, Tengfei Pan
Computer Vision · Cross-listed from cs.LG
Graph neural network for colliding particles with an application to sea ice floe modeling
Ruibiao Zhu
This paper introduces a novel approach to sea ice modeling using Graph Neural Networks (GNNs), utilizing the natural graph structure of sea ice, where nodes represent individual ice pieces, and edges model the physical interactions, including collisions. This concept is developed within a one-dimensional framework as a foundational step. Traditional numerical methods, while effective, are computationally intensive and less scalable. By utilizing GNNs, the proposed model, termed the Collision-captured Network (CN), integrates data assimilation (DA) techniques to effectively learn and predict sea ice dynamics under various conditions. The approach was validated using synthetic data, both with and without observed data points, and it was found that the model accelerates the simulation of trajectories without compromising accuracy. This advancement offers a more efficient tool for
forecasting in marginal ice zones (MIZ) and highlights the potential of combining machine learning with data assimilation for more effective and efficient modeling.
Read the full abstract ▾
Computer Vision
Uncertainty-Guided Inference-Time Depth Adaptation for Transformer-Based Visual Tracking
Patrick Poggi, Divake Kumar, Theja Tulabandhula et al. (+1 more)
Transformer-based single-object trackers achieve state-of-the-art accuracy but rely on fixed-depth inference, executing the full encoder–decoder stack for every frame regardless of visual complexity, thereby incurring unnecessary computational cost in long
video sequences dominated by temporally coherent frames. We propose UncL-STARK, an architecture-preserving approach that enables dynamic, uncertainty-aware depth adaptation in
transformer-based trackers without modifying the underlying network or adding auxiliary heads. The model is fine-tuned to retain predictive robustness at multiple intermediate depths using random-depth training with knowledge
distillation, thus enabling safe inference-time truncation. At runtime, we derive a lightweight uncertainty estimate directly from the model's corner localization heatmaps and use it in a feedback-driven policy that selects the encoder and decoder depth for the next frame based on the prediction confidence by exploiting temporal coherence in
video. Extensive experiments on GOT-10k and LaSOT demonstrate up to 12% GFLOPs reduction, 8.9% latency reduction, and 10.8% energy savings while maintaining tracking accuracy within 0.2% of the full-depth baseline across both short-term and long-term sequences.
Read the full abstract ▾
All authors Patrick Poggi, Divake Kumar, Theja Tulabandhula, Amit Ranjan Trivedi
Computer Vision
Toward Trustworthy Portrait Editing: Evaluation of Demographic Misrepresentation in I2I Models
Huichan Seo, Minki Hong, Sieun Choi et al. (+2 more)
Instruction-guided image-to-image (I2I) editors are increasingly used in consumer and professional visual workflows, where trustworthiness depends not only on
prompt compliance but also on equitable preservation of identity-relevant attributes. We formalize two failure modes: Soft Erasure, where requested edits are weakly realized or silently suppressed, and Stereotype Replacement, where edits introduce unrequested, stereotype-consistent demographic attributes. Using a controlled
benchmark of 5,040 edited portraits, we evaluate these failures across three recent open-weight editors with
vision-
language model scoring and human evaluation. Our results show that identity-preservation failures are pervasive and demographically uneven. In particular, 62–71% of outputs exhibit skin lightening, with Indian and Black source portraits affected at 72–75%, compared with 44% for White source portraits, indicating output-level drift toward lighter or more White-presenting appearances when identity constraints are underspecified. In a mitigation case study,
prompt-level appearance constraints reduce race-change scores for non-White source portraits by up to 1.48 points, while leaving White source portraits largely unchanged, without modifying model weights. These findings show that identity preservation is not a uniform property of I2I portrait editing systems, but an unevenly distributed trustworthiness failure with direct social consequences. At deployment scale, such silent distortions can shape AI-mediated self-representation and reinforce representational disparities. We introduce a controlled audit protocol for
fairness-aware evaluation and governance of generative editing systems. Project page: https://seochan99.github.io/i2i-demographic-bias
Read the full abstract ▾
All authors Huichan Seo, Minki Hong, Sieun Choi, Jihie Kim, Jean Oh
Computer Vision
IRIS: Intent Resolution via Inference-time Saccades for Open-Ended VQA in Large Vision-Language Models
Parsa Madinei, Srijita Karmakar, Russell Cohen Hoffing et al. (+2 more)
We introduce IRIS (Intent Resolution via Inference-time Saccades), a novel training-free approach that uses eye-tracking data in real-time to resolve ambiguity in open-ended VQA. Through a comprehensive user study with 500 unique image-question pairs, we demonstrate that fixations closest to the time participants start verbally asking their questions are the most informative for disambiguation in Large VLMs, more than doubling the accuracy of responses on ambiguous questions (from 35.2% to 77.2%) while maintaining performance on unambiguous queries. We evaluate our approach across state-of-the-art VLMs, showing consistent improvements when gaze data is incorporated in ambiguous image-question pairs, regardless of architectural differences. We release a new
benchmark dataset to use eye movement data for disambiguated VQA, a novel real-time interactive protocol, and an evaluation suite.
Read the full abstract ▾
All authors Parsa Madinei, Srijita Karmakar, Russell Cohen Hoffing, Felix Gervitz, Miguel P. Eckstein
Computer Vision
CHAI: CacHe Attention Inference for text2video
Joel Mathew Cherian, Ashutosh Muralidhara Bharadwaj, Vima Gupta et al. (+1 more)
Text-to-
video diffusion models deliver impressive results but remain slow because of the sequential denoising of 3D latents. Existing approaches to speed up inference either require expensive model retraining or use heuristic-based step skipping, which struggles to maintain
video quality as the number of denoising steps decreases. Our work, CHAI, aims to use cross-inference caching to reduce latency while maintaining
video quality. We introduce Cache
Attention as an effective method for attending to shared objects/scenes across cross-inference latents. This selective
attention mechanism enables effective reuse of cached latents across semantically related
prompts, yielding high cache hit rates. We show that it is possible to generate high-quality
videos using Cache
Attention with as few as 8 denoising steps. When integrated into the overall system, CHAI is 1.65x - 3.35x faster than baseline OpenSora 1.2 while maintaining
video quality.
Read the full abstract ▾
All authors Joel Mathew Cherian, Ashutosh Muralidhara Bharadwaj, Vima Gupta, Anand Padmanabha Iyer
Computer Vision
OmniCT: Towards a Unified Slice-Volume LVLM for Comprehensive CT Analysis
Tianwei Lin, Zhongwei Qiu, Wenqiao Zhang et al. (+12 more)
Computed
Tomography (CT) is one of the most widely used and diagnostically information-dense imaging modalities, covering critical organs such as the heart, lungs, liver, and colon. Clinical interpretation relies on both slice-driven local features (e.g., sub-centimeter nodules, lesion boundaries) and volume-driven spatial representations (e.g., tumor infiltration, inter-organ anatomical relations). However, existing Large
Vision-
Language Models (LVLMs) remain fragmented in CT slice versus volumetric understanding: slice-driven LVLMs show strong generalization but lack cross-slice spatial consistency, while volume-driven LVLMs explicitly capture volumetric semantics but suffer from coarse granularity and poor compatibility with slice inputs. The absence of a unified modeling paradigm constitutes a major bottleneck for the clinical translation of medical LVLMs. We present OmniCT, a powerful unified slice-volume LVLM for CT scenarios, which makes three contributions: (i) Spatial Consistency Enhancement (SCE): volumetric slice composition combined with tri-axial positional
embedding that introduces volumetric consistency, and an MoE hybrid projection enables efficient slice-volume adaptation; (ii) Organ-level Semantic Enhancement (OSE):
segmentation and ROI localization explicitly align anatomical regions, emphasizing lesion- and organ-level semantics; (iii) MedEval-CT: the largest slice-volume CT
dataset and hybrid
benchmark integrates comprehensive metrics for unified evaluation. OmniCT consistently outperforms existing methods with a substantial margin across diverse clinical tasks and satisfies both micro-level detail sensitivity and macro-level spatial
reasoning. More importantly, it establishes a new paradigm for cross-modal medical imaging understanding. Our project is available at https://github.com/ZJU4HealthCare/OmniCT.
Read the full abstract ▾
All authors Tianwei Lin, Zhongwei Qiu, Wenqiao Zhang, Jiang Liu, Yihan Xie, Mingjian Gao, Zhenxuan Fan, Zhaocheng Li, Sijing Li, Zhongle Xie, Peng LU, Yueting Zhuang, Ling Zhang, Beng Chin Ooi, Yingda Xia