Computer Vision
From "What-If" to "What-Is": Counterfactual Thinking-Inspired Semantic Alignment for Visual Brain Decoding
Kaitao Yan, Chi Liu, Congcong Zhu et al. (+5 more)
Visual
brain decoding reconstructs visual content perceived by a person from neural measurements such as fMRI, providing a computational approach to studying how visual information is represented in the
brain. Recent
multimodal representations and
diffusion priors have improved reconstruction realism. However, visually plausible reconstructions may contain incorrect objects, attributes, or relations because a strong generative prior can complete content not sufficiently specified by the decoded representation. Conventional reconstruction metrics mainly assess the final image and may therefore obscure such semantic errors. We propose ConceptAlign, a counterfactual semantic
alignment framework for visual
brain decoding. ConceptAlign pools decoded visual
tokens and projects them into a frozen text-
embedding space, aligning the representation with the ground-truth caption while separating it from scene-preserving near-miss alternatives. Generated offline by an LLM, these alternatives modify one critical object, attribute, or relation while retaining the scene. A margin-based objective learns fine-grained semantic boundaries between the observed stimulus and plausible but incorrect interpretations without requiring LLM calls during inference. We introduce a systematic three-level semantic evaluation framework covering foundational discriminability, counterfactual description discrimination, and representational geometry. Experiments on the Natural Scenes
Dataset show that ConceptAlign improves reconstruction measures, counterfactual semantic discrimination, and representational
alignment over the MindEye2 backbone. Matched negative-source ablations, independent LLM and human-written alternatives, and human evaluation support the effectiveness and robustness of the supervision, with favorable patterns in fine-grained conflicts, limited-data decoding, and cross-subject structure.
Read the full abstract ▾
All authors Kaitao Yan, Chi Liu, Congcong Zhu, Huajie Chen, Gengshen Wu, Minghao Wang, Xiaotong Han, Tianqing Zhu
Computer Vision
A Unified Backbone–Expert Framework with Relation-Token and Residual–Classifier Interfaces for Automatic Modulation Recognition
Zhixiang Deng, Houbiao Li and Zongyong Cui
Automatic modulation recognition (AMR) faces distinct representation bottlenecks under varying observation lengths, where a single model architecture often fails to excel. To address this, we propose a unified backbone-expert framework with a common convolutional state-space backbone and two specialized interfaces. For short sequences, we inject explicit lag-aware complex-plane descriptors as relation
tokens before encoding to compensate for information loss. For long sequences, we design a gated multi-scale residual refinement module to correct the feature map, combined with a fixed-averaging classifier collaboration to harness complementary evidence. Our framework achieves overall average accuracies of 67.28 \pm 0.14% on RML2016.10b and 87.19 \pm 0.77% on HisarMod2019 (mean \pm sample standard deviation over three runs), respectively. The framework's efficacy is further validated through three-seed ablations, native-length cross-configuration tests, and controlled window studies, confirming the benefit of expert-interface decoupling over one-size-fits-all architectures.
Read the full abstract ▾
Computer Vision
HOIMask: Towards Generative Masked Modeling for Human Object Interaction Generation
Yihong Ji, Jinsong Zhang, He Hu et al. (+1 more)
Diffusion-based methods have dominated the HOI generation, as they enable critical contact fusions or signals to guide the
diffusion process. However, they often result in high artifacts and unstable interaction quality due to error accumulation during iterative denoising. In this work, we propose HOIMask, the first generative masked framework for modeling HOI motion in discrete space. HOIMask first encodes both motion sequences and contact-aware signals into discrete 2D human and object
token maps via HOI Vector Quantization (VQ), preserving fine-grained spatial-temporal structure beyond conventional 1D representations. On this basis, a generative masked modeling framework is employed to jointly capture human-object interaction dynamics, leveraging a
transformer architecture designed to model complex spatial-temporal and interaction dependencies. To generate more coherent and physically plausible motions, we further introduce a novel contact-aware reconstruction guidance in discrete space during inference, which fuses contact signals to optimize HOI
tokens that forces the generated motion with higher spatio-temporal consistency. With craftily designed motion interaction
tokens, dedicated architecture and guidance strategy, HOIMask outperforms state-of-the-art
diffusion-based methods, generating more realistic and semantically aligned HOI motions. Please refer to https://jyhflash.github.io/HOIMask/ for more results.
Read the full abstract ▾
All authors Yihong Ji, Jinsong Zhang, He Hu, Hongbo Xu
Computer Vision
Perspective-Invariant Attack with Enhanced Transferability of Adversarial Examples
Kaisheng Liang, Yiming Cao and Bin Xiao
Adversarial examples generated on a surrogate deep neural network (DNN) can often successfully fool other black-box DNN models. This cross-model transferability poses serious security threats to DNNs in practical applications. Input transformation techniques are widely used to enhance
adversarial transferability by increasing the diversity of input images. However, existing methods primarily rely on local operations with limited degrees of freedom (DOF), such as block-wise shuffling and resizing, overlooking global perspective transformations that naturally arise from viewpoint changes. In this work, we propose a Perspective-Invariant Attack (PIA), which introduces a multi-DOF vertex sampling strategy that systematically covers the perspective transformation hierarchy from 2-DOF translation to 8-DOF projective mapping. By generating geometrically diverse input variations, PIA effectively reduces overfitting of
adversarial perturbations to the surrogate model, thereby improving
adversarial transferability. We further propose PIA-Mix, a generic extension that maintains a complementary transformation pool and efficiently combines our perspective transformation with auxiliary methods for improved transferability. Extensive experiments involving various DNN architectures, advanced defense mechanisms, and
multimodal large
language models (LLMs) demonstrate that PIA and PIA-Mix outperform state-of-the-art transfer-based attacks.
Read the full abstract ▾
Computer Vision
Fast Test-Time Refinement for Robust Learned Image Compression
Jiaming Liang, Chi-Man Pun and Weisi Lin
Learned image compression (LIC) has demonstrated remarkable rate-distortion (RD) performance in benign settings. However, the high representational capacity endowed by deep neural networks (DNNs) comes at the expense of increased
adversarial vulnerability. This hinders their adoption as trusted standardized codecs. Recent work has sketched test-time refinement (TTR) as a defense in gray-box scenarios, despite its original purpose of improving benign RD performance. Unfortunately, extensive iterations of TTR incur prohibitive overhead, while the robustness mechanism lacks theoretical understanding. Moreover, TTR has not been evaluated in white-box settings or against attacks beyond $\ell_2$-bounded rate and untargeted distortion objectives. To bridge these gaps, we present a systematic study. Our study reveals an Asymmetric
Adversarial Trajectory (AAT) property in LIC systems: transitioning from
adversarial to benign regions is significantly easier than the reverse process, where
adversarial examples can often be roughly recovered within only 1-2 steps. We provide a two-dimensional Tube Model to explain this phenomenon. Based on AAT, we propose a Fast Test-Time Refinement (FTTR) framework for practical and robust LIC systems. We establish that the robustness arises from the contraction of
adversarial regions induced by the Input-as-Label property of LIC systems, rather than from obfuscated gradients. Extensive evaluations with diverse strong adaptive attacks across multiple LIC systems demonstrate the promise of the proposed FTTR framework. The code is available at https://github.com/chinaliangjiaming/FTTR.git.
Read the full abstract ▾
Computer Vision
CETalk: Continuous Valence-Arousal Control for Audio-Driven 3D Talking Head Generation
Peng Jia, Li Dai, Zhen Xiao et al. (+2 more)
Emotional 3D talking head generation aims to synthesize expressive facial animations with accurate lip synchronization. However, existing methods often rely on discrete
emotion categories, which fail to capture the continuous
evolution of affect. They also overlook the temporal frequency mismatch between audio articulation and emotional expression. In this paper, we propose CETalk, an audio-driven 3D facial animation framework conditioned on continuous Valence–Arousal (VA) representations for fine-grained
emotion control. CETalk predicts a sequence of FLAME parameters through three key components: a Dynamic
Emotion Modulation Module that adaptively scales emotional intensity using audio-derived cues; a Multi-Scale Temporal Modeling mechanism that employs parallel branches to decouple high-frequency articulatory movements from low-frequency emotional dynamics; and a Dynamic Fusion Mechanism that integrates these multi-scale features via an adaptive gating network. To support training and evaluation, we construct 3D-VA-MEAD, a large-scale
dataset with automatically estimated VA annotations and reconstructed 3D facial motions. Extensive experiments demonstrate that CETalk outperforms state-of-the-art methods in both lip-sync accuracy and emotional expressiveness, while enabling smooth and controllable
emotion transitions.
Read the full abstract ▾
All authors Peng Jia, Li Dai, Zhen Xiao, Xueliang Liu, Jia Li
Computer Vision · Cross-listed from cs.LG
EMASAM: a Computationally Efficient Sharpness-Aware Minimization via EMA-Guided Perturbations
Tanapat Ratchatorn and Masayuki Tanaka
Recent progress in optimization research has highlighted the sharpness of the loss landscape as a key factor in narrowing the generalization gap. Motivated by this insight, Sharpness-Aware Minimization (SAM) was proposed as a training strategy that enhances generalization. Despite the promising performance, SAM suffers from its twice computational cost due to its core
algorithm requiring an extra gradient computation during the perturbation step. To overcome this limitation, we introduce Exponential Moving Average Sharpness-Aware Minimization (EMASAM), a computationally efficient variant of SAM. EMASAM does not require the loss gradient in the perturbation step. Instead, EMASAM defines the perturbation direction based on the discrepancy between the main model and the EMA shadow model. This perturbation travels away from the stable average position toward the less stable area, acting as a softer yet cheaper alternative to SAM's worst-case scenario perturbation. Moreover, since EMASAM's perturbation does not rely on noisy mini-batch gradients, it mitigates the gradient-induced instability inherent in SAM. Hence, EMASAM eliminates the need for an extra backpropagation while also preserving the generalization ability of the SAM-style training. Several experiments have been performed and confirm the efficiency and robustness of our method.
Read the full abstract ▾
Computer Vision
ProjFormer: Point Cloud Completion via Geometric-Projective Transformer and Cross-Modal Semantic Constraints
Sheng Liu, Meng Wang, Ruihui Li et al. (+3 more)
Point cloud completion is inherently ill-posed due to severe sparsity and ambiguity in partial observations. Existing multi-view methods alleviate this by incorporating 2D semantics, but often rely on learned
attention and fixed fusion, which lack
geometric consistency and adaptability. We propose ProjFormer, a cross-modal framework that enforces geometry-consistent 2D-3D interaction through explicit projection and adaptive feature routing. A Projective Guided View
Attention module aligns 3D points with multi-view features via deterministic projection, enabling efficient and geometrically consistent aggregation. Building on this, a geometry-aware routing network performs point-wise adaptive fusion of structural and observation-driven features for progressive refinement. Experiments show that, under a lightweight design, ProjFormer delivers competitive performance with improved structural completeness.
Read the full abstract ▾
All authors Sheng Liu, Meng Wang, Ruihui Li, Huilong Pi, Zhuo Tang, Kenli Li
Computer Vision
MODAL: Multi-Modal Object Re-ID via Model-Driven Sparse Decoupling and Text-Image Differential Filtering
Chengbo Huang, Jun-Jie Huang, Long Lan et al. (+5 more)
Multi-modal object re-identification (Re-ID) aims to facilitate cross-camera object
retrieval in complex environments by leveraging complementary information from visual (e.g., RGB, NIR, TIR) and textual modalities. However, existing approaches often lack principled feature disentanglement and coherent multi-modal integration, leading to entangled representations that introduce cross-modal conflicts, obscure discriminative cues, and suffer distribution shift under modality-missing conditions. To tackle these challenges, we propose MODAL, a novel multi-modal object re-identification framework, grounded in coupled sparse coding theory and differential suppression principles. A core component of MODAL is a Multi-modal Feature Sparse Decoupling module, developed in a model-driven deep unrolling manner based on multi-modal coupled sparse coding. It explicitly decomposes multi-modal features into uni-modal specific, bi-modal and tri-modal shared representations, thereby achieving more transparent and effective feature disentanglement. Benefiting from the principled feature disentanglement, MODAL naturally mitigates performance degradation in incomplete-modality scenarios via a Modality-Aware Subspace Activation that selectively activates only the consistently shared subspaces. Moreover, we propose a Text-Image Differential Filtering module that leverages coarse-grained textual semantics to adaptively suppress task-irrelevant responses in the decoupled visual representations, thereby enhancing discriminative information. Extensive experiments on four
datasets demonstrate that MODAL achieves state-of-the-art performance with superior transparency.
Read the full abstract ▾
All authors Chengbo Huang, Jun-Jie Huang, Long Lan, Tianrui Liu, Xueqiong Li, Yuanxi Peng, Xinwang Liu, Meng Wang
Computer Vision
Distribution-free false-alarm calibration and chance-corrected spatial evaluation for industrial anomaly detection
Jie Deng
Studies of industrial visual inspection commonly report the area under the receiver operating characteristic curve (AUROC) and the overlap between anomaly maps and defect masks. Neither measure specifies the false-alarm rate at a selected threshold, while recurrent defect locations and mask geometry can inflate overlap. We combine a distribution-free upper tolerance threshold with a paired-minus-crossed spatial test. This test compares each detector's score-contributing locations with the matched defect mask and with masks from other images; the difference in rates defines spatial-evidence lift relative to the empirical chance-overlap rate. We evaluate three detectors on 120 point-defect images from three ISP-AD modalities and three fixed data splits. Of 378 alarms, 230 overlap the matched mask. Paired and crossed rates are nevertheless similar in eight of nine detector–modality
cells; only DINOv2–ASM has a positive 95% bootstrap lower bound (lift 0.259, 95% interval 0.159–0.347). On the independent
Magnetic Tile Defect
dataset, the same analysis gives lifts of 0.203 (0.169–0.236) for Wide ResNet-50 (WRN50) patch
memory and 0.231 (0.202–0.262) for
Vision Transformer B/16 (ViT-B/16) patch
memory, with one-sided permutation $p=10^{-5}$ for both. When crossed masks are restricted to the same defect class, the lifts remain 0.185 and 0.210. Exact sample planning shows that, with 150 calibration normals, a 95%-confidence distribution-free claim is supported only for target false-positive rates of 1.98% or higher; a 1% target requires at least 299 normals. The results support reporting operating-point performance and chance-corrected spatial evidence alongside AUROC and raw mask overlap.
Read the full abstract ▾
Computer Vision
SA-GEM: Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning for Efficient Remote Sensing Large Vision-Language Models
Kexin Ma, Jing Xiao, Bowen Xing et al. (+2 more)
RS-LVLMs have advanced
multimodal understanding of Earth observation imagery, yet their performance is fundamentally constrained by high-resolution processing, as visual
token counts grow quadratically with linear input resolution while important visual evidence is inherently sparse and increasingly diluted across the expanded sequence. Existing
token pruning methods largely rely on scale-agnostic resolution policies and isolated importance cues, limiting task-aligned granularity adaptation and holistic evidence preservation. To address this, we present Scale-Adaptive and Geospatial Evidence-Modulated
Token Pruning (SA-GEM), a plug-and-play framework that unifies task-adaptive
token granularity allocation with holistic geospatial
token importance modulation. Specifically, a lightweight router selects the resolution based on query-dependent
token granularity, while a
token importance modulator jointly models task relevance, spatial structure, and local redundancy to preserve holistic geospatial evidence. We show that higher resolution is not universally beneficial and, once sufficient granularity is reached,
token quality matters more than
token quantity. Experiments across various
benchmarks demonstrate that SA-GEM achieves consistent gains in both accuracy and efficiency over existing
pruning methods. On XLRS-Bench, it surpasses GeoLLaVA-8K by 2.3% in accuracy with a 2.4 times total inference speedup.
Read the full abstract ▾
All authors Kexin Ma, Jing Xiao, Bowen Xing, Liang Liao, Chia-Wen Lin
Computer Vision
Do Visual Grounding Decoders Need Feed-Forward Networks? A Controlled Study over Frozen Vision-Language Features
Tarun Tomar
Do feed-forward networks (FFNs) in visual grounding decoders add essential computation once a pretrained
vision-
language model has already encoded image and
language context? We compare a four-block
attention-only decoder (A4), a matched four-block
attention-plus-FFN decoder (S4), and an eight-block
attention-only parameter control (A8) over frozen VLM features. A4 matches or slightly exceeds S4 on RefCOCOg and Ref-Adv-s. FineCops-Ref reveals a small A4 deficit of 0.52 percentage points at IoU@0.5 (95% CI [0.12, 0.95] in favor of S4), but A8 recovers it and finishes 0.26 points above S4. Official FineCops levels do not show a monotonic increase in the gap. A4 reduces trainable decoder parameters by 44.4% and cached-decoder latency by 10.1%, although end-to-end latency remains backbone-dominated. These results concern the trainable grounding decoder, not a complete
attention-only VLM.
Read the full abstract ▾
Computer Vision
EgoTac: In-the-wild Tactile Prediction from Egocentric Vision
Wenkang Zhang, Chengbo Yuan, Zicheng Zhang et al. (+2 more)
Touch is fundamental to dexterous manipulation, yet most
egocentric human data increasingly used for
robot learning lacks tactile information. Directly collecting large-scale tactile data is challenging due to sensor limitations, while human
video data is abundant, contact-rich, and easily scalable. This motivates a natural question: can tactile signals be inferred purely from
vision? To address this, we introduce EgoTac, a generalizable model that predicts rich tactile information directly from
egocentric human
videos. EgoTac is trained on a unified corpus of over 5.7M image-tactile pairs, covering both continuous force measurements and binary contacts. By learning from this diverse
dataset, EgoTac captures nuanced touch dynamics across varied interactions. Experiments demonstrate strong performance: in-domain prediction achieves an average force error below 0.06N. On out-of-domain contact prediction
benchmarks, EgoTac consistently outperforms the state-of-the-art contact
estimator. It also captures the rise and fall patterns of real tactile data and enables zero-shot predictions on unconstrained real-world
videos. Scaling analyses further reveal that both data diversity and volume improve performance steadily. Overall, EgoTac provides a scalable pathway to extract tactile priors from
egocentric human
videos, enabling broadly applicable tactile-aware
robot learning. Related resources can be found at https://egotac.github.io/ .
Read the full abstract ▾
All authors Wenkang Zhang, Chengbo Yuan, Zicheng Zhang, Zhengxue Cheng, Yang Gao
Computer Vision
MEDR: Query-Independent Frame Selection via Multi-Signal Event Modeling and Dynamic Rescoring
Xinlei Pu, Weijie Shi, Wen Yang et al. (+6 more)
Frame selection is a fundamental component of
multimodal large
language models, enabling long
videos to be processed under limited visual-
token and computational budgets. Uniform sampling preserves temporal coverage but may miss informative content that appears only briefly. To alleviate this limitation, query-dependent methods can retrieve question-relevant frames. However, because the selected frames depend on the current question, the same visual input cannot be directly shared across different questions, and frame selection must be repeated in multi-turn
video dialogue. This motivates us to seek a query-independent frame selection method that preserves the reusability of a fixed visual input while improving the coverage of informative events beyond uniform sampling. We propose Multi-Signal Event Modeling and Dynamic Rescoring (MEDR), a training-free and query-independent frame selection method. Multi-Signal Event Modeling organizes complementary visual, motion, and text signals into signal-specific temporal events. Dynamic Rescoring then iteratively reevaluates each candidate relative to the current selected set, updating its score according to frame-level signal strength, additional event coverage, and temporal proximity. The resulting fixed frame set is constructed without observing the query and can be reused across different questions. On the standard
benchmark evaluations, MEDR improves model accuracy by 0.63%-0.89% on
Video-MME. On the long-
video subset of LongVideoBench, it improves accuracy by up to 1.23% with Qwen3-VL-8B. MEDR further improves overall accuracy by 0.53%, while reusing exactly the same frame set for every question about a
video.
Read the full abstract ▾
All authors Xinlei Pu, Weijie Shi, Wen Yang, Yi Cao, Hao Chen, Yuanjun Liu, Wenwei Ding, Jia Zhu, Jiajie Xu
Computer Vision
Frequency and Edge-Guided Segment Anything Model for Remote Sensing Image Semantic Segmentation
Feng Gao, Zizhe Pan, Haoting Wang et al. (+4 more)
Remote sensing image semantic
segmentation (RSISS) has attracted significant
attention due to the growing demand for fine-grained land cover information. The Segment Anything Model (SAM), proposed as a foundation
vision model, offers strong
segmentation performance and generalization capabilities for RSISS tasks. However, existing SAM-based approaches face two limitations: (1) Insufficient adaptation of SAM's features to the diverse characteristics of land cover types. (2) Semantic ambiguity at object boundaries, which hinders accurate delineation. To address these limitations, we propose Frequency and Edge-guided SAM (FE-SAM), a scalable and efficient framework for RSISS. Specifically, we introduce a Frequency-Modulated Adapter (FMA) that adaptively decomposes and modulates frequency-domain features based on the input data. It selectively enhances informative high- and low-frequency components corresponding to different land cover types. Furthermore, to improve SAM's ability to capture fine-grained details, we design EGRefiner, which integrates multi-scale edge-enhanced information extracted from the input image. Extensive experiments on three
benchmark datasets demonstrate that FE-SAM outperforms state-of-the-art methods. The source codes are available at: https://github.com/oucailab/FE-SAM.
Read the full abstract ▾
All authors Feng Gao, Zizhe Pan, Haoting Wang, Ruzhuang Hua, Jingchao Cao, Junyu Dong, Qian Du
Computer Vision
MOSS-VL Technical Report
Pengyu Wang, Chenkun Tan, Shaojun Zhou et al. (+29 more)
We present MOSS-VL, an open
vision-
language model family that treats real-time interaction – perceiving while it speaks – as a first-class capability. It is co-designed across the stack: the
language decoder attends to
vision only through gated cross-
attention, so the model can naturally see incoming frames while generating; a synthesized interaction corpus supervises when to speak, when to stay silent, and when to revise; and a staged curriculum concentrates all real-time-specific training in one light final stage over a strong offline foundation. Offline, MOSS-VL-Instruct is competitive at comparable scale and leads temporal-
reasoning video sets. Across four streaming
benchmarks, MOSS-VL-Realtime posts the best average on three (second on the fourth) among open-source streaming models, sweeping the three subsets that squarely test proactive behavior – 66.0 vs. 37.5 for the best baseline on OmniMMI Proactive Alerting. With 11.3B parameters but visual
tokens outside the decoded sequence, MOSS-VL widens its time-to-first-
token advantage over same-backbone Qwen3-VL-8B from 2.8x to 5.1x as visual context grows. We release all five checkpoints, the training curriculum, and the real-time inference code at https://github.com/OpenMOSS/MOSS-VL.
Read the full abstract ▾
All authors Pengyu Wang, Chenkun Tan, Shaojun Zhou, Qirui Zhou, Yanxin Chen, Xingyang He, Huazheng Zeng, Jijun Cheng, Chenghao Wang, Xiaomeng Qian, Pengfei Wang, Zhan Huang, Shanqing Gao, Wei Huang, Longjun Cao, Wu Ran, Jie Liu, Changtai Zhu, Hongkai Wang, Yixian Tian, Chenghao Liu, Zhen Ye, Xinghao Wang, Botian Jiang, Guoguo Feng, Zhaoye Fei, Ruixiao Li, Mingshu Chen, Yang Gao, Qinyuan Cheng, Shimin Li, Xipeng Qiu
Computer Vision · Cross-listed from cs.CL
Handoff-H1: An Orchestrated Vision-Agent System for Material Quantity Takeoff from Construction Blueprints
Bruno Chicelli, Henrique Alves, Rodrigo Anselmo et al. (+3 more)
Converting a set of architectural blueprints into a complete material quantity takeoff requires visual perception across drawing sheets, dimensional and multi-hop
reasoning, and grounding in construction conventions that the drawings never state. We present Handoff-H1, a takeoff system built from three layers: purpose-built computer-
vision models that extract primitives; tool-using
agents equipped with image operations and in-house visual-task tools, including CV-model-backed counting, detection and plan decomposition; and a persistent, hierarchically structured project foundation, grounded in a curated construction knowledge base. We evaluate on the Construction Blueprint Takeoff
Benchmark: 10 real residential blueprint sets paired with consensus-validated expert takeoffs - 2,009 verified line items, restricted for scoring to the 1,348 primary-tier materials that drive an estimate - scored per trade by an LLM judge on material coverage and quantity Precision@25% (P@.25) and combined into a weighted composite. Under identical scoring from the raw PDF, seven frontier and open-weight models span composites of 35-61, and independent professional
estimators - scored against the same reconciled gold standard - post 77.6% (65.5% coverage, 87.9% P@.25). Handoff-H1, working end-to-end from the raw PDF, reaches 81.6% (86.1% coverage, 78.8% P@.25): roughly 20 points above the strongest frontier
agent, and above the independent
estimators by pairing near-human quantity precision with coverage they do not reach. The evaluation harness is public for the open harbor framework; the blueprint sets and ground truth are available upon request for research use.
Read the full abstract ▾
All authors Bruno Chicelli, Henrique Alves, Rodrigo Anselmo, Joshua Weinberg, Felipe Lemos, Jan Baryla
Computer Vision
Generation of Synthetic Fingerphotos with GANs
Conor Miller-Lynch, Sandip Purnapatra, Syed Konain Abbas et al. (+4 more)
Contactless fingerprinting is an emerging approach to biometric authentication that allows users to scan their fingerprints without touching a scanner. Due to the limited amount of contactless fingerprint data available and the security risks associated with sharing real individuals' fingerprints, it is valuable to explore methods of generating synthetic data that can be used in place of - or in conjunction with - real data to develop and evaluate contactless fingerprinting systems. In this paper, we present and evaluate synthetic fingerphotos generated using StyleGAN2-ADA and StyleGAN3, existing image generation architectures. We evaluate the realism,
privacy preservation, and variety of the synthetic fingerphotos by comparing their biometric feature statistics to those of real fingerphotos, computing match scores between real and synthetic fingerphotos, and computing match scores between different synthetic fingerphotos. This paper provides a quantitative comparison point for future evaluations of synthetic fingerphotos. The evaluation code is made available at https://github.com/cmillerlynch/fingerphoto-gan.
Read the full abstract ▾
All authors Conor Miller-Lynch, Sandip Purnapatra, Syed Konain Abbas, Lambert Igene, Faraz Hussain, Soumyabrata Dey, Stephanie Schuckers
Computer Vision
Closed-Form Nonlocal Shrinkage for Multiplicative Image Denoising and SAR Despeckling
Xuran Hu, Mingzhe Zhu, Djordje Stanković et al. (+4 more)
Multiplicative noise poses a challenge in coherent and signal-dependent imaging owing to its intensity-dependent variance and frequently non-Gaussian distribution. We propose a deterministic nonlocal
estimator that combines a logarithmic Yeo–Johnson transformation, patch grouping, an adaptive singular basis, and sparse shrinkage. The orthonormal group dictionary makes the weighted Lasso separable and yields an exact coefficient-wise soft-threshold solution. This solution replaces the iterative inner solver and expresses patch reliability and atom importance through a single threshold field. Since the dictionary is estimated from the noisy group, we introduce a random-
matrix correction governed by the group aspect ratio $\gamma=p^2/K$. The correction links patch size, group size, and shrinkage strength. Experiments cover gamma-corrupted images from three standard
benchmarks and real synthetic aperture radar (SAR) imagery from five sensors. The method gives the best result in 18 of 24 PSNR/SSIM comparisons with twelve published methods and the lowest mean ratio-image deviation across six real SAR configurations. These results support geometry-calibrated nonlocal modeling for structure-preserving image restoration, with SAR despeckling serving as a demanding application. Code is available \href{https://github.com/Teriri1999/Geometry-Calibrated-Closed-Form-Shrinkage-for-SAR-Despeckling}{here}.
Read the full abstract ▾
All authors Xuran Hu, Mingzhe Zhu, Djordje Stanković, Yujie Zhu, Zhenpeng Feng, Yifang Ban, Ljubiša Stanković
Computer Vision · Cross-listed from cs.RO
MotionGS-SLAM: Event-Modulated Gaussian Splatting for Motion-Blur Robust SLAM
Zhiqiang Hu, Shouren Huang and Masatoshi Ishikawa
Current
Vision-based SLAM systems fail catastrophically when motion blur corrupts the visual input, as they attempt the ill-posed inverse problem of recovering sharp content from degraded observations. We present MotionGS-SLAM, which fundamentally reimagines motion blur handling through a paradigm shift: rather than removing blur artifacts, we reformulate the challenge as a well-constrained forward problem that generatively models blur formation within the rendering pipeline. By leveraging event cameras' microsecond temporal resolution and immunity to motion blur, we introduce a novel event-modulated Gaussian kernel that dynamically adapts each Gaussian's rasterization based on precise motion cues. Our dual-modulation mechanism transforms 2D Gaussian projections from isotropic dots into anisotropic, motion-aligned elliptical brush strokes (spatial modulation) while adaptively varying exposure integral sampling density based on local velocity (temporal modulation). This physics-based approach enables joint optimization of intra-exposure camera trajectories and 3D scene geometry through blur-aware photometric and event-based constraints. Extensive experiments demonstrate significant improvements over state-of-the-art methods in trajectory accuracy and map quality under severe high-motion conditions.
Read the full abstract ▾
Computer Vision
DualMiT-Net: Local-Global Transformer-Convolutional Fusion for Breast Mass Segmentation in Mammographic Regions of Interest
Alibek Kamiluly, Milana Muratova, Yash Patel et al. (+1 more)
Breast mass
segmentation is an important step in computer-aided mammography, but it remains difficult because masses can have low contrast, irregular shapes, and boundaries that blend with surrounding breast tissue. To address this problem, we present DualMiT-Net, a dual-branch network that uses both a focused view of the mass and a wider view of the surrounding tissue. The local branch uses a Mix
Transformer (MiT-B5) encoder to learn mass shape, texture, and boundary information, while the global branch uses an EfficientNet-B5 encoder to learn surrounding breast context. Features from the two branches are shared at the deeper encoder levels and are then progressively fused in a single decoder. A spatial gate controls how much global information is added during decoding. We also evaluated four input representations and selected a percentile-windowed mammogram combined with a Gabor texture response. The model was trained and evaluated on the mass subset of the Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM) using a patient-level split. Across three training runs, DualMiT-Net with exponential moving average weights achieved a mean Dice coefficient of 0.9375 and a mean Intersection over Union of 0.8834. It also achieved better Dice and IoU scores than six standard encoder-decoder baselines trained using the same data and training settings. These results show that combining local mass information with wider breast context can provide accurate and consistent breast mass
segmentation.
Read the full abstract ▾
All authors Alibek Kamiluly, Milana Muratova, Yash Patel, Fan Li
Computer Vision · Cross-listed from cs.RO
ForceU-VLA: A Force-Aware Vision-Language-Action Model for Embodied Ultrasound Scanning
Xingzheng Wu, Cheng Zhang, Guihao Yan et al. (+3 more)
Embodied intelligent ultrasound scanning enables the automation and standardization of the ultrasound examination process by integrating perception, decision-making, and execution capabilities. However, existing methods suffer from loosely coupled modeling between force and ultrasound modalities and lack awareness of scanning stages, which limits their ability to capture dynamic probe-tissue interactions. To address these issues, we propose ForceU-VLA, a force-aware
Vision-
Language-Action model for autonomous
embodied ultrasound scanning, which leverages force signals and ultrasound image feedback throughout the scanning process to enable accurate and high-quality ultrasound acquisition. Firstly, we propose a Force-Ultrasound Synergistic Fusion Module (FUSFM) that synergistically fuses ultrasound visual and force-feedback information to provide stable, reliable guidance for probe motion. Secondly, a Stage-Adaptive Modulation Mechanism (SAMM) is proposed to accommodate the task requirements across different scanning stages by adaptively modulating
multimodal features to enhance their representation quality. Additionally, we introduce ForceU-VLA-Data, a real-world, force-aware
embodied ultrasound
dataset that integrates visual, force, and action signals, including data from two organs across five representative clinical scanning views, and comprising 450 expert-collected trajectories with approximately 100,000 synchronized
multimodal frames. Extensive experimental results demonstrate that ForceU-VLA significantly improves contact stability and probe pressure regulation in
embodied ultrasound scanning, thereby effectively enhancing task execution quality and overall system reliability. The source code is available at https://github.com/VMVLab/ForceU-VLA.
Read the full abstract ▾
All authors Xingzheng Wu, Cheng Zhang, Guihao Yan, Xifeng Hu, Zhi Liu, Qing Cai
Computer Vision
MetaReason: Precise Interleaved Multimodal Reasoning via Editing Meta Information for Solving Geometry Problems
Penghao Yin, Haomin Wang, Qihong Tang et al. (+3 more)
Although visual
reasoning is crucial for solving complex geometry tasks, existing
vision-
language models rely heavily on text-only
reasoning. Some recent methods introduce intermediate visual states to facilitate
reasoning, but they are often hindered by inaccurate
geometric representations and low rendering fidelity, ultimately leading to unreliable outputs. To address these limitations, we propose MetaReason, a framework for
multimodal reasoning in plane geometry that leverages structured meta-information to enable accurate auxiliary-line construction. The framework first parses
geometric images into meta-information, performs controllable edits with predefined tools to synthesize high-fidelity visual states, and then conducts
reasoning based on these augmented views. To support this framework, we construct TutorGeo, a comprehensive
dataset containing 17k image-to-meta conversion samples, 60k text-only
reasoning traces, and 60k interleaved
multimodal reasoning traces. Using this
dataset, we combine supervised fine-tuning and reinforcement learning to develop robust
multimodal reasoning capabilities. We also introduce ExamGeo, a
benchmark derived from real-world examination problems that enables systematic evaluation across varying difficulty levels. Experimental results demonstrate that MetaReason significantly outperforms existing open-source models and achieves competitive performance against proprietary models.
Read the full abstract ▾
All authors Penghao Yin, Haomin Wang, Qihong Tang, Xiaoye Qu, Hongjie Zhang, Xiao-Ping Zhang
Computer Vision
FZ-VLM: A Two Stage Florence-Zephyr Vision Language Model Framework for Pulmonary Nodule Characterization and Clinical Decision Making
Pramit Dutta, Jenita Manokaran, Richa Mittal et al. (+2 more)
Lung cancer remains one of the leading causes of cancer-related mortality worldwide, and Computed
Tomography (CT) is a primary imaging tool for screening and followup assessment. After pulmonary nodule detection, radiologists manually assess anatomical location, diameter, margin characteristics, and attenuation type to support risk assessment and clinical decision-making. However, this post-detection workflow is time-consuming and can be affected by inter-observer variability. Existing Artificial Intelligence methods often focus on isolated tasks, limiting their use as a unified, clinically grounded interpretation framework. This study presents FZ-VLM, a two-stage Florence-Zephyr
Vision Language Model framework for unified structured pulmonary nodule characterization in lung CT. The framework uses a fine-tuned Florence-2 model to extract radiological attributes from expert-annotated 2D axial CT slices, while a Zephyr-7B model uses these attributes to generate nodule descriptions, follow-up recommendations, and longitudinal analyses. Results showed that the Stage 1 model achieved 77.18% accuracy for anatomical location, 67.96% accuracy for margin characteristics, and 79.13% accuracy for attenuation type, with a Mean Absolute Error of 2.58 mm for diameter estimation, outperforming evaluated GPT-4-based baselines as well as the human baseline. Expert radiologist evaluation of Stage 2 showed 93.9% accuracy, 98.6% completeness score, 76.1% clinical relevance, and an overall score of 89.5%. Safety analysis showed that most outputs were clinically safe, although some follow-up recommendations still required expert review. To the best of our knowledge, this study presents the first two-stage
Vision-
Language Model framework for structured nodule characterization and clinical decision-making.
Read the full abstract ▾
All authors Pramit Dutta, Jenita Manokaran, Richa Mittal, Ryan Appleby, Eranga Ukwatta
Computer Vision
Registration-Free Hyperspectral Reconstruction from RGB via a Permutation-Invariant Gram-Matrix Principle
Jiangsan Zhao, Masayuki Hirafuji, Seishi Ninomiya et al. (+2 more)
Reconstructing a spatially and spectrally high-resolution hyperspectral image (HR-HSI) from a low-resolution HSI (LR-HSI) and a high-resolution RGB image (HR-RGB) usually assumes precise registration and a known camera response function (CRF). Both assumptions are difficult to satisfy with different sensors. We remove both through a permutation-invariant supervision principle: the Gram
matrix of an unmixed abundance map depends on shared material composition but not on pixel ordering. Matching abundance Gram
matrices therefore allows RGB-to-HSI mapping to be learned without spatial correspondence and without a predefined CRF. Under a full random permutation of HR-RGB pixels, a state-of-the-art fusion method collapses, whereas our reconstruction is unchanged after inverse reindexing for evaluation. Building on this principle, a residual spectral super-resolution function maps HR-RGB directly to HR-HSI without registration, known CRF, or paired supervision. Across indoor, natural-scene, and remote-sensing
benchmarks, the method achieves accuracy comparable to approaches that require these assumptions while remaining robust when they are violated. Loss ablations further show that reconstruction accuracy is largely insensitive to the specific discrepancy used to match the Gram
matrices, indicating that performance arises primarily from the permutation-invariant principle rather than loss tuning.
Read the full abstract ▾
All authors Jiangsan Zhao, Masayuki Hirafuji, Seishi Ninomiya, Jakob Geipel, Wei Guo
Computer Vision
Risk-Adaptive Edge–Cloud Visual Reasoning for Communication-Efficient Autonomous Driving
Meng Ma, Shuyang Li, Naigang Wang et al. (+1 more)
Cloud-hosted
vision-
language models (VLMs) offer greater contextual
reasoning capabilities than smaller onboard models, but frequent visual uploads increase communication overhead and add network and inference latency to tactical decisions. We present a risk-adaptive edge-cloud architecture in which onboard traffic assessment determines when cloud
reasoning is requested. An onboard VLM and a lightweight detector capture temporal traffic conditions and path-relative hazards for conservative local response and selective cloud access. The cloud model provides tactical advice, while validation, vehicle control, and automatic emergency braking remain local. In CARLA experiments, our method matched the task success rate of periodic cloud access while reducing cloud requests by 54.1% and recording fewer automatic emergency braking (AEB) activations. In a delayed-roadwork ablation, semantic events triggered requests before the next scheduled audit. Across three emulated network profiles, the method continued to reduce cloud traffic, although lane changes took longer than with periodic access. Onboard traffic assessment therefore served as a practical trigger for selective VLM inference in these experiments.
Read the full abstract ▾
All authors Meng Ma, Shuyang Li, Naigang Wang, Ruimin Ke
Computer Vision
Benchmarking Frontier Text-to-Image Models on Image-Description Prompts
Sajjad Abdoli, Ghassan Al-Sumaidaee and Ahmed Rashad
Text-to-image models are typically reported on average-case
prompts, which understates the gap between systems on compositionally demanding requests involving precise object counts, multi-object attribute binding, legible embedded text, and explicit spatial constraints. We evaluate four production text-to-image systems: Hunyuan 3.0, Gemini 3 Pro Image ("Nano Banana Pro"), Black Forest Labs FLUX.2, and Ideogram 3.0. The evaluation uses the 48 hardest
prompts drawn from the DataSeeds.AI Sample
Dataset (DSD), selected through an automated complexity-scoring pass over the full corpus. Every generated image is graded using an independent-judge rubric. GPT-5.4-Pro authors an atomic, weighted, mutually exclusive and collectively exhaustive (MECE) evaluation rubric, while Gemini 3.1 Pro Preview independently determines whether each criterion is satisfied. Gemini 3 Pro Image ranks first with a score of 84.8/100, narrowly ahead of FLUX.2 at 82.3/100. Ideogram 3.0 and Hunyuan 3.0 score 65.7/100 and 63.3/100, respectively. Failure analysis shows that the leading systems primarily lose points through object miscounting and
geometric artifacts, whereas the trailing systems more frequently produce garbled text. Ideogram 3.0 also frequently omits requested elements. Full per-sample rubrics, scores, and failure annotations are available from the authors upon request.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Evidence of Absence: Cross-Modal Abductive Risk Perception to Sustain World Models When Vision Fails
Cong Xu and Ravi Sankar
A structured world-state (entities, relations, context, and predictive cues) is designed to preserve prediction-critical content when perception degrades, but it presumes observations to populate it; when the primary visual modality is occluded or degraded, those observations may be missing. We address how to sustain the world model from a complementary modality by treating the absence of expected co-evidence as evidence of a hidden cause. The abductive framework is modality-agnostic; this article instantiates it acoustically. A microphone-array front-end estimates the bearing of engine and tire sources and extracts approach-rate evidence (Doppler when a stable tone exists, a broadband looming readout otherwise); the event "signature present, visual co-evidence absent" then triggers abductive inference of a hidden road user, emitting a calibrated risk advisory rather than a control command. Recoverability of the hidden state is analyzed as an identifiability question separating shared from modality-unique information, and cueing is cast as Neyman-Pearson detection under an explicit false-alarm budget. On real occluded-approach recordings at blind junctions, the method warns a mean 1.7 seconds before line-of-sight entry, matches the sustained-window variant of the published acoustic baseline's detection rate with 42% fewer false alarms, localizes to 3.4 degrees median once in view, is well calibrated (expected calibration error 0.034), and keeps hazard awareness above 0.87 under staged
vision degradation that collapses a
vision-only channel to 0.03. We also measure the method's limits: calibration transfers to an unseen junction almost losslessly, the signature classifier does not, and moving-ego noise is the binding deployment constraint.
Read the full abstract ▾