Computer Vision
Pre-Decoding Acoustic Triage for Budgeted Vision-Language Captioning of Untrimmed Egocentric Video
Masoud Jalayer, Changyi Li and Yu Xiao
Automatically analyzing hours-long
egocentric video is increasingly essential for progress monitoring, quality control, and safety in logistics, construction, and manufacturing. Yet current pipelines that process short, fixed-size windows with a
vision-
language model (VLM) are prohibitively expensive because cost scales with the number of model calls. To reduce this cost, prior work proposes triage policies to select which windows merit a VLM invocation. However, these policies either sample uniformly or rank windows using visual features, which ironically requires the
video decoding that the budget constraints are meant to avoid. We propose audio-first triage: select windows using the lightest modality, scored before any
video frame is decoded, so the approach composes naturally with
token compression or quantization. The novelty lies in the objective, not the representation: rather than a per-frame sound-event detector, we train the selector to trigger once per action. This objective shift improves action coverage by 4.0-10.8 percentage points across all evaluated call rates, using frozen AudioSet-pretrained features without domain-specific sound-event labels. Using fewer than half of the available calls, the triage cuts 9-20% of VLM calls at matched coverage on EPIC-KITCHENS-100 (EK-100), surpasses uniform sampling through the mid-range on Ego4D over 247 clips, and outperforms two recent visual keyframe selectors. Code, the reference implementation and every results file this manuscript reads are at https://github.com/masjalayer/PreDecoding-AcousticTriage.
Read the full abstract ▾
Computer Vision
Multimodal examination answer data with expert-designed Outcome-Based Education rubrics for criterion-level assessment
Jahangir Alam SM, Md Khalid Syfullah, Saad Ahmed et al. (+4 more)
This data article describes a
multimodal collection of scanned examination answers paired with expert-designed Outcome-Based Education (OBE) grading metadata. The collection contains 485 answer submissions from 415 consenting students at four academic institutions. Eight faculty contributors supplied examination materials covering nine subjects and 12 distinct question templates. Each answer-level item links a scanned PDF to a randomized identifier, subject label, question, model answer, criterion definitions, performance-level descriptions, criterion marks, and a total mark. The 12 rubrics contain 47 criteria in total. The scans retain realistic academic content, including handwriting, printed text, equations, tables, code, figures, sketches, and diagrams. CamScanner, Adobe Scan, and conventional scanners contributed variation in illumination, contrast, orientation, compression, and resolution. Diverse handwriting, crossed-out work, revised calculations, and inserted corrections add further visual variability for robustness and generalization studies. Preparation involved heterogeneous-source consolidation, label and text standardization, score validation, identifier randomization, filename randomization, and JSON-to-PDF integrity checks. An answer-level audit confirmed 485 unique identifiers, 485 unique PDF filenames, agreement between each total mark and its criterion-mark sum, and scores within the applicable rubric maximum. The data can support rubric-aware automated evaluation,
multimodal document understanding, criterion-level feedback, score prediction, and
privacy-aware OBE assessment research. Access is restricted to research use and is available from the corresponding author upon reasonable request.
Read the full abstract ▾
All authors Jahangir Alam SM, Md Khalid Syfullah, Saad Ahmed, Munira Akter Mou, A K Z Rasel Rahman, A.K.M. Masudur Rahman, Mohammed Sowket Ali
Computer Vision
Fast and Compact 3D Gaussian Splatting with Polarized Opacity Prior
Zi-Ming Wang, Kai-Wen Duan, Kowei Huang et al. (+2 more)
3D Gaussian Splatting (3DGS) achieves state-of-the-art rendering quality at real-time speeds but suffers from "model bloat" - a large number of redundant, low-opacity Gaussians that inflate
memory usage and training costs. This inefficiency stems from the standard "densify-then-prune" paradigm, which expands the model aggressively before relying on
pruning to achieve compactness. To mitigate this problem, we present an efficient training framework that builds an intrinsically compact representation, replacing the conventional densify-then-prune cycle. Our method leverages a synergistic design: an L2 reconstruction loss to provide error-proportional gradients that stabilize optimization, and a novel Polarized Opacity Prior (POP) to actively manage the Gaussian population. POP steers informative primitives toward full opacity and uninformative ones toward transparency, enabling natural
pruning and accelerating rendering through Early Ray Termination. Experiments on three public
datasets demonstrate that our approach consistently achieves accelerated 3DGS training with significantly fewer Gaussians while maintaining comparable visual reconstruction quality. These results show that the proposed framework provides a simple and effective path toward fast and inherently compact 3DGS training.
Read the full abstract ▾
All authors Zi-Ming Wang, Kai-Wen Duan, Kowei Huang, Akihiro Sugimoto, Shang-Hong Lai
Computer Vision
TransHands: Repurposing Human Pose Encoders as Hand Pose Encoders
Milo Piccioli, Gianluca Amprimo, Claudia Ferraris et al. (+1 more)
Lifting 3D hand poses from 2D
monocular representations remains challenging due to the limited availability of large-scale, diverse 3D-annotated hand
datasets, in contrast to the abundance of human body motion data. We address this limitation by transferring motion representations learned from large body pose corpora to the hand domain. We introduce TransHands, a backbone-agnostic transfer learning framework that enables pre-trained human motion encoders to be effectively adapted for 3D hand pose estimation from 2D pose inputs. Rather than training hand-specific biomechanical models from scratch, TransHands combines a two-stage training and fine-tuning strategy with a lightweight hand-specific input adaptation module that aligns hand kinematics with the representation space learned for full-body motion. We evaluate TransHands across four state-of-the-art motion modeling architectures, including
transformer-based, graph-based, and frequency- domain models. Results demonstrate that motion priors learned from body pose data transfer consistently across architectures, yielding consistent accuracy gains, strong cross-domain generalization, particularly in challenging
egocentric settings, and applicability for downstream tasks in real-world contexts.
Read the full abstract ▾
All authors Milo Piccioli, Gianluca Amprimo, Claudia Ferraris, Gabriella Olmo
Computer Vision
AcroMELD: Recovering Interactive PDF Forms with Structure-Aware Graph Set Transformers
Samuel Abramov
Interactive PDF form fields are often absent from documents that visually resemble forms, leaving users unable to enter data without printing or external editing tools. Detecting the missing widgets is difficult because a field may be indicated by several overlapping cues, born-digital PDFs expose useful but incomplete drawing structure, and dense pages can contain hundreds of fields. We introduce AcroMELD (AcroForm Multi-source Evidence Linking Decoder), a 39.4M-parameter detector that combines a high-resolution visual
transformer with label-free PDF primitives. Its 896-query set comprises 384 visual proposals, 384 structure-seeded proposals, and 128 learned recovery queries. Four graph-set layers exchange information over geometry-biased sparse neighborhoods and cross-attend to PDF structure. A learned same-field relation links co-referent candidates, while a localization-quality head is trained on the containment-aware overlap used by the downstream recovery decision. We define a hash-bound evaluation protocol with disjoint development, calibration, internal-test, and quarantined external-holdout roles. The sealed, single-seed candidate reaches native containment micro-$F_1$ 0.9344 on the internal test and 0.8477 on the one-shot external holdout (95% PDF-cluster bootstrap interval [0.8339, 0.8605]). This passes the registered historical FFGBT-v8 reference by 0.0186 absolute $F_1$. Under the stricter external adapter, however, performance is 0.7786 IoU-$0.5$ $F_1$ and 0.2900 COCO mAP, below a locally evaluated CommonForms-L reference; the signature class receives no prediction at the selected threshold. Thus the result supports the registered operational gate while exposing substantial domain and rare-class limitations.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.MM
Motion-Aware Reasoning from Speech to Mask Tracks: Runner-up Solution for the MeViS-Audio Track of the 8th LSVOS Challenge 2026
Jinxing Zhou, Suiyi Zhao, Yanghao Zhou et al. (+1 more)
Speech-guided referring
video object
segmentation aims to recover the mask tracks of objects specified by a spoken motion description. Here,
speech carries a linguistic instruction rather than acoustic evidence from a sounding object, so a solution must connect
speech recognition, motion-centric temporal grounding, mask tracking, and explicit no-target handling. We introduce
Speech2MaskTrack, our approach for the MeViS-Audio track of the 8th LSVOS Challenge.
Speech2MaskTrack transcribes the spoken query and compiles it into structured constraints over category, count, direction, interaction role, and temporal phase. SAM3.1 enumerates multiple instance tracks, which TRACE ranks using complete-trajectory motion and relation evidence. A frozen lexical presence gate may suppress the ranked SAM3.1 base prediction. When the gate predicts that a target is present, an available full-expression-conditioned SaSaSa2VA track replaces the SAM3.1 mask. Only outputs that remain empty enter GPT-assisted recovery, which invokes SaSaSa2VA again under query- and mask-level verification.
Speech2MaskTrack achieved second place in the official challenge ranking.
Read the full abstract ▾
All authors Jinxing Zhou, Suiyi Zhao, Yanghao Zhou, Ruohao Guo
Computer Vision · Cross-listed from cs.MM
ReART: Reference-Guided Retrieval and Refinement for Emotion-Aware Art Generation
Qianqian Tang, Jiayi Gao, Ting Lei et al. (+1 more)
Emotion-aware artistic image generation requires a model to satisfy semantic content, artistic style, and target
emotion simultaneously. The key challenge is that artistic captions conflate these axes into underspecified free-form text, making fine-grained visual attributes such as brushwork, composition, and tonal atmosphere difficult to ground concretely. We present ReART, a reference-guided
retrieval and refinement framework. Our method decomposes test captions and each image annotation in the EmoArt database into structured visual fields, and performs field-wise
retrieval over subject, layout, brush-line, and tone-mood dimensions to retrieve role-specific visual references that supply the perceptual detail text alone cannot convey; these references are used alongside a structured
prompt for initial synthesis. For samples where any Attribute
Alignment Score (AAS) axis falls below threshold, an AAS-driven refinement loop diagnoses failures, constructs constrained repair plans specifying elements to keep, errors to fix, and operations to avoid, routes references by correction purpose, and performs controlled editing under structural preservation constraints. Our system ranks 2nd in Track 1 of the AffectiveArt 2026 Grand Challenge, achieving a perfect AAS of 1.00 and an overall score of 0.78. Code is available at https://github.com/oceanflowlab/ReART.git.
Read the full abstract ▾
All authors Qianqian Tang, Jiayi Gao, Ting Lei, Yang Liu
Computer Vision
MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis
Lai Wei, Yuchao Chen, Zhenbiao Cao et al. (+5 more)
The application of Large
Language Models (LLMs) to diagnostic decision-making has garnered growing interest. However, existing
benchmarks largely focus on textual
reasoning or isolated visual question-answering (VQA) tasks, lacking holistic integration of clinical narratives and medical imaging, and thus failing to assess the
multimodal diagnostic synthesis capability central to expert clinical judgment. To bridge this gap, we introduce MedReaMM, a
benchmark specifically designed to evaluate models' ability to synthesize heterogeneous clinical evidence consisting of detailed patient histories alongside multiple medical images into accurate differential diagnoses under a complete-information paradigm. Constructed from case reports sourced from top-tier medical journals and curated clinical case databases, MedReaMM comprises 625 expert-validated cases with an average of 2.79 medical images per case and a total of 1,042 standardized diagnoses annotated with ICD-11 codes. These cases predominantly represent rare, atypical, or multi-system presentations that demand expert-level evidence integration beyond routine pattern recognition. We evaluate 23 Large
Multimodal Models (LMMs) and find that most achieve diagnostic accuracy scores below 50%, underscoring a substantial gap in
multimodal diagnostic synthesis capability. Further analysis reveals that medical knowledge proficiency, medical image understanding, and evidence integration are all highly correlated with diagnostic performance.
Read the full abstract ▾
All authors Lai Wei, Yuchao Chen, Zhenbiao Cao, Xiaojin Zhang, Zhongyu Wei, Bangting Wang, Wei Chen, Xiang Bai
Computer Vision · Cross-listed from cs.LG
Does a Modern-Handwriting Warm-Up Help Historical Arabic OCR? A Reproducible, Compute-Matched Evaluation on Muharaf and KHATT
Sumaih Almarshad, Maram Alamri, Dona Aloraini et al. (+4 more)
Whether an intermediate stage of modern Arabic handwriting helps or hurts historical Arabic HTR is usually decided from one implementation and one comparison, too thin a basis for a claim either way. We test stability by running the same nominal ablation four times, letting the base checkpoint, encoder-freezing strategy, epoch budget, precision, and learning-rate schedule vary as they naturally did during development, while holding the normalization, scorer, and interval estimation fixed. Each run compares intermediate training on modern handwriting (KHATT) then fine-tuning on historical manuscripts (Muharaf) against fine-tuning on Muharaf directly. Across the four runs the estimated effect swings from -17.64 to +14.52 CER points and reverses sign. The two extremes are exactly the two runs with an identifiable confound (a fivefold lower learning rate in one; a checkpoint of undisclosed provenance in the other); the two clean runs land at -0.25 and +0.94, i.e. no effect. A tight interval from one implementation says nothing about the next. We then run a compute-matched experiment with identical budgets over three seeds: KHATT warm-up is +2.42 CER points worse than a matched same-domain control (95% interval [+0.60, +4.25]); the part of that gap specific to the handwriting domain is only about 0.6 points a small negative effect under this configuration, not a universal result. We release a SaudiHeritage-OCR package with the normalizer, interval scorer, a verified KHATT decoder, experimental manifests, VLM baselines, and an edition-
alignment protocol, so the result can be checked independently. The Al-Mahd inscription line is held strictly out and is not offered as a
benchmark.
Read the full abstract ▾
All authors Sumaih Almarshad, Maram Alamri, Dona Aloraini, Fares Altuwaim, AlJawharh AlOtaibi, Reem Alyabis, Rayah Aldawsari
Computer Vision
On the Choice of Tensor Estimation for Corner Detection, Optical Flow and Denoising
Freddie Åström and Michael Felsberg
Many image processing methods such as corner detection, optical flow and iterative enhancement make use of image tensors. Generally, these tensors are estimated using the structure tensor. In this work we show that the gradient energy tensor can be used as an alternative to the structure tensor in several cases. We apply the gradient energy tensor to common image problem applications such as corner detection, optical flow and image enhancement. Our experimental results suggest that the gradient energy tensor enables real-time tensor-based image enhancement using the graphical processing unit (GPU) and we obtain 40% increase of frame rate without loss of image quality.
Read the full abstract ▾
Computer Vision
Adapting Dense Vision-Language Relationships for Multi-label Classification with Partial Label
Cheng Chen, Yifan Zhao and Jia Li
Learning multi-label image classification with incomplete annotations is a challenging task that has been widely studied for its superior trade-off between high efficiency and less labor consumption on large-scale
datasets. Predominant methods rely on strong prior assumptions to recover the missing semantics from partial annotations. However, these statistic priors suffer from unstable semantic mistakes and thus lead to catastrophic overfitting. Toward this end, we propose a
Language-driven Dense Semantic Adaptor (LDSA) that excavates prior-adaptive relationships from
multimodal pretrained CLIP models. In our approach, the densely contrastive adaptor is first proposed to construct dense visual contrastive constraints, transferring the task-specific knowledge to visual domains. We then propose a
language-driven interactive decoder with the help of class-specific
prompt tuning, which adapts
language proxies with visual domains. With the collaborative learning of proposed modules, experimental results demonstrate our proposed LDSA achieves a new state of the art on public multi-label classification
benchmarks, and interpretable analyses reveal that our LDSA discovers implicit semantic relationships with the prior-adaptive learning scheme.
Read the full abstract ▾
Computer Vision
On Tensor-Based PDEs and their Corresponding Variational Formulations with Application to Color Image Denoising
Freddie Åström, George Baravdish and Michael Felsberg
The case when a partial differential equation (PDE) can be considered as an Euler-Lagrange (E-L) equation of an energy functional, consisting of a data term and a smoothness term is investigated. We show the necessary conditions for a PDE to be the E-L equation for a corresponding functional. This energy functional is applied to a color image denoising problem and it is shown that the method compares favorably to current state-of-the-art color image denoising techniques.
Read the full abstract ▾
Computer Vision
Self-Calibrating Dense Displacement Fields for Reliable Co-Registration of Large Optical Satellite Imagery
Shoukun Sun, Zhe Wang, Sanaz Salati et al. (+3 more)
Co-registration underlies nearly every multi-temporal and multi-sensor use of optical satellite imagery, and operational products still carry documented offsets well above the fraction-of-a-pixel scale at which change detection, time series, and data fusion degrade. Real image pairs differ along several axes at once (sensor response, scene content, viewing geometry, resolution, mosaic seams), and the last of these is not a single global motion. Existing tools embed a motion model and constants tuned to their development data; a pair that fits is registered precisely, while one that does not either fails to match or returns a result wrong by tens of pixels with no failure reported. Learned matchers add a GPU requirement and carry no accuracy guarantee outside their training distribution. We present SCDF (self-calibrating displacement fields), a training-free, GPU-free
estimator whose motion model is the dense per-pixel displacement field itself, so no scene motion falls outside the model. A single predict–measure–filter loop runs over a resolution pyramid: the accumulated field predicts where each patch of the moving image falls in the reference, RootSIFT matching and a correlation pass measure the displacement there to sub-pixel precision, and filters whose thresholds are all calibrated on the image pair itself decide what survives. One configuration, with no per-
dataset tuning, processes full $8192^2$ scenes on a single CPU core. On 584 constructed-ground-truth pairs built from real Sentinel-2, Landsat-8/9, and NAIP imagery, against seven classical baselines and two zero-shot pretrained matchers, SCDF registers every pair with zero failures, reduces the best baseline's real-pair median end-point error from 6.83 to 4.17m, and cuts its 90th percentile from 17.8 to 7.77m.
Read the full abstract ▾
All authors Shoukun Sun, Zhe Wang, Sanaz Salati, Jiyin Zhang, Hui Wang, Xiaogang Ma
Computer Vision
Targeted Iterative Filtering
Freddie Åström, Michael Felsberg, George Baravdish et al. (+1 more)
The assessment of image denoising results depends on the respective application area, i.e. image compression, still-image acquisition, and medical images require entirely different behavior of the applied denoising method. In this paper we propose a novel, nonlinear
diffusion scheme that is derived from a linear
diffusion process in a value space determined by the application. We show that application-driven linear
diffusion in the transformed space compares favorably with existing nonlinear
diffusion techniques.
Read the full abstract ▾
All authors Freddie Åström, Michael Felsberg, George Baravdish, Claes Lundström
Computer Vision · Cross-listed from cs.RO
TONAV: Task-Oriented Navigation and Action-Velocity Chunk Learning for Articulated Object Quadrupedal Mobile Manipulation
Haoran Lin, Mingyu Yang, Pengfei Qi et al. (+6 more)
Quadruped mobile manipulation requires two tightly coupled capabilities: reaching manipulation-ready configurations and maintaining stable contact throughout articulated-object interaction. However, existing methods often terminate
navigation near the target, leaving a gap between reachability and manipulation readiness, while tracking lag, motion jitter, and contact instability limit continuous interaction. To address these challenges, we present TONAV, a unified framework integrating task-oriented
navigation with action-velocity chunk learning. First, we introduce a position-velocity-coupled teleoperation framework that explicitly captures motion dynamics to improve master-follower consistency and collect smooth, temporally consistent demonstrations. Next, task-oriented
navigation leverages
vision-
language reasoning to decompose high-level instructions into executable subgoals and adaptively refine the
robot base toward a manipulation-ready configuration. Finally, action-velocity chunk learning jointly models joint positions and their temporal transitions under velocity supervision, enabling smooth and stable sustained-contact manipulation. Real-world experiments across diverse articulated-object tasks demonstrate that TONAV achieves higher success rates in both task-oriented
navigation and complete mobile manipulation, mitigating the
navigation-manipulation gap and improving continuous-contact interaction. The project page is at https://haochen611.github.io/TONAV.
Read the full abstract ▾
All authors Haoran Lin, Mingyu Yang, Pengfei Qi, Kehan Chen, Qiang Diao, Liangji Zeng, Wenrui Chen, Yaonan Wang, Kailun Yang
Computer Vision
DECO: Depth-Guided Co-Visibility Reasoning for Low-Altitude UAV Visual Localization
Yibin Ye, Xichao Teng, Shuo Chen et al. (+4 more)
Unmanned aerial vehicles (
UAVs) increasingly require robust visual localization in GNSS-denied environments. A common solution estimates
UAV poses by matching keypoints between
UAV images and geo-tagged orthographic reference maps derived from satellite or aerial imagery, followed by Perspective-\(n\)-Point (PnP) pose solving. However, such reference maps mainly record top-down surfaces such as roofs and ground planes, while vertical structures such as facades and walls are often compressed or missing. Consequently, many visually distinctive keypoints in low-altitude
UAV images have no valid counterparts in the reference map, leading to redundant matches and inaccurate pose estimation. To address this issue, we propose DECO, a DEpth-guided CO-visibility
reasoning framework for low-altitude
UAV visual localization. DECO uses
monocular depth priors to infer local surface geometry and estimate co-visible regions between
UAV images and the reference map. Based on this prior, a Geometry-Saliency Coupled Co-visibility Score is introduced to jointly consider
geometric co-visibility and detector saliency for keypoint ranking. In this way, DECO retains keypoints that are both visually distinctive and geometrically co-visible, improving feature matching and PnP-based pose estimation. Extensive experiments demonstrate that DECO achieves superior localization performance and can be integrated with different depth models, feature detectors, and matchers. The source code will be available at https://github.com/
UAV-AVL/DECO.
Read the full abstract ▾
All authors Yibin Ye, Xichao Teng, Shuo Chen, Xiaokai Song, Dongdong Guan, Qifeng Yu, Zhang Li
Computer Vision · Cross-listed from eess.IV
CiUNet: A Hybrid Swin-CNN UNet for Medical Image Segmentation
Bin Dong and Jinghong Chen
Medical image
segmentation requires high accuracy and robustness, yet practical commercial deployment also demands
privacy preservation and computational efficiency. In this context, the U-Net architecture, which can be inherently decoupled into independent encoder and decoder components, serves as a natural commercial choice. However, pure
Transformer-based variants like Swin-UNet often suffer from insufficient local detail capture and limited interpretability. In this paper, we propose a lightweight hybrid architecture built upon the Swin-UNet framework. Our model integrates a parallel CNN encoder to complement the shallow layer
reasoning of Swin
Transformers with local texture features. To bridge the semantic gap and enhance fine-grained spatial detail recovery, we design an asymmetric feature fusion strategy and introduce cross-layer skip (XSkip) connections that explicitly propagate shallow CNN features into the decoder. We further incorporate novel loss functions and an auxiliary supervision head (Aux-Head) to strengthen training stability, boundary delineation, and intermediate feature interpretability. Extensive experiments on the Synapse multi-organ
segmentation dataset demonstrate that our approach achieves state-of-the-art competitive Dice scores and Hausdorff distances, offering an accurate, efficient, and interpretable solution for clinical deployment.
Read the full abstract ▾
Computer Vision
OVIBench: Benchmarking Online Video Question Answering under Interruption
Naiming Liu, Zhiheng Wu, Shuning Wang et al. (+3 more)
Recent
vision language models (VLMs) have achieved strong progress in
video understanding. However, most existing
video QA research and
benchmarks still follow an offline, single-round paradigm, overlooking realistic interactions where users may interrupt the model during answer generation. To address this gap, we formulate the task of Online
Video Question Answering under Interruption and introduce OVIBench, the first standardized
benchmark for evaluating VLMs in this setting. OVIBench categorizes interruptions into three types: Cancellation, False Trigger, Correction and supports both open-ended and multiple-choice evaluations. To enable large-scale and reproducible testing, we develop an offline simulation protocol that reproduces interruption during generation under a unified temporal setup, together with a multi-dimensional metric suite for assessing interruption understanding and response generation. Experiments demonstrate that OVIBench effectively distinguishes models' interruption-handling abilities, especially in following correction requests. Finally, we construct a train set OVI-Train for interruption-aware fine-tuning. Models fine-tuned on this
dataset achieve significant gains on OVIBench, validating the effectiveness of our
benchmark and data design. OVIBench, OVI-Train, and the evaluation code will be released.
Read the full abstract ▾
All authors Naiming Liu, Zhiheng Wu, Shuning Wang, Tie Zhang, Bowen Liu, Tong Wang
Computer Vision
GAN-Diff : Coupling Pretrained WGAN-GP Features with Conditional Diffusion U-Nets
Saif Ahmed, Asadullah Hil Galib, S.M. Riaz Rahman Antu et al. (+5 more)
Generative
adversarial networks (GANs) can provide efficient image generation, while
diffusion models offer high-quality image restoration but require iterative sampling. This paper presents a hybrid GAN-guided
diffusion framework that uses a pretrained Wasserstein GAN with gradient penalty (WGAN-GP) as a feature prior for conditional
diffusion-based image restoration. Intermediate features from the frozen WGAN-GP generator are incorporated into a
diffusion U-Net through cross-
attention and remain fixed during the DDIM sampling process. The framework is evaluated on two restoration tasks, Gaussian denoising and 2Xsuper-resolution, using CelebA face images. During development, several sources of instability were identified and addressed, including
adversarial learning-rate imbalance, inappropriate
diffusion initialization, excessive corruption, and insufficient parameter averaging. The resulting framework consistently improves the quality of both degraded and low-resolution images. In particular, it improves denoising performance by 4.40 dB in PSNR and super-resolution performance by 3.70 dB over their respective input baselines. These results demonstrate the potential of a frozen GAN feature prior to guide
diffusion models toward stable and effective image restoration.
Read the full abstract ▾
All authors Saif Ahmed, Asadullah Hil Galib, S.M. Riaz Rahman Antu, Ahmed Faizul Haque Dhrubo, Souvik Pramanik, Mohammad Abdul Qayum, Mohsin Sajjad, Mohammad Ashrafuzzaman Khan
Computer Vision
Training-Free VLM Personalization via Calibrated Residual Decoding
Jiaao Yu, Yujian Ma, Xianming Hu et al. (+2 more)
Vision-
language models can be personalized in a training-free manner by directly providing user profiles, preferences, or visual references at inference time, without updating model parameters. However, direct personalized prompting does not guarantee that the model will reliably exploit such evidence. The predictive distribution under the positive user profile often mixes two sources: personalized signals genuinely supported by the current profile, and the model's generic visual or linguistic priors. As a result, from the positive-profile response alone, it is difficult to determine whether a high-confidence answer is supported by the user profile or merely reflects the model's default preference. To address this problem, we propose a training-free calibrated residual decoding framework. Given the same image and question, we construct three evidence conditions: a positive profile , a counterfactual profile , and an empty profile . Our method keeps the prediction under as the anchored base, and explicitly estimates the marginal contribution of personalization from score differences across the three conditions. We further introduce normalized-
entropy-based uncertainty calibration, allowing the strength of personalized enhancement to adapt to the reliability of the residual signal. Experiments on MMPB, YoLLaVA, and MyVLM show that the proposed method improves personalized
multimodal understanding without fine-tuning, with consistent gains on identity-sensitive visual personalization tasks. Additional analysis shows that
entropy calibration stabilizes residual decoding when the contrastive personalization signal is uncertain.
Read the full abstract ▾
All authors Jiaao Yu, Yujian Ma, Xianming Hu, Pengran Wang, Ang Li
Computer Vision
Hyper^2: Unleashing Hyperbolic Geometry's Full Potential via Dual-Space Consistency
Guantian Zheng, Haiyang Xu and Tianyu Gao
HyperbolicCD pioneered
hyperbolic geometry for point cloud completion by replacing the Euclidean Chamfer distance with arcosh(1+alpha||x-y||^2), but the reported gains are modest (3-7% Chamfer reduction across SeedFormer, PointAttN and PMP-Net backbones on PCN and ShapeNet-55). We argue the bottleneck lies elsewhere: the loss is
hyperbolic but the encoder it back-propagates through is Euclidean, so the position-dependent supervision of the loss is averaged away by the chain rule before it reaches the parameters. We call this a cross-geometry mismatch, and make it testable through two model-agnostic indicators, feature-loss correlation r_FL and effective gradient utilisation u_G. On an SVDFormer backbone trained with HyperbolicCD's loss alone we measure (r_FL, u_G) = (0.68, 39%). We propose Hyper^2, a dual-space consistency framework that extends HyperbolicCD by reusing the identical arcosh(1+alpha d^2) functional form as a positional bias on the refinement
attention (a
hyperbolic distance encoding), paired with HyperbolicCD's
hyperbolic Chamfer loss under a single shared
curvature alpha. Both operators are O(N log N) scalar non-linearities on Euclidean distances and together add only ~1.6% FLOPs over SVDFormer. Hyper^2 delivers -22.9% Chamfer on ShapeNet-55 over SVDFormer (well above the 13.2% linear sum of the -12.0% loss-only and -1.2% encoding-only single-space ablations) and -37.5% on the 21 unseen ShapeNet-34 categories. The two indicators remain essentially flat for any single-space configuration but jump together to (0.95, 87%) only when both encoder and loss are
hyperbolic, supporting the claim that
geometric consistency across encoder and loss, rather than either operator alone, is what enables
hyperbolic supervision in point cloud completion. Code is available at https://github.com/Ethan-Zheng136/Hyper-2.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
When Test-Time Adaptation Helps, Harms, or Becomes Inactive: A Condition-Level Study on CIFAR-10-C
Sreeja Guha Majumdar and Aratrika Saha
Test-time adaptation (TTA) aims to improve model robustness under distribution shift by adapting a source model using unlabeled test data. Although methods such as TENT and EATA have demonstrated gains on corrupted data, aggregate accuracy can obscure the conditions under which adaptation fails or provides little benefit. We present a controlled comparison of three TTA strategies—BatchNorm-statistics adaptation (BN-Adapt),
entropy-minimization adaptation (TENT), and reliability-filtered adaptation (a scoped re-implementation of EATA)—against an unadapted source model on the full CIFAR-10-C
benchmark, covering 15 corruption types and 5 severity levels. All three methods improve mean accuracy over the source model by 12.2–13.3 percentage points (Wilcoxon signed-rank $p < 10^{-12}$). However, each method underperforms the source model on 8.0–9.3% of conditions, with failures concentrated in low-severity corruptions where the source model already performs near ceiling, particularly brightness, fog, contrast, and defocus blur. We further find that EATA closely tracks the gradient-free BN-Adapt baseline, with a mean absolute difference of 0.09 percentage points, compared with 1.08 percentage points relative to TENT. This suggests that reliability filtering can substantially restrict effective adaptation, causing EATA to behave more like a BatchNorm-statistics baseline than an
entropy-minimization method. These results show that aggregate accuracy alone can mask systematic TTA failure modes and motivate condition-level evaluation of when adaptation helps, harms, or becomes effectively inactive.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Beyond What Meets the Eye: Unveiling Situational Illusions for Multimodal Large Language Models
Zhiming Yang, Zhuoxi Xiong, Donglin Zhou et al. (+3 more)
Real-world situation appearances can deviate from their underlying physical states, challenging the reliability of
multimodal large
language models (MLLMs) in practical applications. In this paper, we term this phenomenon situational illusions and investigate: (1) how MLLMs perform under such illusions, and (2) how to mitigate the limitations. We first develop a comprehensive where-what-how taxonomy that characterizes where situational illusions occur, what targets they take, and how they arise. Building on this taxonomy, we introduce MSIBench, a
benchmark designed to assess the discrimination, understanding, and
reasoning capabilities of MLLMs under situational illusions. Evaluations of 27 model configurations reveal that current MLLMs are highly vulnerable to these illusions and exhibit 6 typical failure modes related to visual observation, grounding, and
reasoning. To mitigate the limitations, we build on the core idea of systematically inspecting and
reasoning over visual evidence for contextual understanding, developing prompting for closed-source models and supervised fine-tuning for open-source models, respectively. These two simple yet effective methods improve model performances by 20% at most, suggesting a practical path toward more reliable
multimodal perception and
reasoning in complex real-world environments.
Read the full abstract ▾
All authors Zhiming Yang, Zhuoxi Xiong, Donglin Zhou, Wenjun Wei, Shiyao Cui, Jinqiao Shi
Computer Vision
UR$^{2}$-MLLM: Uncertainty-aware Revisit Reasoning in Multimodal Large Language Models for Radiology Report Generation
Yucheng Chen, Yang Yu, Jiazhou Zhou et al. (+5 more)
Radiologists generate diagnostic reports through iterative and selective revisiting of suspicious regions to refine their interpretations. Recent
multimodal large
language models (MLLMs) for radiology report generation (RRG) have shifted from text-only
reasoning toward a “Thinking-with-Images” paradigm, incorporating visual evidence into the
reasoning process. However, existing methods provide static visual evidence without a dynamic revisit mechanism during
reasoning, neglecting how radiologists re-examine uncertain observations. To this end, we propose an Uncertainty-aware Revisit
Reasoning MLLM (UR$^{2}$-MLLM) framework that dynamically revisits uncertain regions during
reasoning for RRG. UR$^{2}$-MLLM is first equipped with uncertainty perception by training on an uncertainty-aware
dataset. We then construct a
multimodal reasoning trajectory
dataset together with a detect-and-copy mechanism, which guides when and where to revisit. Finally, a visual grounding reward refines this behavior through reinforcement learning, aligning the revisited regions with corresponding anatomical structures. Experiments on MIMIC-CXR and IU-Xray show that UR$^{2}$-MLLM achieves state-of-the-art performance, highlighting the value of uncertainty-aware visual revisit
reasoning for reliable and clinically aligned report generation.
Read the full abstract ▾
All authors Yucheng Chen, Yang Yu, Jiazhou Zhou, Yufei Shi, Yongying Lan, Yichi Zhang, Liyi Li, Si Yong Yeo
Computer Vision
SAM3Dual: A 3rd Place Solution to the MOSEv2 Track, 8th LSVOS Challenge
JeongRae Kim, Chaehyun Kim and Changwon Lim
We present SAM3Dual, our third-place solution to the MOSEv2 track of the 8th Large-scale
Video Object
Segmentation (LSVOS) Challenge at ECCV 2026. SAM3Dual is a training-free inference extension of pretrained SAM 3 that explicitly separates temporal
memory into a short-term branch for recent observations and a long-term branch for interval-sampled historical representations. The two
memory responses are combined using a deterministic sequence-relative fusion schedule and conservatively modulated by the previous-frame object confidence. All pretrained SAM 3 parameters remain frozen, requiring no task-specific training, fine-tuning, test-time training, or online parameter optimization. The complete system achieved an official J&F score of 64.37 and ranked third in the MOSEv2 track. This result highlights the potential of reorganizing temporal
memory entirely at inference time to obtain competitive long-term VOS performance while preserving the pretrained model.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
BehaviorWorldGen: Closing the Loop between Action Models and World Simulators via Controllable Behavior-Aware Structured World Generation
Jiaqi Wang, Zhuo Zhang, Haining Guan et al. (+15 more)
Modern driving action models are increasingly improved in a self-improvement loop, where a learned world simulator imagines future observations and the resulting data is fed back to refine the action model. However, the bottleneck of this loop lies in the simulators' inability to generate behaviorally plausible responses by surrounding
agents, making generated data both unrealistic in interaction and imbalanced in distribution. We introduce BehaviorWorldGen, a framework that closes the loop between action models and world simulators through controllable behavior-aware structured world generation. Its core component is BehaviorFlow, a meta-action-conditioned traffic-flow model that injects interpretable behavior controls and jointly generates multi-
agent rollouts. BehaviorFlow realizes the specified
agent behaviors while allowing surrounding vehicles to respond to the ego and to one another. The resulting rollouts are rendered by a world simulator into realistic multi-view observations, which are paired with corrected interaction-aware trajectories for action-model refinement. Since BehaviorWorldGen uses structured trajectories as the interface between its modules, it is compatible with diverse action models and world simulators. Experiments on world generation, scene extrapolation, and policy refinement demonstrate consistent improvements, with the largest benefits concentrated on difficult interactive scenarios.
Read the full abstract ▾
All authors Jiaqi Wang, Zhuo Zhang, Haining Guan, Tingguang Zhou, Haowen Cui, ChuanYe Wang, Zhongyang Zhu, Yulong Zheng, Xuefeng Chen, Zhen Yang, Tianchen Deng, Feiyang Tan, Hangning Zhou, Bo Dai, Lixia Shen, Xiwu Chen, Xiyang Wang, Jiajun Zhu
Computer Vision
VERDICT: Agreement Beats Pixel-Space Verification in Real-Document OCSR
Yani Guan, Dengpan Dong, Shuang Luo et al. (+6 more)
Optical Chemical Structure Recognition (OCSR) converts 2D molecular depictions in the published literature into SMILES, and is increasingly important for constructing large-scale chemical training
datasets. Automation at that scale requires identifying unreliable predictions in the absence of ground truth. Three families of label-free signals were compared on $263$ ACS journal depictions with verified ground truth: model confidence, re-rendering similarity, and agreement among recognizers. Pixel-space re-rendering performed little better than chance (AUROC $0.547$, $95\%$ CI $[0.465,0.629]$), and an oracle-tuned threshold on it reduced correct labels per image from $0.745$ to $0.205$. Agreement among four architecturally distinct recognizers instead reached an AUROC of $0.916$ ($[0.880,0.952]$). The two-of-four rule accepted $81.7\%$ of images at $88.8\%$ precision, the three-of-four rule $52.1\%$ at $98.5\%$. The same pattern held on CLEF-IP, UOB, and USPTO. This distinction is obscured on synthetic
benchmarks, where re-rendered predictions naturally resemble their inputs. A substance filter removed $2{,}193$ false agreements on wildcards and R-group fragments, after which the three-of-four rule rejected all $68$ generic depictions. VERDICT was then applied to PMC Open Access, producing $6{,}146$ structure labels for $4{,}833$ molecules; chemist adjudication of $400$ released labels in two independent samples yielded precisions of $0.995$ for the three-of-four tier and $0.958$ for the two-of-four tier. VERDICT therefore enables validated labels for
multimodal molecular databases linking structure images, machine-readable representations, and source-publication information. In SES AI's Molecular Universe platform, VERDICT further serves as an image-based interface for searching and retrieving molecular records.
Read the full abstract ▾
All authors Yani Guan, Dengpan Dong, Shuang Luo, Zi Wei, Joah Han, Dan Hannah, Yumin Zhang, Qichao Hu, Kang Xu
Computer Vision
When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
Yubo Zhu, Zhehan Kan, Jingyi Yang et al. (+6 more)
Unified
multimodal models (UMMs) can perform both understanding and generation, raising a central question: can visual generation improve understanding? Existing evaluations provide mixed evidence, but confound task difficulty,
reasoning paradigms, and the closed-loop interaction between generation and understanding. We introduce VGAU-Diag, a fine-grained evaluation framework for
vision generation-assisted understanding. It stratifies samples by difficulty, enables unified evaluation of multiple
reasoning paradigms, and uses Oracle-Assisted Reference Protocols. Our analysis shows that generated visual aids help on easier instances but become unreliable as
reasoning complexity increases. Oracle-assisted diagnosis further reveals that the main bottleneck often lies on the visual-understanding side rather than the visual-generation side, as current UMMs struggle to leverage even faithful visual aids. We also show that effective visual generation should target visual-understanding bottlenecks rather than add more
reasoning steps, and identify a three-stage transition from task-irrelevant noise, to misleading plausible guidance, and finally to useful assistance. These findings would be useful to guide the development of better UMMs.
Read the full abstract ▾
All authors Yubo Zhu, Zhehan Kan, Jingyi Yang, Miaolin Chen, Jinbo Xing, Kai Zhu, Zijian Wang, Sheng Zhong, Wei Tong