Computer Vision
FreqAdapt: Frequency-Adaptive Processing for RAW Object Detection
Hanxi Li and Huiling Li
Existing object detection methods predominantly utilize sRGB inputs, which are compressed from RAW sensor data using Image Signal Processors (ISP) originally designed for visualization purposes. Compared to RGB images, RAW images possess favorable noise characteristics and richer information representation, which are crucial for object detection, particularly under challenging conditions such as adverse weather or low-light environments. In this paper, we propose FreqAdapt, a lightweight module for adaptive RAW data enhancement in the frequency domain. Unlike traditional spatial domain processing methods, FreqAdapt innovatively maps ISP operations to the
Fourier frequency domain and performs domain separation based on the physical properties of ISP operations, ensuring each operation is performed in its most suitable domain. Meanwhile, through an adaptive frequency domain encoder that jointly analyzes amplitude spectrum, phase spectrum, and RAW image features, we provide global context for ISP parameter prediction and employ a learnable fusion mechanism to achieve adaptive feature enhancement. Extensive experiments on multiple
datasets with diverse lighting and weather conditions demonstrate that FreqAdapt achieves state-of-the-art performance while maintaining lightweight efficiency and good physical interpretability. Furthermore, our module can be seamlessly incorporated into existing object detection frameworks, providing a novel solution for visual perception tasks in the RAW domain.
Read the full abstract ▾
Computer Vision
Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Prediction
Wei Wei, Yinyuan Zhao and Ruixuan Yu
3D multi-person motion prediction requires modeling both individual kinematics and inter-person interactions. While Flow Matching is effective for multi-hypothesis generation to improve prediction accuracy, directly predicting skeletal sequences from pure noise often compromises structural consistency and introduces unreliable cross-
agent interactions during early noise-dominated integration steps. To address this, we propose a Prior-Guided Residual Flow Matching framework. First, a Deterministic Coarse Prior (DCP) establishes a kinematic anchor, formulating the generative process as a conditional flow over motion residuals to simplify the generative objective and preserve structural stability. Second, a Dynamic Cross-Interaction (DCI) mechanism temporally synchronizes inter-
agent message-passing with the integration progress, ensuring the extraction of reliable social contexts and improving multi-person motion fidelity. Finally, a decoupled joint-motion architecture with bidirectional fusion effectively preserves fine-grained kinematic coherence. Extensive experiments demonstrate that our approach achieves state-of-the-art prediction accuracy across multiple
datasets. Code is available at https://github.com/Wei-Wei-a/Residual-Flow-Matching-with-Dynamic-Cross-Interaction-for-3D-Multi-Person-Motion-Prediction.
Read the full abstract ▾
Computer Vision
DRPFNet: Dual-domain Residual Progressive Fusion Network for RGB-Thermal Object Detection
Zian Wang and Changchun Li
RGB-thermal (RGB-T) object detection aims to fuse complementary information from visible and thermal modalities to achieve robust detection under varying illumination and weather conditions. Current methods typically employ
attention mechanisms or
transformers to perform cross-modal fusion independently at each feature scale, directly combining RGB and thermal features in the spatial domain. However, they still face significant limitations: cross-level knowledge inheritance caused by independent fusion at each scale,suppressing noise continuously due to the lack of bidirectional optimization, and information degradation induced by the absence of frequency-spatial collaboration. To address these issues, we propose DRPFNet, a Dual-domain Residual Progressive Fusion Network that constructs a unified information flow optimization system from three synergistic levels:structure, feature, and enhancement. At the structural level, we establish cross-scale propagation through bottom-up knowledge accumulation and bidirectional enhancement,ensuring smooth information flow. At the feature level, we collaboratively extract RGB high-frequency edges and thermal low-frequency structures via frequency band separation and edge guidance, guaranteeing representation quality. At the enhancement level, we enhance foreground-background discrimination through edge-guided dual-domain refinement,achieving precise object localization.Extensive experiments on two public RGB-T
datasets demonstrate that our method achieves competitive performance with competitive efficiency, validating the effectiveness of this hierarchical collaborative strategy.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Models
Xiaolin Chen, Xuemeng Song, Wenhao Shi et al. (+3 more)
Existing visual
emotion understanding methods typically ignore cultural variations in emotional perception. We introduce culture-conditioned visual
emotion understanding, a task that predicts the culture-specific emotional perception of a given image and explains the underlying rationale. Although related
benchmarks exist, they are limited by inconsistent individual annotations, which hinder the derivation of majority-supported culture-level
emotion labels, and imbalanced cultural coverage. Thus, we present ArtECulture, a
benchmark containing 6,792 artworks with culture-specific
emotion labels and explanations across English, Chinese, and Arabic cultures, with balanced Western and non-Western content. Evaluations of 16 open- and closed-source
Multimodal Large
Language Models (MLLMs) under a zero-shot setting reveal that the task remains challenging, with the best model achieving below 50% accuracy. To address this limitation, we introduce a
retrieval-augmented culture-conditioned
emotion understanding framework, which leverages a concept-based cultural
emotion knowledge base to inject explicit cultural knowledge into MLLMs without additional training. The framework improves both culturally aligned
emotion prediction and grounded explanation generation. Our
benchmark and code will be publicly released.
Read the full abstract ▾
All authors Xiaolin Chen, Xuemeng Song, Wenhao Shi, Xianjing Han, Mong-Li Lee, Wynne Hsu
Computer Vision
Can Text-to-Image Models Draw from the Right Frame of Reference?
Zheyuan Gu, Ruihang Li, Yong Huang et al. (+5 more)
Spatial instruction following has become a crucial requirement for text-to-image (T2I) generation. A common challenge arises when directional expressions are interpreted under different frames of reference. For example, “the left of” may refer to the viewer's image coordinates or to the intrinsic orientation of an object, leading to different expected layouts. Existing T2I
benchmarks reveal important layout failures, yet they rarely isolate whether models can follow a specified frame of reference when it differs from camera view. To mitigate this gap, we introduce FoR-T2I, a
benchmark for evaluating this distinction with 1,200
prompt pairs built from controlled spatial layouts. In each pair, the camera-view (Cam)
prompt states the target relation in camera view, while the frame-of-reference (FoR)
prompt describes the same target placement through an oriented anchor object. Across 22 closed-source and open-source T2I models, mean final accuracy is 41.8% lower on FoR
prompts than on matched Cam
prompts; even the best-performing model achieves only 44.3% FoR accuracy. This suggests that current models struggle more when the same layout is described through an object's orientation rather than directly in image coordinates. We further analyze this gap by relation type and camera view, compare several training-free prompting and feedback-based mitigation strategies, and propose a VLM-gated rewriting approach that selects rewritten
prompts using visual feedback, improving average FoR accuracy from 25.0% to 29.2% under the same generation budget.
Read the full abstract ▾
All authors Zheyuan Gu, Ruihang Li, Yong Huang, Yiqian Zhang, XIangzhao Hao, Jiaxin Niu, Jiahao Hu, Zhenyu Zhang
Computer Vision
When Oracle Conditioning Misleads Deployment: Conditioning-Availability Bias in Echocardiographic Segmentation
Dang P. M. Cao, Hieu D. Pham and Hieu Pham
Conditional
segmentation models may be trained and evaluated with auxiliary signals cleaner than those available at deployment. We study this protocol-level manifestation of shortcut learning and auxiliary-variable shift in phase-conditioned echocardiographic
segmentation. The complementary gap pair measures loss on the deployable oracle-estimated pathway and probes sensitivity on the oracle-random pathway. On held-out CAMUS data, one strong-cyclic, oracle-selected run fails severely with estimated phase, while sensitivity to incorrect phase persists across three runs. On EchoNet-Dynamic, the current
estimator remains usable, but random-phase testing reveals strong latent sensitivity. Deployment-aware checkpoint selection and phase perturbation reduce both gaps with little change in mean Dice. Exploratory subgroup analyses quantify variation across measured strata, and a downstream ejection fraction (EF) audit shows that recovering
segmentation does not necessarily recover EF error or signed bias. Together, the gaps test whether oracle-conditioned performance survives the inference pathway actually available at deployment.
Read the full abstract ▾
Computer Vision
SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference
Shanghao Liu, Renze Chen, Size Zheng et al. (+3 more)
Video diffusion transformers (vDiTs) generate high quality but pay quadratic self-
attention cost, making inference prohibitive at
video-
token scales. The challenge is input-adaptive sparsity: selecting critical Q/K/V
tokens with negligible overhead and executing them for end-to-end gains. We present SPADE, a training-free sparse-
attention engine of three parts: (i) vDiT-SSR, a specification defining 3D blocking candidates and formalizing dynamic masks via Summarizer/
Estimator expressions; (ii) runtime scheme generation using SICS and a head-wise policy; and (iii) an executor with low-overhead index search, flash block-sparse
attention, and kernel grouping. Across Hunyuan-
Video and Wan 2.1/2.2 for text-to-
video and image-to-
video generation, SPADE raises sparsity and speed while preserving quality, accelerating
attention by 2.26x-3.40x and end-to-end inference by 1.49x-1.80x. Our code is open-sourced at https://github.com/6somehow/DAC-SPADE.
Read the full abstract ▾
All authors Shanghao Liu, Renze Chen, Size Zheng, Yuanqiang Liu, Yun Liang, Hailong Yang
Computer Vision
PolyLayout: Multi-room Manhattan Layout Estimation
Gustav Hanning, Shaohui Liu, Rémi Pautrat et al. (+3 more)
Estimating room layouts from multi-view imagery is a core task for indoor scene understanding. Existing methods are typically limited either by poor generalization to new
datasets or restrictive
geometric assumptions of the room shape or camera configuration. Most also estimate rooms independently, failing to exploit shared building structure such as dominant directions, ground plane or ceiling height. We propose PolyLayout, a multi-room layout estimation method that parameterizes room layouts as Manhattan 3D polygons and optimizes them jointly across multiple rooms. The optimization objective is predicted by a neural network on top of robust pre-trained visual features and trained end-to-end with supervision only on output room layouts. At the same time, camera projection and polygon updates remain explicit and model-based. This separation between learned scoring and geometry improves generalization to new
datasets and camera parameters. During optimization, PolyLayout adaptively refines the polygon
topology through iterative wall split and merge operations while jointly utilizing structural cues across rooms. We introduce two new multi-view multi-room layout
benchmarks by providing layout annotations to existing
datasets, and experiments show that PolyLayout outperforms prior approaches, both in terms of accuracy and robustness. Project page: https://ghanning.github.io/PolyLayout
Read the full abstract ▾
All authors Gustav Hanning, Shaohui Liu, Rémi Pautrat, Marc Pollefeys, Kalle Åström, Viktor Larsson
Computer Vision
Zihan Wang, Tong Liu, Zhiwei Wang et al. (+6 more)
Medical visual grounding connects free-form clinical queries to spatial evidence in medical images and is an important component of interpretable medical artificial intelligence. However, general-purpose grounding models are predominantly trained on natural images, while existing medical localization resources remain fragmented across imaging modalities,
datasets, and task formulations. To address this gap, we construct LocAnyMed-200K, a
multimodal medical visual grounding
dataset containing approximately 200K image-query-answer examples across computed
tomography, optical medical imaging, ultrasound, and X-ray. We harmonize heterogeneous detection and localization resources into a unified free-form instruction format that supports one or multiple bounding boxes, point coordinates, and no-target outputs for negative queries. Full-parameter fine-tuning of LocateAnything-3B on LocAnyMed-200K improves F1@IoU 0.50 from 10.64 to 85.59 on a held-out evaluation split, demonstrating that large-scale domain-specific supervision can equip a general grounding model with effective medical localization capabilities. Beyond spatial coordinates, a clinically interpretable grounding system should also communicate the evidence supporting its prediction. We therefore derive LocAnyMed-CoT-20K, a rationale-augmented subset that connects anatomical context, visual observations, and spatial conclusions through structured
reasoning and further improves cross-source generalization through fine-tuning. Together, these resources provide a unified foundation for studying both localization accuracy and rationale quality across heterogeneous medical imaging modalities. The code is publicly available at https://github.com/MiliLab/LocAnyMed.
Read the full abstract ▾
All authors Zihan Wang, Tong Liu, Zhiwei Wang, Tao Huang, Wentao Jiang, Sihan Ma, Shanshan Ye, Xiaohui Yang, Jing Zhang
Computer Vision · Cross-listed from cs.LG
Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging
Siming Fu, Zheming Fu, Ruizhe He et al. (+7 more)
On-policy
distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same
language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules. We present Any-OPD, to our knowledge the first framework for on-policy
distillation between arbitrary pairs of latent flow-matching generators. Any-OPD treats the teacher purely as a black-box sampler and connects the two models at exactly one point: a frozen, model-agnostic
vision representation in which their independently decoded outputs are compared, sidestepping every assumption about latents, features, or architecture. Trajectory correspondence is recovered by matching continuous noise levels instead of step indices, and a brief anchoring phase, in which teacher samples are re-encoded through the student's own VAE, ensures the on-policy gradient measures sample quality rather than domain mismatch. Distilling the 12B FLUX.1-dev into the 2.5B SD3.5-Medium, Any-OPD lifts the student's PickScore from 0.846 to 0.884 and HPSv3 from 9.12 to 10.97, rivaling the teacher at a fifth of its size, where direct latent regression fails to train at all.
Read the full abstract ▾
All authors Siming Fu, Zheming Fu, Ruizhe He, Hualiang Wang, Jie Huang, Xiaoxiao Ma, Mingchen Zhong, Weihu Huang, Xiaoxuan He, Haojun Xu
Computer Vision
Recurrent Contrastive Learning for Imbalanced Medical Image Classification
Zhiyuan Zhu, Xinling Meng, Junxuan Yu et al. (+15 more)
Medical image classification often suffers from class imbalance due to the inherent disparities in disease incidence. Existing approaches, such as class resampling and loss reweighting, mainly improve learning within the observed feature distribution, but do not explicitly enlarge the latent support region of tail classes. As a result, tail-class representations remain overly compact and are easily encroached upon by head classes, leading to biased decision boundaries. In this work, we propose Recurrent Contrastive Learning (RCL) for imbalanced medical image classification. RCL progressively expands the support region of tail classes by recurrently reusing historical feature states across training phases. Specifically, we adopt DINOv3 with LoRA adapters as the backbone to provide robust feature
embeddings. We then devise a Temporal
Memory Queue (TMQ) to preserve corpus-level features across training phases and provide diversified global references for contrastive learning. Based on TMQ, we construct Temporal Anchors (TARs) to form an anchor field around tail classes. This field enlarges the support region of tail classes, suppresses head-class encroachment, and improves inter-class separation. Extensive experiments on three imbalanced medical
datasets demonstrate that RCL achieves consistent improvements over strong baselines. The code is available at https://github.com/dndins/RCL.
Read the full abstract ▾
All authors Zhiyuan Zhu, Xinling Meng, Junxuan Yu, Jiongquan Chen, Qiongying Ni, Tuhang Shao, Yuhao Huang, Luping Zhou, Ruiyang Huang, Yuxue Wang, Rongliang Zhang, Xue Wang, Tianhong Tang, Likun Wang, Junbo Chen, Yong Jiang, Yongping Lu, Xin Yang
Computer Vision · Cross-listed from cs.RO
PLS-Calib: A Partial Least Squares Framework for Event Camera and Odometry Calibration under Ground Motion Constraints
Guangyu Li, Xiao Li, Yujie Wu et al. (+5 more)
Accurate extrinsic rotation calibration between sensors is fundamental to the performance of robotic perception systems. However, most existing calibration techniques rely on full 6-DoF motion to excite all degrees of freedom, which is often infeasible for ground-constrained
robots with limited motion capabilities. Recent approaches designed for such restricted settings, such as Canonical Correlation Analysis (CCA)-based methods, suffer from ill-conditioned covariance
matrices that lead to numerical instability and suboptimal calibration accuracy. To overcome these limitations, we present a novel rotation calibration framework named PLS-Calib that, for the first time, leverages Partial Least Squares (PLS) regression to model the latent kinematic correlations between asynchronous, heterogeneous sensor streams. Specifically, we apply our method to the calibration of an event camera and an odometry onboard a ground
robot. To improve event-based pattern detection, we introduce a polarity-aware event representation, which enhances spatiotemporal contrast in circular calibration targets. Our PLS-based formulation yields a closed-form, stable solution that avoids
matrix singularities inherent in CCA-based approaches. Extensive experiments on both synthetic and real-world
datasets validate the effectiveness of our approach, demonstrating significant improvements in calibration robustness and accuracy over state-of-the-art methods. This work offers a practical and theoretically grounded solution for rotation calibration in constrained robotic systems and opens up new directions for applying statistical learning techniques in neuromorphic
vision.
Read the full abstract ▾
All authors Guangyu Li, Xiao Li, Yujie Wu, Changshuo Wang, Prayag Tiwari, Jiang Cai, Fangwen Yu, Mingkun Xu
Computer Vision
Test-Time Scaling for Safe Text-Guided Image Generation via Intermediate Clean Estimates
Jinya Sakurai, Shueicheng Yan and Xun Xu
Ensuring safety and policy compliance in text-to-image
diffusion models remains a critical challenge, as benign or
adversarial prompts can often elicit prohibited content, e.g. nudity and protected intellectual property. While training-based unlearning methods are effective, they are computationally expensive and prone to catastrophic interference with general capabilities. Conversely, existing test-time defenses are primarily
prompt-centric, relying on modifying textual descriptions only, and overlook the visual signals for detection. In this paper, we propose to leverage the intermediate clean image estimated during the generation process and employ a sparse margin objective to detect prohibited concepts. When a violation is detected, we immediately intervene by optimizing a structured low-rank residual in the text-conditioning space via truncated backpropagation. This design allows weight-preserving detection, keeps non-violating inference latency nearly unchanged as the maximum budget increases, and offers flexibility in safety performance via test-time scaling. Extensive experiments on Stable
Diffusion v1.4 and v3.5 across nudity removal, IP protection, and style erasure demonstrate superior performance across suppression, fidelity and preservation compared to prior weight-preserving baselines, providing a scalable and flexible solution for safe generative deployment.
Read the full abstract ▾
Computer Vision
3DGSI-Assessor: A Large-Scale Dataset and An LMM-based Method for 3D Gaussian Splatting Image Quality Assessment
Yuke Xing, Jiarui Wang, William Gordon et al. (+3 more)
3D Gaussian Splatting (3DGS) has become a dominant representation for real-time novel view synthesis (NVS), yet its storage footprint makes compression indispensable for practical deployment. 3DGS training and compression introduce representation-specific distortions such as floating artifacts and surface scattering, which conventional image quality assessment (IQA) metrics fail to capture. Moreover, the independent compression of
geometric and color attributes may lead to decoupled dimension-specific distortions that must be diagnosed separately, yet existing metrics report only a single overall score. To address these gaps, we present 3DGS-IEval-15K+, a large-scale, multi-dimensional IQA
dataset for compressed 3DGS, comprising 15,200 images from 10 diverse scenes, produced by 6 representative 3DGS
algorithms at systematically designed compression levels and rendered from 20 strategically selected viewpoints spanning both training views and challenging novel views, annotated with 45,600 mean opinion scores (MOSs) across overall, geometry, and color quality. Based on 3DGS-IEval-15K+, we propose 3DGSI-Assessor, an all-in-one 3DGS IQA framework that integrates global semantic and dimension-specific local features within a large
multimodal model (LMM), predicting all three dimensions in a single forward pass. 3DGSI-Assessor achieves state-of-the-art performance on 3DGS-IEval-15K+, and exhibits competitive generalization on other NVS
benchmarks.
Dataset and code will be released at https://github.com/YukeXing/3DGSI-Assessor.
Read the full abstract ▾
All authors Yuke Xing, Jiarui Wang, William Gordon, Zhu Li, Guangtao Zhai, Yiling Xu
Computer Vision
GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs
Zichuan Fu, Shirong Wang, Wenlin Zhang et al. (+10 more)
GUI grounding maps natural-
language instructions to click locations and is essential for reliable GUI
agents. The task remains difficult on high-resolution, densely populated interfaces because a
vision-
language model (VLM) may recognize a requested control without locating it precisely enough for interaction. Most existing methods provide various forms of localization assistance, but still rely on a direct click prediction, allowing visual ambiguity or an inaccurate initial estimate to propagate to the final result. In this paper, we introduce GUI-Lens, a coarse-to-fine grounding framework that allows a general-purpose VLM to determine the target through active visual observations. Specifically, GUI-Lens extracts OCR text and detected UI components from the screenshot and presents their positions as coordinate references. Using the instruction, the current view, and these references, the VLM selects the region and scale of the next view, which is cropped and enlarged to provide finer visual details. This process continues over successively focused views until the target is determined. Proposed crops and clicks are checked against the instruction throughout the process, and the final local position is mapped back to the original screen coordinates. Experiments on four GUI grounding
benchmarks and three general-purpose VLM backends show that GUI-Lens improves overall grounding accuracy by up to 24.9 percentage points and achieves state-of-the-art performance with GPT-5.5.
Read the full abstract ▾
All authors Zichuan Fu, Shirong Wang, Wenlin Zhang, Guojing Li, Yimin Deng, Jingtong Gao, Junjia Qi, Hanyu Yan, Yefeng Zheng, Xiaopeng Li, Wanyu Wang, Xian Wu, Xiangyu Zhao
Computer Vision
Efficient Video Dataset Distillation via Cluster-Guided Prototype Blending
Chongle Ren, Guang Li, Wenbo Huang et al. (+3 more)
Video dataset distillation aims to compress a large
video dataset into a compact surrogate set that preserves its training utility. Most existing approaches synthesize condensed
videos through iterative optimization, whose cost is amplified by the temporal dimension. Rather than further reducing the number of optimized variables, we investigate whether effective distilled
videos can be constructed without gradient-based optimization of the stored
videos. Such a construction-based approach must address three challenges: selecting informative temporal segments, covering diverse intra-class variations under a limited
videos-per-class budget, and increasing the information carried by each stored sample. To this end, we propose ProtoBlend, an efficient select-allocate-blend framework. First, teacher-guided temporal clip selection retains a high-confidence segment from each source
video. Second, cluster-guided prototype allocation partitions the selected clips in the teacher feature space and assigns one distilled slot to each intra-class cluster. Third, each prototype is blended with an in-cluster anchor, while their teacher predictions are combined using the same coefficient to provide mixture-source supervision. Experiments on four trimmed action-recognition
benchmarks demonstrate that ProtoBlend achieves a competitive accuracy-efficiency trade-off without iterative optimization of the distilled
videos.
Read the full abstract ▾
All authors Chongle Ren, Guang Li, Wenbo Huang, Naoki Saito, Takahiro Ogawa, Miki Haseyama
Computer Vision · Cross-listed from cs.MM
Hear to See: Discerning Stateful Listening for Audio-Visual Instance Segmentation
Leiye Liu, Miao Zhang, Jiahong Jiang et al. (+7 more)
Audio-visual instance
segmentation (AVIS) requires accurately identifying and tracking individual sounding objects with pixel-level masks. Existing methods struggle to match overlapping acoustic events with visual instances and handle asynchronous audio-visual dynamics. Therefore, two critical questions arise: how can a model establish precise correspondence between overlapping sound sources and visual instances, and how can a model maintain robust tracking when audio and visual signals are temporally misaligned?This paper proposes Hear to See (H2S), addressing these challenges through two mechanisms. The Acoustic-Semantic Projector (ASP) disentangles mixed audio and establishes hierarchical correspondence from semantic to spatial domains. The Asynchronous Dynamics Modulator (ADM) adaptively adjusts state transitions via audio-modulated Mamba, prioritizing current information during dynamic variations and maintaining continuity in stable periods.Experiments on AVISeg show H2S achieves SOTA performance, attaining 48.54 mAP with a COCO pretrained ResNet50 and surpassing the previous by 7.8%. The code will be open-sourced once the paper is accepted. The source code will be publicly available at https://github.com/leiyeliu/H2S.
Read the full abstract ▾
All authors Leiye Liu, Miao Zhang, Jiahong Jiang, Jingjing Li, Jialong Zhong, Kai Peng, Tingwei Liu, Wei Ji, Yongri Piao, Huchuan Lu
Computer Vision
NanoMorph-3D: An End-to-End Physics-Driven Unrolling Framework for Nanomaterial Reconstruction
Beiyuan Zhang, Hesong Li, Ziqi Wu et al. (+2 more)
Precise 3D characterization of nanomaterials is essential for unlocking structure-property relationships. However, standard electron
tomography is fundamentally limited by the missing wedge problem. Consequently, conventional
algorithms suffer from severe
geometric distortions, a challenge further complicated by pervasive noise interference. Current learning-based methods either rely on physics-blind post-processing or employ end-to-end architectures constrained by local receptive fields, failing to capture complex 3D
topologies. We propose NanoMorph-3D, a unified end-to-end framework grounded in a comprehensive Nanomorphological Taxonomy. Powered by a large-scale synthetic
dataset explicitly modeling non-linear electron attenuation, we design a Physics-Driven Unrolled Network mapping proximal gradient descent into a learnable architecture. To capture complex internal
topologies, we formulate a hierarchical
attention mechanism with Physics-Normalization for long-range 3D dependencies and scale invariance. Crucially, our Dual-Domain strategy leverages Sinusoidal
Attention to explicitly model physical projection trajectories, enforcing strict sinogram consistency to mitigate missing wedge artifacts. Finally, an unsupervised dual-stream mechanism bridges the simulation-to-reality gap. Experiments demonstrate NanoMorph-3D reconstructs diverse
topologies with superior fidelity and speed.
Read the full abstract ▾
All authors Beiyuan Zhang, Hesong Li, Ziqi Wu, Ruiwen Shao, Ying Fu
Computer Vision
Clarity Contrast and Similarity Selection for Multi-Focus Image Fusion
Yicheng Zhang, Haoyou Deng, Zhiqiang Li et al. (+3 more)
Multi-focus image fusion (MFIF) aims to generate an all-in-focus image from multiple images of the same scene focused at different regions. Most existing deep learning-based methods lack explicit interaction between the source images, which limits their performance and interpretability. This paper presents a novel Clarity Contrast and Similarity Selection Network (CSNet), to bridge direct information exchange for MFIF. Specifically, by contrasting the clarity differences between source images within our proposed Clarity Contrast
Attention Module (CCAM), we mutually enhance sharp features while suppressing blurry ones. This allows us to identify the exactly focused regions in each source and locate the focused-defocused boundaries. Moreover, the Defocus Spread Effect (DSE) degrades pixels in all source images around the boundaries. To further refine these ambiguous areas, we introduce a Similarity Selection Strategy, which reconstructs an initial clear image from source images and selects optimal pixels by comparing the similarity among them. Through this interactive approach, CSNet effectively preserves focused regions as well as recovering natural boundaries to fuse an all-in-focus output. Extensive experiments demonstrate that our method achieves state-of-the-art performance both quantitatively and qualitatively. Our code is available on Github: https://github.com/ZYC-HUST/CSNet.
Read the full abstract ▾
All authors Yicheng Zhang, Haoyou Deng, Zhiqiang Li, Wenti Yin, Nong Sang, Changxin Gao
Computer Vision
CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignment
Jing Dai, Qibin Zhang, Weiwei Zhou et al. (+4 more)
Multimodal learning has significantly advanced survival prediction by integrating pathology images with genomic data. However, clinical information, despite its critical role in reflecting a patient' s overall health, remains underutilized due to its discrete, sparse, and low-dimensional nature. Furthermore, the inherent heterogeneity across these modalities pose significant challenges in modeling cross-modal interactions. In this paper, we propose CIGTSurv, a Clinical Information Guided Tri-modal framework for Survival prediction. Specifically, we first design a holistic text template and use pretrained foundation models to transform clinical tabular data into high-dimensional tokenized
embeddings. Using clinical information as an anchor, we then introduce a dual-level interaction mechanism: 1) a local prototype association (LPA) module based on cross-
attention to explicitly learn
token-level correspondences between different modalities, and 2) a global feature
alignment (GFA) loss based on Maximum Mean Discrepancy (MMD) to implicitly enhance cross-modal distribution consistency. Extensive experiments on five TCGA cancer cohorts demonstrate that CIGTSurv achieves state-of-the-art (SOTA) survival prediction performance. Our source code is publicly available at https://github.com/Daijing-ai/CIGT-Surv.git.
Read the full abstract ▾
All authors Jing Dai, Qibin Zhang, Weiwei Zhou, Mingde Xu, Jingsong Liu, Jingdong Zhang, Hongming Xu
Computer Vision
Open-Linguistic Concept Unified Learning for Cross-Site Interpretable Dermatology Image Diagnosis
Chengyu Wu, Junpeng Tan, Wanxiang Luo et al. (+3 more)
Human-interpretable computer-aided diagnosis is crucial for clinical decision making. Concept-based models excel by providing transparent
reasoning and enabling post-hoc, clinician-in-the-loop interventions. However, their rigid
dataset-specific adaptation inherently restricts cross-site generalization. Applying them across diverse modalities, such as dermoscopic and clinical photographs, is challenging due to heterogeneous concept taxonomies varying in availability, granularity, and semantics across cohorts. Consequently, adapting Foundation
Vision-
Language Models (FVLMs) demands costly label engineering and repeated post-training. Existing intervention mechanisms remain rigidly tied to predefined concepts, lacking adaptability and hindering scalable dermatology CAD deployment. To address these bottlenecks, we propose UniCon, an open-linguistic unified concept learning framework for
multimodal interpretable
vision-
language diagnosis. UniCon resolves these challenges through three contributions: (1) A shared semantic representation space via a unified concept prototype codebook, seamlessly coordinating heterogeneous concept systems across modalities without
dataset-specific retraining. (2) Open-linguistic based multi-faceted semantic specifications to overcome sparse textual label limitations, improving boundary sensitivity in uncertain clinical contexts. (3) A robust, cross-site adjustable intervention interface powered by reliability-gated bottleneck aggregation, enabling consistent
reasoning and transferable clinician corrections. Extensive experiments demonstrate that beyond securing top-tier diagnostic accuracy, UniCon successfully bridges disparate clinical taxonomies, unlocking unprecedented cross-site intervention capabilities. Code is available at https://github.com/wuchengyu123/UniCon.
Read the full abstract ▾
All authors Chengyu Wu, Junpeng Tan, Wanxiang Luo, Yaqi Wang, Yandong Wen, Yefeng Zheng
Computer Vision
Self-Supervised Representation-Guided Generative Dataset Distillation
Mingzhuo Li, Guang Li, Linfeng Ye et al. (+4 more)
Dataset distillation compresses a large training set into a compact synthetic set while retaining its downstream utility. Most existing methods target randomly initialized networks, whereas modern
vision systems often adapt frozen pretrained encoders with lightweight modules. Distilled samples should therefore preserve the discriminative geometry of the pretrained representation space, which existing generative objectives do not explicitly consider. We propose self-supervised representation-guided generative
dataset distillation (SRG), a framework that translates the SSL geometry into
diffusion guidance. Specifically, SRG constructs class-wise prototypes from real-image SSL representations and performs guidance through three SSL-space objectives for prototype
alignment, inter-class discrimination, and intra-class assignment. During
diffusion sampling, it adopts a stage-wise guidance strategy: early denoising is anchored to the latent of the real image whose SSL representation is nearest to the assigned prototype, whereas later denoising is guided by the SSL-space objectives. This division preserves the visual realism provided by the generative prior while progressively steering samples toward representative and class-discriminative regions of the SSL representation space. SRG consistently outperforms the evaluated generative baselines across multiple
datasets and IPC settings. A cross-encoder evaluation further indicates transfer across pretrained representation spaces. These results demonstrate the effectiveness of representation-guided generation for
dataset distillation with pretrained SSL models.
Read the full abstract ▾
All authors Mingzhuo Li, Guang Li, Linfeng Ye, Jiafeng Mao, Takahiro Ogawa, Konstantinos N. Plataniotis, Miki Haseyama
Computer Vision
iFAN: Inference-Aware Learning for Plain Mask Transformers
Fang Li, Yu He, Haoyang Tong et al. (+7 more)
Query-based mask
transformers assemble
segmentation outputs through pixel-wise competition among query predictions of the final layer, yet this inference process is not explicitly optimized during training. We identify two key mismatches: the query with the highest probability-mask score does not necessarily produce the most accurate mask, and final-layer decoding may discard superior predictions from intermediate layers. To address these issues, we propose Inference-Aware Learning (iFAN), a general training framework for plain mask
transformers. iFAN introduces Adjusted Probability-Mask Ranking (APMR), which aligns query competition with predicted mask quality and suppresses high-confidence but inaccurate competitors. We further employ Cross-Layer Self-
Distillation (CLSD) to transfer stronger intermediate predictions to the final layer. The ranking and
distillation objectives are training-only, while inference retains efficient final-layer decoding. Experiments on COCO, ADE20K, and Cityscapes demonstrate consistent improvements across panoptic, instance, and semantic
segmentation, as well as across different architectures, backbone scales, and input resolutions. Overall, iFAN improves performance by an average of 1.20 PQ, 1.30 AP, and 0.63 mIoU, with negligible additional parameters, FLOPs and inference latency.
Read the full abstract ▾
All authors Fang Li, Yu He, Haoyang Tong, Lichen Ma, Jingling Fu, Wenxiao Fan, Tongxuan Liu, Luohang Liu, Ke Zhang, Junshi Huang
Computer Vision
CrossScope: A Role-Asymmetric World Model for Joint Dual-Scope Surgical Video Prediction
Wanhao Liu, Jinsong Lin, Rulin Zhou et al. (+11 more)
Visual world models typically learn future dynamics from a single observation stream, limiting their ability to model cooperative systems with multiple independently moving observers. We investigate this challenge in Mother–Child endoscopic retrograde cholangiopancreatography (ERCP), where two flexible scopes provide complementary yet role-dependent views without a calibrated stereo relationship. Unlike conventional multi-view fusion that assumes symmetric information exchange, we formulate role-asymmetric dual-scope future prediction, where cross-view evidence is selectively transferred according to the prediction target and its underlying spatial requirements. We propose CrossScope, a dual-stream
surgical world model that preserves view-specific experts while enabling target-specific evidence routing through geometry-guided residual interactions. CrossScope learns two complementary communication directions:
geometric motion cues from the Mother view guide Child-view future dynamics, while pose-aligned Child appearance supports Mother-view prediction only when valid spatial correspondence is established. This design allows each scope to contribute task-relevant evidence without compromising its view-specific representation. To evaluate this problem, we establish a paired dual-scope
benchmark comprising synchronized phantom and real-world ERCP episodes, with evaluations assessing visual fidelity, structural preservation, target localization, and motion consistency. Experiments demonstrate that CrossScope consistently outperforms strong
surgical video generation baselines, validating the importance of role-aware evidence routing for multi-observer visual world modeling.
Read the full abstract ▾
All authors Wanhao Liu, Jinsong Lin, Rulin Zhou, Chi Kit Ng, Wenbin Pan, Zhiqing Tang, Dongyue Li, Liwei Luo, Yanshen Wu, Panshuo Li, Zhiyong Xiong, Huxin Gao, Tamas Haidegger, Hongliang Ren
Computer Vision
DRIFT: Derailing Denoising Trajectories of Flow-Matching VLAs with Adversarial Patch Attack
Hoseong Tae and Jong-Seok Lee
Flow-matching
vision-
language-action (VLA) models such as pi0 generate
robot actions by integrating a learned denoising velocity field, and have been reported to resist
adversarial perturbations that readily fool autoregressive VLAs. We show that this robustness is largely illusory: it stems from prior attacks ignoring the multi-step denoising ODE. We introduce DRIFT (Denoising Redirection via Input perturbation of the Flow-matching Trajectory), a test-time universal
adversarial patch placed on the
robot's gripper that attacks the denoising velocity field of an off-the-shelf policy. Our central finding is counterintuitive: attacking only the first denoising step is both stronger and cheaper than attacking a wider window of steps, which we explain through a gradient conflict unique to input-space optimization and which is exactly opposite to the training-time
backdoor regime. On pi0 and pi0.5 across four LIBERO suites, DRIFT breaks essentially all originally-solvable tasks with a small single patch, far exceeding action- and
embedding-space attack baselines.
Read the full abstract ▾
Computer Vision
Bridging Online and Offline Handwriting via Differentiable Physical Rendering
Seonmi Park, Seunghyun Shin, Vihaan Misra et al. (+4 more)
Realistic handwritten text generation plays an important role in numerous applications, such as font design, biometric authentication, and robotic calligraphy. Existing methods are typically divided into two independent paradigms: online approaches that estimate handwriting trajectories and offline approaches that synthesize realistic handwriting images. While online models capture structural and temporal dynamics, they often lack fine-grained textures, whereas offline models reproduce realistic appearance but discard stroke order. However, unifying online and offline models remains challenging due to (1) the lack of an explicit physical model linking stroke kinematics to pixel-level appearance and (2) the absence of paired trajectory-image
datasets. Moreover, enabling end-to-end learning requires a differentiable rendering process across motion and appearance domains. To address these challenges, we propose a compact physical brush model that bridges stroke dynamics and visual appearance, together with a differentiable rendering module that converts stroke trajectories into stylized images. By integrating these components, we propose a unified online-offline handwriting generation framework via differentiable brush rendering. The proposed framework consists of four core modules: 1) a text-to-stroke generator that predicts the target stroke conditioned on the given text and style image, 2) a brush parameter observer that extracts brush model parameters from style references, 3) a differentiable brush renderer that maps a stroke sequence and physical brush parameters into a handwritten image, and 4) a zero-shot image refiner that refines rendered images via
diffusion models. Extensive experiments and real-world robotic calligraphy demonstrations validate our approach, achieving both structural and visual fidelity.
Read the full abstract ▾
All authors Seonmi Park, Seunghyun Shin, Vihaan Misra, Dongmin Shin, Ukcheol Shin, Jean Oh, Hae-Gon Jeon
Computer Vision
CRIL-U-Net: Compact Ratio-Interaction Learning for Focal Cortical Dysplasia Segmentation from T1w and FLAIR MRI
Soumen Ghosh, Amit Soni Arya, Tilottama Goswami et al. (+3 more)
Focal cortical dysplasia (FCD) type II is an important structural cause of drug-resistant focal epilepsy, but its small size, heterogeneous appearance, and subtle MRI characteristics make automated
segmentation challenging. Conventional
multimodal networks commonly concatenate T1-weighted (T1w) and fluid-attenuated inversion recovery (FLAIR) images, requiring subsequent layers to learn useful cross-modal relationships implicitly. We propose CRIL-U-Net, a 3D U-Net incorporating a Compact Ratio-Interaction Learning module that combines local spatial features, voxel-wise cross-modal mixing, and bidirectional ratio-inspired interactions. CRIL-U-Net was compared with a conventional 3D U-Net and an input self-
attention U-Net using five-fold cross-validation on 85 FCD subjects and 25 healthy controls. Each architecture was trained independently using Dice-binary cross-
entropy (Dice-BCE) and Focal Tversky-Focal (FTF) losses. With FTF, CRIL-U-Net achieved the highest mean Dice score (0.196 +/- 0.262), compared with 0.136 +/- 0.224 for the U-Net and 0.135 +/- 0.214 for the
attention comparator. It produced nonzero lesion overlap in 44 of 85 cases, compared with 36 for the U-Net. Under FTF, CRIL-U-Net significantly outperformed both comparison architectures after false-discovery-rate correction. These findings suggest that compact cross-modal representation learning can improve FCD
segmentation within a controlled U-Net setting when combined with an imbalance-aware objective, although the remaining zero-overlap rate of 48.2% highlights the need for further validation and methodological development.
Read the full abstract ▾
All authors Soumen Ghosh, Amit Soni Arya, Tilottama Goswami, Subhojit Mandal, John Phamnguyen, Rajat Vashistha
Computer Vision
EditFlow3D: Automated Local Editing of 3D Assets with Trajectory Preservation
Rui Nie, Chuang Wang, Haitao Zhou et al. (+4 more)
Controllable local editing of 3D assets requires precise target localization and appropriate visual guidance. However, existing methods lack a simple yet accurate way to obtain 3D masks and struggle to achieve the desired edit while faithfully preserving the structure and appearance of non-target regions. To address these challenges, we present EditFlow3D, a training-free framework for local 3D editing. Given a source asset and an edit instruction, a VLM-driven workflow interprets the editing intent and automatically constructs a visual guidance image and a refined 3D editing mask, enabling localized editing in the native representation space of a pretrained 3D generative model. Specifically, mask-guided differential flow focuses the edit on the target region, while step-wise trajectory preservation maintains consistency between non-target regions and the source asset without directly replacing intermediate features. Since the existing Edit3D-Bench covers only a limited range of local editing categories, we further introduce EditFlow-Bench as a complementary
benchmark encompassing a broader variety of structural and appearance edits, and evaluate EditFlow3D on both
benchmarks. Quantitative results, qualitative comparisons, and a user study demonstrate that EditFlow3D achieves more accurate target-region editing and better preserves non-target regions than existing 3D editing methods.
Read the full abstract ▾
All authors Rui Nie, Chuang Wang, Haitao Zhou, Jiahe Song, Buyu Li, Sheng Wang, Qian Yu
Computer Vision
Frequency-Decorrelated Temporal Ensembles for EEG–fNIRS Imagined-Handwriting Decoding
Xiao Fan, Hongbin Guo, Yubo Han et al. (+1 more)
Imagined handwriting offers a temporally rich paradigm for non-invasive neural decoding, yet reliable recognition across unseen participants remains difficult because scalp
EEG is noisy and internally generated stroke sequences vary across individuals. The
Multimodal Brain-Computer Interface Grand Challenge provides synchronized
EEG and fNIRS for four-class subject-independent handwriting-trajectory classification. We propose FRED, a task-adapted system that models imagined handwriting as a multi-second motor sequence and trains a compact multi-scale temporal network on three complementary
EEG frequency views. With three seeds per view, cross-band members produce substantially less-correlated errors than same-band replicas, yielding a clean nine-member ensemble accuracy of 0.8076/0.7242/0.7492 on the public/private/overall test partitions without test-set adaptation or output constraints. The submitted pipeline further incorporates transductive pseudo-label training, three
EEG-Conformer members, posterior aggregation, and a paradigm-aware decoder. Because every 12-trial randomization block contains three instances of each class, the final predictions are obtained by Hungarian assignment under the known block quota. On one fixed posterior pool, independent, session-constrained, and block-constrained decoding achieve 0.7600, 0.7758, and 0.7952 overall accuracy, respectively. The complete system reaches 0.8498/0.7718/0.7952, ranking fourth on the private split. A modality audit finds fNIRS-only decoding at chance (0.2511 overall), while adding fNIRS to
EEG changes accuracy by only +0.0025. These results identify frequency-diverse temporal
EEG modeling and protocol-matched structured inference as the principal sources of performance in this sparse-montage
EEG–fNIRS setting. The source code is available at https://github.com/XiuFan719/
EEG-fNIRS-fuse-method-for-MM-challenge.
Read the full abstract ▾
All authors Xiao Fan, Hongbin Guo, Yubo Han, Yi Zhang
Computer Vision · Cross-listed from cs.CR
SpreadMark: Robust Image Watermarking via Spread-Spectrum Embedding
Wei Song, Yuxin Cao, Zhenchang Xing et al. (+4 more)
Invisible image watermarks are increasingly used for deepfake detection and provenance tracking, where they must survive not only incidental distortions but also deliberate removal. We revisit spread-spectrum
embedding, a classical watermarking principle, inside a modern neural post-hoc watermarking architecture. Our starting point is a measurement: in existing encoder-decoder schemes each message bit occupies only a small fraction of the image, a shared contributing factor to their fragility, since removal then need only disturb the region a bit occupies. SpreadMark instead spreads each bit as a dense pseudo-random codeword over the whole image and recovers it by matched-filtering a learned cover-suppressed chip representation, with a parallel convolutional decoding path and sparsification-aware training. A conditional chip-space analysis shows that, under a codeword-independent perturbation model, dense spreading increases the budget required to disrupt matched-filter recovery. Evaluated on COCO and DIV2K against nine schemes, SpreadMark is the only evaluated method retaining high detection under both the regeneration and the latent-space sparsification settings we test, with competitive JPEG and additive-noise robustness. It keeps the embedded watermark imperceptible, maintaining high perceptual quality on both COCO and DIV2K.
Read the full abstract ▾
All authors Wei Song, Yuxin Cao, Zhenchang Xing, Liming Zhu, Jin Song Dong, Yulei Sui, Jingling Xue