Computer Vision · Cross-listed from cs.GR
Hi-TOPS: Hierarchical Topology-aware Scoring Prior for 3D Part Decomposition
Ruoyu Wu, Zhenhong Sun, Xiaoming Gong et al. (+5 more)
Accurate 3D part decomposition requires separating shapes into structurally meaningful components with precise boundaries while preserving articulation seams and thin attachments. Existing approaches often suffer from a structural-scale mismatch:
geometric evidence for separation is most reliable at the meso scale, yet many pipelines operate either too globally to respect joints or too locally to remain robust to noise. We propose Hi-TOPS, a Hierarchical
Topology-aware Scoring Prior that aggregates complementary intrinsic cues into a multi-resolution Flow-Freeze field. Flow regions provide expandable support for primitive coverage, while Freeze regions restrict growth near articulations and thin structures. A TSDF-guided body-surface superquadric fitter then captures dominant cores and residual surface structures, followed by SQ-to-mesh assignment for connected, boundary-aligned parts. Across diverse
benchmarks, Hi-TOPS delivers stable, editable decompositions without semantic supervision or 2D foundation priors.
Read the full abstract ▾
All authors Ruoyu Wu, Zhenhong Sun, Xiaoming Gong, Yuxin Xian, Zhi Wang, Yawen Chen, Huadong Mo, Daoyi Dong
Computer Vision
NISF++: Geometrically-grounded implicit representations of 3D+time cardiac function from 2D short- and long-axis MR views
Nil Stolt-Ansó, Maik Dannecker, Steven Jia et al. (+2 more)
Clinical acquisition in cardiac
magnetic resonance (CMR) imaging involves obtaining cross-sectional planes of the heart along the radial and longitudinal directions. Despite these planes being 2D cross-sectional images of the heart, radiologists understand the 3D spatial and continuous temporal nature of the organ being imaged. The same can not be said about the conventional deep learning architectures used to process CMR images, which rely on in-plane and grid-based operations, and are hence unable to organically integrate information from all imaging planes. This paper builds upon previous work on neural implicit
segmentation functions (NISF) to overcome unaddressed challenges in cardiac function modeling in the CMR domain. For a given subject, our architecture builds a shared 3D+time representations from all available acquisition planes regardless of orientation. By design, predictions along any imaging plane orientation are cross-sections of the same 3D representation, leading to spatio-temporal consistency across all slices. Moreover, our architecture makes the rotation and translation parameters of imaging planes learnable, allowing us to correct for the commonplace respiratory and patient motion between slice acquisitions under a rigid assumption. Furthermore, interpolation of intensities and
segmentation can be performed in 4D at any desired resolution. We perform our study on a 120 subject sub-cohort of CMR imaging data from the UK-Biobank. Our in-plane
segmentation performance is on-par with existing CMR
segmentation methods and explore how the majority of failure cases arise from limitations in the ground-truth
segmentation, for which our representations make predictions with better anatomical accuracy than its original training data. We also evaluate our motion-correction capabilities, displaying quantitative and qualitative improvements in slice
alignment.
Read the full abstract ▾
All authors Nil Stolt-Ansó, Maik Dannecker, Steven Jia, Julian McGinnis, Daniel Rueckert
Computer Vision
LUT: Latent Utility Training for Visual Reasoning
Jiaxuan Kang, Siyu Chen, Mingda Li et al. (+6 more)
Multimodal large
language models have advanced visual understanding, yet perception-intensive
reasoning remains challenging. Recent latent visual
reasoning methods introduce hidden-space computation before answering, but they often rely on costly intermediate supervision, such as bounding boxes, sketches, or interleaved rationales. These strategies focus on how latent states should be shaped, but do not explicitly assess whether the latent is useful for the final answer. We propose LUT, a latent
reasoning framework trained with only standard VQA pairs. LUT centers training on Latent Utility at two levels. At the trajectory level, we propose Utility-Aware Latent
Distillation SFT, which explores answer-relevant latent trajectories, selects qualified trajectories by their information gain, and distills more reliable and learnable supervision through curriculum learning. At the step level, we propose Latent Attribution Policy Optimization, which uses answer-to-latent attribution to differentially optimize latent steps during reinforcement learning. Experiments on perception-intensive visual
reasoning benchmarks show that LUT outperforms previous latent
reasoning methods and remains competitive with latent-text interleaved methods with lower annotation cost.
Read the full abstract ▾
All authors Jiaxuan Kang, Siyu Chen, Mingda Li, Mingjie Liu, Tianyue Wang, Zhaoyang Wei, Yongheng Zhang, Yanchao Hao, Zheng Wei
Computer Vision
MDTD-ArtIR: Benchmarking Image Editing and Restoration Models for Art Image Restoration under Texture-Overlay Degradations
Mridula Vijendran, Shuang Chen and Hubert P. H. Shum
Restoring severely degraded visual media still remains a formidable challenge, as existing methods often hallucinate unnatural textures and contents, struggle with preserving color and texture, or fail to leverage partially retained image information. Existing restoration
benchmarks assume known degradation operators and fail to capture the complex characteristics of artistic damage such as cracks, stains, and color/texture deviation. We introduce a controlled
benchmark for blind restoration of semantic, semi-transparent image media degradations, accompanied by a new, publicly open degradation alpha texture mask
dataset MDTD-Art. We present a new
dataset and
benchmark evaluating state-of-the-art universal restoration models against image editing and
vision-
language models across varying mask opacity levels. Our experiments demonstrate that image editing models consistently outperform specialized restoration architectures for arbitrary degradations, with performance gains amplified by structured
prompt engineering emphasizing detail preservation and structural consistency. These findings position recoverable semantic information and
prompt controllability as critical factors in art image restoration.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Mitigating Backdoors via Decoy Shortcuts and Knowledge Decoupling
Zixuan Zhu, Rui Wang, Lihua Jing et al. (+1 more)
Backdoor attacks pose a serious threat to deep neural networks, especially when training relies on third-party data, allowing adversaries to inject malicious behaviors through data poisoning. In this work, we reveal that
backdoor behaviors tend to be absorbed by a simpler parallel branch when jointly trained with the main network. Motivated by this insight, we propose Trapping and Removing (TR), a simple yet effective training-time defense that introduces a lightweight shortcut branch as a "honeypot" to trap
backdoor knowledge. After training,
backdoors can be removed by discarding the shortcut, without requiring any additional data. To further enhance
backdoor isolation while maintaining benign performance, we design a knowledge decoupling strategy with
entropy-based weight assignment, encouraging poisoned samples to flow through the honeypot while guiding the main network to focus on benign learning. In addition, we introduce an automatic shortcut generation strategy to improve generalization across model architectures. Extensive experiments on four
benchmark datasets and five model architectures demonstrate that our approach effectively mitigates a wide range of
backdoor attacks while preserving performance on benign data. Code: https://github.com/Zixuan-Zhu/TR}{github.com/Zixuan-Zhu/TR.
Read the full abstract ▾
All authors Zixuan Zhu, Rui Wang, Lihua Jing, Jinwen Zhong
Computer Vision
Foveated Probes Recover Localized Binding Information in Vision Foundation Models
Mateusz Michalkiewicz, Mahsa Baktashmotlagh and Guha Balakrishnan
Frozen
vision foundation models are commonly evaluated through a single global image
embedding, but this interface can conflate missing information with information lost at readout time. We study this distinction by keeping a pretrained
vision encoder frozen and varying only the readout applied to its final patch
tokens. We compare standard global readouts against a lightweight foveated readout, which
attention-pools patch
tokens using a learned or question-conditioned query, and against an oracle readout with access to the annotated target region. We evaluate these interfaces on three localized binding problems: a controlled synthetic color–shape binding task under clutter, a color-free crowded shape-detection variant, and a GQA-derived natural-image task where paired questions ask for the colors of different same-category objects in the same image. Global readouts perform near perfectly when the synthetic target appears alone, but collapse under clutter and counterfactual target edits, whereas the foveated readout recovers most of the oracle-accessible signal. On the GQA-derived task, question-independent global image vectors improve only modestly over question-only priors, while question-conditioned foveation substantially improves paired localized color accuracy. A counterfactual nuisance-to-signal ratio explains the synthetic failures: global pooling dilutes localized label-changing evidence while exposing the probe to nuisance variation from irrelevant objects. These results indicate that apparent spatial blindness in frozen
vision models can arise from the global
embedding interface rather than from an absence of spatial information in the frozen patch
tokens.
Read the full abstract ▾
Computer Vision
Generated Images Are Easier to Forget: A Machine Unlearning Perspective for Synthetic Image Detection
Jun Nie, Yonggang Zhang, Tongliang Liu et al. (+3 more)
Robust detection of generated images is critical to counter the misuse of generative models. Existing methods primarily depend on learning from human-annotated training
datasets, limiting their generalization to unseen distributions. In contrast, large-scale
vision models (LVMs) pre-trained on web-scale
datasets exhibit exceptional generalization power through exposure to diverse distributions, offering a transformative paradigm for this task. However, our experimental results reveal that LVMs pre-trained on natural-image-dominated data can effectively capture the features of both natural and generated images, yielding comparably low losses and thus limited discriminative capacity between them. This
prompts a key question: When and how do LVMs exhibit different behaviors when capturing features of natural and generated images? This investigation reveals an insight: during unlearning, LVMs exhibit disparate forgetting dynamics with feature degradation for generated images escalating faster than natural ones. Inspired by the disparate dynamics, we introduce two detection methods: 1) data-free detection, which prunes model parameters to induce unlearning without data access, and 2) data-driven detection, which optimizes LVMs to unlearn knowledge tied to generated images. Extensive experiments conducted on various
benchmarks demonstrate that our unlearning-based approach outperforms conventional detection methods. By recasting the detection task as a problem of machine unlearning, our work establishes a new paradigm for generated image detection.
Read the full abstract ▾
All authors Jun Nie, Yonggang Zhang, Tongliang Liu, Yiu-ming Cheung, Bo Han, Xinmei Tian
Computer Vision
Coverage-Driven Adaptive Keyframe Selection for Video Understanding
Junyang Zhang, Puhan Luo, Chen Tang et al. (+2 more)
Recent advances in large
vision-
language models (LVLMs) have enabled long-
video understanding and analysis. However, processing the large number of frames in a
video incurs substantial computational overhead. Existing methods reduce LVLM inference costs by scoring frame-query relevance before inference and selecting keyframes accordingly. Nevertheless, the distribution of relevant frames varies across queries, and these methods often need to score hundreds or thousands of frames. To address this limitation, we propose CSES, a training-free semantic keyframe selector that adaptively determines the numbers of frames to score and keyframes to select. CSES estimates the prominence of the frame-query relevance profile to guide active acquisition and adapt the temporal coverage of each input. It then formulates keyframe selection as a coverage problem that jointly accounts for semantic relevance, temporal redundancy, and visual redundancy. Active acquisition and keyframe selection terminate based on coverage saturation. The selection objective is monotone and submodular, enabling greedy optimization with a standard approximation guarantee. Experiments with four LVLMs on two
benchmarks show that our method preserves accuracy while scoring $4$-$13\times$ fewer frames and selecting $18.4\%$-$20.5\%$ fewer input keyframes than existing baselines. CSES further achieves a $3.1$-$5.4\times$ speedup in frame selection over baselines.
Read the full abstract ▾
All authors Junyang Zhang, Puhan Luo, Chen Tang, Yuxi Shi, Xiang-Yang Li
Computer Vision
AeroLLE: Constrained Pseudo-Supervision for Nighttime Aerial Image Enhancement with the AeroNight-1.5K Benchmark
Wei Lu, Hongyuan Liu and Si-Bao Chen
Nighttime aerial image enhancement is challenged by spatially nonuniform exposure, mixed illumination, and weak structural evidence, while registered normal-light targets are difficult to capture from moving platforms. Generated normal-light images provide practical appearance guidance but may alter geometry or texture. We introduce \aeronight{}, comprising 1,500 real nighttime aerial RGB images: 1,300 inputs are associated with manually screened pseudo-references, and 200 inputs support unpaired evaluation. We propose AeroLLE, a two-stage framework that first recovers visibility with an HVI Base Enhancer and then performs Spatially Adaptive Exposure–Color Calibration (SAECC). After the Base Enhancer is selected and frozen, SAECC predicts bounded, low-resolution RGB gain and bias fields, restricting the magnitude and spatial variation of the second-stage correction. Experiments under complementary pseudo-paired and unpaired protocols demonstrate improved agreement with screened appearance targets, together with more balanced exposure and color correction across diverse nighttime aerial scenes. These results support constrained, stage-specific calibration as a practical strategy for learning from generated appearance guidance when registered aerial references are unavailable.
Read the full abstract ▾
Computer Vision
FreqAnchorAD: Language-Free Zero-Shot Anomaly Detection via Frequency-Deviation Anchoring
Jianfeng Qiu, Peiyuan Li, Juan Xie et al. (+4 more)
Zero-shot anomaly detection (ZSAD) aims to detect anomalies and localize defective regions in unseen target domains without target training data. Recent ZSAD methods build on pretrained
vision models, particularly CLIP, and construct normal and anomaly references from textual
prompts or learnable visual representations. These methods perform anomaly discrimination primarily in spatial feature spaces, where subtle changes in texture, boundaries, and local structures can be confused with normal appearance variations. Although inconspicuous spatially, such defects can disrupt local texture regularity or boundary continuity, inducing response deviations across frequency bands. However, existing ZSAD methods do not explicitly model these frequency-dependent characteristics. Our image-domain analysis reveals that local defects exhibit spatial-frequency deviations from normal references across low-, middle-, and high-frequency bands, indicating that anomaly evidence is not universally dominated by high-frequency responses. Motivated by this observation, we propose FreqAnchorAD, a frequency-aware framework that organizes frequency-enhanced responses for anchor-relative anomaly discrimination. Specifically, the Local Frequency Compensation Module (LFCM) enhances intermediate patch
tokens with local spatial-frequency cues. The Frequency-Deviation Anchor Projector (FDAP), our core discrimination module, organizes enhanced responses along a source-derived channel coordinate and measures anomaly evidence through relative similarity to normal and anomaly anchors. Finally, Asymmetric Anchor Supervision (AAS) stabilizes normal-anchor
alignment while preserving diverse anomaly patterns. Experiments on thirteen industrial and medical
benchmarks show that FreqAnchorAD achieves state-of-the-art mean performance in image-level anomaly recognition and pixel-level defect localization.
Read the full abstract ▾
All authors Jianfeng Qiu, Peiyuan Li, Juan Xie, Xueliang Ma, Sihang Zhou, Yanning Hou, Ke Xu
Computer Vision
E2Pano: Learning Event-to-Panorama Image Reconstruction
Zhenyang Li, Zongqi He, Jia Pan et al. (+2 more)
Event cameras offer microsecond-level temporal resolution and high dynamic range, potentially facilitating motion-blur-free panoramic imaging from fast rotational scanning. Nonetheless, existing optimization-based methods remain computationally demanding, while prior learning-based reconstruction methods are largely designed for perspective imagery and lack geometry-aware support for panoramic outputs. We present E2Pano, a geometry-guided event-to-panorama pipeline with an end-to-end learnable photometric reconstruction stage. Our framework preserves real spherical coordinates from
geometric mapping throughout the pipeline, employs a lightweight enhancement module with frequency-domain supervision to bridge the event-image domain gap, and leverages a spherical
Transformer with 3D positional
embeddings for photometric reconstruction. Experiments on synthetic data and captured rotational scans show improved reconstruction quality and lower photometric reconstruction cost than optimization-based baselines, together with encouraging transfer to real captures under our acquisition protocol despite training purely on synthetic data. Additionally, we construct PanoScan, a
dataset with 4,370 synthetic and 30 real-world panoramic scenes paired with event streams. Our
dataset and code will be released.
Read the full abstract ▾
All authors Zhenyang Li, Zongqi He, Jia Pan, Shijie Lin, Yifan Peng
Computer Vision
Proteus: A Truncation-Robust Entropy Model for Progressive LiDAR Compression
Yihan Qiu, Xiaodong Lin, Baoquan Zhao et al. (+2 more)
LiDAR point clouds provide explicit, deterministic physical boundaries critical for collaborative safety-critical perception. However, wireless channels inherently impair and corrupt transmitted signals. Existing robust frameworks (such as deep JSCC or MDC) attempt to counter these channel impairments through statistical or parametric estimation, turning exact physical measurements into unverified algorithmic estimates. To address this, we propose Proteus, a learned
LiDAR codec operating on 2D range images. By decoupling the frame representation into independent coders for the significant range bit-planes (SIG) and the insignificant range bit-planes and attributes (INS), Proteus achieves overall stream-level truncation robustness. The non-truncatable SIG block encodes the most significant range bit-planes to establish a necessary, self-contained perceptual lower bound, below which the reconstructed point cloud is severely degraded. Meanwhile, INS employs bit-plane slicing representation and coding, ensuring that range truncation mathematically maps to a deterministic spatial precision degradation. Subordinate attributes are reconstructed via a hybrid lossless-predictive method, leveraging the decoded geometry as a strong structural prior for fine-grained approximation. Furthermore, strategic ordering within INS prioritizes geometry over attributes under bandwidth drops. Experimental results on the Waymo Open
Dataset and SemanticKITTI demonstrate that Proteus tolerates up to approximately 70% bitstream truncation, while outperforming established standards (G-PCC, Draco, and JPEG XL) and the representative learned compressor Unicorn under ideal channel conditions.
Read the full abstract ▾
All authors Yihan Qiu, Xiaodong Lin, Baoquan Zhao, Hailong Jiao, Ge Li
Computer Vision
BRIC-Net: Boundary-Reliable Illumination-Color Interaction for Remote Sensing Image Deshadowing
Wei Lu, Yi Liu and Si-Bao
Shadows in remote sensing images obscure surface appearance and disrupt radiometric continuity, reducing the reliability of visual interpretation and downstream analysis. Remote sensing image deshadowing is an ill-posed inverse problem that requires spatially varying illumination recovery while preserving chromatic and radiometric consistency in non-shadow regions. Existing methods commonly rely on hard shadow masks for compensation or directly regress RGB intensities. Hard masks may inadequately model gradual penumbra variations and are sensitive to localization errors, often producing residual shadows or
halo artifacts; direct RGB regression entangles illumination recovery with chromatic reconstruction and can introduce color casts. To this end, we propose the Boundary-Reliable Illumination-Color Interaction Network (BRIC-Net), which decouples these failures at different representation levels. A Lightness Reliability Prior (LRP) derives reliability-aware guidance from CIELAB statistics. Boundary-Adaptive Gated Mixing (BAGM) performs gated interpolation between shallow RGB and lightness features around uncertain transitions, while Spatial-Channel Mutual Modulation (SCMM) coordinates deeper spatial and channel responses for appearance-preserving illumination recovery. BRIC-Net achieves 29.46~dB full-image peak signal-to-noise ratio (PSNR) on AeroDS-Syn and 27.96~dB on SRGTA. It also obtains the lowest Perception-based Image Quality Evaluator (PIQE) scores on AISD and AeroDS-Real. Region-wise evaluations and component ablations further support its effectiveness in shadow recovery and non-shadow preservation.
Read the full abstract ▾
Computer Vision
Breaking the Horizontal Prior: From Long-Tailed Orientation Bias to Roll-Robust Monocular Depth Estimation
Kaihua Tang, Ziqing Xia, Xiaoxu Zheng et al. (+4 more)
Despite recent advances in
Monocular Depth Estimation, state-of-the-art depth foundation models remain vulnerable to robustness issues. Particularly, even slight camera rolls can result in substantial degradation in depth estimations. We attribute this problem to a previously overlooked phenomenon, termed the Horizontal Prior, which is a manifestation of long-tailed distribution bias: most training images are captured in approximately horizontal orientations due to human visual preferences and photographic habits. While intuitive remedies such as re-balanced data augmentation and horizon leveling provide partial improvements, they fail to fully address the issue. In this paper, we introduce Invariant Depth Constraint (ID-Constraint), a training-time supervision strategy that improves roll robustness by fine-tuning and jointly regularizing the depth backbone with a series of
geometric and spatial
reasoning tasks. These auxiliary objectives encourage the backbone to learn rotation-stable, depth-relevant representations, while the auxiliary prediction heads are discarded after training, leaving the original inference architecture unchanged. Extensive experiments on five
benchmark datasets across four roll settings demonstrate the effectiveness of the proposed method.
Read the full abstract ▾
All authors Kaihua Tang, Ziqing Xia, Xiaoxu Zheng, Xiaoxue Zhang, Michael Bi Mi, Zhan Xu, Dave Zhenyu Chen
Computer Vision
CopyCat: Improving Fine-Grained Subject Consistency in Subject-to-Image Models within Seconds
Peng Zheng, Ruiqi Liu, Rui Ma et al. (+1 more)
Recent subject-to-image models have achieved impressive progress in personalized image generation, yet they still struggle to preserve fine-grained subject-specific details. A major reason is the lack of high-quality fine-grained identity supervision: real paired data are expensive to collect, while synthesized training pairs often preserve only coarse subject appearance and fail to capture subtle subject-specific details. In this work, we propose CopyCat, a lightweight model-refinement framework that improves fine-grained subject consistency within only a few seconds. CopyCat performs a one-time refinement of a pretrained subject-to-image model by attaching a lightweight Fine-grained Consistency LoRA (FCLoRA) and optimizing it using a single proxy image, which is used as both the conditioning image and the reconstruction target. This exact self-reconstruction objective substantially simplifies the optimization task, enabling effective fine-grained refinement within only a few seconds. The refinement is performed only once; the resulting model can be directly applied to diverse unseen reference subjects and
prompts without further subject-specific optimization. We further revisit subject-to-image LoRA training in double-stream
diffusion transformers and find that adapting only the visual stream consistently improves subject consistency. Extensive experiments on DreamBench and XVerseBench demonstrate consistent improvements in fine-grained subject consistency across representative subject-to-image models under both single- and multi-subject settings.
Read the full abstract ▾
All authors Peng Zheng, Ruiqi Liu, Rui Ma, Zuxuan Wu
Computer Vision
Geometry-guided Emotion Modulation for Controllable and Photorealistic Emotional Talking Face Generation
Chenggong Hu, Shaoyin Ma, Yi Wang et al. (+3 more)
Audio-driven emotional talking face generation aims to synthesize realistic
videos with expressive facial dynamics. However, existing methods struggle to balance controllability and visual fidelity. Although implicit representations capture rich semantics, they lack structural guidance, often resulting in averaged emotional expressions. In contrast, explicit
geometric methods offer better control over facial expressions but tend to sacrifice high-frequency texture details. To address it, we propose GemTalk, a
diffusion-based framework that combines the semantic richness of implicit representations with the structural precision of explicit
geometric priors. We introduce a
Vision-guided Audio
Emotion Projection (V-AEP) module to extract implicit emotional lip and expression features. At the same time, a
Diffusion-based
Geometric Priors Generator (D-GPG) generates identity-aware blendshape coefficients as explicit structural priors. Crucially, our Geometry-guided
Emotion Modulation (GEM) module leverages these
geometric priors to recalibrate the magnitude of implicit features, enabling precise, continuous control over emotional expressions, especially
emotion intensity, without sacrificing visual quality. Extensive experiments show GemTalk achieves superior performance in photo-realism, and facial emotional dynamics.
Read the full abstract ▾
All authors Chenggong Hu, Shaoyin Ma, Yi Wang, Li Sun, Mingli Song, Jie Song
Computer Vision
PixelSR: Efficient Screen Content Super-Resolution via Pixel Classification
Zhiheng Li, Lei Chen, Jie Zhou et al. (+1 more)
Screen content images are generally composed of texts and graphics. Compared to natural images, these man-made images contain a large quantity of sharp but repetitive structures. However, existing works in screen content super-resolution underutilize the special characteristics of screen content, leaving a large room to improve model performance and speed up. In this paper, we propose PixelSR, a simple yet effective method to improve super-resolution performance but with faster inference speed. To improve model performance, we classify pixels via pixel binning to compute content
attention in the training phase. Specifically, after binning pixels into content-dependent groups, content
attention is aggregated from pixel features within each group to introduce a content-dependent and non-local receptive field for every pixel. In the testing phase, we utilize the properties of self-repetitiveness and redundancy in screen content to speed up inference without the loss of model performance. We divide targeted high-resolution pixels into three types, which are unique pixels, repeated pixels, and background pixels for each test image. We conduct conventional network processing on unique pixels and cache their predictions in the on-the-fly lookup table. For repeated pixels which have appeared in unique pixels, we directly retrieve prediction results from the lookup table without network processing. For background pixels, we use the nearest neighbor
algorithm to generate high-resolution pixels. The on-the-fly lookup table is cleaned and repeats the procedure above for the next test image. Experiments show our PixelSR achieves state-of-the-art performance with shorter inference time in screen content super-resolution.
Read the full abstract ▾
All authors Zhiheng Li, Lei Chen, Jie Zhou, Jiwen Lu
Computer Vision
WiFuse: An Attention Mechanism for Human Activity Recognition using Fused CSI Amplitude and Delay-Doppler Channel Features
Alison M. Fernandes, Hermes I. Del Monego, Bruno S. Chang et al. (+3 more)
Recently, Wi-Fi sensing has played a significant role in Human Activity Recognition (HAR), as it enables the detection of various activities using only Wi-Fi signals, ensuring
privacy and remaining non-intrusive for the user. However, environmental characteristics such as reflective surfaces, hardware offsets, and other physical impairments affect recognition by the neural network, subsequently causing errors and significantly reducing model accuracy. To overcome this problem we present the WiFuse framework, a dual-stream Channel State Information (CSI) framework for human activity recognition (HAR) that pairs denoised time-domain amplitude variations with 2D-FFT-derived Delay-Doppler motion representations computed from the sanitized channel phase. The fused representation feeds a hybrid ResNet-Temporal Convolutional Network (TCN) neural architecture augmented with channel and spatio-temporal
attention, where the ResNet extracts spatial-spectral features and the TCN models long-range temporal dependencies; a decoupled two-stage transfer learning strategy is employed to improve optimization stability and feature reuse. We conduct extensive experiments on two public
datasets, including comparisons against state-of-the-art methods and alternative hybrid architectures, ablation studies, and cross-
dataset and domain-adaptation evaluations. The proposed framework reaches an overall accuracy of up to 95.28% across the four environments of the XRF55
dataset and up to 98.20% on the multi-user Wi-MIR
dataset. Overall, the results indicate that combining amplitude and Delay-Doppler representations within a dual-stream strategy, enhanced by transfer learning, improves recognition performance under conditions that typically degrade deep neural networks, such as class overlap, multipath propagation, noise, and interference.
Read the full abstract ▾
All authors Alison M. Fernandes, Hermes I. Del Monego, Bruno S. Chang, Anelise Munaretto, Hélder M. Fontes, Rui L. Campos
Computer Vision
Where Does Generative Difficulty Reside? An Empirical Study of Target Representations
Marcel Plocher, Bernhard Schölkopf, Andreas Geiger et al. (+1 more)
The target representation defines the distribution an image generator must learn, yet it is often treated as an interchangeable interface. This assumption is particularly questionable for continuous masked generators, which combine contextual inference from visible
tokens with conditional modeling of each missing
token. We study raw pixels, SD-VAE latents and DINOv2 as well as MAE representation-autoencoder features within a unified masked autoregressive rectified-flow model. Under a shared ImageNet training budget, these spaces exhibit distinct optimization and inference regimes. DINOv2 converges fastest in both iterations and computation but benefits strongly from a wider local denoiser and direct context fusion. Pixels optimize substantially more slowly and require a different prediction, masking, and guidance configuration. MAE reconstructs images more faithfully and exhibits clear semantic clustering, yet produces generations substantially worse than DINOv2. The representations also respond differently to classifier-free guidance and occupy distinct precision-recall trade-offs. Together, our results show that compression, reconstruction fidelity,
token dimensionality, and visible semantic clustering do not individually predict generative behavior. Instead, target representations redistribute difficulty across contextual modeling, per-
token denoising, and inference-time distributional control.
Read the full abstract ▾
All authors Marcel Plocher, Bernhard Schölkopf, Andreas Geiger, Gege Gao
Computer Vision
Diagnosing Under-Development of Irreversible Processes in Video Generation
Jian Xu, Yanning Wu, Delu Zeng et al. (+2 more)
Many physical attributes are irreversible: ice melts but does not re-freeze, paper chars but does not un-burn. Do
video generators respect this? We show the question is hard to measure, and that what can be measured reliably is development rather than reversal. Metrics of local reversal are null-degenerate: a per-clip violation rate scores $0.50$ on pure noise, and a variance-normalized reversal residual sits at its noise ceiling. What survives null-testing is a two-part protocol: progress (a directional attribute correlation) and a stasis rate. Under this protocol, generated
video separates cleanly from real footage, and the gap is human-validated. Across seven text-to-
video models, real reference footage advances ($\rho{=}{+}0.40$, $35\%$ static) while every generator shows near-zero progress and $92$–$100\%$ stasis; nine annotators rate real footage far above generated ($2.75$ vs.\ $0.99$ on a $0$–$4$ scale). The reliable finding is under-development: generators barely advance irreversible attributes rather than reversing them. As a complementary mechanism, we show that post-hoc readout guidance is gameable, whereas enforcing monotonicity by construction in a disentangled attribute latent removes the gameable readout, validated in controlled and semi-synthetic settings.
Read the full abstract ▾
All authors Jian Xu, Yanning Wu, Delu Zeng, John Paisley, Qibin Zhao
Computer Vision
InstancePin: Instance-Addressable Layout-to-Image Diffusion via Coordinate Pinning
Chaoyue Wu, Yunfei Zhang and Si Wu
Layout-to-image
diffusion models have achieved impressive semantic controllability by conditioning generation on category-level
segmentation maps. However, such category-aligned control is not necessarily instance-addressable: multiple nearby objects from the same category are often treated as a shared semantic region, leading to ambiguous boundaries, averaged appearances, and feature confusion among instances. This limitation is particularly evident in urban scene synthesis, where small and crowded pedestrians or vehicles require fine-grained instance separation while preserving global scene consistency. In this paper, we propose InstancePin, an instance-addressable layout-to-image
diffusion framework that pins each object instance with an explicit coordinate anchor. Instead of directly injecting instance masks into the pretrained backbone, InstancePin introduces an independent instance-aware adapter to preserve the category-level generation prior while learning instance-specific spatial control. For each instance, its center coordinate is encoded with
Fourier features and projected into a coordinate
token, which serves as a spatial anchor queried by latent image features through coordinate pinning
attention. To make these anchors spatially meaningful, we further supervise the coordinate
attention maps with instance regions, encouraging each coordinate
token to activate its corresponding object area. Finally, an instance-mask guided fusion module routes pretrained backbone features to non-instance regions and adapter features to instance regions, enabling local instance refinement without sacrificing global semantic fidelity. Extensive experiments on Cityscapes demonstrate that InstancePin mitigates instance
entanglement in dense layouts and improves both image fidelity and semantic consistency.
Read the full abstract ▾
Computer Vision
Representation Transfer of Foundation Models for Ultra-Widefield Retinal Imaging
Mingya Alexa Gong, Da Ma, Lovre Antonio Budimir et al. (+7 more)
Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pretraining strategies influence the transferability of learned representations to weakly supervised ophthalmic imaging tasks. We investigate this question in ultra-widefield (UWF) retinal imaging by evaluating foundation model representations within a patch-based multiple instance learning (MIL) framework for disease classification on UWF images. We compare
Vision Transformer encoders pretrained with supervised, Masked Autoencoder (MAE), and self-
distillation objectives, while keeping the downstream aggregation architecture unchanged. Within a controlled comparison of ViT-B encoders pretrained on ImageNet-1k, the choice of pretraining objective substantially influenced frozen representation transfer, with supervised and self-
distillation-based models outperforming MAE. A contemporary DINOv3 model pretrained at a larger scale achieved the strongest overall performance, with a quadratic weighted kappa of 0.863 for five-class diabetic retinopathy grading, comparable with DINOv1.
Attention analysis further revealed distinct patch-aggregation behaviours associated with the different pretrained representations, while partial fine-tuning substantially reduced the performance gap for MAE. These findings suggest that pretraining strategy influences both representation transferability and the subsequent aggregation of patch-level evidence within MIL, resulting in differences in downstream classification performance.
Read the full abstract ▾
All authors Mingya Alexa Gong, Da Ma, Lovre Antonio Budimir, Ivana Matovinovic, Sven Loncaric, Myeong Jin Ju, Yukun Zhou, Siegfried K. Wagner, Pearse A. Keane, Marinko V. Sarunic
Computer Vision
Element-Aware Group Learning for E-Commerce Image Generation
Jingtong Chen, Jiahui Wang, Xue Zhao et al. (+2 more)
Recent advances in image generation and editing have made
prompt quality a key bottleneck for e-commerce creatives.
Vision-
language models (VLMs) can generate image-editing
prompts from product images and metadata, but further improving their
prompt-writing capabilities requires post-training with feedback from the generated images. Group Relative Policy Optimization (GRPO) is a natural framework for such outcome-level reward optimization. However, it assigns credit only at the full-
prompt level, even though image quality often depends on specific design elements such as composition, background, and the presentation of selling points. Existing fine-grained credit assignment methods typically require step-level supervision or learned critics. To address this, we propose EAGLE-GRPO (Element-Aware Group Learning for E-Commerce Image Generation), which decomposes the group-centered reward over predefined elements. We cast element-level credit assignment as a kernel ridge regression problem and derive a closed-form solution, without additional rollouts or separate credit-assignment models. This yields interpretable per-element advantages and more precise policy updates. Experiments show that EAGLE-GRPO sustains performance gains over more training steps before plateauing and generates
prompts that produce higher-quality e-commerce images than competitive VLM
prompt-writing baselines.
Read the full abstract ▾
All authors Jingtong Chen, Jiahui Wang, Xue Zhao, ShaoGuo Liu, Minghao Li
Computer Vision
Relax Within, Balance Across: Geometry-Guided Load Balancing for Vision-Language Mixture-of-Experts
Ziang Wu, Peng Jin, Qishen Yin et al. (+4 more)
Vision-
language MoE batches contain different numbers of image and text
tokens. Image resolution, image count, tiling, and
prompt length all change this
token mix. We call the standard
token-level Switch auxiliary loss Std-Aux. Std-Aux balances only the mixed load, so large image and text load errors can cancel at one mix. On our main model, the same trained router shows more than a fivefold change in load imbalance across image resolutions. We hold the image and text load profiles fixed and derive the exact load curve as the
token mix varies. The image-text load gap controls sensitivity to the
token mix. Physical preprocessing can also change the conditional profiles. The fixed-profile law excludes such changes. To design a remedy, we examine the router input structure. Image and text occupy distinct regions, while visual
tokens group strongly by source image. The modality boundary motivates separate image and text terms. The image boundary motivates one equal-weight routing instance per image. ReBA, or Relax Within, Balance Across, implements both choices. Across four split backbones, ReBA lowers load on every reported
benchmark input while keeping mean task accuracy comparable to Std-Aux. ReBA also lowers average load over the tested range and worst physical load under resolution and tiling shifts. Code is available at https://github.com/ZiangWu-77/ReBA.
Read the full abstract ▾
All authors Ziang Wu, Peng Jin, Qishen Yin, Munan Ning, Hao Li, Peizhen Zhang, Li Yuan
Computer Vision
Beyond Token-Level Cross-Entropy: Fréchet Distributional Post-Training for Autoregressive Image Generation
Jinhua Zhang, Yisong Lin, Wei Long et al. (+1 more)
Autoregressive image generators are commonly pretrained with
token-level cross-
entropy under teacher forcing, yet evaluated by the distributional quality of decoded images. This creates an objective mismatch, because categorical errors have unequal image-level consequences, and a context mismatch, because inference conditions on model-generated histories. We introduce FD-loss post-training, which adapts a pretrained discrete generator using representation-space Fréchet distance as the sole objective. A dual-pass scheme first constructs detached rollout contexts through gradient-free generation under the model's native inference configuration, then performs differentiable replay with a probability-level straight-through
estimator (STE) that preserves hard argmax decoding in the forward pass while propagating image-level gradients through temperature-scaled probabilities. Only the generator is updated, while the tokenizer and feature extractors remain frozen. Across eight completed configurations from four generator families on class-conditional ImageNet at $256\times256$, FD-loss post-training reduces FID and $\mathrm{FD}_{r6}$ by 41.4% and 52.0% on average. The strongest FID result improves from 2.42 to 1.43 without adding parameters or inference steps.
Read the full abstract ▾
All authors Jinhua Zhang, Yisong Lin, Wei Long, Shuhang Gu
Computer Vision · Cross-listed from cs.AI
Through the LENS: Local Geometric Decomposition of Vision-Language Model Representations
Shalom Kachko, Raz Lapid, Margarita Vald et al. (+2 more)
Vision-
language models (VLMs) process image patches and text
tokens in a shared residual stream, but the local geometry through which the two modalities interact remains poorly understood. Most interpretability methods identify global linear directions, which may miss representations that are globally high-dimensional but locally low-dimensional. We introduce LENS (Local Explanation of Neighborhood Subspaces), a method that decomposes VLM activations into local low-rank Gaussian neighborhoods using a Mixture of Factor Analyzers. Applied to LLaVA-1.5-7B and Qwen3-VL-8B, LENS reveals distinct depth-dependent fusion trajectories consistent with each model's fusion mechanism: LLaVA progressively mixes modalities at later layers, whereas Qwen3-VL mixes them early, partially re-segregates them, and recombines them near the output. An automated
multimodal labeling pipeline assigns concise semantic descriptions to these neighborhoods. Interpolating activations toward neighborhood centroids causally redirects generation within and across modalities and outperforms difference-in-means and VL-SAE in most evaluated conditions; in one LLaVA
vision-to-
vision setting, MFA achieves 5.7 times the VL-SAE score. Human evaluation finds MFA steering competitive with prompting and substantially stronger than the other intervention baselines. Finally, the MFA coefficient space improves Qwen3-VL image-to-rendered-text
retrieval at the deepest evaluated layer from 14.9% to 48.6% R@1. Ablations show that the reported fusion trajectories are stable across component counts, local ranks, and modality-purity thresholds. These results support local
geometric neighborhoods as useful interpretable and causal units for analyzing cross-modal representations in the evaluated VLMs.
Read the full abstract ▾
All authors Shalom Kachko, Raz Lapid, Margarita Vald, Almog Dubin, Moshe Sipper
Computer Vision
Test-Time Curriculum for Open-Set AIGC Detection
Yiqian Zhang, Zheyuan Gu, Xiangzhao Hao et al. (+8 more)
AI-generated image detectors deployed in open-world environments inevitably face distribution shifts as new and stronger generative models continue to emerge. Although existing methods improve cross-generator generalization through better representations or training data construction, they typically follow a static train-once-and-deploy paradigm and cannot adapt after deployment. In this work, we study open-set AIGC image detection from a test-time adaptation perspective. We propose Test-Time Curriculum (TTC), a simple and model-agnostic framework that adapts a detector on unlabeled test data through curriculum-based self-training. TTC starts from highly reliable pseudo-labeled samples and progressively incorporates harder yet informative cases, while enforcing class-balanced selection to reduce biased updates under generator shift. To further improve pseudo-label quality, we introduce Cross-Scale Pseudo-Label Refinement, which aggregates complementary evidence across multiple resolutions for more reliable adaptation, and applies noisy-or fusion at inference to strengthen final predictions. In addition, we construct AIGCGuard, a new
benchmark containing 3,100 representative real images and 124,000 generated images from 40 of the most advanced open-source and proprietary text-to-image models. Extensive experiments on five
benchmarks show that TTC substantially improves overall detection performance under diverse unseen-generator shifts, establishing a practical and effective test-time adaptation framework for open-set generated image detection.
Read the full abstract ▾
All authors Yiqian Zhang, Zheyuan Gu, Xiangzhao Hao, Zefeng Zhang, Jingjia Mao, Jiahao Hu, Jiaxu Miao, Jun Yu, Zhenyu Zhang, Shuohuan Wang, Yu Sun
Computer Vision · Cross-listed from cs.RO
DexMani: Human-Derived Manipulability Guidance for Dexterous Rotation
Xiaoyang Chen, Shengcheng Luo, Haoran Guo et al. (+4 more)
Dexterous object rotation is a sequential contact problem: each support, release, and re-contact decision must both produce the desired object motion, and prepare the hand configuration for continued rotation. Existing reinforcement learning methods discover such movement patterns through trial and error on specific robotic hand embodiments, without explicitly accounting for how each contact transition affects the hand's ability to sustain object rotation in subsequent steps. We introduce DexMani, a framework that transfers human demonstrations as contact-conditioned manipulability
evolution. This prior captures how successful human contact transitions reshape the object-rotation directions available to the hand. DexMani then learns this manipulability
evolution and uses it to guide downstream reinforcement learning, enabling rotation skills to be acquired across
robot embodiments with distinct kinematics and active-contact configurations. Across the Shadow Hand, Allegro Hand, and XHand, DexMani achieves the highest success rates in every evaluated setting for both seen and unseen objects. DexMani reaches an average success rate of 57.5% on LEAP Hand, outperforming other baselines and producing smoother rotatory motions. Project site: https://dexmani.github.io
Read the full abstract ▾
All authors Xiaoyang Chen, Shengcheng Luo, Haoran Guo, Jiaming Jiang, Wanlin Li, Ziyuan Jiao, Chenxi Xiao
Computer Vision
DrawAI: Agentic Benchmark and Workflow for Making Raster Images Editable
Pu Cao, Qingye Kong, Xuedan Yin et al. (+5 more)
Recent image-generation models and
multimodal agents can produce high-quality visuals for increasingly complex visual communication tasks. Yet their raster outputs remain difficult to use directly because meaningful content and relationships are flattened into pixels, preventing users from inspecting, modifying, rearranging, or reusing individual components. We formulate image-to-editable reconstruction, which recovers a structured, directly manipulable artifact from a raster image while preserving its visual and semantic content. The central challenge is to jointly satisfy Fidelity and Editability, which often trade off in practice. To study this task, we introduce DrawAI, comprising an agentic
benchmark, DrawAI-Bench, and a reconstruction workflow, DrawAI-Flow. DrawAI-Bench spans scientific figures, presentation slides, posters, and diagrams, combining real and AI-generated images to reflect practical visual-creation scenarios. It evaluates Fidelity and Editability through a hybrid protocol of 39 criteria: deterministic rule-based metrics measure properties with direct correspondences, while asset-specific
vision-
language rubrics capture semantic and perceptual qualities for which exact matching is misleading. Besides, we propose DrawAI-Flow, a two-stage agentic workflow in which a Parser
Agent turns extracted elements evidence into an explicit reconstruction plan, and a Reconstruction
Agent realizes the plan as executable graphics code through an iterative code-render-validate-revise loop. On DrawAI-Bench, we systematically evaluate thirteen models across five
agent harnesses to study the effects of model capability, harness choice, and workflow design. The results show that reconstruction quality and costs vary substantially across model-harness configurations, while DrawAI-Flow consistently improves editable structure.
Read the full abstract ▾
All authors Pu Cao, Qingye Kong, Xuedan Yin, Xuekun Zhao, Rupeng Yan, Qing Song, Yao Zhang, Lu Yang
Computer Vision
Zero-Cost Virtual RNA: Approximating Immunotherapy Signatures via Cross-Modal WSI Retrieval
Sigrid Vila-Bagaria, Mar Teixidó, Miquel Piñol et al. (+3 more)
Identifying the “Inflamed” immunophenotype in Gastric Adenocarcinoma predicts immunotherapy response but requires an expensive 10-gene RNA signature. While deep learning on standard H&E slides offers a scalable alternative, conventional binary classifiers oversimplify continuous RNA data and introduce label noise. To resolve this, we propose VITA (VIrtual Transcriptomic Approximation). By aligning H&E and RNA into a joint latent space during training, VITA requires only standard H&E at inference to retrieve morphologically similar historical cases and approximate the continuous RNA signature. Achieving 0.72 classification accuracy and a 0.66 Spearman correlation, VITA provides a cost-effective “virtual transcriptomics” pre-screening tool that preserves the continuous phenotypic spectrum without requiring genomic sequencing.
Read the full abstract ▾
All authors Sigrid Vila-Bagaria, Mar Teixidó, Miquel Piñol, Felip Vilardell, Robert Montal, Veronica Vilaplana