Computer Vision
Multimodal Group Emotion Recognition In-the-Wild Towards a Privacy-Safe Non-Individual Approach
Anderson Augusma
This thesis addresses group
emotion recognition (GER) in-the-wild with a focus on
privacy preservation. Unlike traditional
emotion recognition methods that rely on individual-level cues such as face, gaze, or voice analysis, this work uses collective audio-
video signals to infer
emotions at the group level, reducing risks of individual monitoring and surveillance. Two complementary frameworks are proposed. The first is a cross-
attention multimodal architecture for audio-
video fusion, combined with Frames
Attention Pooling (FAP) for temporal aggregation. It is supported by synthetic data augmentation and validated through ablation studies, demonstrating robustness in real-world GER conditions. The second framework, Variational Encoder Multi-Decoder (VE-MD), learns a shared latent space for
emotion classification and structural representation prediction, including body and face cues. Two decoding strategies, DETR-based and heatmap-based, are explored to analyze the role of structural representations in group and individual settings. The thesis makes three main contributions: it clarifies the role of multimodality and structural cues in group-level affective computing; introduces two architectures for
privacy-preserving
multimodal GER; and shows that competitive performance can be achieved without using individual features as input data.
Read the full abstract ▾
Computer Vision
Physics from Video: Identifiability of Time-Invariant Second-Order ODEs under Minimal Trajectory Conditions
Yuanyuan Wang, Wenjie Wang, Kun Zhang et al. (+1 more)
Bridging the gap between visual realism and physical understanding is a core challenge for
video-based world models. We study the structural identifiability of continuous-time physical laws from raw pixels, focusing on whether an encoder-only pipeline can uniquely recover the parameters of second-order linear ODEs. We prove that a level-set slope-coverage condition ensures the learned latent space is locally affine to the true physical state, enabling exact parameter recovery. Our theory provides the first characterization of minimal data requirements across damping regimes, establishing that underdamped systems are identifiable from a single
video clip, whereas other regimes require three diverse trajectories. We further introduce a variance-floor regularizer to stabilize the decoder-free objective and prevent latent collapse. Validated on synthetic and real-world data, our approach demonstrates that interpretable physical constants can be reliably estimated from
video without the need for compute-intensive pixel reconstruction, ensuring both physical correctness and transparency. Code is available at https://github.com/wenjiewang3/PhysicsFromVideo.
Read the full abstract ▾
All authors Yuanyuan Wang, Wenjie Wang, Kun Zhang, Mingming Gong
Computer Vision
Recursive Vision Transformer with Dynamic Depth and Width Adjustment for Resource-Efficient Image Semantic Communication
Zhilong Zhang, Xinhui Zhang, Gongyu Jin et al. (+3 more)
Image semantic communication is a critical component in next-generation wireless communication systems. However, such systems typically suffer from large
memory footprints and high computational complexity, making them difficult to deploy on resource-constrained devices. To address these challenges, we propose a
vision transformer (ViT)-enabled image semantic communication system. In this system, a recursive structure is introduced to iteratively refine semantic features and reduce the parameter count. In addition, three dynamic adjustment strategies are designed to adaptively reduce computational complexity: dynamic depth adjustment, dynamic width adjustment, and joint width-depth optimization. Dynamic depth adjustment adaptively determines the number of recursive modules according to image content and channel conditions, while dynamic width adjustment selectively preserves important
neurons and
attention heads. The joint width-depth optimization further enables flexible computation configurations. Simulation results verify that the proposed recursive ViT-based system, combined with the three dynamic adjustment strategies, reduces the parameter count by 48.7% and achieves higher reconstruction quality than existing baselines under comparable computational complexity.
Read the full abstract ▾
All authors Zhilong Zhang, Xinhui Zhang, Gongyu Jin, Sihua Wang, Danpu Liu, Changchuan Yin
Computer Vision · Cross-listed from cs.NE
Evolving to the Aesthetics of a Vision-Language Model
Stephen James Krol and Jon McCormack
Evolutionary systems have demonstrated remarkable results in creative domains, with recent applications in generative typography, design, and music. However, an open problem remains in designing fitness functions that effectively capture the desired aesthetics of abstract outputs. In this work, we explore two methods for evaluating the aesthetics of a population using
Vision-
Language Models (VLMs). The first method uses CLIP-IQA to predict an aesthetic score for each design. The second method instead pits candidates against each other, with winners determined by a VLM using a custom
prompt specified by the user. The outcomes of these pairwise comparisons are then used to estimate a population ranking via the Glicko rating system. We present these methods in the context of a case study using a custom generative system and compare the resulting rankings with an artist's aesthetic ranking and those produced by other aesthetic evaluation techniques. Additionally, we document the artist's experience using these approaches to evolve designs, critically analysing the strengths and weaknesses of both methods.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
ChWDTA: Channel-wise Wavelet-Domain Transformer Attention and Entropy Modeling for Learned Image Compression
Haisheng Fu, Runyu Yang, Feng Ding et al. (+5 more)
State-of-the-art learned image compression (LIC) schemes are increasingly based on hybrid CNN-
transformer architectures. To further improve rate-distortion performance, we introduce channel-wise wavelet transforms into both the
transformer and
entropy-coding components. First, we propose a channel-wise wavelet-domain
transformer attention (ChWDTA) mechanism. ChWDTA keeps the efficient windowed spatial self-
attention used in modern LIC backbones, but computes the Q/K/V projections on channel-wise wavelet-transformed features before mapping the
attention output back with the inverse transform. The resulting Channel-wise Wavelet-Domain
Transformer Block (ChWDTB) therefore preserves the spatial tokenization pattern of windowed
attention while sparsifying the channel covariance seen by the
attention projections. Second, in the
entropy-coding stage, we introduce a channel-wise wavelet packet (ChWP) decomposition that produces four equal-sized subbands, which better fit channel-wise slice-based autoregressive
entropy modeling. When each channel-wise subband is divided into two slices, we use eight slices for
entropy coding. With this configuration, the proposed scheme obtains BD-rate reductions of -17.82%, -19.15%, and -22.56% on the Kodak, CLIC Professional Validation, and Tecnick test sets, respectively. Even when each channel-wise subband is coded as a single slice, the scheme still retains most of the coding gains with lower complexity. The results confirm the advantage of introducing wavelet transform in CNN-
transformer-based LIC schemes.
Read the full abstract ▾
All authors Haisheng Fu, Runyu Yang, Feng Ding, Siyu Zhu, Jie Liang, Xiaoxiao Li, Zhenman Fang, Jingning Han
Computer Vision
General Covariant Action Modeling: Constructing Generalized Manifolds via Spatio-Temporal Decoupling
Huaihai Lyu, Chaofan Chen, Mingyu Cao et al. (+2 more)
Achieving robust generalization from limited data is a central challenge in
embodied intelligence. Prevailing methods fail by regressing absolute coordinates, which violates the principle of general covariance. Fundamentally, this conflates the intrinsic task geometry with rigid execution patterns, binding policies to specific motion styles and fixed speeds. To resolve this, we propose the Generalized Action
Manifold (GAM) framework that enforces general covariance through structural disentanglement. Specifically, GAM realizes the
manifold by enforcing invariance across two orthogonal dimensions: (1) Temporal Invariance, utilizing an Arc-Length Parameterizer to orthogonalize the spatial path geometry from temporal dynamics, ensuring robustness to velocity variations; (2)
Geometric Invariance, where a Schema-Affine-Factorization mechanism maps trajectories to canonical “world lines” in a pose-normalized coordinate frame. This distinguishes invariant
geometric schemas from affine modulations, ensuring spatial generalizability. By integrating GAM within a structured
Vision-
Language-Action (VLA) architecture, we enable sparse demonstrations to densely populate a continuous, valid action
manifold. Empirical results demonstrate that GAM enables superior transfer and robustness capabilities, outperforming geometry-agnostic baselines.
Read the full abstract ▾
All authors Huaihai Lyu, Chaofan Chen, Mingyu Cao, Yuheng Ji, Changsheng Xu
Computer Vision
VDSB-GWSyn: Diffusion Schrödinger Bridge for Controllable and Anatomically Feasible Guidewire Synthesis in Coronary Angiography
Haoyuan Tang, Zhuo Zhang, Jialin Li et al. (+2 more)
Coronary guidewire endpoint localization is a fundamental capability for computer-assisted PCI, and its importance increases as
robot-assisted PCI is progressively adopted to reduce operator radiation exposure. However, the scarcity of annotated CAG images with guidewires and the limited adaptability of existing guidewire synthesis models remain key bottlenecks for guidewire endpoint localization. To address this issue, we propose VDSB-GWSyn, a
Diffusion Schrödinger Bridge (DSB) model-based framework, enabling synthesis of controllable, high-fidelity guidewire samples under complex anatomical backgrounds. VDSB-GWSyn first uses our shape prior
algorithm to learn the basic guidewire geometry. It then generates guidewire masks under constraints imposed by the vessel
segmentation masks and outputs the corresponding endpoint coordinates. Finally, it synthesizes realistic guidewire samples on real CAG images using DSB conditioned with SPADE. Experimental results show that the guidewire samples synthesized by VDSB-GWSyn achieve favorable ROI-FID and ROI-KID, as well as high IPR scores. In addition, incorporating our synthesized data for synthetic pre-training followed by real fine-tuning substantially improves downstream guidewire endpoint localization, reducing MPE from 16.01~px to 7.71~px and increasing PCK at 3~px from 52.63% to 86.27%, leading to more clinically reliable deployment of
robot-assisted guidewire delivery systems. Moreover, the core design philosophy of controllable device synthesis with strict background preservation and anatomical feasibility constraints has the potential to transfer to other interventional device perception tasks where annotated data are scarce.
Read the full abstract ▾
All authors Haoyuan Tang, Zhuo Zhang, Jialin Li, Shuai Xiao, Jiachen Yang
Computer Vision
Lightweight SAR Ship Detection via Contrastive Distillation
Surendar Devasundaram, Banafsheh Saber Latibari and Abhijit Mahalanobis
Deep convolutional and
transformer-based detectors achieve strong performance for SAR ship detection but are often computationally prohibitive for real-time or onboard deployment. Lightweight models offer improved efficiency yet struggle to capture the complex structural relationships inherent in SAR backscatter. Most existing SAR knowledge-
distillation approaches rely on feature or logit matching, which enforces localized activation similarity while neglecting the
geometric relationships among object representations. We propose a Structured Unified Relational knowledGE
distillation framework for SAR Ship detection (SURGE) that transfers relational geometry from a powerful teacher detector to a compact student detector using a contrastive InfoNCE objective in a shared projection
embedding space. To the best of our knowledge, this work presents the first
transformer-based SAR ship detector knowledge
distillation framework in SAR domain. The framework is architecture-agnostic in the sense that it provides a common region-level
distillation interface for two-stage, one-stage and
transformer-based detectors without modifying their deployed architectures. Experiments on the SSDD and HRSID
benchmarks demonstrate that the proposed method yields substantial improvements for two-stage detectors, achieving up to 6.2 mAP and 8.0 AP75 gains over baseline student and even surpassing teacher performance
Read the full abstract ▾
Computer Vision
Eulerian Gaussian Splatting using Hashed Probability Pyramids
Mia Gaia Polansky, George Kopanas, Stephan Garbin et al. (+2 more)
We introduce a probabilistic splat-based radiance field framework that retains the fast rasterization and test-time efficiency of 3D Gaussian Splatting (3DGS) while replacing heuristic primitive manipulation with gradient-based optimization of a volumetric probability density. Rather than relocating, splitting, or culling Gaussians via hand-tuned densification (e.g., ADC), we treat primitive locations as samples drawn from a persistent, learnable density. We instantiate this density using a novel,
memory-efficient multi-scale hierarchical grid that enables end-to-end gradient-based optimization. To stabilize the optimization, we derive an unbiased gradient
estimator with control variates that markedly reduces variance. By allowing probability mass to flow to where the loss demands, our framework eliminates brittle priors and naturally explores the volume, achieving state-of-the-art reconstruction quality on mip-NeRF 360 while preserving 3DGS-level rendering speed.
Read the full abstract ▾
All authors Mia Gaia Polansky, George Kopanas, Stephan Garbin, Todd Zickler, Dor Verbin
Computer Vision
Robust Cross-Domain Generalization Using Unlabeled Target Data with Source-Domain Supervision
Yuyue Zhou, Shrimanti Ghosh, Michael Xie et al. (+6 more)
It is often desirable to generalize medical imaging AI models trained with dense annotations to data acquired from different ultrasound scanners or clinical sites; however, retraining these models with new annotations is often difficult and costly. We examine this challenge in pediatric wrist fracture assessment using point-of-care ultrasound (POCUS), where fractures are common and can be effectively triaged via ultrasound. AI has shown radiologist-level performance for fracture detection, often aided by high-quality bony structure
segmentation. However, due to significant domain shifts, models perform poorly on data from other centers or probes, and obtaining
segmentation labels across devices is impractical due to manual annotation effort and data
privacy concerns. To address this, we propose a target-informed self-supervised pretraining and model-ensemble strategy. Specifically, our approach combines masked image modeling (MIM) and contrastive learning to learn target-domain structural representations without labels, and introduces a confidence-aware infusion head to adaptively integrate predictions. The source
dataset, collected with a Philips Lumify probe, contained dense labels, while the target
dataset, acquired with a TeleMED portable probe, was unlabeled. The
datasets were kept strictly separate throughout the entire process. Our method used labeled source data for supervised training and leveraged target-domain pretraining to improve generalization. On 318 images from 62 pediatric POCUS
videos, this approach significantly improved cross-device performance, achieving over 6% Dice improvement on the target domain versus the baseline. These results demonstrate a label-efficient and
privacy-preserving approach for cross-device-robust ultrasound AI, offering a framework that can be extended to multi-center studies or
federated learning setups.
Read the full abstract ▾
All authors Yuyue Zhou, Shrimanti Ghosh, Michael Xie, Justin JY Kim, Jessica Knight, Steel McDonald, Vincent Man, Jacob L. Jaremko, Abhilash Hareendranathan
Computer Vision
Seeing through boxes: Non-Line-of-Sight 3D Reconstruction from Radar Signals
Jiachen Lu, Hailan Shanbhag and Haitham Al Hassanieh
Reconstructing object geometry from
radio frequency (RF) signals is fundamentally challenging due to the lensless imaging nature of RF sensing, which leads to low spatial resolution and high noise. Unlike light signals, RF signals can penetrate occlusions and thus capture information about hidden scenes. Existing Non-Line-of-Sight (NLoS) 3D neural reconstruction methods can recover coarse surfaces inside enclosed environments but often suffer from unstable optimization, noisy surface geometry, and surface ambiguity, failing to produce accurate zero-level sets from the signed distance field (SDF). These limitations largely stem from neglecting the role of Line-of-Sight (LoS) geometry outside the enclosed region, which provides valuable physical constraints for modeling signal propagation. In this paper, we introduce a Unified LoS and NLoS neural geometry reconstruction framework GeRaF 2.0 that leverages the outside LoS geometry to model and guide RF propagation from the LoS region into the NLoS region. By integrating visual LoS priors into the neural field formulation, GeRaF 2.0 achieves stable training and physically consistent reconstruction of both visible and hidden geometry, setting a new state-of-the-art in RF-based geometry reconstruction.
Read the full abstract ▾
Computer Vision
GeRaF: Neural Geometry Reconstruction from Radio Frequency Signals
Jiachen Lu, Hailan Shanbhag and Haitham Al Hassanieh
GeRaF is the first method to use neural implicit learning for near-range 3D geometry reconstruction from
radio frequency (RF) signals. Unlike RGB or
LiDAR-based methods, RF sensing can see through occlusion but suffers from low resolution and noise due to its lensless imaging nature. While lenses in RGB imaging constrain sampling to 1D rays, RF signals propagate through the entire space, introducing significant noise and leading to cubic complexity in volumetric rendering. Moreover, RF signals interact with surfaces via specular reflections, requiring fundamentally different modeling. To address these challenges, GeRaF (1) introduces filter-based rendering to suppress irrelevant signals, (2) implements a physics-based RF volumetric rendering pipeline, and (3) proposes a novel lensless sampling and lensless alpha blending strategy that makes full-space sampling feasible during training. By learning signed distance functions, reflectiveness, and signal power through MLPs and trainable parameters, GeRaF takes the first step towards reconstructing millimeter-level geometry from RF signals in real-world settings.
Read the full abstract ▾
Computer Vision
Lightweight Complementary-Cue Fusion for Robust Video Face Forgery Detection
Sunghwan Baek, Tariq Anwaar, Karanveer Singh et al. (+1 more)
Current face
video forgery detectors use wide or dual-stream backbones. We show that a single, lightweight fusion of two handcrafted cues can achieve higher accuracy with a much smaller model. Based on the Xception baseline model (21.9 million parameters), we build two detectors: LFWS, which adds a 1x1 convolution to combine a low-frequency Wavelet-Denoised Feature (WDF) with a phase-spectrum channel derived from Spatial-Phase Shallow Learning (SPSL), and LFWL, which merges WDF with Local Binary Patterns (LBP) in the same way. This extra module adds only 292 parameters, keeping the total at 21.9 million, smaller than F3Net (22.5 million) and less than half the size of SRM (55.3 million). Even with this minimal overhead, the fused models increase the average area under the curve (AUC) from 74.8% to 78.6% on FaceForensics++ and from 70.5% to 74.9% on DFDC-Preview, gains of 3.8% and 4.4% over the Xception baseline. They also consistently outperform F3Net, SRM, and SPSL in eight public
benchmarks, without extra data or test-time augmentation. These results show that carefully paired, handcrafted features, combined through the lightweight fusion block, can provide competitive robustness at a significantly lower cost than comparable frequency-based detectors. Our findings suggest a need to reevaluate scale-driven design choices in face
video forgery detection.
Read the full abstract ▾
All authors Sunghwan Baek, Tariq Anwaar, Karanveer Singh, Rita Singh
Computer Vision · Cross-listed from cs.LG
OISD: On-Policy Internal Self-Distillation of Language Models
Xinyu Liu, Darryl Cherian Jacob, Yang Zhou et al. (+2 more)
Recent reinforcement learning (RL) post-training approaches primarily optimize the final output policy using sparse outcome-level rewards, while largely overlooking predictive signals encoded in intermediate representations. In this paper, we introduce a new paradigm called on-policy internal self-
distillation and propose the OISD framework, which improves
reasoning by transferring on-policy predictive signals from the final layer to intermediate representations. During rollout and Group Relative Policy Optimization (GRPO) optimization, the final layer acts as both the policy and a detached internal teacher for selected intermediate layers, which are guided to align with it through two complementary mechanisms: logit
alignment, which transfers high-level
reasoning behaviors (how to think), and
attention alignment, which enforces consistent
attention patterns (where to look) from the final layer to the selected intermediate layer, both without requiring external privileged information. Our OISD, together with GRPO, employs signed advantage-weighted Jensen–Shannon
alignment to distill informative intermediate representations while preserving policy consistency under a unified acting policy. Experimental results demonstrate the effectiveness of OISD, with substantial and consistent improvements over strong
reasoning RL baselines across four mathematical
reasoning tasks. The code will be released at https://github.com/THE-MALT-LAB/OISD
Read the full abstract ▾
All authors Xinyu Liu, Darryl Cherian Jacob, Yang Zhou, Jindong Wang, Pan He
Computer Vision
A Deep Learning Iterative Framework for Sentinel-1 Stripmap Enhancement Based on Azimuth Doppler Decomposition
Juan Francisco Amieva, Christian Ayala, Roberto Del Prete et al. (+1 more)
Synthetic Aperture Radar (SAR) imagery enables all-weather, day-and-night Earth observation; however, it remains difficult to interpret due to speckle noise and other intrinsic imaging artifacts. Sentinel-1 (S1) constitutes one of the most widely used spaceborne SAR missions, offering systematic global coverage, high temporal resolution, dual-polarization imaging, and free data availability. Among S1 modes, Stripmap (SM) provides the highest resolution, yet speckle noise and spatial constraints often hinder applications requiring finer spatial detail. This motivates the need for effective image enhancement strategies. In this work, we propose a self-supervised enhancement framework for S1 SM imagery based on azimuth subaperture decomposition. The method exploits the physical consistency between subaperture reconstructions and the corresponding full-aperture image to generate paired training data without external sensors, simulated ground truth, or multi-temporal stacks. The proposed framework integrates single- and multi-frame learning and incorporates an iterative inference scheme that progressively refines image quality. Experiments on real S1 SM data show that the proposed approach consistently outperforms the widely adopted self-supervised deep learning baseline MERLIN, in terms of PSNR and SSIM, while MERLIN attains higher ENL, highlighting a trade-off between structural fidelity and speckle smoothing. Overall, the results demonstrate that subaperture-based supervision provides a physically grounded, reproducible, and operationally viable approach for SAR image enhancement using S1 data. It is worth noting that the proposed approach can be extended to other SAR platforms, polarizations, and acquisition modes.
Read the full abstract ▾
All authors Juan Francisco Amieva, Christian Ayala, Roberto Del Prete, Mikel Galar
Computer Vision
Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Models
Jiyao Zhang, Mingxu Zhang, Yitong Peng et al. (+8 more)
Are current
Vision Language Models (VLMs) ready to comprehend and reason about complex
embodied interactions in 3D environments? We introduce
Embodied3DBench, a
robot-centric
benchmark targeting low-level spatial intelligence in
embodied 3D environments. To systematically evaluate these foundational perceptual capabilities, the
benchmark includes 6 task categories divided into two core groups: Spatial Structural Understanding (Grounding, Spatial Relation Prediction, and Multi-view Correspondence) and Interaction-Oriented Perception (Affordance Prediction, Grasp Point Prediction, and Trajectory Prediction). The
benchmark spans 12 subcategories and contains over 21k high-quality question-answer pairs. We evaluate 13 state-of-the-art models, and the results show that while current models exhibit relatively strong high-level spatial
reasoning, such as understanding object-to-object positional relations, they remain fragile in interaction-oriented perception, highlighting a significant lack of robust 3D-aware interaction priors. To actively bridge this capability gap revealed by our
benchmark, we further synthesize a large-scale training
dataset comprising 1.3M QA pairs. Notably, fine-tuning on this
dataset yields significant improvements in low-level spatial intelligence. Ultimately,
Embodied3DBench fills a critical gap by providing both a systematic evaluation framework and a scalable data solution, setting a clear target for the development of interaction-aware
multimodal systems.
Read the full abstract ▾
All authors Jiyao Zhang, Mingxu Zhang, Yitong Peng, Haoxuan Liu, Chenshuo Wang, Yuxing Long, Haoyang Huang, Dongjiang Li, Nan Duan, Hui Shen, Hao Dong
Computer Vision · Cross-listed from cs.CL
Persona Prompting in Multimodal Urban Perception: Descriptive Convergence and Interpretive Variation
Neemias da Silva, Matt Ratto, Myriam Delgado et al. (+3 more)
This study examines how persona prompting shapes
language generated by two
multimodal large
language models in urban perception, a setting for examining subjective interpretations of shared visual evidence. We organize outputs into three functional layers: descriptive grounding (captions), intermediate semantic layer (perception tags), and interpretive framing (justifications). Using approximately 60,000 persona-conditioned annotations from each of two MLLMs, Qwen3-VL and Gemma4, we find that captions converge strongly across persona profiles and show only small attribute-associated differences. Justifications vary substantially more: economic status produces the largest difference in both models, with political orientation and personality also prominent. Paired image-level comparisons confirm larger justification than caption differences for these three attributes. For perception tags, personas sharing the same attribute level produce more similar tag sets than personas with different attribute levels, with the largest separation observed for economic status. Exploratory topic analysis further suggests persona-specific evaluative emphasis. Across models, profile-pair similarity patterns are strongly correlated for all three output types, although agreement is lowest for justifications. Overall, persona prompting affects interpretive framing more strongly than descriptive grounding.
Read the full abstract ▾
All authors Neemias da Silva, Matt Ratto, Myriam Delgado, Rodrigo Minetto, Daniel Silver, Thiago H Silva
Computer Vision · Cross-listed from eess.IV
Accelerating HEVC Intra Partitioning via a CNN-Hierarchical Attention Transformer Hybrid
Krishna Kumar Sharma and Somdyuti Paul
The recursive quad-tree partitioning in High Efficiency
Video Coding (HEVC) incurs considerable computational overhead, with exhaustive rate-distortion optimization for CTU partition prediction consuming the dominant share of encoding time. Although partition prediction through deep learning has emerged as a viable encoding accelerator, an architectural dichotomy remains largely unaddressed: CNNs are computationally efficient but spatially myopic due to their localized effective receptive fields, failing to capture long range semantic relationships and repetitive textures; conversely,
transformer based architectures are better at capturing global context but incur prohibitive CPU latency, a critical liability that impedes deployment which is predominantly CPU-bound. This paper introduces Hybrid Fast
Vision Transformer (HFViT), a hybrid architecture designed to accelerate HEVC intra-mode partition prediction. HFViT fuses a reparameterized depthwise-separable convolutional backbone with a Hierarchical
Attention Transformer (HAT) mechanism, leveraging a carrier
token scheme to enable efficient global information propagation at sub-quadratic complexity. Post-training structural fusion collapses batch normalization into preceding layers to further reduce latency. Comprehensive evaluation reveals the efficacy of HFViT in accelerating HEVC intra-encoding across resolutions. On standard JCT-VC test sequences, HFViT reduces the average VMAF BD-rate penalty by 2.4, 2.6, and 7.9 percentage points on Classes A, B and E, respectively, as compared to the competing ETH-CNN baseline while maintaining CPU inference latency within 8% of the CNN baseline and surpassing it on GPU by 40%, establishing practical viability for real-time encoder integration.
Read the full abstract ▾
Computer Vision
Trajectory Constraints for Imaging Inverse Problems
Chaoyan Huang, Haijie Yuan, Bharath Subramanian et al. (+1 more)
Diffusion models have achieved strong performance in imaging inverse problems, but typically rely on domain-matched pretrained models and external training data. Moreover, errors introduced at intermediate reverse-sampling steps can accumulate and degrade the final reconstruction, with their propagation strongly influenced by how successive states are related. We propose TRACE, a training-free TRAjectory-Constrained rEconstruction framework for imaging inverse problems. TRACE replaces the pretrained
diffusion model with an untrained overparameterized network optimized directly from the observed measurements and organizes reconstruction as a sequence of intermediate states. At each step, TRACE enforces
stochastic proximal consistency (SPC) by inferring each reconstruction from a stochastically perturbed predecessor under measurement consistency, while proximally constraining the resulting estimate to its corresponding unperturbed state. This formulation admits a proximal interpretation, from which we derive stability bounds characterizing the variation between consecutive states under approximate network updates. A local Jacobian analysis further characterizes how SPC regularizes the network-parameterized reconstruction. Experiments on natural-image restoration and CT reconstruction demonstrate competitive performance against both untrained and pretrained neural-prior methods. We further show that the proposed trajectory constraint can be incorporated into existing
diffusion-based inverse solvers to improve reconstruction performance.
Read the full abstract ▾
All authors Chaoyan Huang, Haijie Yuan, Bharath Subramanian, Saiprasad Ravishankar
Computer Vision
Auditing Training-Free 3D Shape Retrieval with Diffused Geodesic Moments
Zhicheng Du, Changyue Liu, Wenji Xi et al. (+5 more)
Reported
retrieval scores for training-free shape descriptors conflate local signal design, normalization, aggregation, codebook fitting, and metric choices, making isolated component evaluation difficult. This paper reframes descriptor evaluation as a protocol audit. We introduce Diffused Geodesic Moments (DGM), a seed-conditioned descriptor that computes sparse implicit heat responses, converts them to distance-like fields, and summarizes each vertex by low-order moments across seeds and scales. DGM is used both as a practical non-spectral baseline and as an instrument for isolating protocol effects. On the registered FAUST
benchmark split (FAUST-Reg) and the TOSCA shape collection, aggregation-matched experiments show that an independent
Geometric Moment Shape Descriptor baseline built on Heat Kernel Signature features (GMSD-HKS) obtains the highest scores in this implementation ($0.621/0.820$ and $0.865/0.963$ mean average precision (mAP)/top-1), Wave Kernel Signature (WKS) remains a strong classical signal, and DGM is useful mainly when sparse solves, non-spectral deployment, or
symmetry-informative seed frames are priorities. The broader finding is methodological: the input field and aggregation protocol can dominate the moment formula. The paper contributes a reproducible protocol-cascade analysis, a cross-shape
alignment diagnostic for functional-map compatibility, and concrete recommendations for designing and reporting training-free shape descriptors.
Read the full abstract ▾
All authors Zhicheng Du, Changyue Liu, Wenji Xi, Zhaotian Xie, Zhuo Deng, Ziheng Zhang, Yang Liu, Lan Ma
Computer Vision
GAP3D: Generative Alignment of VLM Latents to Patch-Level Embeddings for 3D Generation
Polytimi Anna Gkotsi, Andrii Zadaianchuk and Mohammad Mahdi Derakhshani
Recent approaches integrating
vision-
language models (VLMs) as
prompt encoders for generative model conditioning typically rely on expensive end-to-end training or map features to compressed representations, discarding the dense spatial structure required for geometry-aware tasks like 3D asset generation. To address this, we propose GAP3D, a modular,
diffusion-based approach that aligns VLM-generated latents directly to the complete, patch-level feature space of a pre-trained image encoder, enabling a frozen downstream generative model to utilize a VLM as
prompt encoder while maintaining a spatially structured conditioning signal. Evaluated on 3D asset generation, our method bypasses the need for large-scale 3D data by training mainly on general-domain image-text pairs. It also exhibits emergent zero-shot capabilities for
multimodal prompts, despite being trained exclusively on text input. Finally, while currently prioritizing high-level semantics over fine-grained detail, GAP3D demonstrates that the representation gap between VLM and image-encoder feature spaces can be partially bridged through
diffusion-based
alignment, taking the first steps towards a modular integration of foundation models through generative
alignment to dense
embedding spaces.
Read the full abstract ▾
Computer Vision
Resolving Endpoint Underfitting in Diffusion Bridges via Noise Alignment
Yurong Gao, Zicheng Zhang, Congying Han et al. (+2 more)
Diffusion bridge models offer a powerful framework for connecting two data distributions, such as in image restoration and translation. Many existing methods learn this bridge by mimicking the score-matching formulation of standard
diffusion models. In this work, we find that this way leads to an anomalous underfitting phenomenon near the target endpoint, as the process approaches the target distribution ($t \to 0$). This underfitting, characterized by significant drift in the predicted variance and direction, results from an excessively large discrepancy in noise levels between the network's input and its regression target.To resolve this issue, we propose the Noise-Aligned
Diffusion Bridge (NADB).Our approach reformulates the
diffusion bridge by first employing a mean network to provide a cleaner conditional target, and then introducing a novel, noise-aligned mapping relationship. This new formulation resolves the noise mismatch and corrects the underfitting near the target endpoint. Experimental validation across multiple image restoration and image translation tasks demonstrates the effectiveness of our approach. Code is available at https://github.com/gyr02/NADB.
Read the full abstract ▾
All authors Yurong Gao, Zicheng Zhang, Congying Han, Tiande Guo, Xinmin Qiu
Computer Vision
From Pixels to Words – Towards Native One-Vision Models at Scale
Haiwen Diao, Jiahao Wang, Penghao Wu et al. (+18 more)
Current
vision-
language models (VLMs) typically stitch together separate image encoders and
language decoders via multi-stage
alignment, a modular framework that inevitably fragments pixel-level signals across frames and scatters early pixel-word interactions. In parallel, native VLMs, despite impressive performance on single images, remain largely unexplored in multi-image,
video understanding, and spatial intelligence. Hence, we introduce NEO-ov, a native foundation model that learns cross-frame and pixel-word correspondence end-to-end, without any external encoders, auxiliary adapters, or post-hoc fusion. By eliminating module boundaries entirely, NEO-ov enables fine-grained and unified spatiotemporal modeling to emerge natively inside the model. Notably, NEO-ov largely narrows the gap to modular counterparts while excelling at fine-grained visual perception, validating that native "one-
vision" architectures are not only feasible but competitive at scale. Beyond empirical performance, we unveil systematic architectural analyses and detailed training recipes to facilitate subsequent native
multimodal modeling. Our code and models are publicly available at: https://github.com/EvolvingLMMs-Lab/NEO.
Read the full abstract ▾
All authors Haiwen Diao, Jiahao Wang, Penghao Wu, Yuhao Dong, Yuwei Niu, Yue Zhu, Zhongang Cai, Weichen Fan, Linjun Dai, Silei Wu, Xuanyu Zheng, Mingxuan Li, Yuanhan Zhang, Bo Li, Hanming Deng, Huchuan Lu, Quan Wang, Lei Yang, Lewei Lu, Dahua Lin, Ziwei Liu
Computer Vision
Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players
Fangfu Liu, Kai He, Tianchang Shen et al. (+7 more)
World models for interactive
video generation have largely focused on single-
agent settings, where future observations are generated from a single control signal. However, many generated environments require multi-
agent interaction: multiple players,
robots, or
embodied agents act simultaneously within a shared space. Scaling world models to such settings requires a principled multi-
agent design:
agents should remain independently controllable, permutation-symmetric, and support efficient inference while maintaining consistency across time and perspectives. In this paper, we present our generative multi-
agent world model for interactive simulation. It introduces Simplex Rotary
Agent Encoding, a parameter-free extension of 3D RoPE that represents
agents as vertices of a regular simplex in rotary angle space. This gives each
agent a distinct phase while making all
agents permutation-equivalent, enabling scalable
agent identity without learned per-slot identities or a fixed
agent ordering. To avoid dense all-to-all
attention across
agents, we further propose Sparse Hub
Attention, where learnable hub
tokens mediate
token interaction across
agents, reducing cross-
agent attention cost from quadratic to linear in the number of
agents. For real-time rollout, we distill a full-context
diffusion teacher into a causal student that generates temporal blocks sequentially with KV caching, enabling action-responsive generation at 24 FPS. Experiments in multiplayer virtual environments show that our model improves
video fidelity, action controllability, and inter-
agent consistency over slot-based and dense-
attention baselines, while generalizing from two to four players without additional training.
Read the full abstract ▾
All authors Fangfu Liu, Kai He, Tianchang Shen, Tianshi Cao, Sanja Fidler, Yueqi Duan, Jun Gao, Igor Gilitschenski, Zian Wang, Xuanchi Ren
Computer Vision
HarmoVid: Relightful Video Portrait Harmonization
Jun Myeong Choi, Jae Shin Yoon, Luchao Qi et al. (+2 more)
We present a method for harmonizing the lighting of a foreground
video to match a target background scene, adjusting shadows, color tone, and illumination intensity (relightful harmonization). Unlike images, acquiring labeled data for
videos, where identical motions are recorded under different lighting conditions, is practically infeasible and non-scalable. While one way to create such paired data is to apply existing image-based harmonization models frame by frame to a
video, the resulting outputs often suffer from significant temporal jitters. We overcome this problem by introducing a novel lighting deflickering model that can stabilize the global and local lighting flickering artifacts. Our
video diffusion model learns from these upgraded deflickered data with a volume of real and synthetic
videos to generate high-quality
video harmonization results. We further propose an asymmetric alpha mask conditioning technique to learn the clean boundaries from real
videos. Experiments demonstrate that our model achieves strong temporal coherence, naturalness, cleaner boundaries, and physically meaningful lighting behavior, while maintaining strong relighting expressiveness compared to prior image-based and
video-based harmonization methods.
Read the full abstract ▾
All authors Jun Myeong Choi, Jae Shin Yoon, Luchao Qi, Roni Sengupta, Joon-Young Lee
Computer Vision
AREA: Attribute Extraction and Aggregation for CLIP-Based Class-Incremental Learning
Zhen-Hao Xie, Yu-Cheng Shi and Da-Wei Zhou
Class-Incremental Learning (CIL) is important in building real-world learning systems. In CLIP-based CIL, the model performs classification by comparing similarity between visual and textual
embeddings obtained from template
prompts, e.g., “a photo of a [CLASS]”. This seemingly monolithic matching process can be decomposed into two conceptually distinct stages: attribute extraction and attribute aggregation. For example, a model may recognize cat using attributes such as fur texture and whiskers. When learning a new class like car, the model must extract additional attributes like wheels and adjust how they are aggregated in the shared representation space. However, since only data from the current task is available, incremental updates can bias both attribute extraction and aggregation toward new classes, leading to catastrophic forgetting. Therefore, we propose AREA for attribute extraction and aggregation in CLIP-based CIL. To stabilize extraction, we anchor class-level visual and textual attributes on the hyperspherical
embedding space via principal geodesic analysis. To stabilize aggregation, we learn lightweight task-specific experts with scoring and residual refinement, regularized by a variational information bottleneck objective. During inference, we perform routing over task attribute
manifolds via optimal transport for more concise prediction. Experiments show that AREA consistently outperforms SOTA methods. Code is available at https://github.com/LAMDA-CL/ICML2026-AREA.
Read the full abstract ▾
Computer Vision
Personal Visual Memory from Explicit and Implicit Evidence
Viet Nguyen, Thao Nguyen, Vishal M. Patel et al. (+1 more)
Long-term
memory is increasingly important for personalized AI
agents, yet existing
benchmarks and methods remain largely text-centric. Even when images are included, the user-specific information needed for later questions is typically recoverable from text alone, and most
memory systems reduce image turns to generic captions. Yet images often carry personal information that text rarely states – both explicit evidence, such as recurring user-associated entities, and implicit evidence, such as latent user facts inferred from visual or
multimodal cues. We introduce a
benchmark for personal visual
memory that targets both forms of evidence, and propose VisualMem, a hybrid visual–text architecture that augments a text-
memory backend with a structured personal visual
memory module. Rather than collapsing images into captions, VisualMem uses conversational context to resolve identity, ownership, and durable user facts. Experiments show that VisualMem substantially outperforms prior
memory systems on our
benchmark while remaining competitive on standard text-
memory benchmarks, indicating that personal visual
memory is a distinct and important component of long-term
memory for personalized AI
agents.
Read the full abstract ▾
All authors Viet Nguyen, Thao Nguyen, Vishal M. Patel, Yuheng Li
Computer Vision · Cross-listed from cs.CL
OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration
Xinchen Zhang, Bowei Liu, Jiale Liu et al. (+7 more)
Visual outcomes are increasingly central to
multimodal large
language models, making reliable and fine-grained verification essential for scaling generalist foundation models. In this work, we investigate
multimodal meta-verification, which leverages verifier-generated rationales rather than decision-only signals, and explore how to effectively incorporate meta-verification feedback into
multimodal verifier training. We identify two key findings. First, symbolic verifier outputs (e.g., bounding boxes) outperform textual explanations as meta-verification rationales, enabling efficient rule-based reinforcement learning rewards while avoiding reliance on model-based rewards from auxiliary judge models. Second, decoupling reinforcement learning objectives for binary judgment and meta-verification substantially outperforms joint reward optimization, due to intrinsic differences in output structure and learning dynamics. Based on these insights, we train OmniVerifier-M1, a generalist visual verifier leveraging symbolic meta-verification and decoupled reinforcement learning. OmniVerifier-M1 provides robust verification and fine-grained error localization, and further enables M1-TTS, a verifier-driven agentic generation system achieving dynamic region-level self-correction. This approach paves the way for more reliable, interpretable, and fine-grained
multimodal verification, supporting safer and more controllable foundation model deployment.
Read the full abstract ▾
All authors Xinchen Zhang, Bowei Liu, Jiale Liu, Chufan Shi, Yizhen Zhang, Junhong Liu, Youliang Zhang, Zhiheng Li, Yujiu Yang, Ling Yang
Computer Vision
HoloQ-VLA: Uniform W4A4 Quantization of Vision-Language-Action Models
Xinyu Wang, Mingze Li, Sicheng Lyu et al. (+6 more)
Vision-
Language-Action (VLA) models unify perception,
reasoning, and control in a single policy, but their multi-billion-parameter backbones and
diffusion-based action heads make on-device deployment prohibitively expensive. Low-bit post-training quantization (PTQ) is the natural remedy, yet the
diffusion action head that emits continuous control signals is highly sensitive to it: a few weight and activation outliers are enough to destabilize the head, so prior work leaves it at full precision or falls back to mixed-precision schemes, and uniformly quantizing the whole model to low bit-width remains an open challenge. We present HoloQ-VLA, the first training-free PTQ framework that compresses both the
language backbone and the entire
diffusion action head to uniform W4A4 precision without mixed-precision allocation. Instead of trading weight quality against activation quality, HoloQ-VLA targets the two outlier sources with complementary transforms: a weight-adapted rotation composed with an activation-dispersing Hadamard transform, together with per-step scaling that absorbs the dynamic-range drift exhibited by the action head across denoising steps. On LIBERO, HoloQ-VLA compresses Pi-0.5 and GR00T-N1.5 to W4A4 with 98.0% and 87.8% task success rates, matching or exceeding their FP16 references of 97.1% and 87.0%, while reducing the static
memory footprint by 74.2%. Real-world manipulation experiments further demonstrate that HoloQ-VLA maintains smooth and accurate control across diverse real-world scenarios.
Read the full abstract ▾
All authors Xinyu Wang, Mingze Li, Sicheng Lyu, Dongxiu Liu, Kaicheng Yang, Ziyu Zhao, Yufei Cui, Xiao-Wen Chang, Peng Lu
Computer Vision
Bias Leaves a Gradient Trail: Label-Free Bias Identification via Gradient Probes on Concept Decompositions
Thomas Vitry, Kieran Edgeworth, Stefan Wermter et al. (+1 more)
Vision classifiers can exploit spurious correlations, achieving high in-distribution accuracy yet failing under distribution shift. Existing approaches to bias mitigation and analysis often depend on curated
datasets, spurious-attribute or group labels, or retraining, which may be infeasible once a model is deployed or the relevant bias is unknown. We present a bias-label-free, post-hoc method for identifying spurious concepts in frozen
vision models, relying only on standard class labels from a held-out audit
dataset. For each target class, we collect patches from inputs predicted as that class and apply non-negative
matrix factorization to intermediate activations to obtain a bank of interpretable concept vectors. Candidate concepts are then ranked with a bias
estimator derived from their interaction with backpropagated gradients on misclassified examples: bias concepts tend to get activated when correcting false negatives and suppressed when correcting false positives. On Colored MNIST and Waterbirds the method recovers concepts aligned with the known spurious cue, and on CelebA it surfaces decision-relevant directions that only partially coincide with the annotated gender attribute; suppressing the top-ranked concepts at inference time improves worst-group accuracy by up to 17.9 percentage points on Waterbirds and 10.4 on CelebA without any retraining or parameter updates. Our method identifies decision-relevant spurious directions that need not coincide with annotated ones, providing both an interpretable
auditing tool and an actionable debiasing handle for frozen
vision models. Code is available at https://github.com/vitryt/label-free-bias-identification.
Read the full abstract ▾
All authors Thomas Vitry, Kieran Edgeworth, Stefan Wermter, Jae Hee Lee