Computer Vision
ARGaze: Autoregressive Transformers for Online Egocentric Gaze Estimation
Jia Li, Wenjie Zhao, Shijian Deng et al. (+6 more)
Online
egocentric gaze estimation predicts where a camera wearer is looking from first-person
video using only past and current frames, a task essential for augmented reality and assistive technologies. Unlike third-person gaze estimation, this setting lacks explicit head or eye signals, requiring models to infer current visual
attention from sparse, indirect cues such as hand-object interactions and salient scene content. We observe that gaze exhibits strong temporal continuity during goal-directed activities: knowing where a person looked recently provides a powerful prior for predicting where they look next. Inspired by
vision-conditioned autoregressive decoding in
vision-
language models, we propose ARGaze, which reformulates gaze estimation as sequential prediction: at each timestep, a
transformer decoder predicts current gaze by conditioning on (i) current visual features and (ii) a fixed-length Gaze Context Window of recent gaze target estimates. This design enforces causality and enables bounded-resource streaming inference. We achieve state-of-the-art performance across multiple
egocentric benchmarks under online evaluation, with extensive ablations validating that autoregressive modeling with bounded gaze history is critical for robust prediction. We will release our source code and pre-trained models.
Read the full abstract ▾
All authors Jia Li, Wenjie Zhao, Shijian Deng, Bolin Lai, Yuheng Wu, RUijia Chen, Jon E. Froehlich, Yuhang Zhao, Yapeng Tian
Computer Vision
CLEAR-HPV: Interpretable concept discovery for human-papillomavirus-associated morphology in whole-slide histology
Weiyi Qin, Yingci Liu-Swetz, Shiwei Tan et al. (+1 more)
Human papillomavirus (HPV) status is a critical determinant of prognosis and treatment response in head and neck and cervical cancers. Although
attention-based multiple instance learning (MIL) achieves strong slide-level prediction for HPV-related whole-slide histopathology, it provides limited morphologic interpretability. To address this limitation, we introduce Concept-Level Explainable
Attention-guided Representation for HPV (CLEAR-HPV), a framework that restructures the MIL latent space using
attention to enable concept discovery without requiring concept labels during training. Operating in an
attention-weighted latent space, CLEAR-HPV automatically discovers keratinizing, basaloid, and stromal morphologic concepts, generates spatial concept maps, and represents each slide using a compact concept-fraction vector. CLEAR-HPV's concept-fraction vectors preserve the predictive information of the original MIL
embeddings while reducing the high-dimensional feature space (e.g., 1536 dimensions) to only 10 interpretable concepts. CLEAR-HPV generalizes consistently across TCGA-HNSCC, TCGA-CESC, and CPTAC-HNSCC, providing compact, concept-level interpretability through a general, backbone-agnostic framework for
attention-based MIL models of whole-slide histopathology.
Read the full abstract ▾
All authors Weiyi Qin, Yingci Liu-Swetz, Shiwei Tan, Hao Wang
Computer Vision · Cross-listed from cs.CE
Rule-Based Spatial Mixture-of-Experts U-Net for Explainable Edge Detection
Bharadwaj Dogga, Kaaustaaub Shankar, Gibin Raju et al. (+2 more)
Deep learning models like U-Net and its variants, have established state-of-the-art performance in edge detection tasks and are used by Generative AI services world-wide for their image generation models. However, their decision-making processes remain opaque, operating as "black boxes" that obscure the rationale behind specific boundary predictions. This lack of transparency is a critical barrier in safety-critical applications where verification is mandatory. To bridge the gap between high-performance deep learning and interpretable logic, we propose the Rule-Based Spatial Mixture-of-Experts U-Net (sMoE U-Net). Our architecture introduces two key innovations: (1) Spatially-Adaptive Mixture-of-Experts (sMoE) blocks integrated into the decoder skip connections, which dynamically gate between "Context" (smooth) and "Boundary" (sharp) experts based on local feature statistics; and (2) a Takagi-Sugeno-Kang (TSK) Fuzzy Head that replaces the standard classification layer. This fuzzy head fuses deep semantic features with heuristic edge signals using explicit IF-THEN rules. We evaluate our method on the BSDS500
benchmark, achieving an Optimal
Dataset Scale (ODS) F-score of 0.7628, effectively matching purely deep baselines like HED (0.7688) while outperforming the standard U-Net (0.7437). Crucially, our model provides pixel-level explainability through "Rule Firing Maps" and "Strategy Maps," allowing users to visualize whether an edge was detected due to strong gradients, high semantic confidence, or specific logical rule combinations.
Read the full abstract ▾
All authors Bharadwaj Dogga, Kaaustaaub Shankar, Gibin Raju, Wilhelm Louw, Kelly Cohen
Computer Vision
Visual concept ranking uncovers medical shortcuts used by large multimodal models
Joseph D. Janizek, Sonnet Xu, Junayd Lateef et al. (+1 more)
Ensuring the reliability of machine learning models in safety-critical domains such as healthcare requires
auditing methods that can uncover model shortcomings. We introduce a method for identifying important visual concepts within large
multimodal models (LMMs) and use it to investigate the behaviors these models exhibit when prompted with medical tasks. We primarily focus on the task of classifying malignant skin lesions from clinical dermatology images, with supplemental experiments including both chest radiographs and natural images. After showing how LMMs display unexpected gaps in performance between different demographic subgroups when prompted with demonstrating examples, we apply our method, Visual Concept Ranking (VCR), to these models and
prompts. VCR generates hypotheses related to different visual feature dependencies, which we are then able to validate with manual interventions.
Read the full abstract ▾
All authors Joseph D. Janizek, Sonnet Xu, Junayd Lateef, Roxana Daneshjou
Computer Vision · Cross-listed from cs.GR
Gabor Fields: Orientation-Selective Level-of-Detail for Volume Rendering
Jorge Condor, Nicolai Hermann, Mehmet Ata Yurtsever et al. (+1 more)
Gaussian-based representations have enabled efficient physically-based volume rendering at a fraction of the
memory cost of regular, discrete, voxel-based distributions. However, several remaining issues hamper their widespread use. One of the advantages of classic voxel grids is the ease of constructing hierarchical representations by either storing volumetric mipmaps or selectively
pruning branches of an already hierarchical voxel grid. Such strategies reduce rendering time and eliminate aliasing when lower levels of detail are required. Constructing similar strategies for Gaussian-based volumes is not trivial. Straightforward solutions, such as prefiltering or computing mipmap-style representations, lead to increased
memory requirements or expensive re-fitting of each level separately. Additionally, such solutions do not guarantee a smooth transition between different hierarchy levels. To address these limitations, we propose Gabor Fields, an orientation-selective mixture of Gabor kernels that enables continuous frequency filtering at no cost. The frequency content of the asset is reduced by selectively
pruning primitives, directly benefiting rendering performance. Beyond filtering, we demonstrate that stochastically sampling from different frequencies and orientations at each ray recursion enables masking substantial portions of the volume, accelerating ray traversal time in single- and multiple-scattering settings. Furthermore, inspired by procedural volumes, we present an application for efficient design and rendering of procedural clouds as Gabor-noise-modulated Gaussians.
Read the full abstract ▾
All authors Jorge Condor, Nicolai Hermann, Mehmet Ata Yurtsever, Piotr Didyk
Computer Vision
Food Portion Estimation: From Pixels to Calories
Gautham Vinod and Fengqing Zhu
Reliance on images for dietary assessment is an important strategy to accurately and conveniently monitor an individual's health, making it a vital mechanism in the prevention and care of chronic diseases and obesity. However, image-based dietary assessment suffers from estimating the three dimensional size of food from 2D image inputs. Many strategies have been devised to overcome this critical limitation such as the use of auxiliary inputs like depth maps, multi-view inputs, or model-based approaches such as template matching. Deep learning also helps bridge the gap by either using
monocular images or combinations of the image and the auxillary inputs to precisely predict the output portion from the image input. In this paper, we explore the different strategies employed for accurate portion estimation.
Read the full abstract ▾
Computer Vision
VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Models
Yiye Chen, Yanan Jian, Xiaoyi Dong et al. (+5 more)
Vision-
Language-Action (VLA) models have demonstrated strong performance across a wide range of robotic manipulation tasks. Despite the success, extending large pretrained
Vision-
Language Models (VLMs) to the action space can induce
vision-action misalignment, where action predictions exhibit weak dependence on the current visual state, leading to unreliable action outputs. In this work, we study VLA models through the lens of visual conditioning and empirically show that successful rollouts consistently exhibit stronger visual dependence than failed ones. Motivated by this observation, we propose a training framework that explicitly strengthens visual conditioning in VLA models. Our approach first aligns action prediction with visual input via preference optimization on a track-following surrogate task, and then transfers the enhanced
alignment to instruction-following task through latent-space
distillation during supervised finetuning. Without introducing architectural modifications or additional data collection, our method improves both visual conditioning and task performance for discrete OpenVLA, and further yields consistent gains when extended to the continuous OpenVLA-OFT setting. Project website: https://vista-vla.github.io/ .
Read the full abstract ▾
All authors Yiye Chen, Yanan Jian, Xiaoyi Dong, Shuxin Cao, Jing Wu, Patricio Vela, Benjamin E. Lundell, Dongdong Chen
Computer Vision · Cross-listed from quant-ph
QuantumGS: Quantum Encoding Framework for Gaussian Splatting
Grzegorz Wilczyński, Rafał Tobiasz, Paweł Gora et al. (+2 more)
Recent advances in neural rendering, particularly 3D Gaussian Splatting (3DGS), have enabled real-time rendering of complex scenes. However, standard 3DGS relies on spherical harmonics, which often struggle to accurately capture high-frequency view-dependent effects such as sharp reflections and transparency. While hybrid approaches like Viewing Direction Gaussian Splatting (VDGS) mitigate this limitation using classical Multi-Layer Perceptrons (MLPs), they remain limited by the expressivity of classical networks in low-parameter regimes. In this paper, we introduce QuantumGS, a novel hybrid framework that integrates Variational
Quantum Circuits (VQC) into the Gaussian Splatting pipeline. We propose a unique encoding strategy that maps the viewing direction directly onto the Bloch sphere, leveraging the natural geometry of
qubits to represent 3D directional data. By replacing classical color-modulating networks with
quantum circuits generated via a hypernetwork or conditioning mechanism, we achieve higher expressivity and better generalization. Source code is available in the supplementary material. Code is available at https://github.com/gwilczynski95/QuantumGS
Read the full abstract ▾
All authors Grzegorz Wilczyński, Rafał Tobiasz, Paweł Gora, Marcin Mazur, Przemysław Spurek
Computer Vision
UniTrack: Differentiable Graph Representation Learning for Multi-Object Tracking
Bishoy Galoaa, Xiangyu Bai, Utsav Nandi et al. (+3 more)
We present UniTrack, a plug-and-play graph-theoretic loss function designed to significantly enhance multi-object tracking (MOT) performance by directly optimizing tracking-specific objectives through unified differentiable learning. Unlike prior graph-based MOT methods that redesign tracking architectures, UniTrack provides a universal training objective that integrates detection accuracy, identity preservation, and spatiotemporal consistency into a single end-to-end trainable loss function, enabling seamless integration with existing MOT systems without architectural modifications. Through differentiable graph representation learning, UniTrack enables networks to learn holistic representations of motion continuity and identity relationships across frames. We validate UniTrack across diverse tracking models and multiple challenging
benchmarks, demonstrating consistent improvements across all tested architectures and
datasets including Trackformer, MOTR, FairMOT, ByteTrack, GTR, and MOTE. Extensive evaluations show up to 53% reduction in identity switches and 12% IDF1 improvements across challenging
benchmarks, with GTR achieving peak performance gains of 9.7% MOTA on SportsMOT.
Read the full abstract ▾
All authors Bishoy Galoaa, Xiangyu Bai, Utsav Nandi, Sai Siddhartha Vivek Dhir Rangoju, Somaieh Amraee, Sarah Ostadabbas
Computer Vision · Cross-listed from cs.RO
Differentiable Inverse Graphics for Zero-shot Scene Reconstruction and Robot Grasping
Octavio Arriaga, Proneet Sharma, Jichen Guo et al. (+3 more)
Operating effectively in novel real-world environments requires robotic systems to estimate and interact with previously unseen objects. Current state-of-the-art models address this challenge by using large amounts of training data and test-time samples to build black-box scene representations. In this work, we introduce a differentiable neuro-graphics model that combines neural foundation models with physics-based differentiable rendering to perform zero-shot scene reconstruction and
robot grasping without relying on any additional 3D data or test-time samples. Our model solves a series of constrained optimization problems to estimate physically consistent scene parameters, such as meshes, lighting conditions, material properties, and 6D poses of previously unseen objects from a single RGBD image and bounding boxes. We evaluated our approach on standard model-free few-shot
benchmarks and demonstrated that it outperforms existing
algorithms for model-free few-shot pose estimation. Furthermore, we validated the accuracy of our scene reconstructions by applying our
algorithm to a zero-shot grasping task. By enabling zero-shot, physically-consistent scene reconstruction and grasping without reliance on extensive
datasets or test-time sampling, our approach offers a pathway towards more data efficient, interpretable and generalizable
robot autonomy in novel environments.
Read the full abstract ▾
All authors Octavio Arriaga, Proneet Sharma, Jichen Guo, Marc Otto, Siddhant Kadwe, Rebecca Adam
Computer Vision · Cross-listed from cs.GR
Untwisting RoPE: Frequency Control for Shared Attention in DiTs
Aryan Mikaeili, Or Patashnik, Andrea Tagliasacchi et al. (+2 more)
Positional encodings are essential to
transformer-based generative models, yet their behavior in
multimodal and
attention-sharing settings is not fully understood. In this work, we present a principled analysis of Rotary Positional
Embeddings (RoPE), showing that RoPE naturally decomposes into frequency components with distinct positional sensitivities. We demonstrate that this frequency structure explains why shared-
attention mechanisms, where a target image is generated while attending to
tokens from a reference image, can lead to reference copying, in which the model reproduces content from the reference instead of extracting only its stylistic cues. Our analysis reveals that the high-frequency components of RoPE dominate the
attention computation, forcing queries to attend mainly to spatially aligned reference
tokens and thereby inducing this unintended copying behavior. Building on these insights, we introduce a method for selectively modulating RoPE frequency bands so that
attention reflects semantic similarity rather than strict positional
alignment. Applied to modern
transformer-based
diffusion architectures, where all
tokens share
attention, this modulation restores stable and meaningful shared
attention. As a result, it enables effective control over the degree of style transfer versus content copying, yielding a proper style-aligned generation process in which stylistic attributes are transferred without duplicating reference content.
Read the full abstract ▾
All authors Aryan Mikaeili, Or Patashnik, Andrea Tagliasacchi, Daniel Cohen-Or, Ali Mahdavi-Amiri
Computer Vision
SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy
Zhuosen Bao, Xia Du, Zheng Lin et al. (+8 more)
With the deep integration of facial recognition into online banking, identity verification, and other networked services, achieving effective decoupling of identity information from visual representations during image storage and transmission has become a critical challenge for
privacy protection. To address this issue, we propose SIDeR, a Semantic decoupling-driven framework for unrestricted face
privacy protection. SIDeR decomposes a facial image into a machine-recognizable identity feature vector and a visually perceptible semantic appearance component. By leveraging semantic-guided recomposition in the latent space of a
diffusion model, it generates visually anonymous
adversarial faces while maintaining machine-level identity consistency. The framework incorporates momentum-driven unrestricted perturbation optimization and a semantic-visual balancing factor to synthesize multiple visually diverse, highly natural
adversarial samples. Furthermore, for authorized access, the protected image can be restored to its original form when the correct password is provided. Extensive experiments on the CelebA-HQ and FFHQ
datasets demonstrate that SIDeR achieves a 99% attack success rate in black-box scenarios and outperforms baseline methods by 41.28% in PSNR-based restoration quality.
Read the full abstract ▾
All authors Zhuosen Bao, Xia Du, Zheng Lin, Jizhe Zhou, Zihan Fang, Jiening Wu, Yuxin Zhang, Zhe Chen, Chi-man Pun, Wei Ni, Jun Luo
Computer Vision
SynthForensics: Benchmarking and Evaluating People-Centric Synthetic Video Deepfakes
Roberto Leotta, Salvatore Alfio Sambataro, Claudio Vittorio Ragaglia et al. (+5 more)
Modern T2V/I2V generators synthesize people increasingly hard to distinguish from authentic footage, while current evaluation suites lag: legacy
benchmarks target manipulation-based forgeries, and recent synthetic-
video benchmarks prioritize scale over realistic human depiction. We introduce SynthForensics, a people-centric
benchmark of $20{,}445$
videos from 8 T2V and 7 I2V open-source generators, paired-source from FF++/DFD reals, two-stage human-validated, in four compression versions with full metadata. In our paired-comparison human study, raters prefer SynthForensics in $71$–$77\%$ of head-to-head comparisons against each of nine existing synthetic-
video benchmarks, while facial-quality metrics fall within the FF++/DFD baseline range. Across 15 detectors and three protocols, face-based methods drop $13$–$55$ AUC points (mean $27$) from FF++ to SynthForensics and a further $23$ under aggressive compression; fine-tuning closes the gap at a backward cost on legacy
benchmarks; training from scratch shows synthetic and manipulation features largely disjoint for most detectors. We release
dataset, pipeline, and code.
Read the full abstract ▾
All authors Roberto Leotta, Salvatore Alfio Sambataro, Claudio Vittorio Ragaglia, Mirko Casu, Yuri Petralia, Francesco Guarnera, Luca Guarnera, Sebastiano Battiato
Computer Vision · Cross-listed from cs.LG
Temporal Pair Consistency for Flow Matching
Chika Maduabuchi and Jindong Wang
Flow-matching-style generative models learn time-dependent vector fields, but standard objectives supervise each timestep independently, leaving shared-path temporal structure unused. We introduce Temporal Pair Consistency (TPC), a simple training-time objective that couples velocity predictions at paired timesteps along the same probability path. TPC leaves the architecture, probability path, sampler, and inference-time computation unchanged. Under matched backbones, training budgets, solvers, and numbers of function evaluations, TPC improves sample quality across flow matching, OT-CFM, rectified flow, DiT-style backbones, and MeanFlow. On CIFAR-10, TPC improves FM from 6.35 to 3.19 FID and OT-CFM from 3.58 to 2.90 FID at the same NFE. Mechanism diagnostics show lower measured gradient variance, higher paired-gradient correlation, and reduced temporal roughness, while random and local pairing controls are substantially weaker. Higher-resolution ImageNet experiments further show consistent gains for DiT-XL/2 at 256 and 512 resolution and for MeanFlow at 256 resolution, all at matched inference cost.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Reinforced Attention Learning
Bangzheng Li, Jianmo Ni, Chen Qu et al. (+5 more)
Post-training with Reinforcement Learning (RL) has substantially improved
reasoning in Large
Language Models (LLMs) via test-time scaling. However, extending this paradigm to
Multimodal LLMs (MLLMs) through verbose rationales yields limited gains for perception and can even degrade performance. We propose Reinforced
Attention Learning (RAL), a policy-gradient framework that directly optimizes internal
attention distributions rather than output
token sequences. By shifting optimization from what to generate to where to attend, RAL promotes effective information allocation and improved grounding in complex
multimodal inputs. Experiments across diverse image and
video benchmarks show consistent gains over GRPO and other baselines. We further introduce On-Policy
Attention Distillation, demonstrating that transferring latent
attention behaviors yields stronger cross-modal
alignment than standard knowledge
distillation. Our results position
attention policies as a principled and general alternative for
multimodal post-training.
Read the full abstract ▾
All authors Bangzheng Li, Jianmo Ni, Chen Qu, Ian Miao, Liu Yang, Xingyu Fu, Muhao Chen, Derek Zhiyuan Cheng
Computer Vision
CoWTracker: Tracking by Warping instead of Correlation
Zihang Lai, Eldar Insafutdinov, Edgar Sucar et al. (+1 more)
Dense point tracking is a fundamental problem in computer
vision, with applications ranging from
video analysis to robotic manipulation. State-of-the-art trackers typically rely on cost volumes to match features across frames, but this approach incurs quadratic complexity in spatial resolution, limiting scalability and efficiency. In this paper, we propose \method, a novel dense point tracker that eschews cost volumes in favor of warping. Inspired by recent advances in optical flow, our approach iteratively refines track estimates by warping features from the target frame to the query frame based on the current estimate. Combined with a
transformer architecture that performs joint spatiotemporal
reasoning across all tracks, our design establishes long-range correspondences without computing feature correlations. Our model is simple and achieves state-of-the-art performance on standard dense point tracking
benchmarks, including TAP-Vid-DAVIS, TAP-Vid-Kinetics, and Robo-TAP. Remarkably, the model also excels at optical flow, sometimes outperforming specialized methods on the Sintel, KITTI, and Spring
benchmarks. These results suggest that warping-based architectures can unify dense point tracking and optical flow estimation.
Read the full abstract ▾
All authors Zihang Lai, Eldar Insafutdinov, Edgar Sucar, Andrea Vedaldi
Computer Vision
PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generation
Jiahao Zhan, Zizhang Li, Hong-Xing Yu et al. (+1 more)
We introduce PerpetualWonder, a hybrid generative simulator that enables long-horizon, action-conditioned 4D scene generation from a single image. Current works fail at this task because their physical state is decoupled from their visual representation, which prevents generative refinements to update the underlying physics for subsequent interactions. PerpetualWonder solves this by introducing the first true closed-loop system. It features a novel unified representation that creates a bidirectional link between the physical state and visual primitives, allowing generative refinements to correct both the dynamics and appearance. It also introduces a robust update mechanism that gathers supervision from multiple viewpoints to resolve optimization ambiguity. Experiments demonstrate that from a single image, PerpetualWonder can successfully simulate complex, multi-step interactions from long-horizon actions, maintaining physical plausibility and visual consistency.
Read the full abstract ▾
All authors Jiahao Zhan, Zizhang Li, Hong-Xing Yu, Jiajun Wu
Computer Vision
Laminating Representation Autoencoders for Efficient Diffusion
Ramón Calvo-González and François Fleuret
Recent work has shown that
diffusion models can generate high-quality images by operating directly on SSL patch features rather than pixel-space latents. However, the dense patch grids from encoders like DINOv2 contain significant redundancy, making
diffusion needlessly expensive. We introduce FlatDINO, a variational autoencoder that compresses this representation into a one-dimensional sequence of just 32 continuous
tokens -an 8x reduction in sequence length and 48x compression in total dimensionality. On ImageNet 256x256, a DiT-XL trained on FlatDINO latents achieves a gFID of 1.80 with classifier-free guidance while requiring 8x fewer FLOPs per forward pass and up to 4.5x fewer FLOPs per training step compared to
diffusion on uncompressed DINOv2 features. These are preliminary results and this work is in progress.
Read the full abstract ▾
Computer Vision
When LLaVA Meets Objects: Token Composition for Vision-Language-Models
Soumya Jahagirdar, Walid Bousselham, Anna Kukleva et al. (+1 more)
Current autoregressive
Vision Language Models (VLMs) usually rely on a large number of visual
tokens to represent images, resulting in a need for more compute especially at inference time. To address this problem, we propose Mask-LLaVA, a framework that leverages different levels of visual features to create a compact yet information-rich visual representation for autoregressive VLMs. Namely, we combine mask-based object representations together with global
tokens and local patch
tokens. While all
tokens are used during training, it shows that the resulting model can flexibly drop especially the number of mask-based object-
tokens at test time, allowing to adapt the number of
tokens during inference without the need to retrain the model and without a significant drop in performance. We evaluate the proposed approach on a suite of standard
benchmarks showing results competitive to current
token efficient methods and comparable to the original LLaVA baseline using only a fraction of visual
tokens. Our analysis demonstrates that combining multi-level features enables efficient learning with fewer
tokens while allowing dynamic
token selection at test time for good performance.
Read the full abstract ▾
All authors Soumya Jahagirdar, Walid Bousselham, Anna Kukleva, Hilde Kuehne
Computer Vision · Cross-listed from cs.RO
Yi Gu, Yukang Gao, Yangchen Zhou et al. (+7 more)
Pose and motion priors play a crucial role in
humanoid robotics. Although such priors have been widely studied in human motion recovery (HMR) domain with a range of models, their adoption for
humanoid robots remains limited, largely due to the scarcity of high-quality
humanoid motion data. In this work, we introduce Pose Distance Fields for
Humanoid Robots (PDF-HR), a lightweight prior that represents the
robot pose distribution as a continuous and differentiable
manifold. Given an arbitrary pose, PDF-HR predicts its distance to a large corpus of retargeted
robot poses, yielding a smooth measure of pose plausibility that is well suited for optimization and control. PDF-HR can be integrated as a reward shaping term, a regularizer, or a standalone plausibility scorer across diverse pipelines. We evaluate PDF-HR on various
humanoid tasks, including single-trajectory motion tracking, general motion tracking, style-based motion mimicry, and general motion retargeting. Experiments show that this plug-and-play prior consistently and substantially strengthens strong baselines. Code and models are publicly available at https://gaoyukang33.github.io/PDF-HR.
Read the full abstract ▾
All authors Yi Gu, Yukang Gao, Yangchen Zhou, Xingyu Chen, Yixiao Feng, Mingle Zhao, Yunyang Mo, Zhaorui Wang, Lixin Xu, Renjing Xu
Computer Vision
LitS: A novel Neighborhood Descriptor for Point Clouds
Jonatan B. Bastos, Francisco F. Rivera, Oscar G. Lorenzo et al. (+4 more)
With the advancement of 3D scanning technologies, point clouds have become fundamental for representing 3D spatial data, with applications that span across various scientific and technological fields. Practical analysis of this data depends crucially on available neighborhood descriptors to accurately characterize the local geometries of the point cloud. This paper introduces LitS, a novel neighborhood descriptor for 2D and 3D point clouds. LitS are piecewise constant functions on the unit circle that allow points to keep track of their surroundings. Each element in LitS' domain represents a direction with respect to a local reference system. Once constructed, evaluating LitS at any given direction gives us information about the number of neighbors in a cone-like region centered around that same direction. Thus, LitS conveys a lot of information about the local neighborhood of a point, which can be leveraged to gain global structural understanding by analyzing how LitS changes between close points. In addition, LitS comes in two versions ('regular' and 'cumulative') and has two parameters, allowing them to adapt to various contexts and types of point clouds. Overall, they are a versatile neighborhood descriptor, capable of capturing the nuances of local point arrangements and resilient to common point cloud data issues such as variable density and noise.
Read the full abstract ▾
All authors Jonatan B. Bastos, Francisco F. Rivera, Oscar G. Lorenzo, David L. Vilariño, José C. Cabaleiro, Alberto M. Esmorís, Tomás F. Pena
Computer Vision · Cross-listed from cs.LG
It's Not a Lottery, It's a Race: Understanding How Gradient Descent Adapts the Network's Capacity to the Task
Hannah Pinson
Our theoretical understanding of neural networks is lagging behind their empirical success. One of the important unexplained phenomena is why and how, during the process of training with gradient descent, the theoretical capacity of neural networks is reduced to an effective capacity that fits the task. We here investigate the mechanism by which gradient descent achieves this through analyzing the learning dynamics at the level of individual
neurons in single hidden layer ReLU networks. We identify three dynamical principles, namely mutual
alignment, unlocking and racing, that together explain why we can often successfully reduce capacity after training through the merging of equivalent
neurons or the
pruning of low norm weights. We specifically explain the mechanism behind the lottery ticket
conjecture, or why the specific, beneficial initial conditions of some
neurons lead them to obtain higher weight norms.
Read the full abstract ▾
Computer Vision
Toward Reliable and Explainable Nail Disease Classification: Leveraging Adversarial Training and Grad-CAM Visualization
Farzia Hossain, Samanta Ghosh, Shahida Begum et al. (+4 more)
Human nail diseases are gradually observed over all age groups, especially among older individuals, often going ignored until they become severe. Early detection and accurate diagnosis of such conditions are important because they sometimes reveal our body's health problems. But it is challenging due to the inferred visual differences between disease types. This paper presents a machine learning-based model for automated classification of nail diseases based on a publicly available
dataset, which contains 3,835 images scaling six categories. In 224x224 pixels, all images were resized to ensure consistency. To evaluate performance, four well-known CNN models-InceptionV3, DenseNet201, EfficientNetV2, and ResNet50 were trained and analyzed. Among these, InceptionV3 outperformed the others with an accuracy of 95.57%, while DenseNet201 came next with 94.79%. To make the model stronger and less likely to make mistakes on tricky or noisy images, we used
adversarial training. To help understand how the model makes decisions, we used SHAP to highlight important features in the predictions. This system could be a helpful support for doctors, making nail disease diagnosis more accurate and faster.
Read the full abstract ▾
All authors Farzia Hossain, Samanta Ghosh, Shahida Begum, B. M. Shahria Alam, Mohammad Tahmid Noor, Md Parvez Mia, Nishat Tasnim Niloy
Computer Vision
XtraLight-MedMamba for Classification of Neoplastic Tubular Adenomas
Aqsa Sultana, Rayan Afsar, Ahmed Rahu et al. (+5 more)
Accurate risk stratification of precancerous polyps during routine colonoscopy screening is a key strategy to reduce the incidence of colorectal cancer (CRC). However, assessment of low-grade dysplasia remains limited by subjective histopathologic interpretation. Advances in computational pathology and deep learning offer new opportunities to identify subtle, fine morphologic patterns associated with malignant progression that may be imperceptible to the human eye. In this work, we propose XtraLight-MedMamba, an ultra-lightweight state-space-based deep learning framework to classify neoplastic tubular adenomas from whole-slide images (WSIs). The architecture is a blend of a ConvNeXt-based shallow feature extractor with parallel
vision mamba blocks to efficiently model local texture cues within global contextual structure. An integration of the Spatial and Channel
Attention Bridge (SCAB) module enhances multiscale feature extraction, while the Fixed Non-Negative Orthogonal Classifier (FNOClassifier) enables substantial parameter reduction and improved generalization. The model was evaluated on a curated
dataset acquired from patients with low-grade tubular adenomas, stratified into case and control cohorts based on subsequent CRC development. XtraLight-MedMamba achieved an accuracy of 97.18% and an F1-score of 0.9767 using approximately 32,000 parameters, outperforming
transformer-based and conventional Mamba architectures, which have significantly higher model complexity and computational burden, making it suitable for resource-constrained areas.
Read the full abstract ▾
All authors Aqsa Sultana, Rayan Afsar, Ahmed Rahu, Surendra P. Singh, Brian Shula, Brandon Combs, Derrick Forchetti, Vijayan K. Asari
Computer Vision
X2HDR: HDR Image Generation in a Perceptually Uniform Space
Ronghuan Wu, Wanchao Su, Kede Ma et al. (+2 more)
High-dynamic-range (HDR) formats and displays are becoming increasingly prevalent, yet state-of-the-art image generators (e.g., Stable
Diffusion and FLUX) typically remain limited to low-dynamic-range (LDR) output due to the lack of large-scale HDR training data. In this work, we show that existing pretrained
diffusion models can be easily adapted to HDR generation without retraining from scratch. A key challenge is that HDR images are natively represented in linear RGB, whose intensity and color statistics differ substantially from those of sRGB-encoded LDR images. This gap, however, can be effectively bridged by converting HDR inputs into perceptually uniform encodings (e.g., using PU21 or PQ). Empirically, we find that LDR-pretrained variational autoencoders (VAEs) reconstruct PU21-encoded HDR inputs with fidelity comparable to LDR data, whereas linear RGB inputs cause severe degradations. Motivated by this finding, we describe an efficient adaptation strategy that freezes the VAE and finetunes only the denoiser via low-rank adaptation in a perceptually uniform space. This results in a unified computational method that supports both text-to-HDR synthesis and single-image RAW-to-HDR reconstruction. Experiments demonstrate that our perceptually encoded adaptation consistently improves perceptual fidelity, text-image
alignment, and effective dynamic range, relative to previous techniques.
Read the full abstract ▾
All authors Ronghuan Wu, Wanchao Su, Kede Ma, Jing Liao, Rafał K. Mantiuk
Computer Vision
VISTA-Bench: Do Vision-Language Models Really Understand Visualized Text as Well as Pure Text?
Qing'an Liu, Juntong Feng, Yuhao Wang et al. (+6 more)
Vision-
Language Models (VLMs) have achieved impressive performance in cross-modal understanding across textual and visual inputs, yet existing
benchmarks predominantly focus on pure-text queries. In real-world scenarios,
language also frequently appears as visualized text embedded in images, raising the question of whether current VLMs handle such input requests comparably. We introduce VISTA-Bench, a systematic
benchmark from
multimodal perception,
reasoning, to unimodal understanding domains. It evaluates visualized text understanding by contrasting pure-text and visualized-text questions under controlled rendering conditions. Extensive evaluation of over 30 representative VLMs reveals a pronounced modality gap: models that perform well on pure-text queries often degrade substantially when equivalent semantic content is presented as visualized text. This gap is further amplified by increased perceptual difficulty, highlighting sensitivity to rendering variations despite unchanged semantics. Overall, VISTA-Bench provides a principled evaluation framework to diagnose this limitation and to guide progress toward more unified
language representations across tokenized text and pixels. The source
dataset and code are publicly available at https://github.com/QingAnLiu/VISTA-Bench.
Read the full abstract ▾
All authors Qing'an Liu, Juntong Feng, Yuhao Wang, Xinzhe Han, Yujie Cheng, Yue Zhu, Haiwen Diao, Yunzhi Zhuge, Huchuan Lu
Computer Vision
Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention
Chengtao Lv, Yumeng Shi, Yushi Huang et al. (+3 more)
Advanced autoregressive (AR)
video generation models have improved visual fidelity and interactivity, but the quadratic complexity of
attention remains a primary bottleneck for efficient deployment. While existing sparse
attention solutions have shown promise on bidirectional models, we identify that applying these solutions to AR models leads to considerable performance degradation for two reasons: isolated consideration of chunk generation and insufficient utilization of past informative context. Motivated by these observations, we propose Light Forcing, the first sparse
attention solution tailored for AR
video generation models. It incorporates a Chunk-Aware Growth mechanism to quantitatively estimate the contribution of each chunk, which determines their sparsity allocation. This progressive sparsity increase strategy enables the current chunk to inherit prior knowledge in earlier chunks during generation. Additionally, we introduce a Hierarchical Sparse
Attention to capture informative historical and local context in a coarse-to-fine manner. Such two-level mask selection strategy (i.e., frame and block level) can adaptively handle diverse
attention patterns. Extensive experiments demonstrate that our method outperforms existing sparse
attention in quality (e.g., 84.5 on VBench) and efficiency (e.g., $1.2{\sim}1.3\times$ end-to-end speedup). Combined with other efficient solutions, Light Forcing further achieves a $2.0{\sim}3.0\times$ end-to-end speedup across diverse GPUs (e.g., 27.4\,FPS on RTX 5090 and 33.9\,FPS on H100). Code is released via this \href{https://github.com/chengtao-lv/LightForcing}{link}.
Read the full abstract ▾
All authors Chengtao Lv, Yumeng Shi, Yushi Huang, Ruihao Gong, Shen Ren, Wenya Wang
Computer Vision · Cross-listed from cs.LG
Generative Modeling via Drifting
Mingyang Deng, He Li, Tianhong Li et al. (+2 more)
Generative modeling can be formulated as learning a mapping f such that its pushforward distribution matches the data distribution. The pushforward behavior can be carried out iteratively at inference time, for example in
diffusion and flow-based models. In this paper, we propose a new paradigm called Drifting Models, which evolve the pushforward distribution during training and naturally admit one-step inference. We introduce a drifting field that governs the sample movement and achieves
equilibrium when the distributions match. This leads to a training objective that allows the neural network optimizer to evolve the distribution. In experiments, our one-step generator achieves state-of-the-art results on ImageNet at 256 x 256 resolution, with an FID of 1.54 in latent space and 1.61 in pixel space. We hope that our work opens up new opportunities for high-quality one-step generation.
Read the full abstract ▾
All authors Mingyang Deng, He Li, Tianhong Li, Yilun Du, Kaiming He
Computer Vision
Mitigating Long-Tail Bias via Prompt-Controlled Diffusion Augmentation
Buddhi Wijenayake, Nichula Wasalathilake, Roshan Godaliyadda et al. (+3 more)
Long-tailed class imbalance remains a fundamental obstacle in semantic
segmentation of high-resolution remote-sensing imagery, where dominant classes shape learned representations and rare classes are systematically under-segmented. This challenge becomes more acute in cross-domain settings such as LoveDA, which exhibits an explicit Urban/Rural split with substantial appearance differences and inconsistent class-frequency statistics across domains. We propose a
prompt-controlled
diffusion augmentation framework that generates paired label-image samples with explicit control over semantic composition and domain, enabling targeted enrichment of underrepresented classes rather than indiscriminate
dataset expansion. A domain-aware, masked, ratio-conditioned discrete
diffusion model first synthesizes layouts that satisfy class-ratio targets while preserving realistic spatial co-occurrence, and a ControlNet-guided
diffusion model then renders photorealistic, domain-consistent images from these layouts. When mixed with real data, the resulting synthetic pairs improve multiple
segmentation backbones, especially on minority classes and under domain shift, showing that better downstream
segmentation comes from adding the right samples in the right proportions. Source codes, pretrained models, and synthetic
datasets are available at \href{https://buddhi19.github.io/SyntheticGen}{buddhi19.github.io/SyntheticGen}.
Read the full abstract ▾
All authors Buddhi Wijenayake, Nichula Wasalathilake, Roshan Godaliyadda, Vijitha Herath, Parakrama Ekanayake, Vishal M. Patel
Computer Vision
How to rewrite the stars: Mapping your orchard over time through constellations of fruits
Gonçalo P. Matos, Carlos Santiago, João P. Costeira et al. (+2 more)
Following crop growth through the vegetative cycle allows farmers to predict fruit setting and yield in early stages, but it is a laborious and non-scalable task if performed by a human who has to manually measure fruit sizes with a caliper or dendrometers. In recent years, computer
vision has been used to automate several tasks in precision agriculture, such as detecting and counting fruits, and estimating their size. However, the fundamental problem of matching the exact same fruits from one
video, collected on a given date, to the fruits visible in another
video, collected on a later date, which is needed to track fruits' growth through time, remains to be solved. Few attempts were made, but they either assume that the camera always starts from the same known position and that there are sufficiently distinct features to match, or they used other sources of data like GPS. Here we propose a new paradigm to tackle this problem, based on constellations of 3D centroids, and introduce a descriptor for very sparse 3D point clouds that can be used to match fruits across
videos. Matching constellations instead of individual fruits is key to deal with non-
rigidity, occlusions and challenging imagery with few distinct visual features to track. The results show that the proposed method can be successfully used to match fruits across
videos and through time, and also to build an orchard map and later use it to locate the camera pose in 6DoF, thus providing a method for autonomous
navigation of
robots in the orchard and for selective fruit picking, for example.
Read the full abstract ▾
All authors Gonçalo P. Matos, Carlos Santiago, João P. Costeira, Ricardo L. Saldanha, Ernesto M. Morgado