Computer Vision
Dyna-DINO: Efficient ViT Distillation Via Adaptive Representation Anchoring
Jiaqi Zhang, Ashton Lee, Anthony Wong et al. (+3 more)
Vision Foundation Models (VFMs) with
Vision Transformer (ViT) backbones, such as DINOv2, have become essential for downstream tasks like object recognition and semantic
segmentation. The immense computational requirements of backbones often necessitate
distillation into smaller architectures for edge deployment. Feature-based knowledge
distillation (KD) often suffers from the teacher-student gap; the student struggles to imitate teacher's complex feature map due to its limited capacity. To mitigate this bottleneck, we propose Dyna-DINO: Efficient ViT
Distillation Via Adaptive Representation Anchoring, a training curriculum for ViT feature-based knowledge
distillation. By utilizing the teacher's intermediate feature maps as a sequence of progressively more difficult targets, our curriculum allows the student to build a foundational representation before tackling higher-level abstractions. Our results demonstrate that this paradigm significantly accelerates convergence through adaptive difficulty selection across various student model sizes and
dataset scales. With our curriculum, the Dyna-DINO distilled ViT-S achieves 90.1% accuracy on ImageNet-100, a +12.24% improvement compared with baseline. On ImageNet-1K, Dyna-DINO achieves +3.9% and +6.09% improvement for the instance
retrieval task on the Oxford and Paris
datasets, +1.93% improvements on semantic
segmentation task, as well as meaningful performance gain on classification task. Furthermore, the curriculum enables 25.1% savings in training FLOPs and 21% savings in training time on ImageNet-100 by implementing early-stopping for teacher inference during the initial stages of training. Code is available at https://github.com/KevinZ0217/Dyna-DINO
Read the full abstract ▾
All authors Jiaqi Zhang, Ashton Lee, Anthony Wong, John Zou, Sami BuGhanem, Randall Balestriero
Computer Vision
Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers
Fabio De Sousa Ribeiro, Emma A.M. Stanley, Charles Jones et al. (+7 more)
We introduce the first generative foundation model for chest radiograph synthesis trained from scratch at the billion-parameter scale. Existing radiographic AI models often suffer from poor generalisation across patient subpopulations, institutions, and acquisition settings, resulting in limited real-world clinical utility. Controlled, high-fidelity synthesis of chest radiographs is a promising path toward diversifying clinical
datasets and evaluating the robustness of diagnostic models. Therefore, we present the largest specialist generative foundation model for chest radiographs to date, with over 1.3B parameters, trained for 1.6T
tokens on a curated, heterogeneous
dataset comprising 1.2M radiographs and clinical expert-guided metadata. Our model supports controllable radiograph generation and editing across multiple demographic subgroups, acquisition views, and a dozen pathologies. Moreover, we significantly advance the state of the art in radiograph synthesis fidelity, producing images that are indistinguishable from real radiographs to clinical experts.
Read the full abstract ▾
All authors Fabio De Sousa Ribeiro, Emma A.M. Stanley, Charles Jones, Tian Xia, Dominic C. Marshall, Laurent Renard Triché, Christopher V. Cosgriff, Panagiotis Dimitrakopoulos, Sotirios A. Tsaftaris, Ben Glocker
Computer Vision · Cross-listed from cs.LG
3D-DLP: Self-Supervised 3D Object-Centric Scene Representation Learning
Ellina Zhang, Madhaven Iyengar, Amir Zadeh et al. (+4 more)
We introduce 3D-DLP, a self-supervised object-centric representation learning model that decomposes scene-level RGB-D or voxel observations into a set of 3D latent particles. Building on the Deep Latent Particles (DLP) framework, each particle encodes disentangled attributes, including 3D keypoint position, bounding box dimensions, and appearance features, and represents a distinct entity in the scene. The model learns interpretable per-particle
segmentation maps through an end-to-end self-supervised reconstruction objective. We demonstrate on both simulated and real-world
datasets that the learned latent space is interpretable and controllable: by manipulating particle positions and decoding, we can generate novel scene configurations. Furthermore, we show that leveraging these compact 3D latent particles for downstream robotic manipulation improves performance over baselines that either lack explicit 3D information or rely on
memory-intensive dense 3D inputs without object-centric structure. Code and
videos are available at https://eubooks3003.github.io/3d-dlp.
Read the full abstract ▾
All authors Ellina Zhang, Madhaven Iyengar, Amir Zadeh, Chuan Li, Deepak Pathak, David Held, Tal Daniel
Computer Vision
Native Active Perception as Reasoning for Omni-Modal Understanding
Zhenghao Xing, Ruiyang Xu, Yuxuan Wang et al. (+8 more)
Passive models for long
video understanding typically rely on a "watch-it-all" paradigm, processing frames uniformly regardless of query difficulty, causing computational cost to grow with
video duration. Although interactive frameworks have emerged, they often rely on global pre-scanning, and their context cost still scales with
video length. We propose OmniAgent, the first native omni-modal
agent that formulates
video understanding as a POMDP-based iterative Observation-Thought-Action cycle. OmniAgent executes on-demand actions to selectively distill audio-visual cues into a persistent textual
memory, effectively decoupling
reasoning complexity from raw
video duration. To operationalize this, we introduce (1) Agentic Supervised Fine-Tuning to bootstrap native active perception via best-of-N trajectory synthesis with dual-stage quality control, and (2) Agentic Reinforcement Learning with TAURA (Turn-aware Adaptive Uncertainty Rescaled Advantage), which leverages turn-level
entropy to steer credit assignment toward pivotal discovery turns. Crucially, OmniAgent exhibits positive test-time scaling, where performance improves as the number of
reasoning turns increases, validating the efficacy of active perception. Empirical results across ten
benchmarks (e.g., VideoMME, LVBench) demonstrate that OmniAgent achieves state-of-the-art performance among open-source models. Notably, on LVBench, our 7B
agent outperforms the 10$\times$ larger Qwen2.5-VL-72B (50.5% vs. 47.3%).
Read the full abstract ▾
All authors Zhenghao Xing, Ruiyang Xu, Yuxuan Wang, Jinzheng He, Ziyang Ma, Qize Yang, Yunfei Chu, Jin Xu, Junyang Lin, Chi-Wing Fu, Pheng-Ann Heng
Computer Vision
Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games
Shengyuan Ding, Xilin Wei, Xinyu Fang et al. (+4 more)
Deploying
multimodal foundation models as closed-loop policies increasingly requires conditioning actions on observations that are no longer visible. However, existing
benchmarks either expose the full state, conflate hidden-state reconstruction with other
agent skills, or test recall only after an episode has ended. We introduce RNG-Bench (Reconstructive Non-Markov Games), a
benchmark suite designed to isolate a base model's ability to reconstruct past observations and act on them during multi-step interaction. RNG-Bench includes two complementary games: Matching Pairs, where card identities briefly revealed at specific locations must later be recalled, and 3D Maze, where
egocentric views must be integrated into a spatial map. Both games are evaluated under a unified harness with three controlled difficulty axes: grid size, visual pattern, and observation modality. The
benchmark further introduces a head-to-head duel protocol to control for instance-level variance and a
Memory Gap metric that disentangles forgetting from poor action selection. The hardest configurations require contexts of roughly 128K
tokens and 350 image inputs per episode, and remain far from saturated by frontier MLLMs.
Memory Gap analysis shows that most residual errors stem from forgetting earlier observations rather than from suboptimal decision making. Finally, fine-tuning Qwen3.5-9B on optimal-policy rollouts and filtered model demonstrations improves performance on RNG-Bench and transfers to existing
benchmarks without degrading general
multimodal capability.
Read the full abstract ▾
All authors Shengyuan Ding, Xilin Wei, Xinyu Fang, Haodong Duan, Dahua Lin, Jiaqi Wang, Yuhang Zang
Computer Vision · Cross-listed from cs.RO
Do as I Do: Dexterous Manipulation Data from Everyday Human Videos
Bhawna Paliwal, Haritheja Etukuru, William Liang et al. (+3 more)
How can we scalably generate data for robotic manipulation, especially on human-like platforms such as dexterous multi-fingered hands? Learning from human
videos has recently emerged as a likely answer to this question. However, difficulties in estimating hand-object interaction and crossing the human-to-
robot embodiment gap have hindered the adoption of abundant
monocular RGB-only human
videos as the primary source of
robot manipulation data. In this work, we present DO AS I DO, an
algorithm to reconstruct and retarget
monocular RGB human
videos to multi-fingered dexterous robotic hands. DO AS I DO reconstructs hand-object interactions from various
egocentric and exocentric in-the-wild
video sources. The
algorithm then retargets these hand-object interaction estimates into a sequence of actions executable in the real world, yielding
robot-complete manipulation data from disparate human
videos. Overall, DO AS I DO outperforms previous state of the art in estimating hand-object interactions and extracting dexterous manipulation trajectories from RGB
videos, as we show in experiments on
datasets with ground truths and on a
dataset of
video clips collected online. Our experiments enable us to propose an efficacy playbook for practitioners collecting human data for manipulation.
Read the full abstract ▾
All authors Bhawna Paliwal, Haritheja Etukuru, William Liang, Pieter Abbeel, Nur Muhammad Mahi Shafiullah, Jitendra Malik
Computer Vision · Cross-listed from cs.SD
Reference-Driven Multi-Speaker Audio Scene Generation from In-the-Wild Priors
Michael Finkelson, Daniel Segal, Eitan Richardson et al. (+7 more)
Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker
embeddings. These systems operate within
speech-only pipelines that produce clean vocal sequences without the ambient texture of real conversations. We take a different approach. Our method, ScenA, conditions a text-to-audio flow-matching foundation model, pretrained on large-scale in-the-wild data, directly on multiple reference voices and a free-form natural
language prompt that describes an entire multi-speaker audio scene. Leveraging such a foundational model allows us to inherit its capacity for natural, non-studio audio: background noise, room acoustics, overlapping dialogue, and spontaneous paralinguistic events, while adding multi-speaker control without any per-turn structure. Concretely, reference latents are concatenated into the model's
token sequence and distinguished by lightweight identity-aware positional encodings. However, we identify a critical obstacle to this approach: the Reference Shortcut. During training under standard noise schedules, the model can identify the matching reference by acoustic similarity to the noisy target, bypassing the text
prompt entirely. We address this with a high-noise-biased timestep distribution that forces the model to rely on the text
prompt for speaker assignment. We evaluate ScenA on the CoVoMix2-Dialogue
benchmark, showing that it outperforms existing multi-speaker systems on speaker-binding metrics while generating rich conversational audio with overlapping
speech, emotional vocalizations, and ambient sound. Our results demonstrate the advantage of using a general-purpose audio model conditioned on a free-form scene description, rather than passing structured dialog scripts through a
speech-only pipeline.
Read the full abstract ▾
All authors Michael Finkelson, Daniel Segal, Eitan Richardson, Shahar Armon, Nani Goldring, Poriya Panet, Nir Zabari, Benjamin Brazowski, Or Patashnik, Yoav HaCohen
Computer Vision
NeuMesh++: Towards Versatile and Efficient Volumetric Editing with Disentangled Neural Mesh-based Implicit Field
Chong Bao, Yuan Li, Bangbang Yang et al. (+5 more)
Recently neural implicit rendering techniques have evolved rapidly and demonstrated significant advantages in novel view synthesis and 3D scene reconstruction. However, existing neural rendering methods for editing purposes offer limited functionalities, e.g., rigid transformation and category-specific editing. In this paper, we present a novel mesh-based representation by encoding the neural radiance field with disentangled geometry, texture, and semantic codes on mesh vertices, which empowers a set of efficient and comprehensive editing functionalities, including mesh-guided geometry editing, designated texture editing with texture swapping, filling and painting operations, and semantic-guided editing. To this end, we develop several techniques including a novel local space parameterization to enhance rendering quality and training stability, a learnable modification color on vertex to improve the fidelity of texture editing, a spatial-aware optimization strategy to realize precise texture editing, and a semantic-aided region selection to ease the laborious annotation of implicit field editing. Extensive experiments and editing examples on both real and synthetic
datasets demonstrate the superiority of our method on representation quality and editing ability. Project page: https://zju3dv.github.io/neumeshplusplus/
Read the full abstract ▾
All authors Chong Bao, Yuan Li, Bangbang Yang, Yujun Shen, Hujun Bao, Zhaopeng Cui, Yinda Zhang, Guofeng Zhang
Computer Vision
Confidence is Not Reliability: Rethinking MC Dropout in Brain Tumour Segmentation
Xin Ci Wong, Duygu Sarikaya, Kieran Zucker et al. (+2 more)
Glioma
segmentation in multiparametric MRI is a critical component of treatment planning. A
segmentation model that fails silently on treatment-critical sub-regions represents a patient safety risk that overlap-based metrics such as Dice scores cannot expose. We ask whether voxel-level uncertainty estimation via Monte Carlo (MC) Dropout can reliably identify
segmentation errors in clinically critical sub-regions, and whether calibration failure modes are detectable from standard reporting metrics alone. In an empirical two-model case study on 126 BraTS21 patients, we evaluate a high-performance pretrained SegResNet and a locally trained UNet with residual units (UNet-Res). MC dropout preserved
segmentation accuracy ($|\Delta \text{Dice}|$ $<0.01$) while achieving strong uncertainty-error
alignment (AUROC for
entropy (H) $\approx$0.97), indicating uncertainty correctly ranks erroneous voxels above correct ones.
Entropy-based patient stratification identified a high-uncertainty subgroup with substantially lower
segmentation performance (median whole-tumour Dice $0.835$ vs. $0.925$), supporting uncertainty as a practical triage signal. However, global
alignment can mask important region-specific differences. Despite similar AUROC, UNet-Res exhibited near-zero enhancing tumour
entropy ($0.054$) and Expected Calibration Error (ECE) of $0.915$, with a Dice of only $0.714$, indicating severely miscalibrated confidence on the most clinically critical sub-region, a failure mode invisible to standard Dice and AUROC reporting. These findings demonstrate that strong uncertainty-error
alignment is necessary but insufficient for clinical safety: sub-region-specific calibration assessment must accompany AUROC evaluation when selecting models for clinical deployment.
Read the full abstract ▾
All authors Xin Ci Wong, Duygu Sarikaya, Kieran Zucker, Marc De Kamps, Nishant Ravikumar
Computer Vision
A Unified Framework for Efficient Remote Sensing Visual Question Answering: Adapting Dual, Hybrid, and Encoder-Decoder Architectures
Timothy Agboada, Shikha Chandel, Yadav Raj Ghimire et al. (+1 more)
Visual Question Answering (VQA) in the Remote Sensing (RS) domain presents unique challenges due to the high resolution, multi scale object distribution, and semantic complexity of aerial imagery. While general domain Foundation Models have achieved remarkable success, their direct application to RSVQA is hindered by massive domain shifts and the computationally prohibitive nature of full fine tuning. This study presents a comparative analysis of RS Adapter, a Parameter Efficient Fine Tuning (PEFT) strategy, applied across three distinct
Vision Language Model (VLM) architectures: the Dual Encoder CLIP, the Encoder Decoder BLIP, and the Hybrid FLAVA. We introduce a unified architectural surgery pipeline that injects lightweight bottleneck adapters into the
attention and MLP layers of frozen backbones, enabling rapid adaptation with less than 5 percent of trainable parameters. Experimental results on the high resolution RSVQA x
dataset demonstrate that while all adapted models achieve convergence, the Hybrid FLAVA architecture offers a superior balance of
multimodal reasoning and
retrieval capabilities compared to its unimodal counterparts. Our findings establish a new baseline for resource efficient VQA in disaster assessment and urban monitoring.
Read the full abstract ▾
All authors Timothy Agboada, Shikha Chandel, Yadav Raj Ghimire, Leila Hashemi-Beni
Computer Vision
TextRich: A Multi-Domain Benchmark for Detecting AI-Generated Text-Rich Images from GPT-Image-2
Yijin Wang, Shuyi Wang, Wenhan Zhang et al. (+1 more)
Text-rich images often contain
privacy-sensitive, transactional, or decision-relevant information. As recent
multimodal image generation models become increasingly capable of synthesizing realistic textual content and structured visual designs, detecting AI-generated text-rich images has become an important challenge for digital trust and content authenticity. Existing
benchmarks, however, largely focus on object-centric images and provide limited coverage of scenarios where textual semantics and layout organization are central. In this paper, we introduce TextRich, a multi-domain
benchmark for detecting text-rich images generated by OpenAI's GPT-Image-2. The
benchmark contains 12,095 images across six representative categories: commercial posters, infographic charts, academic posters, receipts, tables, and UI screenshots. Using this
benchmark, we evaluate five representative AI-generated image detectors under a zero-shot setting and further explore the capability of a
multimodal vision-
language model for this task. Our results reveal substantial performance variations across text-rich domains, where existing AI-generated image detectors exhibit distinct strengths and failure modes. Although the strongest detector achieves competitive overall performance, it remains ineffective on certain structured categories and highly sensitive to JPEG compression.
Vision-
language models provide a promising complementary approach, but still struggle with highly structured text-rich images. These findings highlight the need for text- and layout-aware detection methods for modern AI-generated images. Our
dataset is released at https://huggingface.co/
datasets/Shuyiww/TextRich.
Read the full abstract ▾
All authors Yijin Wang, Shuyi Wang, Wenhan Zhang, Yuqi Ouyang
Computer Vision
CABLE: Cloud-Assisted Bandwidth-efficient LMM-based Encoding for V2X Systems
Haohua Que, Zhipeng Bao, Qianyi Wu et al. (+1 more)
Cloud-hosted large
multimodal models (LMMs) can provide strong open-vocabulary perception for Vehicle-to-Everything systems, but naively transmitting full-resolution frames from edge to cloud causes severe communication overhead and high cloud-side prefill latency. We present CABLE, a cloud-assisted bandwidth-efficient LMM-based encoding framework for edge-cloud perception. CABLE propagates the previous cloud
segmentation mask on the edge using ego-motion compensation, refines it with residual-motion cues, and consolidates disconnected regions via a corridor envelope to form a robust region of interest (ROI). Only ROI-masked images are uploaded, while the cloud
segmentation output is fed back as the prior for the next frame, forming a mask-to-ROI-to-LMM feedback loop. Experiments on five
datasets (nuScenes, WOD-ZB, Waymo, KITTI, and CADC) show consistent communication savings while largely preserving perception, achieving $73$–$87\%$ ROI pixel-coverage reduction with $5$–$8\times$ estimated LMM prefill speedup at a modest detection-quality trade-off relative to full-frame inference.
Read the full abstract ▾
All authors Haohua Que, Zhipeng Bao, Qianyi Wu, Handong Yao
Computer Vision
OneCanvas: 3D Scene Understanding via Panoramic Reprojection
Bartłomiej Baranowski, Dave Zhenyu Chen and Matthias Nießner
Existing approaches to 3D scene understanding in
Vision-
Language Models (VLMs) either rely on complex, model-specific geometry encoders or large training budgets in pursuit of spatial
reasoning. Instead, OneCanvas aggregates patch features from all views onto a single equirectangular panoramic canvas. Namely, each patch is unprojected to a 3D world coordinate using its depth and camera pose, then placed on the canvas at the continuous longitude and latitude of that point as seen from the canvas origin, with no rasterization or aggregation across overlapping views. A 3D position
embedding of the patch's metric coordinates is added to its feature, restoring the depth lost when collapsing the world position to an angular canvas coordinate. Patches from all frames thus share one spatial coordinate system with no fusion or major architectural modifications of the backbone. The pretrained VLM consumes this representation as if it were an ordinary image. Because the canvas can be centered on any pose of interest, the same representation directly supports situated
reasoning from a specific viewpoint, a common requirement in robotics and
embodied AI. Thanks to this representation, we can also introduce a spatial pretraining curriculum: by procedurally placing patch features of objects, drawn from real images, at chosen 3D world positions on an otherwise empty canvas, we generate on-the-fly supervision spanning a broad range of spatial
reasoning tasks, with answer distributions controlled to reduce spatial
reasoning shortcuts. OneCanvas achieves state-of-the-art results on SQA3D and VSI-Bench and generalizes to out-of-distribution data on SPBench, using an order of magnitude less training compute than its competitors closest in
benchmark performance.
Read the full abstract ▾
Computer Vision
Transformer Geometry Observatory TGO-I: Spectral Geometry Observatory
Kaustubh Kapil and Kishor P. Upla
Despite the widespread adoption of
Vision Transformers (ViTs) and their success across numerous computer
vision applications, the fundamental understanding of their dimensional and representational geometry remains relatively underexplored. To address this gap, we introduce
Transformer Geometry Observatory (TGO), a systematic framework of experiments and analysis pipelines designed to investigate the representational geometry and dynamics of
Vision Transformers. TGO-I, the first installment of the framework, focuses on the spectral geometry of ViT representations. Using a ViT-Small/16 model trained on ImageNet-100, we analyze Effective Rank, Stable Rank, Participation Ratio, Spectral
Entropy, Spectral Flatness, Spectral Anisotropy, covariance structure, eigenspectra, and singular value spectra throughout training. Our results reveal a consistent increase in dimensional utilization, accompanied by decreasing anisotropy, increasing spectral
entropy, increasing participation ratio, and progressively flatter eigenspectra. Contrary to the common intuition that training should concentrate information into a small number of dominant directions, we observe a progressive redistribution of variance across representational dimensions. This phenomenon is particularly pronounced in the final CLS
token representation, which exhibits the highest effective dimensionality and lowest anisotropy within the network.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Seeing Through Occlusion: Deterministic Arm Kinematic Correction for Robot Teleoperation
Thomas M. Kwok, Nicholas Koenig and Yue Hu
Markerless, single-RGB-D-camera motion capture provides a low-cost and non-invasive alternative to conventional marker-based systems for
robot teleoperation; however, depth estimation often degrades in the presence of self-occlusion, particularly during upper-limb motion. This paper presents an Arm Kinematic Correction (AKC) method that improves depth estimation by enforcing
geometric constraints based on constant arm lengths. The proposed approach reconstructs occluded joint depths by leveraging wrist positions and predefined arm lengths via a deterministic formulation based on the Pythagorean
theorem, thereby avoiding the need for complex probabilistic modeling or parameter tuning. Experimental validation against a Vicon reference system demonstrates reliable performance for both static and dynamic joint motions, evaluated using root-mean-square error (RMSE) and Pearson correlation. Furthermore, motion-mapping teleoperation is successfully demonstrated in both simulated and physical
robot environments. The results show that AKC enhances robustness and preserves anatomical consistency under long-duration, severe self-occlusion, even when paired with less reliable temporal filters, highlighting its practicality for real-time applications such as
robot teleoperation and human-
robot interaction.
Read the full abstract ▾
Computer Vision
GUMP-Net: An interpretable model-data-driven intelligent algorithm for multi-class pelvic segmentation
Liheng Wang, Yinghui Zhang, Licheng Zhang et al. (+3 more)
Pelvic
segmentation is one of the most important and fundamental research problems in precise and intelligent diagnosis and treatment, as well as
surgical planning and
navigation for pelvic fractures. By combining an improved geodesic active contour model with deep neural networks, we propose GUMP-Net, an interpretable model-data-driven intelligent
algorithm for multi-class pelvic
segmentation, in which three network modules are designed to constitute the overall
segmentation framework together: the object detection module for automatic level set initialization, the edge detector module for learning an anatomy-aware edge detector function and the iteration module for deep level set
evolution. Leveraging the advantages of level set representation and deep learning, GUMP-Net shows more accurate, robust and consistent
segmentation performance, especially in small training data situation, compared to the state-of-the-art methods. Extensive experiments on pelvic
datasets demonstrate the rationality and effectiveness of the proposed
algorithm. Further experiments extended to ankle
dataset indicate broader applications to other anatomies. The proposed
algorithm not only provides an efficient
segmentation method for complex fracture reduction, but also gives an interpretable
geometric perspective for understanding deep learning
segmentation.
Read the full abstract ▾
All authors Liheng Wang, Yinghui Zhang, Licheng Zhang, Hailin Xu, Qiyong Cao, Chong Chen
Computer Vision
ROSA-TFormer: A Radar-Optical Sensor-Aware Temporal Transformer for Pinus sylvestris Plantation Classification in Northern Shaanxi Using GEE-Derived Sentinel-1/2 Time Series
Nengbo Zhang and Chang sheng
Accurate identification of Pinus sylvestris var. mongolica plantations is important for monitoring afforestation quality and ecological restoration in northern Shaanxi. This paper proposes ROSA-TFormer, a radar-optical sensor-aware temporal
Transformer for P. sylvestris classification using Sentinel-1/2 time-series data generated on Google Earth Engine. The model integrates separate SAR and optical
embedding branches, a sensor-aware gate, and temporal
attention pooling to capture multi-source seasonal features. Experiments on monthly and half-month point-level
datasets show that ROSA-TFormer achieves strong classification performance, with 99.67% overall accuracy, 99.56% macro F1, and 98.91% P. sylvestris F1 on the HalfMonth-dataBig
dataset. Spatial block validation and ablation results further indicate the effectiveness of radar-optical temporal fusion and sensor-aware modeling. The results demonstrate the potential of ROSA-TFormer for point-level P. sylvestris plantation classification, while broader wall-to-wall validation remains necessary.
Read the full abstract ▾
Computer Vision
Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance
Kangsheng Duan, Ziyang Xu, Wenyu Liu et al. (+3 more)
While 10B-level industrial foundation models have pushed the boundaries of image inpainting, their prohibitive computational costs severely hinder practical deployment. Constructing a highly optimized task-specific specialist offers a promising solution; however, extreme structural compression inevitably triggers a severe representation bottleneck. To conquer this, we propose Moebius, a highly efficient lightweight inpainting framework. We systematically reconstruct the
diffusion backbone by introducing the Local-$\lambda$ Mix Interaction ($L\lambda MI$) block. Comprising Local-$\lambda$ and Interactive-$\lambda$ modules, it elegantly summarizes spatial contexts and global semantic priors into fixed-size linear
matrices, preserving complex latent interactions while drastically shedding parameters. Furthermore, to unlock the full representational capacity of this highly compact architecture, we synergistically pair it with an adaptive multi-granularity
distillation strategy. Operating strictly within the latent space to avoid expensive pixel-space decoding, this strategy dynamically balances multiple gradient-based losses to achieve high-fidelity
alignment. Extensive experiments across natural and portrait
benchmarks demonstrate that this optimal synergy enables Moebius to rival or even surpass the generation quality of the 10B-level industrial generalist FLUX.1-Fill-Dev. Remarkably, Moebius achieves this using less than 2% of the parameters (0.22B vs. 11.9B) while delivering a $>15\times$ acceleration in total inference time, setting a new efficiency standard for high-fidelity inpainting. Project page at https://hustvl.github.io/Moebius.
Read the full abstract ▾
All authors Kangsheng Duan, Ziyang Xu, Wenyu Liu, Xiaohu Ruan, Xiaoxin Chen, Xinggang Wang
Computer Vision
When AUC Misleads: Polarization-Aware Evaluation of Deepfake Detectors under Domain Shift
Dat Nguyen, Cosmin Radoi, Romain Hermary et al. (+4 more)
Recent advances in generative AI, such as
diffusion models and face-swapping tools, have enabled the creation of highly realistic deepfakes, leading to real-world harms including financial fraud and non-consensual explicit content. In response, deepfake detection has become an active research area, with recent methods increasingly focusing on improving generalization to unseen manipulations. This is typically evaluated using the Area Under the ROC Curve (AUC) measured separately across multiple
datasets. However, such an evaluation fails to reflect real-world scenarios where detectors face a mixture of data sources and varying artifact types. To address this limitation, we introduce a novel metric, Cross-
dataset AUC (Cross-AUC) that averages per-domain AUCs with a measure of prediction polarization for taking into account the robustness to domain shift. The polarization extent is quantified by the Wasserstein Distance between class score distributions. Cross-AUC not only assesses the generalization capabilities of deepfake detectors under domain shifts more realistically, but it is also interpretable as it better explains the reason behind a drop in performance. Experiments performed on seven
benchmark datasets demonstrate its practical relevance.
Read the full abstract ▾
All authors Dat Nguyen, Cosmin Radoi, Romain Hermary, Marcella Astrid, Nesryne Mejri, Enjie Ghorbel, Djamila Aouada
Computer Vision · Cross-listed from cs.LG
The Reward Was in Your Data All Along: Correcting Flow Matching with Discriminator-Guided RL
Nicolas Beltran-Velez, Felix Friedrich, Zhang Xiaofeng et al. (+4 more)
Score- and flow-matching models often rely on preference-based reinforcement learning for two purposes: aligning with subjective preferences and, surprisingly, recovering properties such as visual realism and coherent object structure that matching-based training is intended to learn from the data itself. We argue that this reflects a structural mismatch. Matching losses measure $\ell_2$ regression error on the velocity or score field under training-time marginals, a proxy poorly aligned with the visual and semantic properties that determine sample quality at inference. Given a reward aligned with these properties, RL sidesteps the mismatch by evaluating the model on its own samples and following the reward landscape directly. The challenge is to obtain such a reward without relying on human preferences, which are expensive and conflate data realism with annotator inclinations. We propose Discriminator-Guided RL (DRL). DRL trains a discriminator to separate data from base-model samples in a pretrained representation space and uses its logit as the reward in KL-regularized RL. The pretrained space restricts the discriminator to perceptually meaningful directions, and the logit estimates the log-likelihood ratio between data and model, which is the optimal reward for targeting the data distribution. Across SiT, JiT, REPA, and RAE, DRL reduces guidance-free FID (e.g., $9.38 \to 2.62$ on SiT) and semantic-space FD (e.g., $88.2 \to 19.3$ on DINOv3 for SiT), with consistent gains across all backbones, and improves human-preference rewards without training on them. It also yields a better Pareto frontier between preference reward and image fidelity under subsequent preference-based post-training, increasing
alignment while reducing low-level artifacts such as oversaturation and excessive brightness.
Read the full abstract ▾
All authors Nicolas Beltran-Velez, Felix Friedrich, Zhang Xiaofeng, Reyhane Askari-Hemmat, Xiaochuang Han, Adriana Romero-Soriano, Michal Drozdzal
Computer Vision
Hand-4DGS: Feed-Forward 3D Gaussian Splatting for 4D Hand Reconstruction from Egocentric Videos
Jeongmin Bae, Seoha Kim, Marc Pollefeys et al. (+3 more)
Dynamic 3D hand reconstruction from
egocentric videos is essential for next-generation computing platforms such as AR/VR and AI glasses. Despite its importance, most prior works focus either on multi-view 3D hand reconstruction or on 4D human body reconstruction.
Egocentric 4D hand reconstruction remains difficult due to rapid hand and camera motion, hand-object and inter-hand interactions, and inherent ambiguity from single-view observations. To address these challenges, we introduce Hand-4DGS, a feed-forward framework for dynamic 4D hand reconstruction from
egocentric videos. Our approach incorporates a mesh-guided representation for structural priors and temporal convolutions to model dynamic motion. We evaluate our framework on H2O and ARCTIC, two
egocentric hand-object interaction
datasets, and show improvements over baselines. Our model can efficiently adapt to unseen
videos from
datasets that are not included in training. In addition, image supervision through differentiable Gaussian rasterization provides an additional signal for appearance optimization and pose adjustment during training and test-time optimization, without ground-truth 3D hand pose annotations.
Read the full abstract ▾
All authors Jeongmin Bae, Seoha Kim, Marc Pollefeys, Mahdi Rad, Youngjung Uh, Taein Kwon
Computer Vision · Cross-listed from cs.CY
The Market in the Model: Latent Diffusion as Neural Economy
Eryk Salvaggio
Valuable critique of generative image models within visual culture and the humanities has emphasized the role of
datasets in shaping the images they produce. Yet, close studies of the ideological positions embedded into the mechanism of the models have been neglected, leaving them imagined as "black boxes." In a bid to expand, rather than replace,
dataset critique, this paper examines the mechanisms of the latent
diffusion model in terms of the problems they were brought in to solve on behalf of computer
vision engineers, and the decisions each component was tasked with automating. I interpret that ensemble through the histories of its parts and the theory of
vision the system inscribes into every generated image. Drawing on Impett and Offert's notion of neural exchange value, I offer this analysis to argue that the model operates as a neural economy: a contained symbolic system that abstracts social communication into commensurable vectors as it transfers the social sphere into parcels for sale. Tracing the training and generation pipelines component by component reveals what each operation displaces, and how it further entrenches the logics of platform and
attention economies over social communication. The paper warns that any critique fixated exclusively on copyright and commodity defenses risks reaffirming the very fetishism the model produces, and argues instead for centering social exchange.
Read the full abstract ▾
Computer Vision
Urdu Katib Handwritten Dataset: A Historical Document Dataset for Offline Urdu Handwritten Text Recognition with CRNN-Based Baseline Evaluation
Ramza Basharat and Muhammad Usman Ali
Automatic Handwritten Text Recognition (HTR) is inherently a challenging task, and its complexity is further increased when dealing with cursive scripts. Although significant efforts have been made on various cursive scripts, research regarding Urdu Handwritten Text Recognition (UHTR) has been relatively limited. This lag of research is primarily due to the unique challenges posed by its script, and the scarcity and unavailability of
benchmark datasets. Therefore, to advance research in UHTR, this study presents a specialized real
dataset called the Urdu Katib Handwritten
Dataset (UKHD). To the best of our knowledge, this is the first offline Urdu handwritten text lines
dataset specifically curated from the materials written by Katibs in historical times. It encompasses a diverse range of flat nib writing variations in the Nastalique calligraphic style. Additionally, the effectiveness of different CRNN-based hybrid models has been evaluated to identify the optimal architecture for Urdu Katib Handwriting Recognition (UKHR). Among the analyzed models, the CNN-BGRU-CTC model showed more robust performance, with low Character Error Rate (CER) and Word Error Rate (WER). This research work aims to support and encourage the research community in developing a robust recognition system for preserving Urdu handwritten literature.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation
Sihan Wang, Xiyao Liu, Lianqing Liu et al. (+1 more)
On-policy self-
distillation (OPSD) trains a model on its own rollouts and uses a frozen copy to provide dense
token-level targets conditioned on a reference target. This works well for LLM
reasoning, but a direct extension to
multimodal large
language models (MLLMs) can create a shortcut: the privileged target may guide
tokens mainly based on the text reference target rather than the image. We propose ViGOS, a visually grounded OPSD framework for MLLM post-training. The student first writes a visual description and then reasons toward the final answer. For valid rollouts, an image-only perception teacher supervises the description, while a privileged
reasoning teacher supervises the
reasoning and final answer on the same student prefix. A reference teacher is used only for invalid rollouts to recover the output format. Across general
vision-
language, expert
reasoning, visual math, spatial grounding, and visual-
language-prior
benchmarks, ViGOS keeps the main benefits of OPSD and improves image-grounded behavior in shortcut-prone settings.
Read the full abstract ▾
All authors Sihan Wang, Xiyao Liu, Lianqing Liu, Zhi Han
Computer Vision
ProductConsistency: Improving Product Identity Preservation in Instruction-Based Image Editing via SFT and RL
Mukund Khanna, Raj Singh Yadav and Kunal Singh
Recent advances in instruction-based image editing have enabled models to perform complex visual edits from natural
language instructions. However, in product-centric scenarios where preserving product features, branding, and textual elements are critical, current open and closed source models often struggle to maintain this fine-grained object identity. This issue is further compounded by the lack of
datasets for instruction-based product image editing with text fidelity constraints, leaving it largely treated as an implicit capability of instruction-based image editing models. In this work, we introduce the ProductConsistency
dataset which is designed to improve product-centric image editing. Our approach includes a supervised fine-tuning (SFT)
dataset of 87k samples for product editing, a reinforcement learning (RL)
dataset with 869 unique product images, and a new
benchmark dataset, the ProductConsistency
Benchmark, to allow rigorous and standardized evaluation of editing models. To guide RL training, we propose a Cyclic Consistency reward that enforces semantic preservation of product identity by using caption similarity between the original product description and captions generated from the edited image. We fine-tune both Qwen-Image-Edit-2511 and Flux.1-Kontext-dev using our
dataset and demonstrate consistent improvements over baseline models in OCR and Perceptual metrics, and MLLM-based evaluations as well, indicating stronger product consistency, text rendering, and overall visual quality; with the Qwen-Image-Edit-2511 model achieving a 5x reduction in the character error rate. The code and pipeline is available at https://anonymous.4open.science/r/ProductConsistency-6FCC/README.md
Read the full abstract ▾
Computer Vision
AMALIA-VL: A Native European Portuguese Open-Source Vision and Language Model
Diogo Glória-Silva, João Cardeira, Manuel Letras da Luz et al. (+8 more)
Large
Vision and
Language Models (LVLMs) have advanced rapidly, yet European Portuguese (pt-PT) remains systematically underserved by existing open-source
multimodal models, which either conflate it with Brazilian Portuguese or severely under-represent it in their training data mixes. We introduce AMALIA-VL, the first open-source instruction-tuned LVLM built natively for pt-PT, pairing a high-resolution
vision encoder with dynamic image tiling and a fully open pt-PT-optimized
language model via a learned connector. We contribute with a purposefully designed three-stage training process -
vision-
language alignment, general visual instruction tuning, and preference optimization - together with a pt-PT-centric
multimodal data mix combining curated and translated public
datasets with novel
datasets that address the near-total absence of European Portuguese
multimodal resources. Our evaluation shows that AMALIA-VL establishes a strong baseline for open-source pt-PT LVLMs. We will release model weights, training data, and construction pipelines along with machine-translated pt-PT evaluation
benchmarks to help democratize pt-PT LVLM development.
Read the full abstract ▾
All authors Diogo Glória-Silva, João Cardeira, Manuel Letras da Luz, Afonso Simplício, Gonçalo Vinagre, Diogo Tavares, Rafael Ferreira, Inês Calvo, Inês Vieira, David Semedo, João Magalhães
Computer Vision
DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration
Yanjie Tu, Qingsen Yan, Axi Niu et al. (+3 more)
All-in-One image restoration aims to develop a unified restoration framework for handling diverse degradation types. Existing end-to-end methods usually regard the restoration process as a black-box mapping, lacking an explicit optimization interpretation. Although deep unfolding provides an interpretable iterative modeling paradigm for image restoration, existing methods mostly rely on fixed degradation assumptions or predefined degradation information, making them difficult to adapt to unified restoration requirements under complex degradations and locally damaged content. This limitation restricts their performance in degradation suppression and structural detail recovery. To address these issues, this paper proposes DVANet, a deep unfolding network inspired by the half-quadratic splitting optimization
algorithm, which formulates unified image restoration under complex degradations as a collaborative unfolding process between degradation-aware observation consistency and visual-prior-guided reconstruction. Specifically, in the degradation-aware observation consistency branch, a degradation representation module is employed to extract global degradation attributes and local degradation cues, and degradation-conditioned mapping is used to enhance the model's adaptability to different degradation types. In the visual-prior-guided reconstruction branch, DINOv3 is introduced to provide structural and semantic information as hierarchical visual priors, thereby complementing the missing structural information in damaged regions and improving detail recovery. Extensive experiments demonstrate that DVANet achieves superior or competitive performance on multi-scenario degradation and cross-domain image restoration tasks, showing favorable degradation adaptability and generalization ability.
Read the full abstract ▾
All authors Yanjie Tu, Qingsen Yan, Axi Niu, Tao Hu, Haokui Zhang, Jiantao Zhou
Computer Vision
PorTEXTO: A European Portuguese Benchmark for Visual Text Extraction
João Cardeira, Diogo Glória-Silva, Manuel Letras da Luz et al. (+4 more)
European Portuguese (pt-PT) is largely absent from Optical Character Recognition (OCR)
benchmarks, which skew toward high-resource
languages. The few
benchmarks that cover pt-PT focus on historical artifacts and literature. This work addresses modern OCR applications, introducing PorTEXTO, the first
benchmark for contemporary and culturally relevant pt-PT visual text extraction. To ascertain quality, we employ an annotation pipeline combining transcriptions from a frontier LVLM with exhaustive review by native speakers. We observe a sharp performance drop from synthetic to real world samples in most models, and find that, currently, specialized
multilingual data is a better driver for pt-PT performance than model size or resolution budget, motivating the release of open pt-PT OCR resources.
Read the full abstract ▾
All authors João Cardeira, Diogo Glória-Silva, Manuel Letras da Luz, Rafael Ferreira, Diogo Tavares, David Semedo, João Magalhães
Computer Vision
Taming I2V models for Image HOI Editing: A Cognitive Benchmark and Agentic Self-Correcting Framework
Jiayi Gao, Qingchao Chen, Yuxin Peng et al. (+1 more)
Current image editing methods excel at static attributes but fail at complex Human-Object Interactions (HOI), a critical challenge unaddressed by existing
benchmarks that conflate HOI with static attributes, relying on global metrics incapable of simultaneously assessing dynamic interaction validity and entangled human-object pair preservation. Thus, we first introduce HOI-Edit, a comprehensive
benchmark with three progressive cognitive levels, which features an automated metric HOI-Eval that reliably evaluates instance-level interaction by letting VLM Q&A after thinking with images containing grounded Human-Object pairs. Considering the task's essence of remodeling dynamic relationships, we
benchmark Image-to-
Video (I2V) models, finding them inherently suited for dynamic editing due to their temporal generation capabilities. Crucially, beyond superior performance, this capability provides a "replay of the failure process," offering unique diagnosability into why errors occur. We thus propose SCPE (Self-Correcting Process Editing), a novel, agentic self-correcting framework that constrains the generation of I2V models through iteratively refined
prompts, enabling the generated
videos to more accurately present the target HOI. Extracted frames from these
videos are the final editing results. On HOI-Edit, SCPE achieves performance competitive with state-of-the-art (SOTA) editing models like Nano Banana on interaction. Code is available at https://github.com/oceanflowlab/HOI-Edit.
Read the full abstract ▾
All authors Jiayi Gao, Qingchao Chen, Yuxin Peng, Yang Liu
Computer Vision · Cross-listed from cs.RO
Sensor Configuration Matters: A Systematic Evaluation of Multimodal SLAM on Quadruped Robots
Roberto Corlito, Fabian Schmidt, Nils Seibert et al. (+3 more)
Autonomous
navigation of quadrupedal
robots in diverse environments fundamentally relies on resilient Simultaneous Localization and Mapping (SLAM). While visual-inertial SLAM has matured across wheeled, handheld, and aerial platforms, a critical evaluation gap remains regarding how hardware-level sensor configurations affect performance under the aggressive dynamics of legged locomotion. Quadrupeds introduce distinct embodiment-induced sensory challenges, including foot-impact shocks, high-frequency mechanical vibrations, and rapid angular rotations, which degrade standard perception pipelines. To address this gap, we present a systematic evaluation of state-of-the-art visual, visual-inertial, and
LiDAR-visual-inertial SLAM methods using the GrandTour
dataset recorded on an ANYmal D quadruped. We isolate and quantify the impacts of camera modalities, shutter techniques, and inertial sensor tiers, analyzing their trade-offs across localization accuracy, algorithmic robustness, and computational resource utilization. Our empirical findings demonstrate that hardware selection has substantial influence on system resilience: stereo configurations consistently outperform
monocular and RGB-D modalities, global shutter cameras significantly mitigate motion-induced tracking failures compared to rolling shutter cameras, and, crucially, standard inertial integration can degrade the performance of primarily
vision-based frameworks under harsh legged locomotion. These insights additionally offer concrete design guidelines for tailoring custom sensor payloads to achieve dependable perception on agile legged systems.
Read the full abstract ▾
All authors Roberto Corlito, Fabian Schmidt, Nils Seibert, Markus Enzweiler, Abhinav Valada, Arne Roennau