Computer Vision · Cross-listed from cs.AI
Privacy-Preserving Full-Body Meshing from mmWave Radar via Mesh Foundation Model Supervision
Shuxing Zhang, Yongquan Ni, Zhenyu Ding et al. (+1 more)
Millimeter-wave (mmWave) radar enables
privacy-preserving human perception, but the extreme sparsity of point clouds from commercial single-chip sensors (mean ~6.5 points/frame; ~28% empty frames) has confined prior art to body-part keypoints or discrete action classification. We present a cross-modal teacher-student framework that lifts commercial radar to full-body, per-frame, metric 3D mesh reconstruction with per-joint uncertainty. Three innovations: (1) a mesh-foundation-model teacher - SAM 3D Body produces whole-body MHR ground truth (70 joints, 18,439 mesh vertices) from a single RGB frame with zero training, slashing annotation cost by orders of magnitude; (2) StudentPoseFormer - set encoding with masked
attention pooling, a temporal
Transformer, and a CVAE multi-hypothesis head that outputs both the pose mean and per-joint variance, honestly reporting where the radar cannot see; and (3) a multi-stage ground-truth quality pipeline (confidence gating, depth validation, temporal smoothing, bone-length consistency, bad-frame rejection) plus systematic information-lever ablations. On the public MM-Fi
benchmark (same TI IWR6843 sensor, cross-subject), our full configuration reaches 7.45 cm 12-joint MPJPE, with ablations proving the causal value of point accumulation (k = 3, -0.34 cm), Doppler (-0.85 cm; -2 cm at the wrist on fast actions), and velocity loss (-0.27 cm). On our own synchronized radar + RGB-D corpus with block-level held-out splits, the pipeline achieves 21.47 cm end-to-end (per-joint hierarchy from 4.8 cm at the hip to 34.7 cm at the wrist - matching physical information limits), could be improved to 15 cm with ~30k diverse samples, and a scaling law shows sample diversity, not volume, is the binding constraint. Deployment inference is radar-only - no camera, no image.
Read the full abstract ▾
All authors Shuxing Zhang, Yongquan Ni, Zhenyu Ding, Yawen Lin
Computer Vision
Beyond Reconstruction Loss in Post-Training Quantization: Balanced Fitting for Large Vision-Language Models
Minchan Kang, Kyeonghye Park, Seungyeon Sa et al. (+3 more)
Post-training quantization (PTQ) enables efficient deployment of large
vision-
language models (LVLMs), but is typically calibrated on a small set while expected to generalize across diverse downstream tasks. Although recent PTQ methods for LVLMs incorporate sensitivity signals, they still minimize reconstruction loss with respect to the full-precision model, potentially over-preserving FP behavior and calibration-specific bias. Rather than treating quantization solely as an error to be minimized, we observe that it can also provide beneficial regularization for certain layers and modalities. Motivated by this observation, we propose Balanced Fitting, a quantization effect-based framework that balances precision and regularization beyond reconstruction-based optimization. By measuring layer- and component-wise quantization effects for weights,
vision activations, and text activations, Balanced Fitting combines fine-grained fitting for sensitive components with coarser fitting to exploit potential regularization benefits. Experiments on multiple LVLMs show that our method consistently outperforms prior PTQ approaches under both weight-only and weight-activation quantization, while lower reconstruction loss does not reliably translate into better downstream performance. The source code is publicly available at https://github.com/kmc3661/BFQ
Read the full abstract ▾
All authors Minchan Kang, Kyeonghye Park, Seungyeon Sa, Seoyoung Cho, Daeshik Kim, Yucheol Cho
Computer Vision
PanoVLN: Towards Effective Panoramic Vision-and-Language Navigation
Zhen Wang, Changpeng Wang, Zhe Liu et al. (+5 more)
Recent
vision-
language models (VLMs) have advanced
vision-and-
language navigation (VLN), enabling models to predict navigation actions from visual observations and
language instructions. In this work, we explore VLN with panoramic observations and introduce PanoVLN. The motivation is straightforward: more complete visual context should enable better-informed navigation decisions. For example, a panorama can reveal a passage outside a perspective camera's field of view, allowing the model to identify the intended route without additional exploration. However, we find that simply replacing perspective images with panoramas yields only limited gains. Our diagnosis suggests that fully exploiting wider visibility requires modifications to action prediction, training supervision, and visual representation. First, wider visibility supports longer-horizon action planning. We make the model predict longer action sequences, enabling larger turns and subsequent movement from a single panorama. Specifically, we introduce a confidence-guided execution (CGE) strategy that dynamically determines how many predicted actions to execute before replanning. Second, wider visibility also brings more complex route choices. We therefore construct training routes with frequent branching points and clear instructions to provide targeted supervision for route selection. Third, panoramic navigation requires understanding spatial relationships across viewing directions, beyond recognizing individual landmarks. We combine semantic and
geometric features from RGB panoramas to capture both scene content and spatial layout without adding visual
tokens. With a 4B backbone and RGB-only input, PanoVLN surpasses the previous SOTA by 11.9% and 8.7% in success rate on R2R-CE and RxR-CE Val-Unseen. Real-world experiments on a quadruped further demonstrate faster navigation with fewer pauses than prior VLN methods.
Read the full abstract ▾
All authors Zhen Wang, Changpeng Wang, Zhe Liu, Zhangyang Qi, Yuxiang Lu, Zimo Zeng, Donglian Qi, Xi Chen
Computer Vision · Cross-listed from cs.LG
A Unifying Framework of Concept-based Explainable AI with Completeness Guarantees
Vojtěch Kůr, Adam Kukučka, Tomáš Brázdil et al. (+1 more)
Concept-based explanations describe neural network predictions through human-understandable properties of inputs called concepts. The field encompasses approaches that differ in how they define and represent concepts and connect them to model predictions. We introduce a theoretical framework that describes these approaches in a common mathematical
language and supports a shared analysis of their properties. For concept discovery, which identifies concepts automatically within a latent space of a trained model, we employ a concept autoencoder view. An encoder extracts concept representations from the model's latent space, and a decoder uses them to reconstruct the original latent representation. The autoencoder's reconstruction error measures how accurately its decoder recovers the original latent representation. We revisit model completeness: how well the concepts can reproduce the model's outputs. We show that model incompleteness of the concepts can be bounded by the autoencoder's reconstruction error. The autoencoder view also provides a common way to define individual concept attributions, which measure each concept's contribution to a prediction. We establish when these attributions sum to the model's prediction, and bound the discrepancy otherwise, thus providing attribution completeness guarantees.
Read the full abstract ▾
All authors Vojtěch Kůr, Adam Kukučka, Tomáš Brázdil, Vít Musil
Computer Vision
CoDrive: Cross-Vehicle World-Consistent Video Generation with Precise Trajectory Control for Cooperative Driving
Yu Meng, Baining Zhao, Junta Wu et al. (+10 more)
Real-world driving is inherently multi-
agent, yet most existing driving world models generate observations from a single ego vehicle. Independently extending them to multiple vehicles does not ensure that different
agents observe a consistent shared world. We present CoDrive, a cross-vehicle, multi-view driving
video generation framework that jointly generates observations of vehicles sharing the same dynamic scene with precise camera-trajectory control. CoDrive interleaves local self-
attention, which models spatiotemporal dependencies among the views of each vehicle, with global self-
attention, which enables information exchange and consistency modeling across vehicles. To explicitly encode their spatial relationships, all camera trajectories are represented in a shared world coordinate system and injected into the
attention layers through projective relative positional encoding. We further adopt a progressive mixed-task training strategy that combines large-scale real-world single-
agent data with synthetic cross-
agent interaction data, allowing the model to benefit from real-world appearance distributions while learning cross-
agent consistency from simulation. For systematic evaluation, we introduce CoDrive-Bench, a
benchmark covering real and synthetic multi-vehicle scenarios and evaluating trajectory controllability, scene geometry consistency, and instance-level consistency. Experiments show that CoDrive improves trajectory controllability and cross-
agent geometric and instance consistency while maintaining competitive visual quality.
Read the full abstract ▾
All authors Yu Meng, Baining Zhao, Junta Wu, Tengfei Wang, Rongze Tang, Haiyu Zhang, Wenqiang Sun, Chen Gao, Zhibo Chen, Xinlei Chen, Yong Li, Xiao-Ping Zhang, Chunchao Guo
Computer Vision · Cross-listed from cs.CR
Optimizing and Securing the Modern Watermarking Channel for Images
Enoal Gesny and Eva Giboulot
To comply with recent regulations requiring traceable generated content, modern watermarking has adopted multi-bit post-hoc watermarking schemes. These modern designs rest on an encoder-decoder pair implemented as deep neural networks. These models are usually treated as pure black-boxes trained end-to-end, with the noise of the watermarking channel modeled through a fixed set of
geometric and valuemetric transforms applied to watermarked images. We argue that this purely empirical approach leads to unquestioned design flaws and a lack of theoretical performance guarantees. This work proposes a general theoretical model of modern post-hoc watermarking schemes grounded in a statistical analysis of the outputs of the encoder/decoder pair. We show that these deep neural networks implicitly define a watermarking channel modeled as parallel AWGN channels, with messages transmitted using BPSK modulation. This imposes a binary alphabet, greatly limiting the capacity of these watermarking systems. Another fatal flaw is their lack of a secret key, making them intrinsically insecure. We make this notion of watermarking security precise for post-hoc schemes by linking it to the possibility of estimating the secret key under a given statistical model of the decoder's output. By putting together the results from this theoretical analysis, we introduce SNW: a novel post-hoc watermarking system that significantly outperforms existing state-of-the-art baselines in terms of capacity while also providing strong security guarantees. Notably, it does not depend on a fixed codebook or binary alphabet, allowing it to reach a rate close to Shannon capacity through the use of capacity-achieving error-correcting codes.
Read the full abstract ▾
Computer Vision
Do Emotion Concepts Generalize Across Sources, Modalities, and Architectures in Vision-Language Models?
Bohao Xing, Xin Liu, Kaishen Yuan et al. (+5 more)
Recent studies suggest that large
language models encode emotion concepts as structured internal representations, but most existing work focuses on text and a single architecture. Therefore, we ask, do emotion concepts generalize across sources, modalities, and architectures in
vision–
language models (VLMs)? To address this, we construct CMES (Cross-Modal Emotion Stimuli), a multi-source collection of emotion-conditioned stories, real facial expressions, synthetic portraits, and synthetic emotion-evoking scenes. For each stimulus source, we extract a separate set of six Ekman emotion vectors from each of three VLMs. We report four main findings as follows: 1) Image-derived emotion vectors form a low-dimensional geometry similar to that of text-derived vectors. Valence is relatively stable across sources, while arousal varies more. 2) Text- and image-derived emotion vectors have modest cosine similarity but still show held-out cross-modal correspondence. Text-derived vectors can also steer image interpretation. 3) Cross-architecture correspondence remains even when native cosine is near zero. Transformations estimated from generic ImageNet activations recover both correspondence and causal transfer without using the six emotion vectors or their labels. 4) After aligning representations across architectures, we construct a shared emotion subspace that preserves affective geometry and selective steering effects. The corresponding consensus emotion vectors also generalize to a held-out fourth architecture at two model sizes. These results suggest that emotion representations can share relational structure and causal effects across sources, modalities, and architectures, even when individual vector directions differ.
Read the full abstract ▾
All authors Bohao Xing, Xin Liu, Kaishen Yuan, Deng Li, Rong Gao, Guoying Zhao, Xiaolan Fu, Heikki Kälviäinen
Computer Vision
SurgGMF: Fully Causal Gaussian Motion Forecasting for Anticipatory Surgical Scene Rendering
Jingqian Sun and Yichao Tang
Dynamic surgical scene modeling is essential for robotic perception, simulation, and decision support. Although existing neural rendering methods enable efficient reconstruction and rendering of deformable surgical scenes, they remain primarily focused on observed-frame reconstruction rather than forecasting future scene states. To this end, we present SurgGMF, a fully causal Gaussian motion forecasting framework for anticipatory surgical scene rendering. Rather than predicting future RGB images directly, SurgGMF forecasts future Gaussian motion states represented by position, scale, and rotation residuals (X/S/R) from historical Gaussian motion fields. To prevent target leakage, we introduce a full-causal-last rendering protocol, where future Gaussian states are rendered without accessing target-frame Gaussian attributes while preserving causal appearance propagation. We evaluate SurgGMF on 12 EndoNeRF and StereoMIS
video slices using neural temporal learners and classical dynamics baselines under a unified forecasting protocol. Learned Gaussian motion forecasting consistently outperforms classical dynamics baselines in render space, demonstrating gains beyond hand-crafted state extrapolation. Latency analysis further reveals an accuracy–efficiency trade-off: under the current implementations, TKAN achieves the highest accuracy, whereas GRU and LSTM provide more favorable module-level latency profiles. These results establish SurgGMF as a reproducible framework for causal Gaussian motion forecasting and advance surgical Gaussian representations from retrospective reconstruction toward predictive scene modeling.
Read the full abstract ▾
Computer Vision
What Visual Generators Need from Teachers: Rethinking Representation Alignment
Yongcong Wang, Hingchin Chen, Mingyu Fan et al. (+6 more)
Representation
alignment speeds up
diffusion transformer training by pulling an intermediate block of the model (student) toward features of a frozen pretrained encoder (teacher). Which teacher layer to align, and for how long, is still set by convention, and each alternative costs a training run. We find that
alignment helps where the student cannot linearly recover the teacher's features, not where it already resembles them. Since a deep teacher layer is largely predictable from the one below, we isolate what each layer adds, its increment, and measure how much of it an unaligned student recovers. The student fills the teacher's hierarchy from the bottom up and stalls near the top, which we call hierarchy filling: even after 400K steps it recovers almost none of the deepest. The recoverability gap is the unrecovered share of an increment, read from one unaligned checkpoint. In short runs that each align one teacher layer at one block, the gap nearly reproduces their ranking by FID improvement, and CKA, a measure of feature similarity, largely reverses it. Representation
Alignment and Recoverability Estimation (RARE) picks the teacher layer with the largest gap before training. During training, it tracks each
token's remaining distance to that layer, the online counterpart of the gap, weights
tokens by it, and phases out the loss once the average distance stops falling. With SiT-B/2 on ImageNet $256\times256$, RARE reaches an FID of 18.02 without guidance and 4.46 with it, ahead of seven
alignment baselines including REPA, iREPA and HASTE. It also trains in 14% fewer GPU-hours than iREPA. Its FID stays below iREPA's across model scales, teachers,
datasets and backbones.
Read the full abstract ▾
All authors Yongcong Wang, Hingchin Chen, Mingyu Fan, Shuo Jiang, Teer Zhang, Yucong Sun, Zijia Wang, Yiming Lu, Chengchao Shen
Computer Vision
Geometry as Address: Routing Attention to Visual Memory for Long-Horizon Camera-Controlled Video Generation
Zesong Yang, Weikai Chen, Liyuan Cui et al. (+10 more)
Long-horizon camera-controlled
video generation requires recovering previously observed content from an ever-growing visual history. Existing approaches either search historical context implicitly or reconstruct it into persistent 3D
memory, facing inefficient
memory access or accumulated
geometric errors. Our key insight is that geometry need not explain the scene–it only needs to determine where visual
memory should be read from, while
attention decides what should be recovered. Based on this insight, we introduce GEAR, a Geometry-Enabled
Attention Routing framework that uses geometry as an explicit
token-level address for visual
memory. Rather than fusing historical observations into a persistent global 3D representation, GEAR retains them as frame latents and uses per-frame geometry only to establish
token-level correspondences with target views, thereby avoiding persistent error accumulation from global fusion. Guided by these correspondences,
Geometric Correspondence
Attention (GCA) selectively injects geometrically matched historical features into noisy target patches during denoising. We further introduce an Invisible Octree to accumulate visibility evidence and reject geometrically plausible but occluded correspondences. Extensive experiments demonstrate that GEAR achieves state-of-the-art visual quality, precise camera control, and revisit consistency, enabling minute-long
video generation along challenging trajectories.
Read the full abstract ▾
All authors Zesong Yang, Weikai Chen, Liyuan Cui, Lutao Jiang, Runze Zhang, Yingda Yin, Xiaoyang Huang, Kai Yan, Keyang Luo, Wangguandong Zheng, Xin Wang, Hujun Bao, Zhaopeng Cui
Computer Vision
DBCF: Dual-Branch Complementary Fusion of Foundation Models for Generalized Deepfake Detection
Fengming Gu, Mingjie He, Zonghui Guo et al. (+2 more)
As image generation and editing technologies have progressed substantially, facial forgeries pose significant challenges to
privacy and public safety. Due to limited ability to capture forgery cues, existing small-scale forgery detection models often struggle to generalize across various domains and unseen manipulations. To address this limitation, researchers have turned to large-scale foundation models, which can provide richer representations and better generalization. Nevertheless, relying on a single foundation model alone remains insufficient for effective forgery detection. While models like CLIP offer robust global semantic cues, they lack the capacity to capture detailed local facial features. In contrast, DINO excels at capturing local structural features of faces, but provides weaker global semantic context. To fully utilize the synergies among multiple foundation models, we propose a hierarchical multi-granular framework that integrates complementary pretrained representations. Specifically, a Global Context Branch (GCB) based on CLIP captures holistic semantic cues, while a Fine-grained Cue Branch (FCB) built on DINOv3 captures localized structural irregularities. In addition, we design a feature fusion module that enables parameter-efficient adaptation of the frozen foundation backbones by adaptively extracting and integrating complementary features from the two models. By jointly leveraging global context and fine-grained cues, our method learns more comprehensive forgery representations and achieves strong cross-manipulation performance. Extensive experiments on multiple
benchmarks demonstrate the benefit of the proposed design, particularly under cross-
dataset and cross-manipulation settings.
Read the full abstract ▾
All authors Fengming Gu, Mingjie He, Zonghui Guo, Jie Zhangb, Shiguang Shan
Computer Vision
From Pixel Generation to Topological Inference: Structural Dual Super-Resolution for Trustworthy Cross-Physical-Domain Trabecular Morphology Learning
Fan Zhang, Yi Zhang and Ling Wang
Clinical CT and UHRCT cannot resolve individual trabeculae, whereas synchrotron radiation microCT (SRuCT) provides high-resolution references but is not applicable for in vivo imaging. The two domains differ by a 32x resolution gap, are only coarsely paired, and exhibit severe physical differences including partial volume effects, noise, and artifacts. Existing super-resolution networks and pretrained-prior methods (GLEAN/StyleGAN2, Stable SR/LDM) underperform because they target pixel generation—diverse details and SSIM/PSNR—and do not model these physical differences. Pixel generation for a 32x resolution gap is intrinsically ill-posed. We propose a paradigm shift from pixel generation to topological inference: deterministically predicting invariant microstructures from macro-scale low-resolution inputs, evaluated by morphological parameters. The core of our 2D morphology learning lies in training on 2D slices while evaluating on 3D morphological parameters, ensuring that the learned representations capture true three-dimensional trabecular
topology rather than 2D pixel statistics. We realize this paradigm via structural dual super-resolution, coupling forward physical degradation (micro-to-macro) with inverse structural inference (macro-to-micro) through structural duality constraints. The method is an end-to-end, few-shot, compact structural dual network (SDN), comprising a bidirectional modeling network, a multi-scale structural consistency discriminator, and four structural duality constraints. On the test set, SDN achieves morphological parameters largely consistent with SRuCT across six metrics, with SSIM reaching 0.8. Trained on 3.2 um SSRF data, the model generalizes well to 3.25 um isotropic BSRF data from an independent source, validating cross-source generalization and confirming trustworthy structural inference rather than pixel generation.
Read the full abstract ▾
Computer Vision
Triangular Resampling for Long-Horizon Motion Generation
Kunhang Li, Yiyi Cai, Xiangyue Zhang et al. (+5 more)
We introduce Triangular Resampling (TR), a post-training method for mitigating long-horizon error accumulation in motion
diffusion models. Built on FloodDiffusion's triangular denoising schedule, TR addresses the mismatch between ground-truth-derived training windows and model-generated inference states. Replacing only completed motion history leaves this mismatch unresolved in partially denoised states within the active window. TR therefore extends rollout-based training to these states, using ground-truth clamping to limit excessive drift. For each replayed sample, TR draws one denoising threshold, shared across latent positions and replay updates, and replays multi-step triangular denoising without gradient tracking. After each update, states below the threshold are replaced with noise-matched ground truth, while those at or above it retain model predictions. The resulting latent window enters the standard training update. This rollout construction supports both supervised training (TR) and distribution matching (TR-DMD). On 120-second motion generation from HumanML3D test
prompts, TR and TR-DMD achieve state-of-the-art FID AUC within their respective non-DMD and DMD comparison groups. Supervised TR reduces FID AUC by 40.9% and FID degradation slope by 55.3% relative to matched post-training without replay.
Read the full abstract ▾
All authors Kunhang Li, Yiyi Cai, Xiangyue Zhang, Fangyuan Tu, Yuhan Wu, Zhixiang Wang, Kaipeng Zhang, Haiyang Liu
Computer Vision
Unified Trajectory Matching Policy Optimization: Diverse T2I Generation and VLA Generalization
Zhiyuan Ma, Jiaming Li, Lingzhen Li et al. (+7 more)
Reward-maximizing reinforcement learning (RL) is widely used to post-train
stochastic diffusion and flow policies for text-to-image (T2I) generation. However, reward-maximizing RL causes policy mode collapse even under reference KL or
entropy regularization, reducing the policy to a single high-reward mode. In T2I, this produces similar images and reward hacking. When extended to
vision-
language-action (VLA) models, the same collapse removes alternative successful strategies and weakens task and scene generalization. To address this limitation, we introduce Unified Trajectory Matching Policy Optimization (Uni-TMPO), a unified RL post-training framework for
diffusion and flow policies. First, Uni-TMPO converts standardized rewards into a target distribution within each trajectory group and derives the policy distribution from trajectory log probabilities. Then, forward Kullback-Leibler optimization matches the two distributions instead of maximizing expected reward. A progress-conditioned coarse-to-fine scheduler efficiently constructs T2I trajectories. Within the unified framework, feedback-conditioned sampling uses updated observations to construct VLA trajectories. Extensive experiments show that Uni-TMPO achieves higher T2I rewards and VLA ID success rates than the strongest baselines. More importantly, it achieves the best T2I reward-diversity-efficiency trade-off and VLA generalization to held-out tasks and scenes, while real-
robot evaluation demonstrates the value of multiple action strategies when the higher-reward target is blocked.
Read the full abstract ▾
All authors Zhiyuan Ma, Jiaming Li, Lingzhen Li, Yu Liu, Xuekai Zhu, Dingkang Liang, Kaiyan Zhang, Jianjun Li, Bowen Zhou, Xiang Bai
Computer Vision · Cross-listed from cs.RO
Natural State-Prediction Accuracy can Hide Weak Controlled Responsiveness in VLA Readouts
Hyungjoon Kim, Wonbin Son, Mi Young Lee et al. (+2 more)
Accurately decoding object states from the internal representations of
vision-
language-action (VLA) models does not establish that the predictions respond faithfully to changes in the target physical state. In natural observations, object state,
robot configuration, occlusion, and task progress vary together, allowing contextual cues to contribute to prediction. In this paper, we introduce an evaluation framework that separates prediction accuracy, target-state responsiveness, and context stability using physically validated observations that cross target coordinates with
robot contexts. We demonstrate that high natural-trajectory accuracy can coexist with weak controlled target-state responsiveness in fixed representation-readout pairs. Comparisons and interventions involving representations, readouts, and training data show that the three properties provide distinct diagnostic information. Furthermore, adding responsiveness and context sensitivity to a failure predictor based on initial state error and physical variables reduces policy-failure prediction error on new initializations relative to the specified baseline while same-observation controlled MAE is also informative. These findings motivate evaluating target-state responsiveness and context stability alongside natural prediction accuracy, and examining their relationship to actual policy behavior and task outcomes.
Read the full abstract ▾
All authors Hyungjoon Kim, Wonbin Son, Mi Young Lee, Jun Young Lee, Seungmin Rho
Computer Vision
V-Gym: Enhancing Agentic Visual Reasoning via Skill-Data Co-Evolution
Bei Yan, Yuecong Min, Jie Zhang et al. (+3 more)
Advances in
multimodal understanding,
reasoning, and tool use enable
agents to tackle increasingly complex visual
reasoning tasks. By distilling past execution experience into reusable skills,
agents can transfer lessons from both successes and failures into future
reasoning, reducing repeated errors and improving capabilities. However, limited experience may produce unreliable, poorly generalizable skills, while static
datasets may lack the targeted and diverse practice needed for refinement. To address this gap, we introduce V-Gym, an autonomous framework that iteratively co-evolves procedural skills and
multimodal practice data from execution trajectories. During skill
evolution, V-Gym analyzes trajectories to distill and refine hierarchical skills, updating procedural guidance and applicability conditions while retaining an update only if it improves validation performance. During data
evolution, V-Gym selects generation seeds by balancing data utility and exploration, then translates trajectory-identified bottlenecks into diverse, targeted practice data that expand the data bank after quality checks. The resulting practice outcomes feed back into subsequent skill updates, closing the loop for continual skill refinement. Experiments across diverse
multimodal reasoning benchmarks show substantial improvements over baselines with multiple backbone models. Its evolved skills generalize across domains and models, while evolved data support more effective skill refinement, enabling autonomous diagnosis, targeted practice, and continual self-improvement.
Read the full abstract ▾
All authors Bei Yan, Yuecong Min, Jie Zhang, Junqi Yang, Shiguang Shan, Xilin Chen
Computer Vision · Cross-listed from cs.LG
Learning What to Recall: Adaptive Multi-Cue Episodic Memory for World Models
Beomsu Kim, Chieh-Hsin Lai, Bac Nguyen et al. (+3 more)
World models predict future observations from current experience and actions, yet prediction can depend on observations seen far in the past. Episodic
memory preserves past observations for later recall; however, as
memory accumulates, it raises a fundamental question: which
memories are useful for the current prediction, and which available
retrieval cues should be trusted to find them? This is challenging because fixed criteria based on recency, pose overlap, or visual similarity can be unreliable across environments and queries. We propose Future-Aware Recall (FAR), a framework that learns episodic recall from future-aware predictive supervision and adaptive multi-cue scoring. During training, FAR measures predictive utility by the conditional log-likelihood of the realized future given recalled context, approximated by negative
diffusion prediction loss, and uses it to train a retriever that remains future-blind at inference. The retriever learns cue-specific relevance and automatically determines which available
retrieval cues, such as time, pose,
vision, and audio, to trust for each query when selecting
memories. Across three complementary settings, FAR outperforms hand-designed recall even with the same
retrieval cues, automatically adapts which available cues to trust, and recalls the right history as the world changes. Together, these results establish FAR as a flexible, principled approach to episodic
memory access in world models.
Read the full abstract ▾
All authors Beomsu Kim, Chieh-Hsin Lai, Bac Nguyen, Amir Bar, Jong Chul Ye, Yuki Mitsufuji
Computer Vision
Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding
Nanxing Hu, Qiwei Yan, Jinchao Zhang et al. (+1 more)
Fine-grained visual understanding requires models to recognize small details within complex images.
Multimodal on-policy self-
distillation (OPSD) addresses this challenge by using a teacher conditioned on evidence-centered crops to supervise a student conditioned on original images along student-generated trajectories. Ideally, teacher corrections, the distributional changes from the student toward the privileged teacher, should be driven by task-relevant visual evidence. However, the designs that make the teacher effective also introduce other interference. Using a lagged or frozen teacher improves training stability but introduces a model-state gap from the evolving student, while cropping enhances task-relevant evidence but also loses the visual context. These two sources of interference make the teacher corrections not purely rely on the visual evidence. We introduce Evidence-Aligned
multimodal on-policy self-
Distillation (EAD), which retains the crop-conditioned teacher as the target but constructs a separate evidence reference for weighting the corrections. To exclude the effect of lagged model-state from this reference, EAD measures prediction changes using the current student. To avoid crop-induced context changes, EAD masks the evidence region in the original image while preserving the other visual context. The change from the student's masked-image prediction to its original-image prediction provides a controlled reference for the direction in which the visual evidence shifts the student's prediction. EAD weights each teacher correction by its cosine
alignment with the reference, i.e., retaining aligned corrections and downweighting the rest. Retaining only 6% of the supervision mass of dense OPSD, EAD consistently outperforms previous state-of-the-art methods.
Read the full abstract ▾
All authors Nanxing Hu, Qiwei Yan, Jinchao Zhang, Guoliang Kang
Computer Vision
CapField-OPD: Learning Continuous Capability Fields via Joint-Anchored Multi-Teacher On-Policy Distillation for Flow Models
Pengyang Ling, Jiazi Bu, Yujie Zhou et al. (+6 more)
Reward-specialized post-training produces strong experts for flow-based generative models, while multi-teacher on-policy
distillation (OPD) consolidates their capabilities into a single student. Existing methods, however, route each
prompt to a single teacher according to its semantic category, implicitly binding the desired capability to
prompt content. This coupling makes capability invocation vulnerable to
prompt perturbations and prevents users from explicitly adjusting the strength of the desired capability at inference time. In this work, we introduce CapField-OPD, an OPD framework that integrates multiple teachers into a continuous capability field through explicit capability coordinates. We use teacher models as anchors to construct this field, with the coordinates determining how their outputs are combined. Each capability configuration thus receives a unique supervision target, and capability control no longer depends on
prompt semantics. Since the training anchors may not be optimal at inference time, we further profile the learned field on a small calibration set. The coordinate with the highest mean reward serves as the recommended default, while coordinates that are frequently optimal offer a promising candidate set for test-time scaling. Extensive experiments on compositional generation, text rendering, and visual aesthetics demonstrate that CapField-OPD consolidates multiple specialized teachers into a single student while preserving or surpassing their performance, reliably invokes the desired capabilities under semantics-preserving
prompt variations, and supports continuous capability control and coordinate-based test-time scaling.
Read the full abstract ▾
All authors Pengyang Ling, Jiazi Bu, Yujie Zhou, Yibin Wang, Zeqiang Lai, Xiaoxiao Ma, Yi Jin, Huaian Chen, Yuhang Zang
Computer Vision
Revitalizing Medical Time Series with Vision-Informed Retrieval: A Vision-Language Perspective
Guoqi Yu, Juncheng Wang and Shujun Wang
Medical time series (MedTS) underpin many clinical classification tasks, yet existing methods usually represent them only as numerical sequences and underuse the morphology that is explicit in waveform inspection. To bridge this gap, we introduce
Vision-Informed
Retrieval (ViRe), which uses a frozen VLM-derived waveform representation as a morphology-aware Query to guide
retrieval from raw numerical MedTS features. Specifically, a
Vision Query is extracted using pre-trained
vision-
language models (VLMs) to obtain morphology-aware priors from waveform plots. A tailored
attention-based cross-modal
retrieval mechanism then uses the
Vision Query to select morphology-relevant temporal and channel evidence from the numerical representation. ViRe demonstrates strong effectiveness against ten established baselines, yielding an overall 6.42% relative improvement over the previous state of the art across six public
benchmarks. Code, training scripts, and reproducibility materials are publicly available in the GitHub Repository: https://github.com/Levi-Ackman/ViRe.
Read the full abstract ▾
Computer Vision
DirectUV: Image-Conditioned UV Texture Generation with Surface-Aware Positional Encoding
Jiantao Lin, Yingjie Xu, Mingzhi Sheng et al. (+3 more)
Generating high-quality UV textures for 3D meshes remains challenging. Multi-view projection pipelines suffer from occlusion and view inconsistency, and recent methods that generate textures directly in UV space still rely on auxiliary modules to supply 3D information, leaving the
attention mechanism tied to UV-grid positions rather than to the underlying surface geometry. This mismatch limits coherence across seams and disconnected UV islands. We propose DirectUV, an image-conditioned UV texture
diffusion framework that operates in the latent UV space of a pretrained image VAE, in which a
Diffusion Transformer denoises the UV latent given a single input image and a coarse UV map. At its core, Surface-Aware Positional Encoding (SAPE) replaces the standard 2D-grid positional encoding with encodings derived from per-
token 3D surface coordinates obtained via UV-to-surface correspondence. As positional encodings define the distance metric used by
attention, SAPE enables
tokens to interact according to 3D positional proximity derived from surface correspondence rather than UV-grid distance, restoring coherence across seams and disconnected islands. A multi-level extension further assigns different
attention heads to progressively finer subdivisions of the same latent UV patch, allowing the model to reason about surface structure at multiple granularities. Experiments show that DirectUV produces sharper and more globally consistent textures than other baselines, with the largest improvements in occluded and view-unseen regions where projection-based methods leave gaps or stretched textures.
Read the full abstract ▾
All authors Jiantao Lin, Yingjie Xu, Mingzhi Sheng, Yangkai Wei, Hao Chen, Ying-Cong Chen
Computer Vision
Backdoor as Probe: Test-Time Adversarial Defense for CLIP
Zhongqi Wang, Jie Zhang, Nie Sen et al. (+3 more)
Test-time
adversarial defense improves the robustness of
vision-
language foundation models such as CLIP without retraining. However,
adversarial activation shifts are typically treated as distortions to suppress, rather than signals to exploit. We turn these shifts into defense signals by repurposing the trigger-to-target mechanism of backdoors. The key is to implant a defender-controlled backdoor as a probe that is weakly activated by clean inputs but strongly activated by
adversarial shifts. Based on this insight, we propose Backdoor as Probe (BaP), a test-time
adversarial defense for CLIP. BaP constructs the probe through a closed-form model edit to a selected MLP layer. It projects the average
adversarial activation shift and a defender-specified semantic direction onto the layer's low-energy input and output activation subspaces to obtain the trigger and target directions, respectively. At inference time,
adversarial inputs produce measurable responses along the target direction for detection. BaP then selectively rectifies detected inputs by optimizing a small perturbation that steers their representations away from
adversarial shifts and toward the clean subspace. Experiments across 16
benchmarks show that BaP improves average robust accuracy from 1.0% to 52.3% while retaining clean accuracy, achieving performance comparable to state-of-the-art methods with up to a \(5.7\times\) inference speedup. BaP further shows the generalization to
adversarial attacks on large
vision-
language models. Project page: https://robin-wzq.github.io/Backdoor-as-Probe/
Read the full abstract ▾
All authors Zhongqi Wang, Jie Zhang, Nie Sen, Zhiyu Chen, Shiguang Shan, Xilin Chen
Computer Vision
Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos
Fangzhou Ma, Ivo Alexander Ban, Eren Homburg et al. (+5 more)
Real-world AI systems must reason about objects that are no longer visible: an AR assistant guiding a user back to an object used earlier, a household
robot retrieving an item someone put away. This requires not just recalling where an object was last seen, but updating its state when it is moved and retaining that update once it leaves view. We refer to this as out-of-sight spatiotemporal
reasoning. We introduce Beyond3D, the first VQA
benchmark to isolate this ability in dynamic egocentric
video: every query targets an object that has been relocated and has since left the field of view. We create our questions from HD-EPIC annotations, building a visibility track for each dynamic object from its 3D position, the camera pose, and the scene geometry to understand at each moment whether it is visible, occluded, or out of view. Beyond3D comprises 9,000 questions in eight types over 135
videos from nine participants, organized as one
reasoning chain: visual grounding (is the target observable now), temporal grounding (when it was last visible and last placed), scene localization (which fixture anchors that location), and 3D spatial perception (where it lies relative to the current viewpoint or another object in the scene). We
benchmark nine general-purpose and spatially specialized VLMs. The best model reaches 42.2% against 29.7% chance and text-only baselines reaching 31.9%, with the largest failures in recovering when an object was last visible, showing that tracking object movement out of sight remains far from solved for current VLMs.
Read the full abstract ▾
All authors Fangzhou Ma, Ivo Alexander Ban, Eren Homburg, Gabriele Goletto, Rémi Pautrat, Mahdi Rad, Chiara Plizzari, Marc Pollefeys
Computer Vision
BMND: Direct Poisson Denoising by N-Dimensional Block Matching and Collaborative Filtering
Christof Duhme, Lars Schiefelbein, Florian Büther et al. (+1 more)
Poisson denoising of scientific data requires methods that account for signal-dependent noise while accommodating different data dimensionalities and preserving quantitative intensity information. We present BMND, a dimension-independent extension of block matching and collaborative filtering for Gaussian and Poisson observations. Building on the two-stage structure of BM3D and BM4D, BMND processes Poisson data directly, without a variance-stabilizing transform, by combining noise-aware patch matching with propagation of signal-dependent noise variances through collaborative filtering and aggregation. A dimension-independent reference-patch traversal scheme supports arrays with an arbitrary number of axes. An optional aggregation-aware mass conservation preserves the observed total intensity after weighted overlap-add. We evaluate the framework on one-dimensional physiological signals, two-dimensional images, and three-dimensional volumes, using controlled noise experiments and measured fluorescence microscopy acquisitions. The experiments demonstrate improved reconstruction quality from noise-aware matching and Wiener filtering, while low-count phantom experiments show reduced denoising-induced intensity loss through mass conservation. The framework provides a unified, non-learning-based approach to denoising across arbitrary data dimensions and is released as an open-source library.
Read the full abstract ▾
All authors Christof Duhme, Lars Schiefelbein, Florian Büther, Xiaoyi Jiang
Computer Vision
Does Native 3D Texture Generation Necessarily Require 3D Assets for Training?
Jiangshan Wang, Zeqiang Lai, Jiayi Guo et al. (+6 more)
Native 3D texture generation synthesizes colors directly in 3D space for a given geometry, conditioned on multi-view reference images. It is generally believed that training such models requires large-scale, high-quality real 3D asset data, whose acquisition remains a long-standing and challenging problem. In this work, we propose Tex-Zero, demonstrating that a high-fidelity native 3D texture generation framework can be trained without 3D assets. Our key observation is that only high-quality and fine-grained color information is essential for 3D texture training, while the required
geometric information is less critical and can be manually constructed rather than obtained from real 3D assets. This finding makes it possible to transform abundant, high-quality 2D images into effective training samples for 3D texture generation. Specifically, we convert high-quality 2D images into 3D training samples by representing each image as a plane in 3D space and applying patch-wise random rotations and aggregation to construct complex
geometric structures. Using these constructed image data, we train the Tex-Zero VAE, which can reconstruct real 3D assets with high quality despite never observing them during training. Building upon the Tex-Zero VAE, we train the Tex-Zero DiT also exclusively on the constructed image data, where the conditioning 2D multi-view images are transformed into planes in 3D space and also encoded by the Tex-Zero VAE, thereby reducing the representation gap and improving generation quality. Extensive experiments show that Tex-Zero generates high-fidelity 3D textures with fine-grained details solely using images as training data, offering a promising perspective on the data paradigm for scaling 3D texture generation.
Read the full abstract ▾
All authors Jiangshan Wang, Zeqiang Lai, Jiayi Guo, Xin Yang, Xin Huang, Jiarui Chen, Ziheng Ouyang, Chunchao Guo, Xiangyu Yue
Computer Vision
WorldAttention: An Efficient Attention Architecture for Interactive Video World Models
Zeyu Zhang, Jinyuan Mao, Dakai An et al. (+6 more)
Leveraging the paradigm of autoregressive
diffusion, text-conditioned interactive
video world models aim to simulate temporally coherent environments guided by textual instructions. While enabling low-latency, long-duration generation is pivotal for embodied AI and simulation-based planning, current frameworks primarily rely on sliding-window mechanisms to bound computational complexity. However, this approach inherently sacrifices historical context, undermining the long-range interactive capabilities. Conversely, maintaining a full-history cache remains computationally prohibitive and
memory-intensive: the quadratic complexity of
attention leads to excessive computational overhead, while the linear growth of the KV cache inevitably leads to GPU
memory saturation. To overcome these limitations, we propose WorldAttention, a system-oriented
attention architecture that achieves high efficiency through the co-design of specialized
attention kernels and hierarchical KV cache management. First, we introduce Hybrid Sparse
Attention (HSA), which integrates linear global
attention supplemented with head-adaptive sparse
attention. Additionally, we design a Hierarchical KV Cache (HKV) that organizes historical KV pairs into semantically indexed pages across multi-tier
memory, enabling fine-grained
retrieval and controlled GPU residency. These two designs are supported by tailored kernels to effectively translate their theoretical efficiency into real-world performance. Extensive experiments on VBench-Long and InterVBench demonstrate that WorldAttention consistently surpasses prior state-of-the-art methods, achieving subject consistency scores of 0.9472 on VBench-Long and 0.9668 on InterVBench, respectively.
Read the full abstract ▾
All authors Zeyu Zhang, Jinyuan Mao, Dakai An, Wangbo Zhao, Hanfeng Lu, Jiasheng Tang, Yinghao Yu, Wei Wang, Bohan Zhuang
Computer Vision
Summarize Before Grounding: Query-Guided Chunk Condensation for Long-Video Temporal Grounding
Nanxing Hu, Xiaoyue Duan, Qiwei Yan et al. (+3 more)
Video temporal grounding (VTG) aims to localize the
video interval corresponding to a
language query. Recent large
vision-
language models (LVLMs) show great potential in solving such a multi-modal
reasoning task. However, long
videos often contain large amounts of redundant information that disturbs LVLMs to mine query-relevant evidence. Instead of dense frame sampling which incurs prohibitive training
memory, previous reinforcement learning with verifiable rewards (RLVR) works typically utilize sparse sampling, which makes training feasible but may miss critical evidence. In this paper, we propose a “summarize before grounding” framework (named “SumGround”) for long-
video temporal grounding. The key of SumGround is to perform query-guided chunk condensation to aggregate and retrieve query-relevant evidence. Specifically, we split the
video into several chunks and perform two-level chunk condensation. First, we introduce query-guided latent summaries, which is represented as KV states of query-guided
prompts, to compress redundant visual
tokens into compact query-relevant chunk summaries. Furthermore, we design an associative summary
retrieval scheme to rank and select chunk summaries that are most likely to contain the event interval. Both query-guided latent summary and associative summary
retrieval schemes are enabled by RLVR. To reduce
memory consumption, we propose a length-aware gradient gating module to selectively stop gradient back-propagated to visual
tokens. Extensive experiments demonstrate that SumGround performs favorably against previous state-of-the-art methods across multiple downstream
datasets, with remarkable gains on long
videos.
Read the full abstract ▾
All authors Nanxing Hu, Xiaoyue Duan, Qiwei Yan, Kailin Lyu, Jinchao Zhang, Guoliang Kang
Computer Vision
Temporal Modelling for Burn Scars on Sentinel-3
Luca Barco, Edoardo Arnaudo, Andrea Bragagnolo et al. (+2 more)
Rapid and accurate burn scar delineation from satellite imagery is essential for post-fire damage assessment. Sentinel-3 OLCI, with daily revisit and 21 spectral bands, suits rapid mapping, yet most pipelines treat acquisitions independently, leaving the pre/post-fire change signal unexploited. We present a
dataset of 246 wildfire activations (2016-2025) from the Copernicus Emergency Management Service, with Sentinel-3 OLCI temporally paired acquisitions. We
benchmark spatial and temporal (ConvLSTM-augmented) variants of three backbones (U-Net, SegFormer, ConvNeXt-UPerNet) under two input modes and spectral configurations. Temporal modeling improves segmentation only when pre-fire frames are included, and a 5-band subset matches the full 21-band OLCI configuration.
Read the full abstract ▾
All authors Luca Barco, Edoardo Arnaudo, Andrea Bragagnolo, Claudio Rossi, Paolo Garza
Computer Vision
Reinforcement Learning from Intermediate Renders for Image-to-Code Generation
Omri Kaduri, Kate Feingold, Phillip Isola et al. (+1 more)
Reinforcement learning is increasingly used to post-train
vision-
language models for image-to-code generation, such as generating SVG code from a reference image, by optimizing rewards computed from the final rendered output. However, relying on a single terminal reward provides sparse feedback that is poorly aligned with the contribution of individual
tokens. A generated program may contain operations that accurately reproduce some parts of the target image alongside others that introduce errors, yet all
tokens are trained from the same final outcome. We observe that many intermediate code prefixes are not only executable, but already produce meaningful partial renders that reflect progress toward the target. This property provides a natural source of denser supervision during generation. Based on this observation, we introduce IR4RL, an RL framework with a
token-level render-progress reward that turns changes between intermediate renders into localized feedback for the generated sequence. We evaluate our approach on Image-to-SVG and Image-to-TikZ generation. Across both tasks, our method improves over supervised fine-tuning and standard GRPO, yielding new state-of-the-art open-source models. This shows that intermediate rendering provides a simple and effective source of process supervision for RL post-training of image-to-code models.
Read the full abstract ▾
All authors Omri Kaduri, Kate Feingold, Phillip Isola, Tali Dekel
Computer Vision
Shaobo Ju, Haiyang Yu, Xuecheng Wu et al. (+5 more)
Temporal
video grounding is a key capability of advanced
Multimodal Large
Language Models (MLLMs) for the thorough understanding of
video events, which is however often limited by repeated actions and visually similar contexts in long
videos. In this paper, we study this issue from the perspective of On-policy
distillation (OPD) and propose a new training regime for MLLMs termed Counterfactual
Attention Policy
Distillation (CAPD). In particular, OPD is a viable solution for MLLMs via providing dense teacher supervision on student-generated trajectories. But its next-
token based teacher-student
distillation is hard to identify the specific
video segments supporting each predicted timestamp, which is critical for temporal grounding. In this case, CAPD measures how masking each temporal group changes the teacher's output distribution. The resulting counterfactual influence calibrates the teacher's
attention and weights
token-level
distillation, allowing the student to learn the temporal evidence that affects boundary prediction. To validate CAPD, we trained it on Qwen3-VL-8B-Instruct using only 2,500 samples for one epoch, and evaluated it on the TimeLens and multiple general
video benchmarks. Experimental results show that CAPD improves average recall by 12.0% relative to GRPO on TimeLens while preserving general
video understanding, achieving comparable accuracy to the base model.
Read the full abstract ▾
All authors Shaobo Ju, Haiyang Yu, Xuecheng Wu, Qiong Wu, Jiacong Wang, Fan Shi, Jun Peng, Yiyi Zhou