Computer Vision
Yi Liu, Jing Zhang, Xiang Feng et al. (+5 more)
Multimodal large
language models (MLLMs) have demonstrated promising capabilities for remote sensing
reasoning, yet they frequently generate visually unsupported
reasoning that contradicts the visual evidence. Existing post-training methods optimize primarily for final-answer correctness, providing limited supervision for identifying and correcting
hallucinations that arise during the
reasoning process. To address this limitation, we propose FaithRS, a
hallucination-aware self-reflection framework for faithful
reasoning in remote sensing. We first construct SFHD-43K, a
dataset of 43K structured, fine-grained
hallucination annotations, and train a detector to identify
hallucination types, underlying causes, and supporting visual evidence. FaithRS then incorporates this diagnostic feedback into a Safe Detector-Guided Corrective Rollout framework, where a safety-aware gate selectively determines whether and how to revise the response. Training is driven by a composite reward that jointly optimizes answer accuracy,
reasoning faithfulness,
hallucination reduction, format compliance, and revision safety. Experiments with Qwen3.5 backbones (0.8B, 4B, and 9B) show consistent improvements across four remote sensing
benchmarks, with gains of up to 11.64 points.
Read the full abstract ▾
All authors Yi Liu, Jing Zhang, Xiang Feng, Xiaoyu Tian, Haonan Guo, Di Wang, Zengmao Wang, Bo Du
Computer Vision · Cross-listed from cs.RO
Visual Sevoing from Lines
Nicolas Andreff, Bernard Espiau and Radu Horaud
In this work, we present a fundamentally new approach to visual servoing using lines. It is based on a theoretical and geometrical study of the main line representations which allowed us to define a new representation, the so-called binormalized Plucker coordinates. They are particularly well suited to visual servoing. Indeed, they allow the definition of a proper image line
alignment concept. The control law which realizes such an
alignment has moreover several properties: partial decoupling between rotation and translation, analytical inversion of the motion equations and global asymptotic stability conditions. This control law was validated both in simulation and experimentally in the specific case of an orthogonal trihedron.
Read the full abstract ▾
Computer Vision
Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes
Yu Wang, Rui Dai, Wei Chen et al. (+5 more)
City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes. We propose Urbex, an agentic
vision-
language framework that instead casts grounding as active spatial search. Urbex represents a city-scale 3D scene as an interactive multi-view environment, allowing a VLM
agent to query landmarks, zoom into local regions, render oblique views, and commit a grounded bounding box. We optimize this tool-use policy with reinforcement learning on CityRefer, using smooth localization rewards and lightweight shaping rewards for efficient exploration. Experiments show that Urbex achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest city-scale ranking baseline. Zero-shot results on STPLS3D-Refer further indicate improved cross-scene robustness.
Read the full abstract ▾
All authors Yu Wang, Rui Dai, Wei Chen, Yi Wang, Bo Dang, Kaikui Liu, Xiangxiang Chu, Yansheng Li
Computer Vision
CityCode: Building Agent-Operable 3D Cities from Sparse Aerial Observations
Jiahui Sun, Ran Bi, Yifei Liu et al. (+8 more)
Urban spatial intelligence calls for 3D cities that
agents can measure, interpret, and modify across successive tasks. Maintaining such cities as persistent, code-based states requires lightweight acquisition and support for consistent state updates without repeated dense reconstruction. We therefore formulate a new setting for grounded urban world construction: generating large-scale, instance-level parameterized 3D cities from one low-altitude aerial RGB image per region and known camera parameters, while recovering metrically scaled buildings and explicit road geometry and connectivity. We introduce CityCode, an
agent-operable representation that unifies metric geometry, urban
topology, visual semantics, and 3D assets through persistent entity identities, with dependency-aware versioned execution for validated updates and historical restoration. We further develop Air2Code to construct this representation by combining visual structure extraction with adapted metric depth estimation. To address occlusion and incomplete coverage, recovered structure and observed appearance guide generative completion and subsequent structure-aware 3D generation for buildings and backgrounds. Experiments on CitySample-Aerial and UrbanScene3D demonstrate substantially more accurate metric structure recovery and superior visual fidelity to the target cities compared with the evaluated baselines. Across 50 task chains of 50 steps each, CityCode supports continued querying, editing, planning, and historical restoration over evolving city states. Together, CityCode and Air2Code turn sparse aerial observations into persistent,
agent-operable urban worlds for long-horizon urban spatial intelligence.
Read the full abstract ▾
All authors Jiahui Sun, Ran Bi, Yifei Liu, Xiaolong Zhou, Yishuo Xu, Yuanqi Li, Congcong Zhu, Lei Yang, Yanwen Guo, Zhihang Zhong, Hongjie Zhang
Computer Vision
ShoeFit: A New Dataset and Dual-image-stream DiT Framework for Virtual Footwear Try-On
Yuhan Li, Zhiyu Jin, Yifan Tong et al. (+5 more)
Virtual footwear try-on (VFTON), a critical yet underexplored area in virtual try-on (VTON), aims to synthesize faithful try-on results given diverse footwear and model images while maintaining 3D consistency and texture authenticity. Unlike conventional garment-focused VTON methods, VFTON presents unique challenges due to (1) Data Scarcity, which arises from the difficulty of perfectly matching product shoes with models wearing the identical ones, (2) Viewpoint Misalignment, where the target foot pose and source shoe views are always misaligned, leading to incomplete texture information and detail distortion, and (3) Background-induced Color Distortion, where complex material of footwear interacts with environmental lighting, causing unintended color contamination. To address these challenges, we introduce MVShoes, a multi-view shoe try-on
dataset consisting of 7305 well-annotated image triplets, covering diverse footwear categories and challenging try-on scenarios. Furthermore, we propose a dual-stream DiT architecture, ShoeFit, designed to mitigate viewpoint misalignment through Multi-View Conditioning with 3D Rotary Position
Embedding, and alleviate background-induced distortion using the LayeredRefAttention which leverages background features to modulate footwear latents. The proposed framework effectively decouples shoe appearance from environmental interferences while preserving high-quality texture detail through decoupled denoising and conditioning branches. Extensive quantitative and qualitative experiments demonstrate that our method substantially improves rendering fidelity and robustness under challenging real-world product shoes, establishing a new
benchmark in high-fidelity footwear try-on synthesis.
Read the full abstract ▾
All authors Yuhan Li, Zhiyu Jin, Yifan Tong, Wenxiang Shang, Benlei Cui, Xuanhong Chen, Hangcheng Zhu, Bingbing Ni
Computer Vision
Omni-RS: Unifying Multi-Task Understanding and Generation in Remote Sensing
Pan Yin. Shuai Yang, Yuanhao Liu, Datao Tang et al. (+6 more)
Practical remote sensing applications often require both understanding and generation capabilities for the same geographic scene. However, most existing methods address either understanding or generation in isolation, while existing unified remote sensing models' generative ability is limited to text-to-image. To bridge this gap, we propose Omni-RS, a unified framework supporting four remote sensing understanding tasks and five generation tasks. Specifically, we introduce the "Scene-Guided Two-Stage Training" strategy, which first adapts a
multimodal large
language model to remote sensing understanding and then uses its hidden states to guide a
diffusion transformer for image generation. To enable joint learning across diverse generation tasks, we construct UniGen, a unified multi-task generation
dataset, by organizing heterogeneous remote sensing data into a unified format. Furthermore, we introduce "Detailed Visual Conditioning", which uses visual conditions encoded by a VAE to guide the generation process, thereby supporting additional generation tasks that require reference images. Extensive experiments demonstrate that Omni-RS performs exceptionally well in understanding tasks and has achieved outstanding results in a variety of generative tasks.
Read the full abstract ▾
All authors Pan Yin. Shuai Yang, Yuanhao Liu, Datao Tang, Jialin Li, Hao Wang, Pei Liu, Kaiyu Li, Cong Zhao, Xiangyong Cao
Computer Vision
AffectWorld: Benchmarking Emotion-Driven Future World Prediction
Changhao Song, Yacong Chen, Hui Gao et al. (+2 more)
The world encompasses both physical environments and human actors. Predicting its
evolution requires understanding both physical laws and how emotions shape human actions. However, existing world model
benchmarks primarily assess physical plausibility and perceptual quality, leaving emotion-driven world
evolution largely untested.We introduce AffectWorld, a
benchmark that evaluates the chain Emotion-Action-World State through three tasks: World-State Consistency, Emotion-Action Consistency, and Causal Consistency. It comprises 1,498 everyday
video samples annotated with plausible emotions, action tendencies, and atomic world-state constraints. A counterfactual subset of 233 samples tests whether changing an
agent's emotion under fixed scene and event conditions yields corresponding changes in actions and consequences. We define eight metrics across the three tasks and an additional counterfactual metric, combining automatic visual assessment with human-calibrated VLM/LLM judgments. Evaluations of five SoTA world models (e.g., Cosmos3) required world-state change scores of 0.100-0.280 and counterfactual emotion sensitivity scores of 0.027-0.070 on a [0, 1] scale. These results reveal substantial gaps in generating the consequences of emotion-driven actions and adapting future predictions to emotional interventions. AffectWorld provides a testbed for advancing world models that capture how emotions shape future physical and social states.
Read the full abstract ▾
All authors Changhao Song, Yacong Chen, Hui Gao, Yazhou Zhang, Peng Zhang
Computer Vision
Robust Logical Anomaly Detection under Capture Shifts via Synthetic Anomalies
Ufuk Efe and Alptekin Temizel
Detectors of logical anomalies, such as missing, extra or wrongly combined parts, usually score a test image by its distance from normal (good) images. They achieve near-perfect image-level AUROC on standard industrial anomaly detection
benchmarks such as MVTec LOCO, but their robustness to changes in capture conditions remains largely unexamined. Under a heterogeneous perturbation protocol in which each test image is randomly kept unaltered or subjected to a brightness reduction or a zoom-out transformation, the normal-reference methods we tested lose 9.7 to 34.1 AUROC points, and two of the strongest, CSAD and SALAD, fall from 97.7% and 96.4% to 68.1% and 62.3%. We propose a framework in which a classifier, trained on normal images and on synthetic anomalies, scores a test image without comparing it with normal data. From a short description of each product, a
language model derives anomaly types and editing instructions, an image editor applies them to normal images, and the classifier learns to separate the edited images from automatically discovered normal sub-modes. With its full training recipe, the framework reaches 94.9% AUROC on clean images, 1.5 to 2.8 points below SALAD and CSAD, and 94.1% under the perturbation protocol. Trained without brightness or zoom-out augmentation, so that neither test perturbation occurs in training, it still reaches 92.5% (93.2% clean). These results suggest that clean
benchmark scores can hide a strong sensitivity to capture conditions, and that learning the decision from both normal images and rule violations may reduce it.
Read the full abstract ▾
Computer Vision
Unsupervised and Supervised Approaches for Infrared Small Target Detection : Overview, Challenges and Perspectives (Part 2)
Thierry Bouwmans, Anastasia Zakharova, Belmar Garcia et al. (+3 more)
Since the rise of supervised deep learning modelet dets, researchers has attempt to solve infrared small targection using various models like conventional neural networks, GANS,
Transformers, Mixture of Experts and recent graph/hypergraph neural networks. Moreover, these deep neural networks methods surpassed the traditional unsupervised models and the low-rank and sparse decomposition unsupervised models reviewed in "Traditional and Recent Approaches for Infrared Small Target Detection : Overview, Challenges and Perspectives (Part 1). In this context, the aim of this paper is to provide a complete survey of the deep learning models in infrared small target detection. First, we categorize the different approaches found in the literature in terms of the modeling models of strategies. Then, we have discussed them in terms of the critical situations that they claim to handle. Furthermore, we present the specific performance evaluation. Then, we conclude with several promising directions for future research.
Read the full abstract ▾
All authors Thierry Bouwmans, Anastasia Zakharova, Belmar Garcia, Marwa Chendeb El Rai, Meghna Kapoor, Badri Narayan Subudhi
Computer Vision
LongForcing: Scaling Temporal Manifold Matching for Long-Horizon World Rollout
Ziyu Zhu, Zhaoxu Sun, Fan Jiang et al. (+5 more)
Autoregressive
video diffusion models have emerged as a foundation for long-horizon world rollout. However, when extended far beyond their training windows, these models suffer from progressive drift: visual quality deteriorates, motion becomes inconsistent, and object identity gradually collapses as generated frames are recursively reused as context. We study a supervision-horizon gap: short-horizon
distillation leaves later rollout states without direct supervision. We show that matching short-window marginals need not determine long-horizon behavior, while longer on-policy trajectories expose delayed failures to training. This motivates temporal
manifold matching at scale for long-horizon world models. Building on this insight, we introduce LongForcing, a
distillation approach that scales teacher supervision to long student self-rollouts. LongForcing preserves the short-horizon teacher-forcing and causal ODE-
distillation stages, while replacing the realscore teacher in the final distribution-matching stage with an extended-horizon bidirectional teacher. The long-horizon teacher supervises extended student self-rollouts using the standard distribution-matching formulation, without modifying the student architecture or increasing inference cost. We evaluate LongForcing on text-to-
video generation and interactive action-conditioned world models, achieving a VBench Quality score of 85.52 and improving WorldRoamBench 60s action score by 23.9 points in first-person ablations. Qualitative stress tests further sample interactive rollouts up to 24 hours, far beyond the training horizon. These results suggest that scaling supervision along the temporal axis is a practical step toward long-horizon world models. The project website and code are at https://github.com/Fantasy-AMAP/LongForcing.
Read the full abstract ▾
All authors Ziyu Zhu, Zhaoxu Sun, Fan Jiang, Mengchao Wang, Junfeng Ni, Ting-Bing Xu, Zhicheng Liu, Mu Xu
Computer Vision
Efficient Scene-Text Refinement via Pixel-Space Grouped Image Generation
Yaoli Liu, Guo-Hua Wang, Yiyang Chen et al. (+4 more)
Generated images often have coherent layouts and visual content, yet local errors in small text and complex characters still limit their readability and practical use. Scene-text refinement requires accuracy, preservation, and efficiency. Existing methods based on variational autoencoders (VAEs) can lose fine strokes during reconstruction, while full-image editing can shift background colors. In this work, we introduce PixText, a model that formulates local text refinement as grouped image generation in pixel space. Conditioned on the original image, PixText generates only the target regions, preserves unedited content, and uses MeanFlow for four-step generation. We further introduce Text Refinement Bench to evaluate text correction and image preservation in generated images. On this
benchmark, PixText achieves 83.06% glyph pass and 94.66% exact target-character OCR accuracy, versus 36.77% and 52.31% for the strongest baseline. Its four-step MeanFlow variant retains an 81.84% pass rate with less than one GPU-second of amortized generation cost per image.
Read the full abstract ▾
All authors Yaoli Liu, Guo-Hua Wang, Yiyang Chen, Qing-Guo Chen, Zhao Xu, Weihua Luo, Yao-Xiang Ding
Computer Vision
Traditional and Recent Approaches for Infrared Small Target Detection : Overview, Challenges and Perspectives (Part 1)
Thierry Bouwmans, Anastasia Zakharova, Belmar Garcia et al. (+3 more)
Infrared small target detection is often used in different civilian and military applications to detect static or moving targets in scenes like in ground, air and sea surveillance. First valuable research has emerged since 30 years ago and the two last decades witnessed very significant publications in this field following the rise of low-rank and sparse decomposition (model-based methods) and deep learning models (data-based methods). Furthermore, several surveys can be found in the literature but none of them addresses an overall review in this field. In this context, the aim of this paper is to provide a complete survey of the traditional and recent approaches in infrared small target detection. First, we categorize the different approaches found in the literature in terms of the modeling models, i.e mathematical models, machine learning models and signal processing models. Then, we have discussed them in terms of the critical situations that they claim to handle. Furthermore, we present the available resources,
datasets and libraries. Then, we conclude with several promising directions for future research. For the sake of reading, this survey is divided into two parts: 1) "Traditional and Recent Approaches for Infrared Small Target Detection : Overview, Challenges and Perspectives (Part 1)" about conventional and low-rank and sparse decomposition (model-based methods), and 2) "Traditional and Recent Approaches for Infrared Small Target Detection : Overview, Challenges and Perspectives (Part 2)" about deep learning models (data-based methods).
Read the full abstract ▾
All authors Thierry Bouwmans, Anastasia Zakharova, Belmar Garcia, Marwa Chendeb El Rai, Meghna Kapoor, Badri Narayan Subudhi
Computer Vision
FALLNEST: Dual-Domain Dataset Condensation of Imbalanced IMU Data for Wearable Fall Detection
Rashi Niyas P, Ritika Jain and Hitika Tiwari
Wearable fall detection is hampered by rare fall events, scarce labeled IMU data, and the cost of training deep models on large, imbalanced corpora. Existing methods either rely on hand-crafted features with shallow classifiers, which limits learning from raw signals, or train deep networks on full
datasets that require re-sampling to counter class skew and demand substantial data and compute. To address this, we propose a novel Fall detection framework via Normalized-gradient and
Embedding matched Synthetic Time-frequency condensation, dubbed FALLNEST, a data-centric approach that distills large, imbalanced IMU
datasets into tiny, class-balanced synthetic training sets. Building on dual-domain time-series condensation (CondTSC), FALLNEST introduces a fall-specific 7-channel input with accelerometer vector magnitude, class-balanced K-Means initialization, and normalized gradient matching with gradient clipping, combined with MMD-based
embedding alignment and multi-view
Fourier-domain augmentation, so that class balance arises from the condensed set itself rather than from re-sampling. We evaluate on SisFall, KFall, and MobiFall using strict subject-wise splits and naturally imbalanced test sets, reporting accuracy, Macro-F1, and PR-AUC over five seeds. A 102K-parameter TinyCNN trained on only 32 to 146 synthetic windows (about only 1% of the training windows) reaches 93.5-95.8% mean accuracy and 0.89-0.98 PR-AUC, with standard deviations of 1.0-1.7% in accuracy, demonstrating the efficacy of our approach.
Read the full abstract ▾
Computer Vision
LiFUSE: Frequency-Spatial Unified Self-Supervised Pretraining for Liver Disease Analysis on Non-contrast CT
Hao Zhang, Yi Huang, Wei Liu et al. (+6 more)
Given the growing global burden of liver tumors and diffuse liver diseases, non-contrast computed
tomography (NCCT) is increasingly used for opportunistic screening because of its low cost and favorable safety profile. However, downstream tasks such as tumor detection and fine-grained disease diagnosis remain challenging due to NCCT's inherently low tissue contrast, lack of hemodynamic enhancement, and complex anatomical background. These limitations make subtle, heterogeneous, and multifocal hepatic lesions difficult to identify. Although large-scale self-supervised pretraining can learn robust representations from unlabeled NCCT scans, conventional voxel-wise reconstruction does not fully exploit the complementary information available in NCCT. Under weak NCCT contrast, low-level intensity patterns become less informative for pathology recognition, limiting the capture of lesion-specific discriminative features. In this paper, we propose the Liver Frequency-Spatial Unified Self-Supervised Pretraining (LiFUSE). The frequency branch applies random band-stop masking and spectrum reconstruction to encourage the model to capture subtle differences between lesions and surrounding tissues, while the spatial branch performs hybrid-granularity masked volumetric reconstruction to accommodate lesions of diverse sizes and morphologies. Finally, a cross-domain consistency scheme couples the spatial and frequency branches, yielding more discriminative representations for hepatic structures and lesions. Extensive experiments on three clinically relevant tasks across internal cohorts and a public target-domain transfer setting show that LiFUSE consistently outperforms strong baselines. These results suggest that unified frequency-spatial learning can improve liver disease analysis on NCCT, highlighting its potential for large-scale screening.
Read the full abstract ▾
All authors Hao Zhang, Yi Huang, Wei Liu, Yuan Gao, Zeju Li, Xu Han, Yu Shi, Ling Zhang, Ke Yan
Computer Vision
Heading Hierarchy Reconstruction for Complex and Long Documents with Level-Wise Multimodal Memory
Qian Chen, Xianyin Zhang, Lifan Guo et al. (+1 more)
Vision-
language models (VLMs) have been widely adopted for document parsing and optical character recognition (OCR), as they can accurately extract page-level elements. However, downstream applications such as
retrieval-augmented generation (RAG) also require coherent document-level structure. Although these models indicate heading levels using different numbers of preceding “\#” symbols, page-by-page prediction often lacks document-level consistency. In practice, input-resolution and context-length constraints make it infeasible for a VLM to jointly process all page images in documents containing tens or hundreds of pages. To address this limitation, we propose a VLM-based framework that incrementally reconstructs heading hierarchies with a level-wise
multimodal memory. The
memory retains a compact subset of all previously processed pages, selected by anchors. Our framework supports flexible anchor selection, and in this work we select the most recent node at each level, allowing the VLM to observe the visual, textual, and layout patterns of headings at every level. For comprehensive evaluation, we also introduce the Complex and Long Document Heading Hierarchy Reconstruction
Benchmark (CLDoc-HHRBench), comprising 250 documents and 11,212 pages with expert-verified annotations. Experiments on CLDoc-HHRBench and three existing
benchmarks demonstrate the effectiveness of our method.
Read the full abstract ▾
All authors Qian Chen, Xianyin Zhang, Lifan Guo, Chi Zhang
Computer Vision · Cross-listed from cs.LG
A Controlled Evaluation of Curvature Inspired Scalar Feature Scaling for Domain Generalization
Sunny Gupta and Amit Sethi
Domain-specific
curvature is an appealing idea for domain generalization (DG), but
curvature learning can denote substantially different computations. This paper evaluates one deliberately narrow implementation: the code calls it HyperDG, but it rescales Euclidean encoder features during training by learned positive scalars and uses an ordinary linear classifier on unscaled Euclidean features at inference. It does not implement a Lorentz or Poincaré representation, geodesic prediction, or a Sharpness-Aware Minimization inner loop. We audit the executed objective and evaluate six conditions on PACS, VLCS, Office Home, and TerraIncognita. The completed design contains 384 runs: six conditions, four held-out targets, and four run seeds per
dataset. Checkpoints are selected using only the three source-validation environments. Inference first averages paired seed differences within each held-out target; targets, rather than 16 seed-by-target observations, are the per-
dataset clusters. The full implementation's equal-
dataset macro point estimate is 1.61 percentage points below native DomainBed ERM (95% clustered CI [-3.17, -0.04]). All four
dataset point estimates favor ERM, but an exact four-
dataset sign-flip test gives p = .125 and Holm-adjusted p = .250; we therefore claim no adjusted evidence of a difference. The cleaner domain-specificity contrast, separate learned source scales without their variance penalty versus one shared learned source scale, is -0.13 pp (95% CI [-0.53, 0.27]). Its 90% CI lies within a revision-stage $\pm 1$ pp smallest effect of practical interest (TOST p = .003), supporting equivalence only for the source-selected suite macro, not every
dataset, target, or final-step endpoint. The study thus supplies a bounded negative result for this scalar training package; it does not test or refute genuine
hyperbolic representation learning.
Read the full abstract ▾
Computer Vision
Closing the Estimation Gap: Training-Free Transductive Inference for Cross-Domain Few-Shot Action Recognition
Changyi Li, Zeyu Yang and Yu Xiao
Cross-domain few-shot action recognition (CD-FSAR) aims to recognise unseen action classes in a target domain from a handful of labelled
videos. Despite advances in representation learning, CD-FSAR inference still relies on prototypes estimated from extremely limited support examples. To assess this bottleneck, we perform an oracle analysis with the representation fixed. Constructing oracle prototypes and decision boundaries from ground-truth query labels improves 5-way 1-shot accuracy by 7–20 percentage points across five target
datasets. This reveals a substantial estimation gap: the representation contains useful discriminative information, but prototypes and boundaries estimated from scarce supports fail to capture each episode's true class structure. Motivated by this, we propose TRACE, a training-free transductive inference framework that improves episode-level estimation without modifying the backbone. TRACE comprises two complementary stages. Confidence-Thresholded Transductive Prototype Refinement (CTPR) refines support-based prototypes using confidently predicted queries. The refined prototypes initialise a lightweight per-episode classifier, which Transductive Classifier Refinement (TCR) further adapts by encouraging confident, class-balanced predictions over all queries. Together, the two stages improve prototype and boundary estimation, enabling better use of information in the unlabelled queries. Across five target
datasets (HMDB51, UCF101, SSv2, Diving48, and RareAct) in both 1-shot and 5-shot settings, TRACE consistently improves a strong CD-FSAR baseline, including gains of more than six percentage points on HMDB51 1-shot, while recovering much of the oracle-estimated gap. These results suggest that improving test-time estimation is a powerful, broadly applicable complement to representation learning for CD-FSAR under extreme support scarcity.
Read the full abstract ▾
Computer Vision
VARI: Robust Text-to-Video Generation via Group-Structured Intent Alignment
Mingxuan Zhu, Hohei Chan, Shiyuan Liu et al. (+2 more)
Modern text-to-
video (T2V) models generate visually compelling
videos, yet meaning-preserving changes in
prompt wording can substantially alter semantic realization and overall quality. Existing
prompt rewriting modifies the inference input, whereas conventional post-training constructs supervision from one
prompt formulation at a time; neither explicitly optimizes reliability across equivalent forms. We introduce VARI (Verified
Alignment over Representations of Intent), a group-structured framework that treats automatically verified formulations of the same visual intent as a shared supervision unit. VARI builds matched form-seed rollout pools, cross-scores each
video against all forms, and applies a wording-independent structured intent audit to identify valid realizations and hard semantic failures. Group-Consistent
Distillation (GCD) aligns every verified form with interchangeable valid realizations, while Form-Contrastive Flow-DPO (FC-DPO) learns group-level preferences from audited positive-negative pairs, prioritizing same-seed, different-form contrasts. On HunyuanVideo-1.5, VARI attains the highest T2V-CompBench, VBench, and StoryEval scores among the evaluated methods. On
prompt-form extensions of T2V-CompBench and VBench, it improves weakest-form
alignment over the strongest
prompt-level baselines by 2.45 and 3.55 points and reduces cross-form drop by 30.5% and 32.2%, respectively, while preserving visual quality. Human evaluation, natural-caption controls,
prompt-enhancement experiments, diversity analyses, and transfer to LTX-2.3 further support the robustness and transferability of these gains, while VARI requires no additional inference-time module.
Read the full abstract ▾
All authors Mingxuan Zhu, Hohei Chan, Shiyuan Liu, Yuanxing Zhang, Lu Chen
Computer Vision
Benchmarking Multimodal Large Language Models for Zero-Shot Human-Object Interaction
Oriol Rabasseda, Zenjie Li, Kamal Nasrollahi et al. (+1 more)
Multimodal Large
Language Models (MLLMs) have demonstrated strong zero-shot
reasoning across diverse domains, but their potential for Human-Object Interaction (HOI) remains unexplored. Unlike traditional HOI approaches that rely on extensive labeled
datasets and fixed interaction taxonomies, MLLMs offer flexible
reasoning, making them well-suited for dynamic, real-world scenarios. In this work, we present the first systematic study of MLLMs for HOI recognition and detection within a pipeline that combines an external object detector for entity localization with zero-shot
multimodal reasoning for interaction understanding. We introduce a
benchmark framework for principled zero-shot evaluation, consisting of (i) a unified taxonomy of zero-shot prompting strategies, and (ii) a methodology that transforms
token-level logits into interaction scores, enabling fair comparison with specialized HOI models under standard metrics. To support
prompt-based evaluation, we additionally construct new question-answer annotations for standard HOI
datasets, transforming them into a format suitable for MLLMs. Building on this, we propose two visual prompting strategies, namely Objects as Context and Black Other Objects, that guide MLLMs to target objects while preserving scene context. Experiments on HICO, HICO-DET, and V-COCO demonstrate state-of-the-art zero-shot performance, achieving 53.50 mAP on HICO and 23.69 mAP on HICO-DET. Overall, our work provides a principled benchmarking framework for MLLMs in HOI, opening the door to new models and
datasets and enabling flexible open-world interaction understanding.
Read the full abstract ▾
All authors Oriol Rabasseda, Zenjie Li, Kamal Nasrollahi, Sergio Escalera
Computer Vision
FlowCred: Hierarchical Credit Assignment and Functional Coordination for Joint Text-to-Video Post-Training
Mingxuan Zhu, Yanwei Huang, Shiyuan Liu et al. (+4 more)
Prompt optimization and generator post-training are two complementary ways to improve text-to-
video (T2V) generation, and recent systems increasingly train both modules jointly. However, optimizing a
prompt policy and a
video generator from the same terminal reward creates two ambiguities:
prompt quality is confounded with
stochastic generation, and the two modules may realize overlapping local improvements. We introduce FlowCred, a two-level framework for joint T2V post-training. First, hierarchical credit assignment uses matched
prompt-seed rollout grids to separate
prompt-level credit from
prompt-conditioned generator residuals under shared randomness. Second, functional coordination probes alternative current-policy
prompts at states from actual generation trajectories, providing generator-aware evidence for
prompt optimization while selectively attenuating generator updates already supported by conditioning changes. Across multiple T2V
benchmarks, FlowCred consistently improves matched joint-training baselines. On LTX-2.5, FlowCred improves a strictly matched joint-training control from 61.47 to 64.46 on T2V-CompBench and from 55.58 to 58.08 on StoryEval, without increasing decoded-
video or reward-model-call budgets. Factorial and realized-update analyses show complementary contributions from
prompt shaping and generator routing, while human evaluation and transfer across backbones and updater families provide further supporting evidence.
Read the full abstract ▾
All authors Mingxuan Zhu, Yanwei Huang, Shiyuan Liu, Xinzhi Zhang, Shujie Li, Yuanxing Zhang, Lu Chen
Computer Vision
Human-Inspired Asymmetric Collaboration for Dual-Student Semi-Supervised Medical Image Segmentation
Huiyi Ren, Ziming Fang, Jiaming Cai et al. (+2 more)
Dual-student learning enriches predictive diversity in SSMIS by introducing a strongly augmented view alongside a weakly augmented view. Insufficient strong augmentation may homogenize the students, whereas excessive augmentation can corrupt anatomical evidence and destabilize collaboration. This challenge echoes feedback-regulated human instruction, where learners receive differentiated tasks and the task difficulty is continually adjusted according to their evolving responses. Inspired by these, we propose Human-Inspired Asymmetric Collaboration (HAC), a closed-loop dual-student framework that maintains weak–strong learning through adaptive difficulty calibration and geometry preservation. Specifically, Reinforcement-guided Difficulty Calibration (RDC) formulates strong-augmentation control as a window-level reinforcement learning process, using augmentation history, realized perturbations, and training responses to dynamically adjust the augmentation magnitude, operation number, and magnitude range. Meanwhile, the Gramian Geometry Preservation Constraint (G2PC) preserves the early-stage three-branch geometry through global volume and pairwise area constraints, preventing shared supervision from homogenizing the two students. Comprehensive evaluations across LA, Pancreas, and BraTS-2019 validate both the segmentation effectiveness and capability-aware difficulty calibration of HAC.
Read the full abstract ▾
All authors Huiyi Ren, Ziming Fang, Jiaming Cai, Rui Xu, Haojie Li
Computer Vision
H-DDCM: Hadamard Codebooks for Fast Generative Image and Video Compression
Or Ronai, Vladimir Kulikov and Tomer Michaeli
Training-free
diffusion-based compression methods repurpose the generative priors of a
diffusion or a flow model to compress data without retraining them as task-specific codecs. Recent codebook-based approaches such as Turbo-DDCM achieve strong perceptual quality at ultra-low bitrates, but their reliance on dense Gaussian codebooks remains costly in both
memory and computation, and scales poorly with the large latent dimensions of modern image and
video generators. Their combinatorial bitstream encoding further becomes computationally heavy at this scale. We introduce H-DDCM, a scalable generative compression framework that replaces Gaussian codebooks with implicit randomized Hadamard dictionaries. The fast Walsh-Hadamard transform allows for implicitly synthesizing full-dimensional dictionaries with $O(d \log d)$ compute for projection and reconstruction and $O(d)$
memory, without materializing the full-dimensional dictionary. We further introduce an efficient table-free lexicographic coding scheme that makes combinatorial encoding practical at larger dimensions. Beyond images, we adapt the framework to an autoregressive
video flow model and develop a streaming compression scheme that periodically re-encodes recently reconstructed frames into a fresh KV-cache prefix, preserving temporal context across segments at no additional transmitted bits. This enables long-form, close to real-time generative
video compression on a single GPU. Across image and
video benchmarks, H-DDCM achieves competitive rate-distortion-perception performance while substantially reducing the
memory and computational overhead of codebook-based training-free generative compression.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
SYNPRED: A Synergistic Approach to Multimodal Learning for Clinical Prediction
Ivana Janíčková, Yen Y. Tan, Thomas H. Helbich et al. (+6 more)
Multimodal medical data captures complementary aspects of disease. It provides a comprehensive basis for diagnosis and the identification of biological associations linked to progression or treatment response. Established contrastive learning approaches exploit shared information across modalities, but fall short in exploiting complementary, modality-specific signals relevant for a task. Here, we propose a synergistic
multimodal variational autoencoder (VAE) framework that separates task-relevant unimodal and
multimodal information from instance-specific variation task-irrelevant, allowing complementary signals across modalities to be leveraged for prediction. The method yields benefits even if modalities are missing at test time. We evaluate the method on a treatment response prediction task using medical imaging and RNA data. Empirical comparison with
alignment-based and
multimodal VAE state-of-the-art approaches demonstrates improved
multimodal prediction performance, and consistent benefits for prediction from a single modality after
multimodal training. The full code can be found here: https://github.com/cirmuw/SYNPRED.
Read the full abstract ▾
All authors Ivana Janíčková, Yen Y. Tan, Thomas H. Helbich, Philipp Seeböck, Julieta Di Marco, Zsuzsanna Bago-Horvath, Ulrike Heber, Thomas Spiegel, Georg Langs
Computer Vision
APT: Feedback-Driven Agentic Post-Training for Text-to-Image Generation
Sirui Zhang, Fangyun Wei, Dong Chen et al. (+5 more)
Post-training is crucial for improving text-to-image models, yet existing pipelines typically rely on manually curated training data and fixed training distributions. We introduce APT (Agentic Post-Training), a fully automated framework that adaptively post-trains a base model according to its evolving capability profile. APT coordinates three
agents to construct evaluation sets from user requirements, diagnose model-specific strengths and failure modes, and generate reinforcement learning data with an adaptive
prompt distribution that emphasizes underperforming capabilities. By repeatedly evaluating, diagnosing, and updating the generator, APT lets the current model's behavior shape its next training tasks, forming a feedback loop inspired by recursive self-improvement. For general-purpose settings without user-specified evaluation scenarios, we further develop an evaluation harness that enables broad and diverse evaluation set generation. Experiments across multiple base models,
benchmarks, and generation architectures demonstrate that APT is model-agnostic,
benchmark-agnostic, architecture-agnostic, and fully automated.
Read the full abstract ▾
All authors Sirui Zhang, Fangyun Wei, Dong Chen, Jinjing Zhao, Yitong Wang, Yunuo Chen, Baining Guo, Zheng-Jun Zha
Computer Vision
LensErase: A Foundation Model for Erase Anything
Yunuo Chen, Xiuyu Wu, Qinhong Yang et al. (+11 more)
We introduce LensErase, a general-purpose mask-based Erase Anything foundation model designed to cover a broad spectrum of real-world erase scenarios. LensErase supports diverse objects, visual text, watermarks, associated visual effects, multiple-object removal, and a wide range of object scales, aspect ratios, and image resolutions. First, to enable such broad coverage, we construct LensErase-100M, a large-scale
dataset containing 101.07 million paired source-mask-target examples spanning diverse erase scenarios. Second, we pretrain LensErase from the text-to-image model Lens, adapting its strong generative prior to mask-based erasing. To efficiently incorporate the source image and erase mask, we propose Mask-Aware RoPE, a 3D rotary positional encoding that directly injects mask information into the source-image
tokens without introducing a separate mask
token sequence. Finally, we refine the model through supervised fine-tuning and reinforcement learning with contract-guided, region-aware rewards, and apply TraceFusion at inference to preserve high-fidelity content outside the effective edit region. Across six public
benchmarks and the proposed LensErase-Bench, LensErase achieves substantial improvements under both reference-based objective metrics and reference-free VLM evaluation. We hope LensErase can serve as a foundational model for erase-related applications and systems.
Read the full abstract ▾
All authors Yunuo Chen, Xiuyu Wu, Qinhong Yang, Jinjing Zhao, Yanchen Dong, Sirui Zhang, Dong Chen, Fangyun Wei, Lei Shi, Ji Li, Dongdong Chen, Guo Lu, Yan Lu, Baining Guo
Computer Vision
Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression
Vatsal, Soumil Malhotra, Priyanshu Chahal et al. (+1 more)
Vision Transformers achieve strong performance but incur high computational costs from quadratic
token interactions. Training-free
token reduction methods improve efficiency, but their robustness remains underexplored. We propose Prune-and Merge ViT, a training-free framework for off-the-shelf models. It combines class
attention-based
pruning with similarity-based merging to reduce redundancy while preserving semantic information. To evaluate robustness we introduce two metrics. Relative Accuracy Drop (RAD) measures relative accuracy degradation under transformations, and Prediction Flip Rate (PFR) quantifies prediction consistency. On pretrained ViT and DeiT, Prune-and-Merge reduces computation competitively while maintaining robustness and better accuracy than existing
token reduction methods. Our analysis suggests that hybrid
token reduction does not introduce
hallucination-like behavior and preserves prediction stability for common image transformations.
Read the full abstract ▾
All authors Vatsal, Soumil Malhotra, Priyanshu Chahal, Sanjeev Kumar
Computer Vision · Cross-listed from cs.RO
Data-Free Continual Learning for World Action Model with Rectified Generative Replay
Kaipeng Fang, Hangxu Liu, Chaojian Shi et al. (+5 more)
World action models (WAMs) built on pretrained
video generation models adapt quickly to new robotic tasks, but they forget previously learned tasks under sequential fine-tuning. Experience replay (ER) mitigates such forgetting by rehearsing data of previous tasks, yet this data is often unavailable in robotics, where a policy is deployed as weights only and its training data is not shared along with it. In this work, we ask whether a WAM can achieve data-free continual learning by regenerating its own past. Our key observation is that a WAM can indeed autoregressively regenerate complete rollouts of previously learned tasks and replay them to mitigate forgetting. However, replaying such raw rollouts still lags far behind replaying real data, which we attribute to two issues. First, the WAM is unable to recognize task completion and keeps generating purposeless motion after the task is done. Second, under open-loop generation, the generated actions gradually drift away from the generated observations. To address both issues, we present rectified generative replay (R$^2$-WAM), which truncates generated rollouts at task completion with an agentic evaluator and filters out rollouts whose actions are physically inconsistent with the observations. On the four LIBERO suites and five real-world tasks, R$^2$-WAM reduces forgetting by over 90% relative to sequential fine-tuning and nearly matches ER without accessing any data from previous tasks. Our analysis further reveals that forgetting in WAMs concentrates in inverse dynamics modeling: a forgotten WAM still produces visually plausible observations of previous tasks while its generated actions fail. Project page: https://kaipengfang.github.io/r2-wam/
Read the full abstract ▾
All authors Kaipeng Fang, Hangxu Liu, Chaojian Shi, Yunsong Zhou, Hongzi Zhu, Minyi Guo, Jingkuan Song, Lianli Gao
Computer Vision
COMPASS: Contextual Moment Prior Alignment for Generalizable AIGI Detection
Zhan Yang, Tiancheng Shen, Weijie Lyu et al. (+4 more)
Generalizable AI-generated image (AIGI) and deepfake detection remain challenging due to the asymmetry phenomenon, where adaptation to limited training distributions can degrade the transferable representations of
Vision Foundation Models (VFMs). Recent subspace adaptation methods mitigate this issue by freezing high-magnitude principal components while adapting the residual subspace. However, such magnitude-based decomposition is data-agnostic and does not explicitly account for target-relevant semantics, potentially leading to semantic misalignment. We propose COMPASS (COntextual Moment Prior
Alignment for Subspace Separation), a data-aware subspace adaptation framework that uses a small unlabeled Semantic Proxy Set to estimate target activation statistics. COMPASS incorporates activation covariance into the decomposition of pre-trained weights, preserving target-relevant representations as a stable semantic reference while the residual adapter learns forgery-related variations. The proxy calibration requires no gradient-based optimization. Extensive experiments demonstrate strong generalization across cross-
dataset, cross-method, joint cross-
dataset-method, and open-world cross-generator evaluations, achieving state-of-the-art performance on both Deepfake and AIGI detection
benchmarks.
Read the full abstract ▾
All authors Zhan Yang, Tiancheng Shen, Weijie Lyu, Shuo Chen, Bernard Ghanem, Yibo Yang, Ming-Hsuan Yang
Computer Vision
FlowSim: Rendering-Free Closed-Loop Simulation for Evaluating End-to-End Driving Systems
Hanfeng Wu, Alvaro Marcos-Ramiro, Michael Schmidt et al. (+1 more)
Closed-loop evaluation of end-to-end (E2E) driving systems typically relies on generating sensor observations and re-processing them through the perception encoder, introducing substantial computational cost and a sim2real gap between simulated and real sensor observations. In this paper, we ask a more fundamental question: is image-space simulation necessary for closed-loop evaluation of E2E driving systems? We propose FlowSim, a rendering-free simulation paradigm that predicts the internal latent representation directly from a previous latent and target-frame symbolic scene conditions. The predicted latent is then processed by the original frozen downstream modules, eliminating image rendering and perception re-encoding. Explicit cross-frame
agent association improves latent transition prediction and rollout stability, while variable-horizon simulation enables long-horizon prediction from a common real anchor without accumulating intermediate rollout errors. Experiments on nuScenes and NavSim demonstrate that FlowSim preserves downstream behavior competitively with, and often exceeds image-space simulation in source-to-reference agreement across variable simulation horizons and both dense and sparse latent representations. By predicting the latent directly, both instantiations of FlowSim simulate each step in 0.04–0.06s, substantially faster than the image-space simulation methods they are compared against. These results establish FlowSim as an efficient and practical alternative to conventional image-space simulation methods.
Read the full abstract ▾
All authors Hanfeng Wu, Alvaro Marcos-Ramiro, Michael Schmidt, Christoph Stiller
Computer Vision
Dynamic Interpolation for Training-Time Prediction Reshaping in Low-Light Image Enhancement
Peirui Zhu, Shijie Hao, Jingxi Liao et al. (+2 more)
Low-light image enhancement (LLIE) is an important image/
video processing problem for reliable visual systems under poor illumination, yet it remains challenging due to the strong coupling among insufficient illumination, noise amplification, and structural degradation. Existing methods typically rely on sophisticated network architectures or engineered objective functions. While effective, the former increases model complexity and deployment burden, whereas the latter relies on empirically tuned loss weights. We revisit supervised LLIE from a training-dynamics perspective and observe that brightness-related errors tend to converge early, while residual noise and structural artifacts often remain insufficiently optimized in later stages. To address this issue, we propose a Dynamic Interpolation Training-only Module (DITM), which introduces no additional learnable parameters and adaptively reshapes the model prediction for supervised feed-forward LLIE training. Specifically, DITM employs brightness-aware interpolation, in which the model prediction is combined with a mean brightness-aligned image constructed by rescaling the model prediction according to the mean brightness of the ground-truth image. By reducing the relative dominance of global brightness discrepancy, DITM makes residual local errors more influential during backpropagation after global luminance has largely stabilized, thereby facilitating artifact suppression and fine-detail restoration in later training stages. DITM is used only during training and is fully removed at inference, introducing no additional inference cost. Extensive experiments on multiple
benchmark datasets and CNN-,
Transformer-, and Mamba-based LLIE backbones show that DITM consistently improves fidelity and perceptual quality while preserving the inference-time architecture and introducing no additional inference-time computation.
Read the full abstract ▾
All authors Peirui Zhu, Shijie Hao, Jingxi Liao, Richang Hong, Meng Wang