Computer Vision
AnomalyAgent: Training-Free Agentic Models for Zero-/Few-Shot Anomaly Detection
Yi Zhang, Jiawen Zhu, Lele Fu et al. (+1 more)
Benefiting from generalizability of
vision-
language models (VLMs) such as CLIP, many zero-/few-shot anomaly detection (AD) approaches have achieved impressive detection performance across various
datasets. Nevertheless, they require substantial training on large auxiliary
datasets to adapt VLMs to anomaly detection, and their inference largely relies on visual-text
embedding similarity-based anomaly scores, lacking
reasoning abilities to detect complex anomalies that require in-depth contextual understanding. To address this limitation, we propose AnomalyAgent, a novel training-free, agentic framework that leverages the advanced
reasoning and generalization capabilities of
multimodal large
language models (MLLMs) for anomaly detection. The key ingredients include 1) a comprehensive anomaly-centric toolset that enables adaptive MLLM-driven, agentic anomaly
reasoning in zero-shot settings, and 2) a customized
memory module that grounds anomaly
reasoning with few-shot, in-context reference examples. We extend evaluation beyond the detection of simple anomalies (e.g., surface defects like cracks and dents and clear lesions) in widely used
benchmarks to more diverse types of anomalies such as logical/contextual anomalies in logistics and manufacturing settings. Extensive experiment results demonstrate that our AnomalyAgent achieves substantially better performance compared to training-free VLM-based AD and generic agentic methods, highlighting its superior generalization capability in both zero-shot and few-shot anomaly detection settings. The code implementation can be find at this address.
Read the full abstract ▾
All authors Yi Zhang, Jiawen Zhu, Lele Fu, Guansong Pang
Computer Vision · Cross-listed from cs.CL
CCS: Clinical Consensus Selection for Radiology Report Generation
Xi Zhang, Yingshu Li, Zaiqiao Meng et al. (+2 more)
Radiology report generation (RRG) is commonly formulated as a single-path generation task, where a
multimodal large
language model (MLLM) produces one decoded report as the final output. While recent progress has largely been driven by scaling training data, model capacity, and
retrieval mechanisms, improving report quality at inference time remains underexplored. In this work, we observe that fixed radiology MLLMs often generate clinically stronger reports elsewhere in their candidate pool than the one selected by default decoding, suggesting that inference-time decision making remains an overlooked bottleneck. To address this, we propose Clinical Consensus Selection (CCS), a decoder-agnostic inference-time selection framework that samples multiple candidate reports and selects the one with the highest clinical consensus across the rollout pool. CCS unifies text-based utilities with a radiology-adapted utility computed by an image–report-trained
multimodal embedder, which measures candidate agreement beyond surface-level textual similarity. Across three
datasets and multiple radiology MLLMs, CCS consistently improves inference-time performance over single-path decoding and generic Best-of-N baselines, with particularly clear gains on clinical metrics. Further analysis shows that image-grounded utility forms a selection axis distinct from textual consensus and that substantial headroom remains for improving RRG at inference time.
Read the full abstract ▾
All authors Xi Zhang, Yingshu Li, Zaiqiao Meng, Jake Lever, Edmond S. L. Ho
Computer Vision
PARCEL: Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient Vision-Language Understanding
Selim Kuzucu, Alessio Tonioni, Vasile Lup et al. (+3 more)
Large
Vision-
Language Models (LVLMs) map visual inputs into dense
token sequences, imposing a quadratic computational bottleneck for inference. Elastic visual-
token compression addresses this by training a single model that can run at multiple visual-
token budgets. However, existing approaches struggle under aggressive compression. Spatial-only compression, as in nested pooling, behaves as an imperfect low-pass filter and induces spectral aliasing that obscures fine-grained detail. Query-only compression, as in nested query resampling, replaces explicit grid-aligned
tokens with non-local summaries and substantially degrades spatial grounding. To resolve this representational conflict, we introduce PARCEL (Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient
Vision-
Language Understanding), a visual tokenization architecture that dynamically partitions the labor of feature extraction. PARCEL establishes spatial pool
tokens as low-frequency layout anchors and conditions elastic query
tokens on these anchors through Pool-Conditioned Query Resampling. This encourages query
tokens to focus on complementary visual features rather than redundant spatial mapping. Extensive evaluations across 27
benchmarks show that PARCEL improves the performance-efficiency Pareto frontier, consistently outperforming existing matryoshka baselines across visual-
token budgets while preserving the "train once, deploy anywhere" paradigm.
Read the full abstract ▾
All authors Selim Kuzucu, Alessio Tonioni, Vasile Lup, Bernt Schiele, Federico Tombari, Muhammad Ferjad Naeem
Computer Vision
Zhuguanyu Wu, Ruihao Gong, Yang Yong et al. (+5 more)
Distribution Matching
Distillation (DMD) is a widely used paradigm for accelerating inference in few-step
video diffusion models. However, DMD-style
video distillation faces two coupled challenges: the fake score must track a continuously evolving generator, making training costly when frequent updates are required, while reverse-KL-style matching can be mode-seeking and conservative for preserving strong motion dynamics. To address these issues, we propose Score Gradient Matching
Distillation (SGMD). SGMD adopts a fake-score perspective by directly optimizing the fake score toward the teacher, while using teacher stop-gradient Fisher as a stable distribution-matching objective. We provide a gradient analysis that motivates this objective choice under ideal tracking. Building on this, SGMD introduces a pair of dual potentials: negative-residual (NR) for outer-loop correction and residual-contraction (RC) for inner-loop tracking. Empirically, compared to DMD2, SGMD achieves an approximately $\sim 3\times$ training speedup and substantially improves motion dynamics for 4-step distilled models while preserving temporal consistency. A human study confirms that SGMD is preferred in motion quality and overall preference, while visual quality and text
alignment remain comparable. Code is available at https://github.com/ModelTC/LightX2V.
Read the full abstract ▾
All authors Zhuguanyu Wu, Ruihao Gong, Yang Yong, Yushi Huang, Xiangyu Fan, Lei Yang, Dahua Lin, Xianglong Liu
Computer Vision
Large Depth Completion Model from Sparse Observations
Zhu Yu, Zhengyi Zhao, Runmin Zhang et al. (+7 more)
This work presents the Large Depth Completion Model (LDCM), a simple, effective, and robust framework for single-view metric depth estimation with sparse observations. Without relying on complex architectural designs, LDCM generates metric-accurate dense depth maps using a
transformer. It outperforms existing approaches across diverse
datasets and sparse observations. We achieve this from two key perspectives: (1) leveraging existing
monocular foundation models to improve the quality of sparse depth inputs, and (2) reformulating training objectives to better capture
geometric structure and metric consistency. Specifically, a Poisson-based depth initialization strategy is first introduced to generate a uniform coarse dense depth map from diverse sparse observations, providing a strong structural prior for the network. Regarding the training objective, we replace the conventional depth head with a point map head that regresses per-pixel 3D coordinates in camera space, enabling the model to directly learn the underlying 3D scene structure instead of performing pixel-wise depth map restoration. Moreover, this design eliminates the need for camera intrinsic parameters, allowing LDCM to naturally produce metric-scaled 3D point maps. Extensive experiments demonstrate that LDCM consistently outperforms state-of-the-art methods across multiple
benchmarks and varying sparsity levels in both depth completion and point map estimation, showcasing its effectiveness and strong generalization to unseen data distributions.
Read the full abstract ▾
All authors Zhu Yu, Zhengyi Zhao, Runmin Zhang, Lingteng Qiu, Kejie Qiu, Yisheng He, Siyu Zhu, Zilong Dong, Si-Yuan Cao, Hui-Liang Shen
Computer Vision
xModel-KD: Cross-modal Knowledge Distillation for 3D Scene Perception using LiDAR
Thenukan Pathmanathan, Kanchan Keisham and Thangarajah Akilan
Point cloud
segmentation is a fundamental task in 3D scene understanding. Its progress is constrained by the high cost and time required for dense 3D annotations, making labeled samples difficult to obtain. Beyond annotation scarcity, different sensing modalities face inherent limitations. 2D images provide rich texture and appearance cues, yet they lack explicit depth and
geometric structure. In contrast, 3D point clouds capture accurate spatial geometry but are sparse and contain no texture information. As a result, relying on a single modality restricts the richness of learned representations and weakens generalization. Although recent multi-modal methods that combine 3D point clouds with 2D images have demonstrated strong performance in tasks such as classification and
retrieval, they typically depend on large-scale labeled
datasets and have not been fully exploited for data-efficient dense prediction. To address these limitations, we propose a novel cross-modal knowledge
distillation framework, xModel-KD, for 3D point cloud
segmentation. Our method exploits the complementary strengths of 2D texture and 3D geometry by learning unified per-point representations through cross-modal
alignment. Specifically, we design a cross-modal fusion encoder trained with a contrastive objective that enforces feature consistency between corresponding 2D and 3D representations across multiple views. By integrating powerful pre-trained backbones with a targeted fusion strategy, the proposed framework effectively transfers appearance cues from images to geometry-aware point features. Experimental results show that cross-modal fusion achieves a 2% absolute improvement in mIoU over a
LiDAR-only baseline, demonstrating the benefit of leveraging complementary multi-modal information for scalable and annotation-efficient 3D scene understanding.
Read the full abstract ▾
Computer Vision
Evaluation of Conversational Agents: Understanding Culture, Context and Environment in Emotion Detection
Martha Teiko Teye, Yaw Marfo Missah, Emmanuel Ahene et al. (+2 more)
Valuable decisions and highly prioritized analysis now depend on applications such as facial biometrics, social media photo tagging, and human
robots interactions. However, the ability to successfully deploy such applications is based on their efficiencies on tested use cases taking into consideration possible edge cases. Over the years, lots of generalized solutions have been implemented to mimic human
emotions including sarcasm. However, factors such as geographical location or cultural difference have not been explored fully amidst its relevance in resolving ethical issues and improving conversational AI (Artificial Intelligence). In this paper, we seek to address the potential challenges in the usage of conversational AI within Black African society. We develop an
emotion prediction model with accuracies ranging between 85% and 96%. Our model combines both
speech and image data to detect the seven basic
emotions with a focus on also identifying sarcasm. It uses 3-layers of the Convolutional Neural Network in addition to a new Audio-Frame Mean Expression (AFME)
algorithm and focuses on model pre-processing and post-processing stages. In the end, our proposed solution contributes to maintaining the credibility of an
emotion recognition system in conversational AIs.
Read the full abstract ▾
All authors Martha Teiko Teye, Yaw Marfo Missah, Emmanuel Ahene, Twum Frimpong, Auxane Boch
Computer Vision
Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence
Artur Jesslen, Olaf Dünkel and Adam Kortylewski
Foundation features from self-supervised
vision models and text-to-image
diffusion models have proven effective for semantic correspondence estimation. However, because these features are learned primarily from 2D image objectives, they lack explicit 3D awareness and often confuse symmetric object sides, repeated parts, and visually similar structures that are distinct in 3D. We introduce a 3D-aware post-training framework that goes beyond available 2D foundation features by incorporating priors from 3D foundation models. Given an image, our method uses SAM3D to estimate object geometry and pose, and refines the pose through render-and-compare optimization. Subsequently, we render PartField descriptors from the reconstructed geometry into the image plane based on the estimated object pose. The resulting geometry-aware feature maps complement DINO and Stable
Diffusion features, while geodesic distances on the reconstructed shapes enable reliable filtering of candidate correspondences. We use the filtered matches as supervision to train a lightweight adapter on top of DINO and Stable
Diffusion for semantic correspondence. In contrast to prior post-training approaches that require pose annotations and rely on coarse spherical geometry, our method automatically obtains instance-specific 3D structure and uses it to guide correspondence learning. Experiments show that our approach improves semantic correspondence over the prior methods while reducing manual
geometric supervision. Code and model can be found at https:/github.com/GenIntel/3D-SC.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
DirectorBench: Diagnosing Long-Form Video Generation with Personalized Multi-Agent Evaluation
Jiamin Chen, Qianben Chen, Jiawen Zhang et al. (+5 more)
Long-form
video generation is rapidly moving from short, single-scene synthesis toward minute-long, multi-shot creation with narrative structure, cinematic control, audio, and cross-modal synchronization. However, evaluating such
videos remains challenging, since existing
benchmarks largely focus on local visual quality, short-horizon temporal consistency, or generic
prompt alignment, and provide limited diagnosis of workflow failures and user-dependent preferences. We introduce DirectorBench, a personalized multi-
agent diagnostic
benchmark for long-form
video generation. DirectorBench evaluates generated
videos with respect to 80 structured metadata entries, 7 user profiles, and 40 checkpoint criteria across 5 dimensions: script, visual, audio, cross-modal, and stability. Instead of reducing quality to a single aggregate score, DirectorBench localizes checkpoint-level bottlenecks and supports profile-aware evaluation. We evaluate 4 long-form
video generation workflows, 6 base LLMs, and 7 user profiles. Across workflows, DirectorBench reveals a between-unit bottleneck: transition quality averages only 0.256 and reaches 0.356 for the best workflow, while
prompt-level user demand fulfillment averages 0.71. We further conduct human evaluation with 14 annotators to validate the
alignment between DirectorBench and human judgment. The results show that DirectorBench captures human-perceptible quality differences and reveals workflow- and profile-dependent failure modes that are hidden by aggregate scoring. These findings highlight the importance of diagnostic and profile-aware benchmarking for long-form
video generation.
Read the full abstract ▾
All authors Jiamin Chen, Qianben Chen, Jiawen Zhang, Yidi Wu, Yuchen Li, Xiaokun Zhang, Wangchunshu Zhou, Chen Ma
Computer Vision
Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation
Jiayi Luo, Qiyan Liu, Tengyang Wang et al. (+8 more)
Autoregressive (AR)
video generation has emerged as a promising paradigm for long-horizon
video synthesis, where each frame is generated conditioned on previously generated
tokens. To accelerate inference, the KV cache is used to avoid redundant recomputation across generation steps. Nevertheless, its growth with generation length introduces increasing
memory and error accumulation, limiting the scalability of AR models to even longer sequences. Existing KV cache compression methods mitigate this issue by selectively retaining only
video tokens deemed important. However, most existing methods assess
token importance using short-horizon signals derived from the current or historical generation context, making these methods prone to overlooking
tokens that appear unimportant at early steps but later become critical for future frames. In this work, we identify an important property of trained AR
video models: although RoPE-modulated queries evolve across autoregressive steps, the underlying canonical pre-RoPE query distribution remains remarkably stable throughout the
video generation process. This approximate stationarity implies that future query distributions are estimable from historical statistics, enabling principled future-aware cache decisions without any additional training. Building on this insight, we propose Future Forcing, a training-free future-aware KV cache policy for AR
video generation. Specifically, Future Forcing first constructs a future query proxy from historical statistics, then scores KV cache
tokens by their importance under this proxy, and finally merges redundant
token pairs within the affine subspace induced by the future query. Extensive experiments show that Future Forcing improves long-horizon consistency under limited KV caches, achieving up to 1.49 improvement in subject consistency on VBench-Long for 60s generation over existing AR
video KV cache policies.
Read the full abstract ▾
All authors Jiayi Luo, Qiyan Liu, Tengyang Wang, JunHao Liu, Jiayu Chen, Cong Wang, Hanxin Zhu, Chen Gao, Xiaobin Hu, Qingyun Sun, Zhibo Chen
Computer Vision
Native Audio-Visual Alignment for Generation
Longbin Ji, Guan Wang, Xuan Wei et al. (+6 more)
Joint audio-
video generation aims to synthesize temporally synchronized and semantically coherent visual-acoustic content. However, existing open-source methods mainly rely on either dual-tower designs with posterior
alignment or fully unified tri-modal designs that mix textual context, audio and
video in one shared space. The former weakens fine-grained audio-
video co-
evolution, while the latter couples semantic conditioning with low-level synchronization. To address these limitations, we propose NAVA, a Native Audio-Visual
Alignment framework for joint audio-
video generation. NAVA is built upon context-conditioned native audio-visual
alignment: it first establishes audio-
video correspondence in a dedicated interaction space, and then uses external context to condition the joint denoising process. Specifically, NAVA is instantiated with an Align-then-Fuse MMDiT architecture, which transitions from modality-aware audio-
video alignment to modality-shared joint denoising. Furthermore, we introduce Timbre-in-Context Conditioning to associate reference timbre cues with corresponding
speech spans to achieve controllable
speech timbre. Experiments on Verse-Bench and Seed-TTS, together with a user study, demonstrate that NAVA achieves superior
video quality, precise audio-visual synchronization, competitive audio quality, and stronger reference-timbre controllability using only 6.3B parameters.
Read the full abstract ▾
All authors Longbin Ji, Guan Wang, Xuan Wei, Chenye Yang, Xiangrui Liu, Zhenyu Zhang, Shuohuan Wang, Yu Sun, Jingzhou He
Computer Vision
Boosting Zero-Shot 3D Style Transfer with 2D Pre-trained Priors
Xin Dong, Yunzhi Teng, Wenfeng Deng et al. (+1 more)
In this work, we focus on zero-shot 3D style transfer that can generate multi-view consistent stylized views of the 3D scene given an arbitrary style image. We primarily tackle the issue of data scarcity in 3D style transfer, which arises when each model is trained on only a single scene, thereby limiting the number of available content images. This scarcity significantly hampers stylization performance, as model optimization relies on a sufficient number of content-style image pairs to provide supervisory signals. Our core idea is to integrate a decoder pre-trained on large-scale 2D image
datasets into the 3D style transfer pipeline, thereby leveraging the prior knowledge encoded in the decoder from learning over numerous content-style image pairs. Our method combines feature Gaussian splatting and deferred stylization, enabling high-quality stylization with the data-sufficient decoder network while ensuring view consistency by unifying view-dependent operations into a view-invariant process. Experiments demonstrate that our Data-Sufficient StyleGaussian (DS-StyleGaussian) model outperforms existing zero-shot 3D style transfer methods in terms of visual quality across various
datasets. This work also suggests that 2D pre-training can serve as a strong enhancement for 3D tasks, bridging the data gap between 2D and 3D.
Read the full abstract ▾
All authors Xin Dong, Yunzhi Teng, Wenfeng Deng, Yansong Tang
Computer Vision
FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detection
Leqi Zhu, Junyan Ye, Kaiqing Lin et al. (+3 more)
The development of generative artificial intelligence technologies has propelled the visual realism of synthetic images to an unprecedented level. Although current interpretable detection methods based on Large
Multimodal Models (LMMs) have made certain progress, they still rely on imitation learning derived from massive volumes of forged data. Consequently, they lack genuine causal
reasoning capabilities and are prone to explanatory
hallucinations. To overcome this bottleneck, we propose FakeVLM-R1, aiming to endow the model with human-like critical thinking capabilities when performing synthetic detection tasks. Building upon Supervised Fine-Tuning (SFT), this framework integrates Group Relative Policy Optimization (GRPO) with a Critical Thinking Chain-of-Thought (CoT) mechanism. During the inference phase, the model executes a "bidirectional dialectical
reasoning" process: while proposing a forgery hypothesis, it must simultaneously invoke physical commonsense to construct an authenticity counter-proof. Furthermore, we constructed the FakeClue++
dataset with high-quality samples, which extensively introduces annotations guided by the physical laws of authentic images, providing a unified authenticity anchor for the model. Experiments confirm that FakeVLM-R1 achieves SOTA performance the evaluated models across multiple
benchmarks. It not only achieves high-precision, logically interpretable detection but also resolves the over-rejection bias of existing methods against real images, demonstrating generalization and robustness against perturbations.
Read the full abstract ▾
All authors Leqi Zhu, Junyan Ye, Kaiqing Lin, Zhiyuan Yan, Conghui He, Weijia Li
Computer Vision
Towards Consistent Video Geometry Estimation
Zhu Yu, Jingnan Gao, Runmin Zhang et al. (+9 more)
This work presents ViGeo, a feed-forward foundation model for recovering spatially dense and temporally consistent geometry from
video sequences. Built upon a plain
transformer architecture without task-specific architectural modifications, ViGeo supports streaming, full-sequence, and long-
video inference within a unified model. The key design is dynamic chunking
attention, which exposes the model to both bidirectional and causal temporal contexts during training and allows it to adapt its
attention pattern at test time without retraining. To improve supervision quality, we further introduce a completion-based data refinement framework. This framework trains a
video depth completion teacher that conditions on sparse and noisy annotations and exploits
video/multi-view context to produce dense, temporally coherent, and geometrically reliable training targets. Beyond depth and point maps, ViGeo also predicts surface normals within the same framework. Trained solely on public
datasets, ViGeo achieves state-of-the-art performance across online, offline, and long-
video depth estimation, surface normal estimation, and
video point map estimation.
Read the full abstract ▾
All authors Zhu Yu, Jingnan Gao, Runmin Zhang, Lingteng Qiu, Zhengyi Zhao, Rui Peng, Yichao Yan, Kejie Qiu, Siyu Zhu, Zilong Dong, Si-Yuan Cao, Hui-Liang Shen
Computer Vision
DualEraser: Joint Video Object and Effect Removal via Balanced Text-Mask Guidance and Decoupled Locator-Preserver
Yuqing Chen, Lin Liu, Haisu Wu et al. (+4 more)
Video object removal frequently struggles to eliminate target objects and their associated complex physical effects (e.g., smoke and light) in real-world scenes. We attribute this challenge to a fundamental semantic–pixel conflict, which manifests at two aspects: condition-level modality dissonance and optimization-level objective
entanglement. In terms of conditioning, modality dissonance emerges from single-modality information incompleteness and cross-modal dominance imbalance. During optimization, two conflicting objectives—high-level semantic erasure and pixel-level background preservation—are inextricably entangled within a single model. To address these conflicts, we propose DualEraser, a novel framework for joint
video object and effect removal. First, a Bipartite Text
prompt and a Multi-Conditional Capability Elicitation (MCCE) mechanism explicitly inject effect semantics and further leverage
multimodal priors to address the limitations of individual modalities. Second, a Learnable Deep CFG Fusion (LD-CFG) module adaptively balances the relative dominance between the text and mask conditions. Finally, we introduce a decoupled expert architecture comprising a Locator for semantic erasure and a Preserver for background pixel
alignment to break the objective
entanglement. Extensive experiments demonstrate that DualEraser achieves state-of-the-art quantitative performance on standard
benchmarks (e.g., gains of 2.16 dB and 1.44 dB on ROSE and VOR-Eval, respectively), while enabling robust removal of complex effects in open-world
videos. https://cyqii.github.io/DualEraser.github.io/
Read the full abstract ▾
All authors Yuqing Chen, Lin Liu, Haisu Wu, Xiaopeng Zhang, Yaowei Wang, Yujiu Yang, Qi Tian
Computer Vision · Cross-listed from cs.LG
Jaa-Yeon Lee, Yeobin Hong, Taesung Kwon et al. (+1 more)
Diffusion models generate highly realistic images but often struggle with precise text-image
alignment. While recent post-training methods improve
alignment using external rewards or human preference signals, their performance heavily depends on reward quality and does not directly address
alignment within the
diffusion process itself. Recent reward-free approaches such as SoftREPA demonstrate that optimizing soft text
tokens via contrastive learning can effectively improve text-image representation
alignment, outperforming standard parameter-efficient fine-tuning baselines. However, the contrastive formulation can excessively penalize negative pairs, which manifests as characteristic failure cases such as over-counting and repetition. To address this issue, we propose a lightweight, reward-free post-training method that refines soft
tokens by integrating contrastive
alignment guidance directly into the score-matching objective of
diffusion models. By assigning
alignment directions at the score level, our approach mitigates these limitations and yields more coherent and semantically faithful generations. Experiments show that our method matches SoftREPA while substantially improving its failure cases, achieving over 35% improvement in counting accuracy on the GenEval
benchmark. Our method is seamlessly applicable to existing
diffusion backbones (SD1.5, SDXL, and SD3), and is complementary to existing RL-based
diffusion post-training methods. Project page: https://jaayeon.github.io/AGSM
Read the full abstract ▾
All authors Jaa-Yeon Lee, Yeobin Hong, Taesung Kwon, Jong Chul Ye
Computer Vision
DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark
Ruofan Hu, Menghui Zhu, Jieming Zhu et al. (+8 more)
Multimodal documents contain diverse elements, such as tables, figures, and layouts, which can complicate
retrieval tasks. While current approaches typically combine dense visual
embedding models with supervised rerankers to achieve high-precision
retrieval, they face inherent limitations. First, the coarse-grained nature of dense
embeddings tends to obfuscate explicit semantics, failing to leverage structurally salient information. Second, supervised reranking models suffer from generalization bottlenecks, as their performance heavily relies on domain-specific training data. Furthermore, existing
benchmarks often lack diverse assessment dimensions and comprehensive relevance annotations, limiting reliable evaluation. To address these challenges, we propose DocRetriever, a plug-and-play framework. It enhances visual
retrieval via a layout-aware sparse
embedding technique, enabling effective hybrid encoding without the overhead of optical character recognition (OCR). We also introduce a generalizable reranker that leverages
reasoning-augmented demonstrations and optimized sampling to improve accuracy in few-shot settings. Finally, we construct a new
benchmark, MultiDocR, to enable more rigorous evaluation. Experiments across diverse
benchmarks validate DocRetriever's superiority over state-of-the-art methods.
Read the full abstract ▾
All authors Ruofan Hu, Menghui Zhu, Jieming Zhu, Bo Chen, Shengyang Xu, Minjie Hong, Xiaoda Yang, Sashuai Zhou, Li Tang, Tao Jin, Zhou Zhao
Computer Vision
VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies
Mingjian Gao, Wenqiao Zhang, Yuqian Yuan et al. (+9 more)
Recent work has begun to equip
vision-
language-action (VLA) policies with explicit intermediate
reasoning. In
embodied control, however, textual chain-of-thought is a poor fit: irrelevant or weakly textual information can interfere with action prediction, while autoregressive text decoding adds too much latency for real-time closed-loop execution. We present VISUALTHINK-VLA, a visual intermediate-
reasoning framework for accurate, low-latency VLA policies. Our bootstrapping philosophy is to guide action with effective visual thinking: VISUALTHINK-VLA bootstraps action prediction through a compact visual-evidence interface that preserves spatial precision while avoiding decoding overhead. Besides, to further improve performance and efficiency, VISUALTHINK-VLA adopts a tailored selective routing mechanism to learn the visual evidence
tokens, enabling low-latency inference while preserving high-capacity specialization. We also introduce VisualEvidence-Kit, a supervision-and-audit resource centered on a VisualEvidence-
Agent that constructs a 754.7k VLA instructions VisualEvidence-Set for route supervision and counterfactual faithfulness tests. Across multiple
benchmarks and real-
robot evaluation, VISUALTHINK-VLA achieves the highest success rate on most
benchmarks while reducing the multi-second latency of
reasoning-augmented baselines to the sub-second regime. For example, on BridgeData V2, it reduces step latency from 8.377,s with ECoT to 0.367,s, achieving a 22.8 times speedup.
Read the full abstract ▾
All authors Mingjian Gao, Wenqiao Zhang, Yuqian Yuan, Yang Dai, Binhe Yu, Zheqi Lv, Haoyu Zheng, Jiaqi Zhu, Zhiqi Ge, Zixuan Wan, Siliang Tang, Yueting Zhuang
Computer Vision
EarlyTom: Early Token Compression Completes Fast Video Understanding
Hesong Wang, Xin Jin, Lu Lu et al. (+4 more)
Video large
language models (
Video-LLMs) have demonstrated strong capabilities in
video understanding tasks. However, their practical deployment is still hindered by the inefficiency introduced by processing massive amounts of visual
tokens. Although recent approaches achieve extremely low
token retention ratios while maintaining accuracy comparable to full-
token baselines, most of them perform compression only at the late stage of prefilling, leaving the efficiency of the
vision encoder unoptimized. In this paper, we first show that
vision encoding contributes a large portion to the time-to-first-
token (TTFT). Therefore, instead of compressing visual
tokens only after the
vision encoder, performing compression inside the encoder still leaves substantial room for exploration. Based on this insight, we propose EarlyTom, a training-free
token compression framework that performs early-stage visual
token compression inside the
vision encoder, enabling significantly better TTFT reduction and higher throughput. In addition, we introduce a decoupled spatial
token selection strategy that improves the overall compression effectiveness. EarlyTom reduces TTFT by up to 2.65x and FLOPs by up to 61% on a single NVIDIA A100 GPU for the LLaVA-OneVision-7B model, while maintaining accuracy comparable to the full-
token baseline. These improvements substantially enhance the practicality of deploying
Video-LLMs in real-world production scenarios.
Read the full abstract ▾
All authors Hesong Wang, Xin Jin, Lu Lu, Chenhaowen Li, Jian Chen, Qiang Liu, Huan Wang
Computer Vision
FRUC: Feedforward Dynamic Scene Reconstruction from Uncalibrated Collaborative Driving Views
Yihang Tao, Yu Guo, Zhengru Fang et al. (+2 more)
We present FRUC, a feedforward 3D Gaussian Splatting framework for dynamic scene reconstruction from uncalibrated collaborative driving views. Existing multi-
agent reconstruction frameworks are often hindered by rigid prerequisites, demanding precise spatial calibration and slow per-scene optimization. In this paper, we rethink this task by conceptualizing a distributed multi-vehicle network as a spatio-temporally unstructured ego-centric multi-camera system, where the core challenge lies in enhancing ego-centric occluded geometry through collaboration without degrading the ego's accurately observed visible geometry, while preserving reconstruction efficiency. For efficient reconstruction, FRUC is built upon a visual grounded
geometric Transformer backbone to enable one-shot, calibration-free inference from a flexible number of multi-vehicle views. To achieve non-destructive
geometric supplementation under uncalibrated cross-
agent misalignment, FRUC first introduces an ego-centric causal occlusion field that explicitly derives occlusion
evolution as latent priors by modeling
agent-wise spatio-temporal correlations. Guided by these occlusion priors, it further formulates cross-
agent integration as a deterministic residual denoising process via zero-initialized injection, turning challenging cross-
agent fusion into bounded residual learning for robust collaborative blind-spot completion. Through extensive evaluations on real-world V2X-Real and UrbanIng-V2X
datasets, FRUC is shown to be a new state-of-the-art for the scene reconstruction of dynamic collaborative driving environments, significantly outperforming existing methods in both rendering quality and efficiency. Code is available at https://github.com/yihangtao/FRUC.git.
Read the full abstract ▾
All authors Yihang Tao, Yu Guo, Zhengru Fang, Haonan An, Yuguang Fang
Computer Vision · Cross-listed from cs.LG
Improving Adversarial Robustness of Attribution via Implicit Regularization
Amir Mehrpanah, Matteo Gamba and Hossein Azizpour
The
adversarial robustness of attributions is a fundamental requirement for reliable explainability in deep learning, yet existing approaches typically rely on computationally expensive explicit regularization. In this work, we show that attribution robustness can arise implicitly from the learning dynamics of standard
stochastic gradient descent. We theoretically motivate this effect through connections between parameter-space and input-space
curvature, and validate it across architectures,
datasets, and attribution methods, with negligible computational overhead. In contrast, we prove that such robustness gains often does not transfer to
attention-based attribution under softmax normalization, due to inherent
entropy constraints, and we validate this limitation experimentally. Finally, we show that replacing softmax
attention with kernel-based
attention restores the robustness gains in
transformer models. Our results highlight learning dynamics as a principled and practical mechanism for robust explainability, and reveal fundamental limitations of
attention-based attribution under normalization.
Read the full abstract ▾
Computer Vision
Genetically Aligned Patient Representations Improve Hematological Diagnosis
Muhammed Furkan Dasdelen, Fatih Ozlugedik, Ilaria Looser et al. (+3 more)
Multimodal alignment of histopathology encoders with transcriptomic and genomic data has been shown to significantly improve performance in downstream diagnostic tasks. Hematological cytology is unique in that visual single-
cell evaluation is often paired with cytogenetics and molecular genetics for blood cancer diagnosis. In this study, we present a framework to align single white blood
cell images with chromosomal aberrations (karyotype) and somatic mutations from targeted gene panels. Our training strategy follows a two-stage approach: (i) self-supervised,
vision-only pretraining of a
transformer aggregator using an iBOT head on a cohort of over 1500 patients, and (ii) genetic
alignment via supervised contrastive loss on acute myeloid leukemia patients. Our genetically aligned patient encoder improves hematological diagnostic tasks, outperforming slide-level histopathology foundation models. Additionally, the model provides off-the-shelf
retrieval capabilities for diseases and genetic alterations. Incorporating genetic data into patient encoders increases the quality of patient representations, providing a framework that aligns with clinical diagnostic workflows and paves the way for future
multimodal hematology-specific AI. The code and model weights are available at https://github.com/marrlab/GenBloom.
Read the full abstract ▾
All authors Muhammed Furkan Dasdelen, Fatih Ozlugedik, Ilaria Looser, Rao Muhammad Umer, Christian Pohlkamp, Carsten Marr
Computer Vision
EVL-ECG: Efficient ECG Interpretation With Multi-Aspect Heterogeneous Knowledge Distillation
Dang Nguyen Hong, Nhi Ngoc-Yen Nguyen and Huy-Hieu Pham
High-fidelity ECG interpretation is increasingly reliant on massive foundation models, yet their deployment in clinical edge-care remains hindered by extreme computational demands. While knowledge
distillation (KD) is a promising solution, traditional methods fail to capture the complex spatio-temporal dependencies of ECG signals when transferring knowledge across heterogeneous architectures. In this paper, we propose EVL-ECG, a framework specifically designed for cross-architecture
distillation of cardiac diagnostic logic. EVL-ECG introduces three ECG-aware innovations: (1) Multi-Head Cross-
Attention Alignment, which harmonizes architectural discrepancies to preserve fine-grained morphological features; (2) Optimal Transport-based Visual Feature Matching, utilizing optimal transport to maintain global structural relationships across ECG leads despite mismatched
token representations; and (3)
Geometric Intra-Architecture Relation Matching, which distills the latent diagnostic
reasoning of the teacher model. Evaluations across ECG
benchmarks demonstrate that EVL-ECG yields improvements of up to 2.4% AUC and 1.1% clinical accuracy over existing baselines. Notably, EVL-ECG establishes an efficient 2B-parameter ECG foundation model, suitable for resource-constrained clinical environments.
Read the full abstract ▾
Computer Vision
SwInception – Local Attention Meets Convolutions
David Hagerman, Roman Naeem, Jakob Lindqvist et al. (+3 more)
Sparse
vision transformers have gained popularity as efficient encoders for medical volumetric
segmentation, with Swin emerging as a prominent choice. Swin uses local
attention to reduce complexity and yields excellent performance for many tasks but still tends to overfit on small
datasets. To mitigate this weakness, we propose a novel architecture that further enhances Swin's inductive bias by introducing Inception blocks in the feed-forward layers. The introduction of these multi-branch convolutions enables more direct
reasoning over local, multi-scale features within the
transformer block. We have also modified the decoder layers in order to capture finer details using fewer parameters. We demonstrate a performance improvement on eleven different medical
datasets through extensive experimentation. We specifically showcase advancements over the previous state-of-the-art backbones on
benchmark challenges like the Medical
Segmentation Decathlon and Beyond the Cranial Vault. By showing that the existing inductive bias in Swin can be further improved, our work presents a promising avenue for enhancing the capabilities of sparse
vision transformers for both medical and natural image
segmentation tasks. Code and pre-trained weights can be accessed at https://github.com/Eiphodos/SwInception.
Read the full abstract ▾
All authors David Hagerman, Roman Naeem, Jakob Lindqvist, Carl Lindström, Fredrik Kahl, Lennart Svensson
Computer Vision
Mesh-Aware Epipolar Matching for Multi-View Multi-Person 3D Pose Estimation in Basketball
Li Yin, Qin Haobin, Tomohiro Suzuki et al. (+3 more)
Multi-view multi-person 3D pose estimation in team sports scenarios remains challenging due to player occlusions, appearance similarity caused by team uniforms, and the scarcity of annotated multi-view data, all of which limit the effectiveness and generalization capability of learning-based methods. In contrast, the performance of training-free approaches is inherently constrained by the accuracy of 2D keypoint detection and the robustness of cross-view association. To address these challenges, we propose Mesh-Aware Epipolar Matching (MAEM), a training-free framework for multi-view multi-person 3D pose estimation. Our method employs a
monocular 3D human mesh recovery model as the frontend and introduces a two-stage epipolar matching strategy based on the recovered mesh outputs. Specifically, the proposed framework combines disjoint-set-union-based clustering with per-joint triangulation to achieve robust cross-view association and accurate 3D pose reconstruction. Experiments on two public multi-view basketball
datasets demonstrate that MAEM consistently outperforms existing training-free association baselines while achieving competitive RGB-only performance in both indoor and outdoor basketball scenarios. MAEM achieves MPJPE/PA-MPJPE scores of 59.8/40.7 mm on SportCenter EPFL and 74.0/51.8 mm on Human-M3 Basketball, highlighting the effectiveness of dense mesh geometry for cross-view association without requiring target-domain training or fine-tuning.
Read the full abstract ▾
All authors Li Yin, Qin Haobin, Tomohiro Suzuki, Calvin Yeung, Mariko Isogawa, Keisuke Fujii
Computer Vision
CityGen: Structure-Guided City-Style Synthesis for Cross-City Autonomous Driving
Zezhong Qian, Zhao Yang, Lu Tan et al. (+4 more)
Autonomous driving systems are commonly trained and evaluated within limited geographic regions, which hinders their scalability when deployed in new cities. However, significant domain shifts in appearance, road
topology, and traffic patterns often cause severe performance degradation under cross-city deployment. Existing approaches based on domain adaptation, data augmentation, or synthetic data generation typically rely on labeled target data, city-specific annotations, or task-specific designs, limiting their scalability and effectiveness for holistic evaluation. In this paper, we introduce CityTransfer-Bench, a geographically disjoint
benchmark for evaluating cross-city generalization across perception,
segmentation, and planning, and propose CityGen, a
diffusion-based generative framework that performs zero-label city adaptation via HD-map-conditioned synthesis guided by city-level visual
prompts. Extensive experiments demonstrate that CityGen consistently improves cross-city robustness across multiple tasks, establishing a scalable and label-efficient foundation for generalizable autonomous driving.
Read the full abstract ▾
All authors Zezhong Qian, Zhao Yang, Lu Tan, Zhihao Yan, Weiyi Hong, Haizhuang Liu, Yawei Jueluo
Computer Vision · Cross-listed from cs.LG
Treatment-Conditioned Diffusion for Forecasting Neurodegenerative Disease Progression
Danylo Boiko and Viktoriia Mishkurova
Forecasting the progression of neurodegenerative diseases, such as Parkinson's disease, is essential for effective long-term planning and personalized therapeutic intervention. Existing systems typically produce scalar clinical scores that ignore the rich structure of longitudinal neuroimaging, while traditional generative approaches suffer from a loss of anatomical details and blurring subtle progression patterns. To address this, we introduce a novel treatment-conditioned
diffusion framework that predicts high-fidelity future
brain states by conditioning the generative process on patients' screening DaTscan images and levodopa equivalent daily dose over one year. The pipeline uses a
Transformer-based encoder to represent non-linear, time-dependent pharmacological dynamics and optimizes generation through a multi-weight region-of-interest mask that focuses on biologically critical areas. Experimental evaluation shows that our framework maintains sharp anatomical boundaries and significantly improves clinical fidelity relative to the baseline, achieving 14.0% lower MSE, 7.2% lower MAE, and 4.9% higher SSIM.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Reducing Experimental Testing in Space Propulsion Film Cooling Analyses by Pixelwise Generative Image Interpolation
Adam T. Müller, Philipp J. Teuffel, Konstantin Manassis et al. (+1 more)
We propose a machine learning approach for image regression from sparse experimental measurements. We show the application of the proposed method on film cooling studies in propulsion system development, aiming to reduce the need for extensive physical testing. Our method employs a lightweight feed-forward neural network with positional encoding to generate images conditioned by input parameters. Validated on real and synthetic data, it achieves high image similarity (RMSE < 8 %, SSIM > 93 %) while maintaining accuracy with a 30 % reduction of measurements. We further propose a knowledge-informed extension for local adaptability of the generated images. This approach significantly reduces required tests while preserving high-quality data, enabling efficient optimization of coolant injector configurations with applications beyond aerospace.
Read the full abstract ▾
All authors Adam T. Müller, Philipp J. Teuffel, Konstantin Manassis, Nicolaj C. Stache
Computer Vision
Train the Agent, Not the Expert: Learning to Harness Heterogeneous Experts for Multi-Turn Visual Reasoning
Yaowu Fan, Tao Han, Dazhao Du et al. (+2 more)
Recent advances in computer
vision have produced powerful specialized models for detection,
segmentation, counting, and other visual tasks. However, these models are typically optimized for isolated tasks, limiting their ability to support general-purpose visual intelligence, particularly when complex
language understanding and dense small-object perception are required. We propose VisHarness, a trainable visual
agent that decouples high-level perception,
reasoning, and decision-making from low-level task execution. Rather than training a task-specific model, we train VisHarness to harness a set of carefully designed heterogeneous visual experts. This paradigm preserves the
agent's general intelligence while leveraging the precision of specialized visual experts for specific tasks. With only lightweight training, VisHarness learns a generalizable policy for harnessing visual experts, enabling it to solve fundamental
vision tasks in diverse complex scenarios through multi-turn interactions with these experts. To enable efficient on-policy reinforcement learning in a live environment, we introduce dynamic visual
memory archiving to reduce the visual-
token overhead that rapidly accumulates during multi-turn interactions with visual experts. Experiments on four representative
benchmarks spanning
reasoning segmentation, generalized referring expression
segmentation, dense small-object detection, and referring expression counting show that VisHarness substantially outperforms existing general-purpose models and is competitive with or superior to task-specific models. Our code, model weights, and training data are publicly available at https://github.com/fyw1999/VisHarness.
Read the full abstract ▾
All authors Yaowu Fan, Tao Han, Dazhao Du, Andy J. Ma, Jia Wan
Computer Vision
DVSM: Decoder-only View Synthesis Model Done Right
Cheng Sun, Jaesung Choe, Min-Hung Chen et al. (+2 more)
Recent Large View Synthesis Models (LVSMs) advocate an encoder-decoder architecture that separates reconstruction and rendering into distinct networks. We re-examine this design. Through controlled experiments, we show that a decoder-only architecture, which represents scenes implicitly as a KV-cache, outperforms encoder-decoder variants while using fewer parameters at identical rendering complexity. Further analysis shows that sharing weights between the color-input reconstruction network and the camera-only rendering network better aligns their features at the same viewpoint, facilitating image synthesis. Building on this finding, our model, dubbed DVSM, further incorporates foundation model priors and stage-wise patch sizing for an improved efficiency-quality tradeoff. Our results establish a new state of the art for novel-view synthesis across multiple
benchmarks, in some cases even outperforming per-scene-optimized 3DGS under dense input views.
Read the full abstract ▾
All authors Cheng Sun, Jaesung Choe, Min-Hung Chen, Ryo Hachiuma, Yu-Chiang Frank Wang