Computer Vision
TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding
Kepeng Yang, Dongxuan Liu, Rongxin Gao et al. (+8 more)
Humans understand anomalous events through a coherent perceptual process in which they identify the focal instance, follow its behavior as the event unfolds, and interpret why it violates the expectations of the surrounding scene.
Video anomaly understanding (VAU) seeks to endow models with a similar capability, moving beyond deciding whether a
video is anomalous toward explaining how the event develops and why it matters. Although recent
vision–
language models (VLMs) can generate detailed and plausible anomaly descriptions, their semantic fluency does not ensure that these interpretations remain grounded in the correct anomaly instance over time. Existing
benchmarks typically evaluate tracking and semantic understanding through separate protocols, leaving such instance–semantic inconsistency largely unmeasured. We therefore introduce TAU-Bench, a track-centric
benchmark for jointly evaluating anomaly instance tracking and fine-grained anomaly understanding. TAU-Bench contains 1,118
videos, 1,454 tracks, and 202,438 pixel-level masks spanning 49 event and 45 scene categories, together with track-centric annotations that connect instance-level identification, event-level understanding, and scene-level
reasoning. To build TAU-Bench at scale, we developed an automated data engine integrating anomaly suitability filtering, anomaly instance track construction, hierarchical caption annotation, and human quality control. Evaluations across representative VLM families show that models producing plausible anomaly interpretations may still fail to localize and track the correct instance reliably, revealing a persistent gap between semantic
reasoning and visual grounding. These findings therefore highlight instance-grounded evaluation as an important step toward more faithful and reliable VAU systems.
Read the full abstract ▾
All authors Kepeng Yang, Dongxuan Liu, Rongxin Gao, Zixin Su, Rui Wu, Shuzhao Xie, Chenxin Li, Panwang Pan, Yuzhi Huang, Yue Huang, Jingyan Jiang
Computer Vision
SciQNet: Two-Stage Multimodal Adaptation for Scientific Image Quality Assessment
Yin-Loon Khor, Yi-Jie Wong, Jing Jie Tan et al. (+1 more)
Scientific images are essential for communicating experimental observations, quantitative evidence and conceptual knowledge. Unlike natural images, their quality depends on both visual clarity and scientific informativeness, making assessment challenging. In this work, we present SciQNet, a two-stage
multimodal adaptation framework for scientific image quality assessment. The first stage performs domain-adaptive pretraining on scientific document images and the second stage conducts task-specific fine-tuning with joint scoring and understanding supervision. For scoring-oriented supervision, we combine instruction tuning with a Huber loss derived from rating-word logits, while understanding-oriented supervision is formulated as multiple-choice visual question answering. Experiments show that using a 40% stratified subset of the domain-adaptive data gives the best performance among the evaluated pretraining fractions, suggesting that pretraining-data relevance may be as important as pretraining-data scale. The final model achieves an SIQA-S score of 92.21, an SIQA-U score of 47.38 and a combined score of 69.80. This work presents our solution to the ICME 2026 Scientific Image Quality Assessment Challenge, which ranked 2nd in the scoring track.
Read the full abstract ▾
All authors Yin-Loon Khor, Yi-Jie Wong, Jing Jie Tan, Ming Jie Lee
Computer Vision
Jiaxuan Li, Qing Xu, Xiangjian He et al. (+4 more)
Cross-modal
alignment of visual and textual representations is fundamental to
multimodal medical image understanding, yet remains hindered by uncertainty in both modalities under real-world clinical conditions. Existing
vision-
language segmentation methods rely on deterministic cross-modal matching, which overlooks aleatoric uncertainty from ambiguous boundaries and epistemic uncertainty from limited training data, leading to fragile performance under domain shift. To address this issue, we propose DistMedVL, a probabilistic
vision-
language framework that introduces a lightweight Probabilistic Cross-Modal Adapter (PCM-Adapter) upon frozen encoders to explicitly model representational uncertainty. Specifically, the PCM-Adapter comprises two sequential modules for progressive probabilistic
alignment. We first devise a Mahalanobis
Alignment Module (MAM) that models textual
tokens as Gaussian distributions and computes patch-text compatibility via Mahalanobis distance, yielding variance-conditioned matching that downweights unreliable feature dimensions. Moreover, we devise a Distribution Flow Module (DFM) that estimates modality-wise confidence parameters and performs
vision-guided refinement of textual distributions, accommodating distributional variation across imaging modalities. Extensive experiments across eight medical
segmentation benchmarks demonstrate that DistMedVL outperforms state-of-the-art methods with only 6.3M trainable parameters, exhibiting superior data efficiency, perturbation robustness and cross-
dataset generalization.
Read the full abstract ▾
All authors Jiaxuan Li, Qing Xu, Xiangjian He, Yue Li, Daokun Zhang, Fiseha B. Tesema, Rong Qu
Computer Vision · Cross-listed from cs.AI
A Unified Framework for Trajectory Prediction with Explicit Planning and Reaction Decomposition
Jiaheng Chen, Jiaxing Li, Tinghe Zhang et al. (+1 more)
Trajectory prediction has shifted toward structured formulations with explicit social modeling. However, existing methods inadequately distinguish the functional roles of social influence in trajectory planning. Observing that
agents typically form motion plans by anticipating others' future behaviors before making local reactive adjustments, we identify social interactions as playing staged roles, namely planning precedes reaction. We propose INTraJ, a unified framework that decomposes social influence into two stages: a planning stage constructs reference trajectories using future social information, and a reaction stage recovers local adjustments from the residual between full-context prediction and the reference. INTraJ supports both multi-target and single-target paradigms. Extensive experiments on four standard
benchmarks, including Argoverse 2, Argoverse 2-ped, ETH/UCY, and SDD, demonstrate consistent improvements, particularly in FDE and long-horizon consistency, with state-of-the-art performance achieved in several settings. INTraJ reframes trajectory prediction as a planning-driven two-stage process, validating that staged social modeling is critical for stable predictions. The code is publicly available at https://github.com/11isnotavailable/INTraJ.
Read the full abstract ▾
All authors Jiaheng Chen, Jiaxing Li, Tinghe Zhang, Chaopeng Guo
Computer Vision
URNet: A Unified Reparameterized Network for Efficient RGB-D Semantic Segmentation
Guoan Xu, Zhengxue Wang, Yang Xiao et al. (+3 more)
Previous RGB-D semantic
segmentation methods commonly employ dual encoders to separately process RGB and depth inputs, followed by dedicated modules for cross-modal feature fusion. However, such designs often inadequately capture depth representations and consequently limit effective cross-modal interaction, while the additional encoder branch introduces redundant computation that hinders lightweight execution. To tackle these challenges, we propose URNet, a Unified Reparameterized RGB-D Network that performs simultaneous multi-modal feature extraction and cross-modal fusion within a single encoder. Specifically, we adopt a reparameterization strategy to compact the network architecture and facilitate fast inference. Within each Reparameterized Block (RepBlock), a Linear Gated
Attention (LGA) module is introduced to fully exploit complementary RGB and depth cues across different feature scales. Furthermore, considering that decoder design has been relatively underexplored in existing RGB-D
segmentation models, we develop a concise yet effective universal decoder, termed the Pyramid Merging Decoder (PMD). Extensive experiments on multiple RGB-D
segmentation benchmarks demonstrate that URNet achieves state-of-the-art performance while maintaining high efficiency. Code will be available at https://github.com/Wild-Stephen/URNet.
Read the full abstract ▾
All authors Guoan Xu, Zhengxue Wang, Yang Xiao, Ligeng Chen, Guangwei Gao, Dongchen Zhu
Computer Vision · Cross-listed from physics.plasm-ph
SafeDivertor: Faithful Divertor Heat Flux Reconstruction from Macroscopic Plasma State Signals via Time-Frequency Prior Exploitation
Hao Si, Zehua Chen, Qingquan Yang et al. (+8 more)
Divertor heat-flux analysis is essential for understanding
plasma-wall interactions and protecting
plasma-facing components in
magnetic-confinement fusion devices, while conventional infrared-based inversion is usually performed after discharge and requires heat-conduction modeling with device-specific material properties, divertor geometry, and boundary conditions. Rather than accelerating this conventional infrared-based inversion paradigm, we introduce a new online-oriented signal-based reconstruction paradigm that directly reconstructs time-resolved radial heat-flux profiles from multi-source macroscopic
plasma-state signals available during discharge. To enable systematic study of this task, we construct DivMPS2HF, a multi-source discharge
dataset that provides the data foundation and
benchmark for signal-based divertor heat-flux reconstruction. We further propose SafeDivertor, a task-driven framework designed to address the key challenges of signal-based heat-flux reconstruction. It employs physical prior-aware initialization to provide radial-distribution guidance for target channels, input perturbation to reduce over-reliance on specific heterogeneous signals, spectral-aware reconstruction optimization to exploit time-frequency priors and preserve transient dynamics, and progressive training to stabilize the optimization of these complementary objectives. Experiments on DivMPS2HF demonstrate that SafeDivertor achieves the best overall performance among the evaluated time-series baselines across all five metrics, establishing a new performance
benchmark for signal-based divertor heat-flux reconstruction. The source code will be released on https://github.com/Event-AHU/OpenFusion
Read the full abstract ▾
All authors Hao Si, Zehua Chen, Qingquan Yang, Xiao Wang, Dengdi Sun, Wanli Lyu, Gaoting Chen, Guosheng Xu, Hang Su, Jin Tang, Jun Zhu
Computer Vision
Dual-Attention and Adversarial Transfer Networks for Sim-to-Real Cross-Orientation Wireless Sensing
Linfeng Du, Kehan Wu, Tong Zhang et al. (+1 more)
Millimeter-wave human activity recognition suffers significant performance degradation when the user's orientation changes relative to the sensing system, yet collecting labeled multi-orientation data is labor-intensive and costly. To eliminate the need for exhaustive multi-orientation measured data, we develop a physics-guided simulator that synthesizes orientation-diverse wireless training data from single-orientation motion. Specifically, to suppress orientation-induced feature variations, we propose a dual-
attention network that extracts activity-discriminative and orientation-robust representations from dual-link Doppler spectrograms. To bridge the simulation-to-reality gap, we introduce an
adversarial unsupervised transfer learning mechanism that aligns feature distributions using only a small number of unlabeled target-domain samples. The S2M-Sense platform shows high fidelity in reproducing real-world signatures, validated against 60.48 GHz mmWave measured data with an average structural similarity index measure (SSIM) of 0.84 between simulated and measured Doppler spectrograms across all 4 activities and 4 orientations. Experimental results show that S2M-Sense achieves 88.33% recognition accuracy using only the dual-link multi-orientation simulated
dataset, which improves to 95% after simulation-to-reality transfer learning with as few as 16 unlabeled measured samples. Both cases with and without transfer learning outperform state-of-the-art cross-domain sensing methods.
Read the full abstract ▾
All authors Linfeng Du, Kehan Wu, Tong Zhang, Rui Wang
Computer Vision
Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streaming
Menglin Han, Yang Ding, Yulei Lu et al. (+6 more)
Real-time long-form avatar audio-
video generation requires causal, continuous synthesis while maintaining audiovisual synchronization and visual consistency. Adapting a pretrained bidirectional model to this setting presents two key dilemmas. First, autoregressively reusing generated blocks as context creates exposure bias, causing errors and visual drift to accumulate over long rollouts. Second, a global
speech utterance does not indicates a causal generator which portion should be spoken next when only limited local audio-
video context is available. We present Vorch-Streamer, a post-training framework that addresses these challenges and enables real-time long-form Text-to-Audio-
Video (T2AV) streaming. We construct a synthetic corpus of 80K avatar clips spanning 12-21 seconds and first train a causal generator with mixed Teacher Forcing and
Diffusion Forcing. We then apply long-horizon Self Forcing with DMD
distillation, exposing the model to its own rollout distribution while preserving the quality of the pretrained bidirectional teacher. To explicitly control
speech progression, an external
language model predicts discrete 25-Hz
speech-planning
tokens, whose continuous features condition the audio
diffusion branch and align each causal block with the content it should speak. With bounded causal context and four-step denoising, Vorch-Streamer jointly generates audio and
video from text at 27.12 FPS, exceeding the 24-FPS real-time playback rate while maintaining competitive audio-lip synchronization and strong identity preservation over long-form generation.
Read the full abstract ▾
All authors Menglin Han, Yang Ding, Yulei Lu, Haoran Yu, Xin Ma, Junyi Chen, Zhangkai Ni, Lin Ma, Yaohui Wang
Computer Vision
Vorch-IR: Long-Form Unified Multimodal Identity Replacement Video Generation
Yaole Wang, Xiaoyu Chen, Xin Ma et al. (+5 more)
Video identity replacement seeks to transfer the identities of one or more subjects while preserving the motion, expressions, and temporal structure of a driving
video. Existing methods largely target single-person settings and often require task-specific structural controls, such as masks or pose representations, limiting their flexibility in general
multimodal editing systems. Progress on multi-person replacement is further constrained by the scarcity of paired training data. We present Vorch-IR, a unified framework that supports single- and dual-person identity replacement, with optional background replacement, in a single model. Built on LTX2, Vorch-IR jointly conditions on a driving
video, indexed reference images, and a textual editing instruction. The reference images need not match the pose, layout, or spatial configuration of the driving
video: their roles as subject or background references are specified through the instruction. Dense visual conditions are fused through self-
attention, while a
vision-
language context establishes semantic correspondence through cross-
attention. We further develop an automatic data construction pipeline that synthesizes paired supervision for all four editing settings. Experiments using automatic metrics and pairwise human evaluation demonstrate strong identity preservation, motion fidelity, and temporal coherence across diverse scenarios. A temporal overlapping inference strategy additionally extends the short-clip model to minute-long generation without autoregressive continuation.
Read the full abstract ▾
All authors Yaole Wang, Xiaoyu Chen, Xin Ma, Yang Ding, Gang Yue, Jingjing Chen, Lin Ma, Yaohui Wang
Computer Vision
ChronoVision: Temporal Reasoning via Latent State Reconstruction
Yifan Shen, Jian Xu, Boyi Li et al. (+6 more)
Multimodal large
language models excel at passive perception but struggle with complex visual cognitive tasks requiring multi-step temporal
reasoning. This degradation largely stems from the inherent ambiguity of
language-based
reasoning, which often fails to accurately articulate continuous visual transformations. To address this, we propose ChronoVision, a
multimodal framework designed to align visual logic with latent imagery. During supervised fine-tuning, a Reconstructive Visual Head predicts the latent representation of the final transformed state, while an ROI
Attention Locating module focuses the model on key visual evidence via semantic span queries. In post-training, we apply reinforcement learning with an implicit process grounding mechanism, guided by a composite reward function that evaluates outcome correctness, latent process
alignment, and unsupervised visual focus. Furthermore, we introduce Vbvr-VQA, a novel
dataset that evaluates temporal tracking by reformulating
video reasoning into a strict image-ordering task. Experiments demonstrate that ChronoVision achieves state-of-the-art performance on Vbvr-VQA with 74.8% in-domain and 71.6% out-of-domain accuracy, alongside a strong 55.0% accuracy on IntPhys2, a highly challenging cross-domain
benchmark.
Read the full abstract ▾
All authors Yifan Shen, Jian Xu, Boyi Li, Yuner Zhang, Tianjiao Yu, Bingxuan Li, Houze Yang, Rushi Wang, Xu Cao
Computer Vision
SCI-CLIP: Segment-Centric Inference with Reference Memory for Training-Free Open-Vocabulary Segmentation
Mohamad Zamini and Diksha Shukla
Training-free open-vocabulary
segmentation remains limited by a missing inference abstraction. Frozen
vision-
language features are produced at patch level, yet dense prediction requires a unit that simultaneously governs feature interaction, spatial support, contextual recovery, and
retrieval-based correction. We present SCI-CLIP, a segment-centric inference framework built around the principle that the same region abstraction should organize all stages of dense open-vocabulary prediction. SCI-CLIP first induces a region-consistent interaction graph over frozen visual
tokens, then reconstructs dense features by propagating values over this graph, augmenting them with selective cross-window support only where local evidence is insufficient. The same segment abstraction is subsequently used to construct and query an offline reference
memory, aligning exemplar
retrieval with the units on which prediction is made. SCI-CLIP turns frozen CLIP-style features into spatially coherent, context-aware, and
retrieval-compatible dense predictions without any training. SCI-CLIP consistently improves the structural quality of dense predictions, the robustness of contextual
reasoning, and the
alignment of exemplar-based correction, yielding stronger open-vocabulary
segmentation across eight
benchmarks. Project code is available at: https://github.com/mzamini92/SCICLIP.
Read the full abstract ▾
Computer Vision
Dual-Output Multi-Exposure HDR Reconstruction via SDR Fusion and Gain Map Inverse Tone Mapping
Jinho Kim, Jinwoo Kim and Seon Joo Kim
We propose DOME-HDR, a dual-output multi-exposure HDR reconstruction framework that jointly produces a perceptually balanced SDR image and a consistent HDR image via gain map inverse tone mapping. Given three bracketed LDR inputs, DOME-HDR first synthesizes a base SDR using a LoRA-adapted latent
diffusion model. A dual cross-
attention fusion module injects complementary structural and color cues from the under- and over-exposed images while anchoring on the mid exposure for stability. The synthesized SDR then guides HPGM, our HDR Prior-guided Gain Map network, to predict a spatially varying gain map for reliable dynamic-range expansion. We evaluate on Kalantari, Tel, and Challenge123 using both full-reference and no-reference metrics, where DOME-HDR achieves state-of-the-art HDR reconstruction quality; ablations further confirm the effectiveness of dual cross-
attention and SDR-guided gain map estimation.
Read the full abstract ▾
Computer Vision
TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMs
Yanqi Wu, Runhe Lai, Xinhua Lu et al. (+5 more)
Despite the remarkable progress of large
vision language models (LVLMs), object
hallucination remains a fundamental challenge that hinders their trustworthy deployment. A key finding motivates our work: real and hallucinated object
tokens are clearly separable in hidden representations, yet this separability is largely lost at the
language-modeling (LM) head. We propose TruthLens, a self-evaluation framework that teaches the LM head to expose a per-object truthfulness signal without any auxiliary model or additional inference cost. Concretely, a rarely-used special
token is repurposed as a reference
token. For each object-
token position, we extract the log-probability assigned to this special
token by the LM head, and define its difference from a predefined constant as the truthfulness score. The model is then fine-tuned with an MSE objective that drives scores toward 1 for real objects and 0 for hallucinated ones, while a divergence constraint preserves the original generation capability. Despite being trained on only a limited set of object categories, TruthLens generalizes effectively to
benchmarks with substantially larger label spaces. Extensive experiments across multiple LVLMs demonstrate state-of-the-art performance; notably, on Qwen2.5-VL-7B, TruthLens outperforms the previous best method on MS-COCO by over 17% in AUROC. Our code is available at https://github.com/wyqstan/TruthLens.
Read the full abstract ▾
All authors Yanqi Wu, Runhe Lai, Xinhua Lu, Qichao Chen, Zhiping Zhou, Jia-Xin Zhuang, Weijiang Yu, Ruixuan Wang
Computer Vision
ALTER: Modeling Longitudinal Changes via Regional Differencing for 3D CT Report Generation
Dongchen Li, Jitao Liang and Wei Li
Computed
tomography (CT) is widely used for clinical diagnosis and longitudinal follow-up, yet automatically generating accurate and complete radiology reports from three-dimensional (3D) CT remains challenging. Existing methods improve fine-grained correspondence between images and text by modeling anatomical regions, but remain centered on the current examination. Consequently, patient-specific longitudinal changes within individual regions remain insufficiently modeled. Meanwhile, interval changes are often distributed across multiple anatomical regions, complicating a coherent assessment of the overall longitudinal state. We propose Anatomically Localized Temporal Evidence Representation (ALTER) to address these limitations. Global Prior Integration (GPI) incorporates the prior CT and report to establish historical context for the current examination. Regional Proxy Differencing (RPD) enables each current anatomical region to retrieve a historical proxy from a single shared encoding of the prior volume and to derive localized interval evidence. Interval Change Fusion (ICF) further combines current abnormality states with region-distributed differences, converting their joint representation into change-aware soft
prompts that guide report generation. ALTER achieves state-of-the-art results on most evaluation metrics across the RadGenome-ChestCT validation and CTRG-Chest-548K test sets. Code and data preprocessing details are available at https://github.com/peytonkarlie/ALTER/tree/main.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction
Yingqing Guo, Hui Yuan, Zijian He et al. (+2 more)
Flow-based generative models are typically sampled by solving a deterministic ordinary differential equation (ODE), whereas reinforcement learning (RL) requires
stochastic rollouts for exploration and optimization. Existing online RL methods for flow models therefore replace the inference-time ODE with a
stochastic differential equation (SDE) during training. Although the ODE and SDE share the same marginal distribution in continuous time, they can behave differently after finite-step discretization. In particular, SDE rollouts often become blurry as exploration noise increases, creating a mismatch between training rollouts and inference-time ODE samples. We introduce LC-GRPO, a flow-based GRPO framework with Langevin correction. Each rollout transition takes an inference-aligned ODE Euler step followed by a
stochastic Langevin correction targeting the marginal at the resulting timestep. The required score is recovered directly from the flow velocity, requiring no additional score model, while the transition remains an isotropic Gaussian with tractable likelihood. We theoretically show that, under suitable conditions, a single Langevin correction step reduces the error of an imperfect ODE Euler step and, under matched randomness, yields a more accurate transition than standard Euler–Maruyama discretization of the reverse SDE. Additionally, experiments on SD3.5-Medium, FLUX.1-Dev, and HunyuanVideo show that LC-GRPO consistently improves reward optimization across text-to-image and text-to-
video tasks, preserves generation quality, and substantially narrows the gap between
stochastic training rollouts and deterministic ODE inference.
Read the full abstract ▾
All authors Yingqing Guo, Hui Yuan, Zijian He, Mengdi Wang, Zheng Ding
Computer Vision
Uncertainty-Aware World Model for Aerial Image-Goal Navigation
Deyi Zhu, Haoyu Fan, Yinan Zhu et al. (+4 more)
Aerial image-goal
navigation requires an unmanned aerial vehicle (
UAV) to reach a target location specified by a goal image. Existing world-model-based methods rank candidate trajectories using predicted futures, but typically rely on only one or a few point predictions, which is inadequate for large-scale outdoor environments with substantial future-state uncertainty. To address this limitation, we propose the Uncertainty-Aware
Navigation World Model (UA-NWM), an efficient latent world model for aerial image-goal
navigation, which formulates trajectory scoring as conditional out-of-distribution detection. UA-NWM represents plausible futures with an uncertainty subspace and decomposes the prediction–goal discrepancy into uncertainty-explainable and unexplainable components. Only the unexplainable residual is used for scoring, enabling robust selection without multiple future samples. Extensive experiments demonstrate that UA-NWM consistently outperforms existing
navigation world models while maintaining low inference latency. Real-world
UAV experiments further validate its practical applicability. Project page: https://duryi.github.io/UA-NWM-Project-Page
Read the full abstract ▾
All authors Deyi Zhu, Haoyu Fan, Yinan Zhu, Weichen Zhang, Shilin Ma, Xinlei Chen, Yansong Tang
Computer Vision
Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs
Ziling Huang, Yuki M. Asano and Shin'ichi Satoh
Multimodal Large
Language Models (MLLMs) have made strong progress in
video understanding, yet long
videos remain difficult: the visual
token budget grows with
video length, so temporally sparse evidence is easily lost. Existing methods, whether frame selection or
agent-based exploration, ultimately rely on a single frame set that must trade off broad temporal coverage against fine-grained, question-relevant evidence: mixing both dilutes decisive evidence, while focusing on either sacrifices the other. We propose VideoRouter (VR), which rethinks long-
video understanding as coordinating complementary evidence views rather than selecting a single subset of frames. VideoRouter first organizes each
video into a question-agnostic temporal hierarchy that partitions it into coarse-to-fine temporally coherent segments. Upper-level nodes capture broad storyline context and event progression, while lower-level nodes preserve fine-grained local details and evidence-bearing moments. This gives rise to two complementary views: a global view for coverage-oriented
reasoning and a local view for detail-oriented evidence recovery. We further introduce a verification-guided router that judges which view is better supported by its own selected evidence and decides the final answer. Across six backbones, routing improves over both views in all settings, and the choice of view is shown to be
dataset-dependent, confirming that no single evidence granularity is universally preferable. On VideoMME and LongVideoBench, our method outperforms state-of-the-art frame-selection methods by 2.5 and 1.3 points, respectively, under the LLaVA-
Video-7B backbone. We will release the code.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.HC
A Multi-Layer System for Ultra-High-Resolution Static 360-Degree Telepresence
Jiapeng Chi, Gerd Bruder, Carsten Neumann et al. (+2 more)
360-degree
video telepresence offers strong immersive potential but remains constrained by the limited resolution of current capture and display hardware. Many telepresence installations feature fixed viewpoints and largely static scenes, yet optimization strategies tailored to such setups have received limited
attention. We present a multi-layer, ultra-high-resolution system for static 360-degree telepresence that combines an 8K panoramic camera with a rotatable 4K pan-tilt-zoom (PTZ) camera. Our approach builds a three-layer representation: (1) a tile-based ultra-high-resolution panoramic background, generated by offline stitching high-detail 4K PTZ scans onto the base 8K panorama to achieve effective resolution beyond native capture, and represented as a set of spatial tiles; (2) a dynamic update layer that composites foreground motions from the 8K stream via real-time high-resolution background matting; and (3) a region-of-interest 4K layer that streams a real-time PTZ view of the selected region and additionally updates the corresponding background tiles over time. We evaluate the proposed system through comparisons with representative
video super-resolution approaches and a user study assessing perceived detail and immersive experience. Our results indicate that tile-based background refinement, together with user-guided updates, provides a practical way to balance panoramic fidelity and interactivity in static 360-degree telepresence.
Read the full abstract ▾
All authors Jiapeng Chi, Gerd Bruder, Carsten Neumann, Carolina Cruz-Neira, Dirk Reiners
Computer Vision
CoordRefer: Coordinate-Aware 3D Visual Grounding from Multiview Images
Haijie Li, Jiaxin Zhang, Dave Zhenyu Chen et al. (+3 more)
Multiview image-based 3D visual grounding predicts a coordinate frame to define a coordinate system and then regresses a 3D bounding box for localization. However, existing methods jointly optimize coordinate frame selection and box regression, leading to coordinate-relative box ambiguity and degraded grounding performance. This ambiguity arises because the same box admits different numerical representations across coordinate frames, creating multiple optimization targets and yielding invalid compromise predictions. To tackle this challenge, we propose CoordRefer, a coordinate-aware framework that decouples coordinate frame selection from coordinate-conditioned grounding. CoordRefer first selects a reference frame to define the coordinate system and then conditions 3D box prediction on the coordinate system. We perform coordinate-aware supervised fine-tuning to establish coordinate frame selection and coordinate-conditioned box regression, followed by Group Relative Policy Optimization with 3D IoU-based rewards to align both stages with downstream grounding quality. On ScanRefer with Qwen3-VL-2B, CoordRefer achieves gains of 11% in Acc@0.25 and 7% in Acc@0.5 over the coordinate-agnostic baseline, while its geometrically refined variant surpasses methods using explicit 3D inputs.
Read the full abstract ▾
All authors Haijie Li, Jiaxin Zhang, Dave Zhenyu Chen, Youyu Chen, Yanmin Wu, Jian Zhang
Computer Vision
EffectLearner: World-Aware Object-Effect Reasoning for Real-World Video Object Removal
Feier Wu, Wanke Xia, Xu He et al. (+8 more)
Video object removal must eliminate not only the target object but also its induced effects while maintaining high-fidelity and spatiotemporally coherent restoration. Existing methods mainly learn object-effect correspondences implicitly from predefined effect categories and fixed data distributions, limiting their generalization to complex real-world scenes involving compositional effects, spatially detached or weakly correlated effects, long-tail physical phenomena, and dynamically evolving interactions. We propose EffectLearner, a semantic-
reasoning-enhanced framework that combines a VLM-based Object-Effect Reasoner with a DiT-based
Video Eraser. Guided by a structured effect-analysis
prompt, the Reasoner performs cross-modal
reasoning over a target-highlighted
video and extracts compact effect-aware context, which guides the
Video Eraser toward comprehensive object-effect removal. Motion-aware mask guidance and motion-consistency supervision further improve removal coverage and spatiotemporal stability under object motion and evolving scene dynamics. To fully exploit the framework in challenging real-world scenarios, we further construct EffectWorld, a paired
video dataset specifically designed for complex object-induced effects, and introduce a progressive training curriculum that combines common supervision with complex-effect data. On the standard ROSE-Bench, EffectLearner outperforms existing baselines on most metrics and achieves clear advantages on both EffectWorld-Eval and the challenging EffectWorld-Wild, demonstrating its ability to deliver high-quality
video object removal in complex real-world scenes.
Read the full abstract ▾
All authors Feier Wu, Wanke Xia, Xu He, Zilang Zhou, Si Chen, Dongxia Liu, Liyang Chen, Qimeng Wu, Zhengbo Zhang, Wenming Yang, Zhiyong Wu
Computer Vision
From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMs
Jiawei Qiu, Yichen Xu, Jianzhe Ma et al. (+5 more)
Timely anticipation of physical hazards is essential for real-world safety, yet existing MLLM evaluations focus on harmful content or general risks, leaving proactive physical hazard prediction underexplored. Sports provide a well-suited testbed: accident causes span diverse injury dimensions and pre-accident spatiotemporal cues draw on
reasoning capabilities shared with broader safety domains such as autonomous driving and fall detection. We introduce SPRINT (Sports Proactive Risk INference Testbed), a
benchmark of 2,888 real-world sports
videos (2,440 accident, 448 safe controls) spanning 14 sports and 3 environmental settings. Accident
videos feature fine-grained annotations of early hazard cues, accident timing, and hierarchical causes; safe
videos are manually verified as accident-free and serve to diagnose
prompt-induced false alarms. Evaluating state-of-the-art MLLMs under diverse
prompts and temporal windows reveals a sharp gap between hazard sensitivity and understanding: the best model exceeds 95% in signaling hazards yet falls below 50% in identifying their causes. Diagnostic experiments further show that explicit danger queries trigger severe false alarms even on hazard-free
videos. These findings indicate that current MLLMs exhibit only superficial proactive safety, lacking stable, cause-grounded early warning, and underscore the need for reliable proactive safety in dynamic physical environments. Data and code will be open-sourced upon acceptance.
Read the full abstract ▾
All authors Jiawei Qiu, Yichen Xu, Jianzhe Ma, Mingyang Yu, Wenbin Zhu, Yang Han, Pinzheng Lv, Wenxuan Wang
Computer Vision
Hierarchical Flow Matching for 3D Point Cloud Generation
Linhao Wang, Qichang Zhang, Ye Su et al. (+1 more)
Generating high-quality 3D point clouds requires capturing both global shape
topology and local
geometric details. Existing flow-based methods rely on continuous normalizing flows (CNFs) that demand expensive ODE solving and trace estimation during training, while
diffusion models require hundreds of iterative denoising steps. Moreover, most approaches adopt single-level generation directly in point space, disregarding the hierarchical structure natural to 3D shapes. We propose Hierarchical Flow Matching (HFM) that extends flow matching to bilevel structure for unconditional 3D point cloud generation. HFM decomposes the task into two levels via optimal-transport flow matching: a Latent Flow Matching models the global shape
manifold in a compact latent space, and a Conditional Point Flow Matching reconstructs detailed point clouds conditioned on the latent code. Both flows are trained with simple MSE regression losses. The resulting straight OT paths enable efficient sampling with as few as 15 Euler steps per flow, while the structured latent space supports downstream tasks including classification. Extensive experiments on ShapeNet and ModelNet
benchmarks demonstrate that HFM achieves competitive or even best performance compared with prior state-of-the-art methods.
Read the full abstract ▾
All authors Linhao Wang, Qichang Zhang, Ye Su, Hao Wang
Computer Vision
OmniMech: All-in-one Multimodal Mechanical Benchmark for 3D Reconstruction
Taiting Lu, Runze Liu, Ziwei Dong et al. (+18 more)
Recent
vision-
language models (VLMs) can generate executable CAD programs from images, but existing methods mainly target coarse, general-purpose 3D objects and rarely address the fine-grained geometry and millimeter-level tolerances required in industrial mechanical design. We introduce OmniMech, the first million-scale
benchmark for evaluating VLMs on executable CAD generation from industrial manufacturing data. OmniMech contains more than 251,000 fully dimensioned and toleranced 2D orthographic drawings, paired with native CAD models, multi-view renderings, mesh, STEP and B-rep representations, and rich semantic annotations. The
benchmark includes four tasks: (1) parametric CAD program synthesis from engineering drawings; (2) diagram-to-3D
reasoning for geometrically and structurally consistent reconstruction; (3) annotation-grounded
reasoning over dimensions, symbols, feature callouts, and manufacturing constraints; and (4) tool-augmented agentic
reasoning using visualization, measurement, CAD execution, and verification tools. Experiments show that current VLMs and CAD-specialized models still struggle with executable program synthesis, fine-grained 3D reconstruction, and reliable enforcement of dimensions and tolerances. We will release the
benchmark data, evaluation code, and tool interfaces to support future research.
Read the full abstract ▾
All authors Taiting Lu, Runze Liu, Ziwei Dong, Sisong Bei, Jingying Zeng, Mingjia Wang, Zhenghao Li, Kaiyuan Lin, Yi-Shan Wu, Yangshoudu Zheng, Hongxing Pan, Kai Zhang, Guoliang Shi, Ling Ma, Yifan Yang, Jiaying Lu, Qi He, Sung-Liang Chen, Yi-Chao Chen, Yincheng Jin, Mahanth Gowda
Computer Vision
HERA: Historical Evidence Routing Adapter for Physical Prediction in Latent World Models
Yuanruyi, Yue Cao, Haojia Gao et al. (+7 more)
Predictive
video models have emerged as promising world models by learning latent visual dynamics from large-scale
video. Yet these models remain challenged by physical events under occlusion, where later predictions may depend on object evidence that is no longer available in the current view. Addressing this challenge requires historical evidence not only to be preserved but also to remain accessible when it becomes relevant to a subsequent prediction. Existing approaches mainly enlarge the temporal context, cache generic
video features, or impose explicit object-centric states, thereby improving the capacity or structure of retained history. However, they do not directly address how relevant historical evidence can be selectively retrieved and integrated into a pretrained predictor without interfering with its native latent workspace. Accordingly, we introduce HERA (Historical Evidence Routing Adapter), a framework for routing retained historical evidence into a frozen latent predictor, and instantiate it with Register-Routed Patch
Memory (RRPM), a lightweight adapter comprising a Structured
Memory Bank,
Memory Registers, and Workspace Registers. On the IntPhys2 Main split, HERA with RRPM improves the pairwise AvgSurprise accuracy of V-JEPA 2-G from 52.57% to 54.35%. Subgroup analysis shows particularly strong improvements on fixed-camera continuity, from 46.15% to 57.69%, and fixed-camera immutability, from 46.15% to 63.46%. These results support historical evidence routing as a practical adaptation strategy for physical prediction in latent world models.
Read the full abstract ▾
All authors Yuanruyi, Yue Cao, Haojia Gao, Guanqiu Guo, Ziyuezhang, Shangqin, Junbo Tan, Bokui Chen, Zhuo Zou, Xueqian Wang
Computer Vision
DynaPix: Can Vision-Language Models Identify the Exact Future?
Thong Nguyen, Vinh-Hien Do, Quynh Vo et al. (+2 more)
Acting in a physical scene requires knowing its real later state, not a plausible one. Current evaluations often accept words or a realistic-looking image, so the predicted state is never checked against the true one. We introduce DynaPix (Dynamic Pixels), a
benchmark that makes prediction checkable. Given a
video clip that stops before a key event and a question about a later moment, a model must pick the true future image from close candidates or a large gallery. The scenes come from a physics simulator, so the correct image and its time are known exactly and the wrong options are deliberately similar. Models often succeed when a visible event marks the target moment, but are near chance when only elapsed time marks it. Gallery search is harder still, as the true image rarely ranks first. People handle the elapsed-time items well, so the difficulty lies with the models, not the questions. Training on scene accounts drawn from the simulator's true record, not a teacher's guess, repairs much of this but not the longer elapsed time case. DynaPix thus exposes a temporal-anchoring gap: models attach a prediction to an event far better than to time itself.
Read the full abstract ▾
All authors Thong Nguyen, Vinh-Hien Do, Quynh Vo, Cong-Duy Nguyen, See-Kiong Ng
Computer Vision
APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning
Sadegh Jafari, Mohiuddin Bilwal, Fan Zhou et al. (+2 more)
Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices.
Pruning and quantization address this, but rely on manual, expert choices and on
algorithms that are hard to apply across architectures. Uniform settings also ignore how differently individual layers respond to compression, which costs accuracy. We introduce APQF, an agentic profiling-guided framework that combines structured
pruning, mixed-precision quantization-aware training, and accuracy recovery in one automated pipeline. A profiling
agent measures how cost is distributed across the model and how sensitive each part is to
pruning, and this evidence drives per-layer
pruning ratios, per-layer bit-widths, and the recovery strategy, all proposed by LLM planners and validated before execution. To our knowledge, APQF is the first framework to combine LLM-guided, profiling-grounded decisions with a fully training-aware
pruning and quantization pipeline for both CNNs and
vision transformers. We evaluate APQF on ResNet, VGG7, ViT, DeiT, and Swin using ImageNet-1k and CIFAR-10. On ImageNet it cuts compute to 5.6-7.7 percent of the original bit-operations, a 13-18x reduction, while keeping accuracy close to the baseline, and under a 200K-image budget it stays roughly 17 points higher in Top-1 than existing joint
pruning and quantization methods. On CIFAR-10 it compresses further than that method on four of five architectures. On VGG7 it reaches 93.15 percent using only 0.41 percent of baseline bit-operations, the only method at that compression level to improve on its full-precision baseline. Ablations show that uniform compression loses the most accuracy at matched compute, and that withholding profiling data from the planner hurts every model. Six LLM planners, including free open-weight ones, all reach 97.4-97.9 percent on Swin-Tiny.
Read the full abstract ▾
All authors Sadegh Jafari, Mohiuddin Bilwal, Fan Zhou, Brian Gelder, Ali Jannesari
Computer Vision
VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing
Ziyun Zeng, Zixuan Wang, Yongsheng Yu et al. (+2 more)
Evaluating generated
videos remains challenging because existing
benchmarks rely on fixed evaluation content, cover only a subset of generation and editing settings, and provide limited evidence for their scores. We introduce VideoArgus, a unified rubric-grounded framework covering five
video generation and editing settings. For each input instance, VideoArgus generates an output-blind, sample-specific rubric once and reuses it to evaluate all corresponding candidate
videos. The rubric defines concrete criteria, scoring rules, failure modes, and evidence plans, which guide criterion-specific VLM QA and visual tools to produce evidence-grounded criterion scores, rationales, and a diagnostic report. We further construct VideoArgus-Bench, containing 1,026 curated input instances built from 653 high-quality images and 416 high-quality
videos, with all
benchmark rubrics pre-generated, frozen, and released. On a separate 1,260-
video human-
alignment set, VideoArgus achieves higher within-input Spearman and Kendall correlations with human judgments than the corresponding
benchmark-specific evaluators across all five tasks. Model rankings also remain largely consistent across different rubric-generation and evaluation-VLM backbones. All code and data are released. Visit our project page: https://zzzmyyzeng.github.io/VideoArgus
Read the full abstract ▾
All authors Ziyun Zeng, Zixuan Wang, Yongsheng Yu, Hang Hua, Jiebo Luo
Computer Vision
CDSeg: A Renderable Gaussian Carrier for Image-to-3D Label Transfer
Wentao Sun, Yiping Chen, Zhengsen Xu et al. (+2 more)
Modern image models provide strong cues about what should be segmented in each view, but their masks do not by themselves determine where those labels should persist in 3D. We present Cross-Domain
Segmentation via Gaussian Splatting (CDSeg), a label-transfer interface that requires no task-specific 3D
segmentation training and uses Gaussian primitives as a renderable label carrier. An external mask source supplies the labels, while renderer-derived visibility determines which 3D primitives receive them. The carrier is instantiated either by completing each input point into one Gaussian, preserving its index, or by reusing the native primitives of an optimized Gaussian scene. CDSeg records pixel–primitive associations during rendering and fuses multi-view masks through voting and a local filter. The resulting labels can be returned to the original points, retained on the native Gaussian scene, or rendered into other views. CDSeg covers promptable, automatic instance, semantic, and
LiDAR settings and processes scenes with millions of primitives in seconds. It obtains 92.35% mIoU on DesktopObjects-360, 95.89% on NeRDS-360, and 65.77% on the full ScanNet-v2 validation split using the provided 2D semantic annotations. CDSeg thereby provides one interface for reusing 2D masks across point clouds, Gaussian scenes, and image views without a task-specific 3D
segmentation network.
Read the full abstract ▾
All authors Wentao Sun, Yiping Chen, Zhengsen Xu, Jonathan Li, John S. Zelek