Computer Vision
ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting
Yuke Li, Weihang Liu, Cheng Zhang et al. (+8 more)
Feed-forward 3D Gaussian Splatting models offer fast single-pass reconstruction,but scaling them to match per-scene optimization quality is fundamentally hindered by the scarcity of large-scale 3D annotations. A practical compromise is predict-then-refine,where post-prediction optimization compensates for the limited capacity of the feed-forward network. However,standard feed-forward 3DGS is trained solely for zero-step rendering error,ignoring whether its output constitutes a good initialization for the downstream optimizer. We present ForeSplat,an optimization-aware training framework that equips feed-forward 3DGS models to produce initializations explicitly designed for rapid,effective refinement. By offloading part of the scene-modeling burden to the optimizer,ForeSplat substantially reduces the capacity pressure on the feed-forward model,making high-quality reconstruction feasible even with compact networks. At its core is MetaGrad,a lightweight multi-anchor meta-gradient training rule that bypasses costly higher-order differentiation through the 3DGS optimizer. MetaGrad unrolls a short inner-loop refinement trajectory,samples anchor states,and back-propagates aggregated first-order gradients to the prediction head as a surrogate optimization-aware signal. This fine-tuning adds no inference cost and enables high-quality reconstruction within seconds after a few refinement steps. We instantiate ForeSplat on diverse backbones,including AnySplat,Pi3X,and a distilled variant tailored for edge deployment. Across all tested architectures,a ForeSplat-trained initialization converges in fewer refinement steps and reaches a higher peak reconstruction quality than its vanilla counterpart,even fully converged. The framework consistently bridges the gap between amortized prediction and per-scene optimization,establishing a practical path toward lightweight,high-fidelity 3D reconstruction.
Read the full abstract ▾
All authors Yuke Li, Weihang Liu, Cheng Zhang, Yuefeng Zhang, Jiadi Cui, Zixuan Wang, Junran Ding, Haoyu Wu, Yujiao Shi, Jingyi Yu, Xin Lou
Computer Vision
FRED: A Multi-Modal Autonomous Driving Dataset for Flooded Road Environments
Connor Malone, Sebastien Demmel and Sebastien Glaser
The Flooded Road Environments
Dataset (FRED) is, to our knowledge, the first multi-modal autonomous driving
dataset specifically targeting the collection of data from scenarios involving water hazards on the road. The
dataset contains images from a 2.3 MP FLIR Blackfly USB3 camera, 64-beam 360 degree point clouds from an Ouster OS1-64
LiDAR, and data from an iXblue ATLANS-C IMU corrected by a Geoflex RTK GNSS, from five separate locations captured both during and after flooding events. The data has been released in two formats: a KITTI-style format for easy integration with existing data tools, and the RTMaps format for direct replay of the vehicle's data capture. We provide semantic labels to enable the training and evaluation of both single-sensor and sensor-fusion methods for water hazard detection. Position and velocity, as well as data captured under dry conditions, are provided to enable the development of location-based detection methods that may incorporate maps, and to evaluate other tasks such as localisation and SLAM.
Read the full abstract ▾
Computer Vision
Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction
Lei Chu and Yuhuan Zhao
Deepgenerative models havebecomeapromisingapproach for human motion prediction due to their ability to capture
multimodal distributions and represent diverse human be haviors. However, generating predictions that are both di verse and jointly consistent among interacting
agents re mains challenging. In addition, most existing approaches are primarily evaluated using single-
agent (marginal) met rics, which fail to fully reflect the joint dynamics of multi
agent interactions. We propose a
diffusion-based frame work that improves multi-
agent motion prediction by lever aging rich contextual information from historical trajecto ries. This information is incorporated through a guidance mechanism to enhance the diversity and expressiveness of predicted motions. To further enforce interaction consis tency, we introduce an energy-based formulation that re fines the joint trajectory distribution while preserving the plausibility of individual trajectories. Extensive experi ments on four
benchmark datasets demonstrate that our approach consistently outperforms existing methods. No tably, our approach substantially improves both marginal (ADE/FDE) and joint (JADE/JFDE) metrics on ETH/UCY over strong marginal baselines. Compared with prior joint prediction methods, it delivers significant gains in marginal metrics while maintaining competitive joint performance.
Read the full abstract ▾
Computer Vision
ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration
Hangyeol Lee and Joo-Young Kim
Diffusion Transformer (DiT) has emerged as a powerful model architecture for generating high-quality images and
videos. In the case of
video DiT, 3D Spatio-Temporal
Attention increases
token length in proportion to the number of frames, sharply increasing computational cost.
Token reduction methods mitigate this cost by exploiting spatial redundancy, but existing approaches rely on inaccurate similarity estimates and lightweight matching
algorithms, resulting in poor matching quality and only marginal acceleration. To overcome these limitations, we propose ORBIS, an SW-HW co-designed accelerator for
video DiT. ORBIS leverages the output activation from the previous timestep to obtain more accurate inter-
token similarity, substantially improving matching quality and enabling a higher
token reduction ratio. We further introduce a Distribution-Aware
Token Matching (DATM)
algorithm that captures global
token distribution and explicitly minimizes
token-pair loss for additional gains. To fully hide DATM latency, we design specialized, deeply pipelined hardware and minimize its hardware cost through quantization, occupying only 2.4% of total area with negligible accuracy loss. Extensive experiments show that ORBIS achieves about 2x higher
token reduction ratio than the state-of-the-art approach, AsymRnR, while delivering up to 4.5x speedup and 79.3% energy reduction compared to an NVIDIA A100 GPU.
Read the full abstract ▾
Computer Vision
PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought
Chaoqi Chen, Qile Xu, Wenjun Zhou et al. (+1 more)
Understanding 3D point clouds through
language remains a fundamental challenge in computer graphics and visual computing, due to the irregular structure of point cloud data and the lack of explicit
reasoning in existing 3D
multimodal models. While Chain-of-Thought (CoT)
reasoning has shown strong effectiveness in LLMs and image-based MLLMs, its extension to 3D understanding remains largely underexplored. In this paper, we propose a data-centric framework for constructing large-scale CoT supervision tailored to 3D point cloud understanding. Our framework consists of a two-stage pipeline that first refines point-text instruction data via
vision-
language-model-based quality evaluation and reference-guided refinement, and then synthesizes high-quality
reasoning paths through Human-in-the-Loop
Prompt Optimization (HiLPO). Using this approach, we build PoCoTI, a CoT-enhanced point-text instruction-following
dataset containing 55K samples with explicit
reasoning paths. Fine-tuning PointLLM on PoCoTI yields PointLLM-R, a
reasoning-capable 3D
multimodal language model. Extensive experiments on generative 3D classification and captioning demonstrate that PointLLM-R achieves state-of-the-art performance and generalizes robustly to real-world scanned point clouds and multi-turn dialogue scenarios.
Read the full abstract ▾
All authors Chaoqi Chen, Qile Xu, Wenjun Zhou, Hui Huang
Computer Vision · Cross-listed from cs.CL
LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
Yifan Dai, Zhenhua Wu, Bohan Zeng et al. (+18 more)
Joint audio-visual
reasoning is essential for omnimodal understanding, yet current
multimodal large
language models (MLLMs) still struggle when
reasoning requires fine-grained evidence from both modalities. A central limitation is that explicit text-based chain-of-thought (CoT) compresses continuous audio-visual signals into discrete
tokens, weakening temporal grounding and shifting intermediate
reasoning toward
language priors. We argue that a unified latent space is a better medium for such
reasoning because it preserves dense sensory information while remaining compatible with autoregressive generation. Based on this insight, we propose LatentOmni, a cross-modal
reasoning framework that interleaves textual
reasoning with audio-visual latent states. LatentOmni introduces feature-level supervision to align latent
reasoning states with task-relevant sensory features and uses Omni-Sync Position
Embedding (OSPE) to maintain temporal consistency between latent audio and visual states. We further construct LatentOmni-Instruct-35K, a
dataset of audio-visual interleaved
reasoning trajectories for supervising latent-space
reasoning. Comprehensive evaluation across multiple audio-visual
reasoning benchmarks demonstrates that LatentOmni achieves the best performance among the evaluated open-source models and consistently outperforms the Explicit Text CoT baseline, supporting latent-space joint
reasoning as a promising path toward stronger omnimodal understanding.
Read the full abstract ▾
All authors Yifan Dai, Zhenhua Wu, Bohan Zeng, Daili Hua, Jialing Liu, Bozhou Li, Yuran Wang, Chengzhuo Tong, Hao Liang, Xiaochen Ma, Junbo Niu, Tianyu Guo, Yang Shi, Yue Ding, Yiyan Ji, Bingyin Mei, Yushuo Guan, Yuanxing Zhang, Pengfei Wan, Fangcheng Fu, Wentao Zhang
Computer Vision
Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness
Hangyeol Lee, Hyojeong Lee and Joo-Young Kim
Diffusion Transformers (DiTs) achieve superior image generation quality but suffer from quadratic computational complexity relative to
token count. While various
token reduction (TR) methods have been proposed to mitigate this cost, they overlook the primary objective of generative models: minimizing recovery error, which requires reflecting output
token similarity. They rely solely on input
token similarity inherited from reduction-only ViT paradigms, leading to a fundamental misalignment with this objective. To bridge this gap, we propose DiTo, a novel TR paradigm that shifts the focus toward output-centric
token reduction. Based on the observation that output
token similarity is consistently preserved across adjacent timesteps, DiTo utilizes prior-step similarities as an effective proxy to establish
token correspondences at a Matching timestep, which are then reused across multiple subsequent Reduction timesteps. To optimize this interleaved scheduling, we propose Pair Match Ratio (PMR)-guided Interval Scheduling to determine the optimal matching frequency. Furthermore, to mitigate localized approximation errors and resulting blocking artifacts caused by repeated reuse, we propose Frequency-aware
Token Matching by incorporating a selection-frequency penalty. Extensive experiments demonstrate that DiTo consistently outperforms existing TR methods with 1.6-3.9 dB higher PSNR at comparable speedups, achieving a superior Pareto frontier.
Read the full abstract ▾
Computer Vision
ConvNeXt-FD: A Fractal-Based Deep Model for Robust Biomedical Image Segmentation
Joao Batista Florindo and Amanda Pontes de Oliveira Ornelas
Biomedical image
segmentation is a critical task in medical diagnosis and treatment planning, enabling precise delineation of anatomical structures and pathological regions. Despite significant advancements, challenges persist due to the inherent variability, noise, and complex morphology present in diverse medical imaging modalities. This paper introduces ConvNeXt-FD, a novel deep learning architecture for robust biomedical image
segmentation, built upon a U-Net-like encoder-decoder framework leveraging the powerful ConvNeXt backbone. Our approach integrates a hybrid loss function combining the Dice coefficient with a boundary-aware regularization term inspired by a differentiable formulation of Fractal Dimension, designed to enhance the model's sensitivity to object boundaries and shape fidelity. We rigorously evaluate ConvNeXt-FD across six distinct biomedical
datasets: BUSI (Breast Ultrasound Images), DDTI (Thyroid Ultrasound Images), FluoCells (Fluorescent
Cell Images), IDRiD (Diabetic Retinopathy Images for Optic Disc
Segmentation), ISIC2018 (Skin Lesion Images), and MoNuSeg (
Nuclei Segmentation). Experimental results demonstrate that ConvNeXt-FD, particularly when initialized with ImageNet pre-trained weights, achieves competitive and often superior performance compared to existing state-of-the-art methods across various metrics, including Dice, Jaccard, Accuracy, Sensitivity, Specificity, and False Positive Rate. The integration of ConvNeXt as a strong encoder, coupled with the boundary-aware regularization, proves effective in capturing both high-level semantic features and fine-grained boundary details, leading to more accurate and reliable
segmentations in challenging biomedical contexts.
Read the full abstract ▾
Computer Vision
Virtual 3D H&E Staining from Phase-contrast Back-illumination Interference Tomography
Anthony Song, Boyan Zhou, Mayank Golhar et al. (+3 more)
Three-dimensional (3D) histopathology of unprocessed tissues has the potential to transform disease management by enabling volumetric characterization of tissue microarchitecture and in-vivo assessment. Back-illumination Interference
Tomography (BIT) is a new phase microscopy technology that provides rapid, non-destructive volumetric imaging of unprocessed tissues. However, translating BIT volumes into clinically interpretable H&E images remains challenging, particularly due to shift-variant contrast and the absence of quantitative validation
benchmarks. We introduce HistoBIT3D, the first voxel-wise paired BIT and fluorescence-labeled
nuclei dataset, enabling quantitative evaluation of structural preservation in unsupervised virtual staining against ground-truth
nuclear distributions. Using this
dataset, we present a novel virtual staining framework that translates BIT volumes with shift-variant contrast into realistic H&E volumes by leveraging bidirectional multiscale content consistency and cross-domain style reuse to enhance structural fidelity and perceptual realism. Our method achieves state-of-the-art realism metrics while significantly improving 3D
nuclei segmentation accuracy and boundary preservation under zero-shot Cellpose evaluation. Together, these contributions establish a quantitatively validated, structurally faithful, and scalable pipeline for 3D virtual H&E staining, advancing the paradigm of slide-free, volumetric computational histopathology. Our data and code are available at: https://github.com/aasong113/HistoBIT3D_VirtualStaining.
Read the full abstract ▾
All authors Anthony Song, Boyan Zhou, Mayank Golhar, Marisa Morakis, Alex Baras, Nicholas Durr
Computer Vision
Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning
Dazhao Du, Jian Liu, Jialong Qin et al. (+7 more)
Video large
language models (
Video LLMs) can achieve strong
video-QA accuracy without reliably tracking spatiotemporal dynamics. A model may answer a motion question from static cues, for example, and give the same prediction even after the underlying motion is reversed. Correctness-based reinforcement learning does not directly address this problem because it rewards the final answer without requiring the policy to respond to task-relevant
video dynamics. We propose Counterfactual Relational Policy Optimization (CRPO), which explicitly trains
Video LLMs to respond to controlled changes in the visual input. For each training example, CRPO constructs a counterfactual
video, such as a horizontally flipped or temporally reversed version of the original, and jointly optimizes rollouts from both
videos under a shared policy. Factual supervision anchors what the model should answer, while counterfactual supervision constrains when that answer should change: predictions should change when an intervention alters task-relevant dynamics and remain stable when the queried property is preserved. This coupling provides a direct behavioral learning signal without requiring ground-truth labels for transformed
videos or annotated spatiotemporal
reasoning traces, while discouraging indiscriminate answer changes. To evaluate this property, we introduce DyBench, a paired counterfactual
benchmark with 3{,}014
videos and a strict pair-accuracy metric. Across paired spatiotemporal evaluations and standard
video benchmarks, CRPO improves sensitivity to motion and temporal changes while improving performance on general
video understanding. The gains also extend to segment reordering, a transformation never used during training, suggesting that CRPO learns sensitivity to
video dynamics beyond the training interventions. The project website can be found at https://ddz16.github.io/crpo.github.io/ .
Read the full abstract ▾
All authors Dazhao Du, Jian Liu, Jialong Qin, Tao Han, Bohai Gu, Fangqi Zhu, Yujia Zhang, Eric Liu, Xi Chen, Song Guo
Computer Vision
RiT: Vanilla Diffusion Transformers Suffice in Representation Space
Le Zhang, Ning Mang and Aishwarya Agrawal
Flow matching with $x$-prediction – regressing the clean data point rather than the ambient velocity – is known to exploit low-dimensional
manifold structure effectively in pixel space \cite{li2025back}. We ask whether a pretrained representation space, while containing a low-dimensional data
manifold of comparable intrinsic dimensionality, offers a distribution more favorable for flow-matching learning. Comparing pixel, SD-VAE, and DINOv2 features along four
geometric axes, we find that pixel and DINOv2 share nearly identical intrinsic dimensionalities (both $\hat{d}\!\approx\!33$) yet DINOv2 exhibits $7.3\times$ higher effective rank, $35\times$ better covariance conditioning, $11.5\times$ lower excess kurtosis, and $1.7\times$ lower on-
manifold interpolation error; SD-VAE latents are consistently intermediate, indicating that the advantage stems from representation-learning objectives rather than mere compression. These statistical properties render the flow-matching regression well-conditioned and remove the need for the specialized prediction heads or Riemannian transport used by prior DINOv2
diffusion methods. We propose the Representation Image
Transformer (RiT): a vanilla
Diffusion Transformer trained by $x$-prediction on frozen DINOv2 features, augmented only by a dimension-aware noise schedule and joint [CLS]-patch modeling. On ImageNet $256{\times}256$, RiT attains FID 1.45 without guidance and 1.14 with classifier-free guidance, outperforming DiT$^\text{DH}$-XL with $19\%$ fewer parameters (676M vs.\ 839M). The resulting ODE is efficiently solvable at coarse discretizations: with classifier-free guidance, $5$ Heun steps already reach FID 2.0 and $10$ steps reach 1.25, without
distillation or consistency training. Code at https://github.com/lezhang7/RiT.
Read the full abstract ▾
Computer Vision
Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow
Chengsheng Zhang, Chenghao Sun, Zhining Xie et al. (+1 more)
Large
Vision-
Language Models (LVLMs) represent a significant leap towards empathetic
agents, demonstrating remarkable capabilities in
emotion understanding. However, the internal mechanisms governing how LVLMs translate abstract visual stimuli into coherent emotional narratives remain largely unexplored, primarily due to the scarcity of visual counterfactuals and the diffuse nature of emotional expression. In this paper, we bridge this gap by introducing a steering-vector-based causal attribution framework tailored for descriptive emotional
reasoning. To this end, we construct a specialized
dataset to demystify the emotional
circuits underlying the three-stage “Adapt-Aggregate-Execute” mechanism. Crucially, we discover a functional decoupling: visual emotional cues are aggregated in middle layers via sentiment-specific
attention heads, but are subsequently translated into narrative generation in deep layers through
emotion-general pathways. Guided by these insights, we regulate the emotional information routing to strengthen
attention flow and amplify the semantic activation to consolidate expression. Extensive experiments on the comprehensive MER-UniBench demonstrate that our methods significantly improve performance via inference-time intervention, effectively mitigating emotional
hallucinations and corroborating the causal fidelity of the discovered
circuits.
Read the full abstract ▾
All authors Chengsheng Zhang, Chenghao Sun, Zhining Xie, Xinmei Tian
Computer Vision
Video as Natural Augmentation: Towards Unified AI-Generated Image and Video Detection
Zhengcen Li, Chenyang Jiang, Liangxu Su et al. (+4 more)
AI-generated content (AIGC) is rapidly improving, creating an urgent need for detectors that generalize across data sources, deployment pipelines, and visual modalities. A strongly generalizable detector should remain robust under distributional variations. However, we identify a consistent failure mode: SOTA AI-generated image detectors often collapse when applied to frames extracted from
videos. Through systematic analysis, we show that this cross-modal gap arises from both entangled synthesis-agnostic
video processing shifts, including color conversion, codec compression, resizing, and blur, and model-specific fingerprints introduced by modern
video generators. Motivated by these findings, we propose VINA (
Video as Natural Augmentation), a unified AIGC detection framework that jointly trains on image and
video data. VINA uses
video frames as physically grounded natural augmentations and further introduces a cross-modal supervised contrastive objective to align image and
video representations under a shared real/fake decision boundary. Extensive experiments on 14 image,
video, and in-the-wild
benchmarks show that VINA delivers bidirectional gains, improves robustness and transferability, and achieves state-of-the-art performance across nearly all evaluated settings without complex augmentation or
dataset-specific tuning.
Read the full abstract ▾
All authors Zhengcen Li, Chenyang Jiang, Liangxu Su, Tong Shao, Shiyang Zhou, Ming Tao, Jingyong Su
Computer Vision
Foresee-to-Ground: From Predictive Temporal Perception to Evidence-Driven Reasoning for Video Temporal Grounding
Zelin Zheng, Xinyan Liu, Ruixin Li et al. (+4 more)
Current
Video-LLM approaches for
Video Temporal Grounding (VTG) typically rely on direct timestamp generation from an unstructured visual-
token stream, often leading to brittle numerics and inconsistent boundaries. To address this, we propose Foresee-to-Ground (F2G), a framework that reformulates VTG as a verifiable Identify-then-Measure problem. F2G integrates Predictive Temporal Perception with Evidence-Driven
Reasoning: it learns boundary-sensitive temporal representations to build a
video-wide evidence pool of candidate event segments, and exposes these segments to the LLM as citable evidence units that bind boundary prediction to explicit event hypotheses. By decoupling event identification from precise boundary measurement, F2G stabilizes grounding and makes predictions verifiable. Extensive experiments demonstrate that F2G consistently improves grounding accuracy across diverse
benchmarks, transfers robustly across different
Video-LLM backbones, and preserves general
video understanding capabilities.
Read the full abstract ▾
All authors Zelin Zheng, Xinyan Liu, Ruixin Li, Antoni B. Chan, Guorong Li, Qingming Huang, Laiyun Qing
Computer Vision · Cross-listed from eess.IV
Entropy-Guided Self-Supervised Learning for Medical Image Classification
Joao Florindo and Viviane Moura
Accurate and robust medical image classification is paramount for early disease diagnosis and treatment planning. However, challenges such as limited annotated data, high intra-class variability, and subtle inter-class differences often hinder the performance of deep learning models. This paper introduces a synergistic deep learning framework that leverages the strengths of self-supervised learning and transfer learning for enhanced medical image classification. Our approach employs two distinct ConvNeXt-Tiny models: one pre-trained on a large-scale natural image
dataset (ImageNet) and another pre-trained using an
entropy-guided Masked Autoencoder (MAE) on the target medical
dataset. Both models are then fine-tuned on specific medical image classification tasks. A final ensemble strategy, based on averaging predicted probabilities, is utilized to combine the complementary insights from these two models. Rigorous experimental validation across four diverse medical imaging
datasets (Breast Ultrasound Images (BUSI), International Skin Imaging Collaboration (ISIC) 2018, Kvasir, and COVID) demonstrates the superior performance and robustness of our ensemble approach. The MAE pre-training significantly improves feature learning on domain-specific data, while the ImageNet pre-training provides strong generalizable features. The ensemble consistently achieves state-of-the-art results, outperforming individual models and existing methods, highlighting the efficacy of combining diverse pre-training strategies for challenging medical image analysis.
Read the full abstract ▾
Computer Vision
Dual-Integrated Low-Latency Single-Lens Infrared Computational Imaging for Object Detection
Xuquan Wang, Guishuo Yang, Dapeng Yan et al. (+5 more)
Computational imaging enables compact infrared systems, but deep-learning pipelines that combine image reconstruction and object detection often introduce substantial inference latency. Most existing acceleration strategies compress the reconstruction network while overlooking physical priors from the optical path, leaving a trade-off between accuracy and speed. We present Physics-aware Dual-Integrated Network (PDI-Net), a low-latency framework that integrates infrared reconstruction with object detection and further embeds optical priors into the learning process. PDI-Net uses a supervised U-Net during training, while a semi-U-Net encoder shares features directly with a YOLO-based detector during inference, avoiding full image reconstruction. To bridge the gap between fidelity-oriented reconstruction features and detection-oriented semantics, we introduce a physics-aware large-small bridge (PALS-Bridge), which uses field-dependent point spread function priors to adaptively modulate multiscale convolutional branches. A physics-informed optical degradation simulation pipeline is also developed for training and validation. The method is deployed on a single-lens infrared camera, reducing system weight by about 50% compared with traditional multi-lens designs. On the M3FD
benchmark under low-SNR conditions, PDI-Net reduces inference time by 84.06% compared with the Rec+Det with
pruning strategy while improving mAP@0.5:0.95 by 5.07%. These results demonstrate compact, low-latency computational infrared imaging for real-time object detection on resource-constrained platforms.
Read the full abstract ▾
All authors Xuquan Wang, Guishuo Yang, Dapeng Yan, Yujie Xing, Xuanyu Qian, Kai Zhang, Xiong Dun, Jiande Sun
Computer Vision
Bounding-Box Trajectories Matter for Video Anomaly Detection
Inpyo Song and Jangwon Lee
Video anomaly detection is critical for public safety and security, yet remains highly challenging despite extensive research due to large variations in appearance, viewpoint, and scene dynamics. Among existing approaches, human pose-based methods have emerged as a major line of research, showing strong performance since many anomalies in public
datasets involve humans and pose representations are robust to appearance changes while providing compact motion descriptions. However, these methods often overlook bounding-box trajectories, although such information is inherently available in pose-based pipelines. In this paper, we explicitly leverage these trajectories as a primary anomaly cue. We present TrajVAD, a framework that models multi-class bounding-box trajectories using normalizing flows to learn normal kinematic patterns. Its trajectory-only variant, TrajVAD-T, eliminates pose estimation, reaches 87.7 AP on ShanghaiTech, and achieves the best results on MSAD among compared methods. TrajVAD-P adds a reliability-gated pose branch and improves performance to 88.6 AUROC and 90.9 AP on ShanghaiTech, establishing bounding-box trajectories as an effective modality for
video anomaly detection.
Read the full abstract ▾
Computer Vision
MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues
Dazhao Du, Liao Duan, Jian Liu et al. (+5 more)
Video temporal grounding (VTG), which localizes the start and end times of a queried event in an untrimmed
video, is a key test of whether
multimodal large
language models (MLLMs) understand not only what happens but also when it happens. Although modern MLLMs describe
video content fluently, their timestamp predictions remain unreliable, while existing remedies either require costly post-training on temporal annotations or rely on coarse training-free heuristics. In this work, we probe the cross-modal
attention of MLLMs and uncover a perception-generation gap. Our key finding is that MLLMs often know the target interval during prefill, but lose this signal when generating the final answer. In the prefill stage, a sparse set of
attention heads, which we call Temporal Grounding Heads (TG-Heads), concentrates query-to-
video attention on the ground-truth interval. During autoregressive decoding, however, the answer
tokens shift
attention away from this interval toward visually salient but query-irrelevant segments. This observation motivates an inference-time read-then-regenerate framework. We first convert TG-Head prefill
attention into a debiased frame-level relevance signal and extract the high-
attention interval it highlights. We then re-invoke the MLLM with visual context restricted to this interval using
video cropping. Without parameter updates or architectural changes, our framework consistently improves
MiMo-VL, Qwen3-VL, and Molmo2 on three
benchmarks, with gains of up to +3.5 mIoU. The project website can be found at https://ddz16.github.io/mllmsknowwhen.github.io/.
Read the full abstract ▾
All authors Dazhao Du, Liao Duan, Jian Liu, Tao Han, Yujia Zhang, Eric Liu, Xi Chen, Song Guo
Computer Vision
EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models
Shiqi Huang, Ziyue Wang, Zhongrong Zuo et al. (+3 more)
Recent
Video Large
Language Models (
Video-LLMs) have demonstrated strong capabilities in
video reasoning through reinforcement learning (RL). However, existing RL pipelines rely heavily on human-annotated tasks and solutions, making them costly to scale and fundamentally constrained by human expertise. Self-evolving frameworks have recently emerged as a promising alternative through autonomous Questioner-Solver self-play. Unfortunately, these approaches are primarily designed for static modalities such as text and images, fundamentally failing to capture the temporal dynamics that are central to
video reasoning. In this work, we propose $\textbf{EvoVid}$, a temporal-centric self-evolving framework that enables
Video-LLMs to improve directly from raw, unannotated
videos. Specifically, we introduce two complementary temporal-centric rewards: a temporal-aware Questioner reward that encourages temporally dependent question generation through temporal perturbation sensitivity, and a temporal-grounded Solver reward that provides automatic temporal supervision via inherent
video segment localization. Extensive experiments across four base models and six
benchmarks demonstrate consistent improvements over both base models and existing self-evolving baselines, achieving competitive performance with supervised methods. These results highlight temporal-centric self-
evolution as an effective and scalable paradigm for
video understanding and
reasoning.
Read the full abstract ▾
All authors Shiqi Huang, Ziyue Wang, Zhongrong Zuo, Han Qiu, Qi She, Bihan Wen
Computer Vision
Ruiqi Liu, Xiaolei Lv, Gengsheng Li et al. (+8 more)
On-policy knowledge
distillation has proven effective for
language models, yet its application to
vision-
language models (VLMs) remains underexplored. We observe that standard on-policy
distillation can improve a student's output quality while failing to strengthen its reliance on visual input: on
vision-critical
tokens, the student's predictions remain largely unchanged whether or not fine-grained visual detail is present, even though the teacher's predictions depend heavily on it.To make this difference observable, we introduce visual advantage (VA), the
token-level log-probability difference when the teacher scores a student-generated rollout with versus without access to fine-grained visual detail. VA is concentrated in a small minority of
tokens, and these high-VA
tokens are the ones that actually carry the visual supervision signal. This motivates a
distillation objective that treats them differently from
language scaffolding, so their contribution is not diluted by the abundant surrounding
language tokens.We propose Visual-Advantage On-Policy
Distillation (VA-OPD), which uses VA at two granularities: rollout-level reweighting by trajectory-averaged VA, and
token-level KL averaged within high-VA and low-VA groups separately. We train on two math
datasets (Geometry3K and ViRL39K) and evaluate on eight
benchmarks covering both mathematical
reasoning and visual understanding, across three teacher sizes (4B, 8B, and 32B) on the Qwen3-VL family. VA-OPD improves over standard on-policy
distillation on every
benchmark, with the gain growing monotonically along both the teacher-size and data-scale axes, suggesting that these factors compound consistently.
Read the full abstract ▾
All authors Ruiqi Liu, Xiaolei Lv, Gengsheng Li, Ximo Zhu, Zhiheng Wang, Zhengbo Zhang, Junkai Chen, Zhiheng Li, Bo Li, Jun Gao, Shu Wu
Computer Vision
SDGBiasBench: Benchmarking and Mitigating Vision–Language Models' Biases in Sustainable Development Goals
Zihang Lin, Huaiyuan Qin, Muli Yang et al. (+1 more)
Assessing progress toward the Sustainable Development Goals (SDGs) requires multi-step
reasoning over visual cues, contextual knowledge, and development indicators, where incomplete evidence use and imperfect evidence integration can introduce hidden prediction biases. Real-world SDG monitoring further spans both qualitative judgments and quantitative estimation. However, existing
benchmarks typically evaluate these aspects in isolation, obscuring systematic biases that emerge when models substitute priors for evidence. To address this gap, we propose SDGBiasBench, a large-scale
benchmark suite for SDG-oriented
vision-
language reasoning. Spanning 500k expert-involved multiple-choice questions and 50k regression tasks, the
benchmark enables comprehensive assessment of both decision-level and estimation-level bias in
Vision–
Language Models (VLMs). Evaluations on SDGBiasBench reveal an intrinsic SDG bias in current VLMs, where predictions are frequently driven by SDG specific priors rather than reliable multi-modal cues. To mitigate such bias, we propose CADE (Contrastive Adaptive Debias Ensemble), a training-free, plug-and-play method that leverages modality-specific answer priors. CADE yields significant gains on the proposed
benchmark, improving multiple-choice accuracy by up to 25% and reducing regression MAE by up to 12 points across multiple VLMs. We hope our work can foster the development of more fair and reliable AI systems for sustainable development.
Read the full abstract ▾
All authors Zihang Lin, Huaiyuan Qin, Muli Yang, Hongyuan Zhu
Computer Vision
MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks
Han Zhang, Wanting Jiang, Tomasz Kornuta et al. (+2 more)
Training
Vision Language Models (VLMs) for
video event
reasoning requires high-quality structured annotations capturing not only what happened, but when, where, why, and with what consequence, at a scale manual labelling cannot support. We present MAVEN (Multi-stage Agentic
Video Event aNnotation), a multi-stage agentic pipeline that turns raw
videos into multi-task training data with Chain-of-Thought (CoT)
reasoning traces, organized around a designated Event of Focus. At its core, MAVEN synthesizes a Multi-Scale Spatio-Temporal Event Description (MSTED) from three complementary caption levels; this explicit intermediate serves as the sole input to downstream Q&A generation across multiple task formats. Crucially, MAVEN supports
agent-driven domain adaptation: given a new
video dataset and target question examples, the
agent redesigns all
prompts top-down without manual re-engineering. A hierarchical refinement loop further classifies annotation errors against a taxonomy, traces root causes to the originating pipeline stage, and applies targeted edits that rewrite
prompts or modify the pipeline structure itself, iteratively improving data quality. We apply MAVEN to label over 5,300 traffic
videos and fine-tune Cosmos-Reason2-8B on the resulting data. On a private CCTV evaluation set, fine-tuning surpasses both Gemini 2.5 Pro and 3.1 Flash, including a $+38.8$-point gain in MCQ accuracy over zero-shot. On AccidentBench, CCTV-only training lifts Cosmos-Reason2 by $+10.7$ MCQ points and matches Gemini 2.5 Pro despite seeing no dashcam
videos; adding
agent-adapted dashcam annotations narrows the gap to Gemini 3.1 Flash, and RL post-training pushes overall performance past both Gemini baselines. Qualitative results on warehouse surveillance and public safety
videos further show the agentic workflow readily adapts the pipeline to new domains.
Read the full abstract ▾
All authors Han Zhang, Wanting Jiang, Tomasz Kornuta, Tian Zheng, Vidya Murali
Computer Vision
Multi-scale interaction network for stereo image super-resolution
Liyi Xu and Lin Qi
Stereo image super-resolution aims to generate high-resolution images by leveraging complementary information from binocular systems. Although previous studies have achieved impressive results, the potential of intra-view and cross-view information has not been fully exploited. To address this issue, we propose a novel multi-scale interaction network for stereo image super-resolution. Specifically, we design a Multi-scale Spatial-Channel
Attention Module that utilizes multi-scale large separable kernel
attention and simple channel
attention to improve intra-view feature extraction. Additionally, we propose a Dual-View Epipolar
Attention Module, utilizing an optimal transport
algorithm to achieve more accurate matching along the epipolar line. Extensive experimental and ablation studies show that our method achieves competitive results that outperform most SOTA methods.
Read the full abstract ▾
Computer Vision
Guided Trajectory Optimization with Sparse Scaling for Test-Time Diffusion
Gang Dai, Yining Huang, Yiming Xia et al. (+2 more)
Test-Time Scaling (TTS) paradigm offers a promising perspective for enhancing the generation performance of
diffusion models. However, current solutions largely restrict their search to predefined noise candidates or suffer from inflexible exploration across the denoising trajectory. To bridge this gap, we propose RTS, a novel Reward-guided Trajectory Scaling method to fully unlock the generative potential of
diffusion models. Unlike existing methods, RTS facilitates the synthesis of refined, high-fidelity images via two core innovations: 1) a coarse-to-fine noise optimization mechanism that exploits historical search experience to actively steer the exploration toward high-reward regions and 2) a unified sparse test-time scaling framework featuring PCA-driven
curvature analysis, which eliminates temporal redundancy by flexiblely allocating compute to a sparse set of key timesteps that represent critical shifts in the denoising direction. Extensive experiments across SD v3, FLUX, and Qwen-Image architectures demonstrate that RTS outperforms baselines, improving the GenEval score by 20.7%, 15.6%, and 12.2%, respectively. Notably, empirical findings indicate that these key points primarily cluster in the mid-stage of the trajectory, distinct from the structure-sensitive early phases and the late attribute refinement phases.
Read the full abstract ▾
All authors Gang Dai, Yining Huang, Yiming Xia, Guohao Chen, Shuaicheng Niu
Computer Vision
Universal CT Representations from Anatomy to Disease Phenotype through Agglomerative Pretraining
Yuheng Li, Yuan Gao, Haoyu Dong et al. (+5 more)
Computed
tomography (CT) is a central to three-dimensional medical imaging, yet CT-based artificial intelligence remains fragmented across task-specific models for
segmentation, classification, registration, and report analysis. Here we present FlexiCT, a family of CT foundation models trained by agglomerative continual pretraining on 266,227 CT volumes from 56 publicly available
datasets, forming a large-scale public resource for CT representation learning. FlexiCT uses agglomerative pretraining across three stages: two-dimensional axial pretraining, three-dimensional anatomical pretraining and report-guided semantic
alignment. This training strategy supports slice-level, volume-level and
vision-
language analysis. Across five downstream task families (
segmentation, classification, registration,
vision-
language understanding and clinical
retrieval), FlexiCT matches or exceeds prior task-specific approaches on multiple
benchmarks. Its
embeddings further organize CT scans along gradients associated with various tumor stages, suggesting that CT foundation models can capture imaging features relevant to disease phenotype characterization. Project page and code are available at: https://ricklisz.github.io/flexict.github.io and https://github.com/ricklisz/FlexiCT.
Read the full abstract ▾
All authors Yuheng Li, Yuan Gao, Haoyu Dong, Yuxiang Lai, Shansong Wang, Mojtaba Safari, James E. Baciak, Xiaofeng Yang
Computer Vision
Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception
Rusiru Thushara, Yasiru Ranasinghe, Jay Paranjape et al. (+1 more)
Vision-
language models (VLMs) often fail under low illumination because their visual grounding is learned predominantly from RGB imagery, whereas thermal infrared preserves complementary scene structure when visible cues degrade. We present Thermo-VL, a wavelength-aware VLM that augments a frozen Molmo-7B backbone with a trainable thermal encoder and a text-guided dual-
attention fusion module. Given aligned RGB
tokens, thermal
tokens, and
prompt embeddings, the fusion module conditions thermal features on both
language and RGB context, then injects a gated residual into the frozen RGB stream so thermal evidence can be incorporated without disrupting Molmo's pretrained RGB-
language interface. We train the model with the standard
language-modeling objective together with auxiliary
alignment and regularization losses that improve cross-modal grounding and reduce over-reliance on RGB. We also introduce a pixel-aligned RGB-thermal instruction-tuning
dataset and Thermo-VL-Bench, a manually screened RGB-thermal VQA
benchmark for low-light and cross-spectrum
reasoning. Experiments show strong gains on challenging thermal-only and RGB+thermal
reasoning tasks, highlighting the value of
prompt-conditioned multispectral fusion. Our
dataset and code are publicly available at: https://thusharakart.github.io/Thermo-VL
Read the full abstract ▾
All authors Rusiru Thushara, Yasiru Ranasinghe, Jay Paranjape, Vishal M. Patel
Computer Vision
Two-Stage Multimodal Framework for Emotion Mimicry Intensity Prediction
Dinithi Dissanayake, Shaveen Silva, Ovindu Atukorala et al. (+2 more)
We present our submission to the Hume-ABAW10 Emotional Mimicry Intensity (EMI) Challenge, which aims to predict six continuous
emotion intensity dimensions: Admiration, Amusement, Determination, Empathic Pain, Excitement, and Joy, from in-the-wild
multimodal video clips. We propose a staged
multimodal framework that combines textual, acoustic, and visual representations, with an optional motion branch. Our approach first trains modality-specific encoders independently and then fuses their learned representations through a lightweight regressor with modality dropout and controlled encoder adaptation. Across our submitted systems, the best validation performance is obtained by the text–audio–
vision–motion fusion model under the expanded 4:1 split, achieving an average Pearson correlation of 0.4722. Although the motion branch yields only very slight gains, its behavior can be interesting to study. Our team was placed third in the EMI challenge, achieving an average Pearson correlation of 0.57 for the test set. Overall, we provide a practical and reproducible baseline for EMI prediction.
Read the full abstract ▾
All authors Dinithi Dissanayake, Shaveen Silva, Ovindu Atukorala, Prasanth Sasikumar, Suranga Nanayakkara
Computer Vision
Learning Emergent Modular Representations in Multi-modality Medical Vision Foundation Models
Yuting He, Chenyu You and Shuo Li
Multi-modality medical
vision (MV) foundation models (FM) are fundamentally challenged by pronounced Non-IID feature statistics across heterogeneous imaging modalities. Monolithic self-supervised optimization on such data induces conflicting gradients, driving representations to collapse toward modality-dominant shortcuts. This work reframes this failure as an imbalance between specialization and coordination in emergent modularity, and proposes Director-Experts (DEX), a modular network that explicitly regulates these dynamics in stacked modules. Each DEX module comprises a pool of experts, dynamically adapted by our image-wise activation strategy, autonomously specializing in modality-dominant statistics, together with a director, updated via our group exponential moving average, which distills multi-expert knowledge into a shared space for semantic integration across modalities, thus driving the emergence of modular representations. We curate a new
benchmark, Medical
Vision Universe, over 4 million images across 10 modalities, which provides a FM-level pre-training with the broadest coverage of distinct imaging modalities to our DEX. Extensive evaluations on 26 downstream tasks demonstrate improved optimization behavior and transferability, indicating DEX as a principled step toward general-purpose multi-modality medical AI. Our code and
dataset will be opened at https://github.com/YutingHe-list/DEX.
Read the full abstract ▾
Computer Vision
CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models
Zhi Liu
Vision-
Language-Action (VLA) models have rapidly converged on a small set of architectural patterns: discrete-
token autoregression (e.g. OpenVLA) and continuous-action flow-matching (e.g. pi-0.5). Yet preference
alignment via Direct Preference Optimisation (DPO) – the de-facto post-training step in
language models – has been studied almost exclusively on autoregressive VLAs. We present CrossVLA, an empirical study of cross-paradigm VLA post-training. Three contributions: (i) a surrogate flow-matching log-probability
estimator that lets DPO operate on continuous-action backbones without probability-flow ODE integration; (ii) a head-to-head comparison of LoRA and DoRA as the parameter-efficient layer for VLA DPO, finding DoRA improves over OpenVLA SFT by a mean +10.4 pp across LIBERO 4-suite (600 trials, 3 seeds) – per-suite +20.0 Object, +11.0 Long-horizon, +8.0 Goal, +2.7 Spatial – with zero seed variance on Object (38/50 on each of 3 seeds); (iii) an inference-time anatomy showing the denoise loop dominates 78.6% of sample_actions latency and prefix-K/V caching a la VLA-Cache caps at a 21% acceleration ceiling – both chunk-level and
token-level cache strategies degrade success rate to 0-80% in our
benchmarks. We further pretrain a multi-view + temporal projection head on 6000 LIBERO frames, achieving 99.5% k-NN recall@1 for same-task
retrieval (36x over random), available as a downstream initialisation. All code, ckpts, training logs, and reproduction scripts are open at https://github.com/lz-googlefycy/vla-lab.
Read the full abstract ▾
Computer Vision
Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding
Lina Zhang, Tonmoy Monsoor, Peizheng Li et al. (+23 more)
While
Multimodal Large
Language Models (MLLMs) have demonstrated remarkable proficiency in general
video understanding, their capacity to interpret involuntary, and spatio-temporally evolving pathologic motor behaviors such as seizure semiology remains largely untested. To address this gap, we introduce Seizure-Semiology-Suite, a clinically grounded
dataset and
benchmark for fine-grained, structured seizure semiology understanding. The
dataset includes 438 seizure
videos annotated with over 35,000 dense labels covering 20 ILAE-defined semiological features. Building on this
dataset, we propose a seven-task hierarchical
benchmark that systematically evaluates MLLMs from low-level visual perception to temporal sequencing, narrative report generation, and seizure diagnosis. To enable clinically meaningful evaluation of generated reports, we further introduce the Report Quality Index for Seizure Semiology (Seizure-RQI). Extensive baselines across 11 open-weight MLLMs reveal systematic weaknesses in laterality
reasoning, temporal localization, symptom sequencing, and clinically faithful reporting. We show that seizure-specific fine-tuning substantially improves performance across tasks, and that a two-stage neuro-symbolic framework achieves an F1 score of 0.96 on epileptic versus non-epileptic seizure classification. Seizure-Semiology-Suite establishes a rigorous
benchmark for evaluating
multimodal models in safety-critical medical
video understanding and guides the development of clinically reliable, domain-adaptive
multimodal intelligence.
Read the full abstract ▾
All authors Lina Zhang, Tonmoy Monsoor, Peizheng Li, Jiarui Cui, Xinyi Peng, Chong Han, Prateik Sinha, Siyuan Dai, Jessica Nichole Pasqua, Colin M McCrimmon, Weiting Liu, Hailey Marie Miranda, Bing Hu, Xiangting Wu, Tengyou Xu, Chunhan Li, Jiaye Tian, Jiarui Tang, Detao Ma, Lingye Kong, Junnan Lyu, Jungang Li, Yan Zan, Junhua Huang, Rajarshi Mazumder, Vwani Roychowdhury