Computer Vision
Simple Domain Generalization for Strong Pixel-Level Image Tampering Detection in Modern VLMs
Yi Tang, Xinyi Shang, Jiacheng Cui et al. (+12 more)
Modern
vision-
language models (VLMs) have significantly improved image generation and editing capabilities, making pixel-level image tampering detection increasingly important yet challenging under cross-model and out-of-distribution shifts. This work studies domain generalization for pixel-level image tampering detection in modern VLMs like ChatGPT, Gemini, Qwen-Image, etc., aiming to learn tampering localization models that remain robust across diverse VLM-generated manipulation distributions. We propose a simple yet effective domain-generalized training framework built on two practical strategies. First, we introduce a balanced minibatch sampling scheme that strategically samples tampered and real images in each minibatch, preventing biased optimization toward either manipulated artifacts or clean-image priors and avoiding training collapse, ensuring that each optimization step receives proper sampled gradient signals. Second, we adopt a simple late-injection strategy, where the detector is first trained on large-scale base data until stable convergence, and then exposed to a small amount of newly selected supporting data from emerging VLM distributions, improving adaptability without overfitting to limited new domains. Together, these components provide a simple yet strong recipe for improving pixel-level tampering localization and OOD robustness across modern VLMs. Despite the conceptual simplicity, our framework outperforms the prior state-of-the-art PIXAR by a large margin of 26.1% and 26.8% relative improvement in average gIoU and cIoU, respectively, across OOD VLMs of GPT-Images-2.0, Gemini-3.1, FLUX.2, and Seedream 4.5. Our code is available at https://github.com/VILA-Lab/PIXAR-DG
Read the full abstract ▾
All authors Yi Tang, Xinyi Shang, Jiacheng Cui, Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao, Tran Dinh Tien, Ahmed Elhagry, Salwa K. Al Khatib, Tianjun Yao, Yonina C. Eldar, Jing-Hao Xue, Hao Li, Salman Khan, Zhiqiang Shen
Computer Vision
FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry
Dingyun Zhang, Lixue Gong and Wei Liu
In line with the prevailing direction of
vision research, we explore the integration of both generation and editing capabilities for
video and image modalities within a single model. Current approaches to collecting
video editing data typically depend on labour-intensive, time-consuming curated procedures–involving object mask annotation, the use of error-introducing pair synthesis via I2V model and ControlNet-like guidance, and VLM-based quality filtering or refinement–and demonstrate limited task scalability. As a result, the diversity of editing tasks remains substantially narrower than that available for image editing models. We develop a pixel-pair temporal warped flow field that can directly generate corresponding
video editing samples in real time from image editing samples, and we demonstrate across multiple levels of
video editing tasks that a model can learn
video editing using only such data. We regard the image modality as a particular form of the
video modality. Accordingly, we design a modality mimic generation loss and a modality mimic editing loss to relatively align the capabilities–and thereby the output distributions–of the two modalities through mutual imitation. Moreover,
language-based visual editing entails the comprehension of the editing instruction and the reference visual content, the localization of the region corresponding to that instruction within the reference visual contents, and the modification of that region alone. Existing approaches predominantly rely on external aids, such as fine-tuning an additional MLLM or explicitly supplying a mask sequence as auxiliary input during inference. In contrast, we aspire for the model to internalize this capability. To that end, we introduce sense-related tasks–for instance, referring expression
segmentation–along with corresponding editing-region-aware latent-level loss and
attention-level loss.
Read the full abstract ▾
Computer Vision
GigaPath-Flash and GigaTIME-Flash: Efficient Pathology Foundation Models for Whole-Slide and Tumor Microenvironment Analysis
Naoto Usuyama, Jeya Maria Jose Valanarasu, Sicong Yao et al. (+27 more)
Foundation models have emerged as a driving force in computational pathology, with the potential to transform cancer diagnosis, prognosis, and treatment selection by learning transferable representations from large-scale histopathology data. A growing landscape of pathology foundation models now spans diverse data sources, architectures, and downstream applications. However, most pretrained models operate only at the image-tile level, use restrictive licenses, and remain computationally expensive, limiting large-scale slide-level clinical and research use. Here, we introduce GigaPath-Flash and GigaTIME-Flash, efficient models for whole-slide pathology AI and spatial proteomics prediction. GigaPath-Flash combines a 22M-parameter ViT-S tile encoder with a 21M-parameter LongNet slide encoder, both pretrained on large-scale real-world histopathology data. Its compact tile encoder is distilled from the billion-parameter GigaPath (ViT-g) teacher and shared by both models. GigaPath-Flash retains 97% of GigaPath's average slide-level performance with 50x less compute. GigaTIME-Flash extends this backbone to predict the tumor immune microenvironment directly from routine H&E images. It surpasses the original CNN-based GigaTIME in prediction quality while running 6x faster and using 8x less GPU
memory. Together with GigaPath and GigaTIME, these models form an open-weight, Apache-2.0-licensed family pretrained on large-scale real-world clinical data. By releasing all models and weights, we provide accessible building blocks for computational pathology, immuno-oncology, and precision health.
Read the full abstract ▾
All authors Naoto Usuyama, Jeya Maria Jose Valanarasu, Sicong Yao, Hanwen Xu, Jaspreet Bagga, Guanghui Qin, Robert E. Kramer, Cliff Wong, Soohee Lee, Hao Qiu, Theodore Zhengde Zhao, Racheli Ben Shimol, Angela Crabtree, Kevin Matlock, Eduardo Alejandro Lozano Garcia, Naiteek Sangani, Alberto Santamaria-Pang, Maximilian Rokuss, Yashna Hasija, Naisargi Manishkumar Patel, Jason Entenmann, Alexandra Q. Bartlett, Bill J. Wright, Bernard A. Fox, Brian Piening, Sheng Zhang, Sheng Wang, Tristan Naumann, Carlo Bifulco, Hoifung Poon
Computer Vision
HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement
Yiyang Cai, Nan Chen, Rongchang Xie et al. (+8 more)
Human-object centric
video personalization (HOCVP) is a core task within subject-driven
video generation. However, existing methods suffer from two key limitations. First, most approaches focusing on inter-subject personalization still struggle to strike a balance between high subject fidelity and accurate interaction patterns between humans and diverse objects, especially when objects represent abstract concepts such as logos. Second, while intra-subject references (e.g., OCR maps, multi-view inputs) are expected to enhance subject fidelity, most existing works lack mechanisms to understand such latent correspondence. To address both challenges, we propose HOMIE, an HOCVP framework that tackles both inter- and intra-subject input settings in a unified manner. Compared to previous approaches, HOMIE proposes a better MLLM integration strategy to extract knowledge of reference-level relationships without compromising the controllability of text encoders or incurring costly re-
alignment. Specifically, we introduce global
multimodal guidance within self-
attention to better align MLLM-derived semantic features with VAE
tokens. Furthermore, we propose modality-reference
embedding to differentiate
tokens from MLLM features and VAE
tokens and associate intra-subject reference image
tokens. Extensive experiments validate that our method achieves state-of-the-art performance across various HOCVP tasks. Project Page: https://yiyangcai.github.io/homie-page.github.io/
Read the full abstract ▾
All authors Yiyang Cai, Nan Chen, Rongchang Xie, Junwen Pan, Chunyang Jiang, Cheng Chen, Wen Zhou, Zhenbang Sun, Wei Xue, Wenhan Luo, Yike Guo
Computer Vision · Cross-listed from cs.LG
Three-Body Scattering for Generative Modeling
Peng Sun, Zhenglin Cheng, Deyuan Liu et al. (+3 more)
Modern generative models typically rely on an
adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional energy can induce sample-level motion and provide direct regression supervision for a one-step generator. Three-Body Scattering Modeling (TBSM) for generation turns the energy distance into a constant-size per-projectile interaction: each projectile is attracted toward one real source and repelled from one independently generated source. Conditioned on the projectile and its condition, its expectation equals the $2$-Wasserstein gradient-flow velocity of $\frac12D_E^2(P_{\theta},Q)$. A batch of $B$ frozen-target events yields $O(B)$ sample-level losses, each using one reference for its condition instead of the minibatch-wide all-pairs field used by methods such as Drifting Models. Tracking this conditional expectation online can reduce field noise. Using scattering in frozen image features, TBSM trains one-step generators on ImageNet-256, achieving FID${}=2.23$ with pixel-space PixelDiT-XL and FID${}=1.63$ with latent-space DiT-XL at NFE${}=1$. We provide a design map relating
diffusion-related supervision, Drift-like dynamics, and GAN-like objectives. These results establish tracked scattering as a route to high-dimensional one-step generation. Code: https://github.com/sp12138/TBSM.
Read the full abstract ▾
All authors Peng Sun, Zhenglin Cheng, Deyuan Liu, Jun Xie, Xinyi Shang, Tao Lin
Computer Vision
Certified Training for Convolutional Perturbations
Benedikt Brückner and Alessio Lomuscio
Vision models have been found to be susceptible to perturbations such as motion blur induced at runtime by a shaking camera. This impedes their deployment in critical applications since phenomena such as slightly blurred
vision might lead to failures, for example an object detector missing objects. While methods such as data augmentation or
Adversarial Training can improve empirical robustness, they lack formal safety guarantees, making it difficult to identify and mitigate hidden vulnerabilities. We introduce a novel Certified Training approach that leverages an efficient encoding of convolutional perturbations to train provably robust models. Our method significantly outperforms
Adversarial Training, achieving, for example, over 80% robust accuracy against motion blur of reasonable intensity on CIFAR10 while maintaining comparable standard accuracy.
Read the full abstract ▾
Computer Vision
Zhe Xin, Hanzhi Chang, Penghui Huang et al. (+2 more)
Dynamic object
segmentation plays a critical role in many visual applications such as static scene reconstruction from dynamic
videos. However, existing optical flow-based methods fail to ensure consistent static/dynamic
segmentation along object boundaries, while 3D reconstruction-based approaches are highly sensitive to reconstruction errors. To address these limitations, we present a dynamic object
segmentation framework that can generate both precise and complete dynamic masks by integrating
multimodal cues including 2D point tracks, 3D reconstruction, and semantic information. We design a network combining
Transformer architectures with feature clustering aggregation modules to perform static/dynamic classification of
multimodal feature trajectories. It enables the model to adaptively determine which type of feature should dominate based on the characteristics of each scene, while also mitigating the impact of feature degradation. Additionally, we introduce a novel point-query-based SAM post-processing method capable of handling multiple objects within a single mask. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in both dynamic object
segmentation and static scene reconstruction tasks.
Read the full abstract ▾
All authors Zhe Xin, Hanzhi Chang, Penghui Huang, Yinian Mao, Guoquan Huang
Computer Vision
Plenoptic Condensation: A Novel Approach to Generalized Scene Reconstruction
Brevin Tilmon, Alex DeJournett, John Leffingwell et al. (+1 more)
We present a novel Generalized Scene Reconstruction (GSR) approach called Plenoptic Condensation (PCon). PCon uses a multi-stage reconstruction pipeline, initially converting images into "soupy" scene elements with low (representational) power, then adaptively condensing the "soup" into "structured" elements of higher power capable of efficiently representing, for example, sharp edges and smooth reflective surfaces. PCon scene models called Reality Models (Relms) enable spatially varying representational power, which is essential for high-fidelity rendering, measurement, and scene understanding. We showcase several in-the-wild PCon reconstructions captured with consumer phone cameras and drones. In one case called "Damaged Fiat", PCon is benchmarked against two state-of-the-art (SOTA) GSR methods: NeRO and RT-Splatting. Referring to Figure 1 below, PCon reconstructs the car hood more than twice as accurately as the SOTA methods. But more importantly, the local damage profile error for PCon is 35 um (0.035 mm), whereas the two other SOTA methods are essentially unable to measure the damage at all. Our project website is available at https://quidient.github.io/pcon-2026.html.
Read the full abstract ▾
All authors Brevin Tilmon, Alex DeJournett, John Leffingwell, Scott Ackerson
Computer Vision
Lossless-INR: Lossless Volumetric Implicit Neural Representations
Kaiyuan Tang, Daniel Burke and Chaoli Wang
Implicit neural representation (INR) methods provide continuous coordinate-to-value mappings and integrate naturally with direct volume rendering, making them attractive for representing volumetric data. However, existing INR-based approaches for volumetric data are inherently lossy, and even small reconstruction errors can propagate through rendering and downstream analysis. In this work, we explore Lossless-INR, a lossless INR framework for 3D scientific volumetric data based on bit-plane decomposition. By decomposing each voxel value into binary bit-planes, we reformulate reconstruction as per-bit binary classification, so that exact recovery reduces to predicting every bit correctly. To make this optimization tractable while keeping the representation compact, we combine an octree block-partitioning strategy that adaptively subdivides complex regions with a ternary feature-grid network whose grid entries are parameterized by a ternary set of values. Experiments on diverse volumetric
datasets show that this design can achieve zero bit-error rate and bit-exact reconstruction, enabling faithful rendering and downstream analysis with a compact representation. The code is available at https://github.com/TouKaienn/Lossless-INR.
Read the full abstract ▾
Computer Vision
O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning
Mei Yuan, Qi Long, Qifeng Wu et al. (+5 more)
Industrial
Video Anomaly Detection (IVAD) aims to identify anomalous objects and events in an industrial process, which is crucial for modern manufacturing and quality control systems. Existing VLM-based anomaly
reasoning methods are capable of detecting open-ended anomalies in general domains. However, their performance declines in industrial settings characterized by intricate object transformations, strict physics, and procedural constraints. To tackle the complexity of such interaction-intensive detection, we introduce a training-free agentic framework for anomaly detection free of domain-specific knowledge, emphasizing object state
evolution like humans inspectors. It is designed to track spatial-temporal dynamics and underlying transformations of detected objects over time, and then reason over the object-wise temporal state trajectories to identify abnormal objects in grounded frames. Our method overcomes limitations of prior approaches that rely on retraining on normal clips or injecting domain knowledge as context for test-time inference. Extensive experiments on three IVAD
datasets demonstrate that our method outperforms frontier VLMs, agentic frameworks, and traditional VAD methods fine-tuned on the respective
datasets, while providing interpretable reports over anomaly processes and types.
Read the full abstract ▾
All authors Mei Yuan, Qi Long, Qifeng Wu, Zhenyang Li, Yizhou Zhao, Lei Wang, Yang Liu, Min Xu
Computer Vision · Cross-listed from cs.AI
SGA: Plug&Play Geometric Verification for Educational Video Synthesis
Jhon Lopez, Carlos Hinojosa and Bernard Ghanem
Recent work leverages Large
Language Models (LLMs) to generate executable code for pedagogical animations using libraries such as Manim. However, ensuring spatial correctness and visual legibility remains challenging, as existing frameworks emphasize pedagogical content while overlooking
geometric occlusions. We propose the Symbolic
Geometric Agent (SGA), a plug-and-play module for code-centric animation pipelines that intercepts LLM-generated code, performs partial execution to extract symbolic scene graphs, and applies targeted refinement when spatial conflicts are detected. We further introduce the Manim Visual Quality Score (MVQS), a deterministic rendering-free proxy for spatial integrity. Experiments on the MMMC-Code
benchmark across four LLM backbones and two agentic pipelines show that SGA achieves a peak MVQS of 73.11 (Code2
Video + GPT-5.1), corresponding to a 16.1% relative improvement over the raw baseline, and improves MVQS in 7 of 8 backbone x pipeline configurations. Additionally, we conduct a human evaluation showing that these improvements translate into human preference, with raters preferring SGA over the raw baseline in 84.4% of comparisons and over a VLM-based critic in 65.0%.
Read the full abstract ▾
Computer Vision
Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention
Wenbo Wei, Jun Wang, Shan Raza et al. (+1 more)
Panoptic
segmentation in complex scenes remains challenging because of occlusions, yet modern approaches often neglect occlusion modelling. In this paper, we propose Position
Embedding Modulation with Occlusion Level
Attention (PEMOLA), a novel occlusion-aware module that can be seamlessly integrated into
transformer-based panoptic
segmentation. To obtain occlusion cues, we train an occlusion classifier on the COCO-OLAC
dataset. The classifier derives the occlusion-level
attention, which serves as spatial guidance, while the occlusion labels are encoded into a learnable
embedding to produce channel-wise weights. Through joint modulation, PEMOLA elegantly introduces the occlusion priors into the position
embedding, thereby improving the occlusion modelling. We further annotate the Cityscapes
dataset with occlusion levels, termed Cityscapes Occlusion Labels for All Computer
Vision Tasks (Cityscapes-OLAC), following the same labelling protocol as COCO-OLAC, to evaluate the cross-
dataset generalisation ability of PEMOLA. Extensive experiments on COCO-OLAC and Cityscapes-OLAC demonstrate that PEMOLA consistently improves panoptic
segmentation quality while introducing minimal computational overhead. These results highlight the importance of occlusion modelling, where incorporating occlusion-level
attention helps deliver robust panoptic
segmentation under occlusion. Code and
dataset are available at https://github.com/wenbo-wei/PEMOLA.
Read the full abstract ▾
All authors Wenbo Wei, Jun Wang, Shan Raza, Abhir Bhalerao
Computer Vision
Sania Bano, Shahzad Ahmad, Santosh Kumar Vipparthi et al. (+2 more)
Understanding human
emotions in spoken conversations is a key challenge in affective computing, with applications in empathetic AI, human computer interaction, and mental health monitoring. However, existing
datasets vary in scale,
emotion distribution, modality
alignment, and data partitioning strategies, which can influence reliable cross-
dataset generalization and minority-
emotion modeling. We introduce SpEmoC a Speaking segment
Emotion for Conversations comprising 306,544 raw clips from 3,100 English
language movies and TV series. From these, 30,000 high quality, class balanced clips are curated, featuring synchronized visual, audio, and textual modalities annotated for seven
emotions through a hybrid pipeline that integrates pretrained models with human validation. SpEmoC uses strict movie- and series-level splits to prevent content overlap between split sets, allowing more reliable evaluation of model generalization. The
dataset also maintains a near-balanced distribution across seven
emotions, including minority classes such as Fear and Disgust, which supports more balanced learning across categories. Extensive experiments, including in-domain benchmarking, cross-
dataset transfer, low-data training, class-imbalance analysis, and modality transfer show that balanced data and careful splitting lead to more stable performance across
emotions when models are evaluated on other
datasets. These results highlight the importance of
dataset design for robust and transferable
multimodal emotion recognition.
Read the full abstract ▾
All authors Sania Bano, Shahzad Ahmad, Santosh Kumar Vipparthi, Sukalpa Chanda, Subrahmanyam Murala
Computer Vision · Cross-listed from cs.LG
Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator
Mateusz Piechocki, Alessandro Capotondi and Marek Kraft
On-device model adaptation is essential to enable lifelong personalization on resource-constrained hardware, but compute, power, and
memory limitations of such devices make end-to-end backpropagation impractical for modern deep neural networks. This work proposes a heterogeneous adaptation pipeline that repurposes a commercial edge AI inference accelerator, Hailo-8L, for frozen-backbone feature extraction during on-device training. The computational graph is partitioned so that the pre-trained backbone is quantized to INT8 and run on the accelerator, while only a lightweight FP32 classification head is fine-tuned on the host CPU, enabling frequent, energy-efficient in-field updates with most weights remaining fixed. Across multiple architectures and
datasets, this pipeline achieves up to 15.4x faster wall-clock training time compared to a Raspberry Pi 5 CPU baseline, offers competitive throughput in favorable settings, and consistently reduces energy per sample. Post-training quantization restoration is shown to be crucial for preserving the quality of accelerator-generated features and mitigating accuracy loss in quantization-sensitive architectures. Overall, the results demonstrate a practical approach to efficient on-device adaptation using inference-oriented edge accelerators. The implementation is available at https://github.com/MatPiech/accelerator-training.
Read the full abstract ▾
Computer Vision
SciForma: Structure-Faithful Generation of Scientific Diagrams
Yuxuan Luo, Peng Zhang, Xinjie Zhang et al. (+3 more)
Structural fidelity is essential to scientific methodology diagrams. To communicate research logic, these diagrams must faithfully render components, directional relations, and textual annotations. Since a single error, such as a reversed arrow or an unreadable equation, can invalidate the entire figure, structural fidelity is inherently conjunctive: correctness on one axis cannot compensate for failure on another. Current open-source models fail to satisfy this criterion. Supervised fine-tuning (SFT) learns plausible layouts but cannot reliably ensure structural correctness, while scalar reward-based post-training obscures which structural dimension has failed. To address this, we introduce SciForma, a framework for the structure faithful generation of scientific methodology diagrams. Specifically, SciForma decomposes diagram quality into three structural axes: Component, Arrow, and Text, guided by a structural inventory. Built on this foundation, we curate SciFormaData-700K for structured training and SciFormaBench-2K for logic-verified evaluation. To close the gap left by SFT, we develop Multi-Dimensional Conjunctive Preference Optimization (M-DPO), which enforces simultaneous correctness across all axes and adaptively routes gradients to the most deficient dimension in post-training. The same structural inventory also enables iterative editing at inference time to correct residual errors. This combination allows SciForma-9B to exceed all open-source baselines and GPT-Image-1.5 on both SciFormaBench-2K and AIBench, bringing open scientific diagram generation close to proprietary-level structural fidelity. Our code and data will be available at: https://github.com/microsoft/SciForma.
Read the full abstract ▾
All authors Yuxuan Luo, Peng Zhang, Xinjie Zhang, Xun Guo, Zhouhui Lian, Yan Lu
Computer Vision · Cross-listed from cs.LG
The Label Complexity of Useful Class-Conditional Prediction Sets under Distribution Shift
Weijia Han, Lisha Qu, Tianxin Zhou et al. (+2 more)
Prediction sets can make deployed classifiers safer by returning several plausible labels when a single prediction is uncertain. Their value depends on classwise reliability: average coverage can meet its target while rare or difficult classes fail repeatedly. This concern is sharper after distribution shift, when calibration labels come from a source environment but reliability is needed on the target. We ask what labeled source data and unlabeled target inputs reveal about class-conditional prediction sets, and when target labels are necessary. Under unrestricted joint shift, two target laws can produce the same observable data while requiring different classwise thresholds; any label-free rule covering both must enlarge its sets on one law. We give a labeled target audit that estimates the missing
quantiles with a simultaneous guarantee. Probability-scale error is invariant to increasing score transformations, and threshold recovery follows under local regularity. At fixed confidence, achieving threshold tolerance $\varepsilon$ with fixed, nonadaptive class-stratified labeled pairs has total complexity $\Theta(K\varepsilon^{-2}\log K)$, or $\Theta(\varepsilon^{-2}\log K)$ labels per class under equal allocation. Class imbalance creates a separate acquisition cost; for foreground class probabilities of order $1/K$, the mixed-stream label complexity is also $\Theta(K\varepsilon^{-2}\log K)$ at fixed confidence. Experiments on action-recognition and image shifts show that marginal coverage can conceal severe class failures and that source classwise calibration depends on the shift. The results connect the information available at deployment to the target labels needed for useful class-conditional prediction.
Read the full abstract ▾
All authors Weijia Han, Lisha Qu, Tianxin Zhou, Zhenda Li, Liying Liang
Computer Vision
Sparse Evidence Can Suffice: Agentic Evidence Seeking for Multimodal Video Misinformation Detection
Haochen Zhao, Yongxiu Xu, Xinkui Lin et al. (+6 more)
Multimodal video misinformation detection is commonly formulated as a holistic
video-understanding task, where the entire
video and its associated content are processed and judged in a single pass. However, real-world misinformation often exhibits a sparse and compositional evidence structure: a reliable decision may depend on only a few coupled clues, while most
video content contributes limited additional information. Exhaustive
multimodal reasoning may therefore introduce substantial redundancy and obscure decisive evidence. This motivates decoupling evidence acquisition from verification: first identifying sparse, decision-relevant clues and then judging veracity based on the acquired evidence. Accordingly, we propose SIEVE, a framework for Sparse Interactive Evidence Verification via Extraction in
multimodal video misinformation detection. An evidence-seeking
agent actively explores the available
multimodal evidence and constructs a compact evidence package, which is then used by a verifier to determine veracity. The
agent is trained with supervised evidence-seeking trajectories and an evidence-aware reinforcement learning objective that promotes informative evidence acquisition while discouraging unnecessary or invalid interactions. Experiments on multiple
video misinformation
benchmarks show that SIEVE consistently outperforms the evaluated baselines and supports reliable verification using compact evidence packages. Moreover, the resulting acquisition process provides an explicit and inspectable evidence trail, improving the transparency and groundedness of
multimodal misinformation detection.
Read the full abstract ▾
All authors Haochen Zhao, Yongxiu Xu, Xinkui Lin, Dong Xie, Jiarui Lu, Yuqi Qian, Yubin Wang, Hongbo Xu, Gaopeng Gou
Computer Vision
VGOcc: Learning Visual-Geometric Gaussians for Vision-Centric 3D Driving Occupancy Prediction
Junhong Lin, Xianda Guo, Kangli Wang et al. (+4 more)
Vision-only occupancy prediction requires recovering a semantic 3D occupancy field from calibrated surround-view images, where each view provides observations with ambiguous depth along camera rays. Existing methods have progressed from dense structured representations to sparse Gaussian primitives, improving the efficiency of 3D scene representation. However, Gaussian learning still relies primarily on image domain features, which provide limited explicit
geometric information for volumetric
reasoning. Our key observation is that effective Gaussian occupancy modeling requires not only sparse primitives, but also richer
geometric and semantic learning cues. In this paper, we propose VGOcc, which learns visual and
geometric cues from foundation models for Gaussian modeling. VGOcc incorporates these cues into primitive initialization and refinement, yielding a representation termed Visual-
Geometric Gaussians tailored to semantic occupancy prediction. Specifically, we propose Visual-
Geometric Gaussian Birth to form spatially balanced Gaussian centers from ray depth hypotheses, while visual semantic features initialize primitive attributes. Next, we design Pose-Aware Feature Learning to combine foundation
tokens with camera
embeddings and calibrated ray information. Features from neighboring views are then aggregated at projected 3D locations for each Gaussian refinement stage. Finally, Gaussian decoder refines birth Gaussians with pose-aware features and renders them into semantic occupancy. Experiments on nuScenes demonstrate that VGOcc achieves state-of-the-art performance in
vision-only 3D occupancy prediction. Codes will be available at https://github.com/JHLin42in/VGOcc.
Read the full abstract ▾
All authors Junhong Lin, Xianda Guo, Kangli Wang, Yuqi Ye, Xiaoyu Liang, Yanlun Peng, Wei Gao
Computer Vision
QIRF Quantum-Inspired Non-Orthogonal Function-Space Compression for 3D Gaussian Splatting
Shizeng Jiang, Hao Zhang, Xuerui Ma et al. (+2 more)
3D Gaussian Splatting (3DGS) achieves high-quality real-time rendering by representing a scene with a large collection of anisotropic Gaussian primitives. However, complex scenes often require millions of Gaussians, resulting in substantial storage and rendering costs. Existing compression methods mainly reduce redundancy through primitive-wise
pruning, attribute quantization, clustering, or neural coding, while redundancy caused by strongly overlapping and non-orthogonal Gaussian basis functions remains largely unexplored. We present QIRF, a
quantum-inspired non-orthogonal function-space compression method for 3D Gaussian Splatting. QIRF models neighboring Gaussian primitives as a local non-orthogonal basis and formulates primitive reduction as a subspace-aware selection problem. Specifically, an analytic Gaussian overlap
matrix and a radiance-response density
matrix are constructed to characterize functional redundancy and rendering relevance. Generalized eigendecomposition is then used to identify the dominant local subspace and select representative Gaussian primitives. An RRDM-based response model and detail-aware safeguarding further preserve visually important high-frequency structures under aggressive
pruning. Experiments on 13 scenes from Mip-NeRF 360, Tanks and Temples, and Deep Blending show that QIRF reduces the Gaussian count and raw PLY storage by 71.7 percent on average, corresponding to approximately 3.54 times compression, while maintaining reconstruction quality comparable to 3DGS and achieving a marginal average PSNR improvement of 0.10 dB. QIRF also improves the average rendering speed over 3DGS by 34.3 percent. These results suggest that non-orthogonal function-space redundancy is an important yet underexplored source of representational redundancy in explicit Gaussian radiance fields.
Read the full abstract ▾
All authors Shizeng Jiang, Hao Zhang, Xuerui Ma, Ying Hu, Tao Zhang
Computer Vision · Cross-listed from cs.RO
UniETP: Unifying Environments for Generalizable Embodied Task Planning
Peiran Xu, Jiaqi Zheng, Ziyou Wang et al. (+1 more)
This paper focuses on the problem of
Embodied Task Planning, where an
agent is required to execute a sequence of atomic actions within an interactive environment to complete a user-specified task. Though a variety of simulators and
datasets have previously been built for this task, these efforts are largely isolated, with each using its own observation format, action type, and task domain. This fragmentation complicates comprehensive model evaluation and hinders the scalability of training data. As an effort towards generalizable
embodied planning, we propose UniETP, a unified interface integrating four commonly-used simulators (AI2-THOR, VirtualHome, Habitat, BEHAVIOR). UniETP is characterized by both standardization and diversity. On one hand, it formalizes all the simulators into a consistent observation and action space, and builds an evaluation system to support complicated task goal. On the other hand, it enhances task diversity and complexity across dimensions like task logic, instance grounding, and instruction understanding, constructing a new
dataset with varied levels of difficulty in an automatic manner. Extensive experiments on the proposed
benchmark are conducted to evaluate the
embodied planning capabilities of recent models and analyze the performance bottlenecks. Codes and data will be available at https://github.com/woyut/UniETP .
Read the full abstract ▾
All authors Peiran Xu, Jiaqi Zheng, Ziyou Wang, Yadong Mu
Computer Vision · Cross-listed from astro-ph.IM
Co-addition and Subtraction of Undersampled Images
Matan Schlanger and Barak Zackay
In astronomical imaging surveys, repeated observations of the same sky patches are taken in order to obtain deeper images and detect new sources. This is the case in the search for many transient phenomena, such as
supernovae, gravitational wave (GW) optical counterparts and other cataclysmic variables. In many such surveys some of the images are undersampled, meaning that the pixel size is too large, and the image suffers from aliasing. For undersampled images, both co-addition of the images and background subtraction are done in a non-optimal manner, which leads to reduced sensitivity and an increased rate of false alarms. We present a new method (named Linear Undersampled Transients & Addition (LUTRA)) that performs both processes in a mathematically proven optimal way, which allows improved performance for many scientific applications. It also allows easy and direct performance of measurements such as photometry and astrometry in a simple manner, while providing results in super-resolution. We demonstrate the performance of the method on public ZTF data and show $\times 1.25$ higher SNR compared to current methods. We provide an open source Python implementation.
Read the full abstract ▾
Computer Vision
SAR Vessel Detection and Gross Tonnage Estimation from Heterogeneous Datasets for Dark Vessel Identification
Davide Paltrinieri, Andrea Diecidue, Roberto Basla et al. (+3 more)
Detecting vessels engaging in illegal activities is of paramount importance for maritime security. One of the major goals is to detect dark vessels, ships that disable their transponders to evade surveillance. Deep Learning (DL) models can detect vessels in Synthetic Aperture Radar (SAR) images, enabling maritime traffic analysis regardless of weather or visibility conditions. However, to detect potential dark vessels, a DL model must select only those that are required to carry a transponder based on their Gross Tonnage (GT). Unfortunately, no public SAR
dataset is available for training an end-to-end DL model for vessel detection and GT regression. In this work, we present a framework that leverages heterogeneous image and tabular
datasets to solve this task. Our solution combines a multi-task DL framework for predicting the location, vessel type, and physical dimensions of ships, cascaded with a non-parametric model for predicting GT from vessel size and category. We perform GT regression by a KNN that measures sample similarity using a hybrid Euclidean and categorical distance. Experiments show that our solution can predict multiple outputs while remaining competitive with state-of-the-art models on individual subtasks, thus enabling the identification of dark vessels. We publish our code on GitHub https://github.com/PaltrinieriDavide/vesseldetection.
Read the full abstract ▾
All authors Davide Paltrinieri, Andrea Diecidue, Roberto Basla, Daniele Casciani, Piero Fraternali, Giacomo Boracchi
Computer Vision
Anticipate Before Acting: Future-State-Conditioned Vision-Language Navigation
Lingfeng Zhang, Zhanguang Zhang, Liheng Ma et al. (+2 more)
End-to-end
vision-
language navigation (VLN) with causal
vision-
language models maps instructions and
egocentric observations directly to actions, but standard behavior cloning supervises only the next action and does not explicitly encourage the policy state to be predictive of future visual outcomes, limiting long-horizon decision making. A privileged-input diagnostic shows that access to an expert-trajectory future image can substantially improve
navigation, indicating that future observations contain rich, actionable cues, though such inputs are unavailable at deployment. Motivated by this signal, we propose Future-State-Conditioned VLN (FSC-VLN), a deployable model that augments a causal policy with a future-query
token and uses training-only future-state supervision to distill information from future observations into the policy state. Concretely, during training we align the future-query representation to a frozen visual
embedding $\Delta$ steps ahead, while inference requires only past and current observations. This design preserves the baseline inference pattern and adds only two learned prefix
tokens, implying minimal overhead. On R2R val-unseen, FSC-VLN improves SR/OSR/SPL over a StreamVLN-style baseline under two training-data regimes, with larger gains on long-horizon episodes; ablations further support the dual-query design that separates future and action queries.
Read the full abstract ▾
All authors Lingfeng Zhang, Zhanguang Zhang, Liheng Ma, Tongtong Cao, Yingxue Zhang
Computer Vision
When 2D Cues Fail: Improving Image Manipulation Localization with Reliable 3D Geometry
Guofeng Yu, Zhiqing Guo, Dan Ma et al. (+1 more)
Existing image manipulation localization (IML) methods rely heavily on 2D forensic cues, such as low-level artifacts, noise traces, and semantic inconsistencies in the manipulated image. While effective in many cases, these cues become much less discriminative when manipulated regions are well blended with their surrounding context in appearance. In such cases, a manipulated region may remain locally appearance-consistent, but still violate the
geometric structure of the surrounding scene. This limitation motivates us to go beyond purely 2D evidence and introduce
geometric reasoning into IML. To this end, we leverage
monocular reconstruction to obtain auxiliary
geometric cues, including depth and surface normals. However, a key challenge lies in the fact that reconstructed geometry on manipulated images is inherently noisy and cannot be used naively. Rather than treating depth and normals as direct evidence, we estimate their reliability and exploit them selectively for localization. Based on this principle, we design a geometry-aware framework (GFrame) that fuses reliable
geometric cues with RGB features and propagates them across scales to improve fine-grained localization. Extensive experiments show that the proposed method achieves excellent performance under limited budget constraints. These results indicate that reliable 3D geometry provides complementary forensic evidence beyond traditional 2D cues for IML. Related code will be released.
Read the full abstract ▾
All authors Guofeng Yu, Zhiqing Guo, Dan Ma, Gaobo Yang
Computer Vision
Benchmarking NACTI Species Recognition in Long-Tailed Regimes
Zehua Liu and Tilo Burghardt
As with most “in the wild” collections of the natural world, the North America Camera Trap Images (NACTI)
dataset exhibits long-tailed class imbalance, with the largest class covering over 50% of its 3.7M images. Building on the PyTorch Wildlife model, we systematically evaluate Long-Tail Recognition (LTR) methodologies to
benchmark species recognition performance, including specialised loss functions and LTR-sensitive regularisation. Our optimised configuration achieves state-of-the-art 99.40% Top-1 accuracy on the NACTI test split, significantly outperforming standard baselines and previously reported top performances. To assess robustness under domain shifts (e.g., night-time captures, occlusion, motion-blur), we extend our evaluation across three independent reduced-bias test sets (including ENA-Detection, Caltech Camera Traps and Missouri Camera Traps). Across these out-of-distribution (OOD) evaluations, our LTR-enhanced model consistently demonstrates substantially stronger generalisation capabilities compared to standard cross-
entropy approaches. However, qualitative and quantitative analyses underline that current LTR optimisations cannot fully overcome representational bottlenecks, resulting in catastrophic predictive breakdown for rare `Tail' classes under severe domain shift. For maximum reproducibility, all
dataset splits, key code, and network weights are published with this paper at https://github.com/ZehuaLiuY/Species-Classification.
Read the full abstract ▾
Computer Vision
SAMRI-3D: Adapting SAM2 for 3D MRI Segmentation with Global Volume Tokens
Zhao Wang, Wei Dai, Hongfu Sun et al. (+2 more)
Foundation models such as Segment Anything Model 2 (SAM2) have transformed natural-image and
video segmentation, and recent work has begun adapting them to medical imaging. These adaptations, however, are largely general-purpose models that treat MRI as one modality among many; large-scale, MRI-specific modelling and benchmarking remain limited, even though MRI's low soft-tissue contrast leaves many boundaries effectively invisible on individual slices. We present SAMRI-3D, a
benchmark and method for 3D MRI
segmentation with SAM2. The SAMRI-3D
benchmark is the largest MRI-only evaluation to date - 10,392 volumes from 34
datasets (27 public, 7 in-house) spanning 12 anatomical domains and 10+ sequences, with explicit seen/unseen splits. Freezing the image encoder and fine-tuning only the lightweight decoder and
memory modules raises mean Dice from 0.58 (zero-shot SAM2) to 0.76, surpassing recent SAM-based medical models (SAMed-2 0.69, Medical-SAM2 0.49, SAM-Med3D 0.37) with strong statistical significance. To target invisible boundaries, we introduce Global Volume
Tokens (GVT): persistent
memory tokens trained with a Truncated Signed Distance Field (TSDF) reconstruction objective that is discarded at inference (zero added cost). This full model, SAMRI-3D, attains the best accuracy (0.78) and lowest variance across all 34
datasets and, uniquely, shows no drop on 8 held-out
datasets (0.79 unseen vs. 0.78 seen); per-sequence analysis confirms the TSDF objective helps most where per-slice contrast is weakest. We will release the
benchmark, code, and models in this paper.
Read the full abstract ▾
All authors Zhao Wang, Wei Dai, Hongfu Sun, Craig Engstrom, Shekhar S. Chandra
Computer Vision · Cross-listed from cs.AI
Do Maps Still Matter for Machines: Revisiting the Role of Choropleth Maps in Foundation Model Spatial Understanding
Zhiwei Wei, Yonghe Sun, Zhenjia Liu et al. (+5 more)
Spatial understanding is crucial for foundation models (FMs), and maps have long helped humans organize and reason about geographic information. This study examines whether choropleth maps remain useful for machine spatial understanding when models can directly process structured geodata. We introduce ChoroplethMap-Bench, a controlled
benchmark containing 2,400 synthetic choropleth maps, corresponding GeoJSON data, and 12,000 questions across five cognitive dimensions: Identify, Spatial Recognition, Compare, Rank, and Delineate. We evaluate 22 open-source and proprietary models under three input conditions: Data Only, Map Only, and Data + Map. The results show that maps substantially improve spatial
reasoning, especially when combined with symbolic data and for tasks requiring higher-level understanding of spatial patterns. We further analyze the effects of map type, color hue, and spatial structure, as well as prompting strategies,
language, geographic context, decoding settings, classification methods, and response stability. Overall, the Data + Map condition achieves the strongest performance, demonstrating that maps remain valuable external representations for foundation model spatial
reasoning.
Read the full abstract ▾
All authors Zhiwei Wei, Yonghe Sun, Zhenjia Liu, Wenjia Xu, Chao He, Weihua Dong, Chunbo Liu, Hua Liao
Computer Vision
HAS: Highlight-guided Attention Steering for Multimodal LLM Video Summarization
Rui Chu and Yingjie Lao
Video understanding has become more and more important with the growth of Artificial Intelligence (AI) for
video generation. Recently,
Multimodal Large
Language Model(M-LLM) has shown its capability in
video understanding.
Video summarization, a specific domain of
video understanding, has proven its importance for efficient
navigation and
retrieval. Both
video understanding and
video summarization require a good selection of key frames in a
video. Current
video summarization methods heavily focus on the selected key frames and correlated segment captions. However, existing approaches overlook the perspective of treating the importance of the frames globally. We argue that using discrete selected frames for summarization will not only reduce the understanding coherence, but also lost important information in the
video, as well as wasting the original capacity of the MLLMs. In this paper, we propose HAS, a Highlight-guided
Attention Steering method for
video summarization. We consider a challenging but practical setting where the
video given to MLLMs for summarize should be continuous but with highlight guidance. HAS mainly consists of two parts: The first part is to find a continuous frame-level highlight distribution for the
video globally. The second part is to apply the highlight distribution as an
attention steering vector for the MLLM, targeting a better understanding of the
video, and thus during the model inference time, putting more
attention on the highlighted frames, while avoiding lost entire information on less highlighted frames through putting less
attention instead of forgetting them. We evaluated HAS on a variety of
benchmarks, and it has shown convincing performance in
video summarization.
Read the full abstract ▾
Computer Vision
Keyframe-Anchored Identity Preservation for Sequential-Action Video Generation
Zhenjie Liu, Binyan Chen, Hao Chen et al. (+2 more)
Identity-preserving text-to-
video generation aims to synthesize a
video that accurately follows a textual description while maintaining the recognizability of a user-specified subject throughout. The IPVG26 challenge extends this framework from a single holistic
prompt to a temporally structured specification. The model additionally receives a sequence of timestamped action captions and must render the subject performing these actions in the specified order. This temporal structure presents a challenge not encountered in previous identity-preserving generation tasks, as the subject must continuously perform a scripted sequence of distinct actions while maintaining a consistent identity. However, end-to-end
video generators are prone to appearance drift as motion accumulates and the depicted actions change. We address this challenge with a training-free, three-stage pipeline framework. An action-aware
prompt polishment stage first rewrites the inputs into image-generation
prompts that specify the terminal state of each action. An identity-preserving generation stage then produces the keyframe sequence by conditioning each frame jointly on the reference identity and its predecessor, thereby decoupling time-invariant appearance from time-varying pose. Finally, an identity-aware inference enhancement stage synthesizes the intermediate segments using multi-reference guidance and identity-driven noise searching, both of which reinforce identity fidelity during sampling. Our method ranked third on the official Track 2 leaderboard, demonstrating competitive performance and strong generality.
Read the full abstract ▾
All authors Zhenjie Liu, Binyan Chen, Hao Chen, Tong Pan, Shangfei Wang
Computer Vision
MoGe-3: Fine-Detail Monocular Geometry Estimation with Self-Guided Sparse Volumetric Refinement
Lingyu Kong, Ruicheng Li, Ruicheng Wang et al. (+4 more)
Monocular geometry estimation has recently achieved impressive performance across diverse scenes. However, state-of-the-art models still face notable distortion in local 3D structure, especially in fine details, like thin structures and small objects. We attribute this limitation to an architectural mismatch: most current models decode 3D geometry within a 2D parameterization, where feature interactions are governed by image-plane proximity rather than true 3D spatial relationships. This inadvertently mixes features from geometrically distant surfaces, resulting in over-smoothed geometry particularly around thin or elongated structure. In this paper, we propose MoGe-3, a fine-detail
monocular geometry estimation model with Self-Guided Sparse 3D Refinement (SSR) that lifts
monocular geometry modeling from 2D image space to 3D space for high-fidelity metric-scale point maps. MoGe-3 lifts the coarse point map from a foundation base model onto a sparse voxel shell and refines it via SSR. The SSR employs sparse convolutions that aggregate features based on 3D spatial locality, avoiding feature mixing across depth discontinuities. Extensive experiments on diverse
datasets demonstrate that MoGe-3 significantly outperforms existing approaches in recovering fine detailed 3D geometry across both quantitative metrics and qualitative visualizations. Project page: https://qft-333.github.io/moge3page/
Read the full abstract ▾
All authors Lingyu Kong, Ruicheng Li, Ruicheng Wang, Sicheng Xu, Chengtang Yao, Jianfeng Xiang, Jiaolong Yang