Computer Vision · Cross-listed from cs.GR
PhysOmni: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction
Xin Zhang, Yabo Chen, Yijie Fang et al. (+5 more)
Recent generative
video models achieve impressive visual quality but remain constrained by limited physical consistency and controllability. Existing
video generation methods provide minimal physical control, and single-image-to-3D conversion approaches often suffer from object interpenetration. Furthermore, physics-based scene-level 3D generation methods exhibit spatial misalignment, stylized artifacts, and inconsistencies with the input data, restricting their use in realistic interactive
video synthesis. We propose PhysOmni, a training-free framework that converts a single image into a physically consistent and controllable
video through holistic scene-level 3D reconstruction. By rep?resenting the full scene geometry in a unified spatial coordinate system, PhysOmni resolves object penetration and
alignment ambiguity. Unlike prior methods, this formulation enables accurate scene?level multi-object interactions and introduces richer, complex control types for advanced mechanics?based manipulation. By decoupling simulation from rendering, PhysOmni bypasses latency-heavy priors, achieving real-time physical interaction previews paired while preserving photorealistic visual fidelity. Experimental results demonstrate that PhysOmni substantially outperforms prior methods in physical fidelity, spatial coherence, and controllability. Project Page: https://zephipage.github.io/PhysOmni/
Read the full abstract ▾
All authors Xin Zhang, Yabo Chen, Yijie Fang, Wanying Qu, Haibin Huang, Chi Zhang, Feng Xu, Xuelong Li
Computer Vision · Cross-listed from cs.LG
FusionCell: Cross-Attentive Fusion of Layout Geometry and Netlist Topology for Standard-Cell Performance Prediction
Haoyi Zhang, Kairong Guo, Bojie Zhang et al. (+2 more)
Standard
cells form the building blocks of digital
circuits, so their delay and power critically influence chip-level performance; yet characterization still relies on slow simulation sweeps, and many fast predictors ignore layout geometry, missing coupling and layout-dependent effects. The challenge is to jointly represent layout geometry and netlist
topology so models capture fine-grained spatial details together with structural connectivity for accurate performance prediction. We introduce FusionCell, a dual-modality predictor that treats routed layout geometry and netlist
topology as inputs and fuses them explicitly in a unified model. A DeiT encoder processes three-layer routed layouts, while a graph
transformer models heterogeneous device/net graphs. The modalities are integrated through a
topology-guided mechanism, where the netlist acts as a structural "map" to actively query relevant physical regions in the layout for joint
geometric and topological
reasoning. We build a 7nm
dataset based on the ASAP7 PDK with over 19.5k
cells spanning 149 types using automatic tools, targeting six metrics: signal rise/fall delay, transition, and power. Experimental results demonstrate that FusionCell reduces regression error, with an average MAPE of 0.92 percent, and improves Spearman/Kendall ranking over baselines, while accelerating the characterization process by orders of magnitude compared to
circuit simulation.
Read the full abstract ▾
All authors Haoyi Zhang, Kairong Guo, Bojie Zhang, Yibo Lin, Runsheng Wang
Computer Vision
JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA
Hyunju Kang, Woohyun Lee, Jaewon Kim et al. (+1 more)
Industrial anomaly detection has been significantly advanced by Large
Multimodal Models (LMMs), enabling diverse human instructions beyond detection, particularly through visually grounded
reasoning for better image understanding. However, LMMs lack domain-specific knowledge, which limits their ability to generate accurate responses in complex industrial scenarios. In this work, we present JUDO, Juxtaposed Domain-Oriented
Multimodal Reasoner, a framework that efficiently incorporates domain knowledge and context in visual and textual
reasoning. Through visual
reasoning, our model segments the defect region by juxtaposing query images with normal images as visual domain context, enabling a fine-grained visual comparative inspection. Furthermore, we inject domain knowledge through supervised fine-tuning (SFT) to enhance context understanding and subsequently guide domain
reasoning through reinforcement learning (GRPO) with tailored rewards, opting for a domain-oriented
reasoning process. Experimental results demonstrate that JUDO achieves superior performance on the MMAD
benchmark, surpassing models such as Qwen2.5-VL-7B and GPT-4o. These results highlight the importance of enhancing domain knowledge and context for effective
reasoning in anomaly understanding.
Read the full abstract ▾
All authors Hyunju Kang, Woohyun Lee, Jaewon Kim, Hogun Park
Computer Vision
Do Vision Models Truly Forget? New Findings from Representation-Level Certification of Visual Unlearning in Vertical Federated Learning
Zhenyu Yu, Yangchen Zeng, Chunlei Meng et al. (+2 more)
Machine unlearning in Vertical
Federated Learning (VFL) has attracted growing interest, yet existing methods certify forgetting solely using output-level metrics. We challenge these works by introducing Mirage, a representation-level
auditing framework that comprises four complementary diagnostics: Linear probe recovery (LPR), centered kernel
alignment (CKA), feature separability scoring, and layer-wise recovery analysis. Extensive experiments across seven
datasets and seven baseline methods following recent VFL unlearning protocols reveal three key findings: (1) Forgetting gap: methods that pass output-level certification still retain substantial class structure in their representations, with LPR exceeding the retrained baseline by up to 15.4 points; CKA shows that these models remain structurally closer to the original than to the retrained reference, while separability scores indicate persistent
geometric discrimination. (2) Unlearning trilemma: no existing method simultaneously achieves high utility, output-level forgetting, and representation-level forgetting. (3) Class-sample asymmetry: class-level forgetting leaves strong representational traces (LPR exceeding 96 percent on several
datasets), whereas sample-level forgetting is indistinguishable from chance (LPR is approximately 50 percent); layer-wise analysis further shows that residual class information persists across network depths. These findings call for representation-aware evaluation standards in
federated unlearning research. Code is publicly available at https://github.com/YuZhenyuLindy/Mirage.
Read the full abstract ▾
All authors Zhenyu Yu, Yangchen Zeng, Chunlei Meng, Guangzhen Yao, Shuigeng Zhou
Computer Vision · Cross-listed from cs.LG
ClaimDiff-RL: Fine-Grained Caption Reinforcement Learning through Visual Claim Comparison
Tianle Li, Xuyang Shen, Yan Ma et al. (+7 more)
Long-form image captioning exposes a reward granularity problem in RL: captions are judged as whole sequences, while the important errors occur at the level of individual visual claims. A good dense caption should be both faithful and informative, avoiding
hallucination without omitting salient details. Yet pairwise preferences, reference-based metrics, and holistic scalar rewards compress these local errors into a single sequence-level signal, obscuring the tradeoff between factuality and coverage. We introduce ClaimDiff-RL, a framework that uses reference-conditioned atomic claim differences as the reward unit for caption RL. Given an image, an actor caption, and a reference caption, a
multimodal judge enumerates visually grounded differences, verifies each difference against the image, assigns open-vocabulary error types and severity levels, and produces per-difference statistics for reward composition. This makes hallucinated claims and omitted salient facts separately measurable and tunable. Experiments show that holistic scalar rewards can reduce
hallucination by increasing missing facts, while ClaimDiff-RL exposes this faithfulness and coverage tradeoff and enables more balanced operating points. On a 160-image human-labeled diagnostic
benchmark, public captioning
benchmarks, and VQA
benchmarks, ClaimDiff-RL improves the
hallucination–missing-fact balance, preserves general capability, and even surpasses Gemini-3-Pro-Preview on several fine-grained Capability dimensions such as object counting, spatial relations, and scene recognition. These results suggest that typed, verifiable claim differences are an effective reward unit for fine-grained and diagnosable caption RL.
Read the full abstract ▾
All authors Tianle Li, Xuyang Shen, Yan Ma, Rongxin Guo, Shaoxiang Chen, Jiacheng Chen, Haochen Wang, Hongyang Tang, Yucong Zhou, Yu Cheng
Computer Vision
Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis
Tianwei Lin, Zhongwei Qiu, Jie Cao et al. (+7 more)
Medical
vision-
language models (VLMs) have rapidly advanced as general-purpose
multimodal assistants, yet their deployment in 3D Computed
Tomography (CT) analysis remains constrained by a persistent mismatch between optimization objectives and clinical rigor. Current Reinforcement Learning (RL) paradigms still rely on lexical proxy signals that induce “Evaluation
Hallucinations”, where models optimize linguistic fluency rather than factual clinical correctness, leading to diagnostically critical errors. To bridge this gap, we introduce the Clinical Abnormality Benchmarking Substrate (CABS), a structured system that decomposes radiology reports into verifiable clinical semantic units. Using CABS, we identify a “Mechanistic Divergence” in standard RL, where surface-similarity rewards drive policy gradients to bypass medical facts. We therefore propose Trajectory-Integral Feedback GRPO (TIF-GRPO), a novel framework integrating control-theoretic principles into policy optimization. By formulating clinical
reasoning as a pseudo-temporal trajectory for anomaly discovery, TIF-GRPO regulates anatomy-aware rewards via an integral feedback loop that penalizes persistent omissions as cumulative state errors and suppresses
hallucinations as excessive control effort. Experiments on 3D CT
benchmarks demonstrate that our approach significantly enhances abnormality detection and clinical faithfulness, establishing a new paradigm for fine-grained regulation in medical VLMs. Our project is available at \href{https://github.com/ZJU4HealthCare/TIF-GRPO}{GitHub}.
Read the full abstract ▾
All authors Tianwei Lin, Zhongwei Qiu, Jie Cao, Jiang Liu, Wenjie Yan, Bo Zhang, Yu Zhong, Wenqiao Zhang, Yingda Xia, Ling Zhang
Computer Vision
You Don't Need Attention: Gated Convolutional Modeling for Watch-Based Fall Detection
Sana Alamgeer, Ronish Kumar, Awatif Yasmin et al. (+2 more)
Existing deep learning approaches for wearable fall detection systems rely on self-
attention mechanisms that impose quadratic computational overhead, distributing weights across all time steps. This global weight distribution impairs the precise localization of the brief impact signatures that characterize falls within short, fixed-length windows. To overcome this challenge, we propose Gated-CNN, a lightweight dual-stream architecture that processes accelerometer and gyroscope streams through independent one-dimensional convolutional feature extractors, followed by (i) a sigmoid gating module that selectively suppresses uninformative background activations while amplifying fall-discriminative features, (ii) a global average pooling layer that compresses each stream into a compact fixed-length descriptor, and (iii) a shared classification head that fuses both descriptors for binary fall prediction. For offline evaluation, we evaluate the model across five wrist-mounted inertial measurement unit (IMU)
datasets, achieving average F1-scores of 93%, 93%, 90%, 91%, and 90% on SmartFallMM, WEDA-Fall, FallAllD, UMAFall, and UP-Fall, outperforming
Transformer baselines. For real-time evaluation, we deployed the model on a Google Pixel Watch 3 and tested across 12 participants. The model achieves an average F1-score of 97% and an accuracy of 98% with zero missed falls, showing that sigmoid gating offers a more structurally aligned and computationally efficient alternative to
attention for commodity smartwatch-based fall detection.
Read the full abstract ▾
All authors Sana Alamgeer, Ronish Kumar, Awatif Yasmin, Muhammad Irshad, Anne H. H. Ngu
Computer Vision · Cross-listed from cs.GR
PiG-Avatar: Hierarchical Neural-Field-Guided Gaussian Avatars
Julian Kaltheuner, Jan Spindler, Sina Kitz et al. (+2 more)
Existing Gaussian avatar methods typically parameterize geometry on a body-template surface, which entangles the avatar's representation space with the template's deformation space and limits the capture of layered, off-body, and non-rigid clothing geometry. We present PiG-Avatar, which addresses this limitation by using the parametric body model solely for kinematic transport, while representing the avatar as Gaussians anchored in a volumetric canonical space governed by a continuous neural field. This decouples representation from template
topology, avoiding the
geometric constraints of surface-based parameterizations. Kinematic coherence is maintained through 3D barycentric anchor transport, which guides motion without constraining geometry and allows anchors to deviate freely from the template surface, yielding dense, stable temporal surface correspondences by construction. To make this unconstrained formulation tractable, we introduce dual-level spatially coherent optimization, combining Sobolev-preconditioned neural-field updates with a novel KNN-based preconditioning of canonical anchor geometry. Together, these mechanisms induce an emergent self-organization of anchor density: anchors migrate toward regions of high
curvature, appearance variation, and non-coherent motion without explicit heuristics. As a result, complex clothing geometry and layered surfaces emerge as natural, high-fidelity outputs. This single representation further supports hierarchical reconstruction across multiple levels of detail, with coarse-level supervision propagating to finer levels through the shared field and coupled anchor graph. On established
benchmarks featuring subjects with complex clothing and challenging non-rigid motion, PiG-Avatar achieves state-of-the-art rendering quality, generalizes robustly to imperfect body model initialization, and renders in real time across all detail levels.
Read the full abstract ▾
All authors Julian Kaltheuner, Jan Spindler, Sina Kitz, Patrick Stotko, Reinhard Klein
Computer Vision
MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation
Yujie Wei, Yujin Han, Zhekai Chen et al. (+20 more)
Video generation is rapidly evolving from single-shot synthesis to complex multi-shot audio-
video (MSAV) narratives to meet real-world demands. However, evaluating such frontier models remains a fundamental challenge. Existing
benchmarks are limited in scope and data diversity, and rely on rigid evaluation pipelines, preventing systematic and reliable assessment of modern MSAV models. To bridge these gaps, we introduce MSAVBench, the first comprehensive
benchmark and adaptive hybrid evaluation framework for multi-shot audio-
video generation. Our
benchmark spans four key dimensions,
video, audio, shot, and reference, covering diverse task settings, varying shot counts of up to 15, and challenging non-realistic scenarios. Our evaluation framework improves robustness through an adaptive self-correction mechanism for shot
segmentation, instance-wise rubrics for subjective metrics, and tool-grounded evidence extraction for complex judgments. Furthermore, MSAVBench achieves high
alignment with human judgments, reaching a Spearman rank correlation of 91.5%. Our systematic evaluation of 19 state-of-the-art closed- and open-source models shows that current systems still struggle with director-level control and fine-grained audio-visual synchronization, while modular or agentic generation pipelines offer a promising path toward narrowing the gap between open- and closed-source models. The
benchmark data and evaluation code are publicly available at https://github.com/ali-vilab/MSAVBench.
Read the full abstract ▾
All authors Yujie Wei, Yujin Han, Zhekai Chen, Yongming Li, Kaixun Jiang, Zhihang Liu, Quanhao Li, Zhiwu Qing, Xiang Wang, Zhen Xing, Ruihang Chu, Lingyi Hong, Yefei He, Junjie Zhou, Junqiu Yu, Yang Shi, Difan Zou, Kai Zhu, Shiwei Zhang, Yingya Zhang, Yu Liu, Xihui Liu, Hongming Shan
Computer Vision · Cross-listed from cs.CL
From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models
Juncheng Wu, Hardy Chen, Haoqin Tu et al. (+6 more)
Recent advances in
vision-
language models (VLMs) emphasize long chain-of-thought
reasoning; yet, we find that their performance on visual tasks is primarily limited by a lack of visual perception as opposed to
reasoning itself. In this work, we systematically study the interplay between perception and
reasoning in VLM post-training by decomposing their capabilities into three separate training stages: visual perception, visual
reasoning, and textual
reasoning, incorporating specialized training data. We demonstrate that visual perception (a) requires targeted optimization with specialized data; (b) serves as a fundamental scaffold that should be solidified through staged training before refining visual
reasoning; and (c) is more effectively learned via RL than caption-based SFT. Our experiments across multiple VLMs demonstrate that staged training consistently improves both visual perception and
reasoning performance over merged training. Notably, models trained with our approach achieve 1.5% higher
reasoning accuracy with 20.8% shorter
reasoning traces, suggesting that superior perception reduces the need for excessive
reasoning. Furthermore, we show that this capability-based staging represents a new curriculum dimension orthogonal to traditional difficulty-based curricula, and combining both yields further additive gains. Our staged-training models achieve superior performance among open-weight VLMs, establishing advanced results on several visual math and perception (e.g., +5.2% on WeMath and +3.7% on RealWorldQA) tasks compared with the base counterpart.
Read the full abstract ▾
All authors Juncheng Wu, Hardy Chen, Haoqin Tu, Xianfeng Tang, Freda Shi, Hui Liu, Hanqing Lu, Cihang Xie, Yuyin Zhou
Computer Vision
Multi-axis Analysis of Image Manipulation Localization
Keanu Nichols, Divya Appapogu, Giscard Biamby et al. (+3 more)
Advanced image editing software enables easy creation of highly convincing image manipulations, which has been made even more accessible in recent years due to advances in generative AI. Manipulated images, while often harmless, could spread misinformation, create false narratives, and influence people's opinions on important issues. Despite this growing threat, there is limited research on detecting advanced manipulations across different visual domains. Thus, we introduce Analysis Under Domain-shifts, qualIty, Type, and Size (AUDITS), a comprehensive
benchmark designed for studying axes of analysis in image manipulation detection. AUDITS comprises over 530K images from two distinct sources (user and news photos). We curate our
dataset to support analysis across multiple axes using recent
diffusion-based inpaintings, spanning a diverse range of manipulation types and sizes. We conduct experiments under different types of domain shift to evaluate robustness of existing image manipulation detection methods. Our goal is to drive further research in this area by offering new insights that would help develop more reliable and generalizable image manipulation detection methods.
Read the full abstract ▾
All authors Keanu Nichols, Divya Appapogu, Giscard Biamby, Dina Bashkirova, Anna Rohrbach, Bryan A. Plummer
Computer Vision
CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Models
Hsiang-Wei Huang, Junbin Lu, Kuang-Ming Chen et al. (+3 more)
Vision-
Language Models (VLMs) achieve strong performance on spatial question answering
benchmarks, yet it remains unclear whether such gains reflect genuine spatial intelligence. We show that existing spatial VLMs lack basic camera motion understanding, a key component of spatial cognition. We propose the Spatial Narrative Score (SNS), an evaluation framework that requires VLMs to generate explicit spatial narratives capturing both scene semantics and camera motion, followed by
reasoning with a frozen proxy LLM. Under SNS, state-of-the-art spatial VLMs exhibit significant performance degradation despite high direct question answering accuracy. To address this gap, we introduce CaMo, a camera motion grounded VLM that achieves consistent performance across SNS evaluation and direct spatial question answering accuracy. Our results highlight the importance of explicit spatial narrative externalization for evaluating VLMs with transferable 3D spatial understanding. Our code, data, and model is available at https://github.com/hsiangwei0903/CaMo
Read the full abstract ▾
All authors Hsiang-Wei Huang, Junbin Lu, Kuang-Ming Chen, Jianxu Shangguan, Cheng-Yen Yang, Jenq-Neng Hwang
Computer Vision
Interpretable Computer Vision for Defect Detection in X-ray Tomography of Aerospace SiC/SiC Composites
Antonio Peña Corredor, Julien Lesseur, Romain Nunez et al. (+2 more)
Non-destructive testing of aerospace SiC/SiC composites via X-ray computed
tomography (XCT) relies on expert visual assessment, with current workflows offering limited traceability for accept/reject decisions. Deep convolutional networks can automate defect detection, yet their black-box nature conflicts with the transparency that industrial inspection practice demands. To close this gap, we introduce p-ResNet-50, a convolutional framework extended with a prototype layer that couples high detection accuracy with case-based explanations. Six learned prototypes are explicitly aligned with expert-defined semantic categories-healthy
matrix,
matrix–air interfaces, pores, line-like defects, and mixed morphologies-so that every classification is traceable to a physically meaningful reference. Two novel regularisation terms, anchor-based and medoid-based, tether prototypes to expert-selected patches and prevent prototype collapse, addressing a known limitation of prototype networks. Latent-space analysis via UMAP delineates semantically coherent sub-domains and maps zones of uncertainty where misclassifications concentrate, giving inspectors an explicit picture of where the model is-and is not-reliable. The framework is validated on an XCT patch
dataset of approximately 12,000 patches extracted from four defect-rich SiC/SiC laboratory specimens. Taking a black-box ResNet-50 as a baseline (ROC-AUC = 0.991), the prototype extension achieves comparable performance (accuracy 0.957 vs. 0.959; ROC-AUC 0.994 vs. 0.993) while trading a slight reduction in sensitivity for higher precision and specificity. Each decision is backed by representative evidence patches, and the model explicitly flags its uncertainty regions. Beyond defect mapping, the framework establishes a reusable methodology for
embedding domain-expert knowledge into prototype networks, applicable to other XCT inspection scenarios requiring traceable, auditable decisions.
Read the full abstract ▾
All authors Antonio Peña Corredor, Julien Lesseur, Romain Nunez, Paul Rivalland, Thomas Philippe
Computer Vision
Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models
Guangzhi Xiong, Qiao Jin, Sanchit Sinha et al. (+2 more)
Large
Vision Language Models (LVLMs) show promise in medical applications, but their inability to faithfully ground responses in visual evidence raises serious concerns about clinical trustworthiness. While visual attribution methods are widely used to explain LVLM predictions, whether these explanations actually reflect the visual evidence underlying the model's decision is largely unverified, since ground-truth annotations for internal model
reasoning are typically unavailable. We address this question for chest X-ray (CXR)
reasoning by developing a causal evaluation framework that retains only CXR-VQA samples for which the expert-annotated region is verified, via counterfactual editing, to be causally responsible for the model's prediction. Using this framework across 11 attribution methods, six open-source LVLMs, and two output modes (direct answer and step-by-step
reasoning), we find that existing attribution methods often fail to identify the evidence used by LVLMs. To address this failure, we propose MedFocus, a concept-based attribution method that localizes clinically meaningful anatomical regions via unbalanced optimal transport and measures their causal effect on model outputs through targeted interventions. MedFocus produces spatial, concept-level, and
token-level attributions and substantially outperforms prior methods, taking a step toward more trustworthy attribution for medical LVLMs. Our data and code are available at https://github.com/gzxiong/medfocus/.
Read the full abstract ▾
All authors Guangzhi Xiong, Qiao Jin, Sanchit Sinha, Zhiyong Lu, Aidong Zhang
Computer Vision
TideGS: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization
Chonghao Zhong, Linfeng Shi, Hua Chen et al. (+4 more)
Training 3D Gaussian Splatting (3DGS) at billion-primitive scale is fundamentally
memory-bound: each Gaussian primitive carries a large attribute vector, and the aggregate parameter table quickly exceeds GPU capacity, limiting prior systems to tens of millions of Gaussians on commodity single-GPU hardware. We observe that 3DGS training is inherently sparse and trajectory-conditioned: each iteration activates only the Gaussians visible from the current camera batch, so GPU
memory can serve as a working-set cache rather than a persistent parameter store. Building on this insight, we introduce TideGS, an out-of-core training framework that manages parameters across an SSD-CPU-GPU hierarchy via three synergistic techniques: block-virtualized geometry for SSD-aligned spatial locality, a hierarchical asynchronous pipeline to overlap I/O with computation, and trajectory-adaptive differential streaming that transfers only incremental working-set deltas between iterations. Experiments show that TideGS enables training with over one billion Gaussians on a single 24 GB GPU while achieving the best reconstruction quality among evaluated single-GPU baselines on large-scale scenes, scaling beyond prior out-of-core baselines (e.g., approximately 100M Gaussians) and standard in-
memory training (e.g., approximately 11M Gaussians).
Read the full abstract ▾
All authors Chonghao Zhong, Linfeng Shi, Hua Chen, Tiecheng Sun, Hao Zhao, Binhang Yuan, Chaojian Li
Computer Vision
PixVerve: Advancing Native UHR Image Generation to 100MP with a Large-Scale High-Quality Dataset
Haojun Chen, Haoyang He, Chengming Xu et al. (+11 more)
Text-to-Image (T2I) models have recently seen notable progress around 1K and 2K resolution. With the extreme desire for better visual experience and the rapid development of imaging technology, the demand for Ultra-High-Resolution (UHR) image generation has grown significantly. However, UHR image generation poses great challenges due to the scarcity and complexity of high-resolution content. In this paper, we first introduce PixVerve-95K, a high-quality, open-source UHR T2I
dataset curated with a carefully designed data pipeline, which contains 95K images across diverse scenarios (each image has a minimum pixel-count of 100M) and seven-dimensional annotations. Based on our large-scale image-text
dataset, we take a pioneering step to extend various T2I foundation models to native 100MP generation with three training schemes. Finally, leveraging both conventional metrics and
multimodal large
language model-based assessments, our proposed PixVerve-Bench
benchmark establishes a comprehensive evaluation protocol for UHR images encompassing visual quality and semantic
alignment. Extensive experimental results on our
benchmark and the constructive exploration of training strategies collaboratively provide valuable insights for future breakthroughs.
Read the full abstract ▾
All authors Haojun Chen, Haoyang He, Chengming Xu, Qingdong He, Junwei Zhu, Yabiao Wang, Zhucun Xue, Xianfang Zeng, Zhennan Chen, Xiaobin Hu, Hao Zhao, Yong Liu, Jiangning Zhang, Dacheng Tao
Computer Vision
SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Prediction
Zhixiong Zhang, Yizhuo Li, Shuangrui Ding et al. (+6 more)
Referring
segmentation grounds natural-
language queries to pixel-level masks, but extending it to complex scenarios with multiple instances, cross-category groups, or open-ended target sets remains challenging. Previous Large
Vision Language Model (LVLM)-based methods represent referred targets with one or more special
tokens sequentially, treating multiple targets as separate outputs rather than a coherent set and offering little incentive to capture set-level properties such as completeness and mutual exclusivity. We reformulate open-ended referring
segmentation as explicit set-level concept prediction and propose Set-Concept
Segmentation (SetCon), which uses LVLM-generated natural-
language concepts, instead of
segmentation-specific
tokens, as semantic conditions for joint mask-set decoding. A hierarchical semantic decomposition first predicts a shared set-level concept defining the target scope and then refines it into fine-grained concept groups aligned with target subsets. To support this, a two-stage annotation pipeline augments existing
reasoning segmentation datasets with hierarchical semantic supervision (236k samples, 784k concept phrases). SetCon achieves state-of-the-art results on image
benchmarks (+3.3 gIoU on gRefCOCO, +12.1 gIoU on MUSE), with margins that grow as the number of referred targets increases. The concept interface also transfers to
video under a detect-and-track setting, yielding new state-of-the-art results on seven referring
video benchmarks, including +10.9 J&F on MeViS and +12.4 J&F on Ref-SeCVOS.
Read the full abstract ▾
All authors Zhixiong Zhang, Yizhuo Li, Shuangrui Ding, Yuhang Zang, Shengyuan Ding, Long Xing, Yibin Wang, Qiaosheng Zhang, Jiaqi Wang
Computer Vision
MetaEarth-MM: Unified Multimodal Remote Sensing Image Generation with Scene-centered Joint Modeling
Zhiping Yu, Chenyang Liu, Jinqi Cao et al. (+4 more)
Multi-modal remote sensing images are vital for Earth observation, yet complete paired observations are often scarce in practice. Existing generative methods commonly address this problem through isolated pairwise modality translation, but their versatility and scalability remain limited as the number of modalities and generation tasks increases. Here, we develop a generative foundation model MetaEarth-MM for multi-modal remote sensing imagery, enabling paired joint generation and any-to-any translation across five modalities within a unified model. Recognizing the intrinsic scene consistency underlying multi-modal observations, we introduce a scene-centered joint modeling paradigm in MetaEarth-MM. Unlike previous methods that rely on direct appearance-level cross-modal mapping, our model organizes the generation around the underlying scene content. Specifically, MetaEarth-MM adopts a decoupled architecture that first infers a latent scene representation from available observations, and then generates target modalities conditioned on this intermediate state. To support training, we further construct EarthMM, a large-scale
dataset comprising 2.8 million multi-resolution global images with 2.2 million aligned pairs. Extensive experiments demonstrate that MetaEarth-MM not only exhibits strong generative capability and robust generalization across diverse generation tasks, but also supports downstream tasks at both data and representation levels, highlighting its potential as a general foundation model for cross-modal Earth observation. The code and
dataset will be available at https://github.com/YZPioneer/MetaEarth-MM.
Read the full abstract ▾
All authors Zhiping Yu, Chenyang Liu, Jinqi Cao, Qinzhe Yang, Siwei Yu, Zhengxia Zou, Zhenwei Shi
Computer Vision
Spatially Prompted Visual Trajectory Prediction for Egocentric Manipulation
Yifan Li, Xinyu Zhou, Yunhao Ge et al. (+2 more)
Language instructions offer a flexible way to specify task semantics but can become ambiguous in cluttered scenes containing multiple visually similar objects and candidate targets. We study Spatially Prompted Visual Trajectory Prediction (SP-VTP), an open-loop setting in which points or bounding boxes in the first
egocentric frame specify an object–target pair and condition the prediction of future 3D end-effector (EE) trajectory chunks. To support this setting, we introduce EgoSPT, a controlled
dataset comprising 2,841
egocentric fork pick-and-place episodes with first-frame object and target annotations and reconstructed 3D EE trajectories. SP-VTP evaluates whether a static task specification remains informative as the viewpoint and scene configuration evolve throughout execution. We also introduce SPOT, a spatially prompted object–target trajectory predictor that integrates first-frame visual and coordinate
prompts with current visual observations and motion history to generate future trajectories. Rather than relying on an explicit object tracker, SPOT maintains a fixed representation of the first-frame task specification. Across three training seeds, spatial prompting consistently reduces translation and endpoint errors relative to a no-
prompt baseline, while controlled
prompt interventions reveal distinct roles for object and target cues at different stages of execution. Together, EgoSPT and SPOT provide a
dataset and baseline for spatially conditioned
egocentric trajectory prediction in open-loop pick-and-place settings.
Read the full abstract ▾
All authors Yifan Li, Xinyu Zhou, Yunhao Ge, Xiaobo Tan, Yu Kong
Computer Vision
VL-DPO: Vision-Language-Guided Finetuning for Preference-Aligned Autonomous Driving
Zhefan Xu, Ghassen Jerfel, Marina Haliem et al. (+3 more)
The rapid growth of autonomous driving
datasets has enabled the scaling of powerful motion
forecasting models. While large-scale pretraining provides strong performance, the standard imitation objective may not fully capture the complex nuances of human driving preferences. Meanwhile, recent advances in
vision-
language models (VLMs) have demonstrated impressive
reasoning and commonsense understanding. Building on these capabilities, this paper presents VL-DPO, a
vision-
language-guided framework that aligns ego-vehicle motion
forecasting models with human preferences. Our approach leverages a VLM as a zero-shot reasoner to automatically generate preference pairs from a pretrained model's rollouts, which are then used to finetune the model via Direct Preference Optimization (DPO). We finetune our models on the Waymo Open End-to-End Driving
Dataset (WOD-E2E) and evaluate performance against held-out human preference annotations using rater feedback score (RFS) and average displacement error (ADE). Our experiments confirm that the VLM's trajectory selection is a high-quality proxy for human preference. Our final model, VL-DPO, yields an 11.94% increase in RFS and a 10.01% reduction in ADE over the pretrained model.
Read the full abstract ▾
All authors Zhefan Xu, Ghassen Jerfel, Marina Haliem, Qi Zhao, Jeonhyung Kang, Khaled S. Refaat
Computer Vision
Probability-Conserving Flow Guidance
Parsa Esmati, Junha Hyung, Amirhossein Dadashzadeh et al. (+2 more)
Diffusion and flow-based generative models dominate visual synthesis, with guidance aligning samples to user input and improving perceptual quality. However, Classifier-Free Guidance (CFG) and extrapolation-based methods are heuristic linear combinations of velocities/scores that ignore the generative
manifold geometry, breaking probability conservation and driving samples off the learned
manifold under strong guidance. We analyse guidance through the continuity equation and show its effect decomposes into a divergence term and a score-parallel term defined invariantly across parameterizations. We prove the divergence term blows up structurally as sampling approaches the data
manifold, motivating a time-dependent schedule alongside score-parallel attenuation. The resulting plug-and-play rule, Adaptive
Manifold Guidance (AdaMaG), targets both terms at no additional inference cost. Finally, we show that several existing saturation and
manifold preserving heuristics can be read as acting on one of these two terms. On image generation
benchmarks, AdaMaG improves realism, reduces visible artefacts, and induces controlled desaturation under strong guidance.
Read the full abstract ▾
All authors Parsa Esmati, Junha Hyung, Amirhossein Dadashzadeh, Jaegul Choo, Majid Mirmehdi
Computer Vision
X-Ray cardiac angiographic vessel segmentation based on pixel classification using machine learning and region growing
E O Rodrigues, L O Rodrigues, J J Lima et al. (+6 more)
This work proposes a pixel-classification approach for vessel
segmentation in x-ray angiograms. The proposal uses textural features such as anisotropic
diffusion, features based on the Hessian
matrix, mathematical morphology and statistics. These features are extracted from the neighborhood of each pixel. The approach also uses the ELEMENT methodology, which consists of creating a pixel-classification controlled by region-growing where the result of the classification affects further classifications of pixels. The Random Forests classifier is used to predict whether the pixel belongs to the vessel structure. The approach achieved the best accuracy in the literature (95.48%) outperforming unsupervised state-of-the-art approaches.
Read the full abstract ▾
All authors E O Rodrigues, L O Rodrigues, J J Lima, D Casanova, F Favarim, E R Dosciatti, V Pegorini, L S N Oliveira, F F C Morais
Computer Vision
Cardiac fat segmentation using computed tomography and an image-to-image conditional generative adversarial neural network
Guilherme Santos da Silva, Dalcimar Casanova, Jefferson Tales Oliva et al. (+1 more)
In recent years, research has highlighted the association between increased adipose tissue surrounding the human heart and elevated susceptibility to cardiovascular diseases such as atrial fibrillation and coronary heart disease. However, the manual
segmentation of these fat deposits has not been widely implemented in clinical practice due to the substantial workload it entails for medical professionals and the associated costs. Consequently, the demand for more precise and time-efficient quantitative analysis has driven the emergence of novel computational methods for fat
segmentation. This study presents a novel deep learning-based methodology that offers autonomous
segmentation and quantification of two distinct types of cardiac fat deposits. The proposed approach leverages the pix2pix network, a generative conditional
adversarial network primarily designed for image-to-image translation tasks. By applying this network architecture, we aim to investigate its efficacy in tackling the specific challenge of cardiac fat
segmentation, despite not being originally tailored for this purpose. The two types of fat deposits of interest in this study are referred to as epicardial and mediastinal fats, which are spatially separated by the pericardium. The experimental results demonstrated an average accuracy of 99.08% and f1-score 98.73 for the
segmentation of the epicardial fat and 97.90% of accuracy and f1-score of 98.40 for the mediastinal fat. These findings represent the high precision and overlap agreement achieved by the proposed methodology. In comparison to existing studies, our approach exhibited superior performance in terms of f1-score and run time, enabling the images to be segmented in real time.
Read the full abstract ▾
All authors Guilherme Santos da Silva, Dalcimar Casanova, Jefferson Tales Oliva, Erick Oliveira Rodrigues
Computer Vision
OP2GS: Object-Aware 3D Gaussian Splatting with Dual-Opacity Primitives
Guiyu Liu, Niklas Vaara, Janne Mustaniemi et al. (+2 more)
3D Gaussian Splatting (3DGS) provides an explicit and efficient scene representation, but its primitives lack inherent object-level identity, hindering downstream tasks such as open-vocabulary scene understanding. Existing methods typically address this by either distilling high-dimensional feature
embeddings into Gaussians or by lifting 2D mask labels into 3D via heuristic refinement. However, feature-based approaches incur heavy storage and decoding overhead, while lifting-based pipelines remain vulnerable to label contamination: Gaussians necessary for appearance reconstruction often receive incorrect object labels during 2D-to-3D projection. We propose OP2GS, an object-aware Gaussian representation that augments each primitive with an explicit instance identity and a dedicated instance opacity $\sigma^{*}$ for object-mask rendering. The original opacity $\sigma$ remains responsible for visual reconstruction, while $\sigma^{*}$ models whether a Gaussian should contribute to a particular object mask. This dual-opacity formulation decouples visual existence from instance occupancy: mislabeled Gaussians can remain available for image rendering while becoming transparent in the object-mask branch. To learn this representation, we introduce a random object loss that optimizes the 1D instance occupancy field using the standard transmittance-based visibility of 3DGS. Semantic descriptors are then attached at the object level through multi-view aggregation, eliminating per-Gaussian feature storage. Compared with feature-training approaches, OP2GS achieves competitive open-vocabulary performance while significantly reducing computational overhead. Compared with training-free pipelines, it leverages physically consistent occupancy learning to resolve visibility ambiguities.
Read the full abstract ▾
All authors Guiyu Liu, Niklas Vaara, Janne Mustaniemi, Juho Kannala, Janne Heikkilä
Computer Vision
Stage-adaptive Token Selection for Efficient Omni-modal LLMs
Zijie Xin, Jie Yang, Ruixiang Zhao et al. (+4 more)
The unified audio-visual understanding capability of omni-modal large
language models (om-LLMs) comes at the high cost of processing tens of thousands of visual and audio
tokens throughout the LLM. To reduce the cost, training-free methods for om-LLM
token selection are being actively developed, operating either pre-LLM or inner-LLM. Recognizing that the non-textual
token redundancy varies across stages and LLM layers, we propose in this paper a novel stage-adaptive
token selection (SEATS) method that operates in both pre-LLM and inner-LLM stages. Given an overall
token budget measured by the
token retention rate (TRR), SEATS performs top-down
token budget allocation in an input-dependent manner. Specifically, a larger TRR is used for pre-LLM
token selection, whilst within the LLM, all
tokens are retained in shallow layers, increasingly decreased TRRs are applied in middle layers, and the non-textual
tokens are fully removed in late layers. Extensive experiments on three representative om-LLMs and six
benchmarks verify the viability of the proposed method. By retaining only 10% of visual and audio
tokens, SEATS preserves 94.9% of the full-
token performance on Qwen2.5-Omni while achieving a 9.3x FLOPs reduction and a 4.9x prefill speedup.
Read the full abstract ▾
All authors Zijie Xin, Jie Yang, Ruixiang Zhao, Tianyi Wang, Fengyun Rao, Jing Lyu, Xirong Li
Computer Vision
A Nash Equilibrium Framework For Training-Free Multimodal Step Verification
Rohit Sinha, Kunal Tilaganji, Tanuja Ganu et al. (+3 more)
Multimodal large
language models often generate
reasoning chains containing subtle errors that lead to incorrect answers. Current verification approaches have notable limitations. Learned critics need extensive labeled data and show inconsistent performance across different tasks. Meanwhile, existing training-free methods simply average scores from different sources, missing a key insight: when these scores disagree, that disagreement itself carries important information about whether a
reasoning step is truly valid or not. We propose a training-free verification approach that treats step-wise verification as a coordination problem among specialized judges. We formalize these judges' interaction as a Nash
equilibrium game where agreement signals valid steps while disagreement reveals instability. Our method computes
equilibrium scores through a closed-form solution, enabling both disagreement-aware filtering and stability-conscious ranking of
reasoning steps. Evaluated across six
benchmarks, our approach achieves consistent improvements of 2.4% to 5.2% over baseline models and shows competitive performance against learned critics, demonstrating that cross-modal agreement (not just average confidence) provides robust verification signals without task-specific adaptation.
Read the full abstract ▾
All authors Rohit Sinha, Kunal Tilaganji, Tanuja Ganu, Nagarajan Natarajan, Amit Sharma, Vineeth N. Balasubramanian
Computer Vision · Cross-listed from eess.IV
FGSVQA: Frequency-Guided Short-form Video Quality Assessment
Xinyi Wang, Angeliki Katsenou, Junxiao Shen et al. (+1 more)
Short-form
video poses new challenges to the quality assessment of user-generated content (UGC) due to its complex generation pipeline, rapid content variation, and mixed distortions. To address this challenge, we propose an end-to-end
video quality assessment (VQA) framework that employs a dense visual encoder based on CLIP, and incorporates compression priors derived from the frequency domain to generate artifact- and structure-aware weight maps for feature aggregation. By explicitly decomposing artifact, structure, and original visual feature branches and adaptively fusing them over time through a learned gating module, the proposed method achieves accurate and efficient quality prediction. Experimental results show that our method achieves strong performance on short-form
video datasets in terms of average rank and linear correlation (SRCC: 0.736, PLCC: 0.787), while maintaining efficient inference runtime. The code and additional results are available at: https://github.com/xinyiW915/FGSVQA.
Read the full abstract ▾
All authors Xinyi Wang, Angeliki Katsenou, Junxiao Shen, David Bull
Computer Vision
CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition
Hongji Yang, Songlian Li, Yucheng Zhou et al. (+4 more)
Recent
diffusion models achieve strong photorealism and fluency in
video generation, yet remain fragile under abstract, sparse or complex conditions, leading to poor performance in professional production workflows such as storyboard sketches and clay render conditions. Existing
video generation models, either inject conditions through adapters or couple a generic
vision-
language model (VLM) within a
diffusion backbone, leaving a capability gap and failing to produce the
videos that align with the user's creative intent. We present CogOmniControl, a
reasoning-driven framework that factorizes controllable
video generation into creative intent cognition and generation. Specifically, we train a specialized CogVLM using authentic anime production data. Compared to generic VLMs, it generates more professional and clear outputs, accurately cognizing user creative intent from sparse and abstract conditions and tuning these cues into dense
reasoning output. Besides, CogOmniDiT unifies the controls from various conditions through in-context generation and is aligned to the CogVLM
reasoning outputs via reinforcement learning. Furthermore, leveraging CogVLM's robust capability in guiding
video generation, we release its potential in planning specific evaluators and enable a Best-of-N selection for the generated
videos. This integration transforms the entire framework into a closed-loop "harness-like" architecture. We further introduce CogReasonBench and CogControlBench, built from professional workflows data that carry genuine creative intent rather than simulated ones. Experiments on two
benchmarks show that CogOmniControl surpassed the existing open-source models. The project website: https://um-lab.github.io/CogOmniControl/
Read the full abstract ▾
All authors Hongji Yang, Songlian Li, Yucheng Zhou, Xiaotong Zhao, Alan Zhao, Chengzhong Xu, Jianbing Shen
Computer Vision · Cross-listed from cs.RO
Minimalist Visual Inertial Odometry
Francesco Pasti, Jeremy Klotz, Nicola Bellotto et al. (+1 more)
Visual-Inertial Odometry (VIO), which is critical to mobile
robot navigation, uses cameras with a large number of pixels. Capturing and processing camera images requires significant resources. This work presents a minimalist approach to planar odometry, showing that just four visual sensors (pixels) and an IMU provide robust motion estimation for differential-drive
robots. Our key insight is that four downward-facing photodiodes that sense the world through optical Gabor masks produce signals that encode speed. Based on this, we jointly optimize the mask parameters alongside a Temporal Convolutional Network (TCN) using a physically-grounded simulator. The resulting model decodes speed from the four photodiode measurements. Pairing these estimates with an IMU's angular speed yields a continuous planar trajectory. We validate our approach with a prototype sensor mounted on a differential drive
robot. Across diverse indoor and outdoor terrains, our system closely tracks the reference trajectories without any real-world fine-tuning. Our work shows that minimalist sensing enables efficient and accurate planar odometry.
Read the full abstract ▾
All authors Francesco Pasti, Jeremy Klotz, Nicola Bellotto, Shree K. Nayar
Computer Vision · Cross-listed from cs.RO
Beyond Binary Success: A Diagnostic Meta-Evaluation Framework for Fine-Grained Manipulation
He-Yang Xu, Pengyuan Zhang, Zongyuan Ge et al. (+5 more)
Fine-grained manipulation marks a regime where global scene context no longer suffices, and success hinges on the tight coupling of local attribute grounding, high-fidelity spatial perception, and constraint-respecting motor execution. However, current
embodied AI
benchmarks collapse these capacities into binary success rates, systematically inflating reported capabilities by up to 70% and masking the architectural bottlenecks that impede real-world deployment. We introduce MetaFine, a diagnostic meta-evaluation framework that disentangles manipulation competency along three axes: understanding, perception, and controlled behavior. Built on a compositional task graph, MetaFine absorbs heterogeneous external
benchmarks and reconstructs them into diagnostic scenarios of varying complexity under a unified protocol. Evaluating state-of-the-art
vision-
language-action (VLA) models through this lens exposes severe dimension-specific failures invisible to conventional metrics. Through targeted causal intervention, we identify the visual encoder's ability to preserve local spatial structure as a key bottleneck for fine-grained precision: improving it directly unlocks previously inaccessible manipulation capabilities without modifying downstream policies. MetaFine further supports hybrid real-sim validation, using limited paired real-world rollouts to calibrate scalable simulation-based estimates for more stable physical benchmarking. By shifting evaluation from ranking to diagnosis, MetaFine turns benchmarking into an actionable compass for repairing the layered capacities underlying genuine physical dexterity. The MetaFine framework,
benchmarks, and supporting resources will be publicly released at our project page: https://metafine.github.io/.
Read the full abstract ▾
All authors He-Yang Xu, Pengyuan Zhang, Zongyuan Ge, Xiaoshuai Hao, Serge Belongie, Xin Geng, Yuxin Peng, Xiu-Shen Wei