Computer Vision
RARF: Region-Aware Rectified Flows for 3D Brain MRI Inpainting
Tomas Guija-Valiente, Blanca Rodriguez-Gonzalez, Norberto Malpica et al. (+1 more)
Medical image inpainting has the potential to improve automated
brain MRI analysis by reconstructing healthy tissue within pathological regions. We introduce RARF, a task-agnostic region-aware rectified flow framework for masked data generation. We instantiate the framework for 3D
brain MRI inpainting as our submission to the BraTS Inpainting Challenge 2026. RARF restricts the
stochastic interpolation process to the inpainting region, while the observed voxels remain fixed and provide patient-specific anatomical context. A three-dimensional neural network receives the partially voided image, with Gaussian noise filling the missing region, together with the inpainting mask and the corresponding timestep. The model is trained using masked flow-matching and reconstruction-consistency objectives, combined with mask-aware preprocessing and data augmentation. During inference, the learned velocity field transports the initial noise toward a plausible reconstruction of the missing tissue, which is then combined with the unchanged observed anatomy. Experiments under the BraTS evaluation protocol show that the proposed approach produces competitive reconstructions while maintaining anatomical consistency. Source code is available at: https://github.com/TomasGuija/rarf.
Read the full abstract ▾
All authors Tomas Guija-Valiente, Blanca Rodriguez-Gonzalez, Norberto Malpica, Angel Torrado-Carvajal
Computer Vision
WorldReward: Reward Modeling for Camera-Conditioned World Models
Yibin Wang, Zehan Wang, Junshu Tang et al. (+13 more)
Camera-conditioned world models generate interactive
videos in which commanded actions should induce the expected scene changes while appearance, geometry, and temporal dynamics remain coherent. Existing rewards assess these requirements separately: geometry-based rewards estimate trajectory execution but cannot judge the visual quality of the executed motion, whereas image-based rewards measure frame quality without capturing action execution or temporal dynamics. We posit that a
vision-
language model (VLM) offers a shared
reasoning space for relating actions to their visual outcomes. However, judging a complete long
video against its full action sequence creates a lengthy, noisy context in which short-lived local action evidence can be missed or diluted. We present WorldReward, a VLM-based pairwise preference reward model that unifies action-consistency and visual-quality evaluation for camera-conditioned world models. WorldReward decomposes paired
videos into action-aligned chunks, organizes each chunk into structured visual evidence, and aggregates chunk-level decisions by voting into separate
video-level action and visual-quality preferences. To train it, we construct a large-scale
reasoning-augmented preference
dataset using structured judgments generated by a frontier VLM and refined through tool-based
agent auditing and targeted human review. We further introduce WorldReward-Bench, a human-annotated
benchmark measuring reward-model agreement with human preferences across action consistency, appearance quality, and motion quality. WorldReward achieves the highest agreement on all three dimensions, exceeding GPT-5.5 by 3.42, 1.45, and 3.56 percentage points, respectively. When used for RL post-training of HY-WorldPlay 1.5, it consistently improves both action execution and visual quality across short- to long-term horizons.
Read the full abstract ▾
All authors Yibin Wang, Zehan Wang, Junshu Tang, Zhimin Li, Yujie Zhou, Jiazi Bu, Pengyang Ling, Feng Han, Zhixiong Zhang, Long Xing, Shengyuan Ding, Ziang Li, Cheng Jin, Yuhang Zang, Jiaqi Wang, Tianyu Pang
Computer Vision
Sparse auto-regressive modeling for scene generation from multi-view images
Thomas Lucas, Maxime Pietrantoni, Philippe Weinzaepfel et al. (+4 more)
Generating complete 3D scenes from sparse, unconstrained views is a fundamental challenge in 3D
vision which requires
reasoning beyond observed content while remaining computationally tractable. Existing feed-forward reconstruction methods are inherently limited to content visible in the input images, while 3D generative modeling is hindered by the high computational cost of dense volumetric representations and the scarcity of large-scale 3D supervision. We introduce SPAR3S, a sparse voxel-aligned 3D latent generative model for conditional scene completion without requiring ground-truth 3D data for supervision. Our key insight is to formulate 3D scene generation in a structured, compact, voxel-aligned 3D latent space where only occupied voxels are represented. We learn this sparse latent space directly from multi-view images using photometric supervision via differentiable 3D Gaussian Splatting. Given a partial set of observed voxels encoded from sparse input views, scene completion reduces to predicting the missing latent
tokens and their spatial support within the voxel grid. To this end, we train a masked autoregressive
transformer that jointly models voxel occupancy and latent
token values, enabling efficient and spatially consistent generation of unseen regions. We demonstrate the effectiveness of our method on synthetic indoor scenes, achieving higher novel-view quality than prior work. We further validate its generalization on RealEstate10k, highlighting its applicability to real-world data.
Read the full abstract ▾
All authors Thomas Lucas, Maxime Pietrantoni, Philippe Weinzaepfel, Wonjune Cho, Bardienus Pieter Duisterhof, Vincent Leroy, Jerome Revaud
Computer Vision
OctWorld: Long-Range World-Consistent Video Generation with Octree-Based 3D Mapping
Zelong Lv, Sicheng Xu, Jianfeng Xiang et al. (+5 more)
We present OctWorld, a
video diffusion framework with persistent 3D
memory for generating explorable, world-consistent, and high-fidelity visual scenes. Given a single image, OctWorld performs stable autoregressive world generation along user-specified camera trajectories. We focus on long-range generation, characterized by extended camera paths and wide viewpoint coverage, where preserving spatial consistency is particularly challenging when previously generated regions are revisited. To address this problem, we introduce OctMap, an extensible and spatially adaptive 3D
memory that progressively fuses generated visual observations and their corresponding depth maps into a global representation. OctMap employs TSDF fusion within a dynamic sparse octree whose spatial resolution adapts to image evidence. This design preserves
geometric and appearance details across diverse scene scales while maintaining low
memory overhead. Experiments demonstrate that OctWorld generates long-range, spatially consistent
videos and outperforms prior methods on both existing
benchmarks and challenging long-range generation settings. OctMap also provides clear advantages over point-based caches and fixed-resolution TSDF volumes. Project page: https://maxtirerror.github.io/octworldpage/
Read the full abstract ▾
All authors Zelong Lv, Sicheng Xu, Jianfeng Xiang, Ruicheng Wang, Yue Dong, Yu Deng, Guangzhong Sun, Jiaolong Yang
Computer Vision
Concept of a Sensor Test Environment for Dusty Agricultural Conditions
Peter Buckel, Johannes Hermann, Jonas Wollmann et al. (+2 more)
Dust in agriculture presents a significant challenge for autonomous agricultural machinery.
Dust can impair the performance of sensors and
algorithms. This work, therefore, presents a concept for a proving ground consisting of an indoor and outdoor area. The indoor area comprises a laboratory test bench where
dust circulates in a closed system and a test
hall where life-size objects can be placed. The outdoor area features dedicated test setups that enable reproducible data to be recorded with and without
dust during real-world agriculture work. The proving ground and the setups are visualized in 3D.
Read the full abstract ▾
All authors Peter Buckel, Johannes Hermann, Jonas Wollmann, Thomas Dietmueller, Timo Oksanen
Computer Vision
GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs
Junqing Du, Fernando Ropero, Erkin Turkoz et al. (+2 more)
3D spatial
reasoning underpins understanding and acting in the physical world, yet it remains unreliable in current
multimodal large
language models (MLLMs). These models falter at precise
geometric measurement, at transforming between
egocentric and allocentric viewpoints, and at grounding fine-grained appearance. The most common remedies fine-tune the model on large-scale curated spatial-
reasoning datasets or attach dedicated encoders for 3D geometry, which typically couples the solution to costly supervision and a specific backbone. We instead introduce GraFT, a training-free framework that supplies the missing 3D structure through a compact, easily maintained 3D scene graph (3DSG). GraFT supplies the graph's 3D structure to the frozen model in three forms: as values computed by symbolic tools, as a bird's-eye-view (BEV) the model reads as an image, and as
egocentric frames the geometry selects. On ScanQA, GraFT improves every metric over the same-backbone baseline, raising CIDEr by 27%. On VSI-Bench, GraFT improves frozen MLLMs by up to 65%, surpassing every proprietary and general-purpose open-source baseline, and several prominent fine-tuned spatial models.
Read the full abstract ▾
All authors Junqing Du, Fernando Ropero, Erkin Turkoz, Yanfeng Zhang, Lu Liu
Computer Vision
Phase-Aware Spatial-Frequency Fusion for Few-Shot Fine-Grained Image Classification
Ruiling Liu, Linyue Zhang, Wenyi Zeng et al. (+5 more)
Few-shot fine-grained image classification (FSFGIC) aims to classify similar images with limited labeled examples. This work highlights the critical yet underutilized role of phase information in capturing structural relationships within an image. This study introduces a novel plug-and-play amplitude-phase integration (API) module that effectively combines local and global frequency amplitude and phase information for obtaining more comprehensive feature descriptors. Additionally, a dedicated network, named PSF-Net, is proposed that adaptively fuses phase-based spatial and frequency information for FSFGIS. The designed PSF-Net can be easily integrated into standard episodic training architectures for end-to-end training from scratch. Extensive experiments on five public
datasets demonstrate that the method outperforms existing state-of-the-art
benchmarks.
Read the full abstract ▾
All authors Ruiling Liu, Linyue Zhang, Wenyi Zeng, Jiamiao Lu, Weichuang Zhang, Changming Sun, Zejun Zhang, Xiao Zhao
Computer Vision
VI3: Grounding Pretrained 3D Foundation Models with Inertial Cues
Ernesto Lozano, Alberto Jaenal and Javier Civera
3D foundation models (3DFMs) excel at predicting camera poses and dense depth from multiple views of a scene, showcasing strong zero-shot generalization. However, as metric scale is not observable from
monocular images, their absolute scale predictions are typically inaccurate. Inertial measurement units (IMUs), present in most devices, naturally complement
monocular cameras by observing scaled motion. We introduce VI3, a model-agnostic framework that metrically anchors a pretrained 3DFM using only IMU readings. VI3 initializes and preintegrates the IMU to obtain a metric motion reference, which is then used to recover the scale of the 3DFM outputs. Our method includes adaptable anchoring strategies tailored to diverse 3DFM architectures. Experiments on synthetic and real aerial
datasets demonstrate that VI3 recovers metric scale without ground-truth supervision while preserving
geometric consistency, acting as a fine refinement under well-conditioned motion and as a strong prior when motion is less informative.
Read the full abstract ▾
Computer Vision
Select, Compress, Reinvest: A Controlled Study of Visual-Token Allocation in Long-Video MLLMs
Prakhar Khatri
Long-
video language models cannot look at every frame: an hour sampled once per second is 3,600 images, and a system keeps only a small fixed slice of that pool. Which frames survive that slice is usually treated as a preprocessing detail; we test whether it should be. Published selectors make the comparison hard because they change the frame scorer, the
prompt boundary, the resolution policy, and the answering model all at once. We hold each fixed and vary one decision at a time: selection, spatial compression, and reinvestment of the savings, across six training-free selection rules, three long-
video benchmarks, and two answering models. Selection is the largest single lever: on LongVideoBench's hour-long bin, eight query-selected frames beat sixteen uniformly spaced ones by 6.9 points, and Orthogonal Matching Pursuit, an unmodified decades-old sparse-approximation
algorithm, matches or comes within a point of every purpose-built selector we compare it against, across all three
benchmarks. Compression is close to free: halving each frame's spatial budget at fixed timestamps costs at most 0.44 points. Reinvestment is where that budget turns back into accuracy: spending the freed
tokens on twice as many compressed frames, at a measured cost no higher than the original eight, returns a further two to three points; compression only pays off once its savings are spent this way. Along the way, an implementation bug in our own AKS baseline and a 0.07 to 3.74 point gap between two harnesses running the same published rules at the same budget show why these comparisons need to happen inside one controlled harness rather than across papers.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.SI
When Vision Meets Graphs: A Survey on Graph Reasoning and Learning
Xinjian Zhao, Wei Pang, Zhixuan Yu et al. (+8 more)
Graphs are a fundamental data structure underlying many problems in the natural and social sciences. Over the past decade, Graph Neural Networks (GNNs) have dominated graph machine learning, supported by solid theoretical foundations. Yet scientists often understand graph structure through
vision: chemists read molecular diagrams and social scientists inspect network visualizations. Despite decades of work on graph visualization, most graph learning pipelines still treat graphs purely as symbolic structures, rarely leveraging the visual form of graphs. We argue that this gap deserves renewed
attention in the era of powerful
vision and
vision-
language models. This survey provides a first systematic overview of the emerging area we term
vision meets graphs, which treats visual depictions of graphs as first-class inputs for
reasoning and learning. We organize existing work into three threads.
Vision for Graph
Reasoning studies how models can use visual depictions of graphs to understand structure and carry out multi-step
reasoning.
Vision for Graph Learning explores how visual features can complement or augment graph encoders beyond known limitations of message passing. Scientific Graphs examines domains where standardized depiction conventions support both
reasoning and learning. Our goal is to clarify what current methods can and cannot do, and to outline a path toward foundation models that perceive and reason about graphs as scientists do.
Read the full abstract ▾
All authors Xinjian Zhao, Wei Pang, Zhixuan Yu, Xiangru Jian, Xiaozhuang Song, Yaoyao Xu, Zhongkai Xue, Dingshuo Chen, Shu Wu, Philip Torr, Tianshu Yu
Computer Vision
SPARK: Input-Conditioned Sparse Activation Modulation for Frozen DiT-based Super-Resolution
Federico Putamorsi, Leonardo Zini, Marcella Cornia et al. (+1 more)
Real-world image super-resolution (SR) increasingly relies on
Diffusion Transformer (DiT) backbones, whose internal activations can be dominated by a small number of massive channels. Yet improving perceptual quality in these models still typically requires fine-tuning the network or attaching additional adapters, leaving this structured activation space largely unexplored for adaptation. We investigate whether dominant channels can instead serve as a compact adaptation interface for frozen DiT-based SR models. We first characterize their behavior in pretrained SR backbones and show through controlled interventions that they strongly affect reconstruction quality. Building on this observation, we introduce SPARK, a lightweight input-conditioned controller that predicts bounded per-channel affine transformations for only the selected channels, while keeping the SR backbone and VAE frozen. Dominant channels are identified through an online activation-ranking procedure, and only a small predictor conditioned on the low-resolution VAE latent is optimized. Experiments on three DiT-based SR backbones across DIV2K, RealSR, and DRealSR show consistent gains in both fidelity and perceptual quality while modulating only eight channels per stream and block. Controlled comparisons further show that these gains cannot be explained by parameter budget or access to the selected channels alone.
Read the full abstract ▾
All authors Federico Putamorsi, Leonardo Zini, Marcella Cornia, Lorenzo Baraldi
Computer Vision
VisCAD: A Foundation Model Suite with Multimodal Industrial CAD Intelligence
Guanlin Li, Zhichao Huang, Huimu Yu et al. (+9 more)
AI-assisted computer-aided design (CAD) for industrial products involves two challenging phases. Part-level generation maps diverse forms of user intent, including renders, text descriptions, 2D drawings, and real photographs, to executable programs in a CAD domain-specific
language. Assembly-level generation must additionally handle interacting parts, plan mating relations, estimate poses, and place all parts correctly. Existing specialized CAD models are commonly trained on narrow input domains, such as renders or texts, and often generalize poorly, while general-purpose frontier models cover broader inputs but perform inconsistently across CAD domains. We present VisCAD, a foundation model suite designed to provide both broad generalization and strong CAD capability for realistic industrial products. At its core is VisCAD-M1, a 27B model trained through mid-training and post-training for part-level design generation. On PubCADBench and RealCADBench, VisCAD-M1 achieves the highest average part-level score among the evaluated models, reaching 0.5540 compared with 0.5496 for the strongest frontier model. Reusing VisCAD-M1 as a test-time verifier can further raise the score to 0.5797, an approximately 5 percent relative improvement over the previous state of the art. VisCAD also includes a domain-specific harness that leverages frontier models for complex assembly generation and demonstrates advantages over general-purpose harnesses in both quantitative and qualitative evaluations.
Read the full abstract ▾
All authors Guanlin Li, Zhichao Huang, Huimu Yu, Yichen Long, Hongsen Liu, Linxin Cai, Qiuhe Hong, Ziqi Liu, Luya Wang, Wenxiang Wu, Ning Zhang, Yuchen Wang
Computer Vision · Cross-listed from cs.AI
SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation
Marco Cipriano, Leonardo Zini, Alexandra Schild et al. (+5 more)
Scalable Vector Graphics (SVG) generation is attracting increasing
attention as generative models improve in expressiveness and controllability. Progress, however, is held back by the lack of domain-specific evaluation protocols: current practice relies on metrics designed for natural images, most notably CLIPScore, which was never trained on vector graphics and aligns only partially with human judgment. We introduce \ours, a human-aligned evaluation framework for text-to-SVG generation. Through controlled caption and image perturbations, we first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations, and that off-the-shelf
Vision-
Language Model (VLM) judges, while more sensitive, respond unevenly across error types and SVG styles. We then introduce a human-annotated
dataset for Semantic
Alignment, measuring how faithfully a generated SVG reflects its caption. Building on it, we develop two complementary evaluators: CLIP scorers adapted to vector graphics and then aligned to human preferences, for fast large-scale evaluation, and a VLM judge trained with supervised fine-tuning and reward-shaped reinforcement learning, for more expressive and interpretable assessment. Using both, we
benchmark major open-source, commercial, and optimization-based SVG generators on an independent caption set.
Read the full abstract ▾
All authors Marco Cipriano, Leonardo Zini, Alexandra Schild, Valentin Teutschbein, Afsana Mimi, Marcella Cornia, Lorenzo Baraldi, Gerard de Melo
Computer Vision
Urban Boundaries, Social Barriers: A Benchmark and Vision-Centric Framework for Mapping Gated Communities and Equity Implications
Minwei Zhao, Weiming Zhang, Jiawang Du et al. (+4 more)
Communities are fundamental spatial units that shape urban form and social life. Whether a residential compound is spatially open or enclosed affects mobility, access to public services, and equity, yet studies of Chinese fengbi xiaoqu remain largely qualitative or small-scale, limiting reproducible city-scale analysis. We address this gap by introducing GBA-GCs, a metropolitan-scale
multimodal benchmark for locally grounded gated/open community recognition in China's Greater Bay Area, covering 37,444 residential compounds with aligned boundary polygons, high-resolution satellite imagery, Chinese metadata, and structured attributes, together with expert-verified labels, inter-annotator reliability, and official evaluation splits. Built on this
benchmark, we present
Multimodal Classifier for Gated Community (MCGC), a
vision-centric
multimodal framework based on DINOv3-SAT that fuses imagery, text, and structured cues via modality-aware cross-
attention and adaptive gating to mitigate modality imbalance. MCGC consistently outperforms strong unimodal and
multimodal baselines. Finally, we apply the validated model to metropolitan-scale mapping and report equity-oriented findings including spatial clustering of GCs, privatized green space, and reduced pedestrian connectivity. The
benchmark, code, and release documentation are available at https://github.com/MinweiZhao/GBA-GCs.
Read the full abstract ▾
All authors Minwei Zhao, Weiming Zhang, Jiawang Du, Qiming Liu, Weiming Zhuang, Pei Nie, Cai Wu
Computer Vision
LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes
Chuyan Chen, Haoxing Chen, Kun Chen et al. (+27 more)
We introduce LLaDA-Image, a unified framework that pairs a 6B
Diffusion Transformer (DiT) trained from scratch with a frozen
vision-
language understanding module built on the LLaDA2.0-Mini
diffusion language model backbone. Instead of relying heavily on paired image-text data from the beginning, we first build a strong visual generative prior through image-only pre-training and mid-training. The generation pipeline comprises 220M samples, 98 of which are real images. For efficient and scalable optimization, we use parameter-free RMSNorm throughout the DiT together with the
Muon optimizer. The resulting unified model produces highly photorealistic images while accurately following fine-grained editing instructions. We further distill LLaDA-Image into LLaDA-Image-Turbo, enabling fast inference in 2-4 sampling steps. On Qwen-Image-Bench, LLaDA-Image achieves overall scores of 53.53 and 53.38 on the English and Chinese tracks, respectively, setting a new state-of-the-art among open-source models on both tracks. To support further research on capable and efficient generative models, we release our model weights, training code, and detailed recipes.
Read the full abstract ▾
All authors Chuyan Chen, Haoxing Chen, Kun Chen, Zhenglin Cheng, Long Cui, Ruishan Fang, Zhangxuan Gu, Zhicheng Huang, Zhenzhong Lan, Yuanting Lei, Haoquan Li, Jianguo Li, Rongchuan Li, Sidu Li, Tao Lin, Deyuan Liu, Jiacheng Liu, Lin Liu, Yuxuan Lou, Zhisheng Lu, Yuxin Ma, Shuheng Shen, Peng Sun, Chaoyang Wang, Hongjun Wang, Xiaomei Wang, Yongxin Wang, Chengzhang Wu, Hongru Wu, Jun Xie
Computer Vision
A Reverse Sign Language Dictionary: Open-Vocabulary Sign Recognition from Continuous Signing via Video Captioning and Description Retrieval
Santiago Poveda-Gutiérrez, Hideki Nakayama and Mayumi Bono
Isolated Sign
Language Recognition (ISLR) is conventionally cast as closed-set classification over gloss labels, which cannot generalize to signs unseen in training and ties every deployment to a gloss-annotated lexicon. We instead recognize signs extracted from continuous signing by (1) captioning a sign-level clip into a free-form procedural description of the articulation with an open-weight
vision-
language model, and (2) retrieving the closest entry from a vocabulary of target descriptions with a
multilingual sentence encoder: a reverse sign
language dictionary that needs no gloss supervision and admits an open vocabulary. On 1,300 sign-level segments from a Japanese Sign
Language (JSL) dialogue corpus annotated with procedural descriptions (against a 2% top-10 chance floor over the 503-entry target vocabulary), fine-tuning the captioner substantially improves seen-class
retrieval:
language and
vision tower fine-tuning raises top-10
retrieval on seen classes from 4.5% (untrained) to 49%, becoming statistically indistinguishable from a standard supervised closed-set classifier (I3D) on two of the three test sets where a closed-set classifier can be evaluated at all. More importantly, unseen-class
retrieval also improves significantly over the untrained pipeline (11.5% -> 21.0% top-10, p=0.0094), a regime in which the closed-set classifier cannot participate. A matcher-side empirical upper-bound analysis shows the sentence encoder already recovers close to 100% of paraphrased gold descriptions, locating a gap in captioning quality that we aim to address in future work. To our knowledge this is the first description-based, open-vocabulary sign lookup from continuous signing without gloss supervision, and the first for JSL.
Read the full abstract ▾
Computer Vision
RealCADBench: Benchmarking Parametric CAD Modeling from Industrial Design Intents
Guanlin Li, Zhichao Huang, Huimu Yu et al. (+8 more)
Parametric computer-aided design (CAD) modeling is difficult to evaluate with a single metric. Existing CAD
benchmarks often emphasize synthetic or CAD-native settings, limited input modalities, or executability and IoUs alone. We introduce RealCADBench, a
benchmark for intent-to-program CAD modeling from real industrial design intents. It contains 12,632 tasks from 19 factory-automation categories and spans text descriptions, 2D engineering drawings, real product pictures, and rendered images for both Part and Assembly modeling. We report results on a 1,770-task evaluation slice: 1,745 Part tasks across four input regimes and RCB-Assm25, a 25-task assembly study used in every reported assembly comparison. Each method generates FreeCAD API Python, which a shared runtime executes to export the 3D model. We evaluate the exported model using executability, Solid IoU, Surface IoU, and a rubric-based visual-semantic identity Judge. Among the nine standalone frontier large models evaluated, no model leads all four metrics. Across six frontier-scale large models, executability ranges from 0.565 to 0.812, Solid IoU from 0.2841 to 0.5379, and Surface IoU from 0.112 to 0.217 across the four Part regimes. The highest regime-balanced composite comes from a different model than the leaders on the four component metrics. On RCB-Assm25, Codex with GPT-5.5 improves executability and both IoU metrics over standalone GPT-5.5, but lowers the Judge score by 6.98 percentage points, leaving GPT-5.5 as the Judge leader. We also observe recurring failure modes, most notably missing fine structures, loss of part identity, and incorrect assembly placement. These results show that execution alone is insufficient to characterize realistic CAD modeling and that frontier models and
agents differ substantially across executability, IoUs, and visual-semantic identity.
Read the full abstract ▾
All authors Guanlin Li, Zhichao Huang, Huimu Yu, Yichen Long, Hongsen Liu, Linxin Cai, Qiuhe Hong, Zongzhen Li, Wei Wang, Xianwen Zhong, Yuchen Wang
Computer Vision
ENEAS: Embedding-guided Neural Ensemble for Adaptive Segmentation
Javier del Pino, Salvador Rodríguez, Alejandro Garabito et al. (+2 more)
We present ENEAS, a unified, text-promptable method for instance tracking and semantic discovery. Text-promptable
segmentation models, including the latest foundation models such as SAM 3, still suffer from temporal
hallucinations, spatial fragmentation, and semantic misclassification: they fail to report target absence when an object leaves the field of view, segment local textures instead of the complete object during extreme close-ups, and prioritize visual features over ontological reality, so that visually similar artifacts such as statues, paintings, or reflections are segmented as target entities. ENEAS works two ways from a single method: precise tracking and high-quality
segmentation of a unique instance, and open-concept discovery of every instance a text query names, resolved by a semantic verification layer. For tracking, we extend the geometrically robust SeC architecture, previously limited to point interactions, with a text-prompting adapter and leverage its temporal
memory, so that the target is held through disappearance without drifting to distractors and kept whole even when it fills the entire view. For discovery, the verification layer combines high-speed visual
embedding matching with conditional VLM refinement, invoking semantic
reasoning only for ambiguous candidates, which filters out the ontological errors that visual-only models cannot distinguish while keeping latency low. Designed with 3D reconstruction in mind, where a single misclassified distractor corrupts the asset, ENEAS unlocks high-quality semantic tracking and
segmentation of
video, of broad libraries, and of collections of temporally or spatially unordered data, together with the discrimination to tell true instances from their doppelgangers: things that look alike but are not the same. The code and models are available at https://github.com/speridlabs/eneas
Read the full abstract ▾
All authors Javier del Pino, Salvador Rodríguez, Alejandro Garabito, Javier Álvarez, Chema Garabito
Computer Vision
KnowVis: Knowledge-Centric Visual Summarization for Video Lectures
Yi Xu, Yifan Hou and Xiaoyu Zhang
Video lectures are valuable educational resources, but their dense and lengthy formats often overwhelm novice learners. This difficulty stems from a fundamental pedagogical mismatch: while
videos deliver transient information linearly, human learning requires constructing interconnected cognitive networks, a task that induces severe cognitive overload for novice learners lacking prior domain knowledge. Existing
video summarization methods fail to resolve this mismatch, as they primarily produce text-heavy, linear condensations that still demand high cognitive effort. To bridge this gap, we propose KnowVis, a framework that transforms linear
video lectures into pedagogically grounded visual narratives. KnowVis first extracts a detailed concept map from
multimodal video content to identify important and challenging threshold concepts, then constructs structured knowledge units, and finally synthesizes engaging visual summaries. Alongside the framework, we introduce a curated
dataset of 125 educational
videos across 10 academic disciplines, paired with 1,079 generated visual summaries. Extensive automated evaluations and a human study demonstrate that, compared to state-of-the-art baselines, KnowVis generates more accurate and clear visuals that successfully reduce cognitive load and significantly improve student learning effectiveness and knowledge retention.
Read the full abstract ▾
Computer Vision
Fill My Mirror: Geometry-Constrained Mirror Inpainting
Ofek Basson, Shimon Vainer, Yacov Hel-Or et al. (+1 more)
Mirrors are common in real-world images, yet producing geometrically consistent reflections with generative models remains challenging. Unlike most objects, mirror appearance depends on scene geometry and viewpoint, making it hard to synthesize using learned appearance priors alone. We address this in the mirror inpainting setting, where the scene is fixed and only the mirror region is generated. Our key insight is that much mirror content is geometrically constrained by the visible scene and need not be hallucinated. We estimate scene geometry and project visible content into the mirror to recover reflection regions determined by geometry. A generative model then completes the mirror region via a two-mask
diffusion strategy balancing
geometric constraints with the model's learned priors, reducing projection artifacts and improving reflection consistency. The method is training-free and applicable to complex real-world scenes. We evaluate on MirrorBench-V2 (synthetic) and real images. Using standard and geometry-aware metrics, we show that explicitly using scene geometry improves consistency.
Read the full abstract ▾
All authors Ofek Basson, Shimon Vainer, Yacov Hel-Or, Ohad Fried
Computer Vision
Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning
Yijun Yang, Shenghe Zheng, Wenbo Li et al. (+8 more)
Despite the remarkable prowess of
Vision-
Language Models (VLMs) in general
multimodal tasks, they remain fundamentally “flat” when
reasoning about the physical world. We argue that this spatial bottleneck stems from a profound dimensional mismatch: while VLMs are trained to interpret 2D projections, true spatial
reasoning demands the recovery of latent 3D geometry and temporal continuity. To conquer this high-dimensional complexity, we advocate a shift from monolithic learning to a “divide and conquer” paradigm. We present FactoSR, a factorized reinforcement learning framework that explicitly interpret the dimensions collapsed by visual projection. At its core, FactoSR decomposes the monolithic problem of world-consistent
reasoning into three orthogonal,
geometric sub-objectives: planar correspondence ($XY$), depth consistency ($Z$), and temporal reversibility ($T$). By optimizing these verifiable constraints within a unified policy learning mechanism, we effectively transform an ill-posed projection recovery problem into a series of tangible
reasoning steps. Extensive evaluations on multi-view and
video benchmarks demonstrate that this elegant decomposition yields substantial gains in 3D and 4D
reasoning, achieving a 5.9% boost on VSI-Bench and 4.5% on All-Angles-Bench. Our findings suggest that reinforcing explicit, factorized 4D consistency is a critical step toward evolving VLMs into robust, world-aware reasoners.
Read the full abstract ▾
All authors Yijun Yang, Shenghe Zheng, Wenbo Li, Jianhui Liu, Haoze Sun, Yanbing Zhang, Jiaxiu Jiang, Lin Song, Haoyang Huang, Nan Duan, Lei Zhu
Computer Vision
SignSeek: Learning Transferable Representations for Sign Dictionary Retrieval
Sobhan Asasi, Ozge Mercanoglu Sincan and Richard Bowden
Sign
language dictionaries are essential resources for sign
language learners, yet automatically retrieving a sign from a dictionary, given only a query
video, remains a challenging problem due to the natural variability between signers. Existing sign representation learning methods are built for closed-set recognition, producing
embeddings that do not generalise to the open-set, signer-independent setting that
retrieval demands. SignSeek closes this gap by contrastively learning sign representations with saliency-guided articulator masking. A contrastive objective aligns same-gloss signs across signers, while our Articulator Saliency-Guided Masking (ASGM) pinpoints the single most critical articulator per sign. This drives two complementary objectives, a masked contrastive
alignment (MAC) loss that sees the sign through a single articulator and a masked prediction (MAP) loss that reconstructs it in latent space from the surrounding spatio-temporal context. Pretrained on 266K samples ($\sim$5,700 glosses) across multiple sign
languages, SignSeek sets a new state-of-the-art performance in cross-corpus
retrieval on ASL-Citizen, WLASL, and NMFs-CSL without any downstream fine-tuning. Strikingly, it achieves zero-shot generalisation to an entirely unseen British Sign
Language (BSL), surpassing methods explicitly trained on BSL, and transfers seamlessly to isolated sign recognition and subtitle
alignment, outperforming prior skeleton-based methods.
Read the full abstract ▾
Computer Vision
Observation-Conditioned Latent Energy Priors for Sparse Implicit Neural Shape Completion
Paul Büschl, Ezequiel de la Rosa, Julia Wolleb et al. (+3 more)
Implicit neural representations (INRs) can model continuous 3D shapes with a shared coordinate decoder and per-instance latent codes. At test time, autodecoder-style models commonly freeze the decoder and optimize a new latent code from sparse off-grid SDF samples. When these samples underconstrain inference, the latent can drift toward regions that fit the observations but decode implausible unobserved geometry. We propose a post-hoc observation-conditioned latent energy prior for frozen INR decoders. The energy scores standardized latents conditioned on a permutation-invariant encoding of the sparse observation set and is used as a residual expert alongside an L2 latent prior selected on validation data. We evaluate on a controlled
cell-
nucleus SDF
dataset and a public MedShapeNet-derived SDF completion
dataset. The proposed L2 objective augmented with conditional energy improves consistently over a validation-selected L2 baseline in the sparsest
cell-
nucleus regimes and, on MedShapeNet, outperforms both L2 and a six-component GMM latent-density prior across all reported readouts. A shuffled-context ablation is consistently weaker than matched context, supporting an observation-specific contribution. These results suggest that lightweight conditional energies can make pretrained INR decoders more observation-aware without retraining.
Read the full abstract ▾
All authors Paul Büschl, Ezequiel de la Rosa, Julia Wolleb, Julian McGinnis, César Nombela-Arrieta, Bjoern Menze
Computer Vision
MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT
Chenguang Zheng, Le Xue, Yichi Zhang et al. (+8 more)
The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advances in 3D medical
vision-
language models have demonstrated remarkable progress, current efforts are limited to regional CT imaging, leaving a critical void in comprehensive whole-body PET/CT analysis. In this work, we introduce MetaStructAtlas, a large-scale
dataset for grounded whole-body PET/CT interpretation that synthesizes
multimodal imaging with integrated anatomical, metabolic, and semantic annotations. MetaStructAtlas provides 490 co-registered 3D PET and CT volumes with 50,470 organ-level
segmentation masks and grounded radiology reports. To facilitate interactive
reasoning, we further developed MetaStructVQA, a standardized 3D grounded visual question-answering
benchmark containing 100,565 QA pairs. This framework explicitly links diagnostic queries to visual evidence across modalities, encompassing anatomical, morphological, and metabolic characteristics. Finally, we evaluate state-of-the-art 3D medical VLMs on MetaStructVQA, establishing a robust foundation for
multimodal representation learning and integrated whole-body
reasoning in
nuclear medicine.
Read the full abstract ▾
All authors Chenguang Zheng, Le Xue, Yichi Zhang, Wenbo Zhang, Zehui Ling, Gang Feng, Xin Gao, Yuan Qi, Yuan Cheng, Zixin Hu, Mei Tian
Computer Vision
Semantic-Aware Subgraph State Space Model for WSI Classification in Histopathology
Feixing Chen, Hao Lu, Lin Luo et al. (+1 more)
Histopathological subtyping relies on the recognition of characteristic histological patterns. These patterns may be expressed by individual tissue structures or by the spatial distribution and co-occurrence of multiple structures, and they often span irregularly shaped tissue regions, termed semantic units in this work. However, conventional patch-based representations may fragment such units and fail to explicitly preserve their internal spatial organization, while efficiently modeling relationships among numerous spatially separated units remains challenging. To address these limitations, we propose the Semantic-Aware Subgraph State Space Model (SASG-SSM), a flexible and efficient framework for whole slide image (WSI) classification. Semantic-Aware Subgraphs (SASGs) first approximate irregularly shaped semantic units by adaptively grouping spatially connected patches guided by class-agnostic visual-semantic priors. By representing patches as graph nodes with adjacency edges, SASGs preserve their internal spatial organization rather than treating them as an unordered set. A Subgraph State Space Module (SG-SSM) subsequently combines a graph neural network encoder for intra-subgraph
topology encoding with a Mamba-based state space encoder for efficient contextualization across large numbers of subgraphs. This module integrates local structural information within semantic units with global contextual information arising from their distribution and co-occurrence across the WSI, while efficiently modeling a large number of spatially distributed regions. Extensive experiments across four WSI subtyping
datasets demonstrate consistent advantages over representative state-of-the-art methods. Further evaluations under small-cohort and few-shot settings demonstrate robustness and data efficiency under limited training data. Code will be released at https://github.com/HLSvois/SASG-SSM.
Read the full abstract ▾
All authors Feixing Chen, Hao Lu, Lin Luo, Yan Xu
Computer Vision
ToPO: Token-Conditioned Preference Routing for Attention-Based Latent Diffusion Models
Juntao Xu, Shihong Li, Hoi Fan Au et al. (+1 more)
Pairwise preference labels rank complete images, yet
Diffusion-DPO applies their effect over many spatial and denoising-time coordinates. For
attention-based, noise-prediction latent
diffusion, ToPO (
Token-Oriented Preference Optimization) constructs a per-minibatch, detached, separable spatial-temporal route from branchwise squared-residual contrast in a frozen reference denoiser. Preferred-branch cross-
attention uses content
tokens to modulate the spatial factor, and an auxiliary pixel-midpoint ordering term is added without local labels or a learned reward model. In matched three-seed retrainings with a shared update schedule, ToPO has higher endpoint estimates than
Diffusion-DPO on all five reported SD-1.5 metrics and on HPSv2, ImageReward, and CLIP for SDXL. It also receives larger raw win shares in an aggregate blind SDXL A/B study. These findings are scoped to the reported equal-update U-Net protocols rather than an equal-compute comparison.
Read the full abstract ▾
All authors Juntao Xu, Shihong Li, Hoi Fan Au, Ning Zhu
Computer Vision
DropClick: Semi-Automated One-Click Segmentation for Agricultural Robotic Data
Patrick Zimmer, Michael Halstead and Chris McCool
Labelling
vision datasets, especially for
segmentation tasks, is a laborious and costly process that stymies novel developments in agricultural robotics. In this paper, we present DropClick, a click-guided
segmentation tool that simplifies the annotation process. Our system utilises single-click inputs on objects to generate pseudo-labels, which can replace manual annotations. DropClick stands out as it is a semi-automated approach and does not require a click for every object in the scene. It can therefore further reduce the required amount of user input drastically. We evaluate our method on two challenging agricultural robotic
datasets, SB20 and BUP20 for plant and fruit
segmentation, respectively. DropClick is first trained on a small subset of just 5 images from the original training data. This DropClick model can then be deployed as a one-click
segmentation system and achieves comparable or higher performance than other one-click methods achieving an mIoU of 70.0 and 72.6 points, for SB20 and BUP20 respectively. DropClick then excels at maintaining high performance when clicks are not given (e.g. dropped); when 50% of the clicks are missing it still maintains an mIoU of 68.9 and 71.3 points, for SB20 and BUP20 respectively. We validate DropClick as a pseudo-labelling approach by taking its outputs to train a Mask2Former instance-based
segmentation model in a semi-supervised manner. In this process, partially removing user input from DropClick yields similar high performance when compared to providing all clicks, at 70.1 vs 70.7 points AP50 for SB20 and no difference for BUP20 at 77.0 for both models; at the same time saving 46.3% of total input for SB20 and 31.9% for BUP20.
Read the full abstract ▾
Computer Vision
Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language
Julian Truetsch, Felix Hauser, Christoph Stiller et al. (+1 more)
Understanding the composition of large-scale autonomous driving
datasets is essential for safety, robustness, and reliable operation across domains. For example, domain shift between locations could lead to the operating environment being misaligned with the training data, resulting in potentially dangerous performance degradation. Yet, existing data analysis pipelines largely rely on metadata, predefined labels, or manual inspection, which provide limited semantic insight or do not scale. This paper studies set difference captioning: given two subsets of images, the goal is to produce a natural-
language hypothesis describing differences between the target and reference set. Building on a two-stage formulation, we adapt the method to autonomous driving by focusing on object-centric patches derived from object detection, which simplifies aggregation and enables attribution of differences to specific object instances or categories. To evaluate this setting in-domain, we introduce a new
benchmark, AD-Diff Bench. Low-concentration experiments assess the suitability of set-difference-captioning approaches to sparse, real-world differences. We restrict our experiments to open-weight models to support reproducibility and ease of deployment. The proposed
benchmark and analysis provide a step towards practical, human-interpretable
dataset introspection for autonomous driving
datasets. Our implementation and
benchmark dataset are available at https://github.com/KIT-MRT/AD-Diff
Read the full abstract ▾
All authors Julian Truetsch, Felix Hauser, Christoph Stiller, Frank Bieder
Computer Vision
CoFiE: Coarse-to-Fine Evidence Selection for Efficient Streaming Video Understanding
Jing Jiang, Yiran Ling, Ruonan Li et al. (+2 more)
Streaming
video understanding requires
Vision Language Models (VLLMs) to process growing
video streams and answer user questions under tight latency constraints. Existing methods improve efficiency through
token pruning and
memory-bank schemes, but mainly reduce visual
tokens after visual encoding. Consequently, downstream
token pruning alone cannot substantially reduce end-to-end latency because the expensive frame encoding cost has already been incurred. We propose CoFiE, a Coarse-to-Fine Evidence Selection framework that decouples evidence selection into a coarse, query-agnostic filtering stage before the
vision encoder and a fine, query-specific refinement stage during LLM prefill. CoFiE introduces Novelty-Guided Frame Filtering to retain visually distinctive candidate frames and Query-Specific Evidence Refinement to select the frames most relevant to the user query. This design removes substantial redundancy before frame encoding while preserving query-specific refinement once semantic information becomes available. Experiments show that CoFiE establishes a new state-of-the-art accuracy-efficiency trade-off across multiple
video understanding
benchmarks, reaching 78.86% accuracy on StreamingBench and 68.72% on OvO-Bench, with improvements of up to 3.15% over prior methods. Even with up to 80% evidence-frame filtering, CoFiE outperforms strong open-source
multimodal models while improving end-to-end inference latency by up to 2.54 times.
Read the full abstract ▾
All authors Jing Jiang, Yiran Ling, Ruonan Li, Dimitrios Stamoulis, Jie Liu
Computer Vision
Do Video Generators Track the World Across Segments? A Benchmark and Method for World-State Reasoning in Video Continuation
Yingmao Miao, Pengfei Zhang, Chaoran Xu et al. (+5 more)
Video generators build long
videos by composing shorter parts, either by generating segments one after another or by autoregressively extending chunks. Each new part usually depends on
memories of historical observations, such as recent frames, selected key frames,
memory banks, or cached features. These
memories preserve visible evidence from the past, but current generators do not reliably turn such evidence into a world-state interface: what holds in the
video world after previous actions and how it should change under the next
prompt. A past frame remains valid history, but it may not describe the state needed by the next segment; some states must instead be inferred from occluded or implicit changes rather than copied from a directly observed frame. This creates a simple but overlooked question for
video continuation: given a previous
video, its
prompt, and a new
prompt, can a model generate a continuation that reflects the state determined by both the historical
video and the new
prompt? To answer this question, we introduce Statebench, a
benchmark that targets this gap by testing continuations over three state categories: past-visible states, occluded-process states, and complex-transition states. We further propose Stateagent, which explicitly maintains an entity-state representation, updates it under the new
prompt, grounds the predicted post-action state as a future end frame, and renders the next
video. Experiments show that our method improves controlled
video continuation by raising the all-case state score (SCS-All) from 45.2 to 69.3, and also benefits story generation at the one-minute scale. Code is avaliable at https://github.com/AMAP-ML/StateAgent.
Read the full abstract ▾
All authors Yingmao Miao, Pengfei Zhang, Chaoran Xu, Meng Yu, Jing Tang, Xiangxiang Chu, Chao Shen, Chenhao Lin