Computer Vision · Cross-listed from cs.RO
World-Action Models for Robot Learning and Control: A Survey
Zuxing Lu, Hongjia Zhai, Guanzhi Wang et al. (+13 more)
Robots operating in open environments act under partial observability, physical constraints, and dynamic task contexts. Beyond mapping observations and
language instructions to actions, they must anticipate how candidate actions may affect future states and task-relevant outcomes. Recent advances in world models,
video generation, and
Vision-
Language-Action (VLA) policies have motivated the development of World-Action Models (WAMs), which couple future world prediction with executable action generation. This survey provides a robotics-oriented review of WAMs. We clarify their scope relative to conventional world models, model-based reinforcement learning, action-conditioned
video generation, and reactive VLA policies, and organize existing methods through a unified taxonomy covering representations, transition modeling, action interfaces, architectures, training pipelines, data modalities, and scaling strategies. We further review applications of WAMs in manipulation,
navigation, and autonomous driving, and we summarize the
datasets,
benchmarks, metrics, and protocols used to evaluate WAM systems. Finally, we discuss key challenges in action
alignment, world-action factorization, spatial and multi-view consistency, long-horizon
memory, neural simulation for closed-loop policy learning, and efficient inference. Taken together, this survey aims to provide a concise technical foundation for integrating predictive world modeling with action generation, toward more reliable
embodied robot intelligence. Project page: https://rcl-robotics.github.io/Awesome-World-Action-Models.
Read the full abstract ▾
All authors Zuxing Lu, Hongjia Zhai, Guanzhi Wang, Huajian Zeng, Jiaqi Yang, Jingyu Liu, Lei Cheng, Yuantai Zhang, Yuheng Qiu, Zezhou Cheng, Ivan Laptev, Danfei Xu, Benjamin Riviere, Giuseppe Loianno, Eric Xing, Xingxing Zuo
Computer Vision · Cross-listed from cs.SD
POLARIS: Training-Free Audio Fingerprinting with Saliency-Based Landmarks and Delaunay Grouping
Jiheng Li
This work presents POLARIS, a training-free audio fingerprinting system that selects landmarks from a locally normalized saliency field and groups them into sparse fingerprints using Delaunay triangulation. To deal with query distortion, POLARIS adds fingerprints from two-hop Delaunay neighborhoods only at query time, without enlarging the reference index. An adaptive configuration applies this expansion only when the original fingerprints do not produce a confident match. We evaluate POLARIS on synthetic distortions from the public PEX Hard Medium
benchmark, excluding queries with pitch or tempo shifts, and on a new
benchmark of real re-recorded music. POLARIS achieves the best performance among the evaluated training-free methods on both
benchmarks. On the real recordings, its adaptive configuration also outperforms the neural NMFP baseline with a comparable measured query time and a smaller logical reference payload. Code,
dataset, and instructions for reproducing all experiments are available at https://github.com/JihengLi/POLARIS.git.
Read the full abstract ▾
Computer Vision
Zero-shot video highlight detection based on text descriptions and synthetic images
Michal Byra, Alberto Presta, Grzegorz Stefanski et al. (+1 more)
Detecting
video highlights, the most informative or engaging moments in a
video, is important for applications such as
video summarization and content recommendation. We propose a zero-shot framework that combines CLIP, large
language models (LLMs), and
diffusion models. Given lightweight
video metadata, such as a title or category, an LLM generates textual descriptions of likely highlight events. These descriptions are further converted into synthetic visual prototypes using a
diffusion model. Textual and visual representations are matched to
video frames using CLIP, enabling frame-level highlight detection without highlight annotations or
dataset-specific training. Experiments on TVSum and SumMe demonstrate strong zero-shot performance, with particularly favorable results on TVSum. The proposed approach provides an effective framework for metadata-conditioned zero-shot
video highlight detection.
Read the full abstract ▾
All authors Michal Byra, Alberto Presta, Grzegorz Stefanski, Krzysztof Arendt
Computer Vision · Cross-listed from cs.RO
Language-Guided Representation Learning for Robust Cross-Sensor Material Recognition
Mashood M. Mohsan, Muhayy Ud Din, Binzhao Xu et al. (+2 more)
Robots need touch to manipulate objects safely and reliably, as many properties, such as softness, texture, and contact stability, are hard to infer from
vision alone. However,
vision-based tactile sensors yield different observations of the same material due to variations in optics, elastomer properties, and illumination, leading to poor generalization when trained on a single or multiple sensors. We propose a
language-guided
distillation framework for learning sensor-robust tactile representations.
Language encodes high-level semantic properties of touch (e.g., rough, soft, slippery) that remain invariant across sensing hardware, providing a natural sensor-agnostic supervisory signal. We construct a 39K-sample touch-
language dataset with human-annotated material labels and train a tactile encoder to align sensor-specific tactile images with
language embeddings in a shared semantic space. We evaluate our approach for few-shot learning and cross-sensor transfer and
benchmark it on six existing tactile
datasets. Our method achieves 95% accuracy in the 100-shot setting, improves cross-sensor transfer by an average of 13.3% accuracy, and yields up to 19% accuracy gains across six existing tactile
datasets. These results demonstrate that
language-guided
distillation enables scalable and hardware-agnostic tactile representation learning. Code and
dataset are available at https://mashood3624.github.io/
Language_Tactile/
Read the full abstract ▾
All authors Mashood M. Mohsan, Muhayy Ud Din, Binzhao Xu, Ahmad Abubakar, Irfan Hussain
Computer Vision
From Visual Feedback to Textual Reviews: A Multi-Agent Vision-Language Framework for Image-Grounded Review Assistance
Utsav Kumar Nareti, Ayush Bansal, Kumari Priya et al. (+4 more)
Visual feedback in the form of user-uploaded images and
videos is becoming increasingly common in e-commerce platforms because it provides authentic evidence of product quality, defects, packaging conditions, and real-world usage. However, visual feedback alone often lacks the contextual explanations and subjective opinions necessary for informed decision-making, while many users provide limited textual feedback due to the effort required to compose detailed reviews. To bridge this gap, we introduce image-grounded review assistance, a novel task that aims to generate editable review drafts from user-uploaded product images. Unlike conventional image captioning, which focuses on objective visual description, the proposed task requires product-specific understanding, sentiment estimation, and evidence-driven review composition under challenging real-world conditions, including degraded image quality, excessive zoom-in, target ambiguity, and partial product visibility. We propose a multi-
agent vision-
language framework consisting of four specialised roles: product grounding, visual sentiment estimation, visual evidence generation, and review synthesis. The framework employs explicit intermediate representations, including product entities, predicted ratings, and evidence summaries, to improve interpretability and visual grounding. Experiments on a curated subset of the Amazon Reviews Electronics
dataset demonstrate the feasibility of generating coherent, product-aware, and sentiment-aware review drafts from visual feedback. To the best of our knowledge, this is the first study to formulate image-grounded review assistance as a multi-
agent vision-
language reasoning problem, providing a practical step toward AI-assisted review authoring in e-commerce systems.
Read the full abstract ▾
All authors Utsav Kumar Nareti, Ayush Bansal, Kumari Priya, Chandranath Adak, Soumi Chattopadhyay, Muhammad Saqib, Saeed Anwar
Computer Vision
CrossDistill: Balancing Quality and Diversity via Trajectory-Level Hybrid Few-Step Distillation
Yuxi Liu, Haoyu Li, Yixiang Cai et al. (+8 more)
Few-step
distillation accelerates
diffusion models but must balance diversity and fidelity: trajectory-based
distillation preserves mode coverage, while distribution matching sharpens samples but can reduce diversity. We show that this tension can be exploited in a noise-regime-dependent way: high-noise steps largely determine global modes, whereas low-noise steps refine local details. We propose CrossDistill, a trajectory-level hybrid
distillation framework that splits the sampling trajectory at a crossover point, applies a trajectory-preserving objective on the high-noise interval and a distribution-matching objective on the low-noise interval, and couples the two stages through the crossover state. In contrast to loss-level mixing, and complementarily to training-time two-stage recipes, CrossDistill explicitly assigns complementary objectives along the noise axis, so that global branching is preserved before local statistics are sharpened. CrossDistill is a noise-level scheduling policy: PCM and DMD are plug-in instantiations, while the noise partition, crossover coupling, and objective ordering are the key design elements. Experiments on text-to-
video diffusion models and qualitative image-to-
video results show that CrossDistill expands the few-step quality-diversity frontier, retaining seed-level variation while achieving competitive visual fidelity.
Read the full abstract ▾
All authors Yuxi Liu, Haoyu Li, Yixiang Cai, Tengxu Sun, Zekun Zhang, Baole Ai, Ang Wang, Jiamang Wang, Lin Qu, Kun Yuan, Kai Zhang
Computer Vision
PC$^2$-AD: Point Cloud Upsampling to Safeguard 3D Anomaly Detection with Resolution-constrained Edge Devices
Yutong Gu, Yingxi Xie, Kejin Huang et al. (+4 more)
Low-cost and low-resolution sensors used in edge deployments can produce test point clouds that are substantially sparser than the normal training data. This train-test sampling-resolution gap changes the local geometry available to a 3D anomaly detector. We propose PC$^2$-AD, a point cloud upsampling framework that compensates sparse test inputs before downstream detection. Target Domain Candidate Generation (TCG) adapts a pretrained upsampler to normal training geometry and generates a dense candidate pool. Geometry-Aware Candidate Filtering (GACF) selects candidates according to
geometric spacing and spatial coverage. Normality-Preserving Point Compensation (NPPC) refines the selection by comparing candidate normality scores with those of their input anchors. The selected points are combined with the unchanged input points and processed by the existing detector. Experiments with six detectors on two Anomaly-ShapeNet settings and Real3D-AD show improvements in the mean of object-level and point-level AUROC for all six detectors in each Anomaly-ShapeNet setting and four on Real3D-AD. These results support point cloud compensation as an input-level approach to improving 3D anomaly detection under low-resolution sensing conditions. Code is publicly available at https://github.com/gyutong406-commits/PC2-AD.
Read the full abstract ▾
All authors Yutong Gu, Yingxi Xie, Kejin Huang, Jian Ning, Hanzhe Liang, Linlin Shen, Jinbao Wang
Computer Vision
Patient-Level, Leakage-Aware Deep Learning for Cross-Center Periapical Radiograph Classification
Md Jubaer Rahman and Ulas Bagci
Dental caries and endodontic disease are among the most common health conditions worldwide, and intraoral periapical radiographs are central to their detection, treatment planning, and follow-up. Automated tooth-level classification of these images, however, lacks reproducible
benchmarks, is often evaluated with image-level splits that leak patients between training and test, and is rarely validated across clinics. This paper presents the first patient-level, leakage-aware classification
benchmark for single-tooth intraoral periapical radiographs on the DentIRO
dataset, which comprises 5,300 images from 3,243 patients across two clinics and four classes: Healthy, Caries, Crowned, and Root Canal. Five transfer-learning models are compared with patient-grouped stratified cross-validation, so that every patient remains within a single fold. DenseNet121 gave the strongest and most stable result at a mean macro-F1 of 0.9787, while the four ImageNet-initialized backbones performed comparably. A controlled comparison on a fixed architecture showed that chest-radiograph pretraining transferred less effectively than ImageNet initialization. Bidirectional cross-center validation produced a small average generalization gap of 0.0077, and Grad-CAM confirmed that predictions rely on clinically meaningful tooth regions rather than acquisition artifacts. The
benchmark offers a rigorous and reproducible baseline for intraoral radiograph classification.
Read the full abstract ▾
Computer Vision
Perceive, Refine, Reason: A Calibrated Pipeline for Measuring Indicators in Strategic Visual Communication on Social Media
Weihong Qi and Chen Ling
Visual content shapes audience perception and opinion on social media, and computational social science increasingly relies on automated tools to analyze images at scale. Yet a measurement gap persists: existing tools rely on predefined categories or produce only coarse image-level labels, while measuring which specific objects appear in an image, how prominently, and where in the frame remains difficult at scale. We introduce Perceive, Refine, Reason (PRR), a calibrated pipeline that turns flexible
vision-
language detectors into auditable measurement instruments for social-scientific research. PRR combines natural-
language category
prompts with pixel-level spatial refinement via the Segment Anything Model (SAM) and a
multimodal LLM arbitration layer whose
reasoning chains externalize domain knowledge and lower the expertise threshold for human-in-the-loop validation. A complementary three-tier auditability framework applies quantification learning to profile per-category reliability, support task-aligned configuration, and statistically correct prevalence estimates. Across four
vision-
language detectors and nine sociological categories, the pipeline yields substantial precision gains over zero-shot baselines, including a 43.3-point improvement for the strongest backbone. Applying PRR to 103,920 Facebook images from U.S. legislators during the 2024 election cycle and linking detections to DW-NOMINATE ideology scores, we find that more conservative legislators display U.S. flags as larger visual elements, with a weaker tendency toward peripheral placement, a spatial pattern invisible to binary detection. PRR provides computational social scientists with a model-agnostic toolkit for accessible, spatially-grounded, and correctable visual measurement.
Read the full abstract ▾
Computer Vision
LIMODENet: Attention-Free Compact Encoders for Information-Preserving Onboard Satellite Image Restoration
Thanh-Dung Le, Vu Nguyen Ha, Ti Ti Nguyen et al. (+1 more)
Onboard satellites must restore a channel-degraded image on a few watts, using neuromorphic accelerators (e.g., BrainChip Akida, Intel Loihi-2) that support no softmax or
attention. We ask which encoder restores best under that constraint and introduce LIMODENet (LinearMix-ODENet), a 0.69M softmax-/QKV-free backbone whose residual stages read as ODE discretizations and which is empirically information-preserving (probe accuracy rises 79.9% -> 98.4% from stem to head). At iso-parameters it restores 1 dB DVB-S2X-degraded EuroSAT better than a CNN autoencoder (+1.75 dB PSNR) and a skip-connection U-Net (+1.07 dB), three seeds, non-overlapping. Unconstrained modern restorers (NAFNet, Restormer) win on fidelity; we decompose that gap: spiking-legal additive skips recover about half, and the rest traces to
attention and channel gating. LIMODENet then converts end-to-end to a spiking network with zero blocked operations, versus 22-24 for the competitors: not the best restorer available, but the best verified deployable within a real power budget.
Read the full abstract ▾
All authors Thanh-Dung Le, Vu Nguyen Ha, Ti Ti Nguyen, Symeon Chatzinotas
Computer Vision
From Density to Biopsy Decisions and Malignancy Prediction: A Benchmark Study of Multimodal Large Language Models Against Radiologists in Digital and Contrast-Enhanced Mammography
Ali Abbasian Ardakani, Afshin Mohammadi, Taha Yusuf Kuzan et al. (+11 more)
Purpose: To compare four
multimodal large
language models (MLLMs) with radiologists of varying expertise in breast density assessment, BI-RADS assessment, biopsy candidacy determination, and continuous malignancy probability estimation using digital mammography (DM) and contrast-enhanced mammography (CEM). Methods: This study included 179 women with paired DM/CEM examinations and reference standards. Four MLLMs (ChatGPT-5.2, Gemini-3.1 Pro, Sonnet-4.6, Muse Spark) interpreted images with and without masks; three radiologists interpreted non-masked images. Results: For binary density classification on DM, radiologist accuracies ranged from 55.81% to 78.60%, exceeding most MLLM values (62.33%-71.63%), while masks added limited benefit. Five-category BI-RADS accuracies were higher for radiologists on DM (56.74-67.44%) and CEM (62.33-82.79%) compared with MLLMs (DM 31.16-45.12%; CEM 40.00-55.81%). Binary biopsy-candidacy accuracies were likewise higher for radiologists (DM 85.12-89.77%; CEM 86.98-92.09%) than for MLLMs (DM 61.39-75.35%; CEM 69.30-82.79%), although CEM improved performance across all readers. Lesion masks substantially improved MLLM continuous malignancy-probability accuracies from 64.65%-71.63% to 72.56-78.60% on DM and from 67.91%-77.21% to 72.56%-81.86% on CEM, approaching radiologist ranges (DM 63.72-82.79%; CEM 81.86-88.84%). The corresponding AUCs for the top masked models overlapped those of the human readers. Overall, Muse Spark, followed by Sonnet-4.6, demonstrated the strongest performance among the MLLMs across domains. Conclusion: Radiologists generally outperformed MLLMs in categorical tasks, while selected masked models approached human performance for continuous malignancy probability estimation, suggesting a potential adjunctive role.
Read the full abstract ▾
All authors Ali Abbasian Ardakani, Afshin Mohammadi, Taha Yusuf Kuzan, Beyza Nur Kuzan, Alisa Mohebbi, Masume Behruzi, Hamid Khorshidi, Ashkan Ghorbani, Elham Asadiara, Zeinab Khorshidi Lotfi, Ansar Rahman, Nedim Christoph Beste, U. Rajendra Acharya, Sepideh Hatamikia
Computer Vision
Floquet Fibre Geometry and Higher-Order Reduced Coordinates for Off-Manifold Transients near Nonlinear Aeroelastic Flutter
Puxue Tan
Assigning reduced coordinates to states near an attracting limit cycle requires the correct invariant-fibre geometry. The classical first-order phase-isostable chart obtained from adjoint Floquet modes projects along the strong-stable quotient fibre, whereas a metric-orthogonal complement of the retained slow bundle generally does not. We prove locally that a chart satisfying the linearised semiconjugacy relation leaves an O(delta^2) invariance residual, while projection along a non-invariant complement generically leaves an O(delta) term. For a nonlinear aeroelastic limit cycle, the metric-normal and strong-stable directions differ by 48.5 to 71.7 degrees, and metric-normal perturbations contain first-order retained phase and slow-amplitude components. Replacing the metric normal by the strong-stable fibre changes the measured residual scaling from delta^1.01 to delta^1.87 without fitted parameters. We then test learned higher-order corrections whose linearisation is pinned to the adjoint-Floquet chart, whose
symmetry is exact, and whose reduced flow is fixed. Although they reduce the registered fixed-normalisation latent residual, post-hoc amplitude recalibration and adjoint-Floquet-targeted future consistency move or reverse the ranking. Because the learned maps already share the baseline's first-order gauge and the future target is supplied by the baseline chart, these diagnostics establish neither an independent positive nor negative higher-order result. Correct first-order Floquet geometry is therefore necessary in this
benchmark, while the additional predictive value of the learned correction remains unidentified by the available representation-dependent diagnostics.
Read the full abstract ▾
Computer Vision
Compositional SVG Generation via VLM-Driven Hierarchical Semantic Parsing
Sehwan Park, Taehoon Kim, Geonhee Han et al. (+3 more)
While
Vision-
Language Models (VLMs) excel at visual
reasoning, generating structured, editable Scalable Vector Graphics (SVG) remains a fundamental challenge. Existing pipelines predominantly yield flat, semantically agnostic collections of paths, where editing a single object requires manually identifying its constituent paths. To address this, we propose a VLM-driven agentic framework for semantic compositional SVG generation. Our pipeline recursively parses visual scenes into semantic and
geometric hierarchies via top-down decomposition, visual grounding, and
prompt-driven amodal occlusion recovery, ensuring each component is geometrically complete. Furthermore, we introduce the Semantic SVG
Benchmark with human-annotated semantic groups and novel sub-component metrics (Semantic Recall/Precision, PERE) to explicitly evaluate structural compositionality and functional editability. Experiments show that our natively predicted structures surpass the upper bounds of existing flat-generation methods in both grouping quality and editability, while maintaining state-of-the-art visual fidelity.
Read the full abstract ▾
All authors Sehwan Park, Taehoon Kim, Geonhee Han, Dohyun Kim, Seung Wook Kim, Paul Hongsuck Seo
Computer Vision
Evaluating Contextual Bias in CNN Image Classification: Evidence from Agricultural Benchmark Datasets
Abhilekha Dalal, Michael Okonoda, Eder Martinez et al. (+1 more)
Convolutional neural networks (CNNs) are typically evaluated using held-out classification accuracy, an approach that presupposes predictions are based primarily on the intended object of interest rather than incidental surrounding context. We test this assumption in CNN-based agricultural image classification by comparing model performance on original images with performance on background-dominated patches extracted from the same images across eight publicly available agricultural
benchmark datasets and four widely used CNN architectures. Background-dominated patches were classified above
dataset-specific random chance for six of the eight
datasets, and substantially above chance for four of them, indicating that contextual information contributes to model predictions for the majority of
datasets evaluated. For these four
datasets, we further evaluated whether this behavior reflected genuine class-discriminative information or was primarily attributable to class imbalance using macro-averaged precision, recall, and F1 together with class-balanced test subsets. The results show that contextual reliance does not admit a single explanation: class imbalance accounts for a substantial portion of the observed signal for some
datasets and architectures, whereas above-chance contextual classification persists after balancing for others. Together with previous evidence from curated object recognition and cancer pathology imaging, these findings support the growing view that contextual bias is a recurring characteristic of CNN-based image classification rather than a phenomenon confined to a single application domain. More broadly, this work provides a systematic framework for quantifying contextual bias across heterogeneous image
datasets by combining
dataset-specific random-chance baselines, contextual bias categorization, macro-averaged evaluation, and class-balanced robustness analysis.
Read the full abstract ▾
All authors Abhilekha Dalal, Michael Okonoda, Eder Martinez, Lior Shamir
Computer Vision
SCOUT-SLAM: Structurally-Coupled Dual Uncertainty-Aware 3DGS SLAM in the Wild
Kumaran Karthik, Pramat Shastri Jois and Suresh Sundaram
Recently, 3D Gaussian Splatting SLAM (3DGS-SLAM) has gained significant momentum in simultaneous localization and 3DGS scene reconstruction. In real-world scenarios with rapid camera motion and cluttered dynamic environments, existing methods rely on the stability of the underlying scene reconstruction to model uncertainty. This leads to a circular dependency between camera tracking accuracy and reconstruction quality: reconstruction instabilities degrade uncertainty modeling, which affects accurate camera tracking and static scene reconstruction. To address this, the paper proposes SCOUT-SLAM, a structurally-coupled dual-uncertainty framework in which both uncertainties are estimated from a shared base network. A low-rank adaptation of this network, trained on multi-view feature consistency, estimates a tracking uncertainty that does not depend solely on the reconstruction quality. A spatially-adaptive prior modulates the network's training objective so that reconstruction instability does not inflate uncertainty on static regions, keeping the shared representation intact for both branches. Evaluations on dynamic
benchmarks (TUM RGB-D, Bonn Dynamic, Wild-SLAM MoCap) demonstrate that SCOUT-SLAM achieves state-of-the-art camera tracking accuracy and artifact-free static scene reconstruction.
Read the full abstract ▾
Computer Vision
TTDF: A Two-Stage Framework for Reliable Surgical Phase Transition Detection
Yushi Guo, Pietro Valdastri and Duygu Sarikaya
Reliable workflow transition detection is important for context-aware
surgical assistance and downstream decision support. However, online
surgical phase recognizers primarily focus on frame-wise accuracy and temporal consistency, rather than the reliability of workflow transition events. Directly converting phase changes into events is unreliable: temporal jitter and workflow-illegal switches produce false or duplicate events, while persistent, workflow-consistent candidates may remain incorrect. To address this limitation, we formulate reliable workflow transition detection as a distinct event-level task operating on outputs of a frozen online phase recognizer. We propose the Two-Stage Transition Detection Framework (TTDF), a causal framework that progressively filters transition candidates. Transition Candidate Extraction (TCE) first applies a minimum-duration requirement and a workflow-graph constraint to remove false candidates caused by temporal jitter and phase transitions not allowed by the workflow graph. Specifically, a candidate is retained only if the predicted target phase persists for a minimum duration and the ordered phase pair belongs to the workflow graph's allowed transition set. TCE thereby produces a high-recall candidate set without additional training. Transition Candidate Verification (TCV) suppresses remaining false candidates using phase-posterior shifts and visual-change cues from frozen DINOv2 features. Events are assessed using a phase-pair-aware one-to-one matching protocol. Experiments on Cholec80 show that TTDF reduces false transition emissions while preserving recall and controlling decision delay.
Read the full abstract ▾
Computer Vision
ESAFusion: LiDAR–4-D Radar Fusion via Local Geometric Complementation and Multiscale Adaptive Interaction for 3-D Object Detection
Gang Ma, Senjie Hu, Junjie Liu et al. (+2 more)
LiDAR–4-D radar fusion combines accurate spatial geometry with motion and reflectivity cues from radar, offering a promising solution for 3-D object detection in complex driving environments. However, sparse radar observations and differences in spatial sampling between the two modalities complicate reliable cross-modal complementation. Moreover, the relative importance of modalities and feature scales varies across spatial regions, making adaptive fusion challenging. To address these challenges, we propose ESAFusion, an evidence-aware and scale-adaptive framework that combines local
geometric complementation with multiscale adaptive interaction. Specifically, we introduce an Evidence-Aware Radar Selection (ERS) module to suppress radar clutter using motion and observation-quality evidence while retaining foreground confidence for subsequent fusion. Then, the Pillar-Level Complementary Encoder (PCE) improves cross-modal complementation under mismatched spatial sampling using local
geometric support from neighboring
LiDAR pillars. We further design an Intra- and Inter-Scale Adaptive Fusion (ISAF) module to adaptively adjust the contributions of different modalities and feature scales in bird's-eye-view (BEV) space. Extensive experiments on the View-of-Delft (VoD)
dataset show that ESAFusion achieves the highest mean average precision (mAP) among the compared methods, reaching 74.60% in the Entire Annotated Area and 88.89% in the Driving Corridor. It also attains the highest average precision (AP) for Cyclist among these methods in both regions while running at 19.23 FPS. Evaluations on VoD-Fog further demonstrate robustness under progressively degraded
LiDAR observations. The source code will be made publicly available at https://github.com/SenJieHu549/ESAFusion.
Read the full abstract ▾
All authors Gang Ma, Senjie Hu, Junjie Liu, Chao Wang, Hui Wei
Computer Vision
Open-UniMo: Towards Unified Motion-Language Understanding and Generation in the Open World
Guocun Wang, Kenkun Liu, Guorui Song et al. (+7 more)
Unified motion generation and understanding is crucial for
embodied AI systems that can both synthesize and interpret human actions in open-world environments. Existing motion-
language models often treat motion as an auxiliary modality of a
language model, leading to text-dominated representations and limited cross-modal interaction. Moreover, the next-
token prediction paradigm is not naturally suited to long motion sequences, where autoregressive generation may accumulate prediction errors. To address these challenges, we propose Open-UniMo, a unified Large Motion-
Language Model (LMLM) trained on million-scale open-world motion-
language data. Open-UniMo promotes modality parity by extending Qwen's vocabulary of about 150K text
tokens with 64K motion
tokens, enabling motion and
language to share a unified
token space. We further introduce motion-consistent Chain-of-Thought
reasoning as an intermediate representation to bridge
language semantics and motion dynamics. Open-UniMo is trained with a two-stage pipeline, where supervised fine-tuning establishes CoT-guided bidirectional motion-
language mapping and Group Relative Policy Optimization (GRPO) improves semantic
alignment while mitigating cumulative errors in autoregressive motion-
token generation. To support comprehensive evaluation, we propose Open-MoBench, a VLM-guided
benchmark for assessing text-to-motion (T2M) generation, motion-to-text (M2T) understanding, and bidirectional consistency. Extensive experiments show that Open-UniMo achieves state-of-the-art performance on both conventional metrics and Open-MoBench. Furthermore, ablation studies reveal that M2T understanding is not primarily limited by motion-
token vocabulary size; instead, coupling M2T with the learnable T2M generation path yields stronger cross-modal representations, demonstrating that generation can facilitate understanding in AR-based motion-
language modeling.
Read the full abstract ▾
All authors Guocun Wang, Kenkun Liu, Guorui Song, Jing Lin, Zhe Huang, Luyuan Zhang, Dake Zhong, Choo Sin Wai, Xiaoguang Han, Haoqian Wang
Computer Vision
Direct Conditional Transition Sampling for Diffusion Inverse Problems
Qi Yu, Hanlin Wu and Xiaohui Sun
Training-free
diffusion inverse solvers typically choose between local measurement guidance and costly clean-space posterior updates. Independent posterior refresh can improve global correction by sampling a clean conditional and re-noising it, but its practical realization requires probability-flow ODE integration and clean-space Markov chain Monte Carlo (MCMC). We propose Direct Conditional Transition Sampling (DCTS), a direct
stochastic-flow approximation to the same ideal refresh target. Rather than explicitly drawing a clean sample, DCTS estimates the measurement-conditioned clean mean along a short inner path and transports Gaussian source noise directly to the next noisy state. A denoiser-compatible sufficient statistic and a covariance-scaled operator update enable this conditional-mean estimation. Experiments on four inverse problems demonstrate that DCTS achieves competitive reconstruction quality with up to $16.8\times$ speedups over competing methods.
Read the full abstract ▾
Computer Vision
Reverse Spatio-Temporal Disease Progression Modelling
Ulugbek Shernazarov, Moucheng Xu and Inomjon Ramatov
Deep learning-based spatio-temporal disease progression models commonly overlook the incubation period of progressive diseases, limiting the use of those models in early interventions, which are vital for not easily reversible diseases such as Alzheimer's. This is because, the existing deep learning based longitudinal disease-progression models are almost always run forward: from an observed baseline they predict future decline. In many clinical settings, however, imaging begins only after pathology is suspected or already visible, the earlier, healthier patient-specific reference was never acquired. To address this, we propose to study reverse disease progression prediction: given later diseased anatomy, reconstruct the unobserved healthier anatomy that preceded it. We use a two-stage model in which a frozen 3D vector-quantised autoencoder defines a compact discrete latent space, while a Neural Ordinary Differential Equation (ODE) learns continuous-time dynamics in that space. A recurrent encoder reads late observations in reverse temporal order, initialises the latent state, and the ODE is integrated backwards across the trajectory. On a controlled Morpho-MNIST
benchmark with a sinusoidal perturbation, our model successfully recovered the unseen previous states from later observations of the non-monotonic trajectory. On longitudinal
brain MRIs from Alzheimer's Disease Neuroimaging Initiative, at the task to recover the previous unseen trajectory towards healthy states of the patients from observed later diseased states, our model outperforms the baselines that uses copy-nearest and mean-observed, with positive disease-reversal scores in every diagnostic stratum. We hope that our work can provide insights and tools towards discovering the incubation periods from single-shot scans, and developing early interventions of diseases based on imaging.
Read the full abstract ▾
Computer Vision
A Building as a Repository: KIR, a Typed Intermediate Representation for Agent-Authored Building Information Models
Dmitry Kuklev
Autonomous
agents that author building information models need more than access to a host API. They need a representation of what they intended, what a compiler decided on their behalf, what was refused, what was observed after execution and what remains unknown. We present KIR, a typed intermediate representation in which a building is authored as a program held in a versioned repository and lowered to host applications as build targets. KIR is organised around seven ways in which a generator writing into a stateful, partially observable host goes silently wrong, and gives each a representation in data: ambiguous selectors become typed refusals with candidates; defaults keep their provenance; obligations that will not be checked are named before execution; vacuous witnesses are rejected statically; a lost transaction response becomes the state UNCONFIRMED with a verify-before-retry rule; the reverse path obeys a census invariant; and decisions are bound by digest to the state they were made against. On a pinned snapshot with 83 operation contracts and Revit as the only backend, offline experiments refuse 29 of 42 stress-test programs with diagnostic codes and no uncaught exception, name 38 of 377 witness obligations as unwitnessable, admit 31 of 100 combinations of execution, witness and acceptance states under seven stated invariants, and find no vacuous witness in 219 certificate runs; a 60-storey tower is 11,263 characters as KIR against 3,709,235 characters of emitted C#. Native Revit runs are reported from project records and kept separate from reproduced results. A controlled comparison with
agents that write host code directly is specified but not yet executed; it is the principal open question.
Read the full abstract ▾
Computer Vision
Small Object Detection in Drone Aerial Imagery with LAF-YOLOv10
Quratulain Nayeem, Fahmina Taranum and Mohammed Mudassir Uddin
General-purpose object detectors lose accuracy on
UAV footage, where targets span only a handful of pixels and onboard compute is limited. Prior work composes independently-validated architectural techniques into one detector, assuming gains reported in isolation transfer once combined. We stress-test that assumption directly. LAF-YOLOv10 integrates four techniques into YOLOv10n: a Partial Convolution C2f (PC-C2f) backbone block, an
Attention-Guided Feature Pyramid Network (AG-FPN), a P2 detection head replacing the large-object P5 head, and Wise-IoU v3 regression, asking whether their combined effect matches what each contributes alone. We train LAF-YOLOv10 three times (seeds 42, 123, 256) on VisDrone-DET2019,
benchmark against unmodified YOLOv10n, and use TIDE error decomposition, per-category breakdown, per-component ablation,
attention/loss comparisons, zero-shot transfer to UAVDT, and held-out/test-dev evaluation to localize where the combination succeeds or fails. Composability does not hold here. LAF-YOLOv10 reaches 24.0+/-0.4% mAP@0.5 at 2.14M parameters, 7.8 points below YOLOv10n (31.8%), a deficit that transfers to UAVDT (-10.0 points) and is confirmed by held-out and test-dev evaluation (23.5%, 22.5%). Background false positives, localization error, and duplicate detections move in the direction AG-FPN and Wise-IoU were designed to push. Ablation traces the deficit to a specific source: the P2/-P5 head swap costs 2.5 points independently plus a 2.5-point interaction penalty when layered onto a backbone weakened by PC-C2f, whose own 2.0-point loss is consistent with a partial pretrained-weight transplant (73/150 backbone tensors transfer). The failure is attributable to a specific interaction, not the four components individually. Composability must be verified directly, not assumed. Code/checkpoints: https://github.com/Mudassiruddin7/Small-Object-Detection-in-
UAV-Imagery.
Read the full abstract ▾
Computer Vision
Sharing standardized image-derived data in computational pathology using DICOM
Daniela P. Schacherer, Christopher P. Bridge, David Clunie et al. (+13 more)
Development and evaluation of computational pathology methods require access to large and diverse
datasets. Over the past decade, various initiatives invested significantly into collecting, centralizing, and sharing pathology imaging data. In contrast, sharing of image-derived data such as region-of-interest delineations or
segmentation masks is less well developed. In this work, we describe our approach to encoding and sharing image-derived pathology data in a standardized manner within the National Cancer Institute (NCI) Imaging Data Commons (IDC), a platform that hosts and provides public access to de-identified radiology and pathology data. The IDC relies on the Digital Imaging and Communications in Medicine (DICOM) standard for data harmonization, yet the adoption of DICOM for pathology image-derived content has remained largely unexplored until now. Here, we present five representative
datasets harmonized by conversion from their original representations into DICOM and shared publicly in the IDC. We demonstrate the benefits of this harmonization, describe contributions to critical open-source tooling, and discuss technical considerations relevant to broader adoption of DICOM for pathology image-derived data.
Read the full abstract ▾
All authors Daniela P. Schacherer, Christopher P. Bridge, David Clunie, Igor Octaviano, André Homeyer, Markus D. Herrmann, Olivier Gevaert, Tabita Ghete, Markus Metzler, Henning Hoefener, Tahsin Kurc, Curtis Lisle, Kenneth Philbrick, Joel Saltz, Yuanning Zheng, Andrey Fedorov
Computer Vision
Selective Tool Use for Agentic Change Visual Question Answering in Remote Sensing
Yakoub Bazi, Mohamad M. Al Rahhal, Mohamed A. Mekhtiche et al. (+1 more)
Change visual question answering (Change VQA) requires understanding semantic changes across bi-temporal remote sensing images. Although
vision language models (VLMs) have shown promising performance on this task, they remain unreliable when answering questions that require explicit transition statistics, area measurements, or spatial information. To address this limitation, we propose a selective tool use framework in which a single VLM either answers directly or invokes a deterministic change analysis tool to obtain question specific evidence. Specifically, the selected tool operates on bi-temporal semantic maps and returns a structured observation, which the same VLM uses to generate its final answer. To support this framework, we construct a tool augmented extension of CDVQA covering eight question families and three tools for transition, spatial, and temporal analysis. Tool use supervision and observations are derived automatically from the original semantic annotations, without additional manual labeling. We then adapt Qwen3.5-4B using Low Rank Adaptation (LoRA) to jointly learn direct answering, tool invocation, and evidence conditioned answering. Experiments on 7,164 test questions show that selective tool use with reference semantic maps improves overall accuracy from 73.77% to 88.79% and average family accuracy from 69.11% to 89.65%. When the semantic maps are predicted automatically, the framework achieves 77.47% overall accuracy and 75.06% average family accuracy. These results demonstrate the benefit of question-specific semantic evidence for Change VQA, while highlighting the influence of semantic prediction quality on the resulting performance. Code and tool-augmented annotations will be made publicly available at https://github.com/yakoubbazi/ToolChangeVQA.
Read the full abstract ▾
All authors Yakoub Bazi, Mohamad M. Al Rahhal, Mohamed A. Mekhtiche, Mansour Zuair
Computer Vision
CGGT: Curve-Grounded Geometry Transformer for 3D Parametric Curve Reconstruction
Zhirui Gao, Renjiao Yi, Yunfan Ye et al. (+4 more)
Recovering editable 3D parametric curves from 2D images is a fundamental challenge in computer graphics, bridging pixel-based perception and vector-based CAD modeling. Existing NeRF- and 3DGS-based methods often rely on dense calibrated views, precomputed 2D edge maps, and costly per-scene optimization, limiting their applicability to casually captured real-world inputs. We propose CGGT, a Curve-Grounded Geometry
Transformer that directly grounds 3D-consistent 2D curve instances in the image space from sparse, unposed multi-view images. CGGT combines a geometry-aware
transformer encoder for multi-view feature learning with a curve-aware masked-
attention decoder for cross-view instance association. In a single forward pass, it predicts camera parameters, dense depth maps, and instance-level 2D curve masks, which are then lifted into 3D and refined through a fast parametric optimization stage to recover compact, editable 3D curve primitives. To support structured curve learning, we introduce Wireframe-100K, a large-scale
dataset comprising 100,000 CAD models with diverse
topologies, realistic multi-view renderings, and accurate parametric curve annotations. Extensive experiments show that our framework achieves substantial improvements in both reconstruction accuracy and efficiency, particularly under challenging sparse-view settings and in separating persistent 3D structural edges from view-dependent image edges caused by silhouettes, textures, and appearance variations. Despite being trained solely on synthetic data, CGGT generalizes well to real-world images, demonstrating its potential for practical CAD-style wireframe reconstruction from unconstrained visual inputs.
Read the full abstract ▾
All authors Zhirui Gao, Renjiao Yi, Yunfan Ye, Ruizhen Hu, Chenyang Zhu, Wei Chen, Kai Xu
Computer Vision
MCIQA-2K: A Multi-Dimensional Dataset and No-Reference Quality Assessment Benchmark for Colorized Images
Yunkai Zhuang, Qihang Yan, Zicheng Zhang et al. (+1 more)
Image colorization is an inherently ill-posed task, since a single grayscale image may correspond to multiple plausible colorized results. Consequently, conventional full-reference image quality assessment (IQA) metrics fail to accurately reflect human perceptual preferences for colorized images. In this paper, we present MCIQA-2K, a large-scale multi-dimensional
benchmark specifically designed for no-reference quality assessment of colorized images. We construct a
dataset containing 2,000 colorized images generated by five representative colorization models, together with human annotations across three perceptual dimensions: color smearing, semantic color misalignment, and global naturalness. Building upon the proposed
benchmark, we further introduce MCIQA, a dedicated multi-branch NR-IQA framework for colorized images. Extensive experiments demonstrate that MCIQA significantly outperforms existing full-reference and no-reference IQA methods on the proposed
benchmark, while also exhibiting competitive generalization capability on several widely-used IQA
datasets. The
dataset and code are publicly available at https://github.com/ARBEZ-ZEBRA/MCIQA.
Read the full abstract ▾
All authors Yunkai Zhuang, Qihang Yan, Zicheng Zhang, Guangtao Zhai
Computer Vision
PuzzleMate: Benchmarking MLLMs for Egocentric Puzzle Assistance
Avijit Dasgupta, Shayon Dasgupta, Zakaria Laskar et al. (+2 more)
Personal AI assistants hold the potential to evolve from digital interfaces into
embodied companions capable of guiding users through complex physical activities. For these assistants to become integral to daily life, they must do more than identify objects; they must provide precise, step-by-step instructions that align with a user's real-time progress. While
Multimodal Large
Language Models (MLLMs) show promise in general visual understanding, their ability to deliver grounded, sequential guidance for fine-grained manipulation tasks remains largely unverified. In this paper, we choose the jigsaw puzzle as a strategic testbed for this capability. Unlike general object recognition, puzzle solving demands high-precision spatial
reasoning, the ability to distinguish between minute
geometric variations, and a rigorous adherence to sequential logic. We investigate this capability through PuzzleMate, a novel framework focused on jigsaw puzzle solving captured through an
egocentric viewpoint. We deploy PuzzleMate in a user-in-the-loop study to evaluate how well state-of-the-art MLLMs perceive the current puzzle state and generate actionable next-step instructions. Our analysis reveals seven key bottlenecks that limit their effectiveness. Building on these insights, we propose a
benchmark that enables systematic evaluation of MLLMs'
reasoning capabilities for puzzle solving. Our findings reveal a substantial performance gap in current models like GPT-5.2 and Gemini-2.5-Pro; while these MLLMs are highly capable, they struggle to navigate the intricate
reasoning and sequential logic essential for jigsaw puzzle assistance.
Read the full abstract ▾
All authors Avijit Dasgupta, Shayon Dasgupta, Zakaria Laskar, C. V. Jawahar, Karteek Alahari
Computer Vision
PRI-Net: A Lightweight Multimodal Framework for 3D UAV Localization
Zhixuan Chen, Jialiang Lu, Zhong Ye et al. (+2 more)
Accurate 3D localization of unmanned aerial vehicles (
UAVs) remains challenging for existing
multimodal approaches due to sparse
LiDAR geometry, modality-imbalanced fusion, and redundant feature transmission over constrained edge-to-server links. To address these limitations, we propose PRI-Net, an efficient and lightweight
multimodal fusion framework for
UAV localization that integrates point cloud splatting, residual
attention fusion, and an information bottleneck. Specifically, a 3D point cloud splatting (3DPCS) strategy is introduced to transform sparse
LiDAR observations into geometrically consistent dense depth maps. A residual
attention fusion (RAF) module is then designed to alleviate modal bias by using an image branch for coarse estimation and a gated fusion branch for refinement. In addition, a
multimodal information bottleneck (MIB) module compacts features by filtering task-irrelevant redundancy. Experiments show that PRI-Net achieves high localization accuracy with lightweight architectures, while reducing feature dimensionality and improving edge-to-server
UAV sensing efficiency and robustness.
Read the full abstract ▾
All authors Zhixuan Chen, Jialiang Lu, Zhong Ye, Yinghui He, Guanding Yu
Computer Vision
DynEoMT: Learning Object Dynamicity from Online Segmentation Queries
Calvin Galagain, Martyna Poreba and François Goulette
Video segmentation models recognize and track objects over time, but they do not indicate whether each segmented region moves independently of the observing camera. This dynamicity attribute cannot be inferred from semantics alone and is confounded by camera ego-motion. We introduce \method, an online framework that augments query-based
video segmentation with region-level dynamicity prediction. It jointly produces the original
segmentation outputs and a dynamic or static state for each predicted region. At inference, DynEoMT uses only the current frame and propagated queries, without optical flow, depth, camera pose, previous RGB frames, or feature maps. Because established
video segmentation benchmarks do not annotate this attribute, we also introduce a class-agnostic offline supervision pipeline using camera-compensated optical flow and confidence-aware temporal filtering. Across VIPSeg, OVIS, YouTube-VIS 2022, and VSPW, DynEoMT achieves balanced accuracies of 84.3, 68.0, 68.6, and 87.6, respectively, while largely preserving
segmentation performance. These results show that
segmentation-region dynamicity can be learned from propagated queries, enabling its online prediction without a dedicated motion-processing pipeline at inference. The complete code will be released as open source to enable full reproduction of the method and experiments.
Read the full abstract ▾
Computer Vision
AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video
Jiaming Tan, Mingliang Zhai, Zhen Li et al. (+3 more)
Interactive
video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency. Existing approaches face a representation trade-off: perspective models operate on local views and must preserve off-screen content over long rollouts, whereas broader spatial coverage is typically obtained by synthesizing full-sphere
videos or constructing explicit 3D representations. Motivated by the complementary roles of global context and selective local acuity in visual perception, we present AlayaVista, a camera-controllable streaming
video world model that decouples panoramic world
evolution from perspective observation synthesis. Given a single perspective image, AlayaVista constructs a 360-degree scene prior using a pretrained panorama expansion model and then evolves the scene as a camera-conditioned panoramic latent state. A latent viewport renderer maps this state to the requested perspective
video latents, while a perspective refiner restores details, suppresses artifacts, and performs super-resolution. To support efficient streaming, we adapt the panoramic generator to chunk-autoregressive generation and distill both panoramic generation and perspective refinement into few-step processes. To provide the supervision required by this design, we construct MUGEN, a large-scale real-world panoramic
video dataset containing 1,318 hours of
videos at resolutions of at least 4K, together with rich semantic and
geometric annotations.
Read the full abstract ▾
All authors Jiaming Tan, Mingliang Zhai, Zhen Li, Yuwei Wu, Chuanhao Li, Kaipeng Zhang