Computer Vision · Cross-listed from cs.RO
EA-Nav: Learning Safe Visual Navigation Policies with Embodiment Awareness
Jialu Zhang, Yong Du, Xianda Guo et al. (+6 more)
Cross-embodiment
navigation is a key challenge in
embodied intelligence. Due to differences in embodiment, the same visual observation may imply different actions for different
agents, making prediction ambiguous when relying solely on
vision. Existing studies mainly rely on reinforcement learning, which requires large-scale interaction and careful reward design, making it difficult to support scalable pretraining and real-world adaptation. In contrast, imitation-learning-based approaches remain limited. To address these challenges, we propose an imitation-learning-based embodiment-aware
navigation framework with a modular multi-stage design. In pretraining, we construct a cross-embodiment
navigation dataset from Internet
videos and introduce embodiment geometry as conditional
tokens to reduce action ambiguity under the same observation. In fine-tuning, we design a
multimodal information injection mechanism based on a decoupled architecture. Specifically, we design a trajectory augmentation strategy to generate high-risk samples, which are used to train spatial perception and risk-aware correction separately, thereby explicitly incorporating embodiment geometry for safe
navigation. Experimental results show that the proposed method effectively improves
navigation performance across different embodiment settings, demonstrating the effectiveness of incorporating embodiment geometry into
embodied navigation.
Read the full abstract ▾
All authors Jialu Zhang, Yong Du, Xianda Guo, Shunwang Sun, Xinqi Liu, Yue Sun, Guodong Lu, Wei Sui, Jituo Li
Computer Vision
Current Injection Spiking Neural Network for Infrared and Visible Image Fusion
Rui Zhao, Zhuoyuan Li, Wenrui Li et al. (+4 more)
Infrared and visible image fusion (IVIF) integrates the complementary information of two modalities into a single image with richer scene content. While existing methods are largely built on artificial neural networks (ANNs), which densely compute over all activations, spiking neural networks (SNNs) communicate through sparse binary spikes and compute only where and when a spike occurs, offering a route to more energy-efficient fusion. However, directly applying SNNs to IVIF creates a fundamental tension: cross-modal fusion relies on fine-grained responses from both modalities, whereas binary spikes can discard complementary cues that remain below the firing threshold. The membrane potential retains these subthreshold responses before firing, letting both modalities jointly shape the output when integrated at this stage. Building on this, we propose CIS-Fuse, a spiking network that performs cross-modal fusion directly at the membrane-potential level. At its core is the current injection spiking (CIS) operator, which injects one modality as a gated auxiliary current into the driving
neuron of the other, so the two integrate before spike firing, with a per-channel learnable injection strength that adaptively regulates the modulation magnitude. Building on CIS, we construct a bidirectional cross-modal fusion (BCMF) module and deploy it on a dual-branch architecture with asymmetric stacking depths, where the two branches develop a clear functional specialization. Extensive experiments on four IVIF
benchmarks and on downstream detection and
segmentation show that CIS-Fuse achieves fusion quality on par with state-of-the-art ANN-based methods while inheriting the energy efficiency of spike-based computation, with roughly an order of magnitude lower inference energy than the similarly-sized ANN-based DCEvo. Code will be released upon publication.
Read the full abstract ▾
All authors Rui Zhao, Zhuoyuan Li, Wenrui Li, Yanchen Dong, Yajing Zheng, Giuseppe Valenzise, Weisi Lin
Computer Vision
Robust Activation Map Rectification for Weakly Supervised Volumetric Segmentation: Temporal Coherence as a Free Lunch
Renshu Gu, Jialiang Chen, Fei Gao et al. (+8 more)
Weakly supervised
segmentation relies heavily on class activation maps (CAMs) to initially localize target regions. However, CAMs are often noisy and prone to catastrophic failures. Existing remedies typically introduce additional training stages or prototype learning, increasing computational cost and reducing robustness. In this paper, we propose a training-free prototype-free framework that rectifies unreliable CAMs by exploiting temporal and structural coherence in volumetric data as a free lunch. Our approach is built on two key components. First, we introduce Variance-Reduced Activation Aggregation (VRAA) which suppresses noise and amplify coherent semantic signals. We provide a theoretical justification by modeling CAMs as high-dimensional random vectors and show that aggregation yields provable variance reduction. Second, we design a Bidirectional Extremity Rectification (BER) mechanism that detects and rectifies implausible activations through bidirectional extremity checks, effectively mitigating extreme-value failures without learning additional parameters. Our method is model-agnostic and can be seamlessly integrated with existing pipelines. Extensive experiments on multiple public
benchmarks demonstrate substantial improvements over state-of-the-art weakly supervised methods, achieving up to 20% Dice and 40% mIoU gains while reducing inference time by more than 5 times. These results indicate that leveraging coherence as an implicit inductive bias yields a principled and efficient approach to stabilizing weakly supervised volumetric
segmentation. Our code will be available.
Read the full abstract ▾
All authors Renshu Gu, Jialiang Chen, Fei Gao, Hang Su, Jun Qi, Jiamin Xu, Yicheng Shen, Jiayu Zhang, Jiaxi Pan, Caiming Zhang, Gang Xu
Computer Vision · Cross-listed from cs.RO
KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding
Zeyu Liu, Zhangzhe Zhu, Yang Zhang et al. (+3 more)
Evaluating the physical consistency of
embodied world models(EWMs) is a critical open challenge. While closed-loop evaluation via simulator rollouts offers a more faithful assessment of physical plausibility than open-loop alternatives, existing frameworks almost exclusively rely on Inverse Dynamics Models(IDMs) for action extraction. Due to the intricate mapping from 2D pixel space to 3D kinematic space, the learned IDMs can be brittle to data outside their training distribution, resulting in unreliable action extraction from the generated
videos with novel objects and scenarios. This creates an unavoidable attribution ambiguity between world model inaccuracies and extractor errors. To reduce this ambiguity, we present KineBench, an IDM-free closed-loop
benchmark for EWMs, built upon an explicit kinematic grounding pipeline. Given a generated
video, KineBench employs cascaded visual foundation models to directly extract 6D end-effector poses from individual frames, which are then executed in a physics simulator for closed-loop validation. Beyond execution-based task success, KineBench incorporates two classical 3D kinematic metrics–Spectral Arc Length (SPARC) and the Maruyama Manipulability Index–to characterize trajectory smoothness and kinematic feasibility from a
robot-centric perspective. Built on 20 diverse manipulation tasks in ManiSkill3, KineBench evaluates EWMs across four progressive suites: basic execution, task transfer, visual out-of-distribution generalization, and complexity-conditioned scaling. Evaluation across frontier models reveals task-complexity-bounded nonlinear scaling in
embodied video generation, providing empirical guidance for future data-scaling strategies.
Read the full abstract ▾
All authors Zeyu Liu, Zhangzhe Zhu, Yang Zhang, Chenyou Fan, Chenjia Bai, Xuelong Li
Computer Vision · Cross-listed from eess.IV
PRISM-DR: Per-lesion Retinal Inference with Specialist Models for Diabetic Retinopathy
Zübeyr Özeren and Tansel Uyar
Diabetic retinopathy is a leading cause of preventable blindness; its early lesions are small, low contrast, and easily missed in manual screening. Most automated detectors handle the four non-proliferative DR lesions: microaneurysms, hemorrhages, hard exudates, and soft exudates, with a single multi-class model, even though these lesions differ sharply in size, color, morphology, and prevalence, so a shared model favors common, easy classes over rare, difficult ones. We present PRISM-DR, a lesion-specific pipeline that trains one single-class detector per lesion, each with its own configuration. From a raw fundus image, the pipeline applies region of interest cropping, fundus-specific preprocessing, four parallel YOLO detectors, tiling, per-lesion ensembling of five cross-validation folds, and an inter-lesion suppression step that resolves overlaps by physical lesion size and clinical priority rather than confidence. Per lesion, the best of five YOLO generations is selected, and augmentation is tuned by
Bayesian optimization. Trained on IDRiD with stratified five-fold cross-validation, the system reaches a test mAP50 of 0.527 and F1 of 0.529, highest AP50 on hard exudates with 0.561. Without fine-tuning, the models transfer well where the imaging scale is close to IDRiD and degrade as field of view and resolution depart. These modest absolute results reflect a small single-source training set and a difficult task; however, treating each lesion as a separate detection problem is a practical alternative to a single multi-class model.
Read the full abstract ▾
Computer Vision
Memory-Augmented Multimodal Large Language Models for Small Object Understanding in Streaming Aerial Videos
Penglei Sun, Yehua Huang, Zhuoli Tao et al. (+8 more)
Language-guided aerial perception aims to understand user-specified tiny targets in complex unmanned aerial vehicle (
UAV) scenes. In real
UAV deployment, the
UAV must respond while it flies, so such perception runs in an online streaming manner, where frames arrive sequentially and the model responds to each one without access to future frames. However, applying current
Multimodal Large
Language Models (MLLMs) to this setting raises two challenges. First, targets viewed from the air are often tiny, yet the visual compression in existing MLLMs treats all regions equally and discards their fine-grained details. Second, understanding a continuous stream requires past-frame context, yet retaining the entire history is infeasible on resource-constrained onboard hardware, whereas discarding it causes the target to drift or disappear. We address the tiny object and streaming challenges from both data and method perspectives. From the data perspective, we present DroneEyes, the first pixel-level and open-vocabulary referring-
segmentation dataset for tiny aerial targets, comprising $2,140$ high-definition
videos and $176,623$ pairs across Object Description and Referring Expression tasks, with dense per-frame masks. From the method perspective, we propose SkyAnchor, an MLLM with two designs to the above challenges: a Semantics-Aware
Token Router that preserves small-target under a reduced visual-
token budget, and a Hierarchical
Memory Bank that keeps the target consistently understood on streams.
Read the full abstract ▾
All authors Penglei Sun, Yehua Huang, Zhuoli Tao, Xiang Li, Runwei Guan, Yaoxian Song, Kaiyong Zhao, Henghui Ding, Bo Han, Yang Yang, Xiaowen Chu
Computer Vision
MoAKE: Toward Unified All-in-One Action Quality Assessment via Mixture of Action Knowledge Experts
Huangbiao Xu, Huanqi Wu, Xiao Ke et al. (+3 more)
Action Quality Assessment (AQA) aims to objectively evaluate performance quality from action
videos. Most existing methods follow a “one-by-one” paradigm, training a separate model for each action type. This setting limits real-world deployment, as it requires prior action-type knowledge to select the corresponding model and suffers from poor generalization across diverse actions. To address these limitations, we study the challenging task of all-in-one AQA, which aims to assess heterogeneous actions within a single unified model. We propose a novel Mixture of Action Knowledge Experts (MoAKE) framework, designed to mitigate negative knowledge transfer caused by large semantic discrepancies among actions. MoAKE learns complementary experts that capture diverse action patterns within a shared semantic space and dynamically aggregates their knowledge to adapt the assessment to the input action. Each expert is tailored with segment-aware prototypes to handle varying temporal lengths, together with an Adaptive Intra- and Inter-Segment Relationship Modeling (AIISRM) module to model multi-granularity temporal dynamics. Furthermore, we establish comprehensive
benchmarks for all-in-one as well as zero/few-shot AQA. Extensive experiments on three long-term
datasets demonstrate that MoAKE significantly outperforms existing methods in the all-in-one setting, while also achieving consistent generalization on three short-term
datasets under zero/few-shot evaluation. Code is available at https://github.com/XuHuangbiao/MoAKE.
Read the full abstract ▾
All authors Huangbiao Xu, Huanqi Wu, Xiao Ke, Jiaxin Cai, Junyi Wu, Jinglin Xu
Computer Vision
Lean-SAM2: Target-Anchored Memory and Encoder Acceleration for SAM2
Xudong Ouyang, Wenlun Zhang, Yimin Xu et al. (+2 more)
The Segment Anything Model 2 (SAM2) has advanced temporal promptable
segmentation, yet its deployment remains hindered by heavy
memory cross-
attention overhead and redundant full-frame visual feature extraction. While recent methods explore efficiency via heuristic
memory pruning and window-based sparse routing, they typically suffer from catastrophic performance degradation in complex
segmentation scenarios replete with occlusions and distractors. To resolve these limitations, we propose Lean-SAM2, a holistic lightweight framework designed to address the above vulnerabilities while systematically eliminating computational redundancies. Specifically, Lean-SAM2 integrates three collaborative mechanisms: (1) Target-Anchored
Memory Pruning (TAMP) safeguards target
tokens against deceptive
attention by modulating raw
attention significance with semantic consistency against
prompt-derived foreground anchors; (2) Temporal Condensation with Insurance
Memory (TCIM) condenses historical context via a visibility-gated fusion while conditionally archiving high-confidence entries in a parallel insurance bank; and (3) Target-Anchored Risk-Aware Routing (TARR) selectively activates the heavy image encoder for target-related windows based on anchor similarity, utilizing a risk-aware fallback policy to trigger full-frame refreshes during volatile transitions. Extensive evaluations across multiple challenging
benchmarks demonstrate that Lean-SAM2 establishes a superior balance between accuracy and efficiency. For example, on the LVOSv2 validation
dataset, Lean-SAM2 achieves overall inference speedups of $1.412\times$ and $1.417\times$ on the SAM2.1-Large and SAM2.1-Base+, respectively, significantly outperforming Efficient-SAM2 while boosting the corresponding $\mathcal{J}\&\mathcal{F}$ scores by $5.0\%$ and $3.6\%$. Code is available at https://github.com/DeawhaleQwQ/Lean-SAM2.
Read the full abstract ▾
All authors Xudong Ouyang, Wenlun Zhang, Yimin Xu, Huazhong Liu, Yunshan Zhong
Computer Vision · Cross-listed from cs.AI
Silent Failures in Multimodal Agentic Search:A Diagnostic Taxonomy and Cross-Judge Evaluation
Zhengxian Wu, Junjie Gao and Kai Yang
Multimodal agentic search systems increasingly rely on external tools to answer knowledge-intensive visual questions. However, existing evaluations mainly focus on final-answer accuracy and may miss failures in the search trajectory. In this work, we study such hidden reliability issues as silent failures. We introduce a six-category taxonomy covering modality shortcuts, phantom grounding, wrong-evidence-right-answer cases, over-
retrieval laundering, cross-modal contradiction, and provenance
hallucination. Based on this taxonomy, we build a trajectory-level diagnostic pipeline that evaluates both answer correctness and evidence-grounding quality under a unified ReAct-style scaffold. Experiments on MMSearch-Plus trajectories across four frontier
multimodal models show that surface accuracy consistently overestimates true trajectory-level correctness. We further use cross-judge validation, blank-image stress tests, and tool ablations to show that silent failures are capability-dependent and often shift rather than disappear. Home-page: https://github.com/DingWu1021/silent-failures-
multimodal-agentic-search
Read the full abstract ▾
Computer Vision
Trace: A Taxonomy-Guided Environment for Multidomain Visual Reasoning
Md Tanvirul Alam
Reinforcement learning with verifiable rewards (RLVR) has substantially improved
language-model
reasoning, yet its extension to
vision-
language models remains constrained by the lack of training data that are simultaneously broad, exactly verifiable, and reproducible. We introduce Trace, a taxonomy-guided environment for multidomain visual
reasoning. Trace factorizes task construction into a scene grammar and an executable task program, separating visual realization from answer computation. A shared semantic state determines the rendered image,
prompt, typed answer, verifier state, and replayable instance trace. The resulting environment comprises 1,000 tasks over 277 scene grammars and 11 visual domains, with controlled semantic and visual variation. RLVR on 64,000 Trace instances improves the macro-average across 24 external
benchmarks by 3.51 percentage points for Qwen2.5-VL-3B and 4.06 points for Qwen2.5-VL-7B, providing evidence that broad procedural training can transfer beyond the generated task distributions. Project page: https://maveryn.github.io/trace/.
Read the full abstract ▾
Computer Vision
Physics-Aware Complex-Valued State Space Model with Scattering-Prior Feature Modulation for PolSAR Image Classification
Fangyan Zhang, Fan Zhang, Shiqi Zhou et al. (+3 more)
Polarimetric synthetic aperture radar (PolSAR) image classification is a representative task for physics-aware GeoAI, where land-cover semantics are closely coupled with electromagnetic scattering mechanisms. Many existing complex-valued networks can preserve amplitude-phase information, but they are often limited in long-range spatial dependency modeling and usually incorporate polarimetric priors only as input-level or shallow auxiliary features. As a result, physical knowledge is insufficiently used to guide deep feature
evolution. To address this issue, this paper proposes CV-SSMNet, a physics-aware complex-valued state-space network with scattering-aware feature modulation for PolSAR image classification. The proposed method builds a complex-valued state-space model (CV-SSM) in the original complex domain to capture long-range spatial dependencies while preserving polarimetric amplitude-phase coupling. Meanwhile, seven physically meaningful scattering priors, are encoded as FiLM-style modulation signals to adaptively recalibrate complex-valued representations during feature
evolution. CV-SSMNet further integrates multi-scale complex convolutions, branch-wise CV-SSM encoding, prior-guided recalibration, and lightweight global context aggregation, enabling physically guided representation learning from local scattering structures to global spatial context. Experiments on three L-band
benchmark datasets and an additional P-band BIOMASS evaluation demonstrate that CV-SSMNet achieves competitive accuracy, improved regional consistency, and better boundary preservation, supporting the effectiveness of
embedding polarimetric scattering mechanisms into complex-valued long-range GeoAI representation learning.
Read the full abstract ▾
All authors Fangyan Zhang, Fan Zhang, Shiqi Zhou, Jun Ni, Carlos López-Martínez, Qiang Yin
Computer Vision
WASABI: Whole-graph Assignment-based Stabilizer for lAne topology By Inter-frame tracking
Tetsuhiro Uchida, Myu Sasaki, Kensho Nakajima et al. (+2 more)
Autonomous driving requires understanding the road as a graph of drivable lanes and their connectivity, beyond the ego lane alone, to follow routes through intersections and reason about cross- and merging-traffic. Recent perception models infer such lane
topology, i.e., lane segments together with their inter-lane connectivity (LCLC), from onboard sensors over a 360-degree BEV view. Due to neural perception's imperfections, their outputs retain structural instabilities such as missed detections, lost or incorrect LCLC, over-detection, and label flicker. This paper presents WASABI, a real-time post-processing pipeline that stabilizes lane
topology outputs both within and across frames by treating lane segments and their LCLC connectivity as joint tracking targets, under onboard real-time constraints (10 Hz / 20 ms / up to 200 input lanes). The pipeline integrates segment tracking with connectivity, noise-robust
topology-aware refinement, and a resource-constrained real-time design. On internal validation data (16 sequences), WASABI improves LCLC detection F1 from 0.834 to 0.948 (+0.114, +13.6%) and reduces centerline lateral error from 2.50 m to 0.95 m, while reducing detection false-positives by 24.6%. Temporal-stability metrics on the same data show LCLC toggle rate reduced by 63.3% and boundary-label flicker rate by 30.2%, confirming across-frame stabilization beyond per-frame accuracy.
Read the full abstract ▾
All authors Tetsuhiro Uchida, Myu Sasaki, Kensho Nakajima, Yasuhiro Shimada, Toru Saito
Computer Vision
Frequency-Hierarchical Active k-Space Sampling for Diagnostic MRI
Ruru Xu, Kian Anvari Hamedani, Zhikai Yang et al. (+1 more)
Active sampling for accelerated MRI must distribute a tight sampling budget across spatial frequencies that carry very different kinds of information. Low frequencies hold most of the anatomical context; high frequencies carry the fine details that drive pathology assessment. Existing active samplers either treat both regions identically or restrict the action space to entire Cartesian rows, which forces a poor compromise at high acceleration. We propose HieraSample, a task-driven framework built around this hierarchy. A cosine-annealed curriculum lowers the acceleration factor from 20x to 4x across 80 acquisition steps while keeping a fully-sampled low-frequency disk at every step; a Mamba-based policy then picks individual high-frequency coordinates from features extracted by dual disease and severity classifiers. The reward is the per-sample reduction in class-weighted cross-
entropy after each action, so a positive reward corresponds directly to a more confident correct prediction. On the fastMRI+ knee
benchmark, HieraSample matches the fully-sampled oracle on ACL diagnosis from 4x to 10x acceleration, and improves on a recent Cartesian baseline by as much as 20.4 AUC points on ACL severity.
Read the full abstract ▾
All authors Ruru Xu, Kian Anvari Hamedani, Zhikai Yang, Ilkay Oksuz
Computer Vision
Look Before You Edit: Attention-Guided Camera Placement and Multi-View Alignment for 3D Gaussian Splatting Editing
Jaeyeon Park, Taeho Kang and Youngki Lee
Text-driven 3D scene editing with 3D Gaussian Splatting (3DGS) typically applies a 2D
diffusion editor to views rendered from fixed training cameras, limiting both the spatial coverage of edits and the user's freedom to target specific objects in complex scenes. We present LB-Edit, a framework that addresses two coupled problems: where to place editing cameras for localized edits, and how to make per-view edits agree with one another so that the 3D scene remains consistent after fine-tuning. First,
Attention-Guided Editing Camera Placement (ACP) probes the
diffusion model's self- and cross-
attention at multiple candidate camera distances to find where
attention is well-contained in the region of interest, then places a compact, geometrically diverse editing camera set at that
attention-optimal distance. Second, Multi-view
Attention Alignment (MAA) steers the editor toward the same edit across views along two axes: it aligns appearance by sharing self-
attention features via
token-level correspondence, and aligns spatial location by lifting cross-
attention maps onto the 3D Gaussians as a shared 3D
attention field, suppressing both appearance and spatial drift. Experiments on multi-object and single-object scenes show that our method achieves the highest user preference in instruction fidelity, multi-view consistency, and editing locality, using as few as 5 editing views and reducing latency by up to 7x over existing methods.
Read the full abstract ▾
Computer Vision
DRGBT-1K: A Large-scale High-quality Benchmark for Dynamic RGBT Tracking
Zhaodong Ding, Chenglong Li, Zeyu Ding et al. (+2 more)
Dynamic RGBT (DRGBT) tracking aims to continuously localize a target when the available sensing modalities and observation platforms vary over time. Compared with conventional RGBT tracking with fixed RGBT inputs and a fixed observation platform, DRGBT tracking is more consistent with real-world collaborative perception systems, where targets may be observed by heterogeneous sensors from different viewpoints. However, existing
benchmarks are still insufficient for systematically evaluating tracker robustness under real dynamic modality variations and cross-platform transitions. To address this limitation, we make the following contributions. 1) We construct DRGBT-1K, a large-scale high-quality
benchmark for DRGBT tracking. It contains 1,045 sequences captured entirely in real-world scenarios and 795K RGBT frame pairs collected using
UAVs and handheld RGBT devices, encompassing diverse real-world scenes, pronounced viewpoint changes, modality variations, and target appearance discontinuities. 2) We provide comprehensive annotations for fine-grained evaluation, including dense bounding boxes, target category labels, challenge attributes, frame-level modality labels and platform labels. DRGBT-1K covers 24 target categories, more than 15 scene types and 15 challenge attributes. 3) We establish a comprehensive
benchmark by evaluating 20 representative
multimodal tracking methods, including conventional RGBT trackers, modality-missing RGBT trackers, and DRGBT trackers under a unified evaluation protocol. 4) We release an unaligned version of DRGBT-1K and derive UGVT-1K to support broader research on unaligned
multimodal tracking and
UAV-ground collaborative tracking. 5) We develop an online evaluation platform for DRGBT-1K and provide a leaderboard that collects all methods evaluated on this
benchmark.
Read the full abstract ▾
All authors Zhaodong Ding, Chenglong Li, Zeyu Ding, Futian Wang, Jin Tang
Computer Vision
Global Building Area Estimation Products: How Accurate Are They?
Saad Lahrichi, Doa'a Allabadi, Kyle Bradbury et al. (+1 more)
Geo-spatial rasters of building footprint area are useful for a variety of tasks, such as monitoring urbanization, improving energy efficiency, and tracking greenhouse gas emissions. There are now multiple global building raster
datasets, however there lacks an independent, comprehensive, and fair assessment of their accuracy. In this work, we evaluate the accuracy of four major global building products: Global Human Settlement Layer (GHSL), Microsoft's TEMPO (TEMPO), The Global Building Atlas (GBA), and Overture. As ground truth for assessing their accuracy, we use ORBITaL-Net, a globally diverse
dataset of manually labeled building footprints. To ensure
fairness, we evaluate products on grids of multiple spatial resolutions, and several conventional performance metrics. Our results indicate that either GBA or TEMPO generally achieves the highest overall accuracy, depending upon the particular evaluation criteria. We also stratify the accuracy of each product by several factors: geographic location, population density, and income groups. The results reveal that product accuracy can sometimes vary significantly with respect to these factors. Notably, all products are significantly less accurate in Africa and Asia. Most products also suffer significant accuracy reduction in high-density urban areas.
Read the full abstract ▾
All authors Saad Lahrichi, Doa'a Allabadi, Kyle Bradbury, Jordan Malof
Computer Vision
Extending a Large View Synthesis Model for Multi-view Panoptic Segmentation
Kwonyoung Ryu, In-Jae Lee, Jonghyun Jin et al. (+3 more)
Large view synthesis models synthesize novel views through cross-view
attention without explicit 3D representations, and recent studies have shown that they learn accurate spatial correspondence from RGB supervision alone. We observe that this correspondence generalizes beyond appearance. When non-photorealistic signals such as binary encoded panoptic labels are passed through the model, they are propagated to novel views with consistent spatial structure. These results indicate that the correspondence learned for RGB view synthesis can also propagate view-independent per-pixel labels. From this observation, we present the first work to extend large view synthesis models beyond appearance rendering to 3D scene understanding. We propose a panoptic
segmentation pipeline that reuses a frozen view synthesis model to propagate panoptic labels from input views to novel views, without 3D reconstruction or any
segmentation-specific training of the view synthesis model. Given panoptic labels on the input views, we encode them into binary channel representations and pass them through the same model to render target-view
segmentation. On ScanNet, our method achieves
segmentation quality on par with Gaussian based approaches requiring explicit 3D reconstruction, while outperforming them in novel view synthesis by more than 7 dB. The label propagation also transfers across
datasets, surpassing these approaches on Replica without any fine-tuning.
Read the full abstract ▾
All authors Kwonyoung Ryu, In-Jae Lee, Jonghyun Jin, Hyunjee Lee, Jongmin Lee, Jaesik Park
Computer Vision · Cross-listed from cs.RO
EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstration
Zuhao Ge, Yuchen Zhou, Weitao Zhou et al. (+8 more)
Robust
embodied robots should be able to recover from failures and retry tasks in order to operate reliably in unstructured and noisy real-world environments. Achieving this capability requires training policies on data that captures recovery behaviors. However, collecting such data through
robot teleoperation is difficult to scale, as it is time-consuming to induce diverse failure states, perform corrective actions, and reset the environment. This challenge is further exacerbated by the high diversity of failure modes, which demands substantially more recovery data than success demonstrations. In this work, we show that
egocentric human data capturing failure recovery processes provides a scalable alternative. By efficiently arranging task-level failure configurations and recording short recovery segments, human operators can generate more than 10x as much valid recovery data per hour compared to
robot teleoperation under our protocol. To address the embodiment gap between human and
robot, we propose EgoRecovery, a co-training framework for learning recovery behavior, where human recovery demonstrations are aligned to a compact corrective-intent space shared with
robot data, which captures the timing and magnitude of correction. Only a small number of
robot recovery demonstrations are required to connect this intent to executable
robot actions. At deployment, a learned recovery gate predicts when correction is needed from
robot observations and activates the corrective intent only in recovery states. Experiments on real-world recovery tasks show that EgoRecovery improves success from failure starts over
robot-only recovery, direct co-training with human recovery data, and direct intent-transfer baselines.
Read the full abstract ▾
All authors Zuhao Ge, Yuchen Zhou, Weitao Zhou, Minglei Li, Xinyu Li, Chao Wu, Hanwen Zhao, Haotian Wang, Zuxuan Wu, Xiaosong Jia, Yu-Gang Jiang
Computer Vision
Efficient Tracking and Understanding Object Transformations
Yihong Sun and Bharath Hariharan
Tracking objects through state transformations is essential for understanding real-world dynamics. However, existing methods are computationally expensive. TubeletGraph recently showed impressive capabilities, but its inference cost (~$4.4$ seconds per object-frame on VOST) precludes any real-time deployment possibilities. We observe that TubeletGraph's overhead arises from building a spatiotemporal partition of the input
video: (1) entity
segmentation is computed densely for every frame regardless of whether a transformation occurs, and (2) every entity in the scene is tracked, scaling cost with scene complexity rather than the number of transformations of interest. To address both, we propose FluxGraph, a reactive variant that uses SAM2's internal multi-mask disagreement as a lightweight trigger for transformation detection, and removes the need for tracking all entities in the given
video. FluxGraph is ~$3.3\times$ faster than TubeletGraph on VOST while improving tracking performance and preserving state graph quality. Furthermore, we also observe consistent speedups of $3.7-10.7\times$ across VSCOS, M$^3$-VOS, and DAVIS17 while maintaining performance. Code is publicly available at https://github.com/YihongSun/FluxGraph.
Read the full abstract ▾
Computer Vision
An Exploratory Analysis of Pain Localization via Explainable Computational Modeling
Ioannis Kyprakis, Stefanos Gkikas, Eric Nichols et al. (+2 more)
Automatic pain localization, which involves identifying the anatomical origin of pain from peripheral physiological signals without patient self-report, is a clinically critical but largely unaddressed problem, particularly for non-verbal patients. This paper presents a systematic comparison of classical feature engineering and deep sequence learning for subject-independent three-class pain localization using the AI4Pain 2026 Challenge
dataset, which comprises four synchronously recorded wearable modalities: electrodermal activity, blood volume pulse, respiration, and peripheral oxygen saturation recorded from 65 participants under controlled TENS-induced pain. A 115-dimensional hand-crafted feature set spanning time-domain, frequency-domain, modality-specific, and cross-modal descriptors is benchmarked against end-to-end deep architectures. Extremely Randomized Trees achieves the highest macro-F1 of 0.539, outperforming the best deep model by 7.4 percentage points, with EDA spectral features emerging as the dominant discriminators. A consistent 26-point gap between pain detection (F1\,=\,0.815) and localization (F1\,=\,0.552) across all models points to a fundamental ceiling imposed by the anatomical diffuseness of peripheral autonomic pathways at 10-second resolution.
Read the full abstract ▾
All authors Ioannis Kyprakis, Stefanos Gkikas, Eric Nichols, Yu Fang, Manolis Tsiknakis
Computer Vision · Cross-listed from cs.LG
Analytic Distribution of Classifier-Free Guidance for Schedule Design
Enze Jiang and Zheng Ma
Classifier-free guidance (CFG) is the default mechanism for conditional generation in
diffusion models, but the distribution sampled by its deterministic guided dynamics is not captured by the usual product-distribution heuristic $p_0^\omega q_0^{1-\omega}$. We analyze CFG through the probability flow ODE and derive exact analytic path-integral representations of the induced distributions for both constant and time-dependent guidance. The resulting formulas show that CFG modifies $p_{t_0}$ by an exponential path-integral correction, and that a time-dependent schedule enters this correction through the weight $\omega(t)-1$. This characterization explains how score discrepancies accumulate along sampling trajectories and motivates Distribution-Guided CFG (DG-CFG), a schedule that balances timestep contributions while accounting for signal strength and low-noise score-error amplification. A toy model with analytic scores closely verifies the predicted distributions. Across Stable
Diffusion~1.5, Stable
Diffusion~2.1, and Stable
Diffusion~XL, DG-CFG yields a stronger diversity–fidelity trade-off and robustly mitigates the saturation and quality degradation caused by strong constant or heuristic guidance. Complete NFE experiments on Stable
Diffusion~1.5 and Stable
Diffusion~2.1 confirm that these gains persist across sampling budgets, while fixed-quality experiments on both backbones show that DG-CFG reaches target metrics with fewer sampling steps.
Read the full abstract ▾
Computer Vision
ReFace: Reorganizing Facial Spatiotemporal Representations for Improved Pain Assessment
Stefanos Gkikas, Yu Fang, Christian Arzate Cruz et al. (+2 more)
Automatic pain assessment from facial
video remains challenging due to the spatial heterogeneity of pain-related facial cues. This study proposes ReFace, a spatial reorganization pipeline that divides facial input into four spatial quadrants before tokenization, rather than processing the entire face as a single region. Evaluated on the AI4Pain
dataset, the proposed approach achieves $56.00\%$ accuracy on the test set using
video only, achieving the highest reported accuracy under the fixed AI4Pain
benchmark protocol among the compared methods. Notably, the four-quadrant configuration processes the same total pixel budget as the full-face input, yet achieves higher accuracy, suggesting that spatial reorganization can improve performance under the proposed tokenization design. A single quadrant region, processing just one quarter of those pixels, remains competitive at a fraction of the computational cost.
Read the full abstract ▾
All authors Stefanos Gkikas, Yu Fang, Christian Arzate Cruz, Muhammad Umar Khan, Raul Fernandez Rojas
Computer Vision
A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities
Stefanos Gkikas, Christian Arzate Cruz, Valentina Becchetti et al. (+3 more)
Pain is a complex and pervasive phenomenon affecting a large percentage of the population, and accurate assessment is essential for effective clinical management and intervention. Computational pain recognition systems enable continuous monitoring, support clinical decision-making, and help mitigate pain-related distress and functional decline. This study introduces a unified tokenization framework for heterogeneous 3D modalities in pain recognition that provides a single processing pipeline across behavioral and
brain-activity 3D data, without requiring separate architectures for each modality or handcrafted inductive biases. The framework preserves spatial, temporal, and time–frequency structure while mapping diverse inputs into a shared
token space. Extensive experiments show that the proposed approach effectively processes facial
videos and fNIRS data in both raw-signal and spectrogram-based representations. On the AI4Pain
benchmark dataset, the proposed framework achieves state-of-the-art performance while maintaining high computational efficiency and enabling real-time assessment on both GPU and CPU hardware.
Read the full abstract ▾
All authors Stefanos Gkikas, Christian Arzate Cruz, Valentina Becchetti, Muhammad Umar Khan, Alessandro Giuseppi, Raul Fernandez Rojas
Computer Vision
Point-Selection Fine-Tuning Framework for Robust Point Cloud Classification
Da Li, Chang Ma and Dongfu Yin
Noisy and corrupted points can substantially degrade point cloud recognition performance, especially under challenging corruption settings. In particular, full fine-tuning of 3D pre-trained models may amplify the influence of outliers and overwrite robustness priors learned during pre-training, while naive parameter-efficient adaptation remains sensitive to corrupted
tokens. To address this issue, we propose PSFT, a point-selection fine-tuning framework that improves robustness while remaining parameter-efficient. PSFT first estimates point-wise influence from pre-pooling features and adaptively retains minimally influential points to suppress outliers. Based on the selected subset, a
prompt generation branch predicts layer-wise
prompt tokens and injects them into a frozen backbone for lightweight downstream adaptation. To further mitigate residual noise after selection, we append a lightweight feature filter with bottleneck MLP transformation and Beta-gated residual blending to refine patch-
token representations before prediction. Extensive experiments show that PSFT consistently reduces corruption error on ModelNet-C and ModelNet40-C across all tested 3D pre-trained backbones, while achieving the strongest ScanObjectNN-C results with ULIP-2 and Uni3D-B among the evaluated tuning strategies. Our implementation can be found at https://github.com/CVChMA/PSFT/tree/master.
Read the full abstract ▾
Computer Vision
SafeGen: Goal-Conditioned Video Diffusion of Safety-Critical Scenarios for VLM-Based Autonomous Driving
Jiangfan Liu, Zexuan Cui, Tianyuan Zhang et al. (+7 more)
VLMs are increasingly deployed in AD systems, creating an urgent need for rigorous safety evaluation under rare yet safety-critical scenarios. Among these, interactions with vulnerable road users represent a major source of real-world failures. However, existing safety-critical scenario generation methods predominantly rely on simulator-based pipelines, which suffer from a substantial sim-to-real gap and often fail to capture realistic, diverse, and unforeseen human-vehicle interaction dynamics. We present SafeGen, a goal-conditioned
diffusion framework for safety-critical scenario generation in VLMADs. Our key insight is to formulate scenario generation as a goal-conditioned
diffusion process, where a predefined catastrophic end-state serves as a strong supervisory signal, guiding the generation of temporally coherent
video trajectories that naturally evolve toward safety-critical outcomes. Building on this formulation, we introduce Context Grounded End State
Reasoning, which leverages VLMs to analyze benign driving contexts and infer latent vulnerabilities in human-vehicle interactions, producing structured end-state specifications that induce high-risk scenarios. Conditioned on these targets, we further propose End State Conditioned
Video Evolution, which grounds semantic threats into physically plausible visual dynamics. Specifically, we instantiate high-risk
agents within the scene via depth-aware
geometric projection, followed by boundary-conditioned
diffusion to generate intermediate frames with consistent motion patterns and temporal coherence. Extensive experiments across 3 VLMADs demonstrate that SafeGen increases the Judge Overall Score, a metric using a VLM judge to evaluate VLMADs' understanding and decision-making, by 24.25% on average compared to SoTA baselines. Furthermore, fine-tuning a VLMAD improves performance in real-world driving scenes by an average of 15.9%.
Read the full abstract ▾
All authors Jiangfan Liu, Zexuan Cui, Tianyuan Zhang, Zonglei Jing, Zonghao Ying, Yaoyuan Zhang, Jiakai Wang, Xiaoqi Jiang, Aishan Liu, Xianglong Liu
Computer Vision
PhenSPINE: A Standardized Benchmark for Spine Pathology Diagnosis
Duong Ngoc Vu, Hai Son Nguyen, Trong-Nghia Nguyen et al. (+4 more)
The accurate diagnosis of spinal pathologies depends heavily on radiological interpretation, yet automated systems are hindered by the lack of diverse, high-quality
benchmarks. In this study, we present PhenSPINE, a
Magnetic Resonance Imaging
dataset comprising 16,813 images from 250 patients, curated to facilitate advanced deep learning research. We propose a robust diagnostic
benchmark that integrates state-of-theart convolutional backbones with a Positional Encoding mechanism to explicitly model the anatomical context of intervertebral discs. Evaluating across four standard MRI sequences, our experiments demonstrate that the Sagittal T2-weighted sequence offers the most robust diagnostic value, achieving a superior Macro F1-score of 50.31%. We find that multisequence fusion strategies yield inferior performance compared to this single-sequence baseline, as the images across sequences in our
dataset are significantly compromised by noise interference from surrounding anatomical regions. This work establishes a robust baseline and offers critical insights into sequence selection for spine analysis.
Read the full abstract ▾
All authors Duong Ngoc Vu, Hai Son Nguyen, Trong-Nghia Nguyen, Bien Tran Van, Trang Mai Xuan, Huan Vu, Thien Van Luong
Computer Vision · Cross-listed from cs.RO
NavVerse: Benchmarking Indoor-to-Outdoor Embodied Navigation in Continuous Robot Simulation
Junzhe Wu, Yue Hu, Zeyu Han et al. (+4 more)
Robots deployed in delivery, campus, and emergency-response settings often need to navigate from buildings to streets within a single continuous episode. Existing
benchmarks usually evaluate indoor and outdoor
navigation separately, and many abstract away
robot execution, leaving exit finding, boundary traversal, adaptation, and kinodynamic failures underexplored. We introduce NavVerse, a physics-enabled
benchmark for indoor-to-outdoor
embodied navigation. NavVerse contains 100 indoor scenes, 50 urban outdoor scenes, and 50 indoor-to-outdoor scenes, and 10,000 episodes spanning Object
Navigation,
Vision-and-
Language Navigation, and Place
Navigation tasks, where
agents search for semantic points of interest such as restaurants or banks.
Agents are evaluated through executable
robot interfaces using task-success, path-efficiency, and safety metrics. Zero-shot experiments with RL, VLA, and modular baselines show that current
agents remain far from solving cross-context
navigation: end-to-end VLAs obtain the highest zero-shot success, while the modular method provides the strongest safety profile. PlaceNav further reveals a clear drop from outdoor to indoor-to-outdoor scenes, indicating that adaptation remains major bottleneck.
Read the full abstract ▾
All authors Junzhe Wu, Yue Hu, Zeyu Han, Po-Hsun Chang, Yinan Dong, Behrad Rabiei, Maani Ghaffari
Computer Vision
A Unified Variational Framework for Deep Weakly Supervised Image Segmentation
Yin King Chu, Lingfeng Li, Sung Ha Kang et al. (+2 more)
We propose a unified variational framework for image
segmentation under sparse pixel-level supervision. Our method is based on a simplex-constrained Potts model with a smooth perimeter regularizer, yielding a convex, smooth energy functional that can be used as a training loss in weakly supervised deep learning paradigms or optimized efficiently using iterative methods. Sparse labels are incorporated into the data fidelity term by constructing a fuzzy membership function via a function extension problem in a Reproducing Kernel
Hilbert Space (RKHS), which can effectively capture inhomogeneous intensity statistics. The derived discrete loss for training standard networks demonstrates robustness and consistent improvements over non-training and partial cross-
entropy (PCE) baselines in experiments, achieving comparable performance without requiring ground-truth
segmentation images.
Read the full abstract ▾
All authors Yin King Chu, Lingfeng Li, Sung Ha Kang, Jianping Zhang, Xue-Cheng Tai
Computer Vision
Domain Shift in Echocardiography: Interpretable Quantification and Prediction of Cross-Dataset Left Ventricular Segmentation
Soroush Elyasi, Nasim Dadashi Serej, Julie Wall et al. (+1 more)
Cross-
dataset generalisation remains a major barrier to clinical deployment of echocardiographic left ventricular
segmentation, yet the sources of this shift are rarely disentangled. We examined whether transfer degradation could be estimated before deployment using handcrafted ultrasound descriptors, VAE latent features, and
segmentation-derived latent features across six echocardiographic
datasets. Geometry-aware preprocessing substantially improved several poor transfer cases, suggesting that much of the apparent domain shift reflects field-of-view and framing inconsistencies rather than intrinsic acoustic differences alone. Intensity z-normalisation changed
dataset separability by less than 0.005, indicating that brightness and contrast are not the dominant shift axis. Absolute Dice drop on held-out source-target pairs was predicted with an R-squared value of 0.612, an MAE of 0.082, and a Spearman rho of 0.681. The variant without LV and fan-shaped features retained approximately 70% of this explanatory power, supporting mask-free transfer-risk monitoring. The most informative discrepancy measure depended on the representation, with CMD strongest in z-normalised handcrafted features, with an absolute r of approximately 0.86 and an R-squared value of approximately 0.70; log-Wasserstein strongest in VAE space, with an r of approximately -0.90 and an R-squared value of approximately 0.81; and log-MMD strongest in LV-
segmentation latent features, with an r of approximately -0.92 and an R-squared value of approximately 0.84. Apparent vendor effects were largely
dataset-confounded. Echocardiographic domain shift is therefore structured and measurable, and its impact on
segmentation can be partly reduced through geometry-aware preprocessing and anticipated using representation-specific transfer-risk estimation.
Read the full abstract ▾
All authors Soroush Elyasi, Nasim Dadashi Serej, Julie Wall, Massoud Zolgharni