Computer Vision · Cross-listed from cs.RO
DexStitch: Learning and Composing Dexterous Skills from Human Videos via Contact-Aware Robotic Inpainting
Byeonggyeol Choi, Seongmin Jung, Unghui Nam et al. (+5 more)
A long-horizon manipulation task consists of primitive skills and the transitions connecting them. Learning skills that can be executed in sequence requires supervision for both, which is costly to collect on a
robot. Human
videos offer a scalable source of demonstrations, but transferring them to
robots requires extracting
robot-compatible observations and actions together with their skill structure. We present DexStitch, a framework that cuts long-horizon human
videos into skills and transitions, converts them into aligned
robot observation-action-skill data, and stitches the learned skills into a requested sequence. A skill classifier first segments the
videos into manipulation skills and transitions. DexStitch then converts the human demonstrations into
robot observations and actions while retaining their corresponding skill labels. During conversion, per-finger human-object contacts guide dexterous hand retargeting and robotic inpainting, preserving contact geometry and finger-object occlusion ordering. To stitch the learned skills into a sequence, a shared skill-conditioned policy is pretrained on the resulting synthetic data and finetuned on a mixture of synthetic data and a small set of single-skill teleoperation episodes. Conditioning on the current and next skills guides each transition toward the following skill, while predicted skill progression enables closed-loop switching. On a held-out session, the classifier agrees with manual annotations on 92.4% of frames. Across four household skills, DexStitch achieves 80.0% single-skill success and 77.5% sequence success.
Read the full abstract ▾
All authors Byeonggyeol Choi, Seongmin Jung, Unghui Nam, Youngbo Song, Sunhong Min, MinHyung Lee, Byonghyo Shim, Yongseok Lee
Computer Vision · Cross-listed from cs.AI
Autonomous 3D Visual-Thermal Inspection for Utility-Scale Solar Farms
Lachlan Tychsen-Smith, Mark Cox, Brett Wood et al. (+6 more)
Utility-scale photovoltaic (PV) inspection is dominated by aerial surveys, which are effective for identifying module-level hot
cells but provide limited visibility of critical components located under the array, including connectors, cabling, fasteners and structural elements. These assets are difficult to inspect consistently at scale, and their defects are often spatially small, partially occluded and difficult to localise from isolated images. We present an automated 3D visual-thermal inspection system (Tricorder) for under-panel PV asset inspection. The approach combines
LiDAR-inertial Simultaneous Localization and Mapping (SLAM) with geometrically calibrated RGB and thermal imagery and machine-learning-based object detection and segmentation. Detected assets are associated with the reconstructed 3D environment, allowing visual and thermal observations from multiple viewpoints to be spatially fused and localised. We demonstrate the approach on utility-scale PV data for MC4 connector detection and connection-state recognition, thermal anomaly identification, and detection of structural components including bolts, nuts and torque tubes. The results demonstrate a pathway from image-based inspection toward spatially grounded autonomous inspection, where defects are detected and linked to specific assets and their 3D locations within the solar farm.
Read the full abstract ▾
All authors Lachlan Tychsen-Smith, Mark Cox, Brett Wood, David Haddon, Kenrick Anderson, Benjamin Duck, Jim O'Malley, Noel W Duffy, Peyman Moghadam
Computer Vision
AttenIR: Aligning Dense Visual Attention with Mesh Warping for Unsupervised Image Retargeting
Yun Zhang, Zhiruo Zhang, Lifei Zhu et al. (+5 more)
Image retargeting aims to resize images while preserving salient content free from distortion and stretching. Recent learning-based solutions reduce reliance on traditional hand-crafted features and optimization procedures, leading to more efficient and generalizable performance. Nevertheless, existing approaches still struggle to preserve global structure and content fidelity. To address these limitations, we propose AttenIR, an unsupervised image retargeting framework requiring no manual labels, based on content-aware mesh warping, which ef- fectively couples dense visual
attention with sparse mesh constraints. We for- mulate the mesh deformation as a vertex displacement regression task. Guided by saliency priors, our method jointly optimizes scale, structure, and geometry preservation losses. By combining dense visual
attention with semantic features, it achieves robust, unsupervised retargeting without manual annotations. Our primary contributions include (1) a lightweight unsupervised method for image retargeting that ensures content fidelity and preserves global structure, and (2) a coarse-to-fine projection strategy to align dense visual
attention priors with sparse mesh constraints, enabling content-aware deformation at the pixel level. Experiments demonstrate that our method achieves competitive and superior performance compared with both traditional and state-of-the-art learning-based methods.
Read the full abstract ▾
All authors Yun Zhang, Zhiruo Zhang, Lifei Zhu, Zhe Zhu, Tong Li, Fang-Lue Zhang, Shengjuan Liu, Xinyuan Zheng
Computer Vision
DT-LoRA: Reaching Beyond the Tangent Space of Low-Rank Adaptation
Yangming Zhang, Ling Yi, Datong Li et al. (+4 more)
Low-rank adaptation (LoRA) often underperforms full fine-tuning. We investigate this performance gap through a tangent-space lens: factorized updates are confined to the tangent space of the current low-rank factors, leaving gradient components outside this space unexploited. An analysis of ten representative LoRA variants reveals that these residual components account for most of the gradient energy and exhibit strong spectral concentration. Motivated by this finding, we propose Dynamic Tangent LoRA (DT-LoRA), which partitions a fixed rank budget into a stable branch and a dynamic branch to capture the principal residual gradient directions. We show that these directions define the optimal update of a local quadratic model under orthogonal-complement and rank constraints.
Matrix-free randomized SVD recovers them without materializing the residual gradient, while controlled write-back preserves accumulated updates and stabilizes training within the allocated rank. DT-LoRA adds no trainable parameters or inference cost over LoRA. Experiments across computer
vision and natural
language processing tasks demonstrate consistent improvements over LoRA and its variants. Moreover, despite being a general-purpose adaptation method, DT-LoRA outperforms task-specific approaches to open-vocabulary domain generalization for semantic segmentation, achieving state-of-the-art results. Code is available at https://github.com/Snipine/DT-LoRA.
Read the full abstract ▾
All authors Yangming Zhang, Ling Yi, Datong Li, Jinliang Ding, Xiaojie Wang, Tiecheng Song, Zhaolong Ning
Computer Vision
From Token Correction to Claim Admission: Training-Free Hallucination Mitigation for Large Vision-Language Models
Hyeonggeun Han, Taehyun Cho and Sangwoo Hong
Large
vision-
language models often exhibit object
hallucination, generating claims that are not supported by the input image. To mitigate this issue, we propose Evidence-Guided Visual Claim Admission (EGVCA), a training-free framework that admits a semantic claim only when it receives sufficient image-dependent support. EGVCA measures this support by comparing the claim likelihood under the observed image and a visual-null input. For captioning, it controls semantic expansion without altering
token generation. For binary visual question answering, it combines atomic visual support with the image-induced change in the model's native Yes–No margin. Experiments on CHAIR, POPE, and MME across three LVLM backbones show that EGVCA substantially reduces object
hallucination while preserving caption quality. It achieves the highest POPE F1 in eight of nine backbone–sampling settings and improves the native MME total score on all three backbones. These results demonstrate the effectiveness of claim-level admission as a simple alternative to
token-level distribution correction.
Read the full abstract ▾
Computer Vision
A Differentiable 3D Scene Graph Metric via Contextual Hellinger Triplet Geometry
U-Chae Jun, Yoonhyuk Choi and Jiwoo Kang
3D scene graph prediction is commonly supervised with local object and predicate classification losses. Although effective for slot-wise label prediction, this formulation does not provide a graph-level semantic distance between a predicted scene graph and its ground truth: semantically mild label errors and structurally disruptive relational errors can be treated similarly, and subject-predicate-object facts are not compared as scene-level units. We propose Contextual Hellinger Triplet Geometry, a graph-level supervision objective that measures the semantic and structural discrepancy between 3D scene graphs. Our key idea is to reinterpret the conventional classifier outputs of a 3D scene graph predictor as an aligned probabilistic scene graph field, where object and predicate predictions jointly define contextual subject-predicate-object facts. This formulation yields a differentiable metric-based training objective with an efficient factorized implementation and can be applied to existing 3D scene graph prediction architectures without modifying their network design. Experiments on 3DSSG demonstrate that our supervision improves downstream graph-conditioned 3D scene generation, while ablation studies confirm the complementary roles of the proposed graph-level terms and user studies show better graph descriptiveness, relation plausibility, and structural preservation.
Read the full abstract ▾
Computer Vision
SWORD: Scene-Grounded Wasserstein Object-Relation Distance for 3D Scene Graph Prediction
U-Chae Jun, Jaeeun Ko, Yoonhyuk Choi et al. (+1 more)
3D scene graph prediction is commonly optimized with object-node and relation-edge classification losses. These objectives encourage exact object and relation label recovery, but they do not directly optimize semantic proximity to the ground-truth scene graph: two wrong graphs can receive comparable penalties even when one is much closer in object roles and relation contexts. A direct graph-distance loss is difficult in general, since graph matching and graph edit distance require correspondence search and edit-cost design. Our key observation is that prior supervised 3D scene graph prediction protocols are conducted on fixed object-node indices and supervised edge slots. This can turn a scene graph comparison into a well-defined aligned graph-distance problem. To this end, we propose SWORD, a Scene-Grounded Wasserstein Object-Relation Distance that builds block-normalized object and relation ground distance metrics from training scene graph annotations through a deterministic base-to-context order, without external semantic priors. SWORD lifts these metrics to Wasserstein distances over aligned node and null-augmented edge marginals, providing semantic graph-distance supervision beyond exact-label penalties while preserving the ground-truth graph as the unique zero-distance solution in aligned marginal probability space. We prove that SWORD is a strict metric on aligned marginal probabilistic scene graphs, preserves scene-graph-grounded substitution ordering, controls additive graph-grounded semantic observables, and yields a surrogate upper-bounding decoded semantic error.
Read the full abstract ▾
All authors U-Chae Jun, Jaeeun Ko, Yoonhyuk Choi, Jiwoo Kang
Computer Vision · Cross-listed from cs.AI
MedEvo: Medical Reasoning Evolution through Knowledge Navigation
Xiaoliang Wang, Zhuo Chen, Chuanyang Zheng et al. (+10 more)
Medical
multimodal reasoning models are typically trained on teacher-generated rationales, discarding the valid
reasoning in students' own failed trajectories, while general-domain correction methods offer little guidance for errors in image reading, anatomical context, or differential diagnosis. We propose MedEvo, which uses medical knowledge, namely a tag taxonomy and error-type diagnosis, to navigate both trajectory repair and data selection. For each failed rollout, a gold-blind auditor locates the first consequential error and provides a minimal, answer-free hint; the same student retains the valid prefix and regenerates the erroneous suffix. After each round, MedEvo identifies capability gaps across medical tags on a held-out feedback set and prioritizes related questions for new rollouts and repairs. With supervised fine-tuning (SFT) alone, MedEvo raises Qwen3.5-9B mean accuracy across seven
benchmarks from 65.57% to 69.12% (47.40% to 54.50% on the
multimodal track of MedXpertQA (MedXpertQA-MM)), surpassing teacher-
distillation SFT (66.93%). It also raises Qwen3.5-35B and Qwen3.5-397B mean accuracy from 67.91% to 73.04% and from 71.83% to 74.95%, respectively. Our analyses show that failed trajectories contain
reasoning worth preserving, that repair success does not imply learning value, and that independent feedback is key to sustained improvement.
Read the full abstract ▾
All authors Xiaoliang Wang, Zhuo Chen, Chuanyang Zheng, Quanbin Wang, Jiangwei Lao, Yingao Liu, Jiajia Liu, Jinming Cao, Honglin Lin, Zhihong Lu, Hao Jin, Jian Wang, Lijun Wu
Computer Vision
Hierarchical Curriculum Learning for Surgical Video Generation: Focusing on Bilateral Axillo-Breast Approach Robotic Thyroidectomy Datasets
Youngbin Kong, Seungho Lee, Jae Kyung Shin et al. (+3 more)
High-fidelity surgical
video generation is increasingly important for surgical education, simulation, and downstream AI training, as conventional simulation pipelines are limited by cost, realism, and scenario diversity. Adapting generic text-to-
video diffusion models to surgical procedures remains difficult, however, due to complex anatomical structures, intricate tool-tissue interactions, the inherently hierarchical nature of surgical workflows, and the scarcity of curated training data outside a few well-studied procedures. We propose a surgical
video generation framework that combines hierarchical procedural annotation with curriculum transfer learning. Surgical procedures are decomposed into three levels (phase, task, and action) and a pretrained text-to-
video diffusion model is progressively adapted from coarse to fine-grained surgical context, supported by a dynamic sampling strategy that handles variable-resolution and variable-length surgical clips. We instantiate the framework on Bilateral Axillo-Breast Approach robotic thyroidectomy, a procedure for which no curated
benchmark currently exists. Relative to the preceding task-level model, the final action-level model reduces FVD by 20.1% and FID by 15.7%, and a matched-budget ablation indicates that these gains arise from the curriculum design rather than from additional training. A user study with six clinicians rates visual realism highly while highlighting remaining gaps in anatomical accuracy and clinical reliability. The proposed framework offers a step toward AI-driven surgical simulation in procedures where curated data are scarce.
Read the full abstract ▾
All authors Youngbin Kong, Seungho Lee, Jae Kyung Shin, Jong-hyuk Ahn, Hyoun-Joong Kong, Kyu Eun Lee
Computer Vision · Cross-listed from eess.IV
Deficit-Driven Stratified Acquisition for Building STRATA-S12: An Open SAR-Optical Corpus for Self-Supervised Learning
Christian Sanchez-Leon, Dongming Chen and Dongqi Wang
Building a co-registered SAR-optical corpus for self-supervised Earth Observation is above all an acquisition problem, yet the main multi-modal
datasets are collected opportunistically, with no per-stratum quota control and no mechanism against the geographic dropout that non-uniform cloud cover leaves behind. We present Deficit-Driven Stratified Acquisition (DDSA), an
algorithm that treats building a co-registered Sentinel-1/Sentinel-2 corpus as a constrained maximum-coverage problem over a hemisphere-corrected (class x season x region) stratification, enforces strict temporal pairing and a multi-stage quality gate, and closes the loop during acquisition with a human-in-the-loop deficit-repair stage. Measured on its own product, the
algorithm yields three results a single-pass acquisition cannot report. Deficit-driven ordering makes a partial run a covering one: it holds every one of the 192 strata populated from the first few percent of the budget, where an abundance-first order leaves most strata empty until the end. Whether a failed attempt can be recovered depends on the temporal persistence of its cause, not the kind of defect:
geometric failures such as nodata are among the most repairable (eta = 41.4%) while cloud is among the least (eta = 18.8%), because a cloudy location is cloudy on most dates of its season. The repair loop supplies 39.5% of the released pairs at no measurable quality cost, with 89.8% surviving manual review. The quality guarantee has a price: 2,928,370 acquisition attempts produced 740,746 released pairs. We release the corpus, STRATA-S12, with STAC/PDS4-aligned FAIR sidecars, and a pre-registered linear-probe check confirms it is fit for self-supervised pre-training. A metadata-level coverage comparison against SSL4EO-S12 and SEN12MS situates the corpus over a shared frame; systematic cross-corpus and cross-method comparison is left to follow-up work.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
Integrating 3D Posture Neutralization with Statistical Shape Modeling for Head-Neck-Shoulder Ergonomic Design
Jinbo Luo, Hang Ren, Yadie Yang et al. (+1 more)
The design of ergonomic products and workspaces requires a deep understanding of human anatomy and posture. In particular, the head-neck-shoulder region is critical for many applications such as cervical orthoses, helmets, and other head-worn or neck-worn devices. This paper presents a framework that integrates posture neutralization with statistical shape modeling (SSM) for the head-neck-shoulder (HNS) region. We acquire a single three-dimensional (3D) scan of the head-neck-shoulder region from each of 200 Chinese adults in a natural posture for each subject. Every scan is then posture-neutralized before SSM. We fit a parametric head-neck model to each scan to separate cervical pose from anatomy, and reset the cervical pose to a common neutral configuration, so that the resulting model captures anatomical shape variation rather than posture-induced deformation. The resulting shape space yields a standardized template and population-wide modes of variation, providing baselines for parametric ergonomic design. We further discuss how the model supports population coverage analysis and the design of products that accommodate anatomical diversity while maintaining neutral cervical posture.
Read the full abstract ▾
All authors Jinbo Luo, Hang Ren, Yadie Yang, Jie Zhang
Computer Vision
ScenePercept: Scenario-Conditioned Perceptual Assessment of Real-World User-Generated Content Images
Shuhan Xia, Yu Chen, Xiang Wang et al. (+3 more)
The rapid development of mobile imaging and social platforms has led to explosive growth in user-generated content images, making their systematic and comprehensive perceptual assessment increasingly important. Existing image perception
benchmarks are typically task-oriented, curating specific scenarios for quality or aesthetic assessment and applying a uniform perceptual criterion across diverse scenarios. This limits their ability to capture the scene diversity and perceptual complexity of real-world UGC images. To address these limitations, we introduce ScenePercept-Bench, a
benchmark designed to reflect the distribution of real-world UGC images, comprising 23,100 images across 21 categories and 105 fine-grained scenes. To account for the influence of scene context on human perception, we design a hierarchical, scene-aware annotation protocol, covering visual quality, aesthetic perception, and scene semantics, with 15 attributes in total. Each image is annotated with an overall score, 15 attribute-level scores, and structured explanatory
reasoning. Building on this
benchmark, we further introduce Scenario-Conditioned Perceptual
Alignment, a training framework that tackles the heterogeneity of perceptual criteria across scenarios. SCPA jointly optimizes absolute score prediction, preference ranking, and cross-scenario calibration, preserving the model general perceptual assessment ability while encouraging it to explore and adapt perceptual criteria across different scenarios. Extensive experiments demonstrate that SCPA achieves state-of-the-art performance in UGC image perception and generalizes effectively to established IQA and IAA
benchmarks.
Read the full abstract ▾
All authors Shuhan Xia, Yu Chen, Xiang Wang, Shuai Li, Peipie Li, Zekun Li
Computer Vision
Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence
Jiahao Meng, Zhongyuan Bao, Yue Tan et al. (+7 more)
Reliable
video reasoning requires models to produce correct answers and expose supporting visual evidence. Open-o3-
Video enables verifiable
reasoning by
embedding timestamps and object-level bounding boxes into
reasoning traces, but is restricted to frames from an initial sparse sampler. We introduce Open-o3-
Video-v2, which formulates
video reasoning as a unified agentic evidencegrounded process. The model identifies missing information, acquires additional observations, grounds the moments and objects supporting its
reasoning, and writes this evidence into the trajectory before continuing or answering. To support this behavior, we construct STGR-v2-CoT135K, including 52K multi-turn agentic grounded
reasoning trajectories, and STGR-v2-RL-13K for reinforcement learning. We further propose Clue-Seek GRPO (CS-GRPO), which tracks how tool calls change accumulated evidence, rewards clue coverage and step-wise gains, and penalizes excessive, repetitive, or uninformative
retrieval. On V-
STaR, Open-o3-
Video-v2-4B achieves 40.4% mAM and 56.3% mLGM, outperforming its Qwen3-VL-4B backbone by 8.5% and 12.6%, respectively. It also delivers consistent gains across general
video understanding, long-
video reasoning, and temporal grounding
benchmarks. These results show that active evidence acquisition and verifiable object-level grounding are complementary for reliable
video reasoning. The code and models are released at https://msalab-pku.github.io/projects/Open-o3-
Video-v2.
Read the full abstract ▾
All authors Jiahao Meng, Zhongyuan Bao, Yue Tan, Haiquan Wen, Renrui Zhang, Anna Wang, Jason Li, Yanwei Fu, Yunhai Tong, Ming-Hsuan Yang
Computer Vision · Cross-listed from cs.RO
Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation
Haoyu Zhang, Siming Li, Zeyu Zhang et al. (+3 more)
Recent coding large
language models (LLMs) can synthesize dynamic 4D worlds from natural
language, but the resulting content remains a visual artifact. It lacks physical semantics such as the collision geometry, mass, friction, inertia, and joint structure, required to enable it to simulate the physical world, and its dynamics are scripted to look plausible rather than designed to execute. What is needed is an executable world: one that a physics engine can load and step, resolving contact physically. We present Code2Worlds-2, an extension of Code2Worlds that turns 4D world generation into an embodied world simulation. We keep the original dual-stream generation pipeline intact and append two modules. First, a SimReady Scene Compiler that turns a generated visual scene into a simulator-compatible representation: it assigns collision meshes via convex decomposition, estimates physical attributes with a
vision-
language model, infers a lightweight support graph, instantiates the engine's contact, friction, and dynamics constraints, and applies a
gravity settling pass, before exporting the scene as MuJoCo MJCF. Second, a Simulator-in-the-Loop 4D Refiner that implements an automated
agent harness-a compile, simulate, critique and refine loop-that converts visual dynamics into executable rigid and articulated dynamics and evolves the physical settings rather than only the code, replacing the purely visual motion critic with a simulator critic that inspects penetration depth, collision pairs, stability, contact forces, joint limits, and
robot reachability. We further propose SimCode4D, a state-based
benchmark with five task templates for evaluating embodied simulation. Experiments on SimCode4D and across natural, indoor, industrial, urban, and service scenes show that Code2Worlds-2 produces physically plausible, simulator-compatible, and
robot-interactable scenes.
Read the full abstract ▾
All authors Haoyu Zhang, Siming Li, Zeyu Zhang, Yi Zhang, Tat-Seng Chua, Hao Tang
Computer Vision
MemAct: Streaming Spatial Understanding with Memory and Active Exploration
Yuxi Wei, Wei Huang, Qirui Chen et al. (+2 more)
Spatial understanding is essential for embodied
agents navigating and interacting with the physical world. As observations arrive incrementally,
agents must reason in a streaming manner. Yet streaming understanding must also be closed-loop: task-relevant evidence may require both recalling past observations and actively acquiring new views. Existing streaming approaches largely emphasize processing incoming observations, leaving task-driven active exploration underexplored. We introduce MemAct, a framework for streaming spatial understanding with
memory and action. A shared
vision–
language model incrementally writes structured historical
memory, retrieves records and past frames on demand, and acquires additional evidence through movement and turning. Retrieved evidence and action observations inform subsequent decisions, enabling the
agent to continue exploring or answer within a unified
reasoning loop. We develop the MemAct suite, a data generation pipeline and collection across real and simulated environments, with approximately 1M training examples for spatial
reasoning,
memory writing,
retrieval, and exploration. Its MemAct-bench comprises 20,000 main QA pairs evaluating current-observation understanding, historical evidence use, and active exploration, plus 2,000 separate Timing questions. MemAct combines supervised fine-tuning for
memory and interaction with counterfactual action optimization using cumulative evidence gains, action costs, and terminal answer quality to improve exploration and stopping decisions. Experiments show that MemAct outperforms existing spatial understanding and streaming baselines and approaches frontier
multimodal models on MemAct-bench while retaining streaming inference efficiency.
Read the full abstract ▾
All authors Yuxi Wei, Wei Huang, Qirui Chen, Lu Hou, Xiaojuan Qi
Computer Vision
FrameLite: Decision-Oriented Aesthetic Cropping with a Compact Vision-Language Model
Jianze Li, Kai Liu, Ziqing Zhang et al. (+4 more)
Aesthetic image cropping improves composition by balancing subjects, context, and image boundaries. Large
vision-
language models bring semantic knowledge to this task, but their scale limits lightweight photography assistance. We propose FrameLite, a compact cropping model built on Qwen3.5-0.8B. Our approach is motivated by the observation that a small model can generate better crops than its first answer suggests. We develop three-stage training to turn this potential into stronger direct predictions. Spatial and composition training establishes localization and framing preferences. Quality-Guided Online Supervision (QOS) then learns the highest-quality complete answers among the current policy's generated candidates. Greedy-Referenced Optimization (GRO) further refines the policy using positive and negative quality differences relative to its current greedy answer. Both online stages use annotation-derived feedback, while inference returns one coordinate answer without search or an external scorer. Experiments show that FrameLite outperforms the compared large-VLM croppers across public
benchmarks with a substantially smaller backbone. Controlled studies confirm that selecting high-quality training answers improves the final model beyond online sampling alone. We will release our code and model weights at https://github.com/JianzeLi-114/FrameLite .
Read the full abstract ▾
All authors Jianze Li, Kai Liu, Ziqing Zhang, Weihang Zhang, Yong Guo, Linghe Kong, Yulun Zhang
Computer Vision · Cross-listed from cs.LG
Calibrated Disambiguation for Partial Multi-label Learning
Zhuoming Li, Yuheng Jia, Mi Yu et al. (+1 more)
Partial multi-label learning (PML) aims to train a classifier on
datasets whose instances are over-annotated with not only relevant labels but also irrelevant labels, which is common when
datasets are collected from crowd-sourcing platform. Existing works primarily approach it from a curriculum learning perspective, leveraging the memorization effect to disambiguate noisy labels and produce robust predictions. However, these methods are based on non-adaptive weighting functions and lack theoretical guidance for optimal weighting. To overcome these issues, a calibrated disambiguation model named PML-CD is proposed. We first formulate the optimal weighting function for curriculum-based disambiguation, which is equivalent to the calibration of the model's predicted confidences, thus providing guidance for curriculum designing. To obtain the optimal weighting function from PML
dataset during the training, a transferable calibrator is designed, which takes the histogram of positive samples' confidences as input, and outputs the optimal curriculum weighting for training. Prototype
alignment regularization is also proposed to promote the model's performance. Experiments conducted on Pascal VOC, MS-COCO, NUS-WIDE and CUB have verified that our method outperforms existing state-of-the-art PML methods.
Read the full abstract ▾
All authors Zhuoming Li, Yuheng Jia, Mi Yu, Zicong Miao
Computer Vision · Cross-listed from cs.RO
From Pixels to Executable Physics: Behavior-Grounded World Models for Embodied Action Consequence Prediction
Ruo Chen, Jiahua Pang, Yingjie Niu et al. (+4 more)
In outdoor environments, varying terrain and contact conditions can cause identical action commands to produce different motion outcomes for quadruped
robots. Predicting these outcomes is essential for reliable navigation and informed path selection. Existing
video world models can generate visually plausible futures, yet often struggle to capture
robot dynamics and terrain interactions, leading to overly optimistic predictions of action outcomes. We present DR-WM, a physically consistent world model that incorporates
robot-terrain contact dynamics to predict the consequences of quadruped actions. Starting from an initial RGB image, our method reconstructs scene geometry and integrates dense, dimensionless priors over terrain contact properties, calibrated using real-world action-response data. The resulting terrain model supports the simulation of physics with spatially varying contact properties such as friction. Given the initial state of the
robot and candidate action sequences, we simulate its future motion and use the resulting camera trajectories and rendered geometry to guide the training of a
video world model. The generated future observations are then used to update the terrain reconstruction, enabling iterative model rollouts without additional real-world images. Extensive experiments demonstrate that DR-WM substantially outperforms geometry-only baselines and baselines with uniform physical parameters on key physical metrics characterizing action outcomes, while achieving the best performance among the evaluated baselines. These physically grounded predictions support more informed action selection: across 90 challenging scenarios, an 8B
vision-
language model augmented with DR-WM outperforms an unassisted 32B
vision-
language model on five of six evaluation metrics.
Read the full abstract ▾
All authors Ruo Chen, Jiahua Pang, Yingjie Niu, Hongyi Lin, Jun Tian, Yang Zhu, Chao Li
Computer Vision
4LC: 4D Latent Cache for Efficient Conditioning in Dynamic World Modeling
Jianhui Zhang, Zuyao Lin, Pengxin Xu et al. (+2 more)
For
video generation in dynamic world modeling, observed static and dynamic content should be efficiently reused to provide conditions for repeated generation at different times and viewpoints. We introduce 4LC, a 4D latent cache with composite visual representations and a flexible spatiotemporal access interface. As a latent cache, 4LC directly reuses visual representations of static and dynamic content to provide generation conditions, avoiding per-request RGB rendering and re-encoding. As a composite cache, 4LC binds visual representations from multiple visual encoders (e.g., VAE, DINO, V-JEPA, VGGT) to shared geometry and motion, allowing the composite representations to describe the dynamic world with rich information. Besides, we design a flexible spatiotemporal interface, which directly reads the cache at independently specified times and viewpoints and combines readouts across observations, providing a common conditioning format for diverse downstream tasks. Building on 4LC, we develop 4LC-
Video to address multiple
video generation tasks through different configurations of source observations, spatiotemporal requests, and historical reuse, including camera control, temporal control, novel-view synthesis, scene revisits, and long-term generation. Our experiments show more efficient condition acquisition, effective spatiotemporal controllability, and scene consistency across tasks using a single 4LC-
Video model without task-specific fine-tuning.
Read the full abstract ▾
All authors Jianhui Zhang, Zuyao Lin, Pengxin Xu, Jingdong Wang, Xingyu Chen
Computer Vision
Smartphone-Based, Field-Deployed AI Inspection System for Power Distribution Infrastructure
Sampath Balaji Masabathula, Srija Vanapalli, Gangadhar Kanchi et al. (+3 more)
Routine inspection of power distribution infrastructure remains challenging for utilities operating geographically dispersed networks, where conventional manual inspection can be difficult to perform consistently and at scale. This work presents a smartphone-based AI inspection system for assessing targeted distribution-infrastructure conditions from images captured by field personnel. The proposed workflow integrates image-quality and viewing-perspective validation with AI-assisted assessment of pole lean, crossarm and top-cleat tilt, voltage level, vegetation-clearance risk, and distribution-
transformer (DTR)-ground clearance. The system integrates deep-learning models with monocular depth estimation,
geometric analysis, and rule-based processing within a common multi-stage inspection workflow designed for practical field operation. The system was pilot deployed across five selected distribution feeders within the Eastern Power Distribution Company of Andhra Pradesh Ltd. (APEPDCL), where 1,222 defect instances were identified across the assessed categories and subsequently tracked through a rectification and verification workflow, with 84 defect instances recorded as rectified during the study period. Feeder-level interruption records were also examined over the study period; however, the available observations do not provide sufficient evidence to establish a reliability improvement attributable to the proposed system. A curated multi-module field-captured
dataset, PD-Defect, is also publicly released to support reproducible research in distribution-infrastructure inspection. The field deployment demonstrates the practical integration of smartphone-based AI-assisted inspection into field inspection and maintenance activities, providing a mechanism for systematic identification, documentation, rectification tracking, and verification of targeted distribution-infrastructure defects.
Read the full abstract ▾
All authors Sampath Balaji Masabathula, Srija Vanapalli, Gangadhar Kanchi, Sreenivasa Rao V.A.N, Vijaya Lalitha V, Sasank Chilamkurthy
Computer Vision · Cross-listed from eess.IV
Background2Background: a self-supervised background removal framework for high dimensional microscopy
Guangrui Ding, Haonan Lin, Rong Tang et al. (+8 more)
High-dimensional microscopy captures features across space, time, and spectrum, but complex backgrounds obscure signals. We present Background2Background (B2B), a self-supervised background removal framework leveraging the backgrounds larger correlation relative to the signal. Local pixel shuffling disrupts signal correlations while preserving background structure, transforming signals into effective noise for Noise2Noise-based background removal. Across fluorescence, stimulated Raman scattering and mid-infrared photothermal imaging, B2B achieves over 40-fold background suppression and reveals hidden signatures.
Read the full abstract ▾
All authors Guangrui Ding, Haonan Lin, Rong Tang, Bethany Weinberg, Jianpeng Ao, Qing Xia, Guorong Hu, Dingcheng Sun, Jiaze Yin, Lei Tian, Ji-Xin Cheng
Computer Vision
PLEDGE: Progressive Lossy Error-Bounded Compression of 3D Gaussian Splatting
Mingze Xia, Yuxiao Li, Baixi Sun et al. (+6 more)
Existing 3D Gaussian Splatting (3DGS) codecs report reconstruction quality after compression, and give users little control over the distortion of individual primitives. We present PLEDGE, a training-free, progressive, error-bounded codec that translates four user-declared tolerances into per-primitive quantization bounds: screen-space position, footprint deformation due to rotation and scale, view-dependent color, and opacity. The bounds apply throughout a declared camera domain, including previously unseen viewpoints. Position and opacity are bounded directly; footprint deformation is bounded under the rasterizer's affine projection model, and color is bounded over all viewing directions. PLEDGE organizes quantized residuals into a progressive bitstream with explicit error bounds for every valid prefix. Experiments across thirteen scenes verify all declared per-mechanism bounds and demonstrate competitive rate-distortion performance. The four tolerances incur substantially different rate costs, so users can spend rate where their application needs it.
Read the full abstract ▾
All authors Mingze Xia, Yuxiao Li, Baixi Sun, Wenbo Li, Pu Jiao, Sheng Di, Franck Cappello, Hanqi Guo, Xin Liang
Computer Vision
Morphology-aware deep learning predicts colorectal cancer survival from whole-slide images
Hikmat Khan, Metin N. Gurcan, Anil Parwani et al. (+3 more)
Background: Five-year survival prediction from hematoxylin-and-eosin whole-slide images could support risk stratification, but the task is weakly supervised, and the tissue morphologies carrying prognostic information remain incompletely characterized. Methods: We conducted a retrospective secondary analysis using TCGA-COAD/READ (n=562) for morphology-specific analysis, Alliance/ CALGB 89803 (n=424) for model development, and SurGen (n=218) for external evaluation. We propose MorphContrast, a morphology-aware contrastive multiple-instance learning framework that uses tissue labels as structural supervision for five-year survival prediction. MorphContrast combines gated-
attention pooling with three auxiliary objectives: patch-level morphology labels define within-patient contrasts, outcome concordance or discordance defines cross-patient contrasts within matched morphologies, and diversity regularization discourages representational collapse. Morphology-matched patches from patients with discordant outcomes serve as hard negatives, encouraging prognostic discrimination within comparable histologic contexts. Results: Morphology-specific models show weak but distributed prognostic discrimination: adipose tissue achieves an area under the receiver operating characteristic curve (AUC) of 0.622, stroma achieves 0.604, and both perform comparably to tumor epithelium (0.601-0.605). On Alliance/CALGB 89803, the single model achieves an AUC of 0.642, while the post hoc ensemble reaches a pooled out-of-fold AUC of 0.70 and a risk-group hazard ratio of 3.79 (95% confidence interval: 2.55-5.63). On the external SurGen colon cancer cohort (n=218), the single model and ensemble achieve AUCs of 0.669 and 0.678, respectively. Conclusions: These findings support tissue morphology as structural supervision for outcome-aware representation learning in colorectal cancer prognosis.
Read the full abstract ▾
All authors Hikmat Khan, Metin N. Gurcan, Anil Parwani, Wendy L. Frankel, Wei Chen, Muhammad Khalid Khan Niazi
Computer Vision
Tanawpamana: Benchmarking Vision-Language Models on Philippine Cultural Heritage Sites
Ivan Yuri De Leon, Hiroki Ouchi and Sakriani Sakti
Vision-
language models are increasingly used to identify and describe places, yet their recognition and knowledge of specific cultures remain poorly understood, particularly in underrepresented regions such as the Philippines. This study introduces the Tanawpamana
Dataset, a repository of Philippine heritage site images with metadata based on the Talapamana registry of the National Commission for Culture and the Arts of the Philippines. We also introduce the Tanawpamana
Benchmark, comprising 4,046 tasks designed to evaluate models' recognition and knowledge of heritage sites. Evaluating seven
vision-
language models, we find that they struggle more with fine-grained visual discrimination among similar heritage sites than with coarse recognition, and that their geographic knowledge is weaker when inferred from image inputs than from text.
Read the full abstract ▾
Computer Vision
Physical-Field-Bridged Material Parameter Inference from Multi-View Deformation Videos
Linrui Shou, Jie Li, Yu Liu et al. (+1 more)
Estimating simulation-ready material parameters from visual observations is crucial for deformable-object manipulation but challenging because material properties are only indirectly revealed through physical responses. We introduce physical-field-bridged material inference, a single-pass framework that uses dense physical responses to connect multi-view RGB deformation
videos with constitutive-type classification and continuous parameter regression. Unlike inverse-physics methods requiring per-scene reconstruction, online tuning, or simulation-in-the-loop optimization, our model predicts force/contact masks, projected motion-flow fields, and projected stress-response maps, then conditions material inference on these fields to reduce reliance on appearance or category shortcuts. We construct DeformFieldBench, a 3DGS MPM learning environment comprising 5,000 simulations across 100 object shapes and five mechanically informative loading modes, with dense pixel-aligned physical supervision. Our method achieves the best median errors on all four continuous parameters, improves stress-hotspot recall from 0.0762 to 0.5121, and attains 0.8676 SSIM/36.82 dB PSNR in same-simulator replay on unseen objects. Under a protocol-matched GSO evaluation, inference and future-state prediction take approximately 30 seconds, compared with 15-120 minutes for per-scene optimization. The method also yields measurable replay agreement on five real-world sequences. These results support dense physical fields as an effective bridge for fast material inference, while cross-simulator and real-world tests reveal remaining domain and identifiability gaps.
Read the full abstract ▾
All authors Linrui Shou, Jie Li, Yu Liu, Tao Huang
Computer Vision
Benchmarking Self-Supervised 3D Medical Vision Foundation Models and Transferability Estimation
Jiaqi Tang, Yu Wang, Yang Liu et al. (+7 more)
Self-supervised pretraining has expanded the pool of 3D medical foundation models, but published evaluations use different
datasets, endpoints and fine-tuning protocols. Models also differ in architecture, pretraining objective and data, while downstream targets range from voxel-level masks to patient-level diagnoses and outcomes. As a result, published findings provide little directly comparable evidence for choosing a model for a new task. Direct comparison of adaptation performance requires fine-tuning candidates on labelled target data, at substantial computational cost. We
benchmark 23 candidates across the same 14 segmentation, classification and regression tasks spanning multiple anatomies and modalities. No candidate ranked among the top three across all segmentation tasks or across all classification and regression tasks; a convolutional encoder could outperform a
transformer trained with the same objective on one task but underperform it on another. We
benchmark transferability
estimators against the fine-tuned rankings and find that their agreement varies across dense segmentation, classification and regression. InsightRank uses task-appropriate frozen-feature statistics and architecture-aware pairwise comparisons to rank candidates before fine-tuning. Across the 23-model pool, mean Kendall tau-b was 0.431 for segmentation and 0.352 for classification and regression. The segmentation Top 3 contained a task winner on four of nine tasks; the image-level Top 3 contained none on the five tasks, despite better full-ranking agreement than the matched conventional
estimators. A three-model shortlist reduced recorded extraction-plus-training job time by 85.8% across seven segmentation tasks. Together, the
benchmark provides a common reference for model selection and InsightRank offers a lower-cost way to prioritize fine-tuning, with shortlist performance that differs by target type.
Read the full abstract ▾
All authors Jiaqi Tang, Yu Wang, Yang Liu, Jing Tao, Fandong Zhang, Zhou Zhen, Xiahai Zhuang, Qingchao Chen, Yining Yang, Yizhou Yu
Computer Vision
FlashVSR+: Efficient Video Super-Resolution via Hybrid Attention and Reward-Yield Distribution Matching
Ruchang Yao, Shi Guo, Junhao Zhuang et al. (+7 more)
Recent
video super-resolution (VSR) models achieve efficient one-step generation through
diffusion distillation. However, high-resolution inference remains costly, and distilled models are difficult to align with human preferences. These challenges call for improvements in both model architecture and training strategy. We present FlashVSR+ to address both limitations. On the architecture side, we introduce hybrid
attention based on the strong locality observed in pretrained VSR models, while retaining sparse global interaction for long-range consistency. It achieves up to $2\times$ DiT speedup and 45% lower peak
memory, enabling spatially untiled 4K VSR on a single 24GB RTX 4090. On the training side, we propose Reward-Yield Distribution Matching
Distillation (Ry-DMD) for preference-aligned one-step generation. Ry-DMD uses reward-weighted teacher rollouts to incorporate black-box rewards into distribution matching, without backpropagating through the reward model. We further show that Ry-DMD generalizes beyond VSR to T2I generation. Together, FlashVSR+ delivers stronger restoration quality and substantially improves high-resolution efficiency, with consistent gains in user preference. We release our code, models, and training data at \href{https://www.flashvsr.tech}{\textcolor{FlashBlue}{www.flashvsr.tech}}.
Read the full abstract ▾
All authors Ruchang Yao, Shi Guo, Junhao Zhuang, Mingde Yao, Xin Cai, Jiahao Zhan, Bin Fu, Xinyuan Chen, Yu Qiao, Tianfan Xue
Computer Vision
CrossPortrait: Cross-Species Portrait Animation via Adversarial Motion Alignment
Gezhong Pan, Zhe Li, Yunfan Tao et al. (+7 more)
Cross-species portrait animation, which aims to animate a source portrait of species A using a driving
video of species B, is particularly challenging due to substantial anatomical differences in facial structures across species. Previous works typically fine-tune human portrait animation models on animal
video datasets but often suffer from driving-species leakage. To address this, we propose CrossPortrait, a novel framework for expressive cross-species portrait animation, supporting diverse scenarios including human-to-animal, animal-to-human and cross-species animal-to-animal (e.g. cat-to-dog) animation. Specifically, we decompose the portrait motion into rigid head poses and expression keypoints. To accommodate species-specific facial structures while facilitating cross-species expression
alignment, we introduce a species-routed expression encoder that incorporates species-specific LoRAs into a unified expression encoder. Furthermore, we introduce expression-level
adversarial domain adaptation (Ex-ADA) to suppress driving-species-specific information in the expression latents, thereby mitigating species leakage and promoting a shared expression space across species. Extensive experiments show that our method enables expressive and identity-preserving portrait animation across diverse species and outperforms other state-of-the-art approaches.
Read the full abstract ▾
All authors Gezhong Pan, Zhe Li, Yunfan Tao, Yongheng Zhao, Shiqi Zhou, Juyong Zhang, Xiaosong Jia, Yanyong Zhang, Shuquan Ye, Shaofeng Zhang
Computer Vision
Kai Liu, Anqi Li, Junxian Li et al. (+4 more)
We propose a unified spatial-density signal to score
token importance for both inference phases: bipartite merge for prefill and KV eviction for decode, yielding measurable gains in each stage. To isolate the two phases, we split
benchmarks by scoring logic. Prefill probes (MMBench, POPE, ScienceQA) evaluate only the first decoded
token; their accuracy reflects prefill logits argmax before any decode KV eviction. Full-pipeline probes (TextVQA, DocVQA, MM-Vet) assess the complete generated sequence and capture both phases. We identify a critical evaluation pitfall: decode
pruning claims validated solely on prefill probes measure prefill quality, not decode performance. Experiments show discarding all visual KV during decode barely affects prefill-probe accuracy but degrades TextVQA and MM-Vet drastically; this insensitivity stems from the scoring rule, not compression intensity. Evaluated on seven model variants (InternVL-3.5, Qwen3-VL, LLaVA-1.5) and three full-pipeline
benchmarks, our method achieves a three-way Pareto frontier across compute,
memory and accuracy. Density-based prefill compression maintains strong accuracy, scales linearly with visual
token count, and delivers 100x faster
token selection at high resolution. For decode, density top-k KV eviction consistently outperforms strong baselines H2O, SnapKV and StreamingLLM on TextVQA and MM-Vet across three retention ratios. At 0.125 keep ratio on TextVQA, it surpasses H2O by 9.6 points; InternVL-3.5 models (2B to 38B) maintain a 7 to 10 point advantage. Compression scales favorably with model size: every measured 8B to 38B model transition reduces accuracy loss instead of worsening it. We suggest MM-Vet or DocVQA as mandatory full-pipeline validation for any decode-phase claims. Our code is available at https://github.com/Kai-Liu001/TwinPrune
Read the full abstract ▾
All authors Kai Liu, Anqi Li, Junxian Li, Zhixin Wang, Zhikai Chen, Renjing Pei, Yulun Zhang
Computer Vision
Gaussian Splatting for Real-Time Scene Reconstruction: Methods, Systems, and Trade-offs
Ashim Dahal, Rabab Abdelfattah and Nick Rahimi
3D Gaussian Splatting (3DGS) allows for high-quality real-time novel-view synthesis. Since its inception in 2023, its explicit and highly configurable scene representation has been extended toward sparse-view generation, dynamic reconstruction, efficiency and compression, streaming, and simultaneous localization and mapping (SLAM). Although Gaussian Splatting can provide real-time rendering, rendering an existing representation is distinct from constructing, updating, or maintaining the scene itself. In this survey, we synthesize current literature, categorize them and provide a connection of current approaches and tools towards an end-to-end pipeline. Explicitly, we categorize GS scene reconstruction into five overlapping families: single-shot generation, dynamic representations, efficiency and compression, streaming and immersive systems, and Gaussian Splatting SLAM. We distinguish render-only throughput from task-specific scene generation, update, tracking, mapping, and optimization costs, and combine controlled sparse-view benchmarking with native evaluations to identify cross-family quality, latency,
memory, and update-cost trade-offs.
Read the full abstract ▾