Computer Vision · Cross-listed from cs.CR
LoRango: It Takes Two LoRAs to Unlock Hidden Behaviors in Diffusion Models
Jin Wei, Rundong Li, Ruihao Yang et al. (+10 more)
Users commonly combine multiple Low-Rank Adaptation (LoRA) adapters to personalize images with different subjects, styles, and visual attributes. Yet inspecting adapters individually does not establish the safety of their composition. We identify and characterize a pair-conditioned attack in text-to-image
diffusion: individually useful and benign-appearing adapters redirect image generation when co-loaded with a specifically matched partner, whose identity serves as the trigger. We introduce LoRango to realize this attack through complementary Signature and Payload adapters. The Signature writes a pair-specific code into intermediate carrier representations, while the Payload uses code-selective responses and opposing signal/reference branches. These branches approximately cancel for standalone adapters and mismatched pairs; matched code-reader
alignment breaks cancellation within native GEGLU blocks and releases the programmed action. Both adapters are exported as ordinary static LoRA files compatible with standard loaders, requiring no
prompt trigger or base-pipeline modification. LoRango achieves matched-pair attack success rates of 97.9% on SD v1.5 and 98.7% on SDXL, compared with 2.8–4.6% when implanted adapters are loaded individually. Further experiments evaluate pair selectivity, standalone fidelity, robustness to deployment variations, and applicability across denoiser architectures. These findings show that individual-adapter inspection is insufficient to assess the security of multi-LoRA personalization and motivate
auditing adapter compositions.
Read the full abstract ▾
All authors Jin Wei, Rundong Li, Ruihao Yang, Yikai Wang, Xiaoyuan Duan, Jianxiong Wu, Yanbo Wang, Chang Xu, Lingyun Zhang, Zhuyang Yu, Ping Chen, Jun Dai, Xiaoyan Sun
Computer Vision
Delving into Asymmetric Information Dynamics for High-Fidelity Virtual Try-On
Zishu Qin, Zhiyu Jin, Pipei Huang et al. (+1 more)
Virtual try-on (VTON) requires precise pixel-level fidelity, yet mainstream
Diffusion Transformers (DiTs) often suffer from texture degradation and structural drift. We identify symmetric interactions in standard joint-
attention mechanisms as a source of these failures. Although such interactions support semantic flexibility in general-purpose editing, they allow
stochastic noise to corrupt deterministic garment features in VTON. We analyze this problem through asymmetric information dynamics and introduce two diagnostic indicators: Conditional
Attention Entropy (CAE) for feature unbiasedness and Injected Information Flux (IIF) for injection effectiveness. Our analysis suggests that symmetric bidirectional
attention can corrupt conditional features and attenuate the conditional signal. To address these limitations, we propose RealFit, a framework that combines Unidirectional Information Flow (UIF) with Decoupled Timestep Modulation (DTM). UIF isolates the garment condition from
stochastic noise to preserve garment identity, while DTM optimizes the modulation scale to maintain a strong conditional signal. The resulting time-invariant condition branch enables a conditional KV cache that reduces inference time by approximately 75%. RealFit offers a principled approach to conditional generation with state-of-the-art fidelity and efficiency.
Read the full abstract ▾
All authors Zishu Qin, Zhiyu Jin, Pipei Huang, Hao Zhou
Computer Vision · Cross-listed from cs.MM
TV-AudioRemover: Joint Text-Visual Guided Sound Removal with Multi-Task Hard-Mixture Curriculum
Xinyue Guo, Jianxuan Yang, Daiguo Zhou et al. (+4 more)
Visual object removal can eliminate a target from
video frames, yet its acoustic trace persists in the soundtrack, causing obvious audio-visual inconsistency. Existing
video inpainting models operate solely on pixels, while audio editing models, especially for the sound removal task, are typically driven by text and therefore rely on limited single-modal control, which is less effective than
multimodal guidance that provides stronger semantic grounding and temporal synchronization cues. In this paper, we present Text-Visual Guided Sound Removal (TV-AudioRemover), a target sound removal framework that leverages the visually edited
video together with a natural-
language instruction to suppress the sound associated with the removed visual object from the original audio mixture. To acquire high-quality training data, we devise a pipeline to construct a million-scale
dataset of single-object audio-visual aligned samples, from which we synthesize mixture-target pairs customized for model training. To effectively leverage visual context and follow instruction intent, we augment the model architecture with task
tokens, generalizable instruction modeling, and modality-specific global guidance. We further adopt multi-task training to strengthen task-role comprehension, and employ a hard-mixture curriculum that leverages semantically similar acoustic mixtures during fine-tuning to enhance fine-grained source discrimination. To support evaluation, we present AV-Remove-Bench, a comprehensive audio-visual object removal
benchmark, along with dedicated objective metrics and an MLLM-based evaluation protocol. Experiments demonstrate that our method achieves state-of-the-art performance on both subjective and objective metrics. Project page: https://yjx-research.github.io/TV-AudioRemover/.
Read the full abstract ▾
All authors Xinyue Guo, Jianxuan Yang, Daiguo Zhou, Jiagao Hu, Yuxuan Chen, Fei Wang, Jian Luan
Computer Vision
MatchFusion: Explicit-Implicit Instance Matching for Spatio-Temporal Multimodal Autonomous Driving
Xiaoyu Li, Jiajia Fu, Long Shi et al. (+7 more)
Sparse instance representations provide a compact interface for spatial
LiDAR-camera and temporal past-current interaction in
multimodal perception and E2EAD. Effective interaction requires reliable instance correspondences despite
geometric discrepancies and heterogeneous semantic representations.
Attention-based methods exploit contextual semantics but often require specialized representation
alignment, increasing computational overhead. In contrast, association based on structured object states is efficient and interpretable but lacks contextual evidence to resolve ambiguous matches. To combine these complementary strengths, we propose MatchFusion, a learnable instance matching and fusion module for spatio-temporal
multimodal autonomous driving. MatchFusion initializes pairwise affinities using
geometric similarity and category consistency, then selectively refines structurally plausible associations using instance
embeddings. The resulting soft matchmap guides a common residual aggregation operator for adaptive information exchange. This unified matching-fusion formulation supports spatial
LiDAR-camera and temporal past-current interaction, using multi-view image-plane geometry and motion-compensated BEV geometry as the respective structural priors. Experiments on nuScenes demonstrate consistent perception gains across diverse front-end configurations. Compared with a prior instance-centric fusion method, the MatchFusion-equipped system achieves higher perception accuracy while reducing FLOPs by 55.3% and GPU
memory usage by 39.3%, with the matching-fusion module accounting for only 3.7% of total perception latency. Integrating temporal MatchFusion into SparseDrive further improves perception within an E2E framework without additional supervision. These results establish explicit-implicit matching as an effective and efficient mechanism for spatio-temporal instance interaction.
Read the full abstract ▾
All authors Xiaoyu Li, Jiajia Fu, Long Shi, Tianyu Du, Ruihang Li, Xian Wu, Lijun Zhao, Yingtao Zhang, Lining Sun, Ruifeng Li
Computer Vision
Less Is More in the Long Tail: Stage-Adaptive Sample Selection for Annotation-Efficient Dense Prediction
Xiaofei Du, Lei Zhang, Shuyu Yan et al. (+2 more)
Deep learning performance generally improves with increasing training data, yet this scaling is fundamentally constrained by annotation cost in large-scale dense prediction tasks with long-tailed category distributions, where pixel- or voxel-level annotation is prohibitively expensive. We propose SASS (Stage-Adaptive Sample Selection), a stage-adaptive data-selection framework for pool-based active learning in long-tailed dense prediction. SASS combines three components: label-free self-supervised gradient scoring, prior-guided category rebalancing with validation-driven feedback, and stage-adaptive acquisition aligned with model training dynamics. This design avoids candidate ground-truth masks during gradient scoring while making acquisition responsive to long-tail imbalance and evolving representations. We evaluate SASS on a
multimodal 3D medical
segmentation testbed comprising over 100,000 samples spanning 108 anatomical structures. SASS recovers 98.3% of full-
dataset performance with a 40% training-pool annotation budget, outperforming BADGE by 5.1 percentage points. Moreover, SASS exhibits a statistically supported less-is-more pattern, surpassing full-
dataset training at the Hard-group level and, at the structure level, for the pancreas and gallbladder. More broadly, SASS shows that annotation-efficient learning depends not only on which samples are selected, but also on how the annotation budget is distributed across categories and when model-derived scores begin to guide selection.
Read the full abstract ▾
All authors Xiaofei Du, Lei Zhang, Shuyu Yan, Manning Wang, Zhijian Song
Computer Vision
Visual Jev: Accurate and Efficient Decisions from Shared Visual Context
Guanxu Yu and Yuhang Yao
Many
vision applications ask several independent, forced-choice questions about the same image. Visual Jev encodes the image and public context once, executes isolated question suffixes as a batch, and reads candidate probabilities from the backbone's
language-model head. Across four
benchmarks, answer-supervised post-training raises equal-weight macro accuracy from 70.6% to 76.1%, with the gain concentrated on the two task families represented in training. At N=32 questions per image, shared batched execution is 8.9x faster in warm amortized time than independent serial execution and remains 3.4x faster than an already-batched baseline that recomputes the prefix, at the cost of higher peak
memory. A matched typed-head control offers no consistent accuracy advantage over the
language-model-head readout. The supported design is therefore simple: adapt the backbone for quality, retain the existing readout, and share execution for efficiency.
Read the full abstract ▾
Computer Vision
Metric-Bench: Exploring In-context Spatial Metric Reasoning in VLMs for Indoor Scenes
Yuling Xi, Haokai Zhang, Muzhi Zhu et al. (+10 more)
Metric
reasoning is a critical and challenging task for
Vision Language Models (VLMs), playing a pivotal role in
embodied AI tasks such as robotic manipulation and autonomous
navigation. However, current spatial
reasoning remains bottlenecked by rigid pixel-level supervision; such localized optimization often compromises general
multimodal intelligence, triggering performance degradation or catastrophic forgetting of broad
reasoning capabilities. To address these limitations, we introduce Metric-Bench, a focused
benchmark designed to guide metric-spatial
reasoning using contextual information. By incorporating in-image reference objects with known physical dimensions, Metric-Bench guides models to implicitly learn the 2D-to-3D mapping without camera intrinsics. We further present MetricReasoner, a task-adapted reinforcement fine-tuning recipe for reference-grounded metric
reasoning, using structured
prompts and verifiable numerical rewards. Extensive experiments on Metric-Bench demonstrate that our approach significantly enhances spatial metric understanding, outperforming existing and even larger proprietary models by 43.1%, while improving downstream
embodied performance over a spatial-specialized counterpart by 30.4% on RoboSpatial overall accuracy and 9.3% on ERQA, and additionally delivering consistent gains on general
benchmarks (15.9% on V$\star$Bench, 88.9% on BLINK), indicating that the proposed adaptation does not necessarily compromise general VLM capabilities.
Read the full abstract ▾
All authors Yuling Xi, Haokai Zhang, Muzhi Zhu, Hao Zhong, Zongze Du, Hengyu Zhao, Chenchen Jing, Yufei Yin, Bin Qin, Yongjie Yang, Zhenbo Luo, Hao Chen, Chunhua Shen
Computer Vision
Identity-Centric Video Summarization via Hierarchical Fusion of Biometric, Appearance, and 3D Body Features
Milad Mirjalili, Enrique Alegre Gutiérrez, Eduardo Fidalgo Fernández et al. (+3 more)
This work presents a
video summarization
algorithm based on multi-object tracking and person reidentification. We integrate facial
embeddings, 3D body-shape features, and visual appearance into a unified tracking framework. These representations enable hierarchical identity assignment and tracking through bidirectional anchoring, which robustly recovers trajectories under severe occlusion or low visual quality. From these stable trajectories, we generate a compact set of summaries for each identity. We select keyframes using a multi-factor weighting scheme that optimizes biometric clarity, social interaction, and motion dynamics, while Adaptive Non-Maximum Suppression ensures temporal diversity. Evaluation on a custom
dataset demonstrates tracking stability, achieving an IDF1 of 97.89% and a MOTA of 95.79%. Compared to Top-K selection, our
algorithm also increases visual diversity by 146%, temporal coverage by 89%, and information retrievability by 3.5%.
Read the full abstract ▾
All authors Milad Mirjalili, Enrique Alegre Gutiérrez, Eduardo Fidalgo Fernández, Víctor González Castro, Rocío Alaiz Rodríguez, Manuel Castejón Limas
Computer Vision
Zhe Zhu, Yiheng Zhang, Peng Li et al. (+9 more)
Part
segmentation is a fundamental problem in computer graphics and 3D
vision. Recent works have expanded 3D part
segmentation beyond fixed taxonomies, but existing approaches typically only address a specific setting, such as text-guided part
segmentation or point-based interaction. In this work, we argue that these settings can be unified as an intent-conditioned generative problem, where different
prompts specify the desired part decomposition. To this end, we introduce PartLLM, a unified
multimodal model that formulates 3D part
segmentation as autoregressive semantic decomposition. Conditioned on an input shape and a user
prompt, PartLLM autoregressively generates semantic part hypotheses as queries for mask prediction and feeds them to a decomposition-aware decoder that jointly predicts coherent part masks. This unified design supports text-guided part
segmentation, interactive
segmentation, and full-shape semantic decomposition with controllable granularity within a single model. Extensive experiments across these task settings show that PartLLM consistently outperforms task-specific baselines, demonstrating the effectiveness of unifying 3D part
segmentation under an intent-conditioned generative formulation.
Read the full abstract ▾
All authors Zhe Zhu, Yiheng Zhang, Peng Li, Zixing Zhao, Honghua Chen, Yaqing Zhang, Le Wan, Zhiyang Dou, Cheng Lin, Yuan Liu, Mingqiang Wei, Wenping Wang
Computer Vision · Cross-listed from cs.RO
Sometimes You Gotta Run Before You Can Walk: Run-then-Walk Scheduling Strategy for VLM Autonomous Driving
Yuqi Ye, Shangkun Sun, Junhong Lin et al. (+6 more)
Recent VLM-based autonomous driving planners adopt GRPO-style reinforcement learning to optimize driving performance. However, existing GRPO recipes either optimize driving efficiency, risking progress-seeking but unsafe behavior, or enforce early safety constraints, leading to overly conservative behavior; both require lengthy training. To solve these problems, we first reveal two distinct RL regimes: a progress regime (Run-GRPO) that aggressively explores high progress, and a safety regime (Walk-GRPO) that restores safety under stable progress. Based on this finding, we propose $\textit{Run-then-Walk}$, a simple yet effective two-stage reward scheduling strategy for GRPO, achieving both better performance and faster convergence. Unlike one-stage RL, which may focus on progress, safety, or a mixture of both within a single training phase, this schedule explicitly separates progress discovery from safety repair. In the $\textit{Run}$ phase, we focus on progress, allowing the policy to escape the conservative bias and discover high-progress modes. In the subsequent $\textit{Walk}$ phase, we introduce endpoint and safety strategy to repair unsafe behaviors from the Run phase. This reversed schedule overcomes the conservatism of Walk-first methods and the unsafe progress-seeking of joint optimization. We validate it with various VLM-based planners on multiple
benchmarks: NAVSIMv1, NAVSIMv2, Navhard, and nuScenes. Extensive experiments demonstrate improved driving performance while requiring 40–50% fewer RL training epochs than the baselines. Code is available at https://github.com/haha-yuki-haha/AutoDrive-P3_with_Run-then-walk.
Read the full abstract ▾
All authors Yuqi Ye, Shangkun Sun, Junhong Lin, Jiayi Zhao, Changhao Peng, Wei Zheng, Guoqing Liu, Tiesong Zhao, Wei Gao
Computer Vision
MorphoSHAP: Rethinking the Unit of Attribution in Explanation for Deep Visual Models
Anirudh Prabhakaran, Alexandre Rocchi and Gianni Franchi
Visual attribution methods typically explain predictions using pixels, superpixels, or regular patches. These representations can localize important regions, but provide limited information about their structure. We introduce MorphoSHAP, a model-agnostic post-hoc method that instead uses morphological shapes as the players of a Shapley attribution game. Using the Tree of Shapes, each shape is described by its scale, geometry, and signed contribution, providing explanations of where the evidence lies, what type of structure carries it, and how strongly it affects the prediction. This shared morphological vocabulary enables spatial, textual, and global class-level explanations beyond image-specific heatmaps. To the best of our knowledge, MorphoSHAP is the first SHAP-based image attribution framework to combine these different forms of explanation. Across five diverse
datasets and three architectures, MorphoSHAP achieves strong insertion/deletion performance and outperforms competing attribution methods on several
benchmarks. Finally, a user study shows that MorphoSHAP provides explanations that are easy to use and are preferred over standard attribution baselines.
Read the full abstract ▾
Computer Vision
LiFR v2: Completion-Augmented Event Propagation for High-Rate Dense Prediction
Tao Wan, Xiaoshan Wu, Yifei Yu et al. (+6 more)
High-rate dense perception in dynamic environments is limited by the low update rate of RGB cameras, as rapid scene changes can occur between frames. Event cameras offer temporally dense but spatially sparse measurements, complementary to spatially dense RGB observations. Direct fusion cannot fully exploit this complementarity, while event-guided propagation fails on newly appearing or disoccluded regions without valid RGB support. We present LiFR v2, a unified propagation-completion-
memory framework for causal anytime and streaming dense prediction from an RGB keyframe and events. LiFR v2 introduces an Event-Guided Completion Module (EGCM) to recover task-relevant representations where propagation is unsupported, and a History
Retrieval Module (HRM) to reuse completed representations across successive queries. The framework supports semantic
segmentation,
monocular depth estimation, and multi-task dense prediction, and we further introduce SHF-Emerge to evaluate rapid object emergence and disocclusion. LiFR v2 achieves 74.37% mIoU on DSEC and 56.13% on SHF-Emerge, improving LiFR-Seg by 1.85 percentage points on the latter, while reducing SHF-Emerge depth RMSE from 1.564 m to 1.118 m over the propagation baseline. It also exceeds 100 FPS for both
segmentation and depth, demonstrating accurate and efficient high-rate perception beyond RGB frame rates.
Read the full abstract ▾
All authors Tao Wan, Xiaoshan Wu, Yifei Yu, Bo Wang, Xiaoyang Lyu, Muxin Liu, Aoxuan Pan, Zhongrui Wang, Xiaojuan Qi
Computer Vision
When Point Clouds Outperform Pixels: Rethinking Zero-Shot Multimodal Anomaly Detection
Chenglin Ye, Lupeng Liu, Dongbo Yu et al. (+2 more)
Zero-shot
multimodal anomaly detection commonly assumes that RGB and point cloud modalities are equally reliable and can contribute uniformly to anomaly localization. We challenge this assumption. Using a set of recently proposed stringent metrics that penalize false anomaly responses in normal regions, we find that point clouds are substantially more reliable than RGB under zero-shot category shift. Motivated by this observation, we propose WOOPS (When Point Clouds Outperform Pixels), a reliability-aware zero-shot
multimodal anomaly detection framework. To strengthen the more reliable
geometric modality, we design a Multi-view Information Decoupling module to suppress heterogeneous information from multi-view point cloud projections and enhance point cloud feature quality. To avoid unconditional fusion, we further introduce a Modality Reliability Calibration module to adaptively calibrate modality contributions according to their reliability. Extensive experiments show that our method achieves the best or competitive performance under the new metrics in both unimodal and
multimodal settings. Further analysis demonstrates that point cloud information also improves RGB-only inference, while ablations verify the effectiveness of both modules. Code will be released upon acceptance.
Read the full abstract ▾
All authors Chenglin Ye, Lupeng Liu, Dongbo Yu, Jun Xiao, Yunbiao Wang
Computer Vision
TRACE: Trajectory Representation and Consistency Estimation for AI-Generated Video Detection
Huangsen Cao, Hongkang chu, Siyao Yu et al. (+3 more)
Recent advances in generative
video models have enabled the synthesis of visually realistic content, posing significant challenges to synthetic
video detection. Existing detectors often rely on appearance artifacts, semantic inconsistencies, and temporal patterns that may be generator-specific, limitating generalization to unseen synthesis models. We investigate whether responses to a pretrained generative model provide more transferable forensic cues. Our key observation is that real and AI-generated
videos exhibit distinct velocity responses under a pretrained Flow Matching
video model. This distinction persists when different pretrained
video-generation backbones are used as probes, suggesting that velocity responses offer transferable forensic signals beyond visual artificts. Motivated by this observation, we propose TRACE (\emph{\underline{T}rajectory \underline{R}epresentation \underline{a}nd \underline{C}onsistency \underline{E}stimation}), a generation-process-aware framework for AI-generated
video detection. TRACE leverages a pretrained
video DiT as a velocity-field probe to extract representations at multiple flow time points, and models cross-frame consistency through velocity differences between adjacent frames. We further introduce a Real-Centered Trajectory Optimization objective that encourages generator-invariant representation learning. Extensive experiments on AIGVDBench demonstrate that TRACE generalizes effectively across diverse generators, substantially outperforming prior state-of-the-art methods on unseen open- and closed-source
video generation models.
Read the full abstract ▾
All authors Huangsen Cao, Hongkang chu, Siyao Yu, Xin Ding, Jianfeng Dong, Yongwei Wang
Computer Vision
Video-HopChain: Multi-Hop Questions and Confidence-Gated Exploration for Video Reasoning Models
Trung Nguyen Quang, Yuhao Dong, Shuo Sun et al. (+4 more)
HopChain has shown on still images that multi-hop data synthesis improves
vision-
language reasoning, because long chain-of-thought
reasoning exposes errors that compound across steps, while most data used for reinforcement learning with verifiable rewards (RLVR) rarely demands a chain of visual evidence, so these weaknesses are likely to stay unexposed. We observe the same problem in
video, where this framework has not yet been explored. We therefore build
Video-HopChain, a
dataset of 22,550 multi-hop
video questions over 13,378
videos, together with a held-out
benchmark of 1,000 questions. Each question chains three to six yes/no questions about moments in one
video, and each yields one of two integers depending on its answer. The final answer is the sum of these integers, so an exact match on that sum gives the verifiable reward that RLVR needs. We first train Qwen3-VL-8B with GRPO on a standard
video dataset, and a second stage on
Video-HopChain then raises the mean over eight
video understanding and
reasoning benchmarks from 55.4 to 57.9 and improves every one of them. Training on such a
dataset, however, exposes a known limitation of GRPO: its learning signal comes from the reward variance within a group, so hard questions whose rollouts are all incorrect and easy questions whose rollouts are all correct both leave the group with no gradient. To recover these groups at the same compute budget, we introduce Confidence-Gated Exploration (CGE). With 8 rollouts per question, CGE samples the first 4 as usual. If these 4 are either all correct or all incorrect, it samples the last 4 with the policy's most confident
token masked inside the
reasoning span, and removes the masked positions from the loss while all 8 rollouts enter the advantage. With CGE, the mean rises further to 59.3. We release the
dataset, the checkpoint, and the data generation and training code.
Read the full abstract ▾
All authors Trung Nguyen Quang, Yuhao Dong, Shuo Sun, Shuai Liu, Shulin Tian, Kim-Hui Yap, Ziwei Liu
Computer Vision
Reading Right, Answering Wrong: How Visual Configuration Changes Affect Evidence Use in VLMs
Dingyang Lin, Yingfeng Luo, Chenglong Wang et al. (+4 more)
Vision-
language models (VLMs) have achieved strong performance on tasks such as visual question answering, yet small image resizes can turn correct answers into errors. We investigate whether changes in visual configuration, such as image tiling and
token arrangement, contribute to this instability. Across seven checkpoints and four
benchmarks, equally small resizes cause more correctness flips when they switch configurations. Surprisingly, in over half of these cases, models answer the question incorrectly but can still read the correct answer when told what to read. Furthermore,
attention interventions in LLaVA-NeXT suggest that configuration changes can weaken the use of readable information during answering. We therefore guide models using field cues and their own transcriptions. With annotation assistance, these forms of guidance together correct 97.2% of errors with readable information. These findings show that configuration changes can affect how models use information they can still read.
Read the full abstract ▾
All authors Dingyang Lin, Yingfeng Luo, Chenglong Wang, Chenwei Zhu, Anxiang Ma, Jingbo Zhu, Tong Xiao
Computer Vision · Cross-listed from cs.RO
Dual Covariance Gaussian Splatting SLAM: Decoupling Rendering and Registration for Robust Real-Time Tracking
Edward Beng Wai Tan and Siew-Kei Lam
ICP-based 3D Gaussian Splatting (3DGS) SLAM tracks in real time by registering incoming frames against map Gaussians, using each primitive's covariance for both rendering and registration. These two uses place conflicting demands on one covariance. The mapper shapes it to minimize photometric error, often flattening it against surfaces, while robust registration typically benefits from measurement uncertainty. We propose a dual-covariance parameterization. Each Gaussian keeps a single mean but holds two covariances: a rendering covariance optimized by the mapper, and a tracking covariance derived from an RGB-D sensor noise model. We further use the tracking covariances as Gaussian anchors for image corners, providing constraints in directions where depth geometry is weak. We evaluate on TUM RGB-D, ScanNet, Replica, and two outdoor sequences recorded with a RealSense D435i on wheeled and handheld platforms. We achieve robust tracking performance across multiple scenes and reduced odometry drift, while tracking at $\sim$ 60 FPS.
Read the full abstract ▾
Computer Vision
SAMI3D-DW: Interactive Segmentation of Any 3D Medical Images
Ping Gong, Shiyuan Su, Fandong Zhang et al. (+4 more)
Interactive
segmentation of 3D medical images supports quantitative analysis of anatomical structures and disease while allowing users to specify and refine their targets. Despite substantial progress by nnInteractive and VISTA3D, reliable
segmentation across diverse clinical targets remains challenging, particularly for complex anatomical structures and the heterogeneous, long-tailed spectrum of pathology. We present SAMI3D-DW V1 (hereafter SAMI3D-DW), an interactive 3D
segmentation model trained on Deepwise's large-scale proprietary medical image
datasets. We evaluate the model under simulated user interactions on a CT/MR
benchmark comprising 4,326 cases from 219 source
datasets, spanning 107 anatomical and pathological categories, organized by a medical taxonomy and evaluated with a category-balanced DSC score. SAMI3D-DW achieves the highest category-macro Dice among evaluated methods in both interaction modes. With one point, it scores 0.5756 versus 0.5316 for nnInteractive, the strongest baseline, rising to 0.7771 versus 0.7495 with five points. With bounding-box initialization, the scores are 0.7129 versus 0.6530. After five corrective clicks, SAMI3D-DW reaches 0.8004 versus 0.7868. For radiologists and clinicians, SAMI3D-DW enables
segmentation of complex anatomical structures, including intracranial vessel trees on CT and MR angiography, with a few clicks. In a preliminary in-house comparison involving neurofibromatosis type 1 (NF1), SAMI3D-DW-assisted tumor annotation took minutes per case and approximately one-fifteenth of the time required for manual annotation, highlighting its potential to support volumetric treatment-response assessment.
Read the full abstract ▾
All authors Ping Gong, Shiyuan Su, Fandong Zhang, Xinchen Han, Haowei Sun, Yiming Li, Yizhou Yu
Computer Vision
Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoning
Dingkang Yang, Yizhou Liu, Wendong Cheng et al. (+5 more)
Generative models have advanced image-conditioned 3D content creation, yet generating controllable and executable 3D scenes from a single image remains challenging. Existing 3D generative approaches can synthesize visually plausible objects and scenes, but their spatial layout estimation is coupled with specific asset generators. They struggle to jointly model object semantics, metric geometry, and scene-level spatial relationships, which are essential for interactive editing, physical simulation, and
embodied applications. We propose Fysiverse-3D-
Vision, a unified
vision-
language-geometry framework for generative 3D scene reconstruction and executable asset construction from a single image. We establish a shared representation where spatial
reasoning and
geometric reconstruction mutually enhance each other, allowing object layouts to be inferred beyond the constraints of individual asset generators. Our model integrates textual supervision, semantic visual cues, and
geometric representations within a unified
Transformer to capture scene context, metric geometry, and object-level interactions. An object-conditioned layout module performs cross-
attention between target object representations and global
geometric features to predict object translation, rotation, and scale. Training progressively learns geometry-
language alignment, introduces layout
reasoning while preserving reconstruction capability, and refines physical consistency through collision-aware optimization. By separating spatial layout
reasoning from asset synthesis, Fysiverse-3D-
Vision provides an adaptable interface for interactive scene editing, object-level manipulations, and executable 3D content generation. Experiments demonstrate that our framework achieves superior
geometric consistency, layout estimation, rendering quality, and physical property understanding compared with existing approaches.
Read the full abstract ▾
All authors Dingkang Yang, Yizhou Liu, Wendong Cheng, Zizhi Chen, Shunli Wang, Yang Liu, Hongsheng Li, Lihua Zhang
Computer Vision
Annual Earth-observation embeddings encode wildfire disturbance and support simplified burned area mapping
Jovana Knezevic, Clement Atzberger, Zhengpeng Feng et al. (+3 more)
Medium-resolution (10-30 m) burned area mapping is vital for monitoring wildfires and their impacts, but remains difficult to scale. Existing methods require either curated fire-specific imagery or dense time-series analysis. Here, we tested whether annual Earth-observation
embeddings retain wildfire disturbance signals sufficiently to map burned areas without either requirement. Using Tessera and AlphaEarth
embeddings, we tested individual burn-scar delineation, mapping of all same-year fires within an area, regional wall-to-wall mapping, cross-continental transfer, and intra-annual fire timing. Tessera strongly encoded wildfire disturbance, allowing even linear models to separate burned from unburned pixels; the signal was weaker in AlphaEarth. Models trained on a single Tessera
embedding matched or exceeded equivalent models using paired pre- and post-fire HLS imagery, and outperformed post-fire imagery alone. The same approach mapped all same-year fires within
benchmark scenes (F1 = 0.90). Applied across California, with no California fire data used for downstream training, it recovered 97% of reference burned area and detected substantially more small and medium-sized fires than GABAM or MCD64A1. Separately, a model trained on 2018-2021 US fires transferred without retraining to 88 European fires from 2024-2025 (F1 = 0.88). For well-detected fires, ignition timing was recovered with a mean absolute error of 13 days. Performance declined for fires ignited near the end of the calendar year, and wall-to-wall deployment produced systematic false positives in some unseen landscapes. Annual
embeddings nevertheless achieve high
segmentation accuracy while moving the burden of dense time series processing upstream, providing a promising path towards simpler regional burned area mapping.
Read the full abstract ▾
All authors Jovana Knezevic, Clement Atzberger, Zhengpeng Feng, Adam F. A. Pellegrini, Srinivasan Keshav, David Coomes
Computer Vision
FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance
Jaihyun Lew, Mingi Jung, Minjun Park et al. (+2 more)
Reference-based image quality assessment (IQA) metrics aim to reflect how humans perceive the perceptual distance between a pair of images. To learn how the human visual system (HVS) operates, recent reference-based IQA metrics heavily rely on human-annotated data. Mean opinion score (MOS)-based pointwise scoring, which assigns a scalar quality value per image, is preferable for annotation but is prohibitively expensive to collect at scale and is known to be noisy due to inconsistent human judgments. As an alternative, two-alternative forced choice (2AFC) pairwise labels have gained popularity due to their reliability and efficiency, but they capture only relative comparisons between pairs. In this paper, we propose a fully automated data generation pipeline that generates pointwise perceptual distance labels between image pairs without any human annotation. Our approach exploits the generative dynamics of
diffusion models as a perceptual distance proxy, where the coarse structure of an image is generated in the early timesteps and the fine details are generated in the later timesteps. Images that fork early in the generation process share only coarse structure and are perceptually far apart; images that fork late differ only in fine detail. We demonstrate that the
diffusion trajectory aligns well with the human visual system, and use this forking moment, FoMo, as a reference-grounded distance label to supervise the training of a reference-based IQA metric. The pointwise labels, which support universal comparison between arbitrary image pairs, enable an information-rich training objective. Extensive experiments across diverse backbone architectures confirm the effectiveness of our generation pipeline, outperforming human-annotated
datasets in multiple
benchmarks.
Read the full abstract ▾
All authors Jaihyun Lew, Mingi Jung, Minjun Park, Wooseok Song, Sungroh Yoon
Computer Vision
Interpretable AI plus Handheld, Portable Retinal Photographs: A Low-Cost Glaucoma Screening Solution for West Africa
Charis Y. N. Chiang, Tarela Sarimiye, Adeyinka Ashaye et al. (+4 more)
Purpose: To develop and evaluate an interpretable artificial intelligence (AI) framework for glaucoma screening from low-cost portable, handheld retinal fundus photographs in a West African population and to compare its performance with clinical tabletop fundus imaging. Methods: We used data from a community-based study of 681 participants (1,362 eyes) in Nigeria, comprising 414 glaucoma, 478 glaucoma suspect, and 470 non-glaucoma eyes. Fundus photographs were acquired using the low-cost handheld, portable Volk Viva retinal camera and the Canon CR-2-AF tabletop camera. We fine-tuned component models separately to each device to perform vessel
segmentation, cup and disc boundary
segmentation, and feature extraction to detect optic nerve head features. A final classification model combined these components to classify scans as glaucoma, glaucoma suspect or non-glaucoma. Feature-weight analysis and Gradient-weighted Class Activation Mapping were used for interpretation. Results: The models performed well on both Volk Viva and Canon CR-2-AF images: Vessel
segmentation: 0.98 Dice Coefficient (DC) (Volk) and 0.94 DC (Canon); Cup and disc
segmentation: 0.95 DC (Volk) and 0.96 DC (Canon); Optic nerve head feature detection: area under the receiver operating characteristic curve (AUCs) of 0.83$\pm$0.03 (Volk) and 0.87$\pm$0.04 (Canon); Classification model: AUCs of 0.85$\pm$0.01 (Volk) and 0.93$\pm$0.01 (Canon). Reports for each image, present model decision confidence scores and decision-rationale visualizations to support clinical interpretation. Conclusions: Volk Viva results were reasonably comparable to Canon CR-2-AF in the component models and not far behind in classification. This shows that interpretable AI combined with low-cost, portable imaging may enhance community-level glaucoma screening, especially in settings with limited specialist access and resources.
Read the full abstract ▾
All authors Charis Y. N. Chiang, Tarela Sarimiye, Adeyinka Ashaye, Martin Buist, Michael A. Hauser, Olusola Olawoye, Michaël J.A. Girard
Computer Vision
C2FXNet: Coarse-to-Fine Scene Expert for Unified Object Detection across Adverse Weather
Tianle Fang, Zhenbing Liu, Chong Yin et al. (+2 more)
Object detection in adverse weather remains challenging because severe degradations weaken visual quality and disrupt semantic feature representations across diverse scenes. Existing methods usually rely on condition-specific designs, which limits their ability to generalize within a unified detector. In this paper, we propose a Coarse-to-Fine Scene Expert Network (C2FXNet) that achieves unified detection through hierarchical scene guidance. Specifically, C2FXNet introduces a dual-level guidance mechanism consisting of a Multi-step
Reasoning Router (MRR), which performs GRU-based recurrent scene
reasoning over compressed multi-scale visual cues and frozen coarse scene prototypes, and a Fine Scene Refinement (FSR) module, which uses image-specific semantic cues to modulate high-level features for local variation handling. Furthermore, a Scene-aware Mixture-of-Experts (SMoE) dynamically combines scene-specific experts under the joint guidance of MRR and FSR. By coupling coarse scene
reasoning with fine-grained semantic refinement, C2FXNet enables robust multi-scene detection without scene-specific training. Extensive experiments on RTTS, ExDark, and our newly constructed Adverse Weather
Dataset (AWD) demonstrate that C2FXNet consistently outperforms state-of-the-art methods across foggy, dark, and clear conditions, reaching 63.70%, 71.14%, and 54.19% mAP on RTTS, ExDark, and AWD, respectively. The source code will be released at https://github.com/PolarisFTL/C2FXNet.
Read the full abstract ▾
All authors Tianle Fang, Zhenbing Liu, Chong Yin, Bolun Li, Haoxiang Lu
Computer Vision
Initialization and Stopping Tolerance in CPU Dermoscopic Segmentation
Wenhao Xu, Yixian Kong, Ting Pan et al. (+2 more)
Contour initialization and numerical stopping can jointly affect the evaluation of active-contour
segmentation. We examine their interaction using the open-source scikit-image Chan-Vese implementation on a resized ISIC 2017 mirror. A fixed development set of 100 images selects a common input channel; all 600 images in the repository's held-out partition are then evaluated. Otsu thresholding is compared with checkerboard-, disk-, and Otsu-initialized contours under default and tighter level-set tolerances. At the default tolerance, Otsu initialization increases mean image Dice from 0.6011 to 0.6660 relative to checkerboard initialization, a paired difference of 0.0649 (95% image-bootstrap interval [0.0452, 0.0860]). Otsu thresholding alone achieves 0.6897. The default disk initializer stops after one iteration on 471 images. Tightening the tolerance reduces the Otsu-seed advantage over checkerboard initialization to 0.0197, with most runs reaching the 500-iteration limit. The default-tolerance advantage also reverses between small- and large-lesion strata. These findings show that an improvement over a generic initializer can coexist with deterioration relative to the threshold baseline. Evaluations should retain the unrefined mask as a comparator and report the initial-field definition, stopping tolerance, and observed iteration counts together.
Read the full abstract ▾
All authors Wenhao Xu, Yixian Kong, Ting Pan, Feilong Wang, Changwei Wang
Computer Vision
Real-Time Atomic-Resolution Electron Phase Imaging without Probe Calibration via Ptychography-Supervised Learning
H. Yue, C.-C. Chen, C.-N. Hsiao et al. (+4 more)
Atomic-scale phase imaging is central to resolving defects, interfaces, and weakly scattering atoms that govern the behavior of nanoscale materials. Electron ptychography delivers sub-ångström phase sensitivity but remains an offline technique, because its iterative reconstruction is computationally expensive and sensitive to experimental calibration, preventing live use during data acquisition. Here, a ptychography-supervised local inference framework is presented that converts four-dimensional scanning transmission electron microscopy (4D-STEM) into an acquisition-compatible phase-imaging workflow. Physics-constrained reference phase maps reconstructed from a single experimental AuPd
dataset serve as teacher labels for a compact model that predicts local phase patches directly from diffraction measurements, without explicit probe input or online iterative optimization. Full-field images are assembled by deterministic overlap stitching. The workflow reaches an online latency of about 0.27 ms per probe position and a throughput of about 20,000 positions per second, an approximately 1,000-fold speed-up over GPU-accelerated ePIE, while preserving atomic-scale
lattice contrast and reciprocal-space fidelity. Without fine-tuning, the same model transfers across materials (WS2), defocus conditions (high-
entropy alloy nanoparticles), and instruments (hBN at 300 kV). The approach amortizes ptychographic redundancy into a fast, generalizable workflow that enables real-time atomic-scale phase imaging for materials microscopy.
Read the full abstract ▾
All authors H. Yue, C.-C. Chen, C.-N. Hsiao, J. Cheng, Y. Liu, X.Z. Liao, Steve F. Shu
Computer Vision · Cross-listed from cs.RO
CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Image
Dongwon Son, Junhyek Han, Yoontae Cho et al. (+5 more)
Robots operating safely in cluttered everyday environments often need to infer scene geometry from partial observations. Methods that detect objects in 2D and reconstruct them independently struggle in such scenes: a missed object is never reconstructed, a merged detection can fuse two objects, and separately reconstructed meshes may overlap or fail to touch their supporting surfaces. We introduce CODA (Complete Once, Decompose Afterward), a generative model that instead reconstructs the complete scene geometry from a single unsegmented RGB-D image, then separates the surface into the surrounding environment and movable objects. Still, generated scene geometry can drift from the observed partial point cloud. To reduce this drift, CODA uses two explicit 3D grounding mechanisms to keep reconstructed geometry consistent with observed surfaces while completing unseen regions. Experiments on HomebrewedDB and our custom cluttered-scene
dataset show more accurate reconstructions and a higher fraction of objects remaining in place under simulated
gravity than both object-first and scene-first baselines.
Read the full abstract ▾
All authors Dongwon Son, Junhyek Han, Yoontae Cho, Minseok Lee, Hong-seok Choi, Jiwook Choi, Hyungjin Kim, Beomjoon Kim
Computer Vision
GameDirector: Decoupling Gameplay Logic from Rendering for Player-Configurable Game World Models
Zijun Lin, Zhiyang Deng, Yuzhe Wu et al. (+2 more)
Recent game world models support realistic visual simulation and interactive gameplay based on player inputs. However, they typically learn environment dynamics from pixel-level supervision, jointly modeling perception,
memory, state transitions, and rendering within a single end-to-end framework. While this design enables open-ended, action-controllable generation, it still falls short of delivering a complete gameplay experience. Games are governed by explicit mechanics, such as health deduction, skill activation, combat rules, and termination conditions. These mechanics depend on precise and consistent state transitions that generative models alone cannot reliably enforce. In contrast, game engines can guarantee such mechanics through hard-coded rules, but provide limited flexibility for player-driven creation. To bridge these paradigms, we introduce GameDirector, the first agentic framework that decouples rule-based gameplay logic from visual rendering. Given player-defined configurations, the framework acts as an intelligent director that interprets visual observations, updates game states, tactically controls NPCs, and enforces gameplay rules. It then translates these decisions into text
prompts that guide the
video world model to render the resulting gameplay. This separation allows players to configure characters, states, and rules much like a game developer while preserving coherent game mechanics. Experiments on three games, using data collected by our automated gameplay
agent, show that GameDirector achieves accurate state tracking, reliable rule following, and improves boss action quality by more than 39.9% over various end-to-end game world model settings. Overall, by externalizing player-controllable game logic, GameDirector establishes a middle ground between hard-coded simulation and generative modeling, enabling more flexible and closed-loop gameplay experiences.
Read the full abstract ▾
All authors Zijun Lin, Zhiyang Deng, Yuzhe Wu, Bihan Wen, Yeying Jin
Computer Vision
Decoupling Disease, Covariates, and Individual Variability: A Unified Disentanglement Framework for Medical Image Classification
Shengjie Zhang, Jinglin Zhang, Zhuangzhuang Jiang et al. (+10 more)
Accurately isolating disease-related features from confounding
covariates (e.g., age, gender, site) and individual variations remains a fundamental challenge in medical image classification. Traditional regression-based approaches may ignore non-linear relations between image features and true
covariates. To overcome this issue, we present a generalized Medical Imaging Disentanglement Learning (MedIDL) framework. MedIDL maps image features into three mutually orthogonal latent spaces through specialized disentanglement heads: a disease classification head guided by a supervised loss, a
covariate-
alignment head constrained by cross-subject similarity matching, and a Gaussian head absorbing individual variations. We evaluated our framework across 7
datasets encompassing diverse imaging modalities. MedIDL outperforms state-of-the-art supervised and self-supervised classification methods in accuracy across all
datasets. Association analyses demonstrate that MedIDL successfully isolates target-specific latent representations. Gradient-based interpretability mappings localize pathognomonic patterns aligning with established clinical literature.
Read the full abstract ▾
All authors Shengjie Zhang, Jinglin Zhang, Zhuangzhuang Jiang, Ziqi Yu, Yipin Zhang, Qi Zhang, Xiang Chen, Haibo Yang, Fei Gao, Longbiao Cui, Yuan Zhou, Xiao-Yong Zhang, Alzheimer's Disease Neuroimaging Initiative
Computer Vision
What Drives Hierarchy-Aware Image Retrieval? Taxonomy Alignment, Objective Choice, and Geometry
Ling Shi
Foundation
vision models provide strong generic representations, yet high class-level
retrieval accuracy does not necessarily imply that an
embedding respects a target semantic taxonomy. We study strict explicit-taxonomy image
retrieval on frozen DINOv2 features and ask: when hierarchical
retrieval improves, how much of the change is associated with the organization of taxonomy-aware supervision, and how much with the Euclidean-
hyperbolic geometry choice? We evaluate higher levels with strict cross-class criteria that exclude finer-grained matches, and compare Euclidean and
hyperbolic projections trained with taxonomy-distance regression or a taxonomy-aware supervised contrastive objective. A compute-matched 2 x 2 Geometry x Loss factorial uses the same 768-256-32 projector capacity, optimization schedule, batch order, and fixed 100-epoch budget; the Loss axis denotes the Regression-to-Taxonomy-SupCon objective-family contrast. On CUB, the objective-family contrasts in mean hierarchy mAP (strict middle/high average, excluding Class/Leaf) are +0.0487 in Euclidean space and +0.0414 in
hyperbolic space, compared with geometry contrasts of +0.0102 and +0.0030. On NABirds Parent-disjoint
retrieval, the corresponding objective-family contrasts are +0.0467 and +0.0440, whereas geometry contrasts are +0.0017 and -0.0009. A semantic-
alignment control shows that the true taxonomy substantially outperforms a structure-preserving shuffled hierarchy, while a NABirds
curvature/radius control does not support stronger negative
curvature as the explanation for the observed hierarchy gains. Across the two taxonomies, the Regression-to-Taxonomy-SupCon contrasts are larger in aggregate than the evaluated geometry contrasts; semantic
alignment also matters separately, while geometry remains hierarchy-dependent.
Read the full abstract ▾
Computer Vision
Shallow to Deep: Aligning Token Pruning with Stage-wise Roles in LVLMs
Shuo Zhang, Jintao Tong, Yixiong Zou et al. (+2 more)
Large
Vision-
Language Models (LVLMs) incur high computational costs from redundant visual
tokens. Although training-free
attention-based multi-layer
pruning in the
vision encoder stage has been explored as an effective strategy, we find that
pruning in shallow layers consistently degrades performance. In this paper, we aim to understand this problem and seek a solution. By analyzing
attention patterns across network depth, we find that shallow layers primarily function as edge detectors with chaotic
attention maps, while deeper layers transition through local subject recognition and unstable semantic aggregation. To address the misalignment between
pruning strategies and network stages, we propose STD, a hierarchical
token pruning framework that adapts
token selection mechanisms to the functional role of each network stage. STD employs High-Frequency Spectral Analysis in shallow layers to deterministically preserve structural edges, uses Gaussian-Smoothed
Attention in intermediate layers to maintain spatial coherence, and introduces a Stability-Adaptive Trigger in deep layers to execute
pruning only during semantically stable phases. Extensive experiments show that STD outperforms state-of-the-art
pruning methods by 1.1% on LLaVA-1.5-7B with 88.9%
token reduction, while also being plug-and-play and highly effective when combined with other methods, and by 2.1% on LLaVA-NeXT-7B with 94.4% reduction, delivering a 3.9x speed-up in the prefilling stage. Our code will be released at https://github.com/Twilight03/STD.
Read the full abstract ▾
All authors Shuo Zhang, Jintao Tong, Yixiong Zou, Yuhua Li, Ruixuan Li