Computer Vision
TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image
Xin Zhang, Yabo Chen, Zixuan Duan et al. (+4 more)
Interactive visual world models must distinguish observation from physical intervention. Camera motion reveals new surfaces, whereas intervention changes object motion, contact, and deformation. Current
video world models are largely driven by appearance priors and often lose physical or spatial consistency over long horizons. We present TourPhysics, an online framework initialized from a single image and a declarative physical configuration. TourPhysics extends PhysOmni, our ACM Multimedia 2026 work, from finite physics-grounded
video synthesis to persistent exploration and manipulation. TourPhysics combines deterministic simulation with
video generation while assigning separate roles to simulator state,
geometric evidence, generator controls, and appearance
memory. For each action, the simulator computes a finite physical and camera trajectory before the corresponding observation is generated. Accepted observations publish the terminal state and update the appearance
memory and subsequent generator controls, while the committed state and simulator geometry remain fixed throughout synthesis and retry. We further separate the simulator geometry used for projection and visibility from the relative depth used to condition the generator. A reference-anchored
memory retrieves accepted static appearance through
geometric cross-view correspondence and incorporates it through a bounded residual that reverts to the native path when no valid correspondence exists. On simulator-defined camera tours and object manipulations, TourPhysics follows prescribed camera and object trajectories more closely than the evaluated baselines, preserves the input scene, and reduces appearance drift during long-horizon revisits.
Read the full abstract ▾
All authors Xin Zhang, Yabo Chen, Zixuan Duan, Haibin Huang, Chi Zhang, Feng Xu, Xuelong Li
Computer Vision
Methane Detection On Board Satellites from Unorthorectified Imagery
Luca Marini, Maggie Chen, Hala Lamdouar et al. (+3 more)
As a potent greenhouse gas, methane is a major driver of
climate change. Its effective mitigation relies on timely detection. Conventional detection methods rely on orthorectification to correct
geometric distortions and matched filters to enhance plume signals, which are steps designed for ground processing and poorly suited to onboard execution. We introduce UnorthoDOS, a
dataset and approach for training machine learning models directly on unorthorectified hyperspectral imagery, bypassing both orthorectification and matched-filter products. Our U-Net models trained on unorthorectified data approach the performance of models trained on orthorectified data (IoU 16.91% vs. 18.47% on all plumes), while both substantially outperform the mag1c matched-filter baseline (IoU 4.76%). We further demonstrate the feasibility of onboard deployment: FP16 compression halves model size with under 0.3% output deviation. The trained ML models and two ML-ready
datasets – orthorectified and unorthorectified hyperspectral imagery from the EMIT sensor – are publicly available at https://huggingface.co/
datasets/SpaceML/UnorthoDOS, with code at https://github.com/spaceml-org/plume-hunter.
Read the full abstract ▾
All authors Luca Marini, Maggie Chen, Hala Lamdouar, Laura Martínez-Ferrer, Dr C. P. Bridges, Giacomo Acciarini
Computer Vision
InterSing: Explicit Interaction Dynamics for 3D Duet Singing Animation and Beyond
Yihan Zhou, Zikai Huang, Yuyang Yu et al. (+3 more)
We present InterSing, a framework for generating realistic 3D head animations for duet singing performances. Unlike solo singing, duet performance requires each singer to balance individual expressiveness with intermittent interaction at musically salient moments, such as phrase boundaries, synchronized rhythms, and call-and-response passages. Because these interactions are sparse and rhythm-dependent, existing audio-driven animation methods and conversational interaction models do not adequately capture their structure. Our key insight is that duet coordination can be represented as a time-varying signal that reflects how strongly performers engage with one another throughout a song. Based on this observation, we introduce interaction logits, an interpretable latent representation that models the degree of cross-performer engagement at each time step. We learn these logits using weak supervision and use them to condition an interaction-aware
diffusion model jointly driven by audio features and interaction dynamics. This formulation enables unified multi-mode generation, spanning independent motion, coordinated behavior, and smooth transitions between them. Experiments show that InterSing generates realistic and expressive singing head animations with stronger coordination and musical
alignment than existing methods, while preserving each performer's characteristic motion style. We further demonstrate that the same formulation generalizes to multi-singer performances and provides intuitive control over when and how performers engage.
Read the full abstract ▾
All authors Yihan Zhou, Zikai Huang, Yuyang Yu, Xuemiao Xu, Cheng Xu, Shengfeng He
Computer Vision
Sound-based Multi-Person 3D Pose Estimation
Yusuke Oumi, Yuto Shibata, Go Irie et al. (+3 more)
Can we recover the 3D poses of multiple people using only sound? This paper presents the first attempt to estimate multi-person 3D poses solely from acoustic signals. Estimating the poses of multiple individuals using acoustic signals is inherently challenging due to the superposition of motion-dependent signal variations. Unlike single-person scenarios, the presence of multiple subjects leads to overlapping acoustic signatures, making it difficult to attribute specific signal changes to an individual's pose. Furthermore, the complexity is compounded by inter-person reflections, which introduce intricate propagation delays that obscure the temporal motion-acoustic relationship. To address these issues, we propose SoundMHPE (Sound-based Multi-person Human Pose
Estimator), a novel encoder-decoder framework consisting of two key components. First, the Acoustic Multi-scale Encoder captures diverse temporal and fine-grained frequency features to isolate subtle acoustic signatures from complex, overlapping signals. Second, the Temporal Pose Decoder employs an
attention mechanism to disentangle multi-person information across successive frames. By jointly accounting for temporal dynamics and inter-person dependencies, this component precisely reconstructs frame-wise individual poses. To validate our approach, we constructed the 6-hour Acoustic Multi-person Pose (AMP)
dataset consisting of 432K synchronized frames of multi-person pose and acoustic data, and demonstrated that our SoundMHPE outperforms baseline models. Project page: https://oumi03.github.io/sound-mhpe/
Read the full abstract ▾
All authors Yusuke Oumi, Yuto Shibata, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
Computer Vision
SimFuse3D: Source-Guided Target Simulation and Confidence-Guided Multi-Stage Localization Reweighting for Cross-Platform 3D Object Detection
Yongchun Lin, Xinliang Zhang, Yun Zou et al. (+7 more)
Changes in sensor height and viewpoint alter object-level point distributions, making cross-platform
LiDAR unsupervised domain adaptation (UDA) difficult. Self-training uses labeled source scans and unlabeled target scans, yet a retained prediction may provide a useful target location while enclosing sparse foreground returns, background clutter, or points inconsistent with the predicted box. We refer to this mismatch as box-point inconsistency. We introduce SimFuse3D, which preserves the target placement and repairs the associated pseudo object using measured geometry from labeled source scans. Object
Memory retrieves a similar labeled source instance. Target Simulation places the retrieved source geometry at the target location, aligns its points with the target viewing geometry, and filters the aligned crop to approximate the target observation. Confidence-Guided Multi-Stage Localization Reweighting (CMLR) maps each target pseudo-object confidence score to a bounded weight shared by RPN localization and R-CNN box regression. All components operate only during adaptation, leaving the detector architecture and inference graph unchanged. Across six cross-platform transfers, SimFuse3D consistently outperforms Pi3DET-Net and achieves the best performance among the compared adaptation methods on nearly all metrics. On nuScenes-to-KITTI, it ranks first among the compared adaptation methods with both evaluated detectors.
Read the full abstract ▾
All authors Yongchun Lin, Xinliang Zhang, Yun Zou, Zhixuan Xiao, Liang Lei, Jianya Guo, Yuqiang Zhai, Xiaofeng Wang, HaiKuo Xu, Haoang Li
Computer Vision
Mitigating Performance Discrepancy in Cross-Domain 3D Class-Incremental Learning
Jinge Ma, Gautham Vinod, Bruce Coburn et al. (+3 more)
3D perception plays a crucial role in real-world applications such as autonomous driving, robotics, and AR/VR. In practical scenarios, 3D perception models need to continually adapt to newly emerging 3D object categories, making class-incremental learning (CIL) particularly important. However, unlike 2D images, 3D point clouds are inherently heterogeneous: objects from the same class may not only come from the clean CAD domain, but also from RGB-D camera scans of varying quality,
video reconstructions, or even corrupted observations. We discover that such heterogeneity introduces a new challenge beyond catastrophic forgetting: the degree of performance degradation can vary substantially across domains, a phenomenon we term performance discrepancy. To investigate this problem, we establish the Domain3D-CIL training and evaluation protocol, which contains point cloud categories from heterogeneous domains. We further adapt a wide range of mainstream CIL methods to the 3D modality. The results demonstrate that this performance discrepancy consistently appears across these baselines. To mitigate this issue, we introduce PolyMem, an exemplar-free approach that implicitly models rich high-order statistics of the feature distribution to enhance cross-domain robustness. Experiments demonstrate that our method effectively alleviates the performance discrepancy while improving the model's performance across domains. Code will be made publicly available upon acceptance.
Read the full abstract ▾
All authors Jinge Ma, Gautham Vinod, Bruce Coburn, Jui-Feng Chi, Siddeshwar Raghavan, Fengqing Zhu
Computer Vision
LetOccVote: Learning Weakly Supervised 3D Occupancy through Consensus
Chi Zhang, Qi Song, Feifei Li et al. (+2 more)
Weakly supervised 3D occupancy prediction reduces the reliance on costly 3D annotations by learning from 2D pseudo-labels generated by
vision foundation models. However, existing methods typically use these imperfect pseudo-labels directly as supervision, making occupancy learning vulnerable to erroneous
geometric and semantic targets. We observe that agreement across repeated observations provides an inexpensive and reliable cue for assessing pseudo-label reliability. Based on this observation, we propose LetOccVote, a weakly supervised Gaussian-based occupancy framework that leverages cross-frame voting to improve both
geometric and semantic supervision. For geometry, Depth Vote exploits cross-frame
geometric agreement to refine supported pseudo depth and reject contradictory estimates before volumetric lifting and depth supervision. For semantics, Semantic Vote aggregates pseudo-semantic observations in a shared 3D space to identify reliable and contested evidence, strengthening reliable semantic supervision while filtering unreliable pseudo-label segments. The entire framework is trained solely with 2D pseudo-label supervision without requiring 3D occupancy annotations. On Occ3D-nuScenes, LetOccVote achieves 53.27 IoU and 20.39 mIoU, establishing state-of-the-art performance among methods with 2D pseudo-label supervision.
Read the full abstract ▾
All authors Chi Zhang, Qi Song, Feifei Li, Jie Li, Rui Huang
Computer Vision
PAPT++: Risk-Aware Adversarial Tuning and Generation for Single Domain Generalization
Zhipeng Xu, De Cheng, Xinyang Jiang et al. (+5 more)
Single domain generalization (SDG) aims to learn a model from one labeled source domain that generalizes to unseen target domains. A common strategy is to enrich the source distribution with augmented or generated samples, and recent text-to-image (T2I)
diffusion models provide a strong generative prior for this purpose. However, diversity alone is insufficient for robust generalization, because useful generated samples should also capture variations that the current classifier finds difficult. Motivated by distributionally robust optimization (DRO), we define a semantic ambiguity set in the class-conditional generative space of a pretrained T2I model and search it for samples with high classification loss under the current classifier. To this end, we introduce PAPT++, a risk-aware
adversarial generation-training framework for SDG. PAPT++ first learns diverse semantic reference images for each class through image-text
alignment and intra-class diversity regularization. These references then serve as denoising targets during classifier-guided
diffusion synthesis, reducing semantic drift while guiding generation toward challenging variations. The generated samples are combined with the source data to update the classifier, and the updated classifier guides the next synthesis round in return. In this way, PAPT++ progressively exposes the classifier to challenging yet semantically consistent variations. Extensive experiments on standard SDG
benchmarks demonstrate the superiority of the proposed PAPT++ method and the effectiveness of its main components.
Read the full abstract ▾
All authors Zhipeng Xu, De Cheng, Xinyang Jiang, Lingfeng He, Huaijie Wang, Dongsheng Li, Nannan Wang, Xinbo Gao
Computer Vision
Weather-Conditioned Depth Anything
Zhaoming Xu, Chan-Wei Hu, Kuan-Ru Huang et al. (+4 more)
Monocular depth estimation foundation models, such as the Depth Anything series, have achieved remarkable performance across diverse domains. However, they still suffer from critical failures under adverse weather conditions, such as fog, rain, snow, or at night. To address this, we present Weather-Conditioned Depth Anything (DA-W), a framework that explicitly disentangles style from content for weather-robust depth estimation. Specifically, we introduce a Style Filter trained on a curated mix of real and synthetic degradation
datasets to extract content-independent, degradation-aware weather
embeddings. This style
embedding is then injected into the Depth Anything backbone using a parameter-efficient, zero-initialized adapter. Such a lightweight modulation allows a single unified model to robustly adapt to diverse conditions, including fog, rain, snow, and low-light, while avoiding catastrophic forgetting of its core generalization abilities in normal conditions. We train the adapter using a pseudo-label
distillation and
alignment strategy. Our comprehensive experiments demonstrate that our proposed DA-W achieves state-of-the-art robust depth estimation, improving AbsRel by an average of 3.7% on our curated weather
benchmarks, while matching or slightly outperforming performance on standard clean
benchmarks. Our project page is available at https://zhaoming-tamu.github.io/WCDA/.
Read the full abstract ▾
All authors Zhaoming Xu, Chan-Wei Hu, Kuan-Ru Huang, Zihao Zhu, Renjie Li, Yang Zhou, Zhengzhong Tu
Computer Vision · Cross-listed from cs.RO
CoLMIN: LLM-based Multi-Decision Path Negotiation for Cooperative Autonomous Driving
Zhe Huang, Zhaoxin Fan, Shuo Wang et al. (+3 more)
Multi-vehicle cooperative autonomous driving enhances the safety and reliability of autonomous driving systems through information sharing among connected vehicles, demonstrating significant potential for improving traffic safety. LLM-based approaches leverage strong
reasoning capabilities of LLMs to enable effective inter-vehicle negotiation and improve cooperative driving performance. However, driving decisions in complex traffic scenarios are inherently multi-solution in nature. As a result, existing negotiation-based methods often converge prematurely to suboptimal solutions, hindering consensus formation and limiting the practical deployment of cooperative autonomous driving systems. To address this challenge, we propose CoLMIN, the LLM-based multi-decision path negotiation framework for cooperative autonomous driving, achieving stable decision consensus through multi-decision path negotiation and reflective
reasoning. To achieve stable and high-quality consensus in cooperative autonomous driving, CoLMIN consists of three key components: (i) an LLM-based Multi-Intent Negotiation module (LMin), which adopts a Negotiator-Evaluator paradigm and generates multiple candidate driving intentions for joint evaluation; (ii) an Evaluation-based Shallow Reflection Module (ESRM), which analyzes negotiation outcomes and provides feedback to guide subsequent negotiations, thereby accelerating consensus formation; and (iii) an LLM-based Deep Reflection Module (LDRM), which performs long-term reflection over negotiation histories to mitigate cognitive fixation and prevent the system from converging to suboptimal solutions. Experimental results in the CARLA simulation environment demonstrate that CoLMIN significantly outperforms existing methods in challenging interactive driving scenarios.
Read the full abstract ▾
All authors Zhe Huang, Zhaoxin Fan, Shuo Wang, Wenjun Wu, Xuan Zhao, Min Liu
Computer Vision
Linguistic Trajectory Encoding for Efficient Long-Horizon Spatial Memory in Embodied Agents
Tianyidan Xie, Shenyi Wang, Qiang Tang et al. (+7 more)
Embodied agents performing long-horizon tasks require a
memory representation in which the state transitions of dynamic objects remain queryable in natural
language across hours-to-days observation horizons. Existing systems either drop fine-grained motion (clip-level
video-
language embeddings), keep it only as raw coordinates (
geometric SLAM), or organise it around immediate task context (
agent working
memories). None of them gives the
agent a per-object timeline whose state transitions are themselves queryable in
language. Our key contribution is Linguistic Trajectory Encoding (LTE), which compresses dynamic object motion histories via a hybrid representation combining natural
language descriptions, sparse spatial anchors, and visual anchors. LTE adapts compression to motion complexity by anchoring periods without reliable observations to the last seen location, while representing motion with
geometric waypoints and linguistic descriptions to preserve accuracy. To evaluate these capabilities across extended time horizons, we construct the Spatial
Memory Benchmark (SMB) from EgoLife multi-day recordings, targeting capabilities absent in existing
benchmarks: semantic trajectory
retrieval and long-horizon object
retrieval. On SMB, the LTE-based system achieves $45.3\%$ success in semantic trajectory
retrieval and $48.7\%$ in long-horizon object
retrieval, outperforming structured-
memory and VLM baselines (best prior: $31.9\%$ and $34.4\%$). LTE achieves trajectory compression by factors of $8.7\times$ to $26.1\times$ with sub-second query latency on $24$\,h
video. On Ego4D natural-
language queries, the system reaches $28.75\%$ / $55.10\%$ R@1/R@5, $+15.80$ / $+31.30$ pts over EgoVLPv2.
Read the full abstract ▾
All authors Tianyidan Xie, Shenyi Wang, Qiang Tang, Mingjie Wang, Zhicheng Qiu, Xuanfu Li, Zhan Xu, Jian Yang, Lanjun Wang, Zili Yi
Computer Vision
Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval
Hyun Seok Seong, Woojin Jun, SuBeen Lee et al. (+1 more)
Partially Relevant
Video Retrieval (PRVR) aims to retrieve untrimmed
videos that contain moments relevant to a text query. Since the target moment occupies only a portion of the
video, PRVR requires
retrieval based on fine-grained understanding beyond coarse
video-level matching. However, existing methods often rely on frozen CLIP frame features, which lack temporal understanding. Even with recent progress in parameter-efficient CLIP adaptation,
video-level predictions can still be supported by imprecise frame-level evidence. In this paper, we propose an Intrinsic Temporal Adaptation (ITA) framework for PRVR. First, our Backbone-Internal Temporal Adaptation allows the last few visual
transformer layers to attend over groups of neighboring frames. This provides temporally aware frame
embeddings while keeping CLIP frozen and training only adaptation parameters. Second, we introduce Affinity-Weighted Gradient Propagation to address the weakly supervised nature of PRVR, softly aggregating top-$k$ frames based on text-frame affinities and propagating learning signals to multiple query-relevant frames. Our method achieves state-of-the-art performance on PRVR
benchmarks, demonstrates robust cross-
dataset transfer, and retrieves substantially more accurate frame-level evidence within ground-truth query-relevant moments. Our code is available at github.com/hynnsk/ITA.
Read the full abstract ▾
All authors Hyun Seok Seong, Woojin Jun, SuBeen Lee, Jae-Pil Heo
Computer Vision
An Attention-Guided Global and Local Fusion Framework for Lesion-Focused Image Classification
Mst Shafia Tasnima, Md Samaun Elaheea, Tanjim Taharat Aurpab et al. (+1 more)
Lesion-focused image classification presents a core analytical challenge, as discriminative signals are often sparse, spatially dispersed, and easily obscured by background noise, while conventional convolutional neural networks (CNNs) process entire images uniformly and may dilute signal relevance. This study hypothesizes that adaptive fusion of global contextual information and lesion-focused local information can improve classification performance compared with using either representation independently. We propose a three-branch,
attention-guided deep learning framework built on Densely Connected Convolutional Network-121 (DenseNet-121) to improve feature attribution, interpretability, and classification reliability. The architecture consists of a global branch that learns representations from full images, followed by Gradient-weighted Class Activation Mapping (Grad-CAM) to generate
attention maps that highlight prediction-relevant regions and produce masked inputs, and a local branch enhanced with a Convolutional Block
Attention Module (CBAM) to extract refined spatial and channel-wise features from these focused regions. An adaptive fusion branch integrates global and local representations by learning instance-specific weights, allowing dynamic prioritization between contextual and localized information. The framework is evaluated on a synthetic Spot Pattern
Dataset (SSPD) and three
benchmark datasets, including skin lesion, guava leaf, and grape leaf image
datasets, where the fusion branch outperformed the individual global and local branches, reaching 97.75% accuracy on the skin lesion
dataset and 99.64% on the guava leaf
dataset. The results highlight the value of
attention-guided architectures in healthcare analytics by improving model transparency, strengthening feature relevance, and supporting more reliable data-driven decision-making in medical image analysis.
Read the full abstract ▾
All authors Mst Shafia Tasnima, Md Samaun Elaheea, Tanjim Taharat Aurpab, Md Musfique Anwar
Computer Vision
CLON: Cue-Calibrated Linguistic Object Onboarding for Zero-Shot 6D Pose Front-Ends
Seojin Ji, Yoojin Kwon and Hyung-Sin Kim
Novel-object 6D pose pipelines depend on front-end hypotheses that retain target instances under clutter and occlusion without overwhelming downstream pose estimation. We introduce Cue-Calibrated Linguistic Object Onboarding (CLON), a training-free proposal-and-matching front-end. CLON converts multi-view CAD renderings into compact linguistic
memories that provide object-specific conditioning to a frozen proposal generator. Constructed offline and reused across scenes, these
memories augment generic
prompts before
segmentation; subsequent template verification assigns object identity. As a complementary onboarding step, CLON derives fixed cue weights from template statistics for semantic, appearance, and
geometric verification. On seven BOP-Classic-Core
datasets, CLON improves detection AP by 8.2 percentage points (pp),
segmentation AP by 6.2 pp, and downstream pose AR by up to 4.1 pp over the evaluated CNOS and SAM-6D front-ends. In a real-world localization-to-grasp study under clutter and occlusion with 100 trials per method, CLON's improved localization doubles grasp success relative to the strongest evaluated baseline. Controlled ablations evaluate linguistic conditioning, multi-view construction, and verification. Analysis of a smaller candidate set with higher class-agnostic recall on five
datasets.
Read the full abstract ▾
Computer Vision
CoMLP: Cooperatively-Gated MLPs for Fine-Grained Cross-Modal Information Fusion in Medical Image Segmentation
Mingyuan Meng, Shuchang Ye, Mingjian Li et al. (+3 more)
Multi-modal medical images and clinical reports provide complementary anatomical, functional, and semantic information for medical image
segmentation. Effectively exploiting these heterogeneous sources requires fine-grained cross-modal information fusion that preserves subtle spatial details while capturing semantic dependencies across modalities. Existing fusion approaches frequently rely on cross-
attention, whose computational burden increases rapidly with spatial resolution, making dense cross-modal interaction difficult on high-resolution feature maps, particularly for volumetric medical images. In this work, we propose CoMLP, a cooperatively-gated MLP module for fine-grained cross-modal information fusion in medical image
segmentation. CoMLP models cross-modal dependencies through cooperative cross-gating, built upon complementary regional and dilated MLP interactions, to capture local and global cross-modal dependencies. We further develop a multi-source fusion architecture in which CoMLP performs both inter-image fusion across imaging modalities and
vision-
language fusion between visual features and textual reports, enabling heterogeneous information to be integrated without relying on dense cross-
attention. Extensive experiments on five medical
segmentation benchmarks, covering 2D/3D images, clinical reports, multiple imaging modalities, and diverse anatomical regions, demonstrate consistent improvements over state-of-the-art multi-modal and
language-guided
segmentation methods. Ablation studies further show that fine-grained interaction at high spatial resolutions and complementary local-global fusion are critical to the performance gains. These results demonstrate the potential of MLP-based interaction as an effective alternative for fine-grained cross-modal information fusion in medical image
segmentation.
Read the full abstract ▾
All authors Mingyuan Meng, Shuchang Ye, Mingjian Li, Zhenyu Zhao, Jinman Kim, Lei Bi
Computer Vision
LUMIN: Lifecycle-Utility-aware and Metadata-Informed Network for Visual Anomaly Detection
Pengfei Yang
Industrial anomaly detection faces two engineering bottlenecks:
memory bank construction latency and inference efficiency. Traditional sampling
algorithms (Farthest Point Sampling, K-Means, etc.) rely on numerous backbone forward passes and iterative distance computations, with construction times ranging from minutes to hours; heavy computation components such as multi-scale feature extraction struggle to meet the millisecond-level real-time requirements of production lines. This paper focuses on sampling efficiency and inference optimization for industrial deployment with two core contributions: (1) PSP (Plugin Sampler Pipeline)—a four-stage adaptive
memory bank sampling pipeline based on 18-dimensional pixel metadata and five complementary visual plugins. PSP completes all sampling with zero backbone forward passes; coarse filtering is sub-second numerical sorting, and metadata extraction is a one-time offline cost. PSP supports progressive deployment and incremental updates. (2) Two engineering optimization strategies—parallel
memory bank similarity computation (reducing inference
memory and latency by over 95%) and stratified pixel sampling for large-scale evaluation (reducing computation time by 20$\times$ while keeping metrics stable). As a vehicle for validation, we introduce LUMIN (Lightweight Universal Manufacturing Inspection Network) with extreme
segmentation-head compression, systematically exploring the accuracy-efficiency frontier against strong baselines. Experiments on five
benchmarks demonstrate that PSP matches state-of-the-art sampling accuracy at near-random construction cost (341$\times$ faster than FPS), while inference optimizations reduce evaluation time by 20$\times$ with negligible accuracy loss.
Read the full abstract ▾
Computer Vision
SeamFlow: Structure-Aware Flow Matching on Edge Probabilities for Artist-Like UV Unwrapping
Yuming Zhao, Zangyueyang Xian, Qijian Zhang et al. (+4 more)
3D surface cutting and UV unwrapping are fundamental problems in computer graphics. Traditional
geometric optimization methods mainly focus on reducing parameterization distortion, but they often overlook visual semantic coherence in seam layouts. Recent autoregressive generative methods improve semantic coherence, yet limited perception of mesh
topology often causes inaccurate local cuts. To address these limitations, we introduce SeamFlow, a novel generative framework for 3D surface cutting. We reformulate the discrete mesh-cutting problem as continuous flow matching in a high-dimensional edge-probability space. Through continuous relaxation, SeamFlow learns a deterministic mapping from a Gaussian prior to a target seam-probability distribution. An
evolution network couples local topological
tokens with global shape priors and guides smooth probability flow through Ordinary Differential Equation solving. Compared with existing autoregressive generative frameworks, SeamFlow improves
topology awareness through edge tokenization while eliminating both 3D spatial projection errors and artificial sequential-order bias. Extensive experiments demonstrate that SeamFlow achieves exceptional semantic coherence and remarkably low parameterization distortion. The project page is https://meshy-dev.github.io/seamflow.
Read the full abstract ▾
All authors Yuming Zhao, Zangyueyang Xian, Qijian Zhang, Rendong Liang, Qin Jia, Ying He, Junhui Hou
Computer Vision · Cross-listed from physics.med-ph
BEAM3R: Beam's-eye-view architecture with Mamba-3 for implicit dose reconstruction
Chen Cheng, Michael Ferraro, James Grover et al. (+2 more)
To enable accurate and rapid
photon control point and
proton beamlet dose calculation in the DoseRAD2026 challenge, we present BEAM3R, a dose estimation framework operating in beam's-eye-view (BEV). Our core innovation combines a Mamba-3 state-space depth-sequence core with physics-based transport conditioning to model long-range depth transport without expensive 3D convolutions. BEAM3R shares a 2D CNN encoder-decoder architecture for
photon and
proton dose tasks, processing per-plane BEV slices.
Proton beamlets are conditioned on water equivalent thickness and remaining range, encoding the parameters determining Bragg peak position.
Photon models use a bidirectional Mamba-3 core to capture dose contributions from materials downstream of the calculation point, while the
proton model uses a forward core with learned energy-prefix
tokens and a Bragg-peak refinement module. To reduce interpolation artifacts and support high spatial resolution, we introduce axial grid
alignment of BEV
lattices with CT slices and an implicit super-resolution representation via sub-pixel phase packing, evaluated by a differentiable Triton-accelerated resampler that reconstructs packed cubic B-spline coefficients directly in CT space. For MRI-based tasks, synthetic CTs (sCT) are generated by a patch-based conditional GAN with a SwinUNETR backbone. On the preliminary DoseRAD2026 test set, CT-to-
photon and CT-to-
proton models achieved 1%/1 mm local gamma pass rates of 96.8% and 96.0%, with stratified plan-level MAEs of 0.0041 and 0.0079. Substituting sCT reduced gamma pass rates to 89.7% for
photon and 75.4%
proton plan level doses, with stratified plan-level MAEs of 0.0093 and 0.0336. Standardised runtimes were 23.4 s and 18.4 s for CT-to-
photon and CT-to-
proton prediction, increasing to 39.7 s and 42.8 s for the corresponding MRI-based pipelines.
Read the full abstract ▾
All authors Chen Cheng, Michael Ferraro, James Grover, David E J Waddington, Emily Hewson
Computer Vision
Where to Look Matters: Learning Influential Views for VLM-based 3D Visual Grounding
Tsung-Chih Chiang, Hsuan-Kung Yang, Jou-Min Liu et al. (+4 more)
Recent zero-shot 3D visual grounding methods leverage
vision-
language models (VLMs) to localize objects in 3D scenes from natural
language queries. However, these methods typically rely on heuristic rules to select which camera views are provided to the VLM, often prioritizing object visibility rather than grounding relevance. We present IVSGround, a framework that learns Influential View Selection for VLM-based 3D visual grounding. Instead of using fixed heuristics, a lightweight view selector is trained to identify views that provide discriminative evidence for grounding. To obtain supervision signals, we generate training signals using feedback from a
reasoning VLM through a two-stage rejection sampling process. During inference, the learned selector predicts query-conditioned influential views for each candidate object, which are then evaluated by a frozen
reasoning VLM through comparative grounding. Experiments on ScanRefer and NR3D show that IVSGround consistently improves grounding accuracy over existing zero-shot pipelines, demonstrating that selecting where to look is crucial for effective 3D visual grounding. Project page: https://ivsground.github.io/
Read the full abstract ▾
All authors Tsung-Chih Chiang, Hsuan-Kung Yang, Jou-Min Liu, Ting-Ru Liu, Chun-Wei Huang, Quan Kong, Chun-Yi Lee
Computer Vision
Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation
Siyuan Liu, Xuze Zhang, Yongshun Wang et al. (+3 more)
Synthetic Aperture Radar (SAR) images have all-weather, day-and-night observation capabilities. However, compared with optical images, their speckle noise and non-intuitive scattering mechanism limit the interpretability of the images. Generative models for SAR-to-optical (S2O) conversion can improve visual interpretability, but existing methods often ignore the constraints on semantic structure, which are necessary for downstream tasks, for the sake of visual effects. We propose a unified collaborative dual-task learning framework, termed BMT (Bridging Modalities and Tasks), that jointly optimizes S2O image translation and semantic
segmentation through a shared hierarchical
Vision Transformer. The framework integrates: (1) a LocalViTBlock that fuses global self-
attention with spatial depthwise convolution through a learnable gating mechanism; (2) an enhanced output module combining multi-scale refinement processing, color correction and anti-aliasing, which calibrates channel-level color statistics through feature fusion; (3) a ControlNet-style conditional injection mechanism that encodes SAR wavelet features and
segmentation labels into a multi-scale feature pyramid and injects them at each encoder layer through zero-initialized convolution; (4) a bounded Kendall uncertainty weighting scheme that prevents either task from dominating the shared representation. We evaluate the framework under both paired and unpaired translation settings, on the public WHU-OPT-SAR paired
dataset and a self-constructed unpaired ship
dataset built from HRSID and DIOR, respectively. The experimental results show that the proposed method achieves competitive S2O translation quality and semantic
segmentation performance. The
dataset and source code have been publicly released at https://github.com/Lewisyuaner/BMT-S2O-main.
Read the full abstract ▾
All authors Siyuan Liu, Xuze Zhang, Yongshun Wang, Licong Pan, Hang Liu, Huihui Li
Computer Vision
HiSfM: Disambiguating Structure-from-Motion via Scaffold-Anchored Hierarchical Reconstruction
Ziding Zhao, Hainan Cui, Peilin Tao et al. (+1 more)
Structure-from-Motion (SfM) is a fundamental tool for sparse 3D reconstruction with broad impact in robotics and
vision, supporting mapping, localization, and large-scale scene modeling. However, conventional pipelines often fail under hard visual ambiguity caused by repeated or symmetric structures, and incur heavy computational cost due to redundant cameras and constraints. We present HiSfM, a hierarchical coarse-to-fine SfM framework that improves robustness and efficiency through scaffold construction. HiSfM first forms strong local communities using geometrical induced heuristics, then connects communities with a compact yet strong skeleton by packing edge-disjoint spanning trees (EDST) while verifying skeletal edges with a two-view disambiguator. We reconstruct a stable scaffold on this verified skeleton, serving as an anchor to capture the essence of the scene, and subsequently absorb remaining images via efficient registration and triangulation for further refinements. Experiments on ambiguity-focused
benchmarks and general
datasets show that HiSfM prevents ambiguity-induced failures while substantially reducing runtime compared to previous methods, and improves completeness over aggressive sparsification methods. Code is available at https://github.com/3dv-casia/HiSfM.
Read the full abstract ▾
All authors Ziding Zhao, Hainan Cui, Peilin Tao, Shuhan Shen
Computer Vision
Counting Beyond Instances: A Benchmark for Group-Individual Object Counting
Rui Wang, Junyi Huang, Jiahui Li et al. (+4 more)
Visual counting is commonly formulated at the instance level, aiming to estimate how many objects of a queried category appear in an image. However, real-world counting often involves higher-level semantic units formed by multiple instances, such as a bunch of grapes, a stack of plates, or a pair of shoes. This exposes a key limitation of existing counting formulations, which mainly focus on what to count, while largely overlooking at which semantic unit to count. We introduce Group-Individual Object Counting (GIC), a new setting that requires models to count both individual objects and semantic groups within a unified framework. To support this new task, we present BunchCount, a real-world
benchmark with 1,330 images, 89,254 individual annotations, and 11,065 group annotations. BunchCount provides paired individual-group annotations within the same image and explicitly records containment relations between each group and its constituent individuals. Experiments on BunchCount show that current advanced counting models perform well on individual instances but fail to count semantic groups more accurately. To mitigate semantic granularity conflict, we propose a counting-unit guided relational counting framework, which exploits group-individual containment relations to regularize cross-granularity representations during training. Our method substantially improves group-level counting while better preserving individual-level counting ability, establishing a strong baseline for counting beyond instances.
Read the full abstract ▾
All authors Rui Wang, Junyi Huang, Jiahui Li, Qiao Yu, Yixue Hao, Long Hu, Baoru Huang
Computer Vision
AngelFingerprint: A Traceable, Explainable, and White-Box Stealthy Watermark for Text-Guided Image Editing
Bo-Han Kung, Futa Waseda, Ching-Chun Chang et al. (+2 more)
Text-guided
diffusion editing raises disinformation concerns, making reliable image provenance essential. While watermarks are commonly used for this purpose, most methods carry a fixed ID that cannot explain what was changed and which
prompt produced it. Furthermore, under open-source white-box access, attackers can easily locate and remove watermarks added as separate modules. Targeting this setting, we propose AngelFingerprint, a novel watermarking framework ensuring edit traceability, explainability, and white-box stealthiness. It integrates a LoRA into the
diffusion model to embed the editing
prompt's CLIP text
embedding directly into the model's weights. An extractor then recovers this
embedding from the image pixels alone. This semantic payload explains the edit, while the weight-integrated design makes it hard to detect and isolate even under full white-box access. Two techniques make this possible: a velocity-
alignment anchor that preserves edit quality, and a specially designed frequency filter that keeps the watermark imperceptible yet recoverable and robust. On the MagicBrush
dataset, our extractor achieves $86\%$ top-1 accuracy in a 200-way
prompt retrieval, versus $20\%$ for
prompt inversion.
Read the full abstract ▾
All authors Bo-Han Kung, Futa Waseda, Ching-Chun Chang, Isao Echizen, Shang-Tse Chen
Computer Vision · Cross-listed from cs.AR
Sustainable Edge Vision via Empirically Calibrated DVFS: Eliminating Thermal Throttling on Passively Cooled Hardware
Aayush Marasini and Zhaoxian Zhou
Passive cooling eliminates the energy overhead and mechanical failure modes of fans, making it attractive for edge deployment, yet sustained Deep Neural Network (DNN) inference on passively cooled edge Systems-on-Chip (SoCs) is bottlenecked by thermal throttling. To address this, we propose an empirically calibrated, state-aware Dynamic Voltage and Frequency Scaling (DVFS) scheduler. Unlike heuristic-driven controllers, our methodology utilizes time-domain guards and absolute temperature bounds, with derivative triggers acting as safeguards against sharp thermal spikes. Evaluated on a passively cooled Raspberry Pi 5 running YOLOv8n, our scheduler eliminates all observed thermal throttling events during sustained 30-minute workloads. It outperforms a temperature-only reactive baseline by achieving a 6.8% higher frame rate (Cohen's d = 8.73) while consuming 1.9% less energy per frame. Furthermore, our optimized passive scheduling surpasses an actively cooled reference system in energy efficiency (Joules/frame), though active cooling remains superior for raw throughput. Through isolated ablations, we show that the dwell guard is necessary for run-to-run reproducibility. Finally, exploratory boundary probes indicate that the passive operating envelope closes at ambient temperatures ($\ge 27^\circ$C) where nonlinear leakage defeats DVFS-based control. These results indicate that, within the mapped envelope, correct scheduling can make mechanical cooling unnecessary for sustained edge inference on this platform.
Read the full abstract ▾
Computer Vision
LookThere! Sparse Vision by Reinforced Selection
Sreehari Rammohan, Yousef Yassin, Anthony Fuller et al. (+3 more)
Vision transformers typically treat every image
token as equally important, yet for most tasks in computer
vision only a fraction are needed. Adaptive computation methods accelerate inference by choosing which
tokens to process, but existing methods struggle at extreme sparsity and require heuristics that may not generalize like
token diversity and
attention scores. We address these limitations with LookThere, achieving a new pareto frontier in performance-compute trade-offs through an end-to-end reinforcement learning framework that jointly trains a shallow input selector and a deep representation extractor. The selector learns where to look and the extractor learns what to see, together saving computation by selecting only what is worth processing for a given task without relying on auxiliary signals. We show that LookThere only selects the task-specific input, excelling at sparse recognition in high-resolution settings (traffic signs, billiards), and maintaining accuracy with as little as 0.2% of the input. It generalizes across tasks and models, including global recognition (ImageNet classification), local recognition (ADE20K
segmentation), zero-shot classification (by
distillation), and regression (counting). Across all settings, LookThere surpasses state-of-the-art selection to provide a general and scalable framework for specialized and efficient adaptive computation.
Read the full abstract ▾
All authors Sreehari Rammohan, Yousef Yassin, Anthony Fuller, Junfeng Wen, Carl Vondrick, Evan Shelhamer
Computer Vision
Enhancing Multimodal Emotion Recognition via Multi-Feature Encoding and Attention-Based Fusion
Xu Lin, Ke Wang, Hui Kang et al. (+1 more)
Multimodal emotion recognition has attracted growing interest due to its importance in human-computer interaction, remote education, and healthcare. This paper proposes a novel
multimodal emotion recognition framework that integrates rich audio and visual feature extraction with an
attention-based fusion strategy. For audio, we extract three complementary feature types: semantic
embeddings from Wav2Vec2, MFCC features, and statistical acoustic descriptors such as pitch, energy, and rhythm. These are aligned and fused via a BiLSTM to capture temporal dependencies. For
video, we propose a ResNet50-BiLSTM architecture that combines deep residual learning and sequential modeling to extract expressive spatiotemporal features from facial sequences. To enhance
multimodal synergy, we introduce a feature-level fusion mechanism based on multi-head
attention, allowing the model to adaptively weigh contributions across modalities. Experiments conducted on the MELD and IEMOCAP
datasets demonstrate that our model significantly outperforms baselines in both accuracy and robustness. Furthermore, ablation studies show that the
attention-based fusion strategy significantly improves performance in unbalanced data settings. Our findings suggest that the proposed framework effectively captures diverse emotional cues from
speech and visual expressions, and offers a practical and generalizable approach for real-world
multimodal emotion recognition tasks.
Read the full abstract ▾
All authors Xu Lin, Ke Wang, Hui Kang, Xinying Wang
Computer Vision
Retinal OCTA Phenotyping with LLM Reporting for Alzheimer's Disease
Progga Paromita Dutta, Jeba Maliha and Md Rafiul Kabir
Early identification of Alzheimer's disease (AD) remains challenging because established assessment methods can be costly, resource-intensive, or unsuitable for population-scale screening. Optical coherence
tomography angiography (OCTA) provides non-invasive visualization of retinal microvasculature, but existing approaches often require diagnostic labels and provide limited measurement-level interpretation. We present an explainable OCTA pipeline that integrates annotation-aware vessel
segmentation, layer-specific vascular biomarker extraction, label-free phenotyping, and measurement-grounded LLM reporting. Using 117 ROSE-1 images from 39 subjects, we apply annotation-matched
segmentation models to superficial vascular complex (SVC), deep vascular complex (DVC), and combined SVC+DVC representations. The models achieve ROC-AUC values of 0.916-0.970 and Dice scores of 0.695-0.781. Six density and fractal-dimension biomarkers form subject-level profiles for exploratory clustering. Analysis of nine held-out subjects identifies an internally consistent lower-density, lower-fractal-dimension phenotype, although the absence of diagnostic labels prevents clinical interpretation. Reports generated using GPT, Gemini, and Llama are evaluated for measurement grounding, citation faithfulness, and diagnostic caution. Overall, the framework provides a transparent, non-diagnostic connection between retinal vascular measurements, exploratory phenotyping, and evidence-linked interpretation for Alzheimer's research.
Read the full abstract ▾
Computer Vision
ReaDiT Guidance: Control for Image and Video Generation using Diffusion Transformer Features
Jay Mahajan, Chang Liu, Rauf Makharov et al. (+3 more)
We present DiT Readout (ReaDiT) Guidance, a lightweight framework for controlling generation with
Diffusion Transformer (DiT) models via their internal feature representations. ReaDiT Guidance uses features from a single DiT block to steer the generative process according to spatial targets - like depth, pose, or edge maps - provided at test time. Furthermore, since modern text-to-
video models are largely built on DiT backbones, ReaDiT Guidance naturally extends to
video generation, enabling camera and motion control. Experimental results demonstrate that our approach achieves competitive or improved results compared to existing feature-based and off-the-shelf adapter-based approaches while requiring fewer parameters.
Read the full abstract ▾
All authors Jay Mahajan, Chang Liu, Rauf Makharov, Viraj Shah, Alexander Schwing, Svetlana Lazebnik
Computer Vision
Denis Zavadski, Sebastian Heid, Damjan Kalšan et al. (+2 more)
Diffusion Transformers (DiTs) have emerged as a dominant architecture for high-quality text-to-image generation, yet their scale poses challenges for efficient deployment. While truncated singular value decomposition (SVD) is a principled tool for parameter reduction, evidence from large
language models (LLMs) suggests that naive low-rank approximation can cause catastrophic failure. In contrast, we find that truncated SVD in DiTs produces smooth degradation even under substantial global compression, with redundancy distributed across projection
matrices throughout the whole network rather than concentrated in a few
transformer blocks. Building on these insights, we introduce SVDtrunc, a two-step block-level compression scheme, first allocating ranks across blocks and compressing the least important ones via truncated SVD under a global parameter budget, and then fine-tuning all blocks with modular knowledge
distillation and a rectified-flow objective. We apply SVDtrunc to FLUX.dev across compression levels ranging from 40-90% of the original parameter count. Across three
benchmarks, GenEval, HPSv2, and DPG, we outperform all competing approaches. Notably, and in contrast to prior work, we retain near-full performance at 68% and remain competitive even at 57% of the original parameter budget. Furthermore, we show that SVDtrunc complements step
distillation and achieves strong results even without fine-tuning, positioning it as a practical continuation of efficiency improvements beyond
diffusion step reduction for large-scale generative models. Project page: https://vislearn.github.io/SVDtrunc/
Read the full abstract ▾
All authors Denis Zavadski, Sebastian Heid, Damjan Kalšan, Stefan Roth, Carsten Rother
Computer Vision · Cross-listed from cs.IR
Jiarui Jin and Anyang Ji
Multimodal Vision-
Language Models (VLMs) have demonstrated remarkable capabilities in cross-modal understanding, yet a fundamental challenge persists when applying them to recommendation: as representations propagate through multi-step
reasoning, both visual and textual signals progressively attenuate - a phenomenon we term cross-modal dilution. To address this, we propose LARK (Latent-Aligned
Reasoning frameworK), a two-stage latent
reasoning framework with complementary
alignment mechanisms within a single VLM. In the first stage, learnable latent
tokens are interleaved with multi-step chain-of-thought (CoT)
reasoning and explicitly aligned with a frozen
vision encoder, serving as visual checkpoints that preserve perceptual details throughout the
reasoning chain. In the second stage, the latent representations are projected via a bridge MLP and trained with item-to-item contrastive learning; to prevent the
reasoning semantics from fading, intermediate features are aligned with the CoT hidden states from the first stage, anchoring the final
embeddings to the model's own
reasoning output. Experiments on three public
benchmarks and one industrial
dataset show that LARK achieves state-of-the-art performance across multiple recommendation architectures, with controlled ablations confirming the distinct contribution of each component.
Read the full abstract ▾