Computer Vision
DINO Eats Geometry: Self-Supervised Sign Representations that Absorb Articulated Hand Geometry
Junyi Hu, Zhewen He, Haomian Huang et al. (+4 more)
Handshape and orientation are
geometric components of sign
language, yet many sign encoders learn from appearance alone. We introduce SignDEG (DINO Eats Geometry), a temporal sign encoder that uses articulated hand geometry as a pre-training target and requires only
video at inference. Frozen DINOv3 features of tracked hands and faces feed per-stream temporal
Transformers.
Geometric latent prediction (GLP) supervises an intermediate block with WiLoR latents, MANO articulation, and 3D joints, including at masked frames.
Geometric relation transfer (GRT) matches cross-
video frame-similarity distributions at the final block. Both objectives are learned jointly with temporal self-
distillation; WiLoR and the geometry heads are then discarded. Under the SHuBERT public-data evaluation protocol, SignDEG reaches 17.9/24.5 BLEU-4 on How2Sign/OpenASL, compared with 16.2/23.2 for SHuBERT. Rank-1 adaptation improves ASL Citizen and Sem-Lex recognition by 0.02 Recall@1 each, and phonological classification improves across 16 attributes. Removing GRT while retaining GLP reduces How2Sign BLEU-4 by 1.4 and ASL Citizen Recall@1 by 0.034. The results support transferring articulated structure into an appearance encoder without running a hand reconstruction model at deployment.
Read the full abstract ▾
All authors Junyi Hu, Zhewen He, Haomian Huang, Zhenhua Li, Zhifei Li, Shuaihang Yuan, Yi Fang
Computer Vision
Object-Aware Background-Controlled Editing via Weighted Velocity Guidance
Wuji Wang, Yue Wu, Chenhao Yi et al. (+1 more)
Training-free image editing steers
diffusion or flow-matching generative models at inference time by modifying
prompt-conditioned denoising velocities. Existing velocity-based editors often apply
prompt-induced residuals globally over the latent space and rely on the model to localize semantic changes implicitly. For object-centric edits, these residuals are rarely zero outside the target object, so small non-target components can accumulate during multi-step integration, causing background drift and unstable object boundaries. We propose Object-Aware Velocity Control (OAVC), a training-free framework that introduces object-level control into the velocity-integration process. OAVC decouples where semantic residuals are allowed to act from how they are injected into the dynamics. It constructs a background-anchored reference interface under the source
prompt and then performs object-localized safe semantic injection under the target
prompt. A constrained injection operator suppresses drift-inducing velocity components, while time-adaptive spatial weighting stabilizes the transition near object boundaries. OAVC requires no training or modification of pretrained model parameters. Experiments on object-centric image and
video benchmarks with image and
video rectified-flow backbones show improved background preservation, structural fidelity, boundary stability, and temporal consistency while retaining effective localized editability.
Read the full abstract ▾
All authors Wuji Wang, Yue Wu, Chenhao Yi, Shuhui Wang
Computer Vision · Cross-listed from cs.RO
IM-ENGINE: Image Editing for Embodied Data Generation
Yian Wang, Junyi Cao, Xiaowen Qiu et al. (+1 more)
Learning-based manipulation requires supervision that is both semantically meaningful and physically executable, but current data pipelines often provide only one of these properties. Human demonstrations capture intent but are costly to collect and constrained by the human-
robot embodiment gap, while simulation can scale data generation but often under-specifies functional behavior. We present IM-ENGINE, a simulator-grounded pipeline that uses image editing as an intermediate representation for
embodied data generation. Given a rendered scene with known geometry, depth,
segmentation, and camera parameters, IM-ENGINE edits the image to inject task-relevant semantics, recovers explicit 3D state using simulator priors and an unchanged anchor object, refines the state in physics, and converts it into
robot-executable supervision. We instantiate the pipeline for dexterous grasp synthesis and goal-state generation. For grasping, IM-ENGINE generates a human grasp in image space, recovers the hand-object interaction, retargets it to a
robot hand, and refines it into physically validated
robot grasps. For goal generation, it edits a rendered scene into a desired outcome, recovers the target-object pose, and refines it into physically valid, semantically meaningful goals and trajectories. This combination of generative semantic priors and simulator grounding enables scalable task-relevant supervision for
robot learning.
Read the full abstract ▾
All authors Yian Wang, Junyi Cao, Xiaowen Qiu, Chuang Gan
Computer Vision
A Comparative Study of GAN-Based Deep Learning Models for Pneumonia Detection in Chest X-Rays
Roshan Paudel, Aashish Ghimire and Pramod Acharya
This study evaluates pneumonia classification in chest X-rays using VGG19, MobileNetV2, ResNet50, and a custom CNN, and explores Generative
Adversarial Network (GAN)-based synthetic data augmentation. MobileNetV2 achieved the highest reported accuracy of 88% with balanced class-wise performance. The custom CNN achieved pneumonia recall of 92.67% and precision of 79.43%, highlighting a precision-recall trade-off. Accuracy, F1-score, precision, recall, confusion
matrices, and training curves were used to assess performance. Synthetic pneumonia images were combined with real images to investigate whether augmentation could improve classification performance. In the reported VGG19 comparison, augmented-data training accuracy reached approximately 100%, while validation accuracy remained near 50%, below the real-data validation accuracy. This experiment therefore did not demonstrate a validation-performance benefit from GAN augmentation. The classifier comparison highlights differences in accuracy and pneumonia recall, while the augmentation experiment indicates the need for further evaluation of synthetic-image quality and training settings.
Read the full abstract ▾
Computer Vision
Adapting Vision Foundation Models to Acoustics for Pose-Free 3D Sonar Reconstruction
Kevin Zhang, Jingxi Chen, Mohamad Qadri et al. (+5 more)
Vision foundation models trained on Internet-scale RGB
datasets enable remarkable capabilities across a range of tasks, from text-to-
video generation to few-shot 3D scene reconstruction. An acoustic foundation model trained on large-scale sonar
datasets could enable similar capabilities in the
underwater domain, where turbidity and low-visibility conditions make conventional RGB foundation models inapplicable. Unfortunately, a lack of freely available large-scale sonar
datasets makes training such a model from scratch impractical. In this work, we demonstrate that
vision foundation models can be efficiently adapted to the sonar setting by (1) exploiting the
geometric relationship between the two sensing modalities and (2) employing accurate physics-based noise models for synthetic data generation. The resulting sonar adaptation models enable new capabilities: For the first time, we experimentally demonstrate sonar-based pose-free 3D reconstruction.
Read the full abstract ▾
All authors Kevin Zhang, Jingxi Chen, Mohamad Qadri, Russell Shomberg, Michael Kaess, Jia-Bin Huang, Adithya Pediredla, Christopher Metzler
Computer Vision
One Model, Two Worlds: Bidirectional Sonar-Optical Translation
Shengji Jin, Trung Tien Dong, Ahmed Lamidi et al. (+3 more)
Translating between imaging sonar and optical cameras is valuable for
underwater perception, but supporting both directions with separate models duplicates storage and computation. A unified bidirectional model is therefore attractive, yet existing approaches largely treat the two directions symmetrically despite their fundamentally different image-formation physics. We argue that sharing a generative model does not require sharing the physics. We introduce the Direction-Asymmetric Realism Bridge (DARB), which retains a shared
diffusion-bridge trunk while routing direction-specific physical priors through asymmetric pathways: range-aware modulation for sonar-to-optical translation and polar ray-dependent processing for optical-to-sonar translation. We further show that
symmetry in training is also costly: applying a common realism schedule reduces sonar-to-optical PSNR by 2.60 dB. Our Adaptive Realism Supervision (ARS) instead determines when, where, and how strongly perceptual supervision is applied from reconstruction quality and gradient balance. Together, DARB and ARS enable one bidirectional model to match the sonar-to-optical specialist within 0.11 dB PSNR, outperform the optical-to-sonar specialist by 0.70 FID, and surpass two independently trained BBDMs on seven of eight metrics.
Read the full abstract ▾
All authors Shengji Jin, Trung Tien Dong, Ahmed Lamidi, Chen Chen, Xiaomin Lin, Yi Sheng
Computer Vision
MobileVLA-R1 2.0: RL-Enhanced Reasoning for Mobile Robot Control
Ting Huang, Yue Huang, Zeyu Zhang et al. (+2 more)
Grounding natural-
language instructions into reliable and executable actions remains a fundamental challenge for
vision-
language-action (VLA) systems on mobile
robots, due to the persistent gap between high-level semantic
reasoning and low-level locomotion and manipulation control. Existing approaches often rely on implicit
reasoning or monolithic action prediction, making it difficult to maintain coherent long-horizon decision making while producing precise and adaptable
robot actions. To address this challenge, we propose MobileVLA-R1 2.0, an RL-enhanced VLA framework that explicitly couples structured
embodied reasoning with executable mobile
robot control. The framework learns multi-granularity
reasoning over
embodied trajectories through supervised Chain-of-Thought (CoT)
alignment and reinforcement learning, improving
reasoning-to-action consistency beyond purely behavioral supervision. To support both locomotion and manipulation, we further introduce a
reasoning-conditioned action decoder that maps
multimodal reasoning representations to task-level action targets, which are subsequently translated into embodiment-specific commands by
robot controllers. This design provides a unified perception-
reasoning-action interface while decoupling high-level action generation from
robot-specific actuation. We conduct extensive evaluations on
language-guided
navigation, quadruped control, and
humanoid mobile manipulation, covering VLN-CE, QUARD, and real-world deployments on Unitree Go2 and G1
robots. MobileVLA-R1 2.0 consistently outperforms strong VLA baselines, achieving an average 1.6 point improvement in SR on VLN-CE and a 10.0 point improvement in full-task success on real-world G1 mobile manipulation tasks over MobileVLA-R1, while demonstrating robust long-horizon instruction following and closed-loop execution across different robotic platforms.
Read the full abstract ▾
All authors Ting Huang, Yue Huang, Zeyu Zhang, Shuicheng Yan, Hao Tang
Computer Vision
VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification
Yixin Wan, Tianle Zheng and Kai-Wei Chang
Multimodal Large
Language Models (MLLMs) perform strongly on general visual understanding tasks such as visual question answering, yet they often struggle with a basic comparative skill: identifying what has changed between two similar images. We introduce VDiff-Bench, a challenging multiple-choice
benchmark for fine-grained Image Difference Identification. VDiff-Bench contains 1,756 four-way questions over image pairs and covers 10 change categories: position, motion, regional image color, overall image color, appearance/disappearance, noise/resolution, texture, substitution/size, OCR/text, and illumination. Each question corresponds to two image inputs with 4 choices: the true difference, two hard negative descriptions, and a "no difference" distractor. To make the task challenging, we specifically curate ground-truth-conditioned negatives that require models to distinguish the actual change from nearby semantic alternatives. Experiments with 11 state-of-the-art open- and closed-source MLLMs show that fine-grained visual comparison remains brittle: models exhibit uneven performance across sources and change categories, with persistent failures on subtle low-level changes like noises and textures. For instance, three 7-8B-scale open-source MLLMs score 52.5-70.6% on semantic changes but only 8.7-33.3% on low-level changes like noise and texture, falsely assuming no changes between two image inputs. Surprisingly, despite strong performance of other closed-source commercial models, Grok 4.3 demonstrate remarkable performance drop on identifying noise and texture differences between images, falling significantly behind large open-source models like Kimi K2.5 and K3. Overall, VDiff-Bench provides a targeted diagnostic for evaluating comparative visual understanding in MLLMs, exposing failures that are not captured by standard single-image
vision-
language tasks.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.NI
Task-Oriented Semantic Communication for Hazard Warning and Remote Operation in Connected Vehicle Platoons
S M Sabit Bananee, Shahriar Hasan, Muhammad Mahbub Alam et al. (+1 more)
Connected and automated vehicle platoons require reliable dissemination of task-relevant hazard information to enable appropriate downstream responses, with unresolved situations additionally requiring remote human intervention. However, the limited channel bandwidth of V2X communication makes high-volume sensor data transmission challenging, motivating the exchange of only task-relevant information rather than raw sensor data. This paper proposes an integrated semantic and task-oriented communication framework for cooperative hazard response and remote operation in automated vehicle platoons. The framework combines onboard instance
segmentation and
monocular depth estimation for task-oriented hazard decision-making with Stable
Diffusion Variational Autoencoder (SD-VAE)-based semantic scene compression. Task-relevant hazard information and the compact semantic representation are jointly disseminated through a broadcast-based Emergency Semantic Message (ESM) over the existing V2X protocol stack. Evaluation demonstrates that SD-VAE consistently outperforms JPEG and JPEG 2000 on task-relevant reconstruction metrics, exhibiting graceful degradation rather than a sharp cliff effect under low-SINR conditions, demonstrating its feasibility for remote scene reconstruction when human intervention is required. Moreover, an extensive campaign of 7,290 simulation runs shows a low overall inter-vehicle collision rate of 0.99%, with collisions occurring mainly under extreme high-speed and short-gap conditions.
Read the full abstract ▾
All authors S M Sabit Bananee, Shahriar Hasan, Muhammad Mahbub Alam, Nafiul Rashid
Computer Vision
Spatial Attention Supervision for Defect Localization: Exploiting Ground-Truth Masks as Training Signal in Diffusion-Augmented Defect Detection
Sajjad Rezvani Boroujeni, Muskan Saraf, Gnana Tulasi Makineni et al. (+2 more)
Ground-truth defect masks in industrial inspection
datasets are typically reserved for evaluation. This paper repurposes them as spatial supervision signals during training of classification networks, teaching a model not just what to predict but where to look. The method adds an activation-based
attention alignment loss that steers convolutional feature maps toward defect regions, in a mixed-supervision formulation that also accommodates samples without masks, such as
diffusion-generated images. Combined with DDPM augmentation, synthetic images contribute quantity while masks contribute spatial precision. We evaluate 85 models (four CNN backbones under a 2x2 data/training factorial over five seeds, plus a Swin-V2-T
transformer baseline) on the MVTec-AD bottle
benchmark, with localization measured on held-out defect images excluded from classifier gradient updates. Main findings: (1)
attention-guided training improves activation-based localization (Pixel-AUROC) by +18.0% for EfficientNetB0 with augmentation (p=0.005, Cohen's d=2.6) and +18.7% for ResNet50 (p=0.008), significant in four of eight CNN settings (uncorrected for multiple comparisons) with no significant change in classification; (2) for EfficientNetB0 a data x training-mode interaction is significant (p=0.002), consistent with a super-additive effect (+13.6% combined vs +1.6% summed individual effects); (3) architectures with weaker spatial representations benefit most, whereas ConvNeXt-T shows no effect, apparently because its depthwise-convolution activations yield spatially uninformative channel-mean maps; (4) unsupervised PatchCore remains the strongest localizer (Pixel-AUROC=0.983), contextualizing the supervised gains. These results show that existing evaluation masks can act as practical training signals that measurably and reproducibly improve where defect classifiers attend.
Read the full abstract ▾
All authors Sajjad Rezvani Boroujeni, Muskan Saraf, Gnana Tulasi Makineni, Tom Bush, Hossein Abedi
Computer Vision
Branch-Centric Tokenization and Test-Time Augmentation for Skeleton Generation
Zhengyuan Li, Chuanyu Pan, Yuanming Hu et al. (+1 more)
Automatic skeleton generation involves predicting both joint positions and skeletal connectivity. However, existing approaches struggle to encode branch structures into
token sequences and do not use test-time computation effectively. We study these choices within a unified autoregressive framework. First, we introduce branch-centric tokenization, a branch-aware representation that places structurally related elements next to each other and encodes connectivity directly in the sequence. Compared with standard BFS-style serialization, this representation yields more compact sequences. Second, we introduce view-augmented generation, a test-time augmentation procedure that applies axis-aligned rotations to the input mesh, maps all predictions back to a common frame, and selects the final skeleton based on mesh coverage and consistency among predictions from different views. Experiments show that our method achieves better skeleton prediction accuracy than state-of-the-art methods. In particular, our method reduces the CD-J2B error by 16.9% on the Articulation-XL2.0
dataset compared to the strongest directly comparable baseline, Auto-Connect. Qualitative results on in-the-wild meshes further demonstrate generalization across diverse inputs.
Read the full abstract ▾
All authors Zhengyuan Li, Chuanyu Pan, Yuanming Hu, Raymond A. Yeh
Computer Vision
From Gaze to Meaning: A Training-Free AI Agent for Unified Grounding and Explanation
Shayan Nasiriboukani, Sara Atito, Mohammad Nezamipour et al. (+1 more)
Understanding human
attention is fundamental for scene interpretation, yet existing approaches often rely on heavily trained models that lack interpretability. Prior methods struggle to jointly reason about gaze targets, attended objects, and visual grounding without extensive supervision. To the best of our knowledge, this work introduces the first training-free Gaze Target
Agent (GTA) for gaze-guided
reasoning across tasks such as gaze target prediction,
attention localization, and object identification. This is achieved by leveraging pretrained
vision-
language models, augmenting them with visually guided
prompts, and employing a
memory-based
retrieval strategy for high-uncertainty samples to improve performance without additional training. We evaluate our approach using both quantitative metrics and qualitative results. Quantitatively, our method achieves state of the art performance on the GazeFollow and GazeHOI
benchmarks. Qualitatively, our
agent provides detailed semantic predictions, predicts the correct targets even when ground truth labels are wrong, and remains flexible without vocabulary constraints.
Read the full abstract ▾
All authors Shayan Nasiriboukani, Sara Atito, Mohammad Nezamipour, Muhammad Awais
Computer Vision
PhysWeep: Does a Video Generator Realize the Physics You Ask For?
Rasul Khanbayov and Hasan Kurban
Image-to-
video generators are often credited with absorbing physical dynamics as implicit world models, a claim the community currently checks with plausibility scores that ask whether a clip looks consistent with real-world motion. Plausibility is the wrong test on its own, because a clip can look natural while encoding the wrong value of the governing physical parameter, and no existing
benchmark measures this gap directly. PhysWeep closes it with a fixed, label-free audit, treating a frozen generator as a black box, recovering the realized parameter from generated pixels, and reporting how often generation is trackable at all, how far the realized value sits from the requested one, and which, if either, of the literature's two proposed failure mechanisms the data support. A deterministic-simulator positive control confirms every score is exactly checkable. Applied to three open generators across six sweep axes, PhysWeep finds a specific, reproducible, previously undocumented failure. Conditional on producing trackable motion, two of the three generate confident, well-fit dynamics that converge to one of a small number of fixed, wrong values selected by the sampling seed rather than by the request, reproducing across two independent model families, two physical systems, and an independent tracker. It matches neither the prior reversion nor the case-based clamping the literature anticipates, because the reversion target is seed-conditional rather than a single global default, and a leave-one-out selection rule rejects both; the in-range faithfulness slope is statistically indistinguishable from zero wherever a response is estimable at all. A
benchmark averaging over seeds would never see this: each sample is confidently locked to a wrong constant, exactly the failure a plausibility score is structurally blind to. We release the protocol, suite, and analysis code.
Read the full abstract ▾
Computer Vision
One Perturbation Is Not Enough: Identifiability and Blind Baselines for Behavioral AI Evaluation
Rasul Khanbayov, Mariam Sohail, Ahmed Abdala et al. (+1 more)
Behavioral evaluations perturb an input and read the induced change in the output in order to certify that a system uses that input. We show that the number of perturbations such a certificate requires is fixed, and that reporting a single perturbation cannot supply it. Where a response ratio is a property of the policy rather than of the test items, the behavioral record is a linear measurement of an exponent vector recording how much the output depends on each input, so perturbations identify input use exactly when their logarithms span the input space. At least $n$ are needed for $n$ inputs, an incomplete design confuses precisely the policies differing along the kernel of its design
matrix, and sharpening a perturbation never substitutes for adding an independent one. We also derive in closed form the score such a test awards a policy that reads nothing, which is far from zero and which none of the probes we survey reports. Instantiating this where the correct response is fixed by dimensional analysis, we run a complete identifying set of three perturbations on three
vision–
language models reporting a physical quantity from
video. All three score far below their own blind bound rather than above it, because each defaults to one of a small set of round calibration values that never matches what the scale asserts; none moves its relabeling response by a single exponent, and none is separable from the same model instructed to ignore the
video.
Read the full abstract ▾
All authors Rasul Khanbayov, Mariam Sohail, Ahmed Abdala, Hasan Kurban
Computer Vision
SolarBench: A global solar energy nowcasting benchmark
Yuhao Nie, Stephen Campbell, Quentin Paletta et al. (+18 more)
As the share of
solar power grows, nowcasting weather-driven
solar variability becomes critical for reliable energy system operation. State-of-the-art approaches increasingly apply deep learning to sky camera and geostationary satellite observations, but fragmented
datasets and inconsistent evaluation make it difficult to determine whether reported improvements generalize across climates, cloud regimes, and photovoltaic (PV) systems. Here we introduce SolarBench, an open global
benchmark for image-based
solar nowcasting. SolarBench harmonizes more than six million sky and satellite images from 11 diverse sites spanning a decade, together with irradiance or PV output and auxiliary atmospheric data. An accompanying toolbox supports reproducible data access, processing, model development, and evaluation. Using SolarBench, we
benchmark representative models and reveal a gap between average
forecasting accuracy and the ability to capture rapid
solar fluctuations. We further quantify predictability across cloud regimes and demonstrate data-efficient adaptation to new PV systems. SolarBench provides an extensible foundation for fair comparison and methodological innovation in
solar nowcasting.
Read the full abstract ▾
All authors Yuhao Nie, Stephen Campbell, Quentin Paletta, Liwenbo Zhang, Tao Jing, Samer Chaaraoui, Jonathan Giezendanner, Andea Scott, Tao Sun, Cong Feng, Max Aragon, Jacques Camier, Adam Jensen, Florian Kotthoff, Yuexing Yang, Yang Ming, Mengying Li, Stefanie Meilinger, Yupeng Wu, Adam Brandt, Sherrie Wang
Computer Vision
Multiple Myeloma Lesion Segmentation on Whole-Body Diffusion-Weighted Imaging via Efficient Anatomical Anticipation and Multimodal Confirmation
Mengmeng Zhang, Shengqian Huang, Junde Zhou et al. (+12 more)
Whole-body
diffusion-weighted imaging (WB-DWI) is widely used for multiple myeloma (MM) assessment, yet automated lesion
segmentation remains challenging due to limited anatomical delineation and the low specificity of marrow hyperintensity. Existing studies have introduced bone region-of-interest (ROI) information and apparent
diffusion coefficient (ADC) maps to mitigate these ambiguities, but practical limitations remain. Bone ROI construction often relies on costly manual annotation, image registration, or dedicated bone models, while ADC is usually incorporated only through simple channel fusion, limiting its ability to provide complementary structural and lesion-discriminative cues. To address these limitations, we propose a two-stage framework for MM lesion
segmentation on WB-DWI. In the first stage, we train a bone ROI generation model from ADC images without dedicated bone labels, providing an efficient and practical anatomical prior for lesion analysis. In the second stage, we propose Anatomy-guided
Multimodal U-Net (AMU-Net), which leverages ADC in a manner consistent with clinical lesion assessment rather than treating it as a generic auxiliary modality. Extensive experiments demonstrate the effectiveness and practicality of the proposed method. It achieves the best overall performance among the evaluated methods, with a mean Dice score of 76.2%.
Read the full abstract ▾
All authors Mengmeng Zhang, Shengqian Huang, Junde Zhou, Xiaoping Wu, Hao Luog, Jing Wanga, Yicheng Sun, Jiao Li, Haibo Zhang, Sheng Xie, Fan Wangg, Qin Wangc, Huadan Xue, Yisheng Lv, Fei-yue Wang
Computer Vision
TBDub: Production-Oriented Visual Dubbing
Bihan Li, Xinyang Li, Zeran Xu et al. (+2 more)
Visual dubbing must synchronize mouth motion with replacement
speech while preserving identity, appearance, and temporal consistency. Although X-Dub provides a strong mask-free
video-editing baseline, its application to livestream and generated-
video content reveals limitations in production-domain robustness, temporal and motion stability, identity and oral-detail preservation, and inference efficiency. We present TBDub, a production-oriented extension of X-Dub that combines task-adaptive post-training with task-aware few-step
distillation. Post-training adapts the
video DiT using production-domain data, production-specific conditioning and filtering, and enhanced audio features to obtain a 30-step Teacher.
Distillation adapts DMD/DMD2 to conditional
video editing and compresses the Teacher into a two-step Student. On 38 TalkVid clips, the Teacher improves all eight reported reconstruction, perceptual, identity, and synchronization metrics over X-Dub. In the MOS evaluation, it improves lip-sync consistency, identity consistency, and visual quality over X-Dub by 0.14, 0.95, and 0.90 points, while the Student achieves the highest lip-sync and visual-quality scores and remains close to the Teacher in identity consistency. In paired end-to-end generation timing from the first VAE encode through the final VAE decode on a single NVIDIA H20 GPU at $512\times512$, the Student reaches 7.13 effective FPS and reduces total latency by $13.93\times$; the DiT stage alone is accelerated by $42.49\times$. The Student largely retains the Teacher's generation quality and audiovisual synchronization. The code is available on GitHub at \https://github.com/TaoLiveAIGC/TBDub, and the 30-step Teacher and two-step Student weights are available on Hugging Face at https://huggingface.co/TaoLiveAIGC/TBDub.
Read the full abstract ▾
All authors Bihan Li, Xinyang Li, Zeran Xu, Meiguang Jin, Junfeng Ma
Computer Vision · Cross-listed from cs.HC
MM-SVGEdit: A Multimodal-Driven SVG Editing for UI Design
Shibo Yang, Yuqing Gao and Zipeng Liu
In the field of UI design, Scalable Vector Graphics (SVG) is widely used as a design medium. However, traditional SVG editing techniques have high entry barriers and require cumbersome manual iteration, while LLM-based editing solutions suffer from low accuracy and poor user controllability. To address these issues, we propose MM-SVGEdit, a
multimodal-driven SVG editing approach that integrates traditional SVG editing and LLM-based methods. We introduce a two-stage strategy in which visual grounding is followed by modification. Both stages support two interaction modalities: natural
language instructions and direct manipulation (mouse and keyboard). We trained and evaluated MM-SVGEdit on a self-constructed
dataset of 14,476 question-answer pairs generated from UIs, covering 11 types of editing operations on both single and multiple UI targets. The results show that MM-SVGEdit improves SVG editing accuracy, efficiency, and user-perceived control while reducing
token consumption and response time.
Read the full abstract ▾
Computer Vision
PASTEL: Panoramic Alignment for Monocular 4D Scene Reconstruction
Yuankun Yang, Yi Wei, Bo Bai et al. (+2 more)
Reconstructing 4D scenes from casually captured
monocular video is vital for applications in virtual reality (VR) and
embodied AI. Recent advances in 4D reconstruction and novel view synthesis have substantially propelled this capability. However, existing reconstruction methods generally cannot recover regions beyond visible camera limits. Consequently, we introduce a new paradigm that achieves 4D scene synthesis by combining visible-region reconstruction from
monocular input with invisible-region generation beyond observable camera boundaries. We present Panoramic
Alignment for Strategic Exploitation of Generative Priors (PASTEL). Specifically, PASTEL proposes panoramic scene
alignment, a novel representation that reformulates the intractable 3D "invisible region" exploration into a tractable 2D directional trajectory planning. This is achieved by reducing the viewpoint planning from 6-DoF search to a 2D directional search with explicit visibility boundaries. By operating within this panoramic space, our method strategically identifies camera trajectories that maximize exploration beyond observable boundaries while minimizing viewpoint deviation. Experimental results show that PASTEL can not only extrapolate plausible scene content beyond the observable boundaries of input
monocular videos, but also substantially boost
monocular 4D reconstruction performance. PASTEL outperforms the previous state-of-the-art method by 0.9dB in full-image PSNR on the DyCheck IPhone
dataset.
Read the full abstract ▾
All authors Yuankun Yang, Yi Wei, Bo Bai, Wenyang Zhou, Li Zhang
Computer Vision
Automatic Red Teaming for Implicit Vulnerabilities of Text-to-Image Models
Chang Ma, Junlin Han, Shuo Chen et al. (+3 more)
Red-teaming Text-to-Image (T2I) models is essential for safe deployment, yet it remains particularly challenging against implicit
adversarial prompts. Unlike explicit
adversarial prompts that can be readily identified and blocked, implicit ones are much harder to detect: the
prompts appear benign on the text surface yet still lead to inappropriate visual content. To address this, we propose
Adversarial Probing for Implicit VulnErabilities (AdvPIE), a
multimodal agentic framework to expose implicit vulnerabilities without requiring access to the parameters of target models. AdvPIE adopts a policy
agent to generate and refine implicit
adversarial prompts based on the feedback from a judge
agent. To construct informative feedback, the judge
agent provides modality-specific safety evaluation at both global and relative levels across iterations. To effectively leverage the feedback, we propose a novel Cumulative
Adversarial Decoding strategy for the policy
agent, which dynamically reweights
token distributions to favor
tokens that lead to more harmful images while preserving sampling diversity. Extensive experiments on standard and safety-aligned T2I models show that AdvPIE1 effectively uncovers implicit vulnerabilities, outperforming various baseline methods.
Read the full abstract ▾
All authors Chang Ma, Junlin Han, Shuo Chen, Runjia Li, Philip Torr, Jindong Gu
Computer Vision
Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentation
Chang Liu, Henghui Ding, Lingyi Hong et al. (+36 more)
This report summarizes the 8th Large-scale
Video Object
Segmentation (LSVOS) Challenge, held in conjunction with ECCV 2026. The challenge evaluates
video segmentation in three complementary settings: complex semi-supervised
video object
segmentation on MOSEv2, text-guided referring
video object
segmentation on MeViSv2-Text, and audio-guided referring
video object
segmentation on MeViSv2-Audio. We describe the tasks and evaluation protocols and review the methods of the top three teams in each track. Across the nine leading solutions, foundation
segmentation models are combined with target-aware
memory,
multimodal reasoning, explicit target-existence verification, agentic interaction, and corrective tracking. These systems illustrate a broader transition from single-model mask propagation toward modular pipelines that reason about object identity, query validity, and temporal reliability.
Read the full abstract ▾
All authors Chang Liu, Henghui Ding, Lingyi Hong, Ning Xu, Linjie Yang, Yuchen Fan, Canyang Wu, Jinrong Zhang, Xusheng He, Ce Bian, Xianjing Han, Jianlong Wu, Mingqi Gao, Sijie Li, Jungong Han, JeongRae Kim, Chaehyun Kim, Changwon Lim, Jungyoon Lee, Gyuil Lim, Doeon Kim, Seong-heum Kim, Pranjal Aggarwal, Sean Welleck, Yiwen Ren, Jianing Liu, Yingxin Wang, Kexin Zhang, Licheng Jiao, Lingling Li, Xu Liu, Jinxing Zhou, Suiyi Zhao, Yanghao Zhou, Ruohao Guo, Liangtao Shi, Jinxia Xie, Xiantao Hu, Ting Liu
Computer Vision
NSFlow: End-to-End Differentiable Neuro-Symbolic Optical Flow for Visual Odometry
Yicheng Lin, Zhipeng Fei, Yuxiu Xu et al. (+3 more)
Sparse optical flow provides stable inter-frame correspondence, playing a key role in Visual Odometry (VO) and Visual-Inertial Odometry (VIO). Classical optimization-based methods, such as Lucas-Kanade (LK), perform well under small displacements but are sensitive to large motions and illumination changes. Modern regression-based learning methods, while more robust in complex scenes, are often computationally heavy and lack explicit
geometric consistency, making them less suitable for efficient VO/VIO front-ends. To bridge this gap, we propose a hybrid neuro-symbolic framework that combines the strengths of both paradigms. Our method uses a Convolutional Neural Network (CNN) to extract robust feature representations, which is fed into a differentiable LK optimizer to estimate optical flow in an end-to-end trainable manner. Through implicit differentiation, gradients are propagated across the iterative solver, enabling joint optimization of feature extraction and flow estimation. The resulting system integrates seamlessly into existing VO/VIO pipelines and runs in real-time on embedded platforms. Experiments show that our method outperforms conventional optimization-based flow in challenging conditions such as dynamic lighting and low texture, while also achieving higher accuracy and lower latency than purely regression-based alternatives. When deployed in a VIO system, our method demonstrates significant performance improvement, achieving an average error reduction of 42% on challenging
datasets while enhancing tracking stability. The code is publicly available.
Read the full abstract ▾
All authors Yicheng Lin, Zhipeng Fei, Yuxiu Xu, WenDong Chen, Cong Li, Bin Han
Computer Vision
GradeTrap: Authority Cues in Images Shift VLM Judgments Despite Explicit Instructions to Ignore Them
Deep Dessai
As
vision-
language models (VLMs) become increasingly capable and are deployed in consequential real-world settings, they must evaluate evidence independently rather than defer uncritically to human authority. We introduce GradeTrap, a controlled evaluation that places two social cues in direct conflict: a student answer, which should attract sycophantic agreement, and a conflicting answer attributed to a peer, teacher, or official answer key, which should attract authority-based deference. Models produce free-form answers while being explicitly instructed to solve independently and ignore all student answers, feedback, and grading marks. We test the models on 60 synthetic real-world trade-off scenarios. Five neutral trials establish a stable model-relative preference, followed by three repetitions of six experimental cues including controls. On the 45-item common intersection across Gemini 3.5 Flash-Lite, GPT-5.6 Luna, and Claude Haiku 4.5, a generic second-answer control yields 5.4% conflicting-answer selection. Relative to that control, pooled within-item changes show no reliable peer-review effect, a 6.9-point teacher-review effect, and a 19.5-point official-key effect. In contrast, a displayed conflicting student answer alone compared to a displayed student reference answer alone only raises selection from 2.2% to 5.2%. Official-key provenance therefore redirects judgements more than a student answer or the generic second-answer control, despite an explicit ignore instruction and an opposing student answer given along with the official key. Effects vary in magnitude across the three models.
Read the full abstract ▾
Computer Vision
DriveZero: End-to-End Driving Beyond Human Demonstrations
Hao He, Chengcheng Hu, Zirun Su et al. (+17 more)
Most end-to-end autonomous-driving systems learn by imitating human driving logs, leaving their learned behavior constrained by the quality and behavioral coverage of the recorded trajectories. This report presents DriveZero, an end-to-end system that learns driving behavior beyond human demonstrations. It decomposes driving into a perception model and an action model, pretrains each in the regime best suited to it, and combines them into one end-to-end planner. The two models call for different learning recipes: perception must understand the world, and benefits from massive and diverse visual data; action must interact with it, and requires closed-loop feedback. On the action side, we introduce DriveRL, a mixed-
agent closed-loop reinforcement-learning framework. It converts real driving logs into interactive worlds, where a privileged teacher policy is trained with PPO through closed-loop rollouts. For the perception model, DriveVFM consolidates multiple frozen
vision foundation models, including DINOv3, SigLIP2, SAM and Depth Anything V2, into a single backbone from raw images alone, requiring no task-specific annotations. DriveZero then unifies the two: a camera-only planner that distills the frozen DriveRL teacher through its rolled-out trajectories. The goal-conditioned teacher can moreover be queried under augmented driving intents, yielding diverse, goal-consistent supervision that logged data cannot provide. On nuPlan, DriveRL with value-guided test-time action search achieves a mean score of 93.57 across the Val14, Test14-hard, and Test14-random community splits in both non-reactive and reactive modes, exceeding the Log-Replay expert on all three splits. DriveZero achieves state-of-the-art performance on NAVSIMv1, NAVSIMv2 and the closed-loop HUGSIM
benchmark without any human trajectory supervision.
Read the full abstract ▾
All authors Hao He, Chengcheng Hu, Zirun Su, Heng Zhang, Haisong Liu, Jinke Li, Haochen Tian, Zhenwei Shen, Hongyang Li, Zhichao Li, Yunchen Yang, Bochao Huang, Siyu Zhang, Kuangye Chen, Xiongjie Zhang, Wentao Dai, Hengchen Dai, Siyuan Liu, Zehao Huang, Naiyan Wang
Computer Vision
Image-Scale Robustness and Visual Recognition Performance: A Cross-Architecture Analysis
Anish Monsley Kirupakaran
The sensitivity of visual recognition models to changes in image scale is well established, yet the factors governing this sensitivity across heterogeneous architectures remain unclear. In this work, we investigate whether scale robustness exhibits a common quantitative structure across modern
vision models. We evaluate 20 pretrained ImageNet-1K classifiers spanning seven architectural families, including convolutional, mobile, efficient, and
Transformer-based architectures. By systematically reducing input image scale, we construct scale-accuracy response curves and define a characteristic scale as a compact measure of the onset of substantial recognition degradation. We then examine the relationship between characteristic scale and baseline recognition accuracy, model parameter count, architectural family, and representation stability. A strong inverse association is observed between baseline accuracy and characteristic scale (Pearson r = -0.890, R^2= 0.792, p < 10^-6). This relationship remains stable under bootstrap resampling, leave-one-architecture-out analysis, and leave-one-family-out analysis. In contrast, parameter count provides negligible additional explanatory power after controlling for baseline accuracy (p = 0.80), while architectural family does not provide significant incremental explanatory power. Furthermore, characteristic scale shows essentially no association with representation stability (r = -0.003, p = 0.991). These results indicate that, across the studied models, scale robustness is strongly organized by baseline recognition performance rather than simply by model size, architectural family, or representation stability. The study provides an empirical framework for characterizing scale robustness across
vision architectures and identifies a reproducible accuracy-scale regularity that warrants further theoretical investigation.
Read the full abstract ▾
Computer Vision
What Does Animal Re-Identification Learn? Linear Biological Concepts and Their Origins in Visual Representations
Robert Nolting, Alexandra Schild, Moritz Weckbecker et al. (+2 more)
Conservation increasingly relies on camera traps that collect more wildlife imagery than experts can manually analyze, making animal re-identification (Re-ID) essential for monitoring individuals and populations. Yet understanding which cues drive model decisions is challenging for ViT-based Re-ID models, whose metric-learning objectives provide no explicit supervision for biological concepts. We ask whether such models nonetheless organize their representations along biologically meaningful axes. Using a DINOv3 backbone fine-tuned for Western lowland gorilla Re-ID with triplet-margin loss, we find that sex and age emerge as linear directions that generalize to held-out individuals, reaching up to 0.91 AUROC and being recoverable from a single image per individual. Activation steering further shows that the sex direction is causally used by the model, flipping a significant fraction of predictions to the opposite sex. Comparing off-the-shelf and fine-tuned backbones shows that Re-ID training does not create these concepts, but relocates them across the network. Finally, data attribution reveals that the representation we find reflects a graded biological axis, is redundantly encoded across the population and shaped by visually ambiguous individuals. Together, these findings show how interpretability can uncover both the biological structure and failure modes of Re-ID representations, providing a step toward auditable computer
vision for wildlife monitoring.
Read the full abstract ▾
All authors Robert Nolting, Alexandra Schild, Moritz Weckbecker, Maximilian Schall, Gerard de Melo
Computer Vision
FujinSplat: Seeing Through Smoke with RAW-Domain Gaussian Splatting
Gengjia Chang, Ziteng Cui and Shuhong Liu
The appearance of a smoky scene is shaped by two processes that a camera records together: the participating medium alters scene radiance in a view-dependent way, and the image signal processor (ISP) then remaps the result through a nonlinear tone and color transformation. Recovering a clean 3D scene requires separating both. Per-view sRGB dehazing acts only after the ISP has entangled them; standard 3D reconstruction ignores the medium and absorbs it into scene geometry and radiance. FujinSplat addresses the problem in the RAW domain, where the two processes remain separable. A per-scene Base ISP is fitted from the scene's hazy RAW captures to its own camera renderings and then frozen, providing a fixed photometric anchor that performs no dehazing. Analyzing expert corrections reveals a compact, low-dimensional correction space identifiable from RAW alone. FujinSplat therefore fits per-view action answers at the training poses and trains a single scene-agnostic controller to regress them from RAW; the corrected views supervise one static 3D Gaussian representation, jointly with a bounded per-view residual that reconciles cross-view photometric inconsistencies. On the RealX3D real-world smoke
benchmark FujinSplat clearly outperforms the strongest comparable baseline, ahead of both physics-based reconstruction and restoration-then-3DGS pipelines. Code is available at https://github.com/I2WM/FujinSplat.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Tri-PvP: Exposing Modality Bias in Omni-Modal Large Language Models through Perceptual-Propositional Evidence Conflicts
Yen-Ting Piao, Shu-Yun Chen, Chin-Hui Chu et al. (+4 more)
Omni-modal large
language models (OLLMs) jointly process
vision, audio, and text, yet their modality bias under cross-modal conflict remains underexplored. Existing
benchmarks conflate two distinct forms of evidence within a single modality: perceptual signals (e.g., a photograph or recording of a dog) and propositional signals (e.g., the declarative claim "this is a dog"), such that any measured modality bias is inherently confounded with evidence-form bias, precluding clean attribution to either source. To address this, we introduce Tri-PvP, an 8,000-sample tri-modal conflict
benchmark crossing
vision, audio, and text, where
vision and audio each take perceptual or propositional form. Evaluating five OLLMs, we find robust visual bias across most models and evidence-type conditions. Crucially, we reveal a systematic asymmetry in evidence-form bias: models exhibit a stronger bias toward perceptual signal in
vision but propositional in audio. Further analyses via layer-wise linear probing and contrastive decoding reveal that modality bias is already linearly decodable from early representation layers and can only be partially mitigated, calling for mitigation strategies beyond surface-level interventions.
Read the full abstract ▾
All authors Yen-Ting Piao, Shu-Yun Chen, Chin-Hui Chu, Chun-Wei Chen, Shih-Yun Shan Kuan, Hung-yi Lee, Yun-Nung Chen
Computer Vision
Depth-to-Image Synthesis-Driven Generative Unguided Depth Completion
Jiayi Yuan, Na Zhao and De Wen Soh
Guided depth completion methods heavily depend on RGB quality and
alignment, while unguided ones often suffer from limited precision due to the absence of explicit visual cues. In this paper, we present Depth-to-Image Synthesis-Driven Generative Unguided Depth Completion (GUDC), a new completion paradigm that innovatively bridges advanced 2D generative models with unguided depth completion, enabling semantics-aware depth inference without real RGB inputs. Our key idea is to exploit ControlNet's powerful depth-conditioned generation capability to synthesize pseudo-images directly from sparse depth, effectively converting the original unguided setting into a semantics-guided one. To address the potential image-depth misalignment caused by depth sparsity, we propose a multi-level dense-to-sparse representation
distillation strategy for ControlNet fine-tuning, where dense-depth features act as teacher signals to distill consistent structural representations for sparse-depth inputs. Furthermore, during pseudo-image-guided completion, we propose a pseudo-image semantic
attention fusion module to adaptively extract informative semantic cues from pseudo-images while suppressing artifacts (e.g., texture
hallucinations). Extensive experiments on KITTI and NYUv2 validate that our GUDC achieves superior accuracy and robustness over existing methods.
Read the full abstract ▾
Computer Vision
Geometry-Aware Test-Time Learning for Quantitative Spatial Reasoning
Gege Zhang, Shuaicheng Niu, Gang Dai et al. (+2 more)
Quantitative spatial
reasoning in visual-
language models (VLMs) aims to infer spatial distances and directional relationships among objects in 3D space from a 2D image and a natural
language query. Despite recent progress, VLM spatial
reasoning remains brittle under distribution shifts, largely due to the high cost of 3D supervision. As a result, models often produce inconsistent or contradictory predictions when faced with novel object configurations or rephrased spatial queries, revealing a misalignment between learned representations and underlying geometry. To address this, we propose TTL-SR, a geometry-aware Test-Time Learning framework for quantitative Spatial
Reasoning that leverages
geometric consistency constraints and unlabeled test data to adapt models to target domains. Specifically, TTL-SR augments the input query with geometrically coupled auxiliary queries, filters unreliable predictions via adaptive
geometric triggering to construct structured
token-level pseudo-labels, and updates model parameters under a geometry-aware multi-objective loss using only test data. Experimental results demonstrate that TTL-SR significantly boosts spatial
reasoning performance, yielding 6.47% and 9.41% accuracy gains for Qwen3-VL-4B-Instruct and SpatialRGPT-VILA-1.5-8B on Q-Spatial-ScanNet
dataset, respectively.
Read the full abstract ▾
All authors Gege Zhang, Shuaicheng Niu, Gang Dai, Lei Sun, Shuangping Huang