Computer Vision
DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences
Yankai Yang, Yancheng Long, Bin Wen et al. (+4 more)
Video multimodal large
language models have made strong progress on open-ended
video understanding, but they still lack precise local spatiotemporal perception. When two
videos share almost the same global semantics and differ only in a short time span or a small region, current models often fail to find the change and provide reliable evidence. We propose DELTAVID, a verifiable proxy-task framework that enhances fine-grained spatiotemporal perception with cross-
video differences. The key idea is to turn cross-
video spot-the-difference into a trainable perception signal, where a model identifies local changes, judges temporal boundaries, and organizes spatial evidence by comparing similar
videos. To make this signal scalable to train and reliable to evaluate, we further introduce DELTAVID-10K and DELTAVID-Bench, which convert controllable local differences in real
videos into evidence-labeled training and test samples. Experiments show that DELTAVID substantially improves performance on cross-
video difference understanding and transfers the learned local evidence ability to general
video understanding
benchmarks, including MMVU, MLVU,
Video-MME, VideoHolmes, VideoMMMU, LVBench, TempCompass, and LongVideoBench. These results show that cross-
video differences are not only an effective way to diagnose fine-grained perception failures, but also a scalable proxy supervision that moves
Video MLLMs from coarse semantic understanding toward fine-grained spatiotemporal evidence
reasoning.
Read the full abstract ▾
All authors Yankai Yang, Yancheng Long, Bin Wen, Fan Yang, Tingting Gao, Han Li, Shuo Yang
Computer Vision · Cross-listed from cs.GR
DANTE-W: Diffuse Albedo Neural Texturing in the Wild
Guangyu Wang, Tianheng Lu, Ruqi Huang et al. (+1 more)
Classical mesh texturing techniques blend captured multi-view images directly, which inevitably suffer from baked-in shading and casted shadows that compromise visual fidelity during relighting. To circumvent this issue, we present a neural texturing framework, namely DANTE-W, to enable high-fidelity diffuse albedo texture recovery from unstructured image collections for large-scale, in-the-wild scenes, which integrates seamlessly with traditional 3D reconstruction pipelines. Given a reconstructed mesh and its surface parameterization, our method fuses view-space generative albedo priors into a coherent texture space via an expressive neural representation, while substantially enhancing fine-grained textural details through physically principled neural rendering. To comprehensively evaluate our method, we curate a
benchmark dataset featuring diverse, fine-grained textures, comprising both real-world in-the-wild scenes and synthetic objects. Extensive experiments verify the effectiveness of our approach in reconstructing accurate albedo textures and boosting relighting fidelity. Project page: dante-wild.github.io.
Read the full abstract ▾
All authors Guangyu Wang, Tianheng Lu, Ruqi Huang, Lu Fang
Computer Vision
FedLAS: Feature-Modulated Bidirectional Label Smoothing for Neural Network Calibration
Thiru Thillai Nadarasar Bahavan, Sachith Seneviratne and Saman Halgamuge
Deep Neural Network (DNN) classifiers suffer from poor calibration when their softmax outputs (predictive confidence) deviate from the empirical likelihoods. This manifests itself as either overconfident incorrect predictions or under-confident correct predictions. Label smoothing (LS) enhances model calibration by introducing
entropy regularization during training through redistributing probability mass from the ground-truth label to the remaining classes. LS, including Margin-based LS (MbLS), have restrictive assumptions: they rely on predefined, uniform smoothing rules and only tackle overconfidence. In reality, samples exhibit diverse characteristics, such as difficulty/ambiguity, that interact with the evolving nature of the model being trained. In training, samples may have various degrees of under- or overconfidence. To overcome this, a mechanism that identifies the specific confidence state of each sample and determines the appropriate degree of smoothing in each training step is needed, tailoring the adjustment to the individual sample. We propose FedLAS: Feature-Modulated Bidirectional Label Smoothing, a plug-and-play
algorithm for label smoothing-based losses. In FedLAS, we introduce a Feature Norm-based Confidence Indicator (NCI) to control smoothing and a Bidirectional Calibration Gating (BCG) module to detect both over and under-confidence. Our
algorithm can be integrated with LS and MbLS based losses when applied to standard DNNs, enhancing performance. Extensive experiments on standard and fine-grained high-resolution
vision benchmarks show that FedLAS consistently improves calibration compared to modern baselines, reducing Expected Calibration Error (ECE) and Adaptive ECE while maintaining Top-1 accuracy. Code: github.com/nadarasarbahavan/FEDLAS
Read the full abstract ▾
Computer Vision
Obliviate: Erasing Concepts from Autoregressive Image Generation Models
Hossein Shakibania, Jonas Henry Grebe, Tobias Braun et al. (+4 more)
The widespread adoption of generative AI models has intensified concerns about misuse, including the creation of unsafe or disturbing imagery. To mitigate such issues, several concept erasure approaches have been proposed to remove harmful content from
multimodal generative models. Yet concept erasure for autoregressive image generation remains largely unexplored, despite the growing relevance of these models in recent trends toward unified
multimodal architectures. In this work, we fill this gap by introducing Obliviate, a guidance-based concept erasure method for autoregressive image generation. Our method builds on three key design choices: KL-based supervision over visual
token distributions, trajectory-level updates over full autoregressive rollouts, and aligned visual prefixes for stable target construction. We evaluate Obliviate on three state-of-the-art autoregressive text-to-image models,
Liquid, Emu3-Gen, and Janus-Pro, covering the erasure of explicit content, graphic violence, and branded imagery. Obliviate consistently outperforms current alternatives, reducing nudity on the defensive RAB
benchmark from 91.58 to 3.15 while preserving overall model utility. Our code is available at https://github.com/
multimodal-ai-lab/Obliviate.
Read the full abstract ▾
All authors Hossein Shakibania, Jonas Henry Grebe, Tobias Braun, Ege Aktemur, Saleh Aslani, Mehmet Görkem Yiğit, Marcus Rohrbach
Computer Vision
AEGIR: Modeling Area Emitters for Indoor Inverse Rendering using Gaussian Splatting
Mohamed Shawky Sabae, Philipp Langsteiner, Jan-Niklas Dihlmann et al. (+1 more)
Inverse rendering requires separating illumination from surface materials, which is highly ambiguous due to their tight coupling in observed images. While Gaussian Splatting is efficient for novel view synthesis, existing relightable methods approximate scene lighting using discrete point lights, global environment maps, or implicit representations. By ignoring the physical spatial extent of real-world emitters, these approaches produce incorrect light attenuation and unrealistic shadows. We present AEGIR (Area Emitters for Gaussian Inverse Rendering), a framework that explicitly models local area emitters within a relightable Gaussian Splatting representation. Joint optimization of emitters, materials, and geometry is challenging due to flexible emitter parameterization, which increases both the number of parameters and the ambiguity between illumination and materials. We address this by introducing a differentiable deferred rendering pipeline that integrates multiple importance sampling with targeted regularization. As a result, AEGIR accurately simulates local light transport and achieves more consistent decomposition. Experiments show that explicit area emitters improve illumination reconstruction and enhance downstream tasks, including novel view synthesis, controlled relighting, and virtual object insertion, particularly in scenes with complex local lighting.
Read the full abstract ▾
All authors Mohamed Shawky Sabae, Philipp Langsteiner, Jan-Niklas Dihlmann, Hendrik Lensch
Computer Vision
Physics-Grounded Disentangled Flow Modeling for Brain Disease Progression Trajectory
Jun Wang and Peirong Liu
Forecasting longitudinal
brain lesion
evolution is critical for disease monitoring and treatment planning. Existing approaches typically learn a direct mapping from a baseline image to a future observation, without explicitly modeling the physical mechanisms underlying the lesion progression. Such an entangled modeling of structural deformation and image intensity variation limits physical plausibility, model generalization, and interpretability. To address this, we propose PDF, a Physics-grounded Disentangled Flow matching framework for longitudinal
brain disease
forecasting. We explicitly decompose the longitudinal modeling of lesion growth into two processes, each learned by a dedicated flow matching network: morphology
evolution, which captures lesion growth and structural deformation; and intensity
evolution, which models signal changes driven by variations in lesion concentration. To enforce physics-grounded constraints, we introduce a PDE-regularized loss based on lesion growth dynamics, that enforces a
diffusion-reaction-advection formulation for morphological
evolution. Experiments on three public longitudinal
datasets spanning diverse
brain diseases demonstrate state-of-the-art performance, validating the effectiveness of the disentangled modeling framework and physics-grounded learning design. Code is publicly available at https://github.com/jhuldr/PDF.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Envisage: Diffusion-Based Rhinoplasty Goal Visualization with Mask-Decomposed Evaluation
Mudit Agarwal and Amit D. Bhrany
Localized generative editing needs localized evaluation: full-image identity metrics are structurally confounded under hard-composited edits. We present Envisage, a FLUX.1-Fill inpainting reference pipeline for rhinoplasty goal visualization from a single frontal photograph. The pipeline combines 8 rhinoplasty clinical presets (the released framework also includes 8 blepharoplasty and 8 rhytidectomy presets), MediaPipe masks, and hard-mask compositing. The composite preserves outside-mask pixels by construction, so full-face identity scores are dominated by copied pixels rather than by the
diffusion backbone. Because full-face identity metrics cannot grade localized edits, we introduce SurgicalScore, a mask-decomposed 0-1 protocol scoring edit direction, edit magnitude, masked LPIPS, realism, and outside-mask preservation; SS_raw assigns 0.919 [0.918, 0.920] to a perfect-predictor control , anchoring the ceiling. On N=211, the paired ArcFace gain (output-to-GT minus input-to-GT) is negative for all methods (Envisage -0.048 smallest, vs. ICEdit -0.139, Kontext -0.242, InstructPix2Pix -0.294; p < 1e-4), with external validation on a 457-pair ASPS/PCA corpus showing a larger negative gap. With SurgicalScore, Envisage achieves the highest score (0.599 [0.579, 0.619]) and leads on both metrics, but the all-negative ArcFace gap shows that full-face identity is poorly aligned with localized
surgical accuracy under hard compositing. A 5-seed GT-oracle (an upper bound, not a deployable result) reduces the residual ArcFace gap by 73% (-0.054 to -0.015), with positive output-to-GT gain on 33.9% of cases, indicating candidate-space headroom for a learned ranker. For localized edits, progress should be measured with edit-region fidelity rather than full-face identity metrics. We release Envisage, SurgicalScore, preset definitions, and matched split manifests.
Read the full abstract ▾
Computer Vision
SIGNET: Motion-Level Knowledge Transfer for Cross-Language Sign Language Translation
Sobhan Asasi, Ozge Mercanoglu Sincan and Richard Bowden
Sign
language translation (SLT) remains challenging due to its high spatio-temporal complexity, long sequences, and the need to model multiple articulators without relying on gloss annotations. Existing approaches are typically tailored to individual
datasets or
languages and struggle to scale, while overlooking the relationships between sign
languages that could inform more effective cross-lingual transfer. We present SIGNET, a framework that enables motion-level knowledge transfer for cross-
language sign
language translation. Our key insight is that, although sign
languages differ in grammar and lexicon, pretrained models capture motion-level visual patterns that can be reused across
datasets and
languages. SIGNET integrates multiple pretrained sign
language backbones through an
attention-based, hand-prior aggregation mechanism that guides a gated fusion network in dynamically selecting the most relevant experts. Comprehensive experiments on four
benchmarks (How2Sign, Phoenix14T, CSL-Daily, and MeineDGS) demonstrate state-of-the-art translation performance, and SIGNET also surpasses prior methods on WLASL for sign
language recognition.
Read the full abstract ▾
Computer Vision
Meshtryoshka: Differentiable Rendering of Real-World Scenes via Mesh Rasterization
David Charatan, Daniel Xu, Richard Szeliski et al. (+2 more)
Differentiable rendering has emerged as a powerful approach for 3D reconstruction and novel view synthesis. State-of-the-art differentiable rendering methods combine a variety of custom representations of 3D geometry and appearance with specialized renderers. However, most downstream tasks in computer graphics rely on 3D meshes. While prior work has attempted differentiable rendering with mesh representations, these approaches are limited to object-centric scenes and fail to reconstruct large-scale, unbounded scenes. In this work, we introduce Meshtryoshka, a novel mesh differentiable rendering framework that combines an off-the-shelf triangle rasterizer with a 3D representation that consists of nested mesh shells which resemble a matryoshka doll. In every forward pass, the mesh shells are extracted anew from a 3D signed distance function via iso-surface extraction, and the opacities for each vertex are computed as a function of signed distance. Each mesh shell is then rasterized independently, and the final image is created via alpha compositing. Crucially, mesh vertex positions are updated only indirectly via gradients that flow through the opacity values into the signed distance function, and hence, our method is compatible with off-the-shelf mesh renderers that need not be differentiable with respect to vertex positions. On object-centric scenes, our method performs competitively with surface-based differentiable rendering techniques. Our differentiable mesh rendering method scales to unbounded, real-world 3D scenes, where it yields high-quality novel view synthesis results approaching those of state-of-the-art, non-mesh methods. Our method suggests that it may be possible to solve the differentiable rendering problem without relying on specialized renderers, only using conventional tools from the computer graphics toolbox.
Read the full abstract ▾
All authors David Charatan, Daniel Xu, Richard Szeliski, George Kopanas, Vincent Sitzmann
Computer Vision
IMU-HOI: A Symbiotic Framework for Coherent Human-Object Interaction and Motion Capture via Contact-Conscious Inertial Fusion
Lizhou Lin, Songpengcheng Xia, Zengyuan Lai et al. (+3 more)
Capturing full-body human motion with object interactions is crucial for AR/VR and robotics applications, yet it remains challenging for conventional
vision-based methods due to occlusions and constrained capture volumes. Inertial measurement units (IMUs) offer a compelling alternative without line-of-sight requirements, but existing IMU-based motion capture assumes an isolated human and ignores object contacts and dynamics. To bridge this gap, we present IMU-HOI, a novel framework that jointly recovers full-body human pose and 6-DoF object trajectory from sparse IMUs on the body and object, explicitly modeling human-object interaction. Our approach first infers probabilistic hand-object contacts directly from IMU streams and uses them as a high-level signal to route between kinematic and inertial
reasoning. These contact cues drive a three-stage fusion pipeline that refines human pose and root translation, and fuses hand-based forward kinematics with object-IMU integration for object motion, yielding coherent, drift-resilient trajectories for both human and object. Experiments on challenging human-object interaction scenarios demonstrate substantial accuracy gains over prior inertial motion capture methods. Moreover, IMU-HOI can be plugged into existing sparse-IMU mocap backbones with minimal changes, effectively extending the scope of purely inertial motion capture from isolated humans to full human-object interaction and joint motion estimation.
Read the full abstract ▾
All authors Lizhou Lin, Songpengcheng Xia, Zengyuan Lai, Lan Sun, Jiarui Yang, Ling Pei
Computer Vision
Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation
Anya Ji, Abhijith Varma Mudunuri, David M. Chan et al. (+1 more)
While recent
vision-
language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs, it remains unclear whether they can recover temporal dynamics when motion is present. To this end, we introduce Animation2Code, a
benchmark for evaluating temporal visual
reasoning via reconstructing executable web animation code from
videos. Animation2Code consists of 1,069 web animation
videos with diverse visual appearances and motion patterns, paired with corresponding HTML/CSS/JavaScript implementations. We propose two human-aligned metrics, appearance similarity and temporal similarity, which allow us to disentangle visual fidelity from temporal
alignment when comparing rendered animations against ground-truth samples. Benchmarking state-of-the-art VLMs on this
dataset shows that current VLMs struggle to maintain temporal consistency in reconstruction, even when achieving high appearance similarity, including under finetuning and iterative refinement settings. Code and data are available at https://anya-ji.github.io/animation2code-website .
Read the full abstract ▾
All authors Anya Ji, Abhijith Varma Mudunuri, David M. Chan, Alane Suhr
Computer Vision
SatSplat: Geometrically-Accurate Gaussian Splatting for Satellite Imagery
Shuang Song, Jiyong Kim and Rongjun Qin
High-resolution satellite imagery demands 3D reconstruction methods that deliver both speed and
geometric accuracy. Recent adaptations of 3D Gaussian Splatting (3DGS) to satellite imagery demonstrate strong efficiency, but reconstruction quality often degrades under diverse illumination across multi-date, high-altitude acquisitions (with small intersection angles), limiting applicability to remote sensing and
vision tasks. We present SatSplat, the first framework to adapt 2D Gaussian Splatting (2DGS) to satellite photogrammetry, with online camera adjustment. We approximate satellite cameras with an affine model and learn a minimal delta parameterization for in-splat camera refinement from dense observations. The formulation is implemented with a 2DGS scene representation. To handle time-varying shadows and illumination changes, we integrate
geometric shadow mapping and per-camera color correction during training. Across the evaluated DFC2019 and IARPA2016
benchmark sites, SatSplat achieves strong
geometric accuracy while significantly outperforming prior 3DGS-based baselines. On our processed DFC2019
benchmark, SatSplat reduces mean absolute error by 11.93% and peak
video memory by 31% relative to the previous state of the art. Our approach enables large-scale digital surface modeling with practical computational efficiency. The project page is available at https://gdaosu.github.io/satsplat/.
Read the full abstract ▾
Computer Vision
Digitizing Coaching Intelligence: An Agentic Framework for Holistic Athlete Profiling using VLM and RAG
Deep Ghosal, Ishani Sen, Wazib Ansar et al. (+1 more)
Athlete assessment is a critical process for tracking physical progress and identifying elite talent. However, during mass recruitment drives, traditional methods rely on manual observation, which is inherently subjective and unscalable, or basic computer
vision (CV) systems limited to quantitative repetition counting. These standard approaches lack the "coaching intelligence" required to evaluate qualitative physiological markers such as form degradation, spinal articulation, and fatigue. This paper presents a novel, LLM-based hybrid agentic framework for automated, holistic athlete profiling that strictly aligns with the Sports Authority of India (SAI) assessment protocols. Orchestrated via LangGraph, our dual-pipeline architecture synthesizes the
geometric precision of CV (MediaPipe) for kinematic tracking with the semantic
reasoning of
Vision-
Language Models (Llama-4-scout). To overcome the latency and
token constraints associated with
multimodal video processing, we introduce a 3 X 3 "Smart Grid" temporal chunking strategy, reducing computational overhead by over 88% while preserving critical temporal continuity. To ensure data integrity and mitigate
hallucination, the framework pioneers an autonomous "LLM-as-a-Judge" self-correction loop that cross-references quantitative and qualitative metrics before persistence. Finally, we implement a dual-persistence
Retrieval-Augmented Generation (RAG) pipeline utilizing a vector search engine (ChromaDB). This enables coaches to bypass rigid SQL databases and perform complex semantic queries (e.g., "Identify athletes with high endurance but poor core
rigidity") using natural
language. Experimental results demonstrate that this multi-
agent approach significantly bridges the gap between raw biometric tracking and actionable coaching insights, offering a scalable, objective solution for national talent identification.
Read the full abstract ▾
All authors Deep Ghosal, Ishani Sen, Wazib Ansar, Amlan Chakrabarti
Computer Vision
KM-Speaker: Keypoint-Based Style Control for High-Quality Speech-Driven 3D Facial Animation and Dialogue Localization
Arthur Josi, Emeline Got, Abdallah Dib et al. (+2 more)
Speech-driven 3D facial animation methods face significant challenges in simultaneously achieving high-fidelity motion and precise artistic control at production quality. Existing controllable models typically learn global style control by relying on large-scale, low-quality in-the-wild
datasets that compromise overall animation realism. Furthermore, these frameworks often lack the fine-grained temporal precision required for demanding tasks such as dialogue localization (e.g., dubbing), where matching specific facial expressions is as critical as lip synchronization. We present KM-Speaker (Keypoint-Matching Speaker), a novel keypoint-conditioned flow-based generative framework that provides both global style guidance and frame-level temporal control from reference performances. We propose a disentanglement strategy that separates audio-driven lip motion from keypoint-driven upper-face dynamics, together with a global style context preservation mechanism to ensure coherent full-face expressiveness. KM-Speaker advances example-based 3D facial animation by achieving high-fidelity motion and flexible controllability in a data-constrained setting, consistently outperforming state-of-the-art methods in lip-sync accuracy, style adherence, and expressive temporal control.
Read the full abstract ▾
All authors Arthur Josi, Emeline Got, Abdallah Dib, Luiz Gustavo Hafemann, Rafael M. O. Cruz
Computer Vision
Matteo Farina, Vishaal Udandarao, Thao Nguyen et al. (+33 more)
Building performant
Vision-
Language Models (VLMs) requires carefully curating large-scale training
datasets, yet the community lacks systematic
benchmarks for evaluating such curation strategies. We introduce DataComp for VLMs (DCVLM), a
benchmark for controlled data-centric experiments to improve VLM training. As part of DCVLM, we collect 160
datasets spanning four data types – image-caption pairs,
multimodal interleaved documents, text-only, and instruction-tuning data – into a corpus of 6T
multimodal tokens. DCVLM allows participants to test curation strategies (filtering, mixing, formatting, sampling) across 1B-8B models and 6.25B-200B
token budgets. Models are then evaluated on a carefully selected suite of up to 52 downstream
benchmarks across 9 domains. We conduct extensive experiments on DCVLM and find that data mixing, not filtering, is key to a high-quality training
dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening at larger scales. The resulting
dataset, DCVLM-Baseline, enables training an 8B VLM to 63.6% accuracy on our 33-task core suite with 200B training
tokens. Compared to FineVision, the state-of-the-art open VLM training
dataset, this represents an improvement of +5.4pp. DCVLM and all accompanying artifacts will be made publicly available at https://www.datacomp.ai/dcvlm/.
Read the full abstract ▾
All authors Matteo Farina, Vishaal Udandarao, Thao Nguyen, Selim Kuzucu, Maximilian Böther, Andreas Hochlehnert, Adhiraj Ghosh, Marianna Nezhurina, Karsten Roth, Joschka Struber, Yuhui Zhang, Sebastian Dziadzio, Elaine Sui, Soumya Jahagirdar, Dhruba Ghosh, Hasan Hammoud, Thomas De Min, Simone Caldarella, Jehanzeb Mirza, Sedrick Keh, Mehdi Cherti, Hilde Kuehne, Bernt Schiele, Serena Yeung-Levy, Muhammad Ferjad Naeem, Federico Tombari, Ana Klimovic, Elisa Ricci, Matthias Bethge, Sewoong Oh, Ameya Prabhu, Alessio Tonioni, Jenia Jitsev, Massimiliano Mancini, Ludwig Schmidt, Nikhil Parthasarathy
Computer Vision
MammoFlow: Multiview Mammogram Synthesis with Anatomically Consistent Flow Matching
Yuexi Du, Leya Barrientos, Laura Sheiman et al. (+3 more)
Multiview mammography relies on paired craniocaudal (CC) and mediolateral oblique (MLO) views to provide complementary projections of a 3D breast volume, enabling precise anomaly localization. However, acquiring high-quality, balanced
datasets remains challenging for deep learning applications. We propose a novel method to synthesize multiview mammograms by leveraging the inherent
geometric relationship between CC and MLO views. To enforce an implicit 3D consistency prior during generation, we develop an
alignment module that searches a 2D affine transformation subspace to establish optimal anatomical correspondence. Leveraging this
alignment, we introduce a pixel-space self-consistency loss based on the Earth Mover's Distance (EMD) between the 1D anteroposterior (AP) axis tissue distributions of the generated images. Integrated into a pretrained flow matching model, MammoFlow forces synthesized pairs to share physically plausible tissue distributions from the chest wall to the nipple. To our knowledge, this is the first work to guide multiview mammogram generation using implicit
geometric tissue correspondence. Our method demonstrates superior image quality, passes expert radiologist evaluation, and generates physically consistent pairs that improve downstream classification AUC by 5%. Code is available at https://github.com/XYPB/MammoFlow
Read the full abstract ▾
All authors Yuexi Du, Leya Barrientos, Laura Sheiman, John Lewin, Hemant D. Tagare, Nicha C. Dvornek
Computer Vision · Cross-listed from cs.RO
The Speedup Paradox: Rethinking Inference Speed-Quality Trade-off in Embodied Tasks
Yujin Wang, Junli Chen, Yixuan Li et al. (+4 more)
Embodied foundation models have recently been widely used to improve
robot generalization and task success rates. Previous works apply lossy efficient-inference techniques such as quantization,
pruning, and asynchronous inference, accepting small action quality degradation in exchange for lower per-inference computation cost and inter-action latency. However, unlike traditional static ML tasks,
embodied tasks involve repeated interaction with the environment, and task-level performance is determined not only by inference cost, but also by closed-loop effects unique to
embodied execution, which remain insufficiently characterized in current efficient-inference studies. In this work, we propose TISED (\underline{T}ask-level \underline{I}nference \underline{S}peedup \underline{E}ffect \underline{D}ecomposition), an analytical framework that unifies diverse lossy lightweighting inference optimization techniques and decomposes their effects on static and dynamic tasks; using it, we uncover several paradoxical effects: (1) on static tasks, optimization can sometimes lengthen end-to-end per-task completion time even as per-chunk time drops; (2) on dynamic tasks, moderate lossy lightweighting optimization can raise task success rate even above the baseline; and (3) the monotonicity and sweet-spot location of both effects can shift with hardware configuration. Together, our findings provide a new perspective on adapting inference optimization techniques to
embodied tasks. Source code: https://github.com/thu-ee-acts-lab/robotics-the-speedup-paradox.
Read the full abstract ▾
All authors Yujin Wang, Junli Chen, Yixuan Li, Shunan Dong, Huazhong Yang, Yongpan Liu, Hongyang Jia
Computer Vision
Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty
Xiao Song, Haonan Qin, Zhaoxu Zhang et al. (+3 more)
Large
vision-
language models (LVLMs) are increasingly used for clinical image understanding, yet they remain vulnerable to
hallucinations–producing textual findings or attributes not supported by the image. We present a
vision-traceable
hallucination detection framework that audits arbitrary LVLM responses via visual evidence grounding, requiring neither modification nor internal access to the hidden states of LVLMs. Given an LVLM response, we extract visually verifiable entities and use a medical-domain-adapted Qwen-VL grounding verifier to localize each entity on the input image. To enhance the robustness of our detection method, we introduce a counterfactual entity perturbation method and estimate visual evidence uncertainty by contrasting factual and counterfactual grounding results. Specifically, we compute an entity-level uncertainty score from the positive confidence, counterfactual confidence, and their grounding overlap for binary
hallucination decision-making. Experiments on multiple medical imaging modalities and LVLM backbones demonstrate that our method consistently improves
hallucination detection performance over recent baselines, while providing interpretable localization evidence and strong cross-model transferability. Code and
dataset are available at https://github.com/Agentic-CliniAI/CounterVHD.
Read the full abstract ▾
All authors Xiao Song, Haonan Qin, Zhaoxu Zhang, Jiong Zhang, Yuqi Fang, Caifeng Shan
Computer Vision
CLEAR-MoE: Shared-Basis Expert Extraction from Frozen Vision Transformers via Calibration-Driven Layer Selection
Md Irtiza Hossain, Humaira Ayesha and Junaid Ahmed Sifat
We present CLEAR-MoE, a four-phase post-training pipeline that converts a frozen pretrained
Vision Transformer (ViT) into a sparse Mixture-of-Experts (MoE) model without updating backbone weights. The pipeline (i) scores feed-forward network (FFN) layers by sparsity, clusterability, and output sensitivity; (ii) decomposes selected layers into a shared low-rank SVD basis and per-cluster residual experts using k-means clustering; (iii) trains lightweight routers supervised by cluster labels; and (iv) dispatches
tokens through pluggable CUDA backends. On Imagenette with DeiT-Small, CLEAR-MoE retains 99.9% of the dense model's accuracy (86.70 +/- 0.02% versus 86.73%). Extensive ablation studies reveal a consistent empirical finding: the shared SVD basis is the primary factor responsible for preserving accuracy. Random routing, learned routing, and three different router architectures produce nearly identical performance, with accuracy varying by at most 0.06 percentage points (86.62%-86.68%). Accuracy also remains stable across different SVD ranks, expert counts (2-8), calibration set sizes (50-500), and random seeds. This behavior generalizes across five ViT backbones (DeiT-Tiny, DeiT-Small, DeiT-Base, ViT-Small, and ViT-Base), covering models from 5.7M to 86.6M parameters, with accuracy differences <= 0.10 percentage points from their dense counterparts. On a GTX 960 GPU, routing and scatter-gather overhead make the CLEAR-MoE FFN 1.3-1.7x slower than the dense implementation. A dispatch microbenchmark further shows that routing is an order of magnitude more
memory-bound than expert
matrix multiplications, identifying fused dispatch kernels as a promising direction for future optimization.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Anatomy-Grounded Synthetic Coronary Angiography for Geometry-Informed Multi-View Matching
In Kyu Lee, Sumin Seo and Jaesik Min
Accurate correspondence matching across multiple angiographic views is the prerequisite for 3D coronary reconstruction and interventional guidance. However, the development of robust deep learning models for this task has been stifled by a fundamental data bottleneck. Obtaining ground truth for matching tasks in angiography pairs is prohibitively expensive and hard to scale. To overcome this barrier, we introduce a physically-grounded data generation framework that synthesizes high-fidelity Digital Reconstructed Radiographs (DRRs) from 3D Coronary CT Angiography (CCTA) volumes. Our framework generates dense, highly accurate 3D-to-2D projection labels by simulating realistic C-arm acquisition geometry on patient anatomy at zero human cost. Leveraging this dense supervision, we propose a Geometry-Informed Matching Module (GIMM) that integrates global feature and anatomical structure into correspondence learning. Unlike real angiography where assessment relies on subjective human annotation, our
dataset provides 2D correspondence labels with paired images, allowing human-free evaluation. We comprehensively evaluate our method on the proposed CT-derived DRR
dataset and demonstrate improvements over other matching baseline models.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
DeVAR: Low-Dose CT Denoising via Visual Autoregressive Modeling
Xizhuo Zhang, Yannian Gu, Zhongzhen Huang et al. (+2 more)
Computed
tomography (CT) plays a crucial role in medical diagnosis, but minimizing radiation exposure while maintaining image quality remains a critical challenge. Low-dose CT (LDCT) protocols reduce radiation risks but inevitably suffer from severe noise and artifacts that compromise diagnostic accuracy. While existing deep learning methods have achieved promising results, there remains a continuous quest for generative paradigms that intrinsically capture global-to-local structural dependencies to better preserve fine anatomical details. To this end, we propose DeVAR, a novel generative framework that applies visual autoregressive modeling (VAR) to LDCT denoising for the first time. Conditioned on global context provided by LDCT prefix
tokens, DeVAR progressively generates discrete
token maps of the target normal-dose CT (NDCT) via next-scale prediction. Because quantization inherently discards high-frequency information, we introduce a residual refiner to capture subtle anatomical structures beyond the capacity of a discrete codebook. Finally, empowered by a dual-representation hybrid training strategy, our hybrid NDCT decoder seamlessly integrates continuous and discrete latents to reconstruct high-fidelity, detail-preserved images. Extensive experiments on two public
datasets demonstrate that DeVAR consistently achieves superior qualitative and quantitative performance compared to state-of-the-art LDCT denoising methods.
Read the full abstract ▾
All authors Xizhuo Zhang, Yannian Gu, Zhongzhen Huang, Shaoting Zhang, Xiaofan Zhang
Computer Vision · Cross-listed from q-bio.QM
Establishing the Minimal Clinically Important Difference (MCID) for Smartphone-Derived Gait Measures in Multiple Sclerosis
Mike D Rinderknecht, Bernhard Fehlmann, Dimitar Stanev et al. (+7 more)
Background: Digital health technologies allow for frequent, remote gait monitoring in people with multiple sclerosis (MS). However, to differentiate daily variability from actual disease progression in longitudinal data, established minimal clinically important differences (MCID) are required. Currently, there is limited literature defining these thresholds for digital gait metrics. Objective: To establish MCIDs for digital gait measures reflecting progression in MS. Methods: Digital gait measures were captured via daily, remote, smartphone-based Two-Minute Walk Tests in CONSONANCE (NCT03523858), a phase 3b study of ocrelizumab in progressive MS. Using an anchor-based approach, median changes from baseline at Week 96 on digital gait measures were computed for patients showing clinically meaningful worsening on either Timed 25-Foot Walk, Ambulation Score, Expanded Disability Status Scale, or 12-item Multiple Sclerosis Walking Scale. These changes were subsequently triangulated to derive the MCID estimates. Results: 243 patients with progressive MS (female: n=125 (51%); mean [SD] age: 49.3 [9.3]; mean [SD] EDSS: 4.8 [1.4]) had digital gait data available at baseline and Week 96. Median changes were generally consistent across anchors. Triangulated MCIDs are: Step Velocity = -0.16 m/s, Step Velocity Scaled to Walking Time = -0.18 m/s, Step Duration = 0.06 s, Step Length = -0.07 m, Total Number of Steps = -28, and Total Distance Walked = -24 m. Conclusion: These MCIDs provide a framework for interpreting meaningful gait changes and integrating digital measures into MS outcome evaluation. Beyond facilitating novel clinical trial endpoints to evaluate treatment efficacy, they enable objective, real-world monitoring to advance personalized patient care.
Read the full abstract ▾
All authors Mike D Rinderknecht, Bernhard Fehlmann, Dimitar Stanev, Cedric Simillion, Ernst Bos, Letizia Leocani, Agne Kazlauskaite, Gary Cutter, Helmut Butzkueven, Licinio Craveiro
Computer Vision · Cross-listed from eess.IV
Measured-Subspace Consistency: A Plug-and-Play Operator for Diffusion Posterior Sampling in Accelerated MRI Reconstruction
Junhyeok Lee and Kyu Sung Choi
Diffusion posterior samplers for accelerated MRI can reconstruct accurately yet still disagree on the acquired k-space across samples, placing posterior variability on coefficients the scanner has already measured. We identify this measured-subspace leakage as a physical-admissibility failure. Under a hard-constraint model it violates the measurement constraint and inflates the reported uncertainty with disagreement about coefficients the scanner has already determined. To quantify this leakage, we introduce complementary measured- and unmeasured-subspace k-space dispersion metrics (MSD/USD). We then present Measured-Subspace Consistency (MSC), a training-free terminal correction that wraps any compatible image-space posterior sampler with a standard multi-coil consistency lock. The ideal lock follows classical range/null-space data consistency. Our contribution is to repurpose it as a black-box posterior audit and correction rather than a new reconstructor or learned sampler. Theoretically, we prove that the ideal transform confines pairwise sample differences to the MRI null space and bound the residual cross-subspace coupling left by practical sensitivity-weighted implementations. Across six base samplers and two MRI anatomies, including out-of-distribution transfer where a knee prior reconstructs
brain, MSC substantially reduces measured-subspace dispersion for Soft samplers (a median 16.5x reduction for DPS across five
brain contrasts, up to ~29x), while preserving unmeasured-subspace diversity and acting as a near-identity map for Consistent ones. Furthermore, MSC maintains or modestly improves PSNR/SSIM, with no retraining, retuning, or significant computational overhead.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
A Zero-Shot Deep Image Prior Framework for Denoising and Deconvolution in Fluorescence Microscopy
Xiangyu Qian, Jing Liu, Yunqing Tang et al. (+2 more)
Fluorescence microscopy images are degraded by noise and diffraction-induced blur, which compromise structural fidelity and limit quantitative analysis. Supervised deep learning methods achieve impressive restoration performance but require large-scale paired
datasets that are difficult to obtain in practice. To address this issue, we propose SDIP, a zero-shot deep image prior (DIP) framework that sequentially performs denoising and deconvolution without external training data. An aSeqDIP-based module first suppresses noise while preserving fine structures through sequential autoencoding regularization. In the deconvolution stage, a wavelet-based background correction step is incorporated before the proposed RLG-DIP module performs artifact-reduced deconvolution. RLG-DIP uses the Richardson-Lucy deconvolution result as a physically consistent guidance prior, integrating the imaging model with the implicit prior of DIP to stabilize the ill-posed deconvolution process. Experiments on the BioSR
dataset across multiple cellular structures demonstrate that SDIP improves both signal-to-noise ratio and resolution, achieving superior visual quality and improved quantitative performance on most evaluated structures. The proposed framework may also provide useful insights for designing physically guided DIP methods for other inverse problems.
Read the full abstract ▾
All authors Xiangyu Qian, Jing Liu, Yunqing Tang, Luru Dai, Qiushi Li
Computer Vision · Cross-listed from cs.RO
DexCompose: Reusing Dexterous Policies for Multi-Task Manipulation with a Single Hand
Dihong Huang, Zhenyu Wei, Zhuxiu Xu et al. (+3 more)
Dexterous manipulation policies can solve individual skills, but composing them to perform multiple tasks with a single hand remains challenging. Adding a new task on top of an existing manipulation skill often imposes conflicting demands on overlapping fingers and contact modes, causing destructive interference between preserving an existing manipulation outcome and executing a new one. We propose DexCompose, a role-aware residual composition framework that reuses pretrained dexterous policies for multi-task manipulation through explicit finger-level action ownership. Given two pretrained full-hand policies, DexCompose first collects successful post-task states from the first skill and performs release tests over candidate finger masks to identify which fingers are necessary for maintaining the established skill state. It then trains two asymmetric residual modules: a bounded residual stabilizer for task preservation, and a context-aware residual that adapts the frozen downstream policy only within the action subspace assigned to the new task. We evaluate the framework on 16 composite dexterous manipulation tasks spanning four object-retention skills and four downstream interactions. DexCompose achieves a 77.4% average composite success rate, demonstrating that structural action ownership with dual residuals offers a promising direction for composing dexterous skills beyond conventional policy chaining.
Read the full abstract ▾
All authors Dihong Huang, Zhenyu Wei, Zhuxiu Xu, Yunchao Yao, Sikai Li, Mingyu Ding
Computer Vision
PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception
Yana Wei, Hongbo Peng, Yanlin Lai et al. (+14 more)
We introduce PerceptionRubrics, a rubric-based evaluation framework that addresses the gap between saturated
benchmark scores and real-world brittleness. Shifting evaluation from holistic semantic matching to rigorous atomic
auditing, PerceptionRubrics pairs 1,038 information-dense images with over 10,000 instance-specific rubrics. These criteria are derived from golden captions constructed via a novel Circular Peer-Review consensus pipeline and then distilled into a dual-stream system of Must-Right (essential facts) and Easy-Wrong (fine-grained details) rubrics. Crucially, PerceptionRubrics implements a Gated Scoring mechanism: unlike linear averages, failure on mandatory visual facts triggers sharp binary penalties. Extensive evaluation yields critical insights: (1) The Reliability Gap: models often verify fragmented elements correctly yet fail strict conjunctive constraints, exposing brittleness in dense domains; (2) Open-Closed Stratification: contrary to
reasoning trends, we reveal a persistent 8% perception deficit between open-source and proprietary frontiers; and (3) Human-Aligned Rigor: our gated metrics substantially out-align conventional
benchmarks, validating that strict perceptual fidelity is the prerequisite for reliable generation.
Read the full abstract ▾
All authors Yana Wei, Hongbo Peng, Yanlin Lai, Liang Zhao, Kangheng Lin, En Yu, Keyu Lv, Han Zhou, Yin Tang, Haodong Li, Mitt Huang, Hangyu Guo, Jianjian Sun, Zheng Ge, Xiangyu Zhang, Daxin Jiang, Vishal M. Patel
Computer Vision
StructSplat: Generalizable 3D Gaussian Splatting from Uncalibrated Sparse Views
Jia-Chen Zhao, Beiqi Chen, Xinyang Chen et al. (+2 more)
We present StructSplat, a feed-forward and generalizable 3D Gaussian reconstruction framework that operates directly on uncalibrated images without requiring camera parameters. Existing methods either rely on per-scene optimization or assume known camera poses, and often entangle geometry and appearance within a unified backbone, limiting reconstruction fidelity and generalization. Our key idea is to adopt a structured representation that organizes geometry, semantic, and texture cues with explicit roles in the reconstruction process. Specifically, we introduce a pixel-aligned feature injection mechanism to enable accurate texture modeling from 2D observations, incorporate semantic-aware priors to improve global consistency, and design a camera
alignment strategy to prevent information leakage and improve generalization. Experiments show that our method significantly outperforms prior approaches on challenging
benchmarks. On DL3DV, our method achieves 28.045 PSNR, surpassing AnySplat (22.377) by +5.67 dB. In cross-
dataset evaluation, our method achieves +1.94 dB over AnySplat on ACID and +1.72 dB on RealEstate10K. Project page: https://structsplat.github.io Code: https://github.com/J-C-Zhao/StructSplat
Read the full abstract ▾
All authors Jia-Chen Zhao, Beiqi Chen, Xinyang Chen, Guangcong Wang, Liqiang Nie
Computer Vision
Learning Topology-Aware Representations via Test-Time Adaptation for Anomaly Segmentation
Ali Zia, Usman Ali, Abdul Rehman et al. (+5 more)
Test-time adaptation (TTA) has emerged as a promising paradigm for mitigating distribution shifts in deep models. However, existing TTA approaches for anomaly
segmentation remain limited by their reliance on pixel-level heuristics, such as confidence thresholding or
entropy minimisation, which fail to preserve structural consistency under noise and texture variation. Moreover, they typically treat anomaly maps as flat intensity fields, ignoring the higher-order spatial relationships that characterise complex defect geometries. We introduce TopoTTA (Topological Test-Time Adaptation), a novel framework that integrates persistent homology, a tool from topological data analysis, into the TTA pipeline to enforce
geometric and structural coherence during adaptation. By applying multi-level cubical complex filtration to anomaly score maps, TopoTTA derives robust topological pseudo-labels that guide a lightweight test-time classifier, enhancing
segmentation quality without retraining the backbone model. The approach avoids reliance on method-specific raw-score thresholding for mask binarisation, preserves connectivity, and generalises across both 2D and 3D modalities. Extensive experiments across six standard
benchmarks (MVTec AD, VisA, Real-IAD, MVTec 3D-AD, AnomalyShapeNet, and MVTec LOCO) demonstrate an average 15% F1 improvement over state-of-the-art unsupervised anomaly detection and
segmentation methods, with the largest gains on anomalies exhibiting complex
geometric or structural variations. These findings suggest that integrating topological
reasoning into test-time adaptation provides a principled route to structure-aware generalisation, bridging the gap between
geometric learning and robust adaptation.
Read the full abstract ▾
All authors Ali Zia, Usman Ali, Abdul Rehman, Umer Ramzan, Kang Han, Muhammad Faheem, Shahnawaz Qureshi, Wei Xiang
Computer Vision
RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning
Yelin Wang, Zijia Song, Shuo Ye et al. (+6 more)
Remote Sensing Image Change Captioning (RSICC) aims to describe changes between bi-temporal remote sensing images and holds significant research and application value. However, most existing methods rely on conventional deep learning architectures, and the limited model capacity constrains performance. Although large-model post-training techniques have achieved great success in general domains, their direct transfer to RSICC remains challenging due to data scarcity and the need for fine-grained change understanding. To address this, we propose RSICCLLM, the first post-training framework for large
vision-
language models in RSICC. Specifically, we design a data generation paradigm, release the instruction
dataset RSICI, and establish a task-specific RSICC
benchmark. We further introduce Difference-aware Supervised Fine-tuning to explicitly extract change representations and guide the model in perceiving and understanding temporal differences. In addition, we propose Dual-Negative Preference Optimization (DNPO), which employs two complementary negative-sample construction strategies to construct the preference
dataset RSICP and further refine model performance. Extensive experiments validate the superior capability of RSICCLLM, which achieves outstanding results with only 7B parameters, surpassing models of substantially larger scales. The code and
dataset will be made publicly available at https://github.com/keaill/RSICCLLM.
Read the full abstract ▾
All authors Yelin Wang, Zijia Song, Shuo Ye, Chuanguang Yang, Miaoyu Wang, Yong Xu, Zhulin An, Yongjun Xu, Zitong Yu
Computer Vision
Exposure Bias Can Alleviate Itself via Directional and Frequency Rectification in Flow Matching
Guanbo Huang, Jingjia Mao, Fanding Huang et al. (+9 more)
Flow Matching (FM) has achieved remarkable generative performance, yet it suffers from exposure bias due to discrepancies between training and inference. Existing mitigation strategies typically rely on static constraints or external heuristics. In this work, we propose that exposure bias itself inherently contains dynamic signals that can guide its own rectification. To leverage this, we introduce DEFAR (DirEctional-Frequency Adaptive Rectification). This framework simulates the single-step inference process during training to identify exposure bias. It utilizes directional and frequency-adaptive feedback signals from the bias itself to enhance the model's bias tolerance. It consists of two key components: (1) Anti-Drift Rectification (ADR). ADR treats inference-time drift as a signal to learn the direction to steer deviated states back toward the target. ADR endows the model with intrinsic active self-rectification capabilities; (2) Frequency Compensation (FC). Empirically, we observe that accumulated bias often stems from a lack of low-frequency components in high-noise stages, and exposure bias carries the missing frequency. FC leverages the bias itself as a self-feedback weighting factor to reinforce the missing frequency components. Experiments on CIFAR-10, CelebA-64, and ImageNet-256/512 show that DEFAR outperforms prior baselines and further demonstrates favorable scalability, compatibility, and inference robustness.
Read the full abstract ▾
All authors Guanbo Huang, Jingjia Mao, Fanding Huang, Fengkai Liu, Xiangyang Luo, Yaoyuan Liang, Jiasheng Lu, Xiaoe Wang, Pei Liu, Ruiliu Fu, Ruqi Huang, Shao-Lun Huang