Computer Vision
What Color is the Sky (for a non-human) ?
Yair Weiss and Ofer Springer
The light of the daytime sky contains a mixture of many colors yet is perceived as blue by human observers. This is largely due to the particular response functions of the human cones. Under these response functions skylight and blue light are metamers: they yield the exact same excitation of the cones. In this paper we ask: is it possible to define the “color” of the sky for other visual systems? We present a simple computational method to determine monochromatic metamers to a given input light for arbitrary visual systems. Using published values on spectral sensitivity functions of various species, we use our method to determine the dominant wavelength of monochromatic metamers to skylight. For a wide range of species (bichromats, trichromats and tetrachormats) we find monochromatic metamers to skylight but the dominant wavelength of the metamer can vary drastically between species and be very different from the color perceived by humans.
Read the full abstract ▾
Computer Vision
Projection-based coupling of infrared thermography and stereocorrelation-based digital image correlation
Jendrik-Alexander Tröger, Lutz Müller-Lohse and Stefan Hartmann
Full-field measurement techniques such as digital image correlation and infrared thermography are prevalent in experimental solid mechanics. Digital image correlation is used to analyze surface deformation, while infrared thermography quantifies surface temperature fields. However, sophisticated procedures are necessary to express both
datasets in the same Lagrangian frame, especially when analyzing non-flat surfaces. In this study, we propose an external projection-based coupling that uses the pinhole camera model to relate two-dimensional temperature data measured by infrared thermography to three-dimensional point coordinates from stereocorrelation-based digital image correlation. Unlike existing multiview approaches, we utilize two independently calibrated industrial-grade systems and augment the experimental evaluation with the pinhole camera model. The projection
matrix of the camera model is calibrated using a single image of a reference object. Through this projection, temperature fields are accurately represented at material points. Our method is particularly suited for, but not restricted to, curved surfaces and is straightforward to embed in existing experimental protocols, as the image registration is kept as is. Additionally, we use radial basis functions as a global interpolation ansatz in both space and time to compute in-plane temperature gradients and even temperature rates on curved surfaces, thereby providing an extensive and information-rich full-field
dataset.
Read the full abstract ▾
Computer Vision
On Test-Time Scaling for Vision-Language Models
Fawaz Sammani, Tzoulio Chamiti and Nikos Deligiannis
Test-time scaling is a paradigm where large models use additional compute at inference to achieve better performance, without changing model weights. While it has been widely studied for Large
Language Models (LLMs), its applicability to Large
Vision-
Language Models (LVLMs) remains less explored and analyzed, with limited analysis of whether, when, and to what extent these approaches transfer to LVLMs. In this work, we ask a simple but fundamental question: can conventional test-time scaling methods developed for LLMs be directly applied to LVLMs? We present the first comprehensive study of test-time scaling for LVLMs, spanning multiple models and model sizes, nine test-time scaling methods, and six diverse
benchmarks. Our main findings is that 1) different from previous findings, small, well-performing models benefit the most from test-time scaling, enabling performance improvements of up to around 30%, reaching large models performance, and often outperforming them, 2) LVLMs lose focus when given more compute than necessary, and 3) Visual information is encoded early in the
reasoning chain, after which the chain is dominated by text-only
reasoning and the contribution of image
tokens drops significantly. Finally, we also provide a global and fine-grained analysis on the quality and information sufficiency of the
reasoning chains produced. Overall, our findings and analysis provide practical guidance and insights into LVLMs and their deployment in research and industry.
Read the full abstract ▾
Computer Vision
HKVLM: Faithful Query–Region Binding for Frozen-Detector Visual Grounding
Bo Ma
Visual grounding often fails even when the target object is present in the proposal pool, because the
language-side referent is bound to the wrong region. We study this binding failure under frozen perception and ask whether an explicit query–region
alignment hook, together with a perception-grounded abstention mechanism, can improve faithful grounding without retraining the detector or the
vision-
language backbone. HKVLM freezes a
language-aligned open-vocabulary detector for localization and learns a lightweight hook that maps referential query
embeddings to detector proposals in a shared space; a verifier abstains when no region sufficiently supports the query. We prove an exact proposal-level diagnostic decomposition, $(1-\mathrm{SeeErr})(1-\mathrm{SayErr})$, separating proposal-coverage failures from conditional binding failures, and a monotonicity result that characterizes the faithfulness–recall trade-off induced by abstention. Across RefCOCO, RefCOCO+, RefCOCOg, and POPE, HKVLM improves over untrained and trained matched-perception binding controls and substantially reduces
hallucination through abstention. Strong coordinate-decoding and end-to-end fine-tuned baselines remain much higher in raw grounding accuracy, and a
reasoning-stress set exposes binding as the main current bottleneck. We therefore present HKVLM as a diagnostic and mechanism-level study of query–region binding under frozen perception, not as an absolute localization leader.
Read the full abstract ▾
Computer Vision
EpiSAM: Character Segmentation in Challenging Stone Inscriptions
Arnav Sharma, Pratyush Jena, Amal Joseph et al. (+1 more)
Stone inscriptions are invaluable sources of historical and linguistic knowledge, yet their automated analysis remains a major challenge due to surface irregularities, erosion, and low visual contrast. Conventional document and handwriting analysis techniques fail to perform well in these scenarios. In this work, we propose character detection as a core strategy for robust inscription analysis. We introduce EpiSAM, a
prompt-guided
transformer framework for character
segmentation in stone inscriptions. Rather than treating characters in isolation, EpiSAM employs a novel neighbor-aware strategy, explicitly predicting adjacent characters alongside the target. These contextual cues resolve boundary ambiguities, improving mask generation and enabling more accurate character
segmentation. Furthermore, we expand an existing stone inscription
dataset by adding dense polygonal annotations for characters, thereby enabling comprehensive research on Southeast Asian epigraphy. Experimental results show that EpiSAM achieves consistent improvements over existing baselines, while also exhibiting strong zero-shot generalization in challenging epigraphic scenarios.
Read the full abstract ▾
All authors Arnav Sharma, Pratyush Jena, Amal Joseph, Ravi Kiran Sarvadevabhatla
Computer Vision
Personalizing MLLMs via Reinforced Multimodal Reference Game
Deepayan Das, Davide Talon, Yiming Wang et al. (+2 more)
Personalizing
Multimodal Large
Language Models (MLLMs) aims to recognize users' unique concepts from visual data and provide personalized responses. Although prior work has shown the benefit of concept descriptions and
reasoning for this task, MLLM descriptions often include information, such as state and context, that does not help and may in fact hinder the unique identification of the target concept among other visually similar items. Effective descriptions of personal concepts should instead be accurate, discriminative, and free of distracting details. To achieve such descriptions, we introduce Reinforced Reference Game (RRG), a learning framework that promotes discriminative descriptions through a novel reinforced
multimodal reference game. The MLLM plays both the roles of speaker and listener in a contrastive game setting, whose goal is to effectively communicate discriminative information about a target concept. Our approach formulates a verifiable contrastive reward over hard positives (dissimilar views of the same concept) and hard negatives (visually similar but different concepts). Empirically, RRG achieves state-of-the-art across multiple tasks on three personalization
benchmarks. RRG generalizes to unseen domains and outperforms existing methods based on concept descriptions and personalization-specific RL frameworks. We will release code and models in the project page.
Read the full abstract ▾
All authors Deepayan Das, Davide Talon, Yiming Wang, Massimiliano Mancini, Elisa Ricci
Computer Vision
DLGStream: Dynamic Language-embedded Guassian Splatting for Open-vocabulary Enabled Free-viewpoint Video Streaming
Zhihui Ke, Yuyang Liu, Xiaobo Zhou et al. (+1 more)
3D Gaussian Splatting~(3DGS) has emerged as a promising paradigm for reconstructing streamable free-viewpoint
video~(FVV) from multi-view
videos. However, 3DGS-based FVVs typically lack user interaction and editing capabilities, which diminishes the immersive experience. Recent research has integrated
language features from CLIP into 3DGS via
distillation, enabling open-vocabulary queries and supporting many downstream applications. Nevertheless, the stringent requirements of FVV, low frame size and high FPS, make current
language Gaussian representations unsuitable for
language-embedded FVV. In this paper, we propose DLGStream, a novel
language-embedded FVV representation that streams time-varying
language features alongside Gaussian attributes to support 4D environment interaction, scene editing, and spatial intelligence. Specifically, we propose a dual-opacity dynamic
language Gaussian representation, which maintains two opacity attributes for color and
language features to deal with performance degradation that occurs when colors and features are jointly optimized. Furthermore, we introduce an interpolation-based deformation field to reduce temporal redundancy. This deformation field can also be used for 4D frame interpolation, boosting FVV sequences from low to high FPS. Experimental results demonstrate that DLGStream achieves superior performance in both on open-vocabulary
segmentation and reconstruction quality with an average frame size of merely 43 KB. The code is available on \href{https://github.com/kkkzh/DLGStream}{https://github.com/kkkzh/DLGStream}.
Read the full abstract ▾
All authors Zhihui Ke, Yuyang Liu, Xiaobo Zhou, Tie Qiu
Computer Vision
Ground4D: Consistency-Aware 4D Reconstruction from Monocular Video
Qing Zhao, Weijian Deng, Pengxu Wei et al. (+1 more)
Learning a 4D scene representation from a single
monocular video that supports dynamic novel-view synthesis while maintaining faithful geometry over time remains challenging. Dynamic Gaussian Splatting achieves strong rendering performance through photometric optimization, yet does not explicitly enforce multi-view
geometric consistency. In contrast, 3D foundation models recover coherent scene geometry and camera motion, but their point-based outputs are not designed for photorealistic rendering. We propose Ground4D, a geometry-grounded framework built on two stages. First, we perform geometry initialization via 3D foundation models, leveraging VGGT in a training-free manner to reconstruct multi-view-consistent 3D geometry and camera poses from
monocular video. The recovered geometry provides a structured and reliable initialization for dynamic Gaussian representations. Second, we conduct geometry-consistency-aware refinement via dynamic Gaussian Splatting, optimizing the representation through differentiable rendering while maintaining multi-view
geometric consistency across both observed and synthesized viewpoints. Furthermore, Ground4D inherently models the continuous 4D dynamics of the scene, naturally supporting rendering at arbitrary timestamps. By integrating foundation-level
geometric priors into dynamic Gaussian optimization, Ground4D achieves stronger reconstruction fidelity and rendering performance, underscoring the role of geometry-grounded constraints in robust 4D scene modeling.
Read the full abstract ▾
All authors Qing Zhao, Weijian Deng, Pengxu Wei, Liang Lin
Computer Vision
RefGlass-GS: A UAV-Enabled Fusion Framework for Photorealistic, Semantic and Interactive Digitization of Reflective Glass Facades via Gaussian Splatting
Zhenyu Liang, Xiao Zhang, Boyu Wang et al. (+5 more)
Existing digitization of buildings with reflective glass facades suffers from
geometric reconstruction distortion, unrealistic view-dependent texture rendering, and difficulties in object-based semantic enhancement. Therefore, we propose RefGlass-GS, a fusion framework that enables end-to-end
UAV-based photorealistic, semantic, and interactive digitization of reflective glass facades. The contributions include: (1) proposing an individual glass panel
segmentation method based on maximum a posteriori estimation with structural regularities, robust to severe reflection and background interference; (2) formulating a
UAV viewpoint planning optimization function that maximizes the coverage of view-dependent appearance for sufficient data capture; (3) developing an optimized Gaussian Splatting framework with a Reflection MLP, a novel deferred shading function, and two enhanced regularization terms for effective modeling of high-frequency near-field reflections; (4) introducing a standardized data organization paradigm for structuring GS-based representations into object-based models, facilitating interactive facility management on digital twin platforms. Experiments on real-world reflective glass facade scenes validate the effectiveness and superiority of the proposed method. Specifically, the glass panel
segmentation achieves an improvement of 0.1927 in mIoU over SOTA methods, and only our method enables instance-level panel extraction. The
UAV view planning improves novel view synthesis for reflective facades by 13.15 dB in PSNR compared to commercially used nap-of-the-object planning methods. The RefGlass-GS modeling outperforms SOTA Gaussian Splatting approaches for reflective scenes with an average improvement of 5.08 dB in PSNR.
Read the full abstract ▾
All authors Zhenyu Liang, Xiao Zhang, Boyu Wang, Zhaolun Liang, Ang Li, Jeff Chak Fu Chan, Mingzhu Wang, Jack C.P. Cheng
Computer Vision
CoGS: Compositional Dynamic Human-Object Scenes Gaussian Splatting from Monocular Video
Jerrin Bright and John Zelek
Reconstructing dynamic human–object interaction scenes from
monocular video is difficult because the human, manipulated object, and background obey different motion models while sharing the same pixels. Existing dynamic radiance-field and Gaussian-splatting methods often entangle these components, causing object motion to leak into the human or static scene, and
monocular human reconstruction remains underconstrained in regions that are rarely observed. We present CoGS, a compositional Gaussian-splatting framework for
monocular human–object scene reconstruction. CoGS decomposes the
video into three coordinated branches: an articulated human initialized from a complete canonical prior, a rigid object field driven by an estimated object trajectory, and a static scene field regularized by weak scene-only planar primitives when available. A six-stage optimization schedule first stabilizes the human and object independently, then fuses them with the scene under full-image supervision, visibility-aware human anchoring, object silhouette and motion constraints, and delayed scene regularization. This design keeps each component responsible for its own geometry and motion while allowing photometric evidence to correct the final composite. Experiments on HOSNeRF and NeuMan show that CoGS improves both human–object interaction reconstruction and in-the-wild human–scene rendering, achieving stronger fidelity and perceptual quality across full-frame and human-focused evaluations. Code will be released upon publication.
Read the full abstract ▾
Computer Vision
ViPSim: Collaborating Visual and Parameter Spaces for Consistent Long-Horizon Embodied World Models
Longyu Chen, Heng Li, Wei Yang et al. (+2 more)
Embodied World Models (EWMs) have emerged as a scalable and risk-free paradigm for advancing
embodied intelligence, enabling the safety-critical evaluation of
Vision-
Language-Action systems. However, their reliability as evaluation
benchmarks and foundational simulators is often hindered by the representation gap between low-dimensional actions and high-dimensional
video synthesis. This gap results in a lack of
geometric correspondence, manifesting as accumulated trajectory drift and inconsistent
robot-object interactions during long-horizon rollouts. To bridge this gap, we propose ViPSim, a framework that achieves consistent long-horizon generation through the synergistic collaboration of Visual and Parameter Spaces. We define the Visual Space as a domain of explicit spatial priors, integrating pixel-aligned projections of end-effector pose, camera perspectives, depth-informed scene geometry, and robotic morphological masks to provide dense structural grounding. Concurrently, the Parameter Space serves as a domain of numerical drivers, injecting raw action sequences and camera
matrices to provide precise motion guidance. By unifying these two spaces, ViPSim ensures that the generated states are simultaneously anchored by
geometric boundaries and steered by numerical commands. Extensive experiments demonstrate that ViPSim is backbone-agnostic and significantly enhances trajectory consistency. Notably, our approach exhibits emergent capabilities in generating complex interactions with deformable objects (e.g., cloth folding) and maintains robust performance in out-of-distribution and cross-embodiment scenarios, providing a high-fidelity foundation for the automated evaluation and predictive control of
embodied agents.
Read the full abstract ▾
All authors Longyu Chen, Heng Li, Wei Yang, Manqi Zhao, Dongsheng Jiang
Computer Vision
PSP: Harnessing Position and Shape Priors for Cross-Domain Few-Shot Medical Image Segmentation
Bin Xu, Yazhou Zhu and Haofeng Zhang
Few-Shot Medical Image
Segmentation (FSMIS) offers a powerful solution to data scarcity but struggles to generalize across different imaging modalities. This performance collapse stems primarily from the drastic texture discrepancies between domains, which mislead models trained on source-specific intensity distributions. While existing methods attempt to align frequency or local texture features, they often fail to decouple semantic structure from domain-specific appearance. To address this, we identify a critical invariance: despite distinct imaging physics, the position and
geometric shape of organs remain robustly consistent across modalities. Therefore, we propose a novel framework that harnesses Position and Shape Priors (PSP) for cross-domain FSMIS. Specifically, PSP first introduces a Position Coordinate
Embedding (PCE) module to inject relative spatial coordinates for rapid organ localization. Subsequently, a Shape Prototype Modulation (SPM) module constructs domain-invariant structural prototypes via explicit shape priors, effectively filtering out texture noise. Furthermore, the Hybrid-Prototype Prediction (HPP) module adaptively calibrates the support prototype to the query feature distribution, mitigating feature misalignment. Extensive experiments on two public medical imaging
datasets demonstrate that PSP significantly outperforms state-of-the-art methods.
Read the full abstract ▾
Computer Vision
Virtual Ring Try-On
Vishnu D. Burkhawala, Zankhana J. Barad, Harshadkumar B. Prajapati et al. (+1 more)
This paper presents an innovative approach that enables the users to capture their hand and try the jewel ring on their hand. The user captures the image of the hand using the React Native base GUI of the mobile application and selects the ring that the user wants to try, and the output image will have the user's hand with the ring image. This approach is implemented using a combination of MediaPipe hand point detection and YOLO-V8 custom object detection. The hand image uploaded by the user first undergoes mediapipe hand point detection. It will give the hand points and a Region of Interest mask where the ring is going to be placed. Then the ring is passed through YOLO object detection, in which ring points are detected, and background is removed. After that, using vector
algebra, the angular discrepancy between the finger's reference axis and the ring's principal axis is computed. Also, ring size is rescaled according to finger thickness, preserving the aspect ratio to maintain perceptual realism. Then the ring is placed on the hand image and the output image is generated and shown on the user screen.
Read the full abstract ▾
All authors Vishnu D. Burkhawala, Zankhana J. Barad, Harshadkumar B. Prajapati, Vipul K. Dabhi
Computer Vision
BREIT: A Framework for Brain Stroke Reconstruction using Multi-Frequency 3D EIT
Djahid Abdelmoumene, Ishak Ayad, Maï K. Nguyen et al. (+1 more)
Multi-Frequency Electrical Impedance
Tomography (MF-EIT) is a non-invasive, low-cost modality that reconstructs electrical property distributions from boundary voltages. For stroke imaging, progress in 3D deep-learning reconstruction is limited by the lack of large-scale
datasets with paired ground-truth (GT) volumes and by non-standardized pipelines for data generation, simulation, and evaluation. We introduce BREIT, a modular framework for 3D MF-EIT stroke reconstruction providing: (i) a neuroimaging-to-EIT pipeline that converts CT/MRI into frequency-dependent GT admittivity volumes; (ii) a self-contained Python 3D Complete Electrode Model (CEM) forward solver for simulating MF-EIT voltages; and (iii) a 3D D-bar implementation supporting non-uniform electrode layouts. Building on BREIT, we propose dFNO-bar, which integrates
Fourier Neural Operators into D-bar by learning a mapping from scattering data $t(\xi)$ to conductivity $\sigma(x){=}\Re\{\gamma\}$. We evaluate dFNO-bar against D-bar, Deep D-bar, and Gauss–Newton reconstructions on UCLH-matched synthetic data, and observe higher
brain SSIM with comparable CC across noise settings.
Read the full abstract ▾
All authors Djahid Abdelmoumene, Ishak Ayad, Maï K. Nguyen, Christian Daveau
Computer Vision
Stochastic Optimal Control Sampling for Diffusion Inverse Problems
Jie Zhang, Youmei Qiu, Hanling Tian et al. (+3 more)
Benefiting from the strong ability to capture data distributions,
diffusion models have become powerful tools for solving image inverse problems. The key is to controllably steer the sampling trajectory toward the measurements while respecting the
diffusion prior. In this work, we introduce
Stochastic Optimal Control Sampling (SOCS), which models the denoising process as a dynamical system and injects control signals via SOC. Previous SOC-based approach addresses inverse problems by optimizing over the entire trajectory, which is computationally expensive. In contrast, we derive a closed-form control update and apply it at each sampling step, pulling the measurement-consistent clean prediction back onto the denoising flow. In SOCS, we can readily modulate the control strength to align with the
diffusion model's native capabilities and thereby enhance perceptual quality. Our method is compatible with a variety of linear
stochastic differential equation backbones. Extensive experiments across a broad spectrum of image inverse tasks demonstrate that SOCS achieves accurate measurement-aligned reconstructions with improved visual fidelity and stronger quantitative performance.
Read the full abstract ▾
All authors Jie Zhang, Youmei Qiu, Hanling Tian, Jingyuan Zhang, Xiang Yin, Xiaolin Huang
Computer Vision · Cross-listed from cs.IR
Multimodal Graph RAG for Long-range Visually Rich Document Understanding
Yi-Cheng Wang and Chu-Song Chen
Multimodal large
language models (MLLMs) are widely applied to visual document understanding. However, comprehending long documents remains an issue by the limited context window. Though recent
multimodal retrieval-augmented generation (MMRAG) can address this challenge by retrieving relevant pages. It still struggles with the visual question answering (VQA) requiring holistic comprehension of a document. To cope with this, knowledge graph (KG) that summarizes global knowledge of a document can provide an effective solution. However, most existing LLM-based KG construction methods handle only the
language modality, leaving the automatic creation of
multimodal KGs (MMKGs) for visually rich documents largely unexplored. In this paper, we introduce a
multimodal graph-based RAG approach to tackle this problem. Existing LLM-based KG methods evaluate the QA performance relying on indirect evidence such as comprehensiveness, diversity, empowerment, and so on. The lack of annotated
datasets for comprehensive document-level VQA poses a significant challenge to effective model evaluation. To overcome this limitation, we also introduce a new
benchmark, DLVQA (document-level VQA), which provides reference summaries and corresponding supporting facts for global document-level questions. Experimental results show that our approach outperforms existing MMRAG or KG-based approaches on multi-hop QA/VQA
benchmarks and DLVQA.
Read the full abstract ▾
Computer Vision
X-Mind: Efficient Visual Chain-of-Thought via Predictive World Model for End-to-End Driving
Bohao Zhao, Chengrui Wei, Guangfeng Jiang et al. (+17 more)
Predicting future states is essential for autonomous
agents, yet current
Vision-
Language-Action (VLA) models fundamentally lack this capability, relying instead on reactive perception-action mapping. While integrating Predictive World Models (PWMs) addresses this gap, existing approaches either incur prohibitive cascaded latency or act as shallow terminal tasks that fail to deeply embed forward-looking
reasoning. To endow VLA models with this
reasoning capability, we propose X-Mind. Rather than treating PWMs as an external auxiliary module, this framework internalizes them as the Visual Chain-of-Thought (Visual CoT). By enforcing a world rollout prior to action, the model is constrained to imagine future
evolution first, yielding a driving policy that is robustly grounded in environmental dynamics and aware of the future consequences its actions will unfold. The challenge here is efficiency, and we tackle it on two fronts. First, we introduce a compact representation of visual thinking: an abstract sketch that fuses a Bird's-Eye-View (BEV) layout with abstract driving priors (e.g.,
navigation intents and traffic rules). Rather than rolling out dense future frames, the model reasons over this sketch as a mental canvas; aided by a Deep Compression Autoencoder (DC-AE), a 12-frame future rollout is reduced to merely 96
tokens, alleviating the long-context computational bottleneck. Second, to accelerate generation further, we propose a recurrent block
diffusion scheme that unrolls the denoising steps across the layers of the large drive model, folding iterative refinement into the backbone's one forward pass. Trained and validated on large-scale real-world data, X-Mind achieves competitive end-to-end driving performance, which makes it a highly practical, low-latency solution that successfully deploys large-scale cognitive
reasoning directly onto resource-constrained vehicle platforms.
Read the full abstract ▾
All authors Bohao Zhao, Chengrui Wei, Guangfeng Jiang, Ruixin Liu, Xuejie Lv, Liu Liang, Sutao Deng, Xiuyang Fan, Pengkun Zheng, Jinyun Zhou, Rui Guo, Hanpeng Liu, Yutong Zheng, Yi Guo, Xinlong Zheng, Qingyu Luo, Zhuangzhuang Ding, Yu Zhang, Hang Zhang, Xianming Liu
Computer Vision
A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models
Nuo Chen, Lulin Liu, Zihao Li et al. (+12 more)
Generative world models hold immense promise as scalable simulators for autonomous systems, particularly for synthesizing rare but safety-critical multi-
agent interactions, such as vehicle collisions. However, current evaluation paradigms index heavily on visual fidelity and semantic
alignment, leaving a critical blind spot: they cannot reliably quantify whether generated dynamics actually obey the fundamental physical laws required for reliable simulation. Assessing this physical plausibility is inherently difficult due to a lack of physical metrics and the challenge of extracting metric-scale kinematics from uncalibrated
video rollouts. To bridge this gap, we introduce CrashTwin, a physics-grounded evaluation framework designed to stress-test the physical trustworthiness of world models. CrashTwin couples a diverse
dataset of multi-
agent collision scenarios, comprising 25K controllable synthetic and 12K in-the-wild real-world collision sequences with a novel calibration-free reconstruction pipeline, enabling the recovery of 3D physical attributes directly from world model rollouts. We propose a diagnostic suite that systematically evaluates three dimensions: spatio-temporal consistency, momentum and kinetic energy conservation, and world-dynamics integrity. Extensive benchmarking of state-of-the-art models reveals a crucial insight: high perceptual quality frequently masks severe physical violations during complex interactions. By quantitatively exposing these failure modes, CrashTwin provides a vital diagnostic tool for developing physically grounded world models capable of reliable real-world simulation.
Read the full abstract ▾
All authors Nuo Chen, Lulin Liu, Zihao Li, Ziyao Zeng, Zihao Zhu, Wenyan Cong, Junyuan Hong, Yunhao Yang, Zhengzhong Tu, Yan Wang, Boris Ivanovic, Marco Pavone, Zhangyang Wang, Yang Zhou, Zhiwen Fan
Computer Vision · Cross-listed from eess.IV
BLUE: A Stale-Pixel Optical-Flow Compositor for Entropy-Efficient Surveillance Video Encoding
Shubham Baid, Akash James, Sahil Chachra et al. (+2 more)
Continuous-recording surveillance systems face a storage problem that codec tuning alone cannot fully solve: even at aggressive CRF settings, a static-camera scene spends most of its bits re-encoding a background that has not changed. We present BLUE, a pre-encode compositor that exploits this structure by maintaining a persistent seed frame of the background and substituting background pixels with seed pixels before the encoder runs. The encoder then emits near-free SKIP macroblocks for the frozen background, while live pixels in foreground regions are carried unchanged at full quality. We evaluate BLUE on all 308 annotated short subclips from the VIRAT Ground Surveillance Release 2.0
dataset using a six-point CRF sweep with both x264 and x265. At CRF 28, BLUE reduces file size by a mean of 34.6% (x264) / 39.4% (x265) on 95.8% / 99.4% of clips respectively. Foreground-region PSNR, computed only over VIRAT object-annotation bounding boxes, is preserved or improved on 60.7% of clips (+0.36 dB mean, +5.48 dB maximum). Full-frame perceptual quality (VMAF) drops by a median of 6.75-8.59 points; we quantify and disclose this trade-off explicitly. A lightweight deployment gate measuring the compositor's own VMAF on a 2-second prefix identifies the 40% of clips where even full-frame quality degradation is near-imperceptible (Delta VMAF <= -2.9), enabling a selective-activation strategy that retains both the storage benefit and acceptable perceptual fidelity.
Read the full abstract ▾
All authors Shubham Baid, Akash James, Sahil Chachra, Nishant Sinha, Kunal Kislay
Computer Vision
FreqOrtho-SR: Frequency-Guided Orthogonal Expert Learning for Real-World Image Super-Resolution
Minh Son Hoang, Dinh Phu Tran, Quyen Nguyen Duc et al. (+2 more)
Diffusion prior-based methods have shown impressive results in real-world image super-resolution (ISR), yet two key challenges persist: balancing pixel-level fidelity with semantic quality, and adapting to diverse degradations. Existing dual-branch approaches freeze the pixel module during semantic training, but the semantic branch can still expand capacity within the pixel subspace, precluding genuine perceptual improvement. Moreover, using a single static adapter cannot generalize across heterogeneous real-world corruptions. To address both issues, we propose FreqOrtho-SR, which comprises: $\textbf{Freq}$uency-guided Mixture of LoRA Experts (FreqMoE), it routes inputs to specialized experts via a non-parametric FFT-based degradation-feature extractor that encodes frequency-domain signatures, enabling stable and interpretable specialization across corruption types; and $\textbf{Ortho}$gonal Gradient Projection (OGP), which reframes the dual-objective optimization as a subspace-constrained problem: by extracting the pixel-fidelity subspace via SVD on combined expert weight deltas and projecting semantic gradients onto its null space, OGP guarantees orthogonality between the two objectives, enabling genuinely complementary learning without mutual interference. Experiments show that FreqOrtho-SR achieves competitive overall performance and a strong fidelity-perception trade-off across multiple
benchmarks with efficient single-step inference. The source code of our method can be found at $\href{https://github.com/sonhm3029/FreqOrtho-SR}{\texttt{sonhm3029/FreqOrtho-SR}}$.
Read the full abstract ▾
All authors Minh Son Hoang, Dinh Phu Tran, Quyen Nguyen Duc, Dam Hoang Phuong, Daeyoung Kim
Computer Vision
CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation
Jinhong Hu, Xiaoping Wang, Shuyin Huang et al. (+3 more)
Understanding and localizing subtle changes between paired images is critical for tasks such as surveillance and image editing. However, traditional Image Change Captioning (ICC) methods lack spatial grounding, limiting their precision. We introduce Image Change Captioning and
Segmentation (ICCS), a new
multimodal task that jointly requires structured change description and pixel-level localization. To address ICCS, we propose the Change-aware Captioning and
Reasoning Chain (CCRC), a dual-chain framework that decouples semantic
reasoning from spatial
segmentation. The first chain, Chain-of-Change-Captioning (CCC), enhances fine-grained change perception via a visual fusion module based on Multi-Head Change-aware
Attention inserted between the visual and
language components of a
Multimodal Large
Language Model (MLLM). CCC also determines whether a change is segmentable. If not, it alone generates the caption. Otherwise, the second chain, Chain-of-Change-Segmenting (CCS), is activated, leveraging spatial priors from CCC and refining masks with a Change-aware
Token Refiner for accurate boundary localization. We evaluate CCRC on both synthetic and real-world change detection
benchmarks with pixel-level supervision. Experiments show CCRC achieves state-of-the-art performance.
Read the full abstract ▾
All authors Jinhong Hu, Xiaoping Wang, Shuyin Huang, Guojin Zhong, Kaitai Liu, Kai Lu
Computer Vision
Mitigating Batch Effects in Histopathology via Language-Mediated Robust Embedding Generation
Yishu Zhang, Shushan Wu, Zhenzhong Zhang et al. (+8 more)
Pathology foundation models (PFMs) have demonstrated strong potential across clinical and scientific applications, yet their performance is often hindered by batch effects, which are non-biological variations across tissue source institutions (TSIs) that distort learned feature representations and impair generalization. Conventional mitigation strategies, such as stain normalization, offer limited success in addressing these high-dimensional, complex artifacts. We present GLMP (General-purpose LLM-Mediated Pathology model), a novel framework that generates robust numerical
embeddings from histology image patches through an intermediate textual representation. By leveraging pretrained general-purpose
multimodal large
language models (MLLMs) and text encoders, GLMP effectively prioritizes biologically meaningful signals over TSI-specific artifacts, thereby improving cross-institutional generalization. To our knowledge, GLMP is the first pathology model to use text descriptions of histological features as an intermediate representation for generating numerical
embeddings from histology images. Our results highlight the untapped potential of broad-domain, non-specialized MLLMs in computational pathology and introduce a new paradigm for building versatile, generalizable, and robust pathology models.
Read the full abstract ▾
All authors Yishu Zhang, Shushan Wu, Zhenzhong Zhang, Didong Li, Huaxiu Yao, Yun Li, Iain Carmichael, Katherine A. Hoadley, Hongtu Zhu, Di Wu, Daiwei Zhang
Computer Vision
LogiCo: A Unified Framework for Logical and Structural Anomaly Detection
Ximiao Zhang, Min Xu and Xiuzhuang Zhou
Current anomaly detection methods primarily focus on structural anomalies, while paying insufficient
attention to anomalies that violate logical constraints. Conversely, top-performing logical anomaly detection approaches address this by modeling global semantic consistency, but perform poorly on subtle structural anomalies due to inadequate detection granularity. In this paper, we propose LogiCo, a unified framework for Logical and structural anomaly detection via Component-level feature reconstruction. Unlike existing methods that rely on explicit global semantic modeling, LogiCo employs a novel component-level feature reconstruction technique to capture inter-component logical constraints. Specifically, LogiCo maps pre-trained image features into a discrete component-level feature space and performs collaborative feature reconstruction at both component and patch levels, enabling it to effectively detect both logical and structural anomalies. Furthermore, to address the specific challenge of count-related logical anomalies, we integrate a
segmentation-map discriminator that extends the model's capability to identify quantitative inconsistencies. LogiCo achieves state-of-the-art performance on both logical and structural anomaly detection across four
benchmarks, including MVTec-LOCO, MVTec-AD, VisA, and Real-IAD, demonstrating its superiority and practical feasibility. The code is available at https://github.com/cnulab/LogiCo.
Read the full abstract ▾
Computer Vision
SATB-VR: Training Few-Step Video Restoration Diffusion Model using SNR-Aware Trajectory Blending
Haoran Bai, Xiaoxu Chen, Xiaoyu Liu et al. (+4 more)
While
diffusion models excel in
video restoration, their reliance on extensive iterative steps limits efficiency. Conversely, aggressive single-step
distillation often compromises fine texture recovery. To achieve an optimal balance, we present SATB-VR, a few-step paradigm that jump-starts the denoising process via an auxiliary predictor, explicitly bypassing early low signal-to-noise ratio (SNR) steps. However, naive joint training of the predictor and the denoiser inherently introduces a severe train-inference discrepancy. To resolve this, we propose the SNR-Aware Trajectory Blending (SATB) strategy. During the forward process, SATB constructs the noisy input by dynamically blending the predictor's output with the ground-truth trajectory based on the SNRs. This forces the denoiser to robustly compensate for initial prediction errors while smoothly converging to the clean data
manifold. Furthermore, we introduce a Denoiser-Driven Consistency (DDC) loss, leveraging the concurrently updated denoiser as a dynamic evaluator to explicitly align internal features and boost predictor accuracy. Extensive experiments demonstrate that, under flexible few-step inference regimes (\eg, $\le 5$ steps), SATB-VR performs favorably against existing approaches on synthetic, real-world, and AIGC
benchmarks.
Read the full abstract ▾
All authors Haoran Bai, Xiaoxu Chen, Xiaoyu Liu, Zongsheng Yue, Sibin Deng, Wangmeng Zuo, Ying Chen
Computer Vision
Predicting Metastatic Risk from Primary Cancer Tissue Architecture via Distance-Aware Spatial Modeling
Sandesh Pokhrel, Hamid Manoochehri, Beatrice S Knudsen et al. (+1 more)
Predicting distant metastasis from the digital H & E slides of the primary tumor is a critical yet challenging task in computational pathology. Multiple Instance Learning (MIL) approaches can attend to subdomains in whole slide images (WSIs) that harbor features of pre-metastatic cancer regions. However, conventional MIL models largely treat tissue patches as unordered bags, discarding the spatial layout that defines how these regions are arranged and interact across the tissue. We propose that metastatic risk is shaped not only by local patch appearance, but also by the
geometric organization of patches in the WSI and the interaction between the tissue compartments. To this end, we introduce Distance-aware Tissue Modeling for Multiple Instance Learning (DTMF-MIL), a spatial MIL framework that reinforces feature
embeddings with explicit distance priors. By computing signed distance functions (SDFs) to capture regions with similar features, and representing each patch with radial-basis distance responses and local SDF statistics, DTMF-MIL learns positions of patches with respect to regional interiors and boundaries. The interactions between similar patches are contextualized across local tissue neighborhoods and used to guide slide-level
attention while pooling patch feature evidence for metastasis prediction. We evaluate DTMF-MIL for prediction of distant prostate cancer metastasis in an internal prostate needle-biopsy cohort (IPC) from a large hospital system and on public TCGA-COAD and TCGA-KIRC
datasets across multiple pathology foundation-model backbones. Across most
dataset, backbone, and metric combinations, DTMF-MIL achieves the strongest results consistently.
Read the full abstract ▾
All authors Sandesh Pokhrel, Hamid Manoochehri, Beatrice S Knudsen, Tolga Tasdizen
Computer Vision
BackTranslation2.0 – A Linguistically Motivated Metric to Assess Sign Language Production
Oliver Cory, Maksym Ivashechkin, Karahan Sahin et al. (+6 more)
Sign
Languages (SLs) are the primary means of communication for millions of deaf individuals, yet existing evaluation metrics for generated SL remain simplistic and poorly aligned with human judgements. We introduce BackTranslation2.0, a linguistically grounded evaluation metric for text-to-sign translation that moves beyond naïve backtranslation. Our approach adopts an agentic framework in which a deterministic pipeline orchestrates a suite of specialised tools to assess four scoring dimensions - grammatical correctness, phonological accuracy, motion fluency, and generation fidelity - aligned with human rater assessments. Tool outputs are not treated independently: a set of large
language model (LLM)-based cross-referential comparison modules evaluates consistency across tools and checks outputs against linguistic expectations, enabling structured
reasoning over grammatical, phonological, and motion-level evidence. Final dimension scores are computed through deterministic weighted formulas over validated tool outputs. To validate BackTranslation2.0, we introduce and evaluate on a British Sign
Language (BSL)
dataset rated in a human rater study across the same quality dimensions, following a protocol developed in collaboration between linguists and deaf experts, benchmarking against six baseline metrics. Our method demonstrates strong correlation with human judgements across all dimensions, providing a more comprehensive, interpretable, and linguistically principled evaluation framework for sign
language production systems.
Read the full abstract ▾
All authors Oliver Cory, Maksym Ivashechkin, Karahan Sahin, Oline Ranum, Jianhe Low, Edward Fish, Anton Pelykh, Ozge Mercanoglu Sincan, Richard Bowden
Computer Vision
SemDynReg: Semantics-Guided Deformation Regularization for Dynamic 3D Gaussian Splatting
Ruitao Chen, Mozhang Guo and Jinge Li
Deformable 3D Gaussian Splatting (3DGS) has emerged as an efficient approach for rendering dynamic scenes in a wide range of 3D applications. However, existing deformation field-based approaches largely lack explicit object-level modeling, often resulting in inconsistent Gaussian deformations within individual objects and unwanted coupling between different objects. To address this limitation, we introduce a semantics-guided framework that enforces dynamic regularization at the object level, aiming to achieve spatially consistent object-wise deformation. Specifically, we first extract
segmentation masks using the Segment Anything Model (SAM) and derive semantic features from input images. An object-ID map is then constructed via feature relevance matching with a predefined object dictionary. Guided by this object-ID map, we identify the pixel-wise top-k contributing Gaussians for each object and impose consistency regularization on their deformation parameters, including position, scale, and rotation. Unlike prior methods that learn deformation fields without explicit object-level constraints, our approach incorporates semantic cues to guide deformation behavior at the object level. Experimental results demonstrate that our semantics-aware regularization improves object-level deformation consistency and outperforms baseline methods in rendering quality, achieving higher PSNR and SSIM and lower LPIPS in dynamic 3DGS rendering. Our project page is available at https://dyn-reg-3dgs.github.io/.
Read the full abstract ▾