Computer Vision
PNEC-Mamba: Prototype-Guided Positive-Negative Evidence Calibration for Hyperspectral Image Classification
Mingzhen Xu, Can Xu, Di Wang et al. (+2 more)
In real-world hyperspectral scenes, pixel representations are often ambiguous due to factors such as spectral similarity, mixed pixels, and local context interference, which may simultaneously encode discriminative evidence and interfering information. Existing methods mainly focus on learning more powerful representations or modeling broader contexts, but rarely investigate whether the learned representations provide reliable evidence or introduce interference into classification decisions. To address this issue, we view hyperspectral image classification from the perspective of pixel-level evidence reliability modeling and propose PNEC-Mamba, a prototype-guided positive-negative evidence calibration framework. The framework progressively establishes semantic references, separates class-related evidence from interference, estimates pixel-level reliability, and performs selective calibration. First, a full-image state-space encoder extracts pixel representations, while dynamic class prototypes provide semantic references that evolve jointly with the feature space. Subsequently, positive and negative evidence is derived from pixel-prototype competition, explicitly separating discriminative cues that support classification from confusing signals associated with competing classes. Based on these evidence relationships, a multi-source uncertainty estimation strategy is introduced to assess pixel-level reliability, enabling stronger evidence calibration for uncertain regions. Finally, a full-resolution consistency refinement step is applied to recover local spatial details and improve boundary coherence in the final predictions. Extensive experiments on three
benchmark datasets demonstrate that PNEC-Mamba achieves superior classification performance compared with state-of-the-art methods.
Read the full abstract ▾
All authors Mingzhen Xu, Can Xu, Di Wang, Haonan Guo, Bo Du
Computer Vision
Assessing the Benefits of Combining Advanced Deep Learning Techniques for Post-Disaster Building Damage Assessment from UAV Imagery
Huy Quang Ung, Guillaume Habault, Roberto Legaspi et al. (+3 more)
Rapid and accurate post-disaster building damage assessment is essential, yet remains a challenging task. Unmanned Aerial Vehicle (
UAV) imagery offers a timely and high-resolution view of affected areas, but existing Computer
Vision (CV) models often demand large annotated
datasets, generalize poorly across geographic regions and their assessment policies, and are confined to the specific tasks they were trained for. Large
Vision-
Language Models (LVLMs) offer a promising alternative through their strong
reasoning and generalization capabilities, but fall short on precise, low-level perception tasks such as object detection and accurate bounding box generation. Furthermore, they often require a substantial amount of data for effective fine-tuning on domain-specific tasks. In this paper, we propose a hybrid framework that decouples detection from damage assessment, combining the precision of CV models with the
reasoning power of LVLMs. A CV model first detects buildings and generates bounding boxes on the image that are then passed to an LVLM for damage classification and contextual interpretation. We evaluated our framework on two real-world
benchmarks: RescueNet and FloodNet. In particular, the best combination under this framework accurately counts intact, partially damaged and completely destroyed buildings, surpassing isolated baselines by up to 2.1 R^2 points, while requiring only limited annotated data for the detection stage. Beyond reporting aggregate gains, we provide a detailed analysis of failure scenarios and edge cases, offering practical insights for practitioners and concrete directions for future work. Our source code and data are publicly available to the research community via the following repository: https://github.com/ungquanghuy-kddi/VLM_GDINO.git
Read the full abstract ▾
All authors Huy Quang Ung, Guillaume Habault, Roberto Legaspi, Hao Niu, Lian Cao, Masato Taya
Computer Vision
PhotoHOI: Synthesizing 3D Hand-Object Interactions from a Single RGB Photograph
Zhenhao Zhang, Jiajun Zhang, Wei Min et al. (+1 more)
Hand-object interaction (HOI) is a fundamental human behavior with broad applications in AR/VR, digital humans, and
embodied interaction. Existing methods typically require predefined object geometry, object trajectories, or task-specific conditions, limiting their use with natural real-world inputs. To address this, we study a more practical problem of synthesizing 3D hand-object interaction sequences from a single RGB photograph and an open-vocabulary
language instruction, and introduce PhotoHOI. PhotoHOI first uses a
vision-
language model to parse the input image and instruction into a structured task specification, including the interaction object, target region, and spatial relation. It then recovers a compact task-relevant 3D scene and plans a smooth collision-aware object trajectory based on the recovered object states, support relations, and surrounding scene geometry. To synthesize hand motion that generalizes to real-world photographs and unseen objects, it learns transferable task-conditioned contact and contact-conditioned grasp priors from large-scale affordance and HOI data. The grasp is further refined in a learned latent space, constraining the optimization to a plausible hand-pose
manifold. Experiments on GRAB and H2O demonstrate improved contact quality and reduced penetration over representative baselines. Results on real-world photographs further demonstrate higher task success and scene consistency, together with generalization to unseen objects and open-vocabulary instructions.
Read the full abstract ▾
All authors Zhenhao Zhang, Jiajun Zhang, Wei Min, Yebin Liu
Computer Vision
SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models
Jing Wu, Jianhua Wu, Jiayi Guan et al. (+5 more)
Vision-
Language Models (VLMs) perform well on commonsense
reasoning tasks but struggle with visual spatial
reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient \textit{Spatio-
vision Language Models (SpatioLM)}, that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-
vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to
embodied manipulation tasks. Code is available at \href{https://github.com/xiaomi-research/spatio-lm}{\faGithub~spatio-lm}.
Read the full abstract ▾
All authors Jing Wu, Jianhua Wu, Jiayi Guan, Jiahong Chen, Jinghui Lu, Hangjun Ye, Bingzhao Gao, Long Chen
Computer Vision
GeoCore-9B: Towards Geo-Aware Generative Foundation Models in Earth Observation
Jeonghyeok Do and Munchurl Kim
Existing generative models for earth observation (EO) predominantly rely on fine-tuning natural image priors, which limits their scalability and introduces perspective biases that conflict with geospatial constraints. To address this, we introduce GeoCore-9B, a 9-billion-parameter generative foundation model, which is the first of its scale to be trained from scratch exclusively on EO data. Unlike previous EO foundation models, GeoCore-9B is built upon a Flow Matching-based
Diffusion Transformer (DiT) and natively conditions generation on text descriptions and continuous geospatial metadata, including ground sample distances, latitudes, and longitudes. To overcome the convergence and spatial disorientation challenges of training at this scale, we propose a Geospatial Semantic
Alignment loss. This objective distills structural Earth surface priors (e.g., terrain and urban areas) from a frozen specialist teacher network, constraining the
diffusion latent trajectory during training without adding inference overhead. Pre-trained on the global-scale Git-10M
dataset, GeoCore-9B demonstrates strong downstream versatility. Beyond standard proxy generative tasks, we show that GeoCore-9B can be effectively adapted for practical EO applications, including highly challenging tasks such as cloud removal and SAR-to-optical cross-modal translation. Extensive evaluations confirm that GeoCore-9B establishes new state-of-the-art performance in both visual fidelity and geographic structural accuracy.
Read the full abstract ▾
Computer Vision
Beyond Illumination: A Conditional Mutual Information-Guided Network for Low-Light Image Enhancement
Ya-nan Guan, Shaonan Zhang, Tao Dai et al. (+5 more)
Low-light image enhancement (LLIE) seeks to restore structural fidelity, natural color rendition, and proper exposure from images captured under inadequate lighting conditions. Recent state-of-the-art approaches, such as CIDNet, adopt a dual-branch architecture comprising a chrominance (HV) branch and an intensity (I) branch to separately model decoupled chromatic and luminance information within the HVI color space. However, these methods overlook the mutual interaction between intensity and chrominance components, which inherently limits their representational capacity and leads to suboptimal enhancement performance. To address this limitation, we propose the Conditional Mutual Information-Guided Network (CMIG-Net), which leverages conditional mutual information as a principled metric to quantitatively assess the contribution of chrominance features conditioned on the available intensity information. In particular, we design a Conditional Mutual Information Calibration (CMIC) module that generates a conditional information map, enabling region-adaptive recalibration of chrominance representations according to local illumination statistics. Furthermore, we introduce a Dynamic Dual-branch Information Restoration (D2IR) module, which adaptively governs bidirectional information flow between the intensity and chrominance branches, guided by both the conditional prior and the instantaneous restoration state. Extensive experiments on paired LLIE
benchmarks demonstrate that CMIG-Net consistently outperforms CIDNet, achieving up to a 0.619 dB gain in PSNR, with a 0.382 dB improvement specifically on the challenging Sony-Total-Dark
dataset.
Read the full abstract ▾
All authors Ya-nan Guan, Shaonan Zhang, Tao Dai, Tianqu Zhuang, Yongchao Qiao, Zhensen Chen, Shu-Tao Xia, Hang Guo
Computer Vision
Transformer Geometry Observatory TGO-III: Semantic Geometry Observatory
Kaustubh Kapil and Kishor P. Upla
With the widespread adoption of
Vision Transformers in modern AI, the need to analyze their inherent representational behavior has become increasingly important. While most existing studies emphasize
token geometries and training dynamics, the
evolution of representational covariance structures and class-level
geometric organization remains comparatively underexplored. In this work, we investigate semantic geometry and class separability as representations evolve across the layers of ViT-Small/16 through TGO-III: Semantic Geometry Observatory. It is a framework designed to analyze the emergence of semantic organization, feature
evolution, and class-wise representation geometry throughout training. The framework employs multiple complementary observatories, including Linear Probe Accuracy, Fisher Ratio, Class Centroid Distances, Local Intrinsic Dimension, and Local PCA Rank, to quantify the progressive
evolution of discriminative representations. Our analysis reveals that class representations become progressively more linearly separable, Fisher discriminability increases, class centroids move farther apart, and local representation
manifolds exhibit structured class-dependent
geometric complexity. These observations provide empirical evidence supporting the Semantic Expansion Hypothesis, suggesting that the
manifold expansion observed in previous observatories is accompanied by the progressive organization of representations into increasingly discriminative semantic structures. Collectively, TGO-III extends the
Transformer Geometry Observatory framework by establishing a direct connection between
manifold geometry, covariance
evolution, and semantic organization during
Transformer training.
Read the full abstract ▾
Computer Vision
Decoupling semantics from vision: A framework for faithful visual-text compression evaluation
Yonghan Gao, Zehong Chen, Lijian Xu et al. (+3 more)
Recent visual-text compression (VTC) methods, typified by DeepSeek-OCR, report impressive high
token compression ratios for long-context modeling tasks by leveraging text-to-image rendering. However, existing evaluation protocols heavily rely on downstream task performance. Such evaluation metrics fail to accurately measure text preservation due to the strong inherent linguistic priors of
Multimodal Large
Language Models (MLLMs). In this work, we introduce a new evaluation framework that decouples MLLMs' capabilities to faithfully assess VTC quality. Within this framework, we further introduce the ZeroSense
Benchmark to ensure low semantic correlation of testing samples. By eliminating textual dependencies, our
benchmark guarantees that the evaluation results are purely reflective of VTC quality, unaffected by the semantic inference capabilities of downstream models. Extensive experiments across multiple
datasets demonstrate that VTC quality and downstream task accuracy diverge significantly, highlighting the necessity of our decoupled evaluation framework.
Read the full abstract ▾
All authors Yonghan Gao, Zehong Chen, Lijian Xu, Jingzhi Chen, Jingwei Guan, Xingyu Zeng
Computer Vision · Cross-listed from cs.LG
WorldDynCache: Risk-Controlled Latent Dynamics Approximation for Diffusion World Model
Leyang Chen, Junyi Wu, Shaoqiu Zhang et al. (+1 more)
Diffusion world models generate high-quality futures, but re- peated
transformer evaluations make inference prohibitively slow. Existing caches reuse intermediate features, selectively update
tokens, or reuse and extrapolate denoising outputs ac- cording to local drift or short native-space histories. These criteria can miss both approximation-induced latent transition defects that accumulate across skipped steps and phase- or condition-dependent changes in the direction of latent evo- lution. We propose WorldDynCache, a risk-controlled latent dynamics approximation framework with two core compo- nents. First, a lightweight latent-transition risk
estimator tracks the accumulated future impact of approximation defects and calibrates its predictions against counterfactual defects ob- served at exact anchors. Second, a condition- and phase- aware lifted latent surrogate approximates latent
evolution without extra
transformer evaluations. On HunyuanVoyager- 13B and Aether-5B, WorldDynCache achieves 4.92 times and 2.15 times speedups, respectively, while attaining the best gen- eration quality among the compared caching methods across WorldScore, PSNR, SSIM, and LPIPS.
Read the full abstract ▾
All authors Leyang Chen, Junyi Wu, Shaoqiu Zhang, Yulun Zhang
Computer Vision
MoCRA: Mixture of Compositional Rank-1 Atoms for 4K All-in-One Video Restoration
Yongcong Wang, Pu Wang, Hingchin Chen et al. (+8 more)
Real-world
video arrives hazy, rainy, dark, or noisy, and a deployable restorer faces three demands at once: no degradation label, native 4K output, and stability in playback. Existing methods answer them separately and break on the joint problem, because per-frame degradation readings flip between frames, downsampled proxies erase the rain and noise they are meant to remove, and dense temporal
alignment does not fit 4K
memory. No paired
benchmark even poses that problem, so we build one. UHV-4K-AIO renders physically modeled haze, rain, sensor noise, and low light over the same 100 clean 4K clips with shared depth and motion, and its construction exposes the split MoCRA is built on: haze and low light survive aggressive downsampling, while rain and noise exist only at native scale. Band-matched compositional conditioning follows, spending conditioning capacity, computation, and supervision in the band where each degradation lives. One dictionary of rank-1 atoms, recomposed sparsely per frame, conditions both a once-per-clip coarse branch and a shallow native-resolution refiner, in 3.6M parameters and with no optical flow. Trained once for all four tasks, MoCRA takes the best task-mean PSNR of eleven retrained image and
video baselines, holds warping error at the level of the flow-based
video models while never estimating motion, and restores native 4K in under half a second, against 1.7 seconds for the fastest baseline.
Read the full abstract ▾
All authors Yongcong Wang, Pu Wang, Hingchin Chen, Runci Bai, Yucheng Xin, Chen Wu, Chengchao Shen, Guangwei Gao, Siyuan Yao, Pengwen Dai, Zhuoran Zheng
Computer Vision
DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agents
Huanyao Zhang, Jiepeng Zhou, Runhao Zhao et al. (+12 more)
Multimodal large
language models (MLLMs) have advanced visual understanding and
reasoning, yet their static parametric knowledge limits their ability to address knowledge-intensive and dynamically evolving open-world problems. To move beyond this limitation,
multimodal deep search has emerged as a key direction for open-world information access, evolving from single-turn factual
retrieval toward long-horizon, multi-turn search guided by visual evidence. However, existing methods typically confine
vision to the input or answer stage, overlooking its role in intermediate
reasoning, and lack designs tailored to long-horizon interaction. Consequently, visual evidence rarely drives continued
retrieval, constraining both interaction depth and
reasoning span. To address these limitations, we propose DeepVoyager-VL, a long-horizon
multimodal deep-search framework for
vision-in-the-loop search. Specifically, we construct a
multimodal event graph to drive data synthesis, yielding problems with intermediate visual dependencies and long
reasoning chains. We then design an
agent framework for active visual acquisition and on-demand image loading. Finally, we fine-tune models on the synthesized data without reinforcement learning. Extensive experiments across ten
multimodal search
benchmarks demonstrate the effectiveness of our method.
Read the full abstract ▾
All authors Huanyao Zhang, Jiepeng Zhou, Runhao Zhao, Yanzhe Shan, Jiaoyang Chen, Bowen Zhou, Bo Li, Fang Wang, Jialong Wu, Zhengwei Tao, Lang Mei, Xiaohan Yu, Liyan Liu, Chong Chen, Wentao Zhang
Computer Vision
PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent
Guangming Fu, Jin Song, Yiyun Fei et al. (+3 more)
Part-level 3D generation has recently attracted increasing
attention for producing structured and editable 3D assets. However, existing methods typically decompose objects according to functional semantics rather than the editable material boundaries (e.g., fabric, wood, metal) required in practical 3D applications such as interior design. Additionally, current methods often generate parts independently, causing computational costs to scale linearly with the part count. To address these limitations, we present PartMat, an efficient material-aware 3D part decomposition pipeline that represents multi-part geometry with a single global latent. Given a reference image and a single whole-object geometry, PartMat decomposes the object into parts that follow material boundaries. First, we propose PartVAE to learn such a unified representation and decode all material parts in a single forward pass, thereby decoupling inference cost from the number of parts. Second, with this representation, a
diffusion model is trained for part generation and refined via reinforcement learning for accurate material assignment and overlap suppression. Finally, to recover fine-grained
geometric details, we introduce a sparse-voxel flow-matching model with part
attention for geometry post-processing. Extensive experiments demonstrate that PartMat significantly outperforms existing baselines in material-aware decomposition accuracy and achieves comparable
geometric quality, while maintaining efficient inference.
Read the full abstract ▾
All authors Guangming Fu, Jin Song, Yiyun Fei, Guoqiu Li, Ruigao Yang, Jianan Jiang
Computer Vision
Detail Continuation over a Trustworthy Coarse Scale for Autoregressive Super-Resolution
Hongyi Fang, Jiahui Wu, Yichen Yue et al. (+2 more)
Hallucination remains a persistent challenge in generative super-resolution (GSR), where reconstructed results may contain visually plausible yet weakly supported content, structural deviations, or unnatural textures with respect to the low-resolution (LR) input. Existing GSR methods have extensively explored the trade-off between perceptual realism and reconstruction fidelity, but the division between preserving reliable coarse-scale information and restoring more uncertain fine details is often handled implicitly within the overall restoration process. Visual autoregressive (VAR) modeling provides a natural opportunity to revisit this issue, as its coarse-to-fine next-scale prediction offers an explicit scale-wise generation interface. However, existing VAR-based SR methods still inherit the original full 1-to-$N$ autoregressive generation path, even though, for super-resolution, coarse-scale information in LR is often relatively more reliable, while long autoregressive chains may accumulate prediction errors. Motivated by these observations, we propose K2N, which reformulates VAR-based SR from full-path generation into a $k$-to-$N$ detail continuation process. Specifically, early coarse-scale states are established directly from LR, while only the remaining finer scales are restored autoregressively. Experimental results show that K2N remains competitive with the VARSR baseline on standard SR metrics, while exhibiting clearer advantages on
hallucination-focused evaluation. These findings suggest that explicitly rethinking the generation path in a scale-wise manner can be a promising direction for improving the reliability of generative super-resolution. Our code will be released soon at https://github.com/BRL-SYSU/K2NSR.
Read the full abstract ▾
All authors Hongyi Fang, Jiahui Wu, Yichen Yue, Benjia Zhou, Dan Zeng
Computer Vision
DAVET: Denoising-Aware Visual Evidence Trajectory Allocation for Diffusion Vision-Language Models
Yongkang Zhou, Xiang Xia, Cheng Yan et al. (+2 more)
Diffusion vision-
language models (dVLMs) iteratively denoise masked responses while conditioning each denoising step on visual evidence, making visual conditioning a substantial recurring inference cost. Unlike autoregressive decoding,
diffusion generation repeatedly revisits the entire response as uncertainty evolves. Our analysis reveals that visual evidence demand is strongly step-dependent, motivating adaptive allocation across denoising steps. Existing inference acceleration methods operate through decoding-side strategies or visual
token compression via
pruning and merging, but do not explicitly treat visual evidence as a resource whose demand evolves across the
diffusion process. Therefore, we present Denoising-Aware Visual Evidence Trajectory Allocation (DAVET), a training-free framework that allocates visual evidence according to the evolving generation state. Starting from a phase-conditioned evidence trajectory, the proposed allocation policy uses operation demand to set an evidence reserve whose allocation at each denoising step is modulated by trajectory risk. DAVET realizes the resulting budgets through a hierarchy of evidence views constructed from a single visual encoding, separating when and how much evidence is needed from how the evidence views are constructed. Evaluated on two representative dVLMs, LLaDA-V and LaViDa, across multiple visual-understanding
benchmarks, DAVET achieves an average speedup of 1.55$\times$ with an average relative performance drop of 1.86%, showing that denoising-aware visual evidence allocation can reduce visual conditioning cost while largely preserving generation quality.
Read the full abstract ▾
All authors Yongkang Zhou, Xiang Xia, Cheng Yan, Fan Xu, Wuyang Zhang
Computer Vision
SecondOpinion: Anatomy-Aware Gated Reasoning for Efficient Medical Image Analysis
Siam Tahsin Bhuiyan, Rashedur Rahman, Sefatul Wasi et al. (+4 more)
Deep learning models for medical image analysis typically apply a fixed amount of computation to every input, regardless of case difficulty. Anatomy-guided dual-stream architectures have been shown to improve diagnostic performance, but they evaluate both streams unconditionally, even on cases a single stream could already resolve confidently. We propose SecondOpinion, a framework in which a fast primary stream processes every case, while a second, anatomy-guided stream is invoked only when GateKeeper, a gating mechanism trained explicitly as a binary correctness classifier, judges that the primary stream's prediction needs additional scrutiny, much as a clinician might seek a second opinion on a difficult case. When activated, the two streams are combined through a lightweight cross-
attention fusion module. We evaluate SecondOpinion on a unified five-class chest X-ray
dataset and a pelvic fracture
dataset, the latter including a held-out, harder subset of fractures that are invisible on X-ray but confirmed via CT. SecondOpinion matches or exceeds prior state-of-the-art performance on both tasks, while activating its anatomy-guided stream on only 9.23% of chest X-ray cases, rising to 24.12% on visible fractures and 45.71% on invisible fractures, an activation rate that tracks task difficulty directly. These results suggest that supervising a gating signal toward correctness, rather than relying on unsupervised confidence, allows a model to allocate anatomical
reasoning where it is actually needed.
Read the full abstract ▾
All authors Siam Tahsin Bhuiyan, Rashedur Rahman, Sefatul Wasi, Riyadul Islam, Syoji Kobashi, Ashraful Islam, Saadia Binte Alam
Computer Vision
Parameter-Dynamic Adaptive Fusion and Calibration Network for RGBT Tracking
Zhaoding Ding, Chenglong Li, Jiandong Jin et al. (+2 more)
Existing RGBT trackers typically employ fusion functions with fixed parameters across different targets and scenarios. Although dynamic-architecture methods improve fusion flexibility by selecting among predefined operations, they still cannot adapt the fusion parameters to the evolving target state. To address these issues, we propose a Parameter-Dynamic Adaptive Fusion and Calibration Network (PAFCNet) for RGBT tracking. PAFCNet dynamically generates target-conditioned parameters for
multimodal fusion and temporal calibration, enabling the tracking process to adapt to target appearance variations and modality quality fluctuations. Specifically, we introduce a Target-Adaptive Hypernetwork (TA-HyperNet) that leverages template representations, which preserve stable target identity and recent appearance changes with less background interference, to generate target-conditioned parameters for subsequent fusion and calibration. Based on TA-HyperNet, we design a target-aware parameter-dynamic fusion module that uses the generated parameters to modulate the fusion process. This enables the fusion module to adapt to changes in target appearance and complex scene conditions. Furthermore, since spatio-temporal information propagation may accumulate tracking noise, we propose a dynamic spatio-temporal calibration module that employs TA-HyperNet to generate calibration parameters for spatio-temporal
tokens. By dynamically calibrating historical information before propagation, the module improves the reliability of temporal representations. Experimental results demonstrate that PAFCNet achieves competitive performance on multiple RGBT tracking
benchmarks.
Read the full abstract ▾
All authors Zhaoding Ding, Chenglong Li, Jiandong Jin, Kewei Ying, Wentao Wu
Computer Vision
Illuminating Visual Identity in Universal Multimodal Embeddings
Jiawei Cao, Junyi Feng, Jiashen Hua et al. (+5 more)
Universal
Multimodal Embeddings (UMEs) aim to unify various modalities and tasks into a shared representation space. In recent years, this field has witnessed substantial progress driven by the development of
Multimodal Large
Language Models (MLLMs). However, a crucial capability, visual identity discrimination, remains underexplored in existing UME methods, despite its critical role in a wide range of tasks, including instance
retrieval, re-identification, and identity preservation in AI-generated content. To bridge this gap, we propose a unified formulation for visual identity discrimination~(VisID) and introduce $\textbf{MVEB}$ ($\textbf{M}$ultimodal $\textbf{V}$isual Identity $\textbf{E}$mbedding $\textbf{B}$enchmark), a large-scale
benchmark curated from both real-world and synthetic
datasets to support evaluation and training. Furthermore, we present a simple yet effective learning framework that jointly optimizes general
multimodal and visual identity representations through a carefully designed identity-aware sampling mechanism. Extensive experiments demonstrate that our approach successfully endows UMEs with strong identity discrimination capability and maintains competitive general
multimodal performance. We believe this work not only illuminates a critical yet neglected capability, but also takes a step toward more holistic universal
multimodal embeddings. Code and data are available at \href{https://chrisclear3.github.io/MVEB}{MVEB}.
Read the full abstract ▾
All authors Jiawei Cao, Junyi Feng, Jiashen Hua, Ziheng Huang, Bing Deng, Kaijie Wu, Chaochen Gu, Jieping Ye
Computer Vision
Investigating Social Bias in Narrative Image Generation
Junyeong Park, Sowon Min, Euna Jang et al. (+5 more)
Text-to-image (T2I) generation models are increasingly embedded in applications such as media content creation and education, raising concerns about how their outputs may reproduce social biases. Prior work has shown that T2I models exhibit social biases, yet existing evaluations largely focus on a photo generation task. As a result, it remains unclear whether and how such biases manifest in more narrative visual formats, such as storyboards and comics, where characters and events are presented across multiple panels. In this work, we compare bias expression across photo, storyboard, and comic generation in six T2I models by adapting BBG, a text-based bias evaluation framework, to image generation. Our results show that proprietary models generate 25.9% biased outputs in photo generation on average, with biased outputs increasing by 9.6pp in storyboard generation and 18.2pp in comic generation. We also find that photos mainly encode biases through subtle visual cues, while storyboards and comics reveal them more explicitly through event sequencing, character positioning, narrative resolution, and textual elements. These findings show that biases that remain less visible in photo generation may surface in narrative visual formats, highlighting the importance of evaluating T2I systems with diverse visual formats beyond photo generation.
Read the full abstract ▾
All authors Junyeong Park, Sowon Min, Euna Jang, Soobin Kim, Jiho Jin, Hyunseung Lim, Gahyeon Bae, Hwajung Hong
Computer Vision
LiveLight: Real-time Streaming Video Relighting with Interactive Control
Yue Ma, Jiangming Wang, Yucheng Wang et al. (+8 more)
We present LiveLight, the first
diffusion-based framework for real-time streaming
video relighting with interactive 3D lighting control. Achieving this is non-trivial, as it requires overcoming three critical challenges: effectively injecting dynamic 3D lighting into a
diffusion model, maintaining high-fidelity generation under an extremely low NFE (Number of Function Evaluations) budget for real-time speed, and facilitating continuous streaming for interactive control. To address these pain points, we propose three key designs. First, for accurate lighting injection, we propose a lightweight adapter that feeds Multi-Plane Light Irradiance (MPLI) conditions-depth-aware irradiance maps encoding 3D lighting geometry-directly into the
diffusion backbone. Second, to prevent rendering quality degradation at low NFEs towards real-time
distillation, we introduce a geometry-guided feedback branch. This training-time constraint leverages a frozen geometry
estimator to enforce depth- and normal-consistent relighting, ensuring geometrically plausible shading without adding inference overhead. Finally, to enable streaming interaction, we develop a progressive rolling-window strategy that maintains a denoising ladder of latent chunks at varying noise levels. By propagating intermediate states, this strategy guarantees temporal coherence and supports arbitrarily long
video relighting with per-frame reference refresh. Extensive experiments on real-world and synthetic
benchmarks demonstrate that LiveLight achieves state-of-the-art relighting quality while running at real-time speed, significantly outperforming offline baselines in temporal stability, lighting controllability, and user preference. To foster real-time interactive relighting research, we will publicly release our models, training data, and synthetic data generator.
Read the full abstract ▾
All authors Yue Ma, Jiangming Wang, Yucheng Wang, Xilai Wang, Zhiyuan Li, Xinyu Wang, Hongyu Liu, Ruofan Liang, Songchun Zhang, Yuxuan Xue, Qifeng Chen
Computer Vision
DecoupleGS: Interactive 3D Gaussian Splatting for End-to-End Autonomous Driving Testing
Siying Li, Ying Ni, Jie Sun et al. (+2 more)
End-to-end (E2E) autonomous driving
algorithms require rigorous closed-loop validation in simulation environments offering high visual fidelity, strong interactivity, and real-time performance. Existing approaches, from game engines to static neural rendering, inherently trade off these requirements and struggle with the dynamic scene composition essential for E2E testing. To bridge this gap, we propose a novel decoupled 3D Gaussian Splatting (3DGS) framework tailored for large-scale E2E evaluation. We fundamentally decompose scenes into a high-fidelity static background and manipulable dynamic
agents using an object-centric canonical representation. To resolve resulting representational conflicts, we introduce three targeted modules: (1) asset compression via perceptual
pruning and vector quantization for real-time traffic rendering; (2) map-guided
geometric registration leveraging semantic
topology to strictly align trajectories; and (3) proxy-based relighting transferring ambient illumination for seamless photometric integration. Extensive experiments demonstrate that DecoupleGS achieves a balanced fidelity-efficiency trade-off, improves metric and photometric consistency, and provides a practical closed-loop sensor simulation platform for E2E autonomous driving evaluation.
Read the full abstract ▾
All authors Siying Li, Ying Ni, Jie Sun, Jian Sun, Haotian Shi
Computer Vision
Pixel Ignores, Superpixel Sees: Adverse Weather Image Restoration via Semantic-Center SSM
Dayu Li, Shihao Zhou, Leizhi Shu et al. (+3 more)
Adverse weather image restoration aims to recover clear visibility from degraded images in complex weather conditions. Existing works attempt to address this problem by modeling relationships between pixels, however, this paradigm defies the spatially non-uniformity fact of degradations and learns non-discriminative features from semantic-conflict regions. In this paper, we propose SSR, a Semantic-center guilded State space model for image Restoration. The key idea of SSR is to shift the conventional scanning strategy of pixel-serial to semantic-guilded one. Specifically, we introduce a Superpixel-guided Selective Scan Mechanism ($\text{S}^3$M), which first partitions the image into perceptually coherent regions via superpixel clustering and then performs relations modeling within the semantic-related regions. Moreover, a Region-level Gating Mechanism (RGM) is developed to perform intra-region calibration by modulating degradation outliers within each semantic superpixel unit along the channel dimension. Extensive experiments on 6 well-established
benchmarks demonstrate that SSR performs favorably against state-of-the-art models with competitive computational cost.
Read the full abstract ▾
All authors Dayu Li, Shihao Zhou, Leizhi Shu, Jin Wu, Chi Man Vong, Jufeng Yang
Computer Vision
Can Urban Blight Be Accessed with Vision-language Models: A Case Study in Detroit
Xiaohao Yang, Aohua Tian, Derek Van Berkel et al. (+2 more)
Addressing urban blight has seen increased focus in the past 15 years. Assessing urban blight is essential for guiding urban planning, targeting rehabilitation, and safeguarding public health, yet traditional residential blight surveys are difficult to maintain at scale due to the labor-intensive cost and long-term cycle. This study introduced a scalable framework for estimating residential blight using open-source large
vision-
language models on multiple views. Structured
prompts guided models to evaluate housing attributes, including roof integrity, wall damage, and broken or boarded openings, producing both binary assessments and probabilistic estimates of disrepair. To evaluate the performance of these visual assessments, we compared professional human annotations of these features across several models, including an ensemble stacking approach based on XGBoost and a weighted scoring system. Results showed that (i) multiple street views can contribute to the improvement of accuracy, (ii) large
vision-
language models have different strengths of inference, (iii) the ensemble learner outperforms individual base models, enhancing robustness across all residential conditions and blight assessment. The practical application of the method allows low-cost tracking and management of housing stock conditions, providing a regularly updatable complement to traditional blight surveys.
Read the full abstract ▾
All authors Xiaohao Yang, Aohua Tian, Derek Van Berkel, Xu Qiang, Mark Lindquist
Computer Vision
SPECTRA: Band-Routed Embedding and Stage-Wise LoRA for Cross-Sensor Fine-Tuning of Geospatial Foundation Models
Xingyan Li, Jordan A. Caraballo-Vega, Jie Gong et al. (+2 more)
Geospatial foundation models (GeoFMs), pretrained on large-scale geospatial data such as Earth observation (EO),
climate, and weather data, have shown promising performance when fine-tuned on diverse downstream tasks. However, there are two challenges of adapting EO-pretrained GeoFMs to practical downstream
datasets. The first challenge is how to handle spectral mismatch: pretrained patch
embeddings expect a fixed set of input bands, whereas downstream sensors may provide different channels. The second challenge is how to reduce fine-tuning cost and make it efficient. While existing work has made efforts on these challenges individually, jointly improving fine-tuning performance under spectral mismatch while reducing adaptation cost remains underexplored. We propose SPECTRA, a parameter-efficient fine-tuning framework that addresses both spectral mismatch and adaptation cost. To handle spectral mismatch, SPECTRA introduces Band-Routed
Embedding (BRE), which maps all available downstream bands into the band space expected by the pretrained GeoFM. By using BRE, all available bands in the downstream
dataset are utilized to improve the selected-band input without changing the pretrained patch
embedding interface. To reduce adaptation cost, SPECTRA further introduces a Stage-wise Transferability-aware LoRA (ST-LoRA) fine-tuning. ST-LoRA estimates stage-wise transferability before fine-tuning and assigns stage-specific LoRA ranks, concentrating trainable parameters on the stages with high transferability for the target task. Across three EO-pretrained GeoFMs and four downstream
segmentation datasets, experiments show that BRE improves performance by utilizing all spectral bands, while ST-LoRA reduces trainable parameters compared with full fine-tuning and standard LoRA. Code is available at https://github.com/big-data-lab-umbc/SPECTRA.
Read the full abstract ▾
All authors Xingyan Li, Jordan A. Caraballo-Vega, Jie Gong, Mark L. Carroll, Jianwu Wang
Computer Vision
IDraw: Artist Verification from Digital Drawing Images
Nayoung Kim, Nan Jiang, Bangjie Sun et al. (+3 more)
As digital drawings are increasingly shared online, reliable authorship verification has become important for protecting artists and resolving disputes. Yet when authorship is questioned, verification may have to rely only on the disputed drawing and reference drawings known to be created by the claimed artist. This setting is challenging for two reasons. First, artist-specific drawing behavior, such as pen pressure and movement speed, is informative but is not available from a completed drawing. Second, similarities in the depicted object or scene can obscure similarities arising from the artist. We propose IDraw, a framework that learns from drawings paired with tablet-pen sensor signals collected from separate training artists. This allows IDraw to infer drawing behavior from completed images during a later authorship dispute, without requiring sensor data from the artist being verified. IDraw also reduces the influence of drawing content by identifying information shared by drawings of the same object across different artists and suppressing it before comparing drawings. To support this approach, we construct the first
multimodal dataset for digital drawing authorship verification, containing 1,110 drawings from 37 artists and 14 types of tablet-pen sensor signals. Evaluated on previously unseen artists across nine image-encoder backbones, IDraw consistently outperforms standard image-based verification and reduces verification error by up to 40%. These results demonstrate that inferring drawing behavior from completed images and suppressing drawing content improve digital drawing authorship verification.
Read the full abstract ▾
All authors Nayoung Kim, Nan Jiang, Bangjie Sun, Jaewon Shin, Sojeong Kim, Jun Han
Computer Vision
Learning Where to Look and How to Judge: Resolution-agnostic Image Quality Assessment with Quality-aware Saliency
Hakan Emre Gedik, Shashank Gupta and Alan Bovik
No-reference image quality assessment (NR IQA) has recently benefited from deep and
multimodal models, yet many SOTA systems still violate at least one basic requirement: they either discard critical quality cues via aggressive resizing, fail to generalize across resolutions, cannot be jointly trained on heterogeneous IQA
datasets with mismatched MOS scales, or require prohibitive computation. We present ReLIQS, a model for Resolution-agnostic Learning for Image Quality with Saliency, which is resolution-agnostic, preserves original-resolution quality cues, learns from multiple subjective studies, and remains computationally efficient and budget-adaptive. ReLIQS is a CLIP-based multiscale patch-driven architecture that learns both where to look and how to judge quality. Fixed-size patches are sampled across multiple resolutions, including the original resolution, and encoded with a CLIP
vision backbone. A lightweight Perceptual Importance
Estimator then predicts IQA-specific importance maps to select a small set of informative patches, and a Latent Quality Axis Module aggregates their
embeddings into a single image-level score. Across authentic, synthetic, and AIGC
benchmarks spanning diverse resolutions and distortions, ReLIQS generalizes better than strong CNN-, CLIP-, and MLLM-based baselines with matching or reduced computational cost.
Read the full abstract ▾
Computer Vision
G-Skin: Learning to Bind 3D Gaussians with Generative Visual Priors
Yuxin Yao, Kendong Liu, Shiqi Zhou et al. (+2 more)
3D Gaussian Splatting has achieved remarkable success in photorealistic and efficient rendering, leading to a rapid increase in 3D assets represented by 3D Gaussian primitives. Directly rigging these assets with arbitrary skeleton
topologies is highly desirable. However, training a feed-forward skinning framework is infeasible due to the lack of high-quality 3D Gaussian rigging
datasets. An alternative solution is to transfer mesh-based techniques to 3D Gaussian-based representation, but 3D Gaussian primitives are not restricted to the surface and lack explicit topological connectivity. Moreover, this kind of method suffers from poor generalization to unseen data due to its strong dependence on training data, while acquiring high-quality rigging data is prohibitively expensive. To address this challenging problem, we propose G-Skin, a novel generative skinning framework designed for expressive and high-fidelity animation with 3D Gaussian representation. To overcome this 3D data scarcity, we introduce a skeleton-controllable image generation model leveraging 2D
vision foundation models to distill powerful motion priors into pseudo-guidance. Guided by these priors, we formulate an optimization pipeline incorporating geometry-aware regularizations, which stabilizes the learning process and ensures smooth, structurally coherent skinning weights. G-Skin also generalizes flexibly to the augmented variants of 3D Gaussian representation designed to mitigate animation-induced rendering artifacts. Extensive experiments validate the effectiveness of our approach, demonstrating clear advantages over state-of-the-art methods. Project page: https://yaoyx689.github.io/GSkin.html.
Read the full abstract ▾
All authors Yuxin Yao, Kendong Liu, Shiqi Zhou, Jiazhi Xia, Junhui Hou
Computer Vision
When Extreme Darkness Meets Motion Blur: MeanFlow for Unified RAW Restoration
Zepu Wang, Jingze Liang, Weijie Xiao et al. (+1 more)
Extremely low-light RAW enhancement aims to recover severely attenuated sensor signals, yet existing methods often focus on illumination and noise while overlooking the motion-induced degradations inherent in practical low-light imaging. We present a framework for robust extremely low-light RAW enhancement under realistic acquisition degradations. First, we introduce See in the Degraded Extremely Dark (SIDED), a new
dataset that applies controlled motion degradation to extremely low-light RAW pairs while retaining their original sensor noise. Second, we propose a unified RAW tokenizer equipped with explicit domain-conditioned representation calibration to align extremely low-light and well-exposed RAW data, followed by a MeanFlow that performs enhancement in a single function evaluation. To our knowledge, this is the first work to formulate extremely low-light RAW enhancement under realistic motion-degraded acquisition and address it with MeanFlow. We further introduce a physics-guided refinement model to strengthen illumination–reflectance consistency, pixel fidelity, and color preservation without incurring additional inference cost. Extensive experiments demonstrate that our framework achieves state-of-the-art performance in extremely low-light RAW enhancement, and robustly handles coupled motion and noise degradations.
Read the full abstract ▾
All authors Zepu Wang, Jingze Liang, Weijie Xiao, Kexin Chen
Computer Vision
STC-Net: Electroluminescence-Based Solar Cell Crack Segmentation for Power Loss Estimation
Shanaka Ramesh Gunasekara, Akila Eranda Devanarayana, Imasha Guruge et al. (+2 more)
Accurate crack assessment in electroluminescence (EL) images is important for photovoltaic (PV) reliability analysis, yet existing
segmentation methods often fail to capture the thin, elongated, and structurally constrained nature of crack defects. This paper proposes a
Solar Topology Crack Network (STC-Net) that incorporates edge priors, spectral priors, and a boundary-
topology refinement module to improve crack continuity and boundary preservation. The framework further extends
segmentation to power-loss estimation by deriving a crack-associated inactive-area proxy from the predicted masks. Experiments on the PVEL-S
dataset show that STC-Net achieves 95.98 MIoU, 98.01 MDice, and 98.00 MAcc during training, and 72.52 MIoU and 80.16 MDice on unseen test samples. These results demonstrate that STC-Net provides accurate crack localization while offering a practical link between EL-based defect
segmentation and PV degradation assessment.
Read the full abstract ▾
All authors Shanaka Ramesh Gunasekara, Akila Eranda Devanarayana, Imasha Guruge, Nuwantha Fernando, Ehsan Asadi
Computer Vision
SpatialQuery: Benchmarking Geometry-Grounded Multi-Instance Spatial Reasoning in Vision-Language Models
Hai Nguyen, Tung Vu and Cong Tran
Vision-
language models (VLMs) achieve strong semantic understanding but remain unreliable in metric spatial
reasoning, particularly when queries require comparing multiple instances of the same object category. We study this problem through the Closest-Instance Distance Query (CIDQ), where a model must identify the nearest visible candidate to a unique reference object and estimate their
gravity-aligned floor-plane distance. We introduce SPATIALQUERY, a training- free framework for CIDQ
reasoning from a single RGB image, together with SPATIALQUERY-1M, a
benchmark containing over one million RGB-only question-answer pairs from 200 indoor scenes. SPATIALQUERY recovers instance-level metric geometry and transforms it into a canonical Bird's-Eye View through Scene Cubifying, which represents objects as uniformly sized, category-coded blocks to emphasize their relative floor- plane locations. We further propose Uncertainty-Aware Chain-of-Thought (UA-CoT) prompting, which incorporates geometry- derived per-instance uncertainty into the VLM
reasoning process. Without task-specific fine-tuning or architectural modification, SPATIALQUERY with Qwen3-VL-8B achieves a Floor-MAE of 0.259 m, an Unc-Acc@0.3 m of 90.5%, and a proximity-decision accuracy of 84.18%, outperforming fine-tuned spatial specialists, general-purpose VLMs, and closed-source frontier models. Code,
benchmark resources, and an interactive demo are available at https://namhai1810.github.io/SpatialQuery/.
Read the full abstract ▾
Computer Vision
UniSim-SLAM: Feed-Forward SLAM with Unified Sim(3) Optimization
Inha Lee, Dongjae Jeong, Junhee Lee et al. (+1 more)
Recent
geometric foundation models enable feed-forward inference for SLAM, but their predictions are strongly dependent on the input view set, which leads to
geometric inconsistencies and trajectory drift when results are chained over long sequences. Online deployment further exposes a trade-off between the low latency of two-view tracking and the constraint richness of multi-view inference. We introduce UniSim-SLAM, an integrated system that runs lightweight two-view keyframe tracking in the frontend and performs periodic multi-view submap refinement in the backend. To combine predictions defined in heterogeneous local coordinates with inconsistent scales, we formulate a unified multi-level factor graph on $Sim(3)$ that jointly optimizes global keyframe poses and submap poses. The graph integrates temporal view-to-view odometry edges, view-to-submap bridge edges with depth-statistics scale anchoring, and submap-to-submap tie and scale constraints to enforce consistent similarity relations across submaps. Experiments on TUM RGB-D and 7-Scenes show that UniSim-SLAM achieves state-of-the-art accuracy in the uncalibrated setting, reducing trajectory error by $38.5\% $ on TUM RGB-D and $45.9\%$ on 7-Scenes compared to prior best results. Project page: https://
vision3d-lab.github.io/unisim-slam/
Read the full abstract ▾
All authors Inha Lee, Dongjae Jeong, Junhee Lee, Kyungdon Joo