Computer Vision
What Breaks in Multi-Image Schema Application? Factorial Oracle Interventions Reveal an Instance-Information Bottleneck
Yiliu Xu, Xinjin Li and Yu Ma
Modern
vision-
language models can accept multi-image contexts, but aggregate accuracy rarely reveals where a multi-image
reasoning pipeline fails. We study a controlled schema-application task in which a model must locate two target characters in a 16-image sequence, identify their visual attributes, consult an item-specific visual codebook, and compose two ordered nonce code names. We introduce a within-item 2-by-2 factorial Oracle design: Oracle-A supplies a verified textual schema, Oracle-B supplies a verified ordered target-appearance representation, and both interventions retain the original images and final question. Across 72 new items and 1,188 analyzed model responses, Oracle-B increased strict exact accuracy by 54.6 percentage points (95% confidence interval: 46.3 to 63.0) and outperformed Oracle-A by 60.6 points (Holm-adjusted p-value no greater than .0002). An auxiliary
retrieval diagnostic reached only 49.1% ordered-appearance accuracy even after removing code-name generation. Providing both Oracle sources yielded 100% accuracy across all models. These results localize the dominant recoverable deficit to upstream ordered instance recovery rather than primarily to downstream schema use and composition.
Read the full abstract ▾
Computer Vision
CORE: Contrast-and-Reflect for Self-Supervised Prompt Optimization in Anomaly Synthesis
Haoxiang Rao, Fang Zhao, Zhao Wang et al. (+3 more)
Industrial anomaly detection (AD) is bottlenecked by the scarcity of anomalous samples, making high-fidelity anomaly synthesis crucial for improving downstream performance. However, existing methods either rely on scarce real anomaly references or auxiliary data, while lacking complex
reasoning and iterative optimization capabilities for generating realistic defects in open-set scenarios. To address these challenges, we propose CORE, a cost-efficient and training-free
prompt optimization framework based on Large
Multimodal Models (LMMs), which unifies zero-shot and one-shot anomaly synthesis. CORE first introduces an Anomaly Context Grounding module to transform sparse user context into defect-aware generation context, which is subsequently instantiated as structured anomaly
prompts that disentangle normal-region preservation from anomaly-specific variation, enabling controllable and faithful defect generation. Furthermore, CORE autonomously refines
prompts through a self-supervised contrast-and-reflect mechanism. This design empowers general-purpose LMMs to synthesize realistic anomalies under both zero-shot and one-shot settings without task-specific training. Extensive experiments on MVTec-AD and VisA demonstrate that CORE synthesizes realistic and diverse anomalies, yielding superior improvements in downstream AD performance. Furthermore, its model-agnostic and plug-and-play paradigm enables effortless integration with diverse LMMs, facilitating adaptation to unseen open-set scenarios.
Read the full abstract ▾
All authors Haoxiang Rao, Fang Zhao, Zhao Wang, Weijia Li, Chenyang Si, Caifeng Shan
Computer Vision
LED-BENCH: A Benchmark for Large-format Engineering Drawing Understanding
Ke Li, Tianyi Wang, Enning You et al. (+4 more)
Real-world engineering drawings are often large-format and information-dense. Answering practical engineering questions therefore requires both fine-grained understanding of engineering information and complex
reasoning over objects, relations, and evidence distributed across different regions or views. Existing
benchmarks cover a range of engineering-drawing understanding tasks involving local regions, individual views, specific elements, or more constrained visual settings. However, systematic evaluation of understanding over realistic, large-format engineering drawings remains limited. To address this gap, we introduce LED-BENCH, a multidisciplinary
benchmark designed to evaluate fine-grained understanding and complex
reasoning over large-format engineering drawings, spanning the mechanical, architectural, and transportation domains. Extensive evaluation of 11 representative
multimodal large
language models, including 8 proprietary models and 3 open-source models, reveals substantial remaining difficulty: GPT-5.6 Sol achieves 52.65% average accuracy, performance varies across engineering domains, and all evaluated models perform worse on questions requiring broader evidence integration. These results highlight fine-grained understanding, evidence integration, and complex
reasoning over large-format engineering drawings as persistent challenges for current
multimodal models.
Read the full abstract ▾
All authors Ke Li, Tianyi Wang, Enning You, Chaoqun Wan, Zhen Huang, Shuxian Liang, Xian-Sheng Hua
Computer Vision
Frame Evidence:A Benchmark for Reference Frames and Visual Evidence in Multi-Image Spatial Reasoning
Chunxi Luo, Qishuai Ding, Xingchen Peng et al. (+6 more)
A spatial answer depends on the reference frame specified in
language and its supporting visual evidence. Item accuracy alone obscures whether correctness persists when either changes. We introduce Frame Evidence, a
benchmark of 320 matched triplets and 960 Chinese multiple-choice questions from captured and rendered scenes. One family compares three grounded reference frames over fixed images. The other pairs image deletion annotated as insufficient with answer-preserving addition. Joint metrics require correct answers across all three frames and paired evidence variants. Across 20 configurations on 288 held-out questions, GPT three-shot reaches 50.00% item accuracy, but the best three-frame group accuracy is 5/48 (10.42%), reached by GPT two-shot. Few-shot prompting and task-specific adaptation improve item accuracy without consistently improving joint correctness. Input controls show where images support matched success, while repeated executions reveal answer instability beneath similar scores.
Read the full abstract ▾
All authors Chunxi Luo, Qishuai Ding, Xingchen Peng, Hengwei Ye, Ke Zhang, Wenxin Lian, Tianyi Lu, Chunlong Yu, Zhenya Lin
Computer Vision
SaTPO: Text-to-Image Safety Alignment via Transition Preference Optimization
Ziyi Fang, Jingzhi Li, Xiaojun Jia et al. (+3 more)
Safety
alignment of text-to-image
diffusion models aims to suppress harmful content without compromising useful generation capabilities. However, preferences over completed images compress multidimensional generation factors into a coarse signal, potentially driving changes beyond those required for safety. To address this, we propose Safety Transition Preference Optimization (SaTPO), which enforces step-wise
alignment during denoising. SaTPO leverages the insight that an unsafe
prompt and its safe rewrite naturally form a tightly coupled preference pair when conditioned on a shared parent state and noise realization. By anchoring optimization on identical parent states, SaTPO eliminates historical trajectory divergence, confining safety adaptation to an incremental
alignment while preserving task-relevant semantics. Multi-scale drift constraints and benign replay further limit behavioral deviation and preserve ordinary generation. We additionally construct CoProPV, a refined
dataset ensuring visually verified contrasts. Experiments on Stable
Diffusion v1.5 and SDXL demonstrate improved safety-quality trade-offs. Compared with AlignGuard, PerspectiveVision unsafe-detection rates on Inappropriate Image
Prompts decrease from 12.70% to 6.00% and from 15.04% to 9.30%, respectively, while FID improves from 32.28 to 25.95 and from 36.50 to 27.51. Warning: this paper includes potentially offensive content.
Read the full abstract ▾
All authors Ziyi Fang, Jingzhi Li, Xiaojun Jia, Xingxing Wei, Wenqi Ren, Haijun Zhang
Computer Vision
Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling
Jing-Xuan Zhang, Genshun Wan, Jianqing Gao et al. (+3 more)
Lipreading remains challenging due to the inherent ambiguity of visual
speech cues and their sensitivity to head pose and
video resolution. In this work, we propose
FLARE, a framework for robust visual
speech representation learning via cross-modal
distillation and joint face-lip modeling.
FLARE leverages semantically rich Whisper representations as the acoustic teacher signal, converting them into discrete
distillation targets through a random-projection quantizer and pretraining the model with a masked prediction objective. To capture complementary visual information,
FLARE employs a dual-stream architecture that jointly processes lip region-of-interest (ROI) and full-face inputs, where the lip stream captures finegrained articulatory motion and the face stream encodes broader facial dynamics. During pretraining, auxiliary audio input is incorporated to facilitate cross-modal
alignment, and modality dropout is applied to encourage robust visual-only representations. For downstream lipreading,
FLARE serves as a visual encoder coupled with either a lightweight
Transformer decoder or an LLM-based decoder. Evaluated on LRS2, LRS3, and the out-of-domain WildVSR
benchmark,
FLARE achieves strong lipreading performance, with word error rates of 11.7%, 13.2%, and 34.3%, respectively. A progressive design study confirms the contribution of each major component. Further analyses demonstrate that joint face-lip modeling improves robustness under degraded
video resolution and large head pose variation, while yielding more phoneme-discriminative visual representations. Code and pretrained models are available at https://github.com/jxzhanggg/
FLARE.
Read the full abstract ▾
All authors Jing-Xuan Zhang, Genshun Wan, Jianqing Gao, Jia Pan, Yumei Zhang, Lichen Zhang
Computer Vision
TextVerifier: Seeing Beyond Recognition to Verify Chinese Text Rendering
Yiyang Chen, Guo-Hua Wang, Yaoli Liu et al. (+4 more)
Text-to-image models increasingly generate readable Chinese text that still contains local rendering defects, such as a character with an extra or missing stroke or a malformed component. Such errors are invisible to OCR and
vision-
language models, which are trained for recognition and are therefore invariant to the very structural deviations that define a rendering error. We formalize text rendering verification as a target-conditioned problem: given a target character and an image crop, decide whether the crop is a correct rendering of that character. Our verifier TEXTVERIFIER feeds the text tower a character's Ideographic Description Sequence (IDS) rather than a single-character symbol, and is trained on 12.7M crop-IDS pairs plus label-free "same-character, different-quality" hard pairs sampled from the training trajectory of a text-region inpainting model. To evaluate this ability we build TEXTRENDERBENCH-ZH, the first per-character
benchmark for generated Chinese text, annotating position, identity, and correctness for 7,358 characters. TEXTVERIFIER reaches an F1 of 0.844, far above OCR confidence (0.526), a general VLM (0.329), and TextPecker (0.590). Used as a per-character reward for Flow-GRPO post-training, it further reduces rendering errors, showing that its score can both detect and correct errors.
Read the full abstract ▾
All authors Yiyang Chen, Guo-Hua Wang, Yaoli Liu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Xiujun Ma
Computer Vision
Cross-Scenario Industrial Defect Detection via Domain-Robust Training and Metric-Aligned Selection
Dikai Li, Weijia Wang and Yuan-Gen Wang
Automated defect detection of industrial boards must remain highly reliable under cross-scenario variations in lighting, surface contaminants, and production line conditions. This makes cross-scenario defect detection a domain generalization problem, wherein models are required to detect, localize, and classify defects across unseen scenarios while minimizing false alarms on clean boards. For the IDA 2026 Track 1 challenge, we propose a cross-scenario defect detection method that includes four complementary modules: cross-scenario robust training, small-defect-aware learning, score calibration and inference threshold selection, and metric-aligned checkpoint selection. These four modules respectively introduce cross-scenario robust normalization and DSU-style statistic perturbation with progressive random convolution, NWD and parameter-free spatial
attention, the VFL learning objective, and a custom SWAD variant driven by the Track 1-oriented proxy. Experimental results demonstrate that our model achieves a Track 1 score of 0.564 with a screening score of 0.696 on the test
dataset on a single RTX 4090 GPU. The code is available at https://github.com/GZHU-DVL/Dikai.
Read the full abstract ▾
Computer Vision
RDCompGS: Rate-Distortion Optimization for Compression of 3D Gaussian Splatting with Image Codecs
Huiping Ren, Shuai Wan, Luqian Bai et al. (+2 more)
3D Gaussian Splatting (3DGS) enables high-quality, real-time novel view synthesis but requires significant storage and transmission resources. Mapping Gaussian attributes to 2D representations enables efficient compression using mature image codecs. However, Gaussian representations that achieve comparable rendering quality are generally treated similarly during optimization, despite substantial differences in coding rate, leading to limited rate-distortion (R-D) performance after compression. To address this limitation, we propose RDCompGS, a 3DGS compression method that enables overall R-D optimization of Gaussian representations with respect to subsequent coding. A differentiable estimate of the coding rate is incorporated into the R-D optimization with rendering distortion. The rate constraint adjustment model further regulates the R-D trade-off across different compression conditions. Extensive experiments on three
benchmark datasets demonstrate that RDCompGS achieves average Bjøntegaard Delta Rate reductions of 37.84% and 49.31% with H.265/HEVC and JPEG XL, respectively. The estimated rate also shows a strong correlation with the actual bit-rate under different compression conditions.
Read the full abstract ▾
All authors Huiping Ren, Shuai Wan, Luqian Bai, Zhang Chen, Fuzheng Yang
Computer Vision
Reducing the Gap Between Spatial and Space-Filling Curve Neighbors in 3D Point Cloud Segmentation
Konan Sasaki, Junya Komori and Kazuhiro Hotta
As a lightweight and highly accurate method for nearest-neighbor search in 3D point clouds, approaches based on space-filling curves (SFC) have been proposed. However, due to the inherent properties of SFC, a gap arises between proximity relationships along the SFC and those in the original Euclidean space. In this study, we propose a loss function that reduces the influence of this gap. Specifically, we compute the distances between adjacent points along the SFC and train the model to optimize feature similarity between pairs of points that are distant from each other in the original 3D space. We incorporate the proposed method into a high-performing 3D point cloud segmentation model and evaluate its effectiveness on three
datasets. The results demonstrate that the proposed method outperforms conventional methods on both indoor and outdoor point cloud
datasets.
Read the full abstract ▾
Computer Vision
HyrCap: Hybrid Rank-Calibration of Action Proposals for Temporal Event Understanding
Rui Chen, Xingyu Chen, Pengxin Xu et al. (+2 more)
Temporal action detection in long untrimmed
videos suffers from inaccurate localization and low-quality proposals, especially with sparse annotations and ambiguous boundaries. For open-ended
video understanding and LLM-driven event analysis, reliable proposals connect temporal localization, event annotation, and semantic understanding. However, conventional temporal action detection (TAD) pipelines lack explicit modeling of proposal localization quality, cross-representation support, and inter-proposal relations. We propose Hybrid Rank-Calibration of Action Proposals (HyrCap), a proposal-centric query-based TAD framework for temporal event understanding. Hybrid combines complementary
video features with different localization cues, while Rank-Calibration adjusts proposal ranking confidence to reflect localization reliability and candidate competition. HyrCap treats dense query-based predictions as an over-complete temporal event hypothesis space. Cross-Representation Consensus Calibration, Localization Quality Estimation, and Proposal Redundancy Suppression improve proposal ranking reliability, localization quality, and candidate discrimination. Inspired by LLMs' capabilities in fine-grained semantic recognition, event description, and
reasoning, we introduce HyrCap for Semantic Action Understanding, using calibrated proposals for standard TAD and as inputs to LLM-based semantic understanding. We define a staged evaluation protocol to separately measure event localization, segment-level semantic understanding, and combined event understanding performance. Experiments on public
benchmarks show that HyrCap achieves state-of-the-art performance across multiple metrics and provides a finer-grained paradigm and evaluation protocol for query-based event segmentation and annotation.
Read the full abstract ▾
All authors Rui Chen, Xingyu Chen, Pengxin Xu, Kai Chen, Junzhi Yu
Computer Vision
Spectral World Renderer: Unified 3D Hyperspectral Unmixing
Chia-Ming Lee, Yu-Jou Hsiao, Ming-Ching Chang et al. (+3 more)
Cameras and the foundation models built on them recognize what a 3D scene contains, but not what it is made of: surfaces of the same colour can be different materials. Hyperspectral unmixing explains each spectrum as a mixture of material signatures, yet in 3D it is applied to rendered images after the views are composited, or tied to slow implicit fields, and no multi-view data with known composition exist to check the result. We present the Spectral World Renderer (SWR), which unifies world generation and unmixing. A coding
agent builds executable 3D worlds that are rendered with measured spectra, so every hyperspectral image comes with exact labels and sub-pixel proportions; the resulting SWR-Bench provides eight indoor and outdoor worlds in which decompositions can be verified. SWR then unmixes inside rendering: each Gaussian of a compact 3D representation is decoded into material abundances before compositing, so one render yields spectra and material maps, and any surface can be queried by material name, a clicked pixel or a reference spectrum. Without material labels, SWR identifies materials best in all eight worlds, has the best mean PSNR on real captures, trains 3.7-12.9x faster than unmixing baselines, renders 34.9 labelled views/s and separates same-coloured materials.
Read the full abstract ▾
All authors Chia-Ming Lee, Yu-Jou Hsiao, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung Hsu
Computer Vision
Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence
Yuanyuan Gao, Zhirui Liu, Yuning Gong et al. (+9 more)
Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor
datasets are dominated by room-scale captures and costly
LiDAR acquisition. Online real-estate platforms offer abundant panoramas of diverse multi-room homes, but sparse viewpoints and texture-poor regions make reconstruction severely under-constrained. We introduce Holi-Spatial 360, a generative reconstruction and data curation framework for converting these panoramas into annotated multi-room 3D environments. Because reconstruction quality bounds subsequent generation, our Correspondence-Guided
Geometric Optimization converts feed-forward reconstruction priors into reliable cross-view constraints and builds a coherent 3D Gaussian Splatting representation as 3D spatial
memory. Adapted single-image and
video priors then restore appearance and complete continuous trajectories while preserving observed content. The reconstructed scenes support arbitrary RGB-D rendering and an automated curation pipeline that combines end-to-end 3D grounding with VLM-based verification. Holi360 contains over 1K multi-room 3DGS scenes, more than 100K instance annotations, and 2M spatial question-answer pairs. We also introduce the Holi360
benchmark to evaluate the spatial understanding of VLMs in real-world, multi-room indoor environments. Experiments demonstrate superior reconstruction and curation quality under sparse panoramic inputs, while fine-tuning VLMs on Holi360 substantially improves spatial
reasoning capabilities.
Read the full abstract ▾
All authors Yuanyuan Gao, Zhirui Liu, Yuning Gong, Yifei Liu, Yifan Zhan, Kuixuan Jiao, Xiaolong Zhou, Yifan Yang, Hongjie Zhang, Xue Yang, Dan Xu, Zhihang Zhong
Computer Vision · Cross-listed from cs.RO
Robota-Embed: Unified VLA Embeddings for Robot In-Context Learning and Evolution
Yunze Liu, Jing Wang, Shenrui Wu et al. (+16 more)
Retrieving useful context for a
robot requires matching its task, execution stage, and the continuation that should follow. We introduce Robota-Embed, to our knowledge the first
vision-
language-action (VLA)
embedding model to jointly encode all three modalities in both current-state queries and recorded future continuations. Predictive
retrieval training aligns these endpoints, enabling historical continuations to serve as context at inference time. We construct a multi-source corpus of 1,695,684 cleaned training anchors and introduce Robota-Embed-Bench, with 4,000 queries and a shared 8,000-entry gallery. Robota-Embed achieves 47.33% clip-level and 81.77% episode-level R@1, exceeding action-free Gemini
Embedding 2 by 22.09 and 33.47 percentage points. Controlled
retrieval studies show that removing native action degrades
retrieval more than removing
vision. We study two complementary applications. Embed2ICL uses retrieved experience for training-free and training-based policy conditioning, increasing success from 47.0% to 56.3% in DROID-Sim and from 86.4% to 92.8% on the LIBERO difficult subset, while improving task progress on real
robots. Embed2Evolve uses execution feedback to grow task-local
memory and refine context use around a frozen GPT-6 Astra policy. Across five ManiSkill tasks, it achieves 68% success, compared with 24% for GPT-6 Astra Direct and 33% for a harness independently evolved with Omni-Embed-Nemotron-3B under the same protocol. Relative to the latter, it uses approximately 79.5% fewer policy
tokens per test rollout, counting both successes and failures. These results show that a unified, action-aware
retrieval space can support
robot in-context learning and task-local context
evolution.
Read the full abstract ▾
All authors Yunze Liu, Jing Wang, Shenrui Wu, Wenkun He, Guanqi Zhan, Xianzheng Ma, Junyu Chen, Wenhang Ge, Haozhan Tang, Yuchao Gu, Zhuoyang Zhang, Zekai Wang, Dantong Niu, Zhen Dong, Yuke Zhu, Trevor Darrell, Linxi "Jim" Fan, Song Han, Han Cai
Computer Vision
Robust Real-Time Monocular Human Pose Estimation with Insole-Based Sensor Fusion
Liang Yu, Tao Zhu, Xiangqing Lu et al. (+2 more)
Real-time monocular human pose estimation is an important sensing component for human-centered intelligent systems. While recent camera-based methods have achieved impressive accuracy, fundamental challenges persist, particularly when visual observations are incomplete, such as during lower-body occlusion or short out-of-frame intervals. In these challenging but common scenarios,
vision-only systems often struggle to reconstruct physically plausible lower-limb motion and reliable foot-ground contact. Due to the highly unobtrusive and wearable nature of smart insoles, this paper investigates whether simple plantar pressure and acceleration sensors can effectively resolve these visually induced limitations. To systematically investigate this, we first collect a synchronized
multimodal dataset and then propose a part-decoupled temporal fusion framework. The proposed model fuses foot-sensing features into the lower-body branch to compensate for incomplete visual evidence, while a lightweight coordinator exchanges information between the upper- and lower-body temporal representations to maintain whole-body consistency. The framework is evaluated with three representative real-time visual backbones based on 2D keypoints, 3D landmarks, and SMPL-based human mesh recovery. We evaluate our framework on three representative real-time visual backbones. Quantitative experiments demonstrate that incorporating simple foot sensors consistently improves pose accuracy, motion smoothness, and gait-related measurements over
vision-only baselines, with the most significant gains observed under occlusion and short visual-dropout conditions. Ultimately, our results demonstrate that unobtrusive foot sensing provides vital and complementary physical observability when camera observations are temporarily unavailable.
Read the full abstract ▾
All authors Liang Yu, Tao Zhu, Xiangqing Lu, Wenjian Luo, Liming Chen
Computer Vision
ActICL: Post-Training for Agentic Multimodal In-Context Learning
Yanshu Li, Ruixiang Tang, Junyi Jessy Li et al. (+1 more)
Multimodal in-context learning (ICL) is an important capability of
multimodal large
language models (MLLMs). Its performance is closely tied to the demonstrations it uses, making demonstration
retrieval a core problem for ICL. Common methods use fixed scoring methods to retrieve all the demonstrations at once, which may introduce redundancy and noise. Context-aware
retrieval methods have emerged to address this problem by allowing incremental
retrieval from a growing context. However, none of them lets the MLLM learn how to judge what new demonstrations it needs and then carry out progressive
retrieval on its own. To this end, we introduce ActICL, which trains an MLLM
agent to acquire and use an ICL context via tool-augmented
reasoning. An ActICL
agent analyzes its own information gaps during iterative
reasoning and tool-use actions. Based on these gaps, it performs targeted
retrieval to acquire a set of initial candidates. It can then use the existing context to further filter the candidates and refine the context, or it can iteratively perform a new
retrieval. When the
agent judges that the context is sufficient for the task, or when the resource limit is reached with the user request met, it sends the acquired context to an answering model, either the
agent itself or another MLLM. We use a two-stage post-training procedure that jointly optimizes the
agent's acquisition and answering. With an adaptive number of demonstrations, ActICL improves average performance by 11.6 percentage points over standard similarity-based
retrieval across seven
benchmarks and three MLLMs, and by 4.4 points compared to the strongest prior baseline. The contexts acquired by ActICL
agents can also be used effectively by frontier models. When GPT-5.6-Sol performs 4-shot ICL, ActICL's contexts improve its performance by 3.5 points over standard
retrieval, averaged over three
benchmarks.
Read the full abstract ▾
All authors Yanshu Li, Ruixiang Tang, Junyi Jessy Li, Elias Stengel-Eskin
Computer Vision
DeBoneDiT: Depth-Driven Conditional Bridge Diffusion Transformers for Bone Suppression in Chest X-Ray Images
Yifei Sun, Fenglei Fan, Junhao Jia et al. (+4 more)
Chest X-Ray (CXR) is a primary modality for diagnosing pulmonary diseases, yet the overlap of bony structures often obscures critical radiographic findings, increasing the risk of diagnostic uncertainty. While Dual-Energy Subtraction (DES) is the clinical
benchmark for bone suppression, its widespread adoption is primarily hindered by specialized hardware requirements and increased radiation exposure. Recently,
diffusion-based generative models have shown promise for this task, yet they are still constrained by reconstruction-biased latent compression, prior-induced domain gap, and a scarcity of high-quality paired
datasets. To address these challenges, we propose DeBoneDiT, a depth-driven conditional bridge
diffusion Transformer architecture for efficient high-resolution bone suppression. We first introduce Depth Auto-Encoder (DAE), a structure-aware
vision tokenizer for latent compression. By incorporating a multi-level depth feature loss derived from the pretrained Depth Anything V2 encoder, DAE preserves both visual fidelity and structural consistency. Second, we formulate bone suppression as a source-conditioned Brownian bridge
diffusion process, leveraging CXRs as both the
diffusion prior and structural guidance to narrow the source-to-target domain gap and maintain anatomical plausibility. Furthermore, we have constructed and released SZCH-X-Rays, the largest publicly available high-quality paired
dataset for bone suppression to date, comprising 741 pairs of CXR and DES soft tissue images. Extensive experiments and downstream evaluation demonstrate that DeBoneDiT achieves superior bone suppression performance with reduced computational overhead.
Read the full abstract ▾
All authors Yifei Sun, Fenglei Fan, Junhao Jia, Wenming Deng, Hongxia Xu, Changmiao Wang, Ruiquan Ge
Computer Vision
Learning to Select Features with Manifold Deep Unfolding PCA
Jingjing Liu, Pei Tai, Xianchao Xiu et al. (+1 more)
Unsupervised feature selection (UFS) is of great significance in image processing and pattern recognition. Nevertheless, existing model-based methods often rely on manually tuned hyperparameters, while deep learning-based methods generally require massive training data. To address these issues, this paper proposes an efficient
manifold deep unfolding feature selection network (MDUFS-Net). Technically, we first incorporate the $\ell_{2,0}$-norm into the classic principal component analysis (PCA) framework, then unfold the alternating direction method of multipliers (ADMM)-based iterative
algorithm into an end-to-end trainable network, thereby enabling automatic parameter learning. Moreover, a
manifold optimization strategy is developed to improve the numerical robustness. On this basis, we further extend MDUFS-Net into an enhanced variant called MDUFS-Net+ by introducing the $\ell_{0}$-norm for more discriminative feature representations. Extensive experiments validate that our proposed MDUFS-Net and MDUFS-Net+ outperform state-of-the-art methods in the UFS task, with promising performance in clustering. Our code is available at https://github.com/xianchaoxiu/MDUFS.
Read the full abstract ▾
All authors Jingjing Liu, Pei Tai, Xianchao Xiu, Wanquan Liu
Computer Vision
VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness
Zihao Lin, Haoyu Dong, Haobo Xu et al. (+10 more)
LLM system capabilities depend not only on model parameters but also on the executable harnesses that organize information acquisition, tool use,
reasoning, and
memory. Recursive harness
evolution can automate their design, but how it can reliably discover generalizable systems from limited, noisy, and repeatedly reused feedback remains unclear. We study this question in agentic
video understanding, where rich
multimodal evidence and coupled mechanisms make harness optimization challenging. We first introduce VideoHarness-BRSI, a recursive self-improvement framework that evolves
video harnesses around a frozen
vision-
language model. Analyzing over 4,000 evolved candidates, we find it competitive with hand-engineered systems but identify three interconnected failure modes: validation overfitting, search-budget dilution, and the early-experience trap. We trace all three to a common root: the same imperfect validation evidence drives both harness revisions and the proposer's search strategy. We therefore propose VideoHarness-SRSI, an evidence-governed framework in which every search decision rests on evidence that is transferable, sufficient, and gathered before the search commits. It restricts the proposer to content-free feedback, informs exploration with instrumented warm-up experiments, and develops mechanisms in isolated probe-race branches before factorial composition and joint refinement. Across four
video understanding
benchmarks, VideoHarness-SRSI improves the equally weighted mean held-out score by 6.2% relative to VideoHarness-BRSI and reduces the mean validation-test gap by 37.9%. Reliable harness
evolution thus depends on how search is organized and informed, not only on how well it optimizes validation performance.
Read the full abstract ▾
All authors Zihao Lin, Haoyu Dong, Haobo Xu, Chi-Chih Chang, Xuying Ning, Yinglong Xia, Shuang Yang, Zhuokai Zhao, Xinying Song, Dongfang Liu, Lifu Huang, Yang Song, Qifan Wang
Computer Vision · Cross-listed from cs.RO
From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents
Yue Zhang, Han Lin, Zun Wang et al. (+3 more)
Simulated environments enable scalable training data generation for embodied
agents, but more data does not necessarily provide what a particular
agent needs to learn. Existing environment-generation pipelines largely optimize generic properties such as realism, diversity, or task coverage, while different pretrained policies can fail for different environmental reasons. We therefore introduce WEDGE, a framework for weakness-guided navigation learning in which the failure modes of a pretrained
agent guide the generation of training environments at each adaptation round. Given a pretrained policy, WEDGE uses an LLM to propose executable weakness features from contrasting successful and failed rollouts, automatically validates them on the diagnostic set, and selects the strongest validated weakness based on its success-rate gap. The validated weaknesses are then mapped to the environmental factors that control them. WEDGE varies these factors to construct environments in the difficulty regimes where the policy struggles, producing targeted training experience for adaptation. Across three navigation
agents with different adaptation mechanisms, we find that policies exhibit distinct weakness profiles and benefit from different training environments. Weakness-conditioned adaptation consistently outperforms matched random and hard-example alternatives on ProcTHOR, while the resulting gains for Uni-NaVid and Gemini also extend to HM3D after ProcTHOR adaptation. Iterative re-diagnosis further shows that dominant weaknesses can persist or shift as the policy improves, providing updated guidance for subsequent environment construction. Together, these results show that policy failures are not only diagnostic signals, but actionable guidance for constructing training environments that target a policy's current weaknesses.
Read the full abstract ▾
All authors Yue Zhang, Han Lin, Zun Wang, Yidong Huang, Zaid Khan, Mohit Bansal
Computer Vision
WinARDet: Compressed Window Auto-Regression for Long-Horizon Vision-Language Grounding
Chenxu Wang, Hangyi Kuang, Bo-Wen Yin et al. (+6 more)
MLLM-based detectors formulate
vision-
language grounding as autoregressive coordinate generation, yet in dense scenes, they often collapse into a box repetition loop, repeatedly emitting identical boxes until the decoding budget is exhausted. We trace this failure to historical-answer
attention takeover: Under full-history autoregression, repeated coordinate
tokens in the growing answer history attract disproportionate
attention, crowding out the
attention sinks needed to attend to new visual evidence. % To break this loop, in this paper, we propose Compressed Window Auto-Regression (WinAR). This new mechanism partitions a long answer sequence into consecutive generation windows and restricts full
token-level autoregression to the current window only while compressing each previous window into a fixed set of latent
tokens that condition subsequent decoding. This design reduces the repeated exposure to raw historical coordinates that triggers the
attention takeover, yet preserves cross-window dependencies essential for coherent multi-object generation. On RSDense, which contains 200-2,000 objects per image, our model WinARDet improves RexOmni F1-score (IoU=0.5) from 32.16 to 39.36 over the baseline, while reducing the redundancy rate from 35.48% to 13.00%. WinARDet also generalizes across decoding strategies and output formats, including parallel box decoding and polygon generation, providing a practical paradigm for long-horizon
vision-
language grounding across scene densities.
Read the full abstract ▾
All authors Chenxu Wang, Hangyi Kuang, Bo-Wen Yin, Yuxuan Li, Quan-Sheng Zeng, Yang Zhang, Xu Guo, Xiang Li, Qibin Hou
Computer Vision · Cross-listed from cs.CL
Vision-Language Models Integrate Conflicting Evidence Like Biased Bayesian Observers
Josh Barua, Kyle Mahowald and William Rudman
Vision-
language models must reconcile conflicting text and images. Prior work has characterized the resulting behavior in terms of modality biases–e.g., a "blind faith in text"–but why preferences vary across models, domains, and input conditions remains unclear. We develop a predictive account from a
Bayesian perspective, positing that VLMs combine conditionally independent evidence from each modality with prior beliefs, subject to modality-specific biases. The resulting two-parameter cue combination model accurately predicts VLM preferences under unseen combinations of input degradation and transfers to new forms of unreliability without refitting. Separating evidence strength from integration bias explains how VLMs can follow text even while weighting visual evidence more heavily, reconciling reports of text and
vision dominance under a single combination rule.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Train to Stitch: Topology-Preserving Regularization for Interoperable Representations
Akshay Malhotra, Nicolas Zilberstein and Shahab Hamidi-Rad
As large, powerful pre-trained models become more prevalent, there is growing interest in developing methods that allow different neural networks to be seamlessly combined with little or no direct communication between them. Take autoencoder-based data compression as an example: the ability to independently choose architectures and train encoder and decoder models, while still ensuring they work together, could fundamentally transform how such models are built, deployed, and updated. In this work, we introduce a novel approach that leverages topological regularization to align the latent spaces of two independently trained autoencoder models, entirely eliminating the need for coordination during training. We provide a conditional theoretical analysis showing how the structural agreement encouraged by our regularizer can translate into approximate
geometric compatibility between independently learned representations. Reconstruction experiments validate this approach empirically, substantially narrowing the interoperability gap across multiple
datasets.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Robust Explanatory Analysis for Structured Optimization Networks (REASON) in Learning
Vahid Jebraeeli, Hamid Krim and Keisuke Fujii
We seek a hierarchical resolution of residual error under a composite learning objective as a principled mechanism for identifying and exploiting additional information, particularly under distribution shift and perturbation. Inspired by the System 1/System 2 paradigm, we formulate this principle as a two-loop mechanism serving a common inference goal. A fast, System 1-like loop resolves the residual addressable by the available evidence; the remaining epistemic residual triggers a medium speed, System 2-like loop that actively acquires additional evidence and measures its informational value through residual reduction. Over nested solution spaces, the medium-loop optimum is provably no worse than the fast-loop optimum. Experiments show that the resulting two-loop model consistently improves upon the fast loop at approximately half the computation of the medium loop, including under unseen corruptions, and, when provided a cleaner observation, outperforms staged classifiers and learning-to-defer methods at equal computational cost.
Read the full abstract ▾
Computer Vision
The Impact of Detector Architecture, Model Scale, and Scene Density on Multi-Pedestrian Tracking
Pedro Manoel Herminio Alves, Dalton Cézane Gomes Valadares and Cláudio de Souza Baptista
Pedestrian tracking in
video is important for applications such as surveillance and scene analysis, but it becomes more challenging in crowded scenarios with frequent occlusions and rapid motion. This paper analyzes how detector architecture, model scale, scene density, and inference regime affect the tracking of multiple pedestrians. We compare the YOLO11, RT-DETR, and YOLO26 detector families with the ByteTrack and BoT-SORT trackers on sequences from the MOT17, MOT20, and SOMPT22
benchmarks. For YOLO26, we also compare traditional overlap filtering via Non-Maximum Suppression (NMS) with end-to-end inference. The RT-DETR family achieved the highest mean scores on the HOTA tracking quality metric across all three
benchmarks with both trackers, with a more pronounced advantage on MOT20, albeit at a higher computational cost. On MOT17 and SOMPT22, YOLO26 variants, especially YOLO26-L with NMS, offered favorable combinations of quality and speed compared with RT-DETR. In the YOLO26 inference comparison, the end-to-end mode did not yield consistent tracking gains over the NMS-based regime. Taken together, these results provide a practical guide for selecting detectors and trackers for this task based on application priorities.
Read the full abstract ▾
Computer Vision
VLX-Go: Robust Embodied Tracking through Multi-Stage Learning and On-Policy Refinement
Kyusong Lee, Heting Ying and Tiancheng Zhao
Embodied visual tracking requires a
robot to identify a designated target from visual observations and control its own motion to follow that target through an environment. Following an isolated person is a comparatively simple demonstration of embodied visual tracking, but following the intended person among distractors remains challenging. Existing methods exhibit a substantial drop in success from single-target to distractor tracking, motivating a focus on the shared spaces that person-following
robots must navigate. We present \method{}, a
vision–
language waypoint policy built around the hypothesis that visual and
language representations should be aligned before tracking-specific motion is learned. The policy combines complementary frozen visual encoders, multiscale temporal context, a
transformer backbone, and a lightweight head for
robot-centric waypoint prediction. Its training progresses through
vision–
language alignment,
multimodal reasoning mid-training, and tracking specialization on 19,532 oracle episodes. Compared with 41.54% distractor-tracking success for our earlier OmTrackVLA model, the complete multi-stage recipe achieves 55.80%. We then explore on-policy refinement using filtered oracle labels for states visited by the policy. This preliminary study adds 2.42 percentage points of distractor-tracking success, reaching 58.22%, while reducing collision rate by 1.99 pts. The final policy records the highest distractor- and ambiguous-tracking success rates among the compared methods. These results support staged
multimodal learning as a promising direction for tracking among distractors and motivate further study of on-policy refinement for person following in shared spaces.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Integrated Novelty Detection Systems: From Sensor Networks to Multi-Novelty Computer Vision
Irené Tematelewo
Novelty detection is crucial in various technological and scientific domains. Its importance spans from ensuring the reliability of sensor networks to enhancing the adaptability of computer
vision systems. As AI systems become increasingly sophisticated and complex, effective anomaly detection is essential for maintaining reliability and accuracy. This work addresses novelty detection in weather sensor networks and computer
vision. In weather sensor networks, particularly for monitoring rainfall, detecting blocked rain gauges can significantly impact data reliability, food security, and disaster management. This work evaluates four existing quality control methods for detecting blocked rain gauges and demonstrates their limitations, particularly in detecting clusters of faulty sensors. By comparing methods such as the Conditional Mixture Model (CMM), PSWQ (Fz-filter), Kernel Regression, and Gaussian Method, we identify their shortcomings and propose three novel quality control techniques that integrate probabilistic fault detection and consistency-based diagnosis framework to improve timely detection of blocked sensors, while reducing false alarms. In the realm of computer
vision, current systems often fail to handle multiple types of novelty due to their focus on single-novelty scenarios. We highlight the need for integrated systems capable of handling multiple types of novelty. To support this, we introduce the Novel Snapshot Serengeti (NOVEL-SS)
dataset, derived from the Snapshot Serengeti camera trap data, which includes various forms of novelty. We also present the DCA (Detect, Characterize, and Accommodate) baseline system, designed to handle multiple novelty by detecting, characterizing, and accommodating them. This work aims to advance the robustness and adaptability of novelty detection systems across both weather sensor networks and computer
vision applications.
Read the full abstract ▾
Computer Vision
MMKG-R1: Constructing Multimodal Knowledge Graphs via Reinforcement Learning
Zongyu Wu, Yilong Wang, Xiaochen Wang et al. (+5 more)
Multimodal knowledge graphs (MMKGs) organize information across different modalities to support
multimodal Graph
Retrieval-Augmented Generation (RAG), which aims to improve the performance of
multimodal large
language models (MLLMs). The effectiveness of these
multimodal Graph RAG methods largely depends on the quality of the constructed MMKGs. However, most existing methods construct MMKGs without explicit guidance on graph quality, while reinforcement learning (RL)-based methods that rely on final-answer correctness may provide only indirect feedback on MMKG quality. To address these limitations, we propose MMKG-R1, a novel MMKG-quality-driven RL framework for training an MLLM as a
multimodal knowledge constructor. To be specific, MMKG-R1 introduces multiple quality-aware rewards to directly evaluate both the quality of information extracted from individual knowledge sources and the quality of the assembled MMKG. These rewards are then used to guide the RL training. Experiments across multiple
datasets show that MMKG-R1 constructs MMKGs that improve downstream question-answering accuracy.
Read the full abstract ▾
All authors Zongyu Wu, Yilong Wang, Xiaochen Wang, Minhua Lin, Yuan Zhong, Fenglong Ma, Xiang Zhang, Suhang Wang
Computer Vision
NavAble: A Large-Scale Dataset and Synthetic Data Generation Pipeline for Blind Navigation
Hochul Hwang, Jahir Sadik Monon, Soowan Yang et al. (+9 more)
Reliable recognition of accessibility-critical objects (e.g., audible pedestrian signals, door-activation buttons, and handrails) is essential for assistive navigation technologies that support safe, independent mobility for blind and low-
vision (BLV) users. Yet these categories are underrepresented in existing large-scale
vision datasets, and the few
datasets that include them suffer from limited class coverage, inconsistent annotations, and poor diversity. We introduce NavAble, a large-scale
dataset spanning 11 accessibility-critical object classes, combining 41K curated open-source images with 8K newly collected, densely annotated real-world images. To extend scale and diversity, we develop a synthetic data generation pipeline that renders high-fidelity 3D assets across 37 environments under diverse styles and viewpoints, yielding 452K frames with rich ground-truth annotations from 565 assets. The pipeline minimizes manual effort by combining automated web image crawling, VLM-based filtering, SAM 3D-based asset generation, and convenient camera trajectory configuration, enabling large-scale data generation at a diversity unattainable through manual collection. Experiments show that fine-tuning existing segmentation models on the NavAble
Dataset achieves reliable accessibility-object segmentation (>80% mIoU on our held-out test set). We also demonstrate the effectiveness of synthetic data augmentation, which yields up to +3.5 mIoU over real-world-only fine-tuning. Beyond
benchmark evaluation, validation on egocentric data from two mobility-assistive
robots demonstrates improved perception reliability in real-world navigation scenarios. Together, NavAble establishes a scalable foundation for BLV navigation perception, directly addressing the data scarcity and embodiment gaps. The
dataset and pipeline are publicly available at https://huggingface.co/
datasets/NavAble/NeurIPS_2026_BLV.
Read the full abstract ▾
All authors Hochul Hwang, Jahir Sadik Monon, Soowan Yang, Shiven Umeshbhai Patel, Anh Nhat Hong Nguyen, Kien Nguyen, Keshav Garg, Dylan Gage, Duretti Hordofaa, Anshu Anjna, Eshed Ohn-Bar, Donghyun Kim
Computer Vision
SonoWorld: Towards Interactive Pose-Controllable Ultrasound World Models
Zefan Yang, Zheyuan Zhang, Boris Maihe et al. (+7 more)
Robotic ultrasound can standardize image acquisition and reduce operator workload and variability, but policy learning remains limited by the lack of realistic, controllable simulation environments. While controllable world models have advanced rapidly in natural-image domains, ultrasound world modeling remains challenging because each frame captures only a cross-section of unobserved 3D anatomy, causing small probe motions to induce large appearance changes. Existing approaches rely on
robot-tracked poses and latent-space prediction, but do not directly generate the observations required for interactive control. We propose SonoWorld, a causal ultrasound world model that learns
video dynamics from unlabeled clinical
videos and grounds them to 6-DoF control using a small set of freehand clinical scans tracked with an EM sensor. SonoWorld is trained in three stages: pretraining a latent-action-conditioned
video diffusion model on around 6.9 million unlabeled multi-organ ultrasound frames; grounding the learned
video dynamics to metric probe poses with a small EM-tracked thyroid
dataset; and applying distribution matching
distillation to enable causal, long-horizon autoregressive generation with selectively caching eviction
tokens at inference time. We evaluate controllability through pose-following fidelity, round-trip consistency, and axis-specific control under synthetic actions on held-out tracked scans, and further validate it with a user study. In model-based action discrimination, SonoWorld correctly ranks the target-directed probe motion over an equal-magnitude opposite motion in up to 74.8% of trials. To our knowledge, SonoWorld is the first causal, pose-controllable ultrasound world model learned primarily from freehand clinical
videos, providing a data-driven interactive environment for robotic ultrasound data acquisition and policy evaluation.
Read the full abstract ▾
All authors Zefan Yang, Zheyuan Zhang, Boris Maihe, Yikang Liu, Hang Yu, Ankush Mukherjee, Xiao Chen, Pingkun Yan, Terrence Chen, Shanhui Sun