Computer Vision
Advancing All-Weather Building Damage Mapping to the Instance Level: Outcomes and Insights from the 2026 Bright Challenge
Hongruixuan Chen, He Huang, Haifeng Wang et al. (+19 more)
Rapid post-disaster response requires timely, building-level information on whether structures remain intact, are damaged, or are destroyed. Post-event optical imagery, however, may be unavailable because of cloud, smoke, or darkness. The Bright Challenge evaluated all-weather building damage mapping from a submeter-resolution pre-event optical image and a post-event SAR image. Participants were required to detect and delineate each building and assign exactly one of three mutually exclusive damage labels. The challenge extended the globally distributed Bright
dataset with instance-level annotations for about 291,000 buildings across 16 disaster events spanning seven disaster types. The final phase was evaluated exclusively on two 2025 events absent from training: a wildfire event in California and a hurricane in Jamaica. A total of 157 participants made 1,289 submissions, and 46 teams entered the final phase. The two winning solutions achieved test mAPs of 0.182 and 0.181, approximately 8.7 times the public baseline of 0.021, but remained far below the best in-domain holdout score of 0.513. Across teams ranked in both phases, performance declined sharply and the rank order changed substantially. The two leading solutions independently favored modality-specific encoding, staged or late optical–SAR fusion, and an optical-dominant separation of building localization from damage recognition. The winning method additionally used scene-aware threshold adjustment and pseudo-label adaptation. These results identify cross-event generalization and stable severity discrimination as the principal remaining challenges. All data, annotations, baseline code, and winning solutions are publicly available at https://github.com/ChenHongruixuan/BRIGHT.
Read the full abstract ▾
All authors Hongruixuan Chen, He Huang, Haifeng Wang, Jian Song, Junjue Wang, Weihao Xuan, Hamish Mitchell, Jiepan Li, Wei He, Liangpei Zhang, Zijie Wang, Chen Zhong, Jiazhen Zhao, Lei Hu, Ting Hu, Hongyan Zhang, Gregory Angelides, Miriam Cha, Clifford Broni-Bediako, Junshi Xia, Taylor Perron, Naoto Yokoya
Computer Vision
AI-generated Images Challenge Visual Trust in High-risk Scenarios
Yi-Zhi Wang, Yichen Xiao, Linan Yue et al. (+5 more)
Rapid advances in image generation are eroding the evidentiary value of visual content in settings where authenticity can affect public safety and personal reputation. Yet existing detection
benchmarks rarely examine synthetic images in public- and individual-safety contexts, where misleading visual content may carry substantial risks. Here we introduce SafeIMG, a safety-oriented
benchmark spanning 12 public- and individual-safety scenarios generated using GPT Image 2. Unlike
benchmarks centred on generic imagery and image-level labels, SafeIMG evaluates not only whether detectors recognise synthetic images, but also whether their decisions reflect human-identified anomalies. To this end, SafeIMG provides human annotations that localise suspicious regions and explain local artefacts and higher-level commonsense or physical inconsistencies. We evaluate specialized synthetic-image detectors and
vision-
language models (VLMs), and find that neither provides reliable detection. The strongest VLM identifies only 49.5% of generated images, whereas the best specialised detector identifies 33.1%, compared with 81.7% accuracy for human evaluators. Model explanations cover only 29.8% of human-annotated anomalies and predominantly capture local defects in text, faces and hands. Their coverage falls to 15.0% for commonsense conflicts and 12.0% for physical inconsistencies, while detection performance deteriorates further after dissemination-induced image degradation. These findings show that current detectors lack the accuracy, explanatory
alignment and robustness needed to evaluate AI-generated images reliably across public- and individual-safety settings.
Read the full abstract ▾
All authors Yi-Zhi Wang, Yichen Xiao, Linan Yue, Weibo Gao, Yichao Du, Pengfei Fang, Shimin Di, Min-Ling Zhang
Computer Vision · Cross-listed from cs.LG
QFedPolyp: A Communication- and Inference-Efficient Federated Learning Framework for Polyp Segmentation
Madan Baduwal and Priyanka Paudel
Background and Objective: Automatic polyp
segmentation supports computer-aided diagnosis and early colorectal cancer detec- tion. Centralized deep learning requires hospitals to share sensitive medical data, while
federated learning preserves
privacy but introduces high communication costs through repeated transmission of full-precision model parameters. We propose QFedPolyp, a communication- and inference-efficient
federated learning framework for collaborative polyp
segmentation. Methods: QFedPolyp combines quantization-aware training with low-precision model communication. Each hospital locally trains a lightweight U-Net on private data while simulating quantization during training. Clients transmit quantized model parameters to a central server, where they are reconstructed and aggregated using
Federated Averaging. Evaluation is performed on Kvasir-SEG, CVC-ClinicVideoDB, PolypGen, and BKAI-IGH NeoPolyp. Results: Full-precision
federated training achieves Dice scores of 0.910 on Kvasir-SEG and 0.930 on CVC-ClinicVideoDB. Uni- form 8-bit communication reduces transmission cost by approximately 4 times while preserving competitive
segmentation accuracy. Quantized models also achieve up to 1.5 times faster inference than full-precision models. Conclusions: QFedPolyp enables
privacy-preserving collaborative polyp
segmentation with reduced communication overhead and faster inference. The resulting lightweight models are suitable for real-time clinical deployment.
Read the full abstract ▾
Computer Vision
SCALE: Self-Supervised Constraint-Aware Layout GEneration for Local P&R DRV Fixing at Advanced Nodes
Chia-Tung Ho, Haoyu Yang, Guanglei Zhou et al. (+6 more)
As
semiconductor manufacturing advances toward sub-2nm nodes, local place-and-route (P&R) design-rule violation (DRV) fixing is increasingly limited by complex rule interactions, dense multi-layer routing geometries, and foundry-specific constraints. While Large
Language Models (LLMs) have recently demonstrated strong capabilities in EDA scripting and documentation, their application to visual layout understanding remains largely unexplored: diagnosing DRC violations from layout imagery demands precise
geometric reasoning and foundry-specific rule knowledge absent from general-purpose VLM training. We propose SCALE, a framework with a self-supervised layout-generation stage for local DRV fixing at advanced nodes. Multi-layer layout geometry is serialized into structured text, and a fine-tuned
language model learns to reconstruct randomly masked polygons from surrounding BEOL context alone without violation labels. At inference, natural-
language rule constraints and high-temperature sampling steer generation toward diverse, violation-prone layout variants validated by an industrial signoff DRC checker, producing DRC-annotated layout–violation pairs used to fine-tune a domain-adapted DRC-VLM. This VLM provides rule-aware
geometric guidance for local DRV repair, boosting state-of-the-art
agents' solve rates by +12–25% (up to 97%) on 100 real sub-2nm cases spanning enclosure, spacing, width, and color-spacing violations.
Read the full abstract ▾
All authors Chia-Tung Ho, Haoyu Yang, Guanglei Zhou, Yoshi Nishi, Yaguang Li, Walker Turner, Cunxi Yu, Yiran Chen, Brucek Khailany
Computer Vision
Closing the Loop: Training-Free Revisit Consistency for Autoregressive Generative Rendering
Wenchao Ma, Changran Liu, Sharon X. Huang et al. (+1 more)
Recent conditional
video generation models have shown promising potentials to transform 3D engine renderings, such as depth maps and untextured geometry, into photorealistic
videos for gaming and immersive content creation. These applications require long-horizon auto-regressive generation that continuously synthesizes new frames while preserving a persistent 3D world. Auto-regressive generators synthesize
video chunk by chunk with a bounded KV cache, so when the camera revisits a location after its context has been evicted, the model often regenerates inconsistent appearance, even though the conditioning renderings (e.g., depth) remain perfectly aligned with the underlying geometry. We address this revisit inconsistency without any post-training by exploiting correspondences the 3D engine already provides: temporal correspondence retrieves pose-matched historical latent chunks into the KV cache as loop-closure
memory, while spatial correspondence from camera pose and depth reprojection biases
token-level
attention toward geometrically corresponding regions of the retrieved chunks. We demonstrate our method on loop-closure trajectories mined from TartanAir and TartanGround
dataset to mirror complicate real-world application scenarios, where it outperforms existing training-free baselines on revisit consistency without losing overall
video quality. Project Page: https://wenchao-m.github.io/ClosetheLoop.github.io/
Read the full abstract ▾
All authors Wenchao Ma, Changran Liu, Sharon X. Huang, Haomiao Jiang
Computer Vision
Toward High-Fidelity 3D Point-Cloud Learning for Brain Folding Morphology Prediction Using Trans-Unet
Geran Zhao, Xiaotian Li, Poorya Chavoshnejad et al. (+4 more)
Learning high-fidelity point-cloud features in the 3D space poses significant challenges, including permutation invariance, lack of local context, difficulty in fine-grained surface reconstruction, and high computational cost. In this article, we propose Trans-Unet, a novel framework that addresses these issues by first tansforming 3D point-cloud data into a 2D grid domain and then employing a U-shaped hybrid model that integrates Convolutional Neural Networks, and self-
attention mechanisms. The proposed Trans-Unet effectively learns and reconstructs precise features from high-resolution 3D point-cloud data (with 40,401 points in surface and 2,382 points in fiber) derived from a predefined finite element
brain patch growth model, enabling accurate prediction of
brain folding patterns. By combining multiple techniques, Trans-Unet leverages the complementary strengths: the 3D-to-2D transformation preserves fine-grained structural information while significantly reducing computational cost and the curse of dimensionality; convolutional blocks capture hierarchical, low-level local representations; and the self-
attention mechanism models global, high-level semantics and long-range dependencies. The
dataset consists of 3D point-clouds containing both
brain surface patches and fiber information generated by a large-scale finite element model. Trans-Unet is applied to predict
brain surface folding from the initial state (state 0 or states 0-2) to the final state (state 3). Experimental results demonstrate that Trans-Unet achieves high-resolution predictions of
brain patch growth, surpassing existing methods in both fidelity and accuracy.
Read the full abstract ▾
All authors Geran Zhao, Xiaotian Li, Poorya Chavoshnejad, Mir Jalil Razavi, Akbar Solhtalab, Lijun Yin, Guifang Fu
Computer Vision · Cross-listed from physics.med-ph
A Dual Path Framework with Hotspot Guided Fusion for Three Dimensional CT to PET Synthesis in Head and Neck Cancer
Mohd Maaz Khan and Oluwaseyi Oderinde
18F-FDG PET/CT plays a central role in staging, treatment planning, and response assessment for head and neck cancer by providing functional information that complements anatomical CT imaging. However, PET acquisition requires radiotracer administration, specialized infrastructure, and additional cost, limiting its availability for repeated imaging. We present a proof of concept deep learning framework for synthesizing PET like images directly from routine CT scans with the goal of providing complementary metabolic information that may support imaging triage and clinical decision support rather than replace diagnostic PET. Forty-four patients from the publicly available QIN-HEADNECK
dataset were retrospectively analyzed using five fold cross-validation. We propose a fully three dimensional dual path architecture consisting of (i) a regression U-Net optimized for voxel-wise quantitative SUV estimation and (ii) a conditional generative
adversarial network optimized for realistic PET texture. Their outputs are integrated using hotspot guided
Laplacian pyramid blending, allowing quantitative information from the regression pathway to be preserved within metabolically active regions while leveraging
adversarial texture synthesis elsewhere. The proposed framework achieved a mean absolute error of 0.00395, PSNR of 39.19 dB, and SSIM of 0.9634 on reconstructed three dimensional PET volumes. Qualitative evaluation demonstrated accurate localization of many FDG-avid lesions while producing anatomically realistic background texture. Consistent with previous CT to PET synthesis studies, the principal limitation was systematic underestimation of SUV within highly metabolically active tumor regions.
Read the full abstract ▾
Computer Vision
What Happens to Accuracy When Photo Lineups Contain Non-Mated Rank-One Images From Large Galleries?
Genesis Argueta, Kevin W. Bowyer, Michael King et al. (+1 more)
One-to-many facial identification is commonly used to match a probe image from surveillance
video against a gallery of driver's license and/or booking photos. The
algorithm's rank-one image from the gallery, or a human examiner's selection from the
algorithm's top-ranked images, may then be placed in a photo lineup shown to a witness. Witness selection of the gallery image in the photo lineup may then lead directly to the person in the gallery image being arrested. This facial identification process is involved in a number of wrongful arrests. This work specifically examines whether the probability of a witness making an incorrect identification increases with the size of the gallery searched. We compare photo lineup accuracy when the "suspect" image is drawn from galleries of 500, 5,000, and 24,000 images. We find that larger galleries increase both the likelihood of a witness making an incorrect identification and their confidence in that (incorrect) identification. These results raise questions of whether an image resulting from such a facial identification process should be used in photo lineups and of whether results of a photo lineup alone should constitute probable cause for arrest.
Read the full abstract ▾
All authors Genesis Argueta, Kevin W. Bowyer, Michael King, Jayeeta Dhar
Computer Vision
Risk-Routed Implicit Boundary Refinement for Robust Ultrasound Image Segmentation
Jingguo Qu, Xinyang Han, Xiang Wang et al. (+8 more)
Accurate boundary delineation in medical ultrasound (US) images remains challenging because of speckle noise, low contrast, acoustic shadowing, and acquisition variability. We propose Risk-routed Implicit Boundary Refinement (RIBR), a compact framework that refines convolutional
segmentation through spatially weighted implicit residuals. A coordinate-conditioned implicit branch uses regional features to predict logit corrections, and a learned gate controls their contribution using predictive uncertainty and local appearance. Geometry- and speckle-aware regularization provides contour supervision and discourages probability fluctuations away from boundaries. We compare RIBR with ten baselines on nine
datasets covering lymph nodes, breast lesions, thyroid nodules, and prostate, including four external lymph-node and breast cohorts. Across three seeds, RIBR achieves the best overall
dataset macro-average among the evaluated configurations, with $76.67{\pm}0.42\%$ Dice and a 95th-percentile Hausdorff distance (HD95) of $29.55{\pm}0.55$ pixels on the resized evaluation grid. It obtains the lowest HD95 on all nine
datasets, with task-dependent Dice rankings. These results support controlled implicit residual refinement for boundary-sensitive US
segmentation. Source code is available at https://github.com/jinggqu/ribr.
Read the full abstract ▾
All authors Jingguo Qu, Xinyang Han, Xiang Wang, Yuqi Yang, Tonghuan Xiao, Sheng Ning, Jing Qin, Ann Dorothy King, Winnie Chiu-Wing Chu, Jing Cai, Michael Ying
Computer Vision · Cross-listed from cs.CL
Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms
Abu Tyeb Azad, Fahim Ahmed, Ishita Sur Apan et al. (+7 more)
Document packets, multiple documents concatenated into a single file, are common in government and administrative workflows, yet splitting them into their constituent documents is difficult, especially for low-resource
languages. We introduce Khondo (Bangla for split/segment), the first
benchmark for document packet splitting on Bangladeshi government forms. Unlike prior English and OCR-text-based
datasets, Khondo is bilingual (Bangla–English) and
vision-native; where models operate directly on page images. It spans five concatenation schemes, from sequential to fully shuffled, across 14 administrative domains, with ground-truth boundaries, domain types, and page order. Zero-shot evaluation of MLLMs shows they cluster pages into their source documents fairly well but struggle in restoring the original page order once shuffled. To isolate what drives this difficulty, we run two controlled analyses, varying the
prompt instruction and then the packet
language. Both primarily affect ordering rather than clustering: (a) explicit page-order instructions are necessary but insufficient, and (b) English packets are ordered more reliably than Bangla, making page arrangement the dominant challenge and
language a secondary but consistent factor. Khondo establishes page-order reconstruction as a key open problem in
vision-based, low-resource document understanding, and provides a controlled
benchmark for measuring progress toward solving it. Our
dataset and code is available at https://huggingface.co/
datasets/Mausul/khondo
Read the full abstract ▾
All authors Abu Tyeb Azad, Fahim Ahmed, Ishita Sur Apan, Ezharuddin Jubaer, Sumaiya Karim Katha, Armun Alam, Amin Ahsan Ali, Aman Chadha, Md Mofijul Islam, AKM Mahbubur Rahman
Computer Vision · Cross-listed from cs.LG
Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection
Othmane Harraq and Tamer Aldwairi
Talking-face (TF) deepfake generation synthesizes photore- alistic facial
video from a static source image and an au- dio signal, producing forgeries that current image-based detectors consistently fail to identify. Unlike face-swap ma- nipulation, TF synthesis has no underlying real
video from which to inherit physiological characteristics, making re- mote photoplethysmography (rPPG) a uniquely motivated detection modality for this forgery category. We propose a detection framework that extracts per-
video rPPG wave- forms via RhythmFormer and trains a suite of lightweight classifiers to distinguish real from synthesized physiologi- cal signals. Evaluated on the TF subset of Celeb-DF++ un- der a strict subject-independent protocol, where test identi- ties are completely separated from training identities, our 1D ResNet achieves an AUC of 0.806 and EER of 27.8%, placing it within 2.4 points of the best published general- purpose detector (Effort, ICML 2025) while operating ex- clusively on the physiological channel. We document a con- trolled reproduction study of DeepFakesON-Phys, the rep- resentative prior rPPG detector, demonstrating degrada- tion from AUC 0.999 on legacy face-swap data to 0.622 on the TF subset of Celeb-DF++. We further show that detec- tion difficulty is strongly method-dependent: AUC ranges from 0.985 (Real3DPortrait) to 0.690 (IP-LAP) across the seven TF generators, with the ranking remaining perfectly stable across all evaluation protocols. This spread reflects an interpretable physiological property of each generator rather than evaluation noise, and constitutes the primary theoretical contribution of the work.
Read the full abstract ▾
Computer Vision
Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints
Liqiang Jing, Xiong Zhou, Siddharth Varia et al. (+3 more)
While Large
Vision-
Language Models (LVLMs) exhibit strong perceptual capabilities, they remain vulnerable in visual
reasoning tasks. Existing
benchmarks largely focus on symbolic mathematical or scientific problems and simple
vision-centric tasks, offering limited assessment of complex visual
reasoning and logical consistency, a critical requirement for reliable
reasoning systems. We introduce ConVBench, a complex
vision-centric
reasoning benchmark in which each image is paired with two logically equivalent questions across six categories: action and state, complex counting, spatial
reasoning, causal and intent understanding, commonsense
reasoning, and temporal perception. To complement this
benchmark, we define two evaluation metrics, logical consistency and robust accuracy, that jointly assess both the correctness and consistency of model responses. We further present ConVLM, which improves LVLM
reasoning through Group Relative Policy Optimization (GRPO)-based reinforcement learning with a novel consistency reward. This method leverages automatically generated logically equivalent question-answer pairs and a dual-reward design combining accuracy- and consistency-based signals, encouraging agreement between paired responses. The framework functions effectively with or without strict answer supervision.
Read the full abstract ▾
All authors Liqiang Jing, Xiong Zhou, Siddharth Varia, Neha Anna John, Xinya Du, Vassilis N. Ioannidis
Computer Vision
Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-On
Yong Liu, Xiaolong Fu, Zihang Xu et al. (+10 more)
We present Oxygen-TryOn, a unified foundation model for any-item virtual try-on. Rather than repurposing a general-purpose image editor, Oxygen-TryOn is fashion-native, built for try-on through a dedicated data engine and try-on-specific training. Given one or more reference items (clean product shots or in-the-wild worn-on photos) and a single target subject image, it synthesizes a photorealistic image of the subject wearing the items across virtually any fashion category. Prior systems handle a single garment category in a studio setting, and recent multi-reference methods remain garment-centric; in contrast, Oxygen-TryOn supports diverse items and scenarios, including full- and half-body views, a variable number of references, and free multi-item composition, while faithfully preserving both subject identity and item appearance. Instead of mask-based inpainting, we reformulate try-on as a multi-reference, understanding-driven generation task. We build a data engine that collects, manufactures, annotates, and filters high-quality try-on data at scale, and design a three-stage recipe of continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL). The RL stage uses a hybrid reward combining an in-house try-on reward model with a proprietary, rubric-guided general-purpose model, jointly supervising fine-grained consistency and instruction-level quality. It also follows general editing instructions (e.g., pose changes) in the same pass. Across public
benchmarks and our in-house Oxygen-TryOn Bench, it achieves state-of-the-art consistency and realism on single-item try-on and leads on multi-item try-on, matching or surpassing both leading proprietary systems (Nano Banana Pro, GPT-Image-2, Seedream5 Lite) and open-source models (FLUX.2).
Read the full abstract ▾
All authors Yong Liu, Xiaolong Fu, Zihang Xu, Wen Xue, Xueheng Li, Lin Song, Yuan Zhang, Chuyang Zhao, Haoyang Huang, Nan Duan, Yipeng Sun, Yan Li, Simiu Gu
Computer Vision · Cross-listed from quant-ph
Hash-QNeRF: Multiresolution Hash Encoding for Quantum Neural Radiance Fields
Digonto Biswas, Tana Ballove, Anjan Bandyopadhyay et al. (+2 more)
Neural Radiance Fields (NeRF) have revolutionized novel view synthesis, yet their classical implementations remain computationally intensive for high-fidelity rendering. QNeRF recently demonstrated the feasibility of training NeRF on gate-based
quantum computers by combining amplitude
embedding, parameterized
quantum circuits (PQCs), parity-based measurements, and volumetric rendering. However, QNeRF relies on classical sinusoidal positional encoding for spatial coordinates, which scales poorly with scene complexity and resolution. In this work, we replace the sinusoidal positional encoding for spatial coordinates with the multiresolution hash encoding from Instant-NGP while keeping the view-direction encoding, amplitude MLP,
quantum circuit, parity measurement, output scaling, and volumetric rendering pipeline unchanged. This hybrid design, Hash-QNeRF, retains the
quantum radiance prediction step while benefiting from the fast convergence and
memory efficiency of learnable hash grids. On a synthetic Blender scene, we achieve a final training loss of 0.003534, corresponding to approximately 24.5 dB PSNR on the fitted batch. Noise resilience experiments using Qiskit FakeKyiv and FakeTorino backends yield state fidelities of 0.93 to 0.98, indicating that hash encoding does not degrade the
quantum circuit's noise tolerance.
Read the full abstract ▾
All authors Digonto Biswas, Tana Ballove, Anjan Bandyopadhyay, Sutanu Mangal, Arun Kumar Pati
Computer Vision · Cross-listed from cs.LG
Self-Poisoning in Adaptive Out-of-Distribution Detection: A Sharp-Threshold Theory and Certified Label-Free Calibration
Vishnu Bindu Balachandran
Test-time adaptive out-of-distribution (OOD) detectors update a
memory bank from the unlabelled stream. We show this adaptation obeys a provable dynamical law. Modelling bank
impurity as a generalized Pólya urn, we prove almost-sure convergence to a mean-field
equilibrium whose slope acts as a reproduction number. Below one,
impurity stays benign. Above one, the bank is fully poisoned and the detector collapses. The measured admission kernel is affine ($R^2 \ge 0.996$) with slope just below one in every encoder family (a protocol signature), so this detector class is near-critical by design, and across 96 settings the predicted threshold matches the empirical collapse, where ungated dictionaries lose up to $0.163$ AUROC. We then prove that a certified admission gate, reading only a frozen reserve, severs the feedback loop and removes the transition at every contamination rate, even adversarially, while controlling false positives label-free. For the complementary static-calibration failure under drift we give CDC, which restores nominal FPR label-free on all tested drift-affected
cells. Finally we prove a two-world impossibility
theorem. Drift and contamination are indistinguishable without labels, forcing a closed-form power ceiling our procedure approaches. Together these give a complete possibility/impossibility characterization of label-free adaptive OOD detection.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.SE
Pixels for Programs? A Cross-Provider Case Study of Input-Token Accounting for Source Code as Text and Images
Ronak Bhalgami
Long source-code contexts consume many text
tokens, motivating the proposal to render code as images for
vision-
language models. Recent work asks whether models can still solve code tasks after this transformation. We examine a different systems question: how commercial APIs count the resulting requests. We present a reproducible measurement case study of provider-reported input
tokens for raw source text and a compact rendered-image representation. The
benchmark pairs requests across five programming
languages, nine source lengths from 20 to 2,000 lines, and 15 available model aliases exposed by Anthropic, OpenAI, and Google Vertex AI. These aliases collapse to approximately five distinct accounting signatures and are not independent model replications. Across 675 complete text/image pairs, aggregate image-to-text ratios are 0.135, 0.194, and 0.242, corresponding to reported input-
token reductions of 86.5%, 80.6%, and 75.8%, respectively. These totals conceal materially different break-even behavior: Anthropic and OpenAI images receive lower counts at every tested size, while Gemini images require 6.95 times as many
tokens at 20 lines and cross below text only at 200 lines in the aggregate. A targeted audit also reproduces non-monotonic Gemini image accounting across a page boundary. This study measures black-box request accounting for one compact rendering pipeline. It does not measure semantic fidelity, task accuracy, latency, monetary cost, or coding-
agent efficiency. We release the scripts, revision-pinned corpus specification, raw usage records, validators, and deterministic analysis needed to reproduce and extend the study.
Read the full abstract ▾
Computer Vision
3D-Aware VLMs with Implicit and Explicit Geometries
Wenhao Li, Xueying Jiang, Quanhao Qian et al. (+4 more)
Despite rapid progress, most existing
vision-
language models (VLMs) built from 2D visual inputs often struggle when handling various 3D tasks that require fine-grained spatial understanding and
reasoning. To bridge this gap, we present VLM-IE3D, a unified framework that enhances the 3D spatial awareness of VLMs by equipping them with both implicit and explicit 3D geometries learned from RGB
videos. Our VLM-IE3D introduces Implicit Geometry
Tokens (IGTs) that capture high-level
geometric priors from input
videos, as well as complementary Explicit Geometry
Tokens (EGTs) that encode detailed
geometric structures from reconstructed 3D attributes. On top of that, VLM-IE3D comes with a 3D-aware adapter that effectively fuses the two types of
geometric representations with 2D visual cues. This RGB-only design injects strong 3D inductive biases for fine-grained spatial understanding and
reasoning without requiring any additional 3D inputs. Extensive experiments show that VLM-IE3D achieves superior performance consistently across various 3D tasks including 3D
video detection, 3D visual grounding, 3D dense captioning, and spatial
reasoning. Code and models are available at https://github.com/Vegetebird/VLM-IE3D.
Read the full abstract ▾
All authors Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Ran Xu, Shijian Lu, Gongjie Zhang
Computer Vision
Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers
Sicheng Mo, Yuheng Li, Ziyang Leng et al. (+2 more)
Multi-
agent interactive world models should not only generate consistent observations, but also maintain world states that persist across
agents and evolve across views. Existing autoregressive
video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi-
agent and multi-view settings. We present WorldWeaver (W^2), a streaming multi-
agent video diffusion model that augments rollout with cross-
agent world state registers: learnable
tokens that store shared world information, track individual
agent status, and are dynamically updated after each generated chunk. We ground these registers with supervision signals spanning individual
agent status, global state views including bird's-eye views, and scene text. We further improve the architecture with a Mixture-of-
Transformers design that uses separate weights for world state modeling and visual frame modeling. Extensive experiments in two-
agent Minecraft
video generation show that explicit world-state modeling improves logical consistency and generation quality.
Read the full abstract ▾
All authors Sicheng Mo, Yuheng Li, Ziyang Leng, Krishna Kumar Singh, Bolei Zhou
Computer Vision
Unified Video Dense Prediction from Disjoint Data
Yihong Sun, Seoung Wug Oh, Jiahui Huang et al. (+2 more)
Scene understanding requires simultaneous prediction about geometry, appearance, and semantics. However, existing task-specific annotations are fragmented across incompatible, domain-specific
datasets. Current unified systems circumvent this by restricting training to fully co-annotated data, or by incurring the large computational cost of pseudo-labeling. To mitigate this, we introduce UniD, a unified
video model that jointly predicts eight dense scene properties-depth, surface normals, semantic
segmentation, boundaries, human parts, albedo, shading, and materials-all learned from disjoint, domain-specific
datasets. We propose a simple yet effective
distillation step in which per-task experts supervise a unified backbone through lightweight task projectors, eliminating the need for annotation overlap or pseudo-labeling. Our key insight is that the strong visual priors of a pretrained
diffusion model are sufficient to bridge the domain gaps introduced by disjoint training sources, enabling robust generalization to scene-task combinations never seen during training. UniD achieves competitive performance against per-task specialists and multi-task baselines, with strong generalization to out-of-distribution scenarios and enhanced temporal and cross-task consistency. Code and
video results are available at https://unid-
video.github.io/.
Read the full abstract ▾
All authors Yihong Sun, Seoung Wug Oh, Jiahui Huang, Bharath Hariharan, Joon-Young Lee
Computer Vision
Inference-Time Scaling of Diffusion Models via Progressive Seed Pruning
Rogerio Guimaraes and Pietro Perona
Diffusion and flow-matching models dominate conditional image generation, yet inference-time scaling for these models is far less developed than for autoregressive
language models. Because final quality is highly sensitive to the initial noise seed, many approaches spend extra compute on seed search or resampling under a black-box reward, but typically maintaining a constant
memory footprint throughout inference. We show that relaxing this constraint enables an underexplored inference-time scaling axis: by front-loading exploration, evaluating many seeds early, and
pruning aggressively, we can use a fixed compute budget more effectively. Progressive Seed
Pruning (\PSP) scores intermediate denoised estimates and progressively narrows the candidate set so that only promising trajectories are fully denoised, while keeping the total number of model evaluations fixed. Across
diffusion and flow-matching backbones, \PSP \ consistently improves reward-guided selection and achieves higher GenEval scores (automated) and better human evaluation on
prompt-
alignment than best-of-$N$, importance-sampling, and tree-search baselines at matched compute. Project page: https://www.
vision.caltech.edu/psp. Code: https://github.com/rogerioagjr/psp.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation
Yu Qi, Zhang Ye, Xinyi Xu et al. (+6 more)
Compositional generalization is essential for
robot to follow diverse instructions. However, pretrained policies are known to take shortcuts, deferring to salient cues rather than grounding
language. We introduce a diagnostic framework that localizes this failure to individual instruction factors, e.g., reusable semantic components such as color, verb, object, size, and spatial attribute. Our framework formalizes instruction factor bias, the tendency of fine-tuned policies to over-rely on dominant factors as shortcuts, and quantifies it through two metrics: Factor Dominance Rate (FDR), capturing pairwise bias between factors, and Factor Dominance Hierarchy (FDH), aggregating these into a global ranking. Evaluation on six foundation policies reveals broadly consistent ordering, i.e., color $\geq$ object $\geq$ spatial $\geq$ verb $\geq$ size, with color dominant, and verb and size most under-grounded. We further show the diagnosis is actionable: a bias-aware data collection strategy that reallocates a fixed budget toward under-grounded factors outperforms baselines in simulation and on a real
robot using half the demonstrations, thereby enabling more sample-efficient and generalizable policy learning.
Read the full abstract ▾
All authors Yu Qi, Zhang Ye, Xinyi Xu, Yuxuan Lu, Amitoj Sandhu, Boce Hu, Haojie Huang, Jonathan Tremblay, Lawson L.S. Wong
Computer Vision
GraphVid: Interactive Graph-Controllable Video Generation
Vedant Shah, Onkar Susladkar, Tushar Prakash et al. (+5 more)
Controllable
video generation remains challenging due to the difficulty of specifying precise multi-object interactions using text
prompts or motion-control inputs that primarily constrain pixel movement. In practice, trajectory-based control often requires users to draw accurate tracks for multiple objects, which scales poorly with scene complexity and becomes ambiguous under occlusion or overlap. To enable flexible yet precise multi-subject control, we introduce $\textbf{GraphVid}$, a graph-conditioned image-to-
video generation model that enables interactive control through structured interaction graphs. We further curate $\textbf{GraphVid-Bench}$, a large-scale interaction-centric
video dataset with structured relational annotations to enable training of interaction-aware
video generation models. Despite using substantially less training data and fewer trainable parameters than prior motion-control methods, GraphVid delivers strong controllability and
video quality. Compared with Motion-I2V, GraphVid reduces FID by up to 39.9% and FVD by 37.6%, while improving PSNR (9.87=>15.98) and SSIM (0.38=>0.61). Our results highlight the potential of structured semantic interfaces as a powerful paradigm for controllable
video generation.
Read the full abstract ▾
All authors Vedant Shah, Onkar Susladkar, Tushar Prakash, Kiet Nguyen, Tianjiao Yu, Adheesh Juvekar, Muntasir Wahed, Ismini Lourentzou
Computer Vision
Synthetic data generation framework for quality control automation in gravure printing
Korota Arsène Coulibaly, Mohamed Hamlich, Khalid Hmali et al. (+1 more)
Quality control in printing, particularly in rotogravure printing, still depends on slow, costly, and subjective manual inspection. Automated surface defect detection is critical for maintaining high-quality standards in rotogravure printing. Deep learning models give prospects for automation. However, training robust deep learning models, such as YOLO or
Vision Transformers, is heavily hindered by the extreme scarcity of real-world industrial defects images. To overcome this limitation, this paper introduces a novel synthetic data generation framework tailored for rotogravure printing quality control. The proposed pipeline automatically generates high-fidelity images of specific printing defects (creases, streaks, misregistration, etc.) and outputs corresponding bounding boxes and annotations. To validate the framework, a synthetic
dataset of 7533 images was generated and used to train the state-of-the-art object-detection model RFDETR. Experimental results demonstrate that the model trained on our synthetic data achieves a Mean Average Precision (mAP) of 80.9% on real industrial testing samples. This framework provides a zero-cost, rapid-deployment solution for automating defect inspection in printing lines without requiring massive manual data collection.
Read the full abstract ▾
All authors Korota Arsène Coulibaly, Mohamed Hamlich, Khalid Hmali, Andrea Trombin
Computer Vision
Self-Supervised Learning of Structured Dynamics from Videos
Lukas Knobel, Andrew Zisserman and Yuki M. Asano
Understanding motion in
video is a fundamental challenge for visual learning, as frame-to-frame change entangles two sources of dynamics: camera motion and object motion. This decomposition has remained underexplored in representation learning, partly because these factors are tightly coupled in natural
videos and difficult to supervise separately. Yet recovering it is important for learning robust motion representations that separate meaningful object dynamics from camera-induced variation. We study whether such structured motion representations can be recovered from frozen features of a pretrained image
vision transformer. We propose the Structured Dynamics Model (SDM), which explicitly separates the dominant source of temporal change from residual dynamics through future-feature prediction, rather than representing
video change with a single entangled latent or with unstructured, spatially dense transition
tokens. Training combines self-supervised learning on real
video with weak supervision of scene dynamics on synthetic Kubric data. We evaluate SDM on ProbeMotion, a new evaluation suite spanning synthetic and real
videos with camera motion, object motion, and combined dynamics. SDM outperforms backbone baselines using global CLS or average-pooled features, and compares favorably to strongly supervised representations such as VGGT on several probes, despite using substantially weaker supervision. These results suggest that pretrained image models can be readily repurposed into structured
video-dynamics representations, providing a useful inductive bias for learning and analyzing latent
video dynamics.
Read the full abstract ▾
Computer Vision
Scene Parameter Saliency via Differentiable Light Transport
Linas Beresna and Eugene Fiume
Gradient-based saliency methods reveal which input features most influence a neural network's output, and are a standard tool for model interpretability. We observe that differentiable renderers, which are conventionally used for parameter optimisation, produce an analogous form of saliency: given any scalar metric evaluated on a rendered image, a single reverse-mode differentiation pass yields per-parameter gradients that identify which scene elements most influence the metric. We call these gradient fields metric saliency maps. Unlike neural saliency, which propagates attribution through learned weights, metric saliency propagates through the image formation process itself, including multi-bounce light transport, capturing parameter dependencies that are semi-opaque to manual inspection. We compute metric saliency maps for qualitatively different objectives: psychovisual glare indices, mean scene luminance, and neural perceptual scores. The saliency rankings differ substantially across metrics for the same scene, with parameters that dominate one objective being negligible for another. The saliency map is specific to the metric, not an intrinsic property of the scene. Our results suggest that differentiable renderers produce derivative images that are as informative for scene understanding as the primal images they were designed to generate.
Read the full abstract ▾
Computer Vision
Yijun Liang, Yunjie Tian, Yijiang Li et al. (+4 more)
On-policy self-
distillation (OPSD) removes the external teacher required by on-policy
distillation (OPD), but still needs asymmetric information between teacher and student so that the self-teacher provides a stronger learning signal. Existing methods create this asymmetry through privileged answers or visual evidence. We ask whether both can be removed, yielding a simpler form of OPSD driven purely by input conditioning. We propose Visual Contrastive Self-
Distillation, namely VCSD, which converts image-content removal into an on-policy self-
distillation signal. At each student-generated response prefix, the EMA teacher produces two next-
token distributions under the same
prompt and prefix: one conditioned on the original image and the other on a content-erased control. Their
token-wise log-probability difference highlights candidates whose likelihood is specifically increased by the instance-level visual content. We use this contrast to sharpen the teacher's original-image distribution within its plausible support, and distill the resulting full-distribution target into the student. On ViRL39K, VCSD consistently outperforms matched OPSD across Qwen3-VL and Qwen3.5 models. It improves the seven-
benchmark aggregate from 72.51% to 76.26% on Qwen3-VL-8B and from 74.97% to 79.24% on Qwen3.5-9B, with gains at all six evaluated scales. VCSD requires no external teacher, privileged answers, visual evidence signals,
reasoning traces, or additional inference-time cost.
Read the full abstract ▾
All authors Yijun Liang, Yunjie Tian, Yijiang Li, Yuqi Jia, Kaishen Wang, Tianyi Zhou, Di Fu
Computer Vision
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
Junsong Chen, Jincheng Yu, Yitong Li et al. (+11 more)
We introduce SANA-
Video 2.0, a hybrid
video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality
video up to 720p on a single GPU, SANA-
Video 2.0 matches full-softmax
video DiTs in quality while retaining the favorable long-sequence scaling of linear
attention. To avoid quadratic
attention throughout, Hybrid Linear-Softmax
Attention combines gated linear
attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank
token interactions that pure linear
attention lacks. To propagate these refreshed representations across depth, Block
Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-
Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-
Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax
video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with
video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse
attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution
video generation.
Read the full abstract ▾
All authors Junsong Chen, Jincheng Yu, Yitong Li, Shuchen Xue, Haozhe Liu, Jingyu Xin, Yuyang Zhao, Tian Ye, Zhangjie Wu, Zian Wang, Daquan Zhou, Ping Luo, Song Han, Enze Xie
Computer Vision
UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging
Rafsan Jany, Shadab Tanjeed Ahmad, Ahsan Bulbul et al. (+3 more)
Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary information, yet acquiring paired clinical data remains a significant challenge in real world scenarios. While cross-modal knowledge
distillation addresses this, existing methods often struggle with large modality gaps and the propagation of noise from uncertain source-domain predictions. To overcome these challenges, we propose UnDA, an anchor-guided framework for unpaired cross-modal
distillation. Our approach introduces a backbone-agnostic
Alignment Module that extracts semantically structured class
tokens via an
attention based pooling mechanism. To ensure robust knowledge transfer, we propose Uncertainty-Weighted Optimal Transport (UCT-OT), which dynamically weights feature-level
alignment based on prediction confidence, effectively suppressing noisy supervision. Furthermore, a per-class ProtoNCE objective maintains stable prototype
memories to enforce global discriminability across unpaired batches. Evaluations on representative
segmentation tasks under strictly unpaired settings show consistent improvements in accuracy and boundary precision in the target modality, demonstrating that meaningful structural knowledge can be transferred across heterogeneous data sources without paired
datasets.
Read the full abstract ▾
All authors Rafsan Jany, Shadab Tanjeed Ahmad, Ahsan Bulbul, Tahsinul Islam, Md Azam Hossain, Abu Raihan Mostofa Kamal
Computer Vision
Towards Robust Iris Recognition Through Occlusion Identification and Conditional Diffusion-Based Reconstruction
Kamrul Hasan, Mylene C.Q. Farias and Oleg V. Komogortsev
Iris recognition is a reliable biometric approach that identifies individuals using the distinctive and stable texture of the iris. However, recognition performance can degrade when discriminative iris texture is partially occluded by eyelids, eyelashes, specular reflections, or other acquisition artifacts. Existing approaches often perform recognition directly on degraded samples or rely only on the remaining visible iris region, which may be inadequate when substantial texture is corrupted. To address this limitation, we propose an occlusion-aware iris recognition framework with three sequential modules: occlusion-type identification,
diffusion-based reconstruction, and deep-learning-based recognition. First, a residual 2D CNN-based network determines whether an iris image is non-occluded or belongs to one of the controlled occlusion categories. Second, the occluded image, binary mask, and predicted occlusion type condition a denoising
diffusion probabilistic model to reconstruct the corrupted region. Finally, VGG19-HPMNet, a modified VGG19 model with horizontal pyramid mapping, extracts discriminative global and part-wise local iris features for recognition. Experiments on the CASIA-Iris-Thousand
dataset under a controlled synthetic-occlusion protocol show that the proposed framework improves iris recognition performance by identifying the occlusion type, reconstructing masked regions, and re-evaluating the restored iris samples.
Read the full abstract ▾
Computer Vision
ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing
Yueyi Liu, Chi Zhang, Sen Cui et al. (+1 more)
Test-Time Tuning (TTT) on pretrained
diffusion models has emerged as a powerful paradigm for
video editing. However, there exists a foundational mismatch between the distribution-mapping nature of generative models and the single-point optimization of standard TTT. In this paper, we demonstrate that this mismatch triggers Prior Collapse, a degenerate state where the model discards the text conditions and spatial latents, collapsing generations to the source
video, or entangling the features of distinct regions. To resolve this, we propose ElasticTTT, a novel framework that preserves the prior generative distribution and rescues generative elasticity. Specifically, we propose Target Distribution Regularization to prevent sharp memorization minima, Contrastive CFG to guide inference away from source biases, and Asynchronous Noise Schedule to preserve unedited regions. Extensive evaluations, supported by theoretical analysis, demonstrate that ElasticTTT successfully preserves the generative prior of the base model, achieving state-of-the-art performance on one-shot
video editing.
Read the full abstract ▾
All authors Yueyi Liu, Chi Zhang, Sen Cui, Miao Liu