Computer Vision · Cross-listed from cs.AI
Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design
Hongyang Du, Lan Yan, Christian Flores et al. (+1 more)
Professional graphic design is a long-horizon agentic task in which structured, editable artifacts emerge from many interdependent actions, yet outcomes admit no reliable programmatic oracle. We introduce a continual adaptation framework in which a frozen frontier model operates professional design software through more than 230 tools, while an external procedural
memory of natural-
language skills accumulates and refines reusable design procedures from experience. The
memory widens by acquiring procedures for recurring uncovered subtasks and deepens by revising existing procedures against their own successful and failed executions, while a matched replay gate admits only changes that repair failures without regressing observed successes. Five rounds over 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, grow the bank from 76 documentation-derived skills to 139 and raise GenEval2 execution success on Claude-Sonnet-4 from 72.7% to 99.3% (+11.99 points in generation quality), with 61.8% and 67.6% win rates against the no-skill
agent across four specialized design
benchmarks on Claude-Sonnet-4 and Claude-Opus-4.6. We further show the two mechanisms are effective in combination: on 200 held-out briefs from user-traffic
benchmark, widening or deepening alone reaches a 49.4% / 48.6% win rate over the no-skill
agent, while their combination reaches 58.5% (p = 0.025). Procedural
memory offers a practical route to continual adaptation of
agents under noisy, unverifiable feedback.
Read the full abstract ▾
All authors Hongyang Du, Lan Yan, Christian Flores, Asim Kadav
Computer Vision
MintAct: A Unified Visual Agent for Digital Environments
Mingfei Gao, Rui Tian, Haiming Gang et al. (+11 more)
We present MintAct, a family of
vision-
language models that unifies UI grounding, multi-step
navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and 8B scales. Through careful design of our environments, data, and training recipes, MintAct models match the performance of per-domain specialists across all of these capabilities. To enable this, we develop a scalable environment and reinforcement learning (RL) infrastructure. On the environment side, we host hundreds of concurrent instances across heterogeneous per-domain backends, serving both trajectory data collection and online RL. To enable efficient and scalable RL training, an asynchronous framework keeps explicit control over the cross-domain training distribution and remains stable under noisy environment feedback and off-policy drift. Experimental results show that MintAct achieves state-of-the-art performance (48.9 on OSWorld-Verified) across a wide range of
benchmarks at comparable model sizes.
Read the full abstract ▾
All authors Mingfei Gao, Rui Tian, Haiming Gang, Bohan Zhai, Le Zhang, Yuanzheng Gong, Di Feng, Ege Özsoy, Kaixin Ma, Vishwesh Kirthivasan, Oğuzhan Fatih Kar, Roman Bachmann, Anders Boesen Lindbo Larsen, Afshin Dehghan
Computer Vision
OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation
Wenxue Li, Peiyan Guan, Haoyang Jiang et al. (+14 more)
Omni reference-to-
video (R2V) generation is emerging as a general paradigm that enables flexible
video customization from diverse and compositional visual references. However, existing
benchmarks fall short of these emerging capabilities: their test cases cover only a limited range of reference types and compositions, and their evaluation protocols largely assess holistic reference consistency, overlooking whether reference factors are properly preserved, disentangled, and routed. Meanwhile, the high cost of constructing omni R2V training data makes suitable training resources scarce. To address these gaps, we introduce OmniVBench and the Omni-R2V
Dataset, establishing a shared foundation for evaluating and training omni R2V models. OmniVBench expands R2V evaluation across broader reference types, richer reference compositions, and more flexible instruction-driven customization, covering 7 task families and 18 fine-grained tasks. Complementing this task design, we introduce factor-grounded evaluation with 12,172 case-specific checklist items, explicitly assessing whether intended reference factors are faithfully preserved, correctly disentangled and bound to their targets, and realized according to the instruction. We further introduce the Omni-R2V
Dataset, bringing industrial-grade training resources for diverse R2V tasks to the broader research community. Drawing primarily on a large-scale corpus of professional
video footage, it comprises 340K processed training samples spanning diverse reference types and multi-reference compositions. Specifically, we develop task-specific pipelines for reference-target pair construction, offering a practical and scalable recipe for omni R2V data construction. Extensive evaluation of advanced open- and closed-source R2V models reveals substantial performance variation across diverse tasks and distinct capability patterns across evaluation dimensions.
Read the full abstract ▾
All authors Wenxue Li, Peiyan Guan, Haoyang Jiang, Junxian Cai, Hualuo Liu, Chunjie Zhang, Chong Guan, Kai Huang, Songlian Li, Taiyi Wu, Yongjian Yu, Xiaotong Zhao, Alan Zhao, Quande Liu, Xi Chen, Yu Liu, Lei Zhu
Computer Vision
Traffic Sign Recognition for Autonomous Driving Using Branched YOLOv2 and Geometric Features
Arefeh Rezaei
Traffic sign recognition (TSR) is an important perception task for autonomous driving and advanced driver-assistance systems, where a system must both localize traffic signs and determine their semantic classes efficiently. This work presents a TSR system based on YOLOv2 for simultaneous detection and classification. Two complementary modifications are studied. First, YOLOv2 is extended with intermediate prediction layers, forming a branched architecture that can terminate inference early for easy cases and reduce computation time. Both whole-image and
cell-wise branching strategies are investigated. Second,
geometric information is introduced to reduce classification errors between visually similar signs. An unsupervised
Bayesian image-
segmentation method produces binary representations that are compared with class-specific
geometric templates inside YOLOv2 bounding boxes. This information is used either during inference or as an additional signal during training. A dedicated
dataset is constructed by combining GTSDB and GTSRB samples using seamless cloning and controlled image transformations. Experiments cover ten traffic-sign classes, with 3,000 training and 300 test samples. The selected branched architecture reports 0.647 s runtime and 0.680 mAP, compared with 0.6607 s and 0.680 mAP for baseline YOLOv2.
Geometric verification during inference increases mAP to 0.713, while the
geometric-feature training variant achieves 0.697 mAP with a reported runtime of 0.6608 s.
Read the full abstract ▾
Computer Vision
PRIME: Perception Feedback with Situational Memory Embeddings in VLA Models
Erik Deinzer, Naya Baslan, Luca Paparusso et al. (+3 more)
Current
Vision-
Language-Action (VLA) models for autonomous driving operate primarily through feedforward inference across the perception–
reasoning–planning hierarchy. While modern architectures maintain temporal recurrence within the perceptual module, early perception remains blind to downstream
reasoning and
navigation goals, processing visual inputs agnostically without prioritizing cues informed by prior decisions. To bridge this gap, this paper introduces
PRIME, a learned feedback mechanism that conditions the VLA perceptual queries on a novel Situational
Memory. By aggregating latent representations of past perception,
reasoning,
navigation goals, and predicted behaviors across an L-step window via cross-
attention,
PRIME enables intent-driven perceptual
attention at minimal computational cost, adding only a maximum of 29.7M parameters (0.41% of the 7.3B-parameter base model). Evaluated on the Bench2Drive closed-loop
benchmark,
PRIME achieves a state-of-the-art Driving Score of 82.47 (+4.73 over ORION) and a Success Rate of 60.00% (+5.38 percentage points), the highest reported Driving Score among published VLAs trained on Think2Drive demonstrations.
Read the full abstract ▾
All authors Erik Deinzer, Naya Baslan, Luca Paparusso, Narunas Vaskevicius, Peter Knott, Luigi Palmieri
Computer Vision · Cross-listed from cs.RO
GALA: Geometry-Aware Latent Action Modeling for Vision-Language-Action Model Pretraining across Embodiments
Yichen Liu, Puzhen Yuan, Xiang Zhu et al. (+2 more)
Learning large-scale
vision-
language-action (VLA) models from multi-embodiment
datasets remains challenging due to heterogeneous action spaces across end effectors. Although latent action models (LAMs) can learn embodiment-agnostic action representations from diverse
video data, existing image-based LAMs often fail to capture fine-grained end-effector articulation, particularly finger-level
geometric changes in human and dexterous
robot hands. To address this limitation, we propose GALA, a Geometry-Aware Latent-Action modeling framework that augments image-based latent actions with 3D end-effector
geometric motion. However, naively incorporating point clouds yields fine-grained action representations with limited shared semantics, hindering cross-embodiment pretraining. To address this issue, we introduce the Unified End-effector Motion Representation (UEMR), which preserves fine-grained motion information while improving the cross-embodiment generalizability of latent actions. Building upon UEMR, GALA combines visual latent actions that capture scene-level dynamics with
geometric latent actions that capture shared fine-grained end-effector articulation, providing effective supervision for VLA pretraining from multi-embodiment data, including action-free ego-centric human
videos. Experiments on fine-grained motion probing, cross-embodiment
retrieval, and downstream VLA evaluation demonstrate GALA's effectiveness in modeling generalizable fine-grained motions across embodiments, achieving 68.3% RoboCasa-GR1 success rate and 75.5% real-world success rate. Code, appendix, and demos are available at https://puzhenyuan.github.io/GALA-website/.
Read the full abstract ▾
All authors Yichen Liu, Puzhen Yuan, Xiang Zhu, Yanjiang Guo, Jianyu Chen
Computer Vision
Info3R: Information-Adaptive Test-Time Training for 3D Reconstruction
Sunghyun Baek, Hanna Bae, Minchan Kwon et al. (+1 more)
Transformer-based models have recently achieved strong performance on 3D reconstruction from images, and recent works extend them to process
video streams in an online manner for real-world deployment. However, existing methods overlook two key signals when handling long image streams: the importance of each incoming frame and the information saturation of the model's internal state. In this paper, we propose Info3R, a novel information-adaptive test-time training method for the online 3D reconstruction. We introduce an information-aware state update that modulates the state update strength based on the redundancy and informativeness of each incoming frame. To restore the state's plasticity – its capacity to incorporate new observations – we propose a dynamic state reset, triggered by the cumulative magnitude of state updates and the model's prediction confidence and accompanied by an anchor-to-world
alignment. Our method achieves consistent improvements on camera pose estimation,
video depth estimation, and 3D reconstruction, while substantially mitigating the performance degradation in the long sequence evaluation. Notably, on KITTI Odometry, our method achieves on average 1.68x lower ATE than LongStream, demonstrating its robustness on extended outdoor sequences.
Read the full abstract ▾
All authors Sunghyun Baek, Hanna Bae, Minchan Kwon, Junmo Kim
Computer Vision
The Role of Radiometric Features in Cross-Site Leaf-Wood Segmentation of LiDAR Point Clouds
Roman Kaharlytskyi, Derek T. Robinson and Roberto Guglielmi
Leaf-wood
segmentation of individual trees from
LiDAR point clouds is essential for quantitative structure models (QSMs) used in non-destructive biomass estimation. Existing
segmentation methods typically exclude radiometric features (e.g., intensity, return number) to maximize cross-sensor compatibility. We challenge this design choice by evaluating cross-site and cross-platform generalization: training on the public Heidelberg
dataset (terrestrial TLS, 1550nm) and testing on a novel
dataset from Ontario, Canada (RPA-LS, 905nm). Results show that geometry-only methods - including state-of-the-art deep learning models trained on high-density
LiDAR datasets - fail to generalize to the sparse, top-down geometry of aerial scans, achieving F1 scores <= 0.56. Incorporating radiometric features (intensity, return number, number of returns) improves F1 to 0.61, but more critically, increases wood recall by 119% from 0.16 to 0.35. Furthermore, geometry-only approaches often result in fragmented stem and branch components. We find that leveraging radiometric features preserves greater structural connectivity, resulting in more coherent architectures that are better suited for QSM reconstruction. We demonstrate that while
geometric patterns are view-dependent and prone to overfitting scan patterns, radiometric features encode physical material properties that generalize across disparate sensors and environments.
Read the full abstract ▾
Computer Vision
Catena: A Comprehensive Software Suite for Large-Scale Connectomics
Samia Mohinta, Pedro Gómez-Gálvez, Shi Yan Lee et al. (+5 more)
The gold standard
datasets for mapping connectomes are electron microscopy volumes of densely labeled neural tissue at nanometer resolution. Yet reconstructing and proofreading neuronal arbors and annotating all synapses requires pipelining multiple software tools that are often fragmented, inconsistently maintained, or proprietary, hindering reproducibility and automation. Here, we introduce Catena, an open-source, comprehensive, developer-centric software suite for connectomics that integrates modules for 3D
neuron and organelle
segmentation, synapse detection, microtubule tracking, and neurotransmitter inference. Catena organizes its modules in composable, chunk-wise processing pipelines in a completely documented, extensible, and adaptable design. We further reduce compute and ground-truth data requirements with pretrained machine learning models, facilitating fine-tuning. Catena ships fully containerized modules that encapsulate evolving dependencies for consistent execution across workstations and clusters. By consolidating open components, shareable models, and containerized runtimes, Catena delivers a reproducible and scalable approach to mapping cellular connectomes from electron microscopy volumes. Code and documentation: https://github.com/Mohinta2892/catena.git
Read the full abstract ▾
All authors Samia Mohinta, Pedro Gómez-Gálvez, Shi Yan Lee, Daniel Franco-Barranco, Michael Clayton, Stephan Preibisch, Jan Funke, Albert Cardona
Computer Vision
Benchmarking the Explanatory Quality of Open-Weight Vision-Language Models in Face Recognition
Laurent Colbois and Sébastien Marcel
Vision-
Language Models (VLMs) have recently been proposed as promising tools for face recognition, as they can produce natural
language explanations alongside similarity scores. This capability is considered appealing for face comparisons in forensic contexts, which require decisions to be transparent and auditable. However, existing evaluations of VLMs for that use case focus mostly on recognition accuracy, while the validity of generated explanations remains unquantified. In this work, we introduce a benchmarking framework for VLM-based face recognition that treats explanation quality as a core evaluation axis. We propose two criteria that explanations should satisfy: relevance, i.e., reliance on identity-stable facial features; and faithfulness, i.e.,
alignment with the visible image content without hallucinated features. We jointly develop a methodology enabling the quantification of relevance and faithfulness of evaluated models, based on constraining model outputs to a structured explanation format that supports automated querying and
auditing. Using this framework, we
benchmark several families of open-weight VLMs, jointly evaluating face verification accuracy and explanation quality. Our results highlight remaining shortcomings of produced explanations, and emphasize the need for such explanation quality metrics to get a complete picture of model performance. The proposed
benchmark and open-source evaluation harness provide a foundation for proper benchmarking and future fine-tuning of explainable face recognition systems.
Read the full abstract ▾
Computer Vision
Chronosphere: Space-Time Tessellation of Local Climate Experts
Daniel Cher, Eric Xing, Kexing Li et al. (+3 more)
We introduce Chronosphere, a spatio-temporal neural field that learns representations of
climate. A central challenge in geographic representation learning is modeling environmental processes whose spatial and temporal complexity varies widely. Yet existing location encoders typically fix a single level of detail everywhere. Global bases such as spherical harmonics spread capacity uniformly across space and time. Localized bases resolve only predefined regions. Learned tessellations adapt, but are inefficient at representing higher frequencies. Chronosphere unifies these approaches, pairing an adaptive tessellation of learnable sites on the spacetime
torus $S^2\times S^1$ with a shared bank of local basis functions. Both where capacity is placed and how much detail each region carries adapt to the data, across space and time. Trained to reconstruct climatology, Chronosphere matches or leads state-of-the-art location encoders across spatial and temporal tasks, with the largest gains under spatial and temporal transfer.
Read the full abstract ▾
All authors Daniel Cher, Eric Xing, Kexing Li, Brian Wei, Isaac Corley, Nathan Jacobs
Computer Vision
Morphology-Aware Ambiguity Learning for Wafer Defect Decision Support
Seungjun Chu and Seokhyun Chung
Wafer map defect recognition is commonly formulated as a fixed-taxonomy classification problem that assigns each wafer to a single defect class. However, some wafers exhibit morphologies near class boundaries, for which forcing a single prediction may be less informative than providing plausible diagnostic alternatives. This paper proposes a morphology-aware ambiguity learning framework that supports three diagnostic actions: automatic single-class diagnosis, assisted diagnosis with two plausible defect classes, and full review. Using the radial, angular, and
geometric characteristics of training wafer maps, the framework constructs a class-level ambiguity
matrix representing defect-class pairs with similar morphology and plausible diagnostic alternatives. It guides the model to learn plausible alternative classes rather than treating all incorrect classes equally. During inference, the
matrix determines whether an uncertain prediction can be represented by a meaningful two-class diagnostic set or should be escalated for full review. Experiments on WM-811K show that the proposed framework outperforms conventional approaches in defect recognition and diagnostic decision support, providing meaningful two-class alternatives while reserving full review for cases with unresolved ambiguity. Illustrative cost analyses further show the potential cost advantage of the proposed routing strategy. The diagnostic behavior of the framework remains consistent across different backbone architectures.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
The Weight Is Over - Interactive Diffusion on Consumer GPUs
Frieder Ganz and Maximilian Müller
On-device inference is booming, but the momentum is almost all in
language models.
Diffusion pipelines are
memory hungry, latency-sensitive, and require orchestrating an embedder, a
transformer, a decoder, and often further postprocessing that is not as standardized as LLM inference loops are. We navigate the trade-off between performance, quality, and model footprint to reach as many client devices in the wild as possible. We make three contributions: an
embedding translator that maps a small text encoder into a large encoder space to cut weight and latency; a reproducible sweep recipe for navigating the speed/quality/
memory triangle in
diffusion pipelines; and an interactive on-device image generation editor achieving sub-second TTFI on recent GPUs.
Read the full abstract ▾
Computer Vision
Object Detection Benchmarks are Incomplete: The Role of Label Errors and Annotation Uncertainty
Sarina Penquitt, Jonathan Klees, Antonia van Betteray et al. (+4 more)
While object detection has advanced through improved architectures and open-vocabulary models, we provide strong evidence that
benchmark quality is limited by annotation incompleteness. Across four widely used
datasets (COCO, Pascal VOC, Cityscapes, KITTI), re-annotation reveals substantial increases in annotated objects (e.g., up to +60% on KITTI and +40% on COCO), driven primarily by previously unlabeled small, occluded, or densely packed instances. While some differences arise from
dataset-specific annotation conventions, we consistently find that missing annotations are the main source of label errors across all
datasets. To achieve high data quality, we introduce a scalable annotation pipeline that emphasizes high recall and captures ambiguity through soft labels aggregated from at least 11 annotators per object. The resulting annotations improve coverage and align well with human calibration. We show that
benchmark performance is highly sensitive to annotation quality, although model rankings remain largely stable. We introduce two large-scale
benchmarks: (i) an uncertainty-aware object detection
benchmark, and (ii) a label error detection
benchmark grounded in real label errors. We show that current detectors are strongly depended on annotation quality and are misaligned with human perception. Current label error detection methods, which have been shown to perform well on synthetic noise, struggle to achieve high recall and precision on real label errors. Our results highlight the need for future object detection
benchmarks to move beyond deterministic annotations toward high-recall, uncertainty-aware evaluation that maximizes valid instances and better reflects real-world ambiguity.
Read the full abstract ▾
All authors Sarina Penquitt, Jonathan Klees, Antonia van Betteray, Parssa Jashnieh, Peter Stehr, Matthias Rottmann, Lars Schmarje
Computer Vision · Cross-listed from stat.ML
How Many Posterior Samples? Calibrated Stopping for Adaptive Sensing
Vincent Corlay and Andriy Enttsel
In classification-oriented adaptive sensing, posterior samples characterize uncertainty at the current measurement state and can serve two roles: they may guide the next sensing direction, while their class labels provide votes for the candidate classes and determine whether sensing should continue. We focus on the stopping layer that turns these votes into a declaration, without modifying the posterior sampler or sensing directions. A natural plug-in rule declares when the observed vote share exceeds a threshold. We show that this threshold is not itself a confidence guarantee: when the underlying vote mass equals the threshold, the plug-in rule declares about half the time. As alternatives, we calibrate a fixed-sample rule and a finite-horizon sequential rule to a prescribed false-declaration probability, and study exact curtailment, which stops a fixed-pool rule once its final verdict is forced. We then derive how one-round declaration probabilities determine posterior-sample cost and classification accuracy along a sensing path. On MNIST with DDRM and a fixed PCA-guided probe sequence, curtailment saves up to 62% of posterior samples. Among the evaluated rules at matched operating points, sequential stopping reduces the cost the most. At a high accuracy, that same sequential rule can trade more posterior samples for fewer measurements.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.SP
Classification-oriented adaptive sensing via posterior sampling
Andriy Enttsel, Maxime Rousselot and Vincent Corlay
Recent advances in
diffusion models have enabled high-performance, instance-adaptive compressed sensing through posterior sampling, without task-specific policy training. Existing methods select sensing probes by maximizing total posterior signal variance and are therefore primarily reconstruction-driven. We introduce a classification-driven extension motivated by the closed-form posterior covariance of a class-conditional Gaussian mixture model, which decomposes into within-class and between-class uncertainty. Using calibrated soft classifier outputs, we estimate these uncertainty terms from
diffusion posterior samples and propose a classification-oriented criterion for selecting the dominant sensing direction in the unmeasured subspace. Experiments on MNIST and CIFAR-10 compare the resulting classification accuracy, measurement cost, and reconstruction quality with those of reconstruction-oriented counterparts. The results identify regimes in which semantic posterior uncertainty yields a more favorable classification–measurement trade-off and quantify the associated reconstruction cost.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention
Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen et al. (+2 more)
Multimodal survival models can combine complementary prognostic information from whole-slide images and genomic profiles, but effective fusion remains challenging amid external cohort shift and computational complexity. To address these challenges, we propose MIST,
multimodal survival prediction with genomic-guided histology
attention. MIST represents genomic features as
tokens and allows them to query compact foundation-model-derived histology context
tokens before survival prediction. This design enriches molecular information with histology context rather than merging separately encoded modalities only at the final stage. Training combines discrete-time survival prediction with genomic feature masking, WSI dropout, and paired WSI-genomics contrastive
alignment. Across four external evaluations in colon, renal, lung, and glioblastoma cohorts, MIST improves external C-index over standard fusion baselines in the primary comparisons. These results support genomic-guided histology
attention as a compact and effective strategy for
multimodal oncology outcome prediction. Our code is available at https://github.com/samiyavuuz/MIST .
Read the full abstract ▾
All authors Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen, Jana Lipkova, Benedikt Wiestler
Computer Vision
VideoReloc: Long-Term Indoor Video Relocalization against a Kilobyte-Scale Semantic Scene Graph
Qianru Li, Xuyang Chen, Xuqin Wang et al. (+6 more)
Given a compact semantic scene graph, long-term indoor
video relocalization estimates a map-frame trajectory after lighting and furniture changes. Visual methods rely on appearance and become unreliable under these changes; localizing one frame at a time from object classes and geometry instead leaves sparse, ambiguous evidence. We introduce VideoReloc, whose adaptive clips use odometry to gather spatial evidence until object and motion criteria are met, adapting query length to the observed scene. Its run-level decision rechecks conflicting placements using evidence accumulated across connected clips, stabilizing the trajectory beyond adjacent-clip tracking. Hypothesis-first registration proposes poses from object triplets and verifies each using clip-wide object centers and box surfaces. Orientation-aware refinement uses box faces,
gravity and wall directions to resolve ambiguity in camera orientation and refine the full pose. This reframes sparse-map relocalization as verification of spatially extended
video queries, moving discriminative support from stored appearance to temporal context and permitting a 100 kB map of class-labelled boxes. On RIO10 and ReplicaCAD, the all-frame localization success rate at 1 m/10$^\circ$ is 73.5% and 61.1% under causal evaluation, rising to 90.6% and 74.8% with clip closure. The evaluated per-frame scene coordinate regressors reach up to 47.6% and 49.8%, respectively, with maps of 12.6-42 MB. Project page: https://videoreloc.github.io
Read the full abstract ▾
All authors Qianru Li, Xuyang Chen, Xuqin Wang, Zhenghao Zhang, Hongyi Luo, Tao Wu, Daniel Cremers, Lu Liu, Yanfeng Zhang
Computer Vision
A Principled Approach to Unsupervised Anomaly Detection
James Myles, Matthew Baugh, Johanna P. Müller et al. (+2 more)
Traditional unsupervised anomaly detection (UAD) methods are designed to flag or localise deviations from a normative distribution, ignoring the underlying generative mechanisms of the anomalies. Yet the nature of an anomaly is often as important as its presence. We reformulate UAD as a
Bayesian inverse problem, in which the objective is to infer the most probable corruption responsible for each observation. Our framework yields a probabilistic anomaly score as the energy of the inferred corruption parameters, and serves as a principled recipe for developing new UAD
algorithms. We derive several existing methods as instances of the general framework, each corresponding to the same energy score under different modelling choices. Experimentally, we study the framework's components in a controlled setting, and improve object-class AUROC on the MVTec AD
dataset by 2.3% by adapting the underlying corruption model. Finally, we validate the framework on a
brain MRI
benchmark, achieving strong detection performance while producing estimates of pathology intensity, bias, and geometry. Code is available at https://github.com/jgmyles/inverse-uad.
Read the full abstract ▾
All authors James Myles, Matthew Baugh, Johanna P. Müller, Bernhard Kainz, Yingzhen Li
Computer Vision
PointLAM: Local Attentive Mamba for Efficient Point-based 3D Object Detection
Xuanming Shang, Weijia Zhang and Chao Ma
3D object detection from
LiDAR point clouds faces a fundamental dilemma: voxel-based methods achieve efficiency at the cost of
geometric quantization, while point-based methods preserve fidelity but suffer from prohibitive computational bottlenecks. Specifically, point-based architectures are crippled by slow downsampling strategies (e.g., FPS) and expensive dynamic neighbor queries (e.g., k-NN) coupled with costly continuous interactions. To tackle these systemic inefficiencies, we propose PointLAM, a highly efficient and powerful point-based architecture driven by two synergistic innovations. First, to resolve the downsampling bottleneck, we develop the
Laplacian Point Sampler (LPS). LPS employs an implicit discrete
Laplacian high-pass filter and Doubly Sorted Sampling to achieve fast, structure-aware foreground preservation. Second, to overcome local modeling latency, we design the Local Hadamard Aggregator (LHA). LHA decouples spatial indexing from feature representation using transient grids, and replaces complex continuous interactions with a Hadamard Gating mechanism for
topology-aware, attentive modulation. By coupling this local gating with Bi-Directional Mamba (BDM) layers for global sequence modeling, we formulate the Local Attentive Mamba (LAM) block. Powered by this architecture, PointLAM achieves competitive performance on nuScenes and Waymo for point-based detectors. It rivals highly optimized voxel competitors while requiring a fraction of the computational footprint, demonstrating marked superiority in detecting small instances and handling extreme sparsity. Project page: https://pointlam.github.io/.
Read the full abstract ▾
Computer Vision
XCalib Depth-Guided Geometric Optimization for Dense Thermal-Visible Video Registration
Aurelien Godet, Gabriel Jobert and Mauro Dalla Mura
Image registration is a vital preprocessing step in
multimodal perception tasks, including image fusion, object detection, and semantic
segmentation. In Advanced Driver- Assistance Systems (ADAS), spatial misalignment between visible (RGB) and infrared (IR) cameras -caused by non-coincident optical axes and field-of-view differences- introduces non-uniform parallax and visual ghosting. Classical keypoint-based methods are restricted to global homographies that fail under dynamic depth, while unconstrained dense flow
algorithms lack structural regularization and suffer from temporal instability. In this paper, we propose XCalib, an unsupervised dense thermal-visible registration framework that bridges this gap. Rather than serving as an absolute metric calibration tool, XCalib leverages virtual pinhole camera parameterization strictly as a
geometric constraint space. By optimizing effective relative pose and intrinsics alongside predicted
monocular metric depth, XCalib restricts the search space of spatial displacements to physically valid projection geometries. Our key contributions are: (1) a novel registration paradigm that uses camera parameterization as an implicit regularizer for dense cross-modal warping; (2) Normalized Edges Correlation (NEC), a robust structural similarity metric tailored to cross- spectral
alignment; and (3) extensive quantitative and qualitative evaluations across public ADAS
datasets, demonstrating superior temporal stability and
alignment accuracy over unconstrained dense flow baselines.
Read the full abstract ▾
Computer Vision
Beyond Benchmark Scores: Auditing Medical Vision-Language Models for Chest X-Ray Tuberculosis Screening
Mushir Akhtar, M. Tanveer and Mohd. Arshad
A medical model's
benchmark score does not establish that the same conclusion holds under a different evaluation. This study tests whether claims about model ranking, score reliability and screening performance survive changes in cohort,
prompt, negative spectrum, specified prevalence and operating threshold. We audit three medical
vision-
language models (BioMedCLIP, CheXficient, and MedSigLIP) and a general-domain OpenCLIP comparator on 12,200 chest radiograph records from four
datasets (Montgomery, Shenzhen, TBX11K, and VinDr-CXR). Five fixed
prompt families yield 244,000 model–image–
prompt scores. No model leads every cohort and reliability criterion.
Prompt-family changes alter AUROC in 21 of 48 multiplicity-controlled comparisons. Replacing healthy controls with sick non-tuberculosis controls reduces AUROC by 0.075–0.306 across all four models. On VinDr-CXR, the three medical models distinguish tuberculosis from no-finding controls substantially better than from pneumonia or lung tumor; their AUROC point estimates for both named diseases fall below 0.5. CheXficient has documented VinDr-CXR pretraining exposure, which limits the interpretation of its results. Thresholds chosen for 95% sensitivity on TBX11K training retain that constraint by point estimate in only four of sixteen target evaluations. A five-seed supervised source model reaches 0.999 AUROC on TBX11K validation but 0.629 on each of two external cohorts. Conservative exclusion of perceptual-overlap candidates narrows this gap without closing it. These retrospective, single-task results show that discrimination, score reliability and threshold retention support different portability claims. Evidence for chest X-ray tuberculosis screening should identify the complete evaluation specification rather than attribute clinical portability to a checkpoint alone.
Read the full abstract ▾
Computer Vision
SFVO: Decoupled Confidence-Guided Stereo-Flow Visual Odometry with Bidirectional PnP
Kai Zhang, Guoyang Zhao and Jun Ma
Deep learning-based visual odometry (VO) has achieved significant progress, yet most existing methods focus on a
monocular approach, which suffers from scale ambiguity. Stereo VO provides real metric by its nature, but remains less studied in deep learning VO due to its high computational cost and modeling complexity. Recent advances in stereo matching and optical flow estimation have made dense visual correspondence increasingly accurate and reliable, but their complementary
geometric information has not been fully exploited for VO. In this paper, we present SFVO, a correspondence-driven stereo VO framework that directly builds upon pretrained stereo matching and optical flow models. SFVO exploits pretrained stereo matching and optical flow models to estimate stereo and temporal correspondences. Instead of learning pose directly from images, SFVO maps learned correspondences into
geometric constraints and predicts which points are trustworthy. To improve the reliability of visual correspondence-based
geometric constraints, we introduce decoupled confidence maps for rotation and translation. This design better aligns the characteristics of visual correspondence and 6-DoF transformations. Extensive experiments on outdoor and indoor
datasets demonstrate that SFVO achieves robust and accurate pose estimation with strong generalization capability. The code will be released.
Read the full abstract ▾
Computer Vision
Balanced Prompt Adaptation against Entropy-Induced Collapse for Test-Time Binary Segmentation
Zhengshan Wang, Joshua Charles Webster-Ford, Yifei Tian et al. (+3 more)
Entropy minimization is a standard objective for test-time adaptation (TTA), but it can fail in imbalanced binary
segmentation. Unlike image classification, dense
segmentation aggregates thousands of pixel predictions, allowing the larger predicted class to dominate the update, pull minority predictions toward itself, and produce a degenerate mask as predictions saturate and their
entropy gradients vanish. We theoretically establish this collapse in a shared-shift model. This analysis motivates Balanced-Anchor
Prompt Adaptation (BAPA), which combines two complementary modules. The Class-Balanced Anchors (CBA) module selects high-confidence anchors separately from each predicted class and gives foreground and background equal total loss weight, preventing the larger region from dominating the update. Dynamic
Prompt Adaptation (DPA) refreshes these anchors after each prediction update and optimizes only text-side
prompt residuals while keeping the
vision-
language encoders frozen. This
prompt-only update refines the foreground-background decision boundary without altering the pretrained dense visual representation. Across experiments from four domains, BAPA achieves the highest mean Dice among the evaluated methods. Factorized ablations further validate the complementary roles of CBA and DPA, supporting balanced
prompt adaptation as an effective alternative to
entropy minimization for test-time binary
segmentation.
Read the full abstract ▾
All authors Zhengshan Wang, Joshua Charles Webster-Ford, Yifei Tian, Xinxin Wang, Long Chen, Weiping Ding
Computer Vision
ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation
Boni Hu, Xiong Wei, Haoming Huang et al. (+20 more)
Generative world models offer controllable and repeatable closed-loop simulation for end-to-end and
vision-
language-action driving policies, but production deployment exposes three unresolved requirements: faithfully reproducing a mixed fisheye-pinhole rig at native resolutions; reconciling causal, per-timestep interaction with long-horizon stability and low latency; and preserving scene identity when a location is revisited. We present ZYT-World, a single architecture that natively generates four fisheye views with field of view > 180{\deg} and three pinhole views. Projection-specific Plucker adapters encode camera geometry, ego-motion adaptive layer normalization provides global motion control, and a lightweight pixel-aligned layout conditions traffic participants and signals through instance-level boxes, headings and colors. Heterogeneous training combines full-rig
geometric coverage with high-resolution detail. Teacher forcing, causal consistency
distillation, self-rollout distribution matching
distillation, and RigCritic transform a 40-step bidirectional teacher into a one-step, per-latent streaming generator, with RigCritic evaluating the seven-view rig jointly. A 19M-parameter variational autoencoder decoder (TinyVAE), W8A8 quantization, and our inference engine reduce decoding, backbone, and incremental-execution costs, respectively. Finally, cross-trajectory pairs derived from real captures train a plug-in implicit-
memory module that preserves place-specific evidence. On the internal multi-view test set, the one-step model retains more than 90% of the teacher's PSNR and SSIM, while FID, FVD, and LPIPS stay within 11% of the teacher. Under the generator-only timing in Figure 2, it is 107.7 times faster than the 40-step bidirectional teacher. TinyVAE decodes 59.8 times faster than Wan. 30s rollouts and cross-trajectory revisits show the intended long-horizon and
memory behavior.
Read the full abstract ▾
All authors Boni Hu, Xiong Wei, Haoming Huang, Yong Huang, Chenbo Wang, Yi Yang, Jiancheng Wang, Ruicheng Zhu, Zhimin Yang, Guanglai Liu, Qiaowan Jin, Dongzhuo Wang, Haiwei Kuang, Jiajun Fan, Yue Wu, Jiaxin Wei, Hao Sun, Feihong Yan, Yuyao Zhou, Wei Bi, Kaixuan Wang, Zichao Guo, Xiaozhi Chen
Computer Vision
SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation
Ronghui Li, Jun Dong, Zhongyuan Hu et al. (+8 more)
Large
language models (LLMs) provide limited support for sign
language interaction. Unifying sign
language translation (SLT) and generation (SLG) to enable sign
language as both input and output can reduce switching between separate models during sign-text interaction. We present SignGPT, a unified, pose-based framework for gloss-free SLT and SLG. SignGPT integrates part-aware hierarchical representations of body, hand, and facial motion into a shared
language model and employs asymmetric multi-
token prediction and progressive training for bidirectional modeling. We evaluate SignGPT on How2Sign (ASL) and Phoenix-2014T (DGS) through
benchmark comparisons, qualitative analyses, and component ablations. An exploratory study with 12 Deaf ASL signers assesses an LLM-mediated sign-to-sign response pipeline, highlighting the potential of unified modeling to support sign
language conversation (SLC).
Read the full abstract ▾
All authors Ronghui Li, Jun Dong, Zhongyuan Hu, Zunnan Xu, Jun Zhou, Liyuan Chen, Shuoling Liu, Jiangpeng Yan, Jie Guo, Xiu Li, Linchao Bao
Computer Vision
Diffusion-Based Tumor Inpainting for Renal Segmentation under Clinical Data Scarcity
Ekaterina Sedykh, Salme Ussanov, Dmytro Fedorenko et al. (+1 more)
Deep learning
segmentation of renal tumors requires large annotated
datasets, yet clinical deployments typically offer only a handful of tumor-positive cases from the target site. We propose a
diffusion-based inpainting framework that synthesizes anatomically plausible renal tumors within healthy CT scans, requiring no additional annotation, and provide the first systematic comparison of 2D, 2.5D, and full 3D (MAISI) synthesis strategies for this task. Training the
diffusion model on public data (KiTS23, KIRC) and evaluating nnU-Net
segmentation on a internal cohort across three low-data regimes, we find that 2.5D and 3D augmentation substantially reduce false positives (from $\sim$18–20% to $\sim$3–6%) while maintaining Dice, whereas 2D provides no consistent benefit. Crucially, the proposed 2.5D method matches full 3D synthesis on every metric at substantially lower computational cost, indicating that local volumetric consistency alone is sufficient for effective augmentation in data- and resource-scarce clinical settings.
Read the full abstract ▾
All authors Ekaterina Sedykh, Salme Ussanov, Dmytro Fedorenko, Dmytro Fishman
Computer Vision · Cross-listed from cs.AI
Listen Before You Speak: Response Planning from Listener Facial Reactions for Conversational Speech Generation
Yunji Chu
Conversational
speech depends on dialogue context and the listener's immediately preceding behavior. We propose ReACT-TTS, a two-stage framework that uses a one-second pre-response listener facial sequence to plan the next utterance's
emotion and prosody before
speech realization. On a strict dyadic MELD protocol, Temporal conditioning yields higher mean macro-F1 and VAD concordance than Text-only across ten seeds, while accuracy remains essentially unchanged. Ablations show that temporal modeling performs best among the visual variants and that an explicit early-to-late difference is unnecessary; correct listener reactions also outperform cyclic mismatches on average. In a contextual-appropriateness study with 20
speech researchers, 76% of judgments prefer Temporal, 9% Text-only, and 15% report no preference. We further connect the predicted response style to a Grad-TTS backbone for end-to-end
speech realization. Overall, the results support pre-response listener dynamics as complementary cues for conversational response planning. The source code is available at https://github.com/CYJ1/ReACT-TTS_public.
Read the full abstract ▾
Computer Vision
Wan Xu, Yuanfan Guo, Kevin Han et al. (+2 more)
Despite the remarkable progress in
Multimodal Large
Language Models (MLLMs), prevailing Chain-of-Thought (CoT) paradigms remain confined to the natural-
language expression space. Consequently, they inherently incur excessive linguistic overhead, leading to information dilution and weak visual grounding. To address this challenge, we propose Dense
Reasoning Trace (DRT), a paradigm that departs from natural-
language-centered CoT by expressing
reasoning as compact structured traces, which include concise intermediate states with symbolic connectors and disentangle visual observations from logical deductions. First, we introduce the Dense Trace Initialization to internalize the DRT
reasoning mode into the model, substantially improving
token efficiency while preserving visual evidence. To further enable the model to faithfully capture the logical relations within traces, we propose the Trace-Grounded Reinforcement Learning framework, which builds reference traces through a tri-perspective verification pipeline and employs Trace-Grounded GRPO with structured rewards, encouraging the model to generate concise DRT-style traces with reduced
hallucination and stronger logical grounding. Extensive experiments on challenging
reasoning benchmarks show that DRT achieves 5.5$\times$
token efficiency improvement while improving 1.3 accuracy points over the Qwen3-VL baseline. These findings suggest that complex
multimodal reasoning may not require verbose natural-
language traces, opening a more efficient path for next-generation MLLMs. Our code and data are available at: https://github.com/HIT-leaderone/DRT
Read the full abstract ▾
All authors Wan Xu, Yuanfan Guo, Kevin Han, LaLa Chen, Wangmeng Zuo
Computer Vision
Configurable Multi-Stage Vision Pipeline for Crop Disease and Pest Diagnosis
Naga Ganesh, Chandrashekar M S, Lakshmi Pedapudi et al. (+2 more)
FarmerChat is Digital Green's farm advisory service for smallholder farmers. When something looks wrong with a crop, the farmer takes a photograph and sends it, and that photograph is the whole question: no symptom described, no crop named, often no text at all. The service has to determine whether the picture can be used, what crop it shows, and what is wrong with it, from images taken on cheap phones in a field, in poor light and with a moving camera. The system doing this today cannot be adjusted. It has no adjustable thresholds for photograph rejection, crops and problems cannot be added, and there is no confidence cut-off to set. We study about 1.16 million photographs sent to FarmerChat from Ethiopia, India, Kenya and Nigeria. The production quality gate rejected 46.8% of the images it judged, over a quarter of those reaching diagnosis returned no crop name, and 35.8% of the labelled problems filed under "disease" are pests, identifiable without the crop. We therefore split the work into three stages: a quality gate (M0), a crop detector (M1), and a disease or pest detector (M2). Route A fills all three with one fine-tuned
vision-
language model (Qwen3-VL-4B) answering in a single call. Route B fills each with a small specialist model (DaViT, YOLO26). We replace our production GPT-4o quality gate with a small MobileNetV3 gate at 86.9% F1 in 12 ms. On one test set scored the same way for every system, a hierarchical DaViT-Base achieves 95.41% crop accuracy against 91.46% for the production baseline. It also leads on diagnosis and never declines to answer, while every
language model in the comparison leaves a large share of rows with no diagnosis. The fine-tuned model retains two capabilities the specialists do not have: one call for all three stages, and a request for a better photograph when the image cannot support an answer.
Read the full abstract ▾
All authors Naga Ganesh, Chandrashekar M S, Lakshmi Pedapudi, Aakash Singh, Vineet Singh