Computer Vision · Cross-listed from cs.AI
SPARC: A Multi-Agent System for Electrical Circuit Question Answering
Mushtari Sadia, Zhenning Yang, Umme Habiba Lamia et al. (+3 more)
Electrical
circuit diagram QA tasks require complex mathematical
reasoning, which remains challenging for
multimodal LLMs. We present SPARC, a multi-
agent system that answers questions over
circuit diagrams by grounding
reasoning in executable physics-based simulations. SPARC uses LLM
agents to synthesize, execute, and analyze simulation programs, improving accuracy and reliability by design. It achieves 83% accuracy, with up to a 58% absolute improvement over baselines, while enabling systematic error diagnosis.
Read the full abstract ▾
All authors Mushtari Sadia, Zhenning Yang, Umme Habiba Lamia, Nishat Shawrin, Ang Chen, Amrita Roy Chowdhury
Computer Vision · Cross-listed from eess.AS
Pixel-TTS: Image based Text Rendering for Robust Text-to-Speech
Adarsh Arigala, Arjun Gangwar, S Umesh et al. (+1 more)
Recent advances in pixel-based text modeling show that representing text as images enables models to exploit visual cues for
language understanding. Grounding text in its visual form allows structurally similar characters with different Unicode encodings to produce similar
embeddings, benefiting cross-lingual and zero-shot scenarios. Conventional text-based approaches treat each character independently, limiting generalization to unseen characters and requiring
embedding expansion during cross-lingual adaptation. We propose Pixel-TTS, a text-to-
speech framework for visually grounded
speech synthesis. It renders text as images and projects them through a 2D convolutional layer to generate
embeddings. This design eliminates
embedding matrix expansion during fine-tuning while improving robustness to unseen characters and orthographic variations. Extensive experiments show Pixel-TTS achieves competitive performance with strong baselines, faster convergence and robust zero-shot generalization.
Read the full abstract ▾
All authors Adarsh Arigala, Arjun Gangwar, S Umesh, Yova Kementchedjhieva
Computer Vision
Automated 3D Kinematic Monitoring for Circadian Activity and Anomaly Detection in Juvenile Fish
Chih-Wei Huang, Chang-Wen Huang, Chung-Ping Chiang et al. (+1 more)
Precision aquaculture faces a "phenotyping bottleneck" in tracking high-resolution behavioral traits, as conventional methods cannot quantify instantaneous three-dimensional (3D) physical exertion. To address this, we present a high-throughput 3D behavioral phenotyping framework integrating deep learning object detection with binocular stereo
vision for real-time monitoring of juvenile tilapia in high-density environments. The system automates non-contact body length estimation and reconstructs 3D swimming trajectories from absolute spatial coordinates. By eliminating 2D perspective distortions, this approach precisely quantifies 3D velocity and acceleration, marking the first estimation of true physical swimming speeds in free-roaming juveniles. Results show the framework successfully establishes circadian locomotor baselines, serving as an early warning system for physiological stress and providing an objective metric for fish vitality.
Read the full abstract ▾
All authors Chih-Wei Huang, Chang-Wen Huang, Chung-Ping Chiang, Tsung-Wei Pan
Computer Vision
Is My Vision-Language Data in Your AI? Membership Inference Test (MINT) Demo 2
Daniel DeAlcala, Gonzalo Mancera, Julian Fierrez et al. (+3 more)
We present the Membership Inference Test (MINT) Demo 2, a framework designed to improve transparency in machine learning training processes. MINT is a technique for experimentally determining whether specific data were used during machine learning model training. We establish the theoretical framework and propose multiple architectures for MINT depending on the amount of information known about the models that are being audited. Experimental results using a popular face recognition model, 4 state-of-the-art LLMs, and multiple, diverse, and large-scale public image and text databases achieve promising accuracy levels in the detection of training data of up to 90%. Building on these results, we introduce a comprehensive web platform1 that expands these capabilities to image and text modalities. The platform integrates a diverse technological stack, including MINT, aMINT, and gMINT, allowing users to audit a wide range of models. This demonstrator aims to promote AI transparency and provides a practical tool to foster compliance with emerging AI regulations.
Read the full abstract ▾
All authors Daniel DeAlcala, Gonzalo Mancera, Julian Fierrez, Aythami Morales, Ruben Tolosana, Ruben Vera-Rodriguez
Computer Vision
MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios
Haitian Wang, Ruoxi Sun, Quantong Qiu et al. (+5 more)
Recent advancements have significantly expanded the theoretical context windows of
Multimodal Embedding Models (MEMs). However, larger context windows do not necessarily translate into effective comprehension and representation of long-context
multimodal inputs, which remains a critical bottleneck for real-world deployment. To address the lack of systematic evaluation in this setting, we introduce MMLongEmbed, the first comprehensive
benchmark for evaluating MEMs in long-context scenarios. MMLongEmbed comprises four
retrieval tasks spanning multiple context-length ranges, covering text, document, and
video modalities. Through extensive evaluation of state-of-the-art models, we find that current architectures rely heavily on superficial feature matching and struggle to capture deep semantic and structural dependencies. We further observe that performance degradation varies systematically with context length and key information placement. Moreover, models exhibit substantially different robustness to redundant contextual information across modalities. For reproducibility, the
benchmark and code are publicly available.
Read the full abstract ▾
All authors Haitian Wang, Ruoxi Sun, Quantong Qiu, Juntao Li, Junhui Li, Hua Chen, Jinxiong Chang, Min Zhang
Computer Vision · Cross-listed from eess.SP
C-MambaPose: Phase-Preserving Complex Mamba for Cross-Environment Wi-Fi Human Pose Estimation
Phuc H. Nguyen and Van-Dinh Nguyen
Cross-environment generalization remains a fundamental challenge for Wi-Fi-based human pose estimation (HPE), as environmental multipath can drastically alter Channel State Information (CSI) for the same human motion. Existing approaches often discard CSI phase or collapse complex signals into real-valued features early, losing motion-sensitive phase dynamics. We present C-MambaPose, a phase-preserving complex-valued state-space framework for robust cross-environment Wi-Fi 3D HPE. Our key insight is to preserve complex phase relationships throughout spatiotemporal modeling, rather than using phase only as an input feature. C-MambaPose sanitizes synchronization-induced phase errors and employs a Spatiotemporal Complex Mamba encoder with dynamic multi-scale receptive fields to capture long-range phase-amplitude dynamics. Cross-
attention then maps the learned representations to human joints, followed by a skeleton-aware GraFormer decoder for anatomically coherent pose estimation. Experiments on MM-Fi show that C-MambaPose achieves strong performance across all settings, with the largest gains in unseen environments, reaching $298.5$ mm and $317.6$ mm MPJPE on the two cross-environment protocols using only $3.78$ M parameters and no pretraining. Ablations further show that prematurely discarding complex phase relationships substantially degrades performance, demonstrating their importance for cross-environment Wi-Fi HPE.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
On the Controllability-Fidelity Frontier in Diffusion Editing
Yi Hu, Leying Yi, Emily Davis et al. (+1 more)
Diffusion-based generative models enable powerful image editing capabilities, but achieving precise control while maintaining fidelity and safety remains challenging. We present a comprehensive theoretical and empirical study of controllable
diffusion-based image editing, analyzing the trade-offs between adherence to user intent, preservation of non-target content, and output quality. Our work spans text- and mask-guided edits, point/drag manipulation, and inversion-based pipelines. We derive mathematical formulations of editing objectives and analyze dynamics of noise injection, score guidance, and inversion error. We provide theoretical bounds on reconstruction error, stability under repeated edits, and locality of changes. We propose algorithmic frameworks (with pseudocode) for mask-localized and instruction-guided editing, and present extensive experiments comparing state-of-the-art methods (e.g.\ TF-ICON \cite{lu2023tficone}, DragFlow \cite{zhou2025dragflow}, InstructPix2Pix \cite{brooks2023instructpix2pix}, UltraEdit \cite{zhao2024ultraedit}) on multiple tasks and metrics (FID, identity similarity, CLIP
alignment, artifact scores, etc). Our results reveal key failure modes, such as identity drift,
prompt sensitivity, and compositional errors. We also discuss ethical considerations in image editing, including misuse risks, bias, consent, and concept erasure techniques (e.g.\ MACE \cite{lu2024mace}, ANT \cite{li2025ant}, EraseAnything \cite{gao2024eraseanything}) as safeguards. We conclude with best practices and future directions for responsible, high-fidelity
diffusion-based editing.
Read the full abstract ▾
All authors Yi Hu, Leying Yi, Emily Davis, Finn Carter
Computer Vision · Cross-listed from physics.optics
Beyond the Thin-Layer Limit: Differentiable Volumetric Training for Visible-Range Diffractive Neural Networks
Dineth Jayakody and Dushan N. Wadduwage
Diffractive deep neural networks (D2NNs) promise miniaturized, power-efficient, light-speed optical front-ends for machine
vision, yet the most mature demonstrations remain in the terahertz regime, built from readily fabricated millimeter-scale
neurons. Translating D2NNs to the visible range, where nearly all
vision pipelines operate, was long blamed on the difficulty of fabricating nanoscale
neurons; but even after recent advances removed that barrier, visible-range D2NNs matching their terahertz counterparts remain out of reach. We identify the true obstacle as the thin-layer approximation underlying nearly all D2NN training, which treats each diffractive layer as an infinitely thin mask. It fails not because of the short wavelength, as is commonly assumed, but because the low-refractive-index materials (n approximately 1.3-1.5) used at visible wavelengths require relief structures thick enough that intra-layer diffraction and phase accumulation become significant. To overcome this, we introduce a differentiable beam-propagation ($\partial$BPM) layer that models each element as a finite-thickness volume and propagates light through it during training, keeping the fabrication-compatible height map end-to-end trainable without full-wave simulation in the loop. Across MNIST, Fashion-MNIST, and CIFAR-100 classification and imaging, $\partial$BPM training substantially reduces the design-to-device mismatch, and full-wave FDTD validation raises classification accuracy from 50% to 90% without re-optimization. The $\partial$BPM layer thus offers a scalable, physics-aware bridge between efficient optical neural-network optimization and fabrication-consistent diffractive design.
Read the full abstract ▾
Computer Vision
Sung-Wook Lee, Xuhui Kang and Yen-Ling Kuo
Discrete
Vision-
Language-Action (VLA) models typically formulate action generation as next-
token prediction over discretized action spaces, conditioning each
token autoregressively on prior context. While effective, this paradigm incurs high inference latency and largely ignores the temporal structure inherent in action trajectories. Recent efforts introduce parallel decoding to improve efficiency, enabling faster inference, but lack explicit mechanisms for modeling
token dependencies. We introduce TBD-VLA, a discrete
token-based VLA framework that incorporates block
diffusion to enable temporal action generation. We partition action sequences into temporal blocks and perform masked discrete
diffusion within each block, while maintaining autoregressive generation across blocks. This design unifies temporal autoregression and parallel action decoding, achieving both strong temporal coherence and improved inference speed. In addition, the explicit temporal modeling enables asynchronous execution of action chunks (e.g., Real-Time Chunking) via temporal in-painting. TBD-VLA significantly outperforms prior VLA approaches in both simulation and real-world manipulation tasks, offering a scalable path toward fast, temporally aware, discrete VLA models. Project webpage: https://tbd-vla.github.io/
Read the full abstract ▾
Computer Vision
C3VD-DEFCOL: A Deformable Colonoscopy Dataset with Time-Resolved 3D Ground Truth and Realistic Appearance
Ethan Luk, Mayank V. Golhar, Anthony Song et al. (+5 more)
3D reconstruction could improve colonoscopy by estimating mucosal coverage and alerting clinicians to missed regions during screening. However,
algorithm development is limited as no current
datasets provide both a realistic in vivo appearance and dense, time-resolved 3D ground truth, especially under non-rigid deformation. We present C3VD-DEFCOL, a framework and
dataset for evaluating deformable colonoscopy reconstruction with paired geometry and realistic texture. Starting from C3VD/C3VDv2 colon meshes and camera trajectories, we generate controlled deformations of the colon surface, including peristaltic waves and centerline motion, and render per-frame depth, surface normals, optical flow, camera poses, and time-stamped 3D meshes. We then use the rendered geometry, primarily depth, to condition an LTX-2.3-based sim-to-real translation model that produces RGB clips with in vivo-like mucosal color, texture, vasculature, and specular appearance while preserving the underlying 3D scene structure. The resulting
dataset contains 110
videos from 11 unique colon mesh geometries, with varying camera trajectories, appearances, and parameterized deformation regimes, including three peristaltic severity levels that serve as controlled evaluation axes. We evaluate the generated
videos using appearance realism,
geometric consistency, and temporal consistency metrics, and use the paired ground truth to
benchmark the downstream task of pose estimation in deformable 3D reconstruction. Our experiments show how pose estimation error increases with increasing deformation severity, providing a controlled stress test that is not possible with existing in vivo
datasets. Overall, C3VD-DEFCOL is designed as a reproducible, quantitative evaluation platform for testing deformable 3D reconstruction
algorithms, with the goal of reducing the domain gap between synthetic
datasets and in vivo colonoscopy.
Read the full abstract ▾
All authors Ethan Luk, Mayank V. Golhar, Anthony Song, Raúl Iranzo, Víctor M. Batlle, Lalithkumar Seenivasan, José M.M. Montiel, Nicholas J. Durr
Computer Vision
The Cross-Architecture Substrate: A Domain-Transcendent, Calibration-Surviving Geometric Invariant of Modern Vision Encoders
Yousef Radwan
Different
vision neural networks – trained to classify, contrast, reconstruct, or match images to text – should have correspondingly different internal representations. We report that they do not. After training, the top sixteen principal directions of variation inside thirteen modern
vision encoders converge to the same sixteen-dimensional
geometric object. We call this the cross-architecture substrate and study it with PCA, centred kernel
alignment (CKA), and Pang 2026 calibration. The substrate transports across four visual domains (natural photographs, medical CT, satellite, microscopy) at median Procrustes-CKA 0.679, and across eight domains (adding sketches, depth, thermal infrared, astronomy) at 0.604, every pair >0.40. It survives Pang calibration globally (7.4x disc-vs-MAE separation, n=13,394) and locally (4.82-5.30, p<10^{-44}). It is not pixel statistics (0.263), not Gabor features (0.31), not a random projection (0.041), and emerges in the first 10% of training while accuracy keeps climbing. We deliver four applications: a label-free transferability filter beating LogME (3x faster, +0.15 Kendall-tau); a four-way domain detector (99.6% accuracy); a frozen low-shot probe (16 dims beat 768-dim DINOv2 by 3.78pp at N=50 labels per class); and a teacher-free
distillation auxiliary matching trained-teacher KD on 33 pairs (7.56pp peak gain at 10% label fraction). The substrate does not cross modalities, does not help cross-paradigm
distillation, and does not predict transfer quality (rho=0.08 against transfer accuracy).
Read the full abstract ▾
Computer Vision
VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning?
Didi Zhu, Changrui Chen, Stefanos Zafeiriou et al. (+1 more)
When a
multimodal large
language model answers a visual
reasoning question correctly, is the prediction actually supported by the task-critical visual evidence? Correct answers can coexist with flawed
reasoning, making accuracy alone an incomplete test of grounding. We introduce VisualFLIP, a paired
benchmark with 1,374 images arranged as same-question perturbation pairs across cardinality, attribute, spatial, and logic tasks. Each pair keeps the question fixed but minimally changes the evidence so the gold answer deterministically flips. We evaluate 24 MLLMs with pair accuracy, which requires solving both sides of a pair, and Collapse Rate (CR), which measures how often a model that solves at least one side repeats the same non-empty answer for both images. Together, these metrics show that paired correctness and evidence dependence are related but distinct: capable models can still fail to update after task-critical visual changes, and collapse becomes more severe for some models when the edited image follows an earlier answer in a sequential setting. Further details are available on our project page: https://didizhu-judy.github.io/VisualFLIP/
Read the full abstract ▾
All authors Didi Zhu, Changrui Chen, Stefanos Zafeiriou, Jiankang Deng
Computer Vision
The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
Lujun Li, Lama Sleem, Niccolo Gentile et al. (+4 more)
Recent
vision-
language models (VLMs) excel at
multimodal understanding and
reasoning, yet their fine-grained visual perception remains underexplored. A natural extension of “How many r are there in Strawberry?” asks: how small a visual pattern can a VLM reliably perceive? As such, we introduce FineSightBench, a new
benchmark that systematically probes this limit by separating perception tasks (pixel-level recognition of letters, shapes, objects) from
reasoning tasks (spatial
reasoning, counting, ordering over small targets) across controlled scales of 4–48px. Through comprehensive experiments and detailed failure mode analysis on state-of-the-art models, we reveal a sharp dissociation: perception saturates around 12px, while
reasoning remains limited even at larger scales, with persistent numeracy and sequence errors. These findings expose fundamental deficiencies in VLMs' fine-scale visual
reasoning that demand more rigorous evaluation.
Read the full abstract ▾
All authors Lujun Li, Lama Sleem, Niccolo Gentile, Yangjie Xu, Yewei Song, Wenbo Wu, Radu State
Computer Vision · Cross-listed from cs.RO
MinNav: Minimalist Navigation Using Optical Flow For Active Tiny Aerial Robots
Aniket Patil, Mandeep Singh, Uday Girish Maradana et al. (+1 more)
Navigation using a
monocular camera is pivotal for autonomous operation on tiny aerial
robots due to their perfect balance of versatility, cost and accuracy. In this paper, we introduce MinNav, a
navigation stack based on optical flow and its uncertainty to fly through a scene with static and dynamic obstacles and unknown-shaped gaps without any prior knowledge of the scene components and/or their locations/ordering. We further improve success rate by using the activeness of the
robot to move around in an exploratory way to find obstacles and navigate. We successfully evaluate and demonstrate the proposed approach in many real-world experiments in various environments with static and dynamic obstacles and unknown-shaped gaps with an overall success rate of 70%. To the best of our knowledge, this is the first solution to tackle all the aforementioned
navigation cases without prior knowledge using a
monocular camera. Our approach is on par in performance with depth based methods with factors of magnitude less computation required and can readily run onboard tiny aerial
robots. The accompanying
video, supplementary material, code and
dataset can be found at https://pear.wpi.edu/research/minnav.html
Read the full abstract ▾
All authors Aniket Patil, Mandeep Singh, Uday Girish Maradana, Nitin J. Sanket
Computer Vision · Cross-listed from cs.GR
Frequency-Scale Saliency for Spectral Descriptor Analysis in 3D Shape Retrieval
Jianru Shen
Classical spectral descriptors such as the Heat Kernel Signature and Wave Kernel Signature are widely used for non-rigid 3D shape
retrieval, yet their failure modes remain poorly understood. We present a frequency-scale saliency framework that audits these descriptors by quantifying the
retrieval-level contribution of each descriptor scale interval through ablation. We introduce class spectral fingerprints to characterize category-level scale dependence, and show that descriptor similarity between class pairs is substantially correlated with
retrieval failure, with a Spearman correlation of 0.479. Experiments on SHREC'11 demonstrate that short scales dominate
retrieval performance while long scales are harmful, that HKS and WKS exhibit distinct scale dependence patterns, and that saliency-weighted
retrieval improves mAP on hard categories by 0.156, with cross-fold and random-weight controls confirming that the gain is stable and not due to arbitrary reweighting.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Land cover and flood type govern the detection limits of satellite-based flood mapping across diverse global flood events
Venkatesh Kolluru, Rajat Shinde, Abdelhak Marouane et al. (+6 more)
Floods are among the most destructive natural hazards, and their increasing frequency under
climate change makes satellite-based inundation mapping essential for disaster response. Geospatial foundation models pretrained on satellite archives offer geographic transferability, but their operational reliability across diverse, unseen events remains uncharacterized. Here we deploy Prithvi-EO-2.0 across 19 out-of-distribution flood events (2017-2025) spanning six continents, eight
climate zones, and six flood mechanisms, validating against two independent reference products. Detection accuracy depended jointly on land cover and flood type, with cropland yielding the highest agreement (IoU=52%) and riverine events the strongest detection (F1=0.69), while tree cover and built-up areas showed near-zero detection (IoU=4%) regardless of flood mechanism. Dual-reference validation revealed that apparent model error partly reflects definitional inconsistency between reference products rather than detection failure. Iterative pipeline testing identified 23 failure modes, with pipeline engineering dominating initial error over model capacity. These findings establish environment-dependent detection boundaries for operational satellite flood mapping.
Read the full abstract ▾
All authors Venkatesh Kolluru, Rajat Shinde, Abdelhak Marouane, Caden Helbling, Deepak Shah, Othneil Drew, Iksha Gurung, Manil Maskey, Rahul Ramachandran
Computer Vision
DALE-CT: Depth-Aware 2D Slice Encoders Learn an Anatomical World Model of Chest CT
Evan W. Damron, Mahmut S. Gokmen, Mitchell A. Klusty et al. (+3 more)
Chest CT is among the highest-volume imaging exams in medicine, yet expert voxel-level annotations are scarce and costly, motivating encoders that learn directly from unlabeled scans. We present DALE-CT, a family of 2D slice-based
Vision Transformers trained from scratch on chest CT with the heuristics-free LeJEPA objective. We introduce depth-aware slab sampling, which draws self-supervised views from across a physical $z$-axis slab rather than a single slice, implicitly tasking the 2D encoder with representing how anatomy changes between neighboring slices. The frozen representations trace each scan as a smooth anatomical trajectory, recover cranio-caudal slice ordering without labels, and distinguish slices by the anatomy they contain rather than by position alone. This anatomical world model emerges without any 3D or positional supervision, and an otherwise-identical encoder trained on slices in isolation never develops it. Building on this backbone, we introduce dense auxiliary supervision into the pretraining objective, using anatomical and abnormality masks to supervise patch and slice
tokens alongside the self-supervised loss, and we compare the resulting variants against a DINOv2 baseline continually pretrained on CT-RATE. Among nine public and in-house models evaluated under the same protocol, DALE-CT-2S is the strongest 2D model in-domain, reaching 0.825 Macro AUROC on CT-RATE, within 0.024 of COLIPRI-CRM and without any text supervision. We subsequently scale the supervision-free configuration to a $\sim$287k-scan multi-source pool, to our knowledge the largest reported chest-CT pretraining corpus. The resulting DALE-CT-0-L posts the best 2D external-transfer point estimates, and we release it as our recommended backbone with the full model family, training code, and evaluation pipeline.
Read the full abstract ▾
All authors Evan W. Damron, Mahmut S. Gokmen, Mitchell A. Klusty, Caroline N. Leach, Emily B. Collier, V. K. Cody Bumgardner
Computer Vision
Quantum-Enhanced Similarity Measures for Polarimetric Materials Classification
Sara Shojaei, Seyed Mohamad Ali Tousi, Emma Bennett et al. (+6 more)
We present a
quantum–classical hybrid pipeline for polarimetric material classification that casts this as a point-matching problem. Voxel cubes, containing polarized light reflections, are used to train an encoder to produce 32-dimensional
embeddings for the voxels of the cubes. At inference, the encoder head is discarded and the
embeddings are encoded as probability amplitudes of
quantum states. Next, a SWAP-test
circuit estimates the fidelity between each of the 32D
embeddings from the query cube and a
dataset of anchor cubes. The aggregated fidelity serves as materials similarity scores, and the class of the anchor with highest aggregated fidelity is deemed as the class of the queried material. We evaluate our approach on a
dataset of 23 materials ($\approx$800 samples each) derived from their Mueller
matrices. The point-matching approaches from the proposed
quantum SWAP-test and a classical classifier using Optimal Transport are compared. Our results demonstrate the competitive classification accuracy alongside open-set discrimination potential, establishing it as a viable path toward NISQ-based material recognition.
Read the full abstract ▾
All authors Sara Shojaei, Seyed Mohamad Ali Tousi, Emma Bennett, Param Sangani, Ali Shiri Sichani, Ilker Ersoy, Hadi Ali-Akbarpour, Filiz Bunyak, G. N. DeSouza
Computer Vision
DroneDAR: Long-Range Drone Distance Estimation Using Monocular Vision and Bounding-Box Features
Knut Peterson, Zaid Mayers and David Han
Accurate distance estimation for small drones in long-range imagery is important for tracking and situational awareness, yet remains challenging due to extreme target scale variation, background clutter, and noisy visual cues. This paper studies
monocular drone distance estimation using image crops together with bounding-box geometry, a practical setting in which a detector provides a candidate drone region and the model predicts range from appearance and box-derived features. We evaluate a Droneranger-style baseline, and introduce a new DroneDAR (Drone Detection And Ranging) model that combines a convolutional backbone with explicit bounding-box cues through a lightweight gating mechanism. Experiments analyze how backbone capacity, crop resolution, and regression loss functions affect performance across distance regimes. We further examine common failure modes at long distances, including sensitivity to bounding-box noise and reduced texture detail in the crop. The results provide guidance for designing and training range
estimators that remain robust under real-world long-range conditions and highlight directions for improving reliability when drones occupy only a few pixels.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
A case study of evaluating AI agents on a neuroscience data-to-discovery pipeline
Kai A. Horstmann, Ethan Lin, Alice A. Robie et al. (+2 more)
Agentic AI offers a promising path to automating software development bottlenecks in scientific research pipelines, particularly for stages that take domain experts days to months to build and where correctness and robustness matter more than implementation details. We present an empirical study of general-purpose coding
agents on a fly optogenetics data-to-discovery pipeline. We assess
agents on tasks and
datasets substantially larger than existing
benchmarks and evaluation criteria grounded in domain expert standards. We show that
agents can solve several pipeline stages, suggesting stage-level automation is tractable. By analyzing
agents' code iterations, we show they struggle most without a pre-defined criterion, when they must instead use their scientific judgment to assess their current solution. Mirroring scientists, they sometimes attempt visual inspection of intermediate outputs for self-evaluation, but largely fail to interpret what they see or act on it appropriately. Solving the end-to-end pipeline requires stringing together successes across all stages, which is beyond
agents' current abilities. We identify challenges largely absent from existing
benchmarks, including computational resource management and generalization to held-out data. Finally, we distill principles for constructing scientific tasks and rigorous evaluation criteria for open-ended problems.
Read the full abstract ▾
All authors Kai A. Horstmann, Ethan Lin, Alice A. Robie, Jennifer J. Sun, Kristin Branson
Computer Vision · Cross-listed from eess.IV
Multi-planar 2D-U-Net Segmentation of 3D-CT Abdominal Organs augmented by Spatial Occurrence Maps
Daria Kern, Negar Chabi, Souraj Adhikary et al. (+1 more)
This work proposes a lightweight 2D-U-Net-based framework for segmenting five abdominal organs in large field-of-view 3D CT scans. The method combines coarse-to-fine
segmentation, predictions from multiple anatomical planes, and additional fuzzy 3D spatial maps that provide anatomical location cues to improve
segmentation accuracy. We combine multi-planar 2D-U-Net models augmented by a spatial occurrence map. The approach involves two main stages. First, the abdominal volume of interest region is detected by traversing the whole scan axially with a 2D-U-Net and determining the x-y-z-minimum and -maximum extents of the 5 abdominal organs of interest. Second, we use spatial occurrence maps to enhance our multi-planar 2D-U-net architecture inside the bounds from the former stage. The method is evaluated on 80 CT scans from various public sources. The results show Dice improvements of about 4% at maximum compared to the same model trained without spatial occurrence maps.
Read the full abstract ▾
All authors Daria Kern, Negar Chabi, Souraj Adhikary, Andre Mastmeyer
Computer Vision
Cross-View Urban Traffic Dataset: Drone-Supervised Ground Truth for Monocular Bird's-Eye View Localization
Prakhar Bhardwaj, Simone Weikl, Kilian Mang et al. (+1 more)
We introduce a
dataset and
benchmark for cross-view urban traffic perception built from synchronized ego-centric bicycle
videos and aerial drone
videos recorded at real urban intersections. The
benchmark targets two linked tasks: cross-view identity matching between street-view and drone-view object tracks, and ego-to-bird's-eye-view prediction using aerial supervision. In contrast to prior urban driving and V2X
datasets, our
benchmark provides identity-level
alignment across radically different viewpoints together with standardized evaluation, annotation tooling, and baseline implementations. This setting is motivated by intersection-centric traffic analysis, where identity preservation, local interactions, and global spatial structure must be reasoned about jointly across views. We evaluate methods at both the track and frame levels, including cross-view ID precision/recall/IDF1, near–far breakdowns, temporal stability, and consistency metrics. We also provide baseline results for wedge-based cross-view matching and for three BEV prediction baselines: inverse perspective mapping, a MonoLayout-style learned baseline, and a regression baseline. The results show that the
benchmark is feasible but challenging: cross-view matching achieves strong recall yet remains limited by over-assignment and temporal inconsistency, while ego-to-BEV prediction benefits from aerial supervision but remains far from saturated under lightweight
monocular sensing. We hope that this
benchmark will support future research on cross-view perception, urban scene
alignment, and ego-to-global traffic understanding.
Read the full abstract ▾
All authors Prakhar Bhardwaj, Simone Weikl, Kilian Mang, Elia Jonas Sandtner
Computer Vision
Struct-Searcher: Agentic Structural Thinking Advances Multimodal Deep Information Seeking
Fan Zhang, Vireo Zhang, Shengju Qian et al. (+7 more)
Deep research
agents have attracted increasing
attention for their ability to collect large-scale online information to acquire target knowledge, with recent efforts shifting from purely text-based information seeking to
multimodal settings. However, existing agentic workflows are largely aligned with evidence accumulation models, which linearly aggregate evidence and lack principled mechanisms for handling contradictory information across heterogeneous modalities. Towards this end, we propose Struct-Searcher, a structural agentic workflow grounded in belief revision theory that explicitly maintains an evolving
multimodal structural graph throughout the
reasoning process, enabling effective conflict-aware
multimodal deep information seeking. Extensive experiments across multiple
benchmark datasets and backbone models demonstrate that Struct-Searcher is (1) plug-and-play and model-agnostic, yielding an average relative accuracy improvement of 17.2% on BrowseComp-VL across five different backbones. (2) top-performing, consistently outperforming state-of-the-art
vision-
language models (VLMs) and deep research
agents, with relative accuracy improvements of 3.7% on MM-BrowseComp, 1.5% on HLE-VL, and 0.7% on BrowseComp-VL over the second-best competing approach.
Read the full abstract ▾
All authors Fan Zhang, Vireo Zhang, Shengju Qian, Haoxuan Li, Zheng Lian, Hao Wu, Yuan Gao, Xinyu Geng, Xin Wang, Pheng-Ann Heng
Computer Vision
What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction
Jewon Yeom, Hanseul Kim, Jeongjae Park et al. (+3 more)
Video world models are increasingly used to provide predictive visual representations, yet it remains unclear which pretraining signals induce action-relevant structure in their latent spaces. We study this question through a unified probe-based evaluation across diverse encoder families, including image-only self-supervision,
video pretraining with and without latent prediction, reconstruction-based autoencoders,
diffusion models, and shortcut-forcing dynamics models. Using a common inverse-dynamics probing objective, we find that action-relevant structure is driven primarily by temporal
video pretraining rather than pixel reconstruction fidelity: models with strong pixel decoding quality can exhibit near-zero action recoverability, while
video-pretrained self-supervised encoders consistently achieve the best Pareto trade-off between visual fidelity and action prediction. Comparing V-JEPA and VideoMAE further shows that most gains arise from natural-
video temporal context, with feature-level latent prediction providing a smaller additional benefit. These trends transfer across robotic
benchmarks, though CALVIN reveals that static-environment tasks can partially mask the importance of temporal structure by allowing strong image priors to suffice. Finally, inverse-dynamics supervision substantially improves robustness to visual corruption, suggesting that action-aware objectives regularize latent geometry beyond clean-setting performance. Our results identify temporal predictive structure – not reconstruction fidelity – as the primary ingredient underlying action-relevant
video representations.
Read the full abstract ▾
All authors Jewon Yeom, Hanseul Kim, Jeongjae Park, Sungmok Jung, Jaejin Lee, Taesup Kim