Computer Vision
WADE: A Reasoning-Annotated Benchmark for Multi-Instance Floating-Waste Grounding with Compact Vision-Language Models
Md. Asaduzzaman Shuvo, Ahsan Farabi, Md. Abdul Ahad Minhaz et al. (+4 more)
Floating waste in inland waterways threatens aquatic ecosystems and requires timely monitoring under cluttered, multi-object conditions. Existing aquatic-waste
datasets provide limited geographic coverage, sparse multi-instance annotations, and little supervision beyond boxes and labels. Compact
vision-
language models (VLMs) therefore remain insufficiently evaluated for jointly localizing, classifying, counting, and explaining floating waste. We introduce WADE, a
reasoning-annotated
benchmark containing 2,167 images from rural Bangladesh, 13,608 bounding boxes, and ten waste categories. Each annotation is associated with class-level recognition rules covering visual cues, likely confusions, and discriminative features. We evaluate six VLMs under zero-shot, two-shot,
reasoning-guided, and fine-tuned settings using detection, counting, and
hallucination metrics. For resource-efficient adaptation, we jointly fine-tune Qwen3-VL-2B on boxes, labels, and
reasoning chains using QLoRA. Fine-tuning increases recall from 0.0248 to 0.2339 and F1 from 0.0257 to 0.2163, while reducing image-level
hallucination from 0.6836 to 0.0883. However, over three-quarters of instances remain undetected, establishing WADE as a challenging
benchmark for dense floating-waste grounding with compact VLMs.
Read the full abstract ▾
All authors Md. Asaduzzaman Shuvo, Ahsan Farabi, Md. Abdul Ahad Minhaz, Mahedi Hasan, Israt Khandaker, Ibrahim Khalil Shanto, Muhammad Nomani Kabir
Computer Vision
Quality Inspection of Printed Circuit Board Pin Insertion via Semantic Segmentation and Board-Level Feature Extraction
Nils Rabeneck, André Kiunke, Nicole Hoess et al. (+1 more)
Quality control during printed
circuit board (PCB) assembly is a critical step in ensuring reliable electronic products. Detecting misaligned pins during or after pin insertion remains a particularly challenging inspection task. This paper presents an automated defect detection method for identifying incorrectly inserted pins on PCBs. The proposed pipeline combines semantic
segmentation using a U-Net architecture with contour-based feature extraction and logistic regression for board-level pass/fail classification.
Segmentation masks are used to derive contour representations of individual pins, from which board-level features -such as average contour size- are extracted and used to train a logistic regression classifier. We evaluate the method on two
datasets: an industrial collection of real-world PCB images, and a publicly available PCB pin-inspection
dataset with substantially different visual characteristics. To assess the effectiveness of the proposed approach, a comparison against PatchCore, an anomaly detection technique new to be applied to pin inspection, as well as instance
segmentation-based pin detection is made. The developed method achieved Area Under the Receiver Operating Characteristic Curve (ROC-AUC) values of 0.990 on a random test set split from the industrial data and 1.000 on the public
dataset indicating strong separation between pass and fail boards. The results indicate that the proposed approach is a promising candidate for automated pin inspection in industrial environments and achieves strong performance on
datasets with substantially different visual characteristics after
dataset-specific training.
Read the full abstract ▾
All authors Nils Rabeneck, André Kiunke, Nicole Hoess, Wolfgang Mauerer
Computer Vision
Motion-Based Tokenization for Cross-Dataset Egocentric Gaze Modeling
Virmarie Maquiling, Zhuojiang Cai and Enkelejda Kasneci
Gaze is increasingly used as an input signal for
vision and
multimodal models, yet no consensus exists on how to represent it across
datasets. Raw traces preserve detail but are noisy and device-dependent, while coarse event labels are easy to model but can discard local motion structure. We formulate event-aligned, fixed-horizon angular displacement as an interpretable, event-conditioned motion vocabulary and compare it with event-only, spatial, absolute-angle, learned vector-quantized, and continuous representations. To assess transfer alongside target predictability and
token collapse, our evaluation combines next-
token prediction with target-domain regret, low-order target references, paired bootstrap, order sensitivity, motif overlap, and frozen structural probes. In an event-aligned headset
benchmark, angular-motion
tokens have lower target-domain regret than frozen-codebook VQ
tokens in one transfer direction, while the reverse direction is inconclusive. The probes reveal complementary representation properties, and event-only
tokens show that low perplexity can retain little motion information. On a third
egocentric dataset, a matched comparison of I-VT, native, and frame-span interfaces shows that event construction materially changes transfer: native events have the lowest regret into EGTEA, while frame-span events have zero motif overlap and fail severely as a source. Motion-based tokenization therefore provides a compact representation for event-aligned
egocentric gaze streams, while the evaluation identifies how target predictability and event construction shape cross-
dataset conclusions.
Read the full abstract ▾
Computer Vision
Results of the 1st Asynchronous CASTLE Challenge at the Joint Egocentric Vision Workshop in Conjunction with CVPR 2026
Luca Rossetto, Werner Bailer, Cathal Gurrin et al. (+5 more)
This report summarizes the contributions and results of the 1st Asynchronous CASTLE Challenge at the Joint
Egocentric Vision Workshop in conjunction with CVPR 2026.
Read the full abstract ▾
All authors Luca Rossetto, Werner Bailer, Cathal Gurrin, Graham Healy, Omar Shahbaz Khan, Stevan Rudinac, Klaus Schöffmann, Allie Tran
Computer Vision
AquaFlow: A Monocular Gaussian Splatting SLAM for Underwater Streaming Reconstruction
Yingxiang Xu, Kerui Ren, Wenqi Guo et al. (+4 more)
Recent
monocular 3D Gaussian Splatting (3DGS) streaming reconstruction methods have achieved impressive performance by balancing reconstruction quality and efficiency. However, extending these frameworks to
underwater scenes remains challenging due to severe visual degradation, such as light attenuation and scattering, which degrades camera pose tracking and distorts scene geometry. To address these challenges, we propose AquaFlow, a
monocular Gaussian Splatting streaming reconstruction framework for efficient and high-fidelity
underwater reconstruction. Specifically, AquaFlow fine-tunes a 3D
vision foundation model on large-scale
underwater data for robust pose and pointmap estimation, and introduces a medium-guided incremental Gaussian initialization strategy for streaming mapping. Furthermore, we develop a streaming-compatible hybrid scene representation that integrates structured, distance-conditioned neural Gaussians with a physics-inspired optical model to compensate for
underwater image formation effects, enabling accurate scene reconstruction. We evaluate AquaFlow on a comprehensive
dataset of 62 diverse
underwater trajectories, collected from both public
benchmarks and in-the-wild web
videos across various scales. Extensive experiments demonstrate that AquaFlow achieves state-of-the-art tracking and rendering performance, reducing average localization error by 13.2% and improving PSNR by 4.74 dB compared to WaterSplat-SLAM.
Read the full abstract ▾
All authors Yingxiang Xu, Kerui Ren, Wenqi Guo, Changjian Jiang, Tao Lu, Linning Xu, Mulin Yu
Computer Vision
NemoSplat: Feed-Forward 4D Gaussian Splatting for Media-Aware Underwater Reconstruction
Xiaopeng Guo, Wai Chung Tse, Yipeng Zhu et al. (+3 more)
Reconstructing photorealistic scenes in unconstrained
underwater environments remains challenging due to severe media-induced light scattering and unpredictable dynamic objects. Recent feed-forward visual foundation models have demonstrated remarkable capabilities in generalized novel view synthesis and tracking. However, when directly applied to aquatic
videos, optical attenuation and motion interference fatally corrupt their feature aggregation, leading to severe tracking and reconstruction failures. To overcome these limitations, we present NemoSplat, the first feed-forward 4D Gaussian Splatting framework tailored for media-aware dynamic reconstruction directly from uncalibrated marine
videos. Beyond providing robust estimations of camera poses and dense scene depth, we devise a Promptable Dynamic Disentangler that utilizes a confidence-aware fusion strategy of learned dynamic probabilities and optional semantic text priors, effectively isolating massive transient entities. Furthermore, to counteract visual degradation, a Media-Aware Gaussian Predictor is formulated to jointly estimate intrinsic 3D Gaussian attributes alongside physical media parameters, rendering pristine scene appearance in a single forward pass. Additionally, we introduce a large-scale
underwater dataset with massive dynamic elements to facilitate training and evaluation. Extensive experiments on our
dataset demonstrate that NemoSplat achieves state-of-the-art tracking accuracy and high-fidelity rendering. Homepage: https://nemosplat.hkustvgd.com
Read the full abstract ▾
All authors Xiaopeng Guo, Wai Chung Tse, Yipeng Zhu, Hanwen Zhang, Huajian Huang, Sai-Kit Yeung
Computer Vision
Yujie Qi and Luyan Zhang
Referring Expression
Segmentation (RES) aims to generate a pixel-level mask for the object specified by a
language expression. Recent methods based on
multimodal large
language models (MLLMs) often rely on one-pass coordinate prediction for visual localization, which serializes continuous spatial locations as discrete text
tokens and may lead to localization bias and
alignment errors. To address these issues, we propose DRAgent, an MLLM-driven discriminative
reasoning (DR) framework for RES. Instead of requiring the MLLM to generate localization coordinates, DRAgent first constructs a detector-generated candidate space and then uses the MLLM as a visual-semantic target discriminator. Specifically, the MLLM performs reliable target selection among potential distractors through a two-stage DR mechanism, which first screens high-recall candidates and then performs instance-wise verification. The selected target box is subsequently used as a spatial
prompt for a foundation
segmentation model to produce the final pixel-level mask. Furthermore, we construct a self-consistency-filtered
reasoning-chain data pipeline for LoRA-based fine-tuning, providing more reliable supervision for enhancing the MLLM's discriminative
reasoning capability. Experiments demonstrate that DRAgent achieves competitive performance on RefCOCO, RefCOCO+, and RefCOCOg.
Read the full abstract ▾
Computer Vision
FOVEA: Focused On-Demand Visual Evidence Adaptation for Cache-Friendly Multimodal Speculative Decoding
Hengjie Zhu, Dayan Wu, Zihao Zhang et al. (+6 more)
Multimodal speculative decoding accelerates
vision-
language models by allowing a lightweight draft model to propose candidate
tokens for parallel verification by a larger target model. Existing methods typically condition the drafter on a fixed visual interface, such as a predefined visual-
token budget or a static compressed representation. However, our controlled visual-budget analysis shows that visual demand varies substantially across tasks and decoding stages, which means more visual input is not always beneficial. Actually, insufficient evidence may weaken visual grounding, while excessive context adds overhead and may disrupt drafting. We propose FOVEA (Focused On-demand Visual Evidence Adaptation), a cache-friendly approach that builds a reusable visual
memory and dynamically retrieves a bounded subset for a draft state. A cumulative-mass rule determines both how many and which entries are selected. The selected entries are aggregated into a visual readout and fused with the current draft hidden state through a lightweight gated residual correction. Rather than inserting visual
tokens into the autoregressive context, the correction modifies only the representation passed to the
language-model head. Experiments across multiple
vision-
language backbones and
multimodal benchmarks show that FOVEA improves draft acceptance and end-to-end decoding speed, achieving up to $2.13\times$ speedup over autoregressive decoding. These results demonstrate that state-conditioned evidence
retrieval is an effective alternative to reusing a fixed visual representation throughout
multimodal generation.
Read the full abstract ▾
All authors Hengjie Zhu, Dayan Wu, Zihao Zhang, Xinze Liu, Jingxuan Yu, Peng Fu, Zheng Lin, Weiping Wang, Ding Wang
Computer Vision
Large-Small Model Collaboration for Zero-Shot Surgical Phase Recognition
Yiyi Zhang, Ying Zheng, Wenxin Fan et al. (+5 more)
Task-specific lightweight models for
surgical phase recognition excel at capturing temporal dynamics but generalize poorly under domain shift. Conversely,
surgical foundation models (FMs) offer superior transferability via large-scale pretraining, yet their lack of explicit temporal modeling often yields temporally inconsistent predictions, leading to degraded performance. To exploit the complementary strengths of both paradigms, we propose Large-Small Temporal adaptation (LaST), a novel large-small collaborative framework that enables zero-shot adaptation to unseen clinical domains. In LaST, the FM initiates the pipeline by generating frame-level phase priors that serve as initial weak supervision. To effectively utilize these noisy phase priors, we introduce an iterative temporal refinement scheme that integrates dynamic quality control to filter reliable predictions and dual-model cross-learning to mitigate confirmation bias. Simultaneously, the lightweight model leverages its intrinsic temporal modeling ability to progressively correct inconsistent predictions and enhance overall accuracy across iterations. At the end, a cycle replay strategy is employed to close the loop: the refined, more accurate predictions are utilized as upgraded supervision signals for the subsequent iterations, fostering a self-reinforcing
evolution of both label quality and model capability. Extensive experiments demonstrate that LaST achieves robust adaptation to unseen domains for zero-shot
surgical phase recognition, outperforming the baseline (PeskaVLP) by 24.85%-43.17% in accuracy and even surpassing fully supervised linear probing and several state-of-the-art few-shot approaches. Codes will be released at https://github.com/YIYIZH/LaST.
Read the full abstract ▾
All authors Yiyi Zhang, Ying Zheng, Wenxin Fan, Yu Zhu, Yuchen Yuan, Litao Zhao, Zheng Li, Pheng-Ann Heng
Computer Vision · Cross-listed from cs.LG
Liqing Zhang
Neurobiological studies and representation learning have observed that representations of objects belonging to the same category in high-dimensional neural spaces exhibit low-dimensional object
manifold characteristics, and different object
manifolds are linearly separable in these neural spaces. However, these experimentally observed phenomena lack rigorous theoretical validation to date. This paper proposes a new
stochastic separability
theorem for
embedding manifolds of two different object categories. First, we establish a projection measure concentration
theorem for
embedding manifolds under general conditions. We develop a new two-layer measure concentration analysis technique, which unifies two estimation bounds via the law of total expectation to derive measure concentration inequalities. Based on the measure concentration
theorem, we further prove a
stochastic separability
theorem for
embedding manifolds of two different object categories. If two
datasets have distinct means and bounded total variances, their samples become linearly separable with high probability, provided that the projection direction satisfies a non-singularity condition. The main contributions of this paper are twofold: 1. We prove the projection concentration properties of
embedding manifolds in high-dimensional spaces by using two-lawyer tail-bound inequalities. 2. We identify a non-singularity condition for the
stochastic separability between
embedding manifolds, and rigorously prove the
stochastic projection separability
theorem. The
theorem not only uncovers
geometric and statistical properties of the object
embedding manifolds, but also provides a novel mechanism for representation learning in deep networks.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
UniMem: Unifying Visual and Textual Memory in VLAs for Non-Markovian Tasks
Lars Osterberg, Maggie Wang and Mac Schwager
While
Vision-
Language-Action (VLA) models have leveraged internet-scale pretraining and task-specific finetuning to achieve strong performance on long-horizon tasks, they often struggle with non-Markovian tasks that require
memory. Existing approaches to
memory typically involve additional
Vision-
Language-Models (VLMs) for long-term
memory management, introducing a
memory bottleneck and a fractured training pipeline. Conditioning on multiple historical frames can provide the VLA with access to more descriptive features of past scenes, but can degrade performance if frames are chosen at arbitrary, fixed intervals. To address these limitations, we present UniMem, a framework that unifies high-level visual and textual
memory and low-level control under one backbone. UniMem employs an event classifier to initiate textual and visual
memory updates, a keyframe encoder for dense visual
memory, and a keyframe caching technique to minimize overhead during policy rollouts. We evaluate UniMem across five simulation and four hardware tasks targeting sequential and spatial
memory, demonstrating that our unified, single-model system outperforms fixed-interval image sampling baselines (93.4% vs. 68.2%) in simulation and hierarchical VLM-VLA baselines (80.0% vs. 43.5%) in hardware, while offering faster inference and a simple training pipeline for easy adoption. Project website: https://losterberg3.github.io/unimem-vla/
Read the full abstract ▾
Computer Vision
Toward Sub-1 kB Identity-Preserving Face Compression: A Benchmark of Codecs, a Custom Learned Codec, and Studies of Resolution, Demographic Fairness, Recompression, and Adversarial Robustness
Petr Hurtik and Jakub Sochor
Storing face images under a hard sub-kilobyte budget, as required for identity documents, smart-card biometrics and bandwidth-constrained verification, forces a codec to discard most of the signal while keeping what a face matcher actually reads: identity. Generic codecs optimize pixel fidelity, not the
embedding distances that drive verification, so which codec, resolution and setting best preserve identity at 1024 bytes or less, and how that degrades at 512, is unclear. We
benchmark ten general and face-specific codecs across resolutions, byte budgets, two
datasets (controlled Color FERET, in-the-wild AI-Solutions-KK) and four anchor face matchers, with a fourteen-model ViT and CNN roster confirming the ranking is backbone-invariant. We then train a custom identity-preserving codec that hits the byte budget exactly via binary search over a frozen gain table, and run four studies: resolution, demographic
fairness, recompression, and no-box
adversarial robustness. Sub-kilobyte identity preservation is feasible, but which codec to deploy depends entirely on the budget. At 1024 bytes and the 112 px working resolution the problem is close to solved: modern codecs hold Color FERET equal-error rate under 0.35 percent on the ArcFace anchor. At 512 bytes the field re-sorts: AVIF, HEIF, JPEG XL and legacy JPEG collapse to 28 to 98 percent false-non-match rate at FMR 1e-4, while WebP, JPEG-AI and our byte-budgeted learned codecs stay out of that band, with 24.3 percent for WebP against 6.9 percent for our accurate variant in the wild. That re-sort, not the 1024-byte ranking, is the operational result: a codec chosen at 1 kB is not the codec to deploy at half that.
Read the full abstract ▾
Computer Vision
Following Motion for Sequential Modeling in Video Frame Interpolation
Jaehyun Park and Nam Ik Cho
State Space Models (SSMs) have surfaced as a promising architecture in
Video Frame Interpolation (VFI), as they can capture long-range dependencies with linear computational complexity. However, their predefined scanning order limits their effectiveness in modeling the dynamic motion trajectories inherent in VFI problems. To tackle this challenge, we propose Motion-Guided Mamba for
Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI. MGMVFI introduces Motion-Guided Serialization (MGS), which leverages optical flow to define a motion-adaptive 1D input order for the SSM. This aligns the causal state updates with semantically related
tokens, enabling motion-consistent feature propagation, particularly for large and dynamic motions. Additionally, to mitigate the unreliable feature representations caused by inaccurate optical flow estimates, we introduce contextual synthesis that utilizes the surrounding spatial context for robust inter-frame feature synthesis. These components are seamlessly integrated within our tailored Mamba architecture, which also employs a lightweight refinement block to enhance local detail reconstruction at a reduced computational cost. Extensive experiments on standard VFI
benchmarks demonstrate that MGMVFI achievesstate-of-the-artperformance,particularly on complex and dynamic motions, thereby establishing a new direction for sequence modeling in
video interpolation.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Mapping the Concept Landscape: Structural Perception of Global Distributions for Transparent Data Pruning
Dongyue Wu and Tao Ma
Existing data
pruning methods predominantly rely on high-dimensional feature
embeddings to measure sample importance. However, these compressed vectors often obscure fine-grained semantic interactions, leading to suboptimal coverage of rare semantic concepts in the pruned subsets. In this paper, we propose Mapping the Concept Landscape (MCL), a novel structural perception framework for transparent data
pruning. Instead of abstract
embeddings, we represent each image-caption pair as an explicit sample-level graph comprising entities, events, and attributes. By integrating these individual graphs into a comprehensive
dataset-level graph, we characterize the global distribution of semantic concepts and quantify their rarity across the entire corpus. Based on this structured perception, we develop a greedy concept-coverage maximization
algorithm that iteratively selects samples to maximize the marginal gain of high-value, under-represented concepts. Experimental results on various
benchmarks demonstrate that our method not only achieves superior
pruning efficiency compared to state-of-the-art methods but also provides a transparent and interpretable audit trail for the selection process.
Read the full abstract ▾
Computer Vision
VeCAS: Vessel-Focused Contrast-Free Angiogram Synthesis for Vascular Interventions
De-Xing Huang, Chen-Yu Wang, Hao Liang et al. (+10 more)
X-ray angiography relies on iodinated contrast
agents to visualize vascular structures during image-guided interventions. However, contrast administration carries risks of adverse events, motivating the development of contrast-free alternatives. Generating X-ray angiograms directly from non-contrast X-ray images offers a potential solution, but existing approaches remain limited by (i) insufficient control over vascular localization and (ii) inefficient modeling of redundant background content. To address these challenges, we propose VeCAS, a two-stage vessel-focused contrast-free angiogram synthesis framework that separates vascular structure localization from angiographic appearance synthesis. In Stage I, a discriminative model localizes vascular structures in non-contrast X-ray images, while cross-modality latent
distillation transfers vessel-sensitive knowledge from X-ray angiograms during training. In Stage II, a vessel-focused inpainting model synthesizes angiographic appearance within the localized vascular regions while preserving the non-vascular background. Experiments on an in-house lower-limb vascular intervention
dataset show that VeCAS outperforms the comparison methods in terms of vascular structural fidelity and image quality. Visual Turing tests and physician assessments indicate the perceptual realism of the synthesized angiograms. In addition, robotic guidewire
navigation experiments in vascular phantoms show that VeCAS guidance reduces the time to target by 41.4% and the number of operation steps by 40.7% compared with non-contrast guidance. Together, these results suggest the potential of VeCAS to serve as “meta contrast
agent” for vascular interventions.
Read the full abstract ▾
All authors De-Xing Huang, Chen-Yu Wang, Hao Liang, Xiao-Hu Zhou, Mei-Jiang Gui, Tian-Yu Xiang, Qin-Yi Zhang, Chen Wang, Xiao-Liang Xie, Shi-Qi Liu, Ming-Yuan Liu, Zhen-Chang Wang, Zeng-Guang Hou
Computer Vision
SiZeUp: Fast 3D Proxy from Aerial Images via Depth Ordinal Loss
Wenjun Zhou, Yunshan Li, Qiaoyu Zhu et al. (+4 more)
We present SiZeUp, a fast and scalable approach for constructing large-scale 3D urban proxy models directly from calibrated oblique aerial imagery. Our method adopts a height-from-footprint representation, reducing 3D building abstraction to a low-dimensional optimization problem in which building footprints are extruded by a single height parameter. To enable efficient and robust height estimation, we introduce an ordinal depth consistency loss that enforces agreement between the relative depth ordering of rendered proxies and depth priors predicted by a
monocular depth model. This is realized through a differentiable renderer that maps parametric building proxies into multi-view depth images, allowing gradients to be propagated from depth supervision to building heights. Our ordinal formulation produces stable optimization in practice and avoids explicit feature matching or dense point cloud reconstruction. Rather than relying on metric depth, which can be unreliable under
monocular scale ambiguity, our ordinal depth consistency loss operates on relative depths, providing a more reliable signal across views. Combined with an efficient dynamic view selection, our approach achieves a 23-52$\times$ speedup over state-of-the-art proxy reconstruction pipelines while maintaining comparable proxy-level coverage and volume consistency, making it well suited for large-scale urban modeling tasks.
Read the full abstract ▾
All authors Wenjun Zhou, Yunshan Li, Qiaoyu Zhu, Weidan Xiong, Hao Zhang, Daniel Cohen-Or, Hui Huang
Computer Vision
Direct, Parallel, or Sequential? A Comparative Study of Training-Free Multi-Subject Image-to-Video Generation
Yanliang Qi, Kexi Chen, Muchao Ye et al. (+1 more)
Text-conditioned image-to-
video (I2V) generation has advanced rapidly, yet generating
videos with multiple subjects remains challenging. A model must simultaneously preserve the appearance of each subject, assign distinct motions, and maintain coherent spatial and temporal interactions. This paper presents a systematic study of three representative paradigms for training-free multi-subject I2V generation: direct, parallel, and sequential generation. Direct generation applies a pretrained I2V model to the complete reference image and
prompt, requiring all subjects and motions to be synthesized jointly. Parallel and sequential generation instead decompose the reference image and
prompt into subject-specific visual and textual conditions. Parallel generation synthesizes each subject independently and subsequently composes the resulting
videos, reducing the complexity of each generation step at the cost of weaker inter-subject context. Sequential generation first synthesizes a background
video and then progressively introduces individual subjects. This preserves accumulated scene context but introduces sensitivity to subject ordering and error propagation. We empirically evaluate the three paradigms across diverse multi-subject scenes, comparing appearance preservation, motion fidelity, temporal consistency, and inter-subject coherence, while also characterizing their distinct failure modes. Our findings reveal the strengths and limitations of each paradigm and offer practical insights for designing controllable multi-subject
video generation systems.
Read the full abstract ▾
All authors Yanliang Qi, Kexi Chen, Muchao Ye, Haomiao Ni
Computer Vision · Cross-listed from cs.RO
Reproducible Vision-Guided 6-DoF Robotic Manipulator with a Mixed Stepper-Driver Architecture and Browser-Native Control
Lasan Perera, Deneth Priyadarshana, Dulana Pitiwaduge et al. (+2 more)
We present the NeuralNexus Arm, an open, low-cost 6-DOF robotic manipulator built by an undergraduate engineering team, together with the design decisions and debugging experience needed to reproduce it. The arm is driven by a single STM32H743 microcontroller on a custom printed
circuit board (PCB) and combines two stepper-driver strategies on one controller: push-pull 3.3 V step/direction outputs for onboard TMC2209 drivers on the three wrist joints, and open-drain outputs for external CL57T and DM542 drivers on the three high-torque proximal joints. We describe the mechanical design, mixed-driver electronics, interrupt-driven firmware, a MATLAB/Simscape-based inverse-kinematics pipeline, a browser-native control interface using the Web Serial API, and a lightweight
vision pipeline for object localisation and autonomous pick-and-place tasks. We also document non-obvious hardware and firmware failure modes encountered during the transition from a development board to the custom PCB as reproducibility guidance. All design files and firmware are released openly. The platform actuates all six axes under coordinated control at a 2 kHz update rate and executes both manual and pre-recorded motions from the browser interface.
Read the full abstract ▾
All authors Lasan Perera, Deneth Priyadarshana, Dulana Pitiwaduge, Isitha Dinujaya, Mokshan Colambage
Computer Vision
VersaDB: A High-Performance AI Storage Database for Unifying Mutimodal Datasets
Cong Wang, Zelin Liu, Yang Luo Ran Zhang et al. (+6 more)
The AI field has been rapidly developing, leading to the emergence of a large number of AI training
datasets of various types. These
datasets contain different modalities, including text, images, audio, etc., and may come in various data storage formats. With the advancement of AI hardware, AI computation units like GPUs, TPUs, and NPUs can greatly accelerate the training speed of AI models, which in turn increases the demand for faster data processing. When using existing AI processing frameworks to handle
datasets with different modalities and storage formats, processing speeds may be suboptimal due to issues such as data layout and the way users handle the data. Therefore, using a unified database to store multiple data formats can better manage and optimize data access. In this paper, we introduce VersaDB, a database designed specifically for AI
datasets with various modalities. We implemented a page-based storage system, separating structured and unstructured data. Additionally, we generated B+ tree-based index files to accelerate data access. VersaDB supports automatic sharding and maintains a hierarchical metadata management system, with corresponding metadata maintained at the page, shard, and global levels, forming the foundation for the efficient operation of the database. We also focused on ease of use by providing APIs for directly converting
datasets into VersaDB, as well as APIs for converting popular AI data storage formats (e.g., CSV, TFRecord, .bin) into VersaDB.Our experiments show that using VersaDB can achieve up to 5.35x acceleration and maintain consistent performance across different parallelism levels.
Read the full abstract ▾
All authors Cong Wang, Zelin Liu, Yang Luo Ran Zhang, Zhijian Guo, Hui Zhang, Fan Yu, Yanfei Cao, Naijie Gu, Jun Yu
Computer Vision
GuidedFlow: An Attention-Guided Framework for Anomaly Detection in Additive Manufacturing
Sosmita Paul and Krishna Roy
Additive Manufacturing (AM) plays a vital role in the ongoing industrial revolution. However, quality control remains crucial and challenging due to printing defects or potential cyber-physical intrusions. Image or
video-based anomaly detection is a key effort towards addressing these challenges. Various approaches have been explored in this domain, including reconstruction-based,
embedding-based, and flow-based methods. Though normalizing flow-based methods address some of the core challenges of unforeseen defects and generalization while maintaining detection performance, existing approaches struggle with tiny/stringing defects common in 3D printing. In a small-data setting, this poses a limitation in generalization. To address these limitations, we propose GuidedFlow, a novel
attention-guided normalizing flow model for anomaly detection and localization. GuidedFlow employs a pre-trained ResNet model, fine-tuned on the domain
dataset. An
attention-guided spatial and temporal flow framework models the dynamics across multiple scales and frames. A Spatio-Temporal
Attention Network (SAN) enables the flow model to prioritize relevant contextual cues from input frames. We evaluate GuidedFlow on our AM3D-AD
dataset, consisting of benign and anomalous real 3D printed object images and
videos. We also conduct a comparative study using the MVTec-AD industrial image anomaly detection
dataset. Experimental results demonstrate that GuidedFlow outperforms most of the state-of-the-art models with enhanced detection accuracy and AUROC.
Read the full abstract ▾
Computer Vision
OmicSync: Reliability-Aware Spatial Multi-Omics Clustering with Evidence-Constrained LLM Reasoning
Rabeya Tus Sadia, Qiang Ye and Qiang Cheng
Spatial multi-omics technologies jointly profile gene expression, surface
proteins, and histology at each tissue spot, yet most spatial domain discovery methods provide only cluster assignments, without indicating assignment reliability, modality contributions, or why a domain decision should be trusted. We present OmicSync, a reliability-aware spatial multi-omics framework that couples unsupervised domain clustering with evidence-constrained LLM
reasoning using model-derived per-spot signals, including assignment confidence, epistemic routing uncertainty, and modality-routing weights. These signals are converted into structured evidence dictionaries and used to generate standard, stepwise, counterfactual, contrastive, and uncertainty-focused explanations. OmicSync integrates a KAN-GCN backbone with spatial encoding, cross-modal fusion, uncertainty-aware routing,
cell-type supervision, and missing-modality imputation. We further introduce OmicSync-R, which closes the
reasoning-clustering loop by using automatically computed
reasoning-quality scores as REINFORCE rewards, allowing
reasoning coherence to shape the latent structure without backpropagating through the
language model. Across four 10x CytAssist FFPE spatial proteomics
benchmarks, OmicSync achieves the best average rank on Human Tonsil (1.44), Glioblastoma (1.78), and Tonsil Add-on (1.22), and second-best on Human Breast Cancer (2.33). OmicSync-R further improves ARI on Human Breast Cancer from 45.73 to 46.72 and outperforms existing methods on six of nine clustering metrics. Together, OmicSync and OmicSync-R enable reliability-aware, spot-level auditable spatial domain discovery guided by evidence-constrained
reasoning.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Neural Operator based Multi-Field Reconstruction of Inner Solar Boundary State
Vignesh Kumar Pandian Sathia, Reza Mansouri, Dustin J. Kempton et al. (+2 more)
The
Solar wind is a continuous flow of charged particles emanating from the
solar surface and governed by complex, interacting magnetohydrodynamic processes. Accurate specification of inner-boundary conditions is essential for heliospheric modeling and
solar-wind prediction. In many practical applications, only a subset of interacting multi-field variables is directly available, but for a comprehensive view of
solar wind prediction and downstream magnetohydrodynamic simulations, a more complete boundary state is required. In this work, we study the problem of learning the multi-field multi-scale
solar magnetohydrodynamic state at 30
solar radii ($R_\odot$) using operator learning. Specifically, given the radial velocity and radial
magnetic field, we aim to reconstruct the non-radial velocity and
magnetic field components, radial and non-radial current density, thermodynamic density, and pressure components. This mapping is highly nonlinear, spatially coupled, and multi-scale, making it a challenging task for data-driven scientific machine learning. To address this problem, we employ a Local Neural Operator (LocalNO) that learns mappings between input and output function spaces while retaining locality and resolution-awareness. Unlike conventional regression models and autoencoder models, neural operators are better suited for learning structured field-to-field transformations arising from physical systems. The resulting predictions along with inputs are intended to serve as boundary condition variables for future inner-heliospheric modeling pipelines.
Read the full abstract ▾
All authors Vignesh Kumar Pandian Sathia, Reza Mansouri, Dustin J. Kempton, Pete Riley, Rafal A. Angryk
Computer Vision
Can We Perform Online RL for Image Editing without Editing Rewards?
Qichao Ma, Jikang Cheng, Ling Liang et al. (+3 more)
Reinforcement learning (RL) enables direct preference optimization for image editing through editing-specific rewards, which remain less developed due to costly triplet supervision and complex task-dependent calibration. In contrast, text-to-image (T2I) generation benefits from a mature and diverse reward ecosystem spanning semantic
alignment, aesthetics, realism, glyph shape, and other visual preferences. Extending this ecosystem to image editing would substantially broaden the range of visual preferences accessible to RL-based optimization, prompting the central question: Can We Perform Image Editing RL without Editing Rewards? In this paper, we argue that the standard image editing dimensions have potential to be mapped to the T2I reward space: image quality can transfer directly,
prompt following can be aligned through a description of the desired visual state, and reference consistency admits a coarse semantic conversion by encoding the source content to preserve. However, editing instructions specify relative changes, whereas T2I rewards require self-contained target descriptions; moreover, semantically valid captions from generic
vision-
language models may be incompatible with the frozen reward. Hence, we further introduce Lever-Edit, a two-stage framework that learns a reward-aligned captioner for counterfactual target descriptions, freezes it, and optimizes the editing policy solely with the transferred T2I reward. Experiments show competitive editing
alignment and source preservation against editing-reward-based fine-tuning, while outperforming intuitive transfer baselines.
Read the full abstract ▾
All authors Qichao Ma, Jikang Cheng, Ling Liang, Zhaofei Yu, Tiejun Huang, Renye Yan
Computer Vision
LagrangeGS: Non-Conservative Lagrangian System on Dynamic 3D Gaussian Splatting
Shogo Sato, Takuhiro Kaneko, Shoichiro Takeda et al. (+4 more)
Dynamic 3D Gaussian Splatting (3DGS) achieves photorealistic reconstruction of time-varying scenes, and recent physics-aware extensions improve extrapolation by explicitly predicting velocity fields. However, these extensions merely fit vector fields to visual deformations without satisfying Lagrangian mechanics, leading to three major issues: (i) physically inconsistent trajectories, (ii) lack of time-reversibility, and (iii)
geometric collapse during long-term extrapolation. In this paper, we propose LagrangeGS, which formulates dynamic 3DGS as a non-conservative Lagrangian system. While this Lagrangian formulation fundamentally solves (i), a direct application of general LNNs to dynamic 3DGS requires a large velocity-Hessian inversion for millions of Gaussian particles. To overcome this computational bottleneck, we approximate the velocity-Hessian as an identity
matrix, decoupling particle dynamics for computational tractability. For (ii), we restrict the non-conservative forces to be explicitly time independent, enabling consistent backward integration. Finally, to address (iii), we introduce local rigid
alignment that regularizes particle trajectories. Extensive evaluations on dynamic scene
benchmarks demonstrate that LagrangeGS enables stable long-term extrapolation, consistent time reversal, and counterfactual physics-based editing without retraining.
Read the full abstract ▾
All authors Shogo Sato, Takuhiro Kaneko, Shoichiro Takeda, Tomoyasu Shimada, Riku Inoue, Kazuhiko Murasaki, Ryuichi Tanida
Computer Vision
ByteAction: Byte-space Action Recognition Foundation Model
Fangcheng Li, Zhen Yu, Kejun Wu et al. (+2 more)
Byte-space Action Recognition (BAR) aims to recognize human actions directly from compressed image bitstreams without any pixel decoding. By operating entirely in byte space, BAR is inherently independent of file integrity and pixel-level reconstruction, making it naturally applicable to
privacy-sensitive scenarios and robust against bitstream corruption. In this paper, we propose ByteAction, a BAR foundation model that achieves accurate action recognition on corrupted image bitstreams. ByteAction follows a dual-view byte-level recognition framework. It constructs weakly and strongly corrupted bitstream views, which are augmented by Bitstream Pattern Augmentation (BPA) and encoded with a shared ByteFormer backbone. The model is optimized with both classification and corruption consistency objectives. Specifically, we propose Bitstream Pattern Augmentation (BPA), which reshapes one-dimensional byte sequences into two-dimensional byte
matrix and applies region-level erasure to encourage the model to learn robust cross-region byte dependencies. We further propose a Corruption Consistency Training strategy that constrains the model to maintain stable predictions across different corruption severities through bidirectional KL divergence. Experiments on the image bitstream from Stanford40, PPMI, and PASCAL VOC 2012 Action demonstrate that ByteAction achieves state-of-the-art corruption robustness across all scenarios while maintaining competitive intact bitstream performance.
Read the full abstract ▾
All authors Fangcheng Li, Zhen Yu, Kejun Wu, Qiong Liu, You Yang
Computer Vision
Object-Uni: A Unified Model for Object-Centric Spatial Understanding and Controllable Generation
Mining Tan, Yinuo Wang, Ziqi Zhou et al. (+6 more)
Unified models for visual understanding and generation have made rapid progress, yet they still lack the ability to understand and manipulate the spatial states of object instances. Existing models can describe objects in natural
language, but they struggle to precisely represent continuous object poses and generate geometrically consistent images under target viewpoints. To mitigate this, we propose Object-Uni, a unified model for object-centric spatial understanding and controllable generation. Specifically, we formulate object-centric spatial intelligence as a unified problem connecting pose perception, spatial
reasoning, pose-conditioned generation, and object-centric novel view synthesis. We treat object pose as an explicit
geometric variable shared by understanding and generation, rather than merely a prediction label or control signal. To make pose usable by
multimodal large
language models, we propose a viewpoint-based orientation abstraction that maps orientation into structured viewpoint descriptions while preserving continuous
geometric supervision. We further construct an object-centric spatial
benchmark (UniSpatial-80K) and train a unified model with an object-
token-grounded pose anchor to associate each instance with its pose state. Experiments show that our model improves object-level pose understanding and pose-controllable generation, moving unified models from describing objects toward manipulating spatial states.
Read the full abstract ▾
All authors Mining Tan, Yinuo Wang, Ziqi Zhou, Weize Quan, Sifei Li, Jingdong Chen, DanDan Zheng, Libin Wang, Weiming Dong
Computer Vision
Seeing the Unseen: Semantic-in-Gaussian for Sparse-View 3D Generalization
Zeyang Bai, Yunpeng Wang, Yunbiao Wang et al. (+1 more)
Generalizable 3D Gaussian Splatting (G-3DGS) has emerged as a promising approach for novel view synthesis undersparse-view settings. However, existing frameworks remain restricted by pixel-aligned Gaussian estimation, whichstruggles in partially observed or occluded regions and often leads to incomplete surfaces or structural collapse. Toaddress these challenges, we propose SeeU (Seeing the Unseen), a novel G-3DGS framework. We frame its core design asSemantic-in-Gaussian: semantic-conditioned refinement in Gaussian space. Specifically, we introduce a Cross-viewEntropy-Aware (CEA) module that aggregates multi-view semantic and
geometric cues into compact
embeddings. Theseembeddings guide the Conditional Gaussian
Transformer, which applies residual updates to coarse Gaussians, helpingrecover under-constrained regions of partially observed structures while preserving surface consistency. Comprehensiveexperiments on multiple
benchmarks demonstrate that SeeU consistently improves rendering quality and structuralcompleteness while retaining efficient feed-forward inference. Especially under challenging extrapolation settings,SeeU achieves an average improvement of 2.44 dB in PSNR compared to recent SOTA G-3DGS methods.
Read the full abstract ▾
All authors Zeyang Bai, Yunpeng Wang, Yunbiao Wang, Jun Xiao
Computer Vision
LoViF 2026 The First Challenge on Unified Removal of Raindrops and Reflections: Methods and Results
Zewei He, Xi Tong, Yu Chen et al. (+49 more)
This workshop paper comprehensively reviews the First Challenge on Unified Removal of Raindrops and Reflections. The challenge aims to address a frequently encountered practical problem in the field of autonomous driving, i.e., raindrop-reflection composite degradation on rainy days. This competition attracted 149 registered participants and received 12 valid final submissions with corresponding fact sheets, significantly contributing to the progress of unified removal of raindrops and reflections. All the methods are developed and evaluated on our real-shot RainDrop and ReFlection (RDRF)
dataset. A detailed analysis of the submitted methods and corresponding results is provided in this report, which highlights effective approaches and provides interesting insights for future research.
Read the full abstract ▾
All authors Zewei He, Xi Tong, Yu Chen, Xingyu Liu, Xin Li, Zepeng Wang, Jiagao Hu, Fuhao Li, Yuxuan Chen, Fei Wang, Daiguo Zhou, Minmin Yi, Chuanrui Zhang, Liwen Zhang, Yeongjin Jeong, Hyunjin Cho, Jiwon Lee, Minsang Kim, Jae Woong Soh, Jin-Hui Jiang, Rong-Lin Jian, Chih-Chung Hsu, Youngjin Oh, Junhyeong Kwon, Junyoung Park, Jae Hyun Park, Sung Ju Lee, Nam Ik Cho, Vishwajeet Shukla, Himanshu Baurai, Zhiqi Zhang, Kui Jiang, Zhaocheng Yu, Runzhe Li, Dawei Fan, Hao Li, Zhanshuo Zhang, Fan Ji, Jiangmeng Li, Xiongxin Tang, Fanjiang Xu, Shangquan Sun, Anh-Kiet Duong, Petra Gomez-Krämer, Jean-Michel Carozza, Ruibo Zhang, Dexiang Hong, Xinyan Liu, Shengeng Tang, Weidong Chen, Tzu-Hsuan Weng, Min-Te Sun
Computer Vision
Hybrid Generative-Discriminative Object Placement
Siyuan Zhou and Li Niu
As an important operation of image composition, object placement aims to predict the plausible placement (location, scale) for the inserted foreground object. Previous object placement methods can be divided into generative methods and discriminative methods, both of which cannot balance efficiency and effectiveness well. In this work, we propose a semi-generative method in the middle ground between them. In particular, we assign uniformly distributed anchors on the background. Then, we fuse foreground and background features to predict the rationality score for each anchor and predict plausible placement sets for positive anchors. Extensive experiments on the OPA
dataset show that our method can strike a good balance between efficiency and effectiveness.
Read the full abstract ▾
Computer Vision
MorphoCLIP: Text-Supervised Contrastive Learning for Perturbation Matching in Cell Painting Images
Sukhrobbek Ilyosbekov, Shubham Gajjar and Rongfei Jin
Cell Painting microscopy captures how
cells change after a chemical or genetic perturbation. Connecting these images to the perturbations that produced them could make large imaging screens easier to search and interpret, but the task remains difficult because biological effects are subtle and technical variation is substantial. We introduce MorphoCLIP, a contrastive model that links
Cell Painting profiles with text descriptions of compounds, CRISPR knockouts, and ORF overexpressions. The model keeps its
vision and
language backbones frozen and trains only a compact cross-channel module and projection layers, so it can be trained on a single consumer GPU. On held-out CPJUMP1 data, MorphoCLIP searches in both directions: from a
cell image to its perturbation description and from a description to matching
cell images. In both cases, a correct match appears among the top ten results much more often than expected by chance. Adding a replicate-
alignment loss makes profiles from repeated experiments more consistent, although this improvement does not yet translate into reliable gene-compound matching. Gene-aware labels and plate correction also show no consistent
retrieval benefit. These findings suggest that text supervision can help organize chemical and genetic
Cell Painting data. Matching compounds with genetic perturbations, however, remains an open problem.
Read the full abstract ▾