Computer Vision · Cross-listed from q-bio.QM
BananaVLM: A Domain-Adapted Vision Language Model for Banana Crop Disease Diagnosis
Sangam Kumar Jena and Pandarasamy Arjunan
Banana crop diseases threaten food security across the world, yet field diagnosis remains difficult because of limited expert access and visual similarity among disease classes. We present BananaVLM, a domain-adapted
vision-
language model for banana disease diagnosis built on LLaVA-v1.5-7B and fine-tuned with Low-Rank Adaptation (LoRA). To address the lack of
multimodal agricultural instruction data, we introduce BananaInstruct, an automated pipeline that converts raw disease images into $\approx$80,000 question–answer pairs without manual annotation, using LLaVA-1.5-13B for symptom descriptions and Mistral-7B for agricultural Q&A generation and label grounding. Evaluated against 14 open-source and 5 closed-source VLMs on a nine-class
benchmark, BananaVLM achieves 92.21% in-domain and 83.28% out-of-domain (OOD) classification accuracy, outperforming the best closed-source baseline, Gemini~2.5~Pro, by 49.8 and 63.3 percentage points, respectively. Binary healthy/diseased identification reaches 98.38% OOD accuracy with 1.00 recall. A controlled LoRA–DoRA comparison shows that DoRA performance degrades across epochs, while LoRA remains more effective for fine-grained classification. Qualitative evaluation using nine LLM judges yields G-Eval win rates of 0.81–0.96, while five domain experts prefer BananaVLM in 98.48% of 990 blind pairwise comparisons. These results show that lightweight domain adaptation with automated instruction tuning is an effective and scalable approach for specialized agricultural AI. Code,
datasets, and model weights are available at https://github.com/samy101/banana-vlm
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Contrastive World Models
Bonnie Li
World models trained via pixel reconstruction can struggle in visually complex environments, where irrelevant information dominates the objective and distract the model from information relevant to planning and control. We present Contrastive World Models, an approach for learning latent dynamics models without pixel reconstruction. Building on Dreamer, we replace observation reconstruction in the standard world model objective with a Deep InfoMax-like lower bound that maximizes the mutual information between state-action sequences and local patch features of future observations, encouraging state representations to retain information that is predictive of the future without requiring the model to reconstruct visually irrelevant details. We evaluate our approach in small-scale experiments across three settings of increasing visual complexity. Our method matches Dreamer and a momentum prediction baseline in the default setting, and substantially outperforms both once distractors or natural
video backgrounds are introduced, while also training more efficiently by removing the pixel decoder entirely. Our approach is general and makes minimal assumptions beyond access to state-action sequences and future observations. These results suggest that contrastive, infomax-based objectives are a principled and promising direction for building world models that are robust to visual nuisance factors, a property particularly relevant for transferring model-based RL
agents to the real world.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
UI-Venus-2 Technical Report
Venus Team, Zhuohan Cai, Haoxing Chen et al. (+28 more)
Multimodal GUI
agents have emerged as a promising paradigm for digital task automation, yet transitioning from
benchmark-oriented models to dependable real-world applications remains challenging due to limited environment coverage, brittle task construction, and unreliable reward verification. In this work, we present UI-Venus-2, a general-purpose foundation GUI
agent designed to operate across mobile, web, and desktop environments through a unified closed-loop
reasoning-action framework. To bridge the gap toward practical deployment, we jointly scale three critical dimensions: (1) Environments, expanding coverage to more than 170
multilingual mobile apps and native desktop operating systems; (2) Tasks, employing a deep-research pipeline for function-grounded instruction generation; and (3) Verification, adopting trace-level and sample-level evaluators with visual keypoints and multi-model voting to ensure reliable RL signals for training. Furthermore, we integrate safety-aware mechanisms to ensure controlled execution of consequential actions. By offering a capable, efficient, and open-source foundation, UI-Venus-2 advances the field toward more generalizable, verifiable, and self-reflective
agents for real-world applications.
Read the full abstract ▾
All authors Venus Team, Zhuohan Cai, Haoxing Chen, Jiaxuan Chen, Weizhi Chen, Changlong Gao, Zhangxuan Gu, Yuan Guo, Yusong Hu, Jianrong Jiang, Jianguo Li, Runze Li, Jinzhen Lin, Zhenyu Ma, Changhua Meng, Han Peng, Xinyu Qiu, Shuheng Shen, Zhongyi Shui, Weiqiang Wang, Ming Wen, Zhuoer Xu, Hang Yan, Kaiwen Yang, Ruilin Yao, Nanjun Yu, Zhengwen Zeng, Lianrui Zhang, Yunzhu Zhang, Zhe Zhao, Beitong Zhou
Computer Vision · Cross-listed from eess.IV
Enhancing MRI Brain Tumor Edge Detection: A Hybrid Preprocessing Approach Utilizing CLAHE
Shahid-E-Kaiser Md. Tashrif, Munshi Md Arafat Hussain, Sheikh Nahian et al. (+1 more)
Accurate boundary delineation of
brain tumors in
Magnetic Resonance Imaging (MRI) is a critical yet formidable challenge in neuro-oncology due to inherent scanner noise, complex anatomical structures, and uneven illumination. Traditional edge detection
algorithms, while computationally lightweight and mathematically interpretable, frequently fail to capture the diffuse, localized boundaries of edema when relying solely on global preprocessing and manual parameter tuning. To overcome these limitations, we propose a hybrid automated edge detection pipeline. Our approach integrates an optimally configured Contrast-Limited Adaptive Histogram Equalization (CLAHE) layer into a comprehensive morphological preprocessing framework, followed by a deterministic sequential parameter sweep to fully automate threshold selection. The proposed hybrid model demonstrated enhancement in detecting critical anatomical structures in a publicly available
benchmark database from Kaggle. By intelligently amplifying localized gradients without overwhelming the image with background noise, our method achieved higher Recall (Sensitivity). Consequently, the overall F1-Score elevated, and the Structural Similarity Index (SSIM) improved, all while maintaining a highly efficient execution. This establishes our optimized pipeline as a highly practical and near real-time operational model for clinical diagnostics, offering a compelling alternative to computationally heavy deep learning approaches.
Read the full abstract ▾
All authors Shahid-E-Kaiser Md. Tashrif, Munshi Md Arafat Hussain, Sheikh Nahian, Sumaiya Islam
Computer Vision · Cross-listed from cs.SE
STEP: A Modular Silent Trial Engine for Operational Evaluation of Digital Pathology AI in Routine Workflow
Gabriele Campanella, Matthew Croken, Olga Lukatskaya et al. (+4 more)
Prospective silent trials provide an important bridge between retrospective validation of artificial intelligence (AI) models and their use in clinical care by evaluating model performance and operational reliability on live clinical data without influencing patient management. In computational pathology, conducting silent trials requires integration across laboratory information systems, digital pathology infrastructure, computational resources, and model inference pipelines, and these workflows are often implemented using application-specific software. We developed the Silent Trial Engine for Pathology (STEP), a reusable software platform for orchestrating prospective silent trials of computational pathology AI models across heterogeneous clinical and computational environments. STEP separates common trial orchestration from institution-specific data access and compute infrastructure through modular adapter interfaces. The platform supports scheduled case discovery, per-slide inference submission, deterministic idempotency, failure recovery, result and ancillary-data ingestion, persistent trial and run state, and audit logging, with compute adapters supporting local execution and high-performance computing environments using LSF and Slurm. STEP was deployed at three institutions to support prospective silent evaluation of EAGLE, an AI model for predicting EGFR mutation status from hematoxylin and eosin-stained whole-slide images. By separating trial-level workflow logic from site-specific integrations, STEP enables a common execution framework to operate across heterogeneous pathology environments while maintaining durable and auditable trial state. This approach may reduce duplicated engineering effort and facilitate systematic real-world evaluation of computational pathology AI before interventional clinical deployment.
Read the full abstract ▾
All authors Gabriele Campanella, Matthew Croken, Olga Lukatskaya, Jane Houldsworth, Ricky Kwan, Peter Schüffler, Chad Vanderbilt
Computer Vision · Cross-listed from cs.CL
ReVA: A Region-Aware Visual Assistant for Visually Grounded Question Answering
Anoop Senthil
Multimodal Large
Language Models (MLLMs) have achieved remarkable progress in Visual Question Answering (VQA), yet they continue to struggle with questions requiring precise spatial
reasoning and fine-grained visual understanding. These limitations often manifest as object, attribute, and spatial
hallucinations, where models generate confident but visually unsupported responses due to insufficient region-level and fine-grained visual grounding. To address this challenge, we propose ReVA, a region-aware VQA model that employs a frozen CLIP ViT-L/14
Vision Transformer (ViT) and a Qwen2.5-7B-Instruct large
language model (LLM) connected through a dual bridge that aligns both whole-image and region-level representations with the LLM's
embedding space. The image bridge maps final
transformer block features into image
tokens. The region bridge maps cropped features from enriched intermediate features across ViT blocks so early texture and later object cues are more evident, into K region
tokens for every bounding box. ReVA uses a detector stack that supplies automatic zero-shot bounding boxes that are both question-agnostic and question-dependent, using RAM++ (Recognize Anything Model), spaCy, and Grounding DINO. The image
tokens and region
tokens are concatenated as an LLM
prompt prefix to jointly encode scene-level context and fine-grained regional evidence when answering questions. Evaluated on VQAv2, MMBench, POPE, and SEED-Bench, ReVA achieves 82.85% mean F1 on POPE, compared with 81.14% for an image-
token baseline without region
tokens. These results demonstrate that explicit region-aware visual representations reduce object
hallucination and improve the factual grounding of MLLMs.
Read the full abstract ▾
Computer Vision
Variable-Granularity Tokenization for High-Resolution Object Detection
Khayrul Islam
ViT detectors fix a uniform
token grid before any learned stage. A native-resolution aerial detector must then choose between resolving few-pixel objects and staying inside compute and
memory limits. We introduce VGTok, a training-free tokenizer that sets patch granularity per region from pixels, ahead of the encoder. VGTok scores each region by multi-scale morphological top-hat separability from its surround, then thresholds those scores at a per-image percentile, which fixes the
token budget. A structure-tensor gate ($\lambda_{\min}$) refines only where two-dimensional object structure supports it, leaving one-dimensional clutter coarse. The resulting
token set is a strict partition of the image. In a Co-DETR detector with an EVA-02 ViT-L encoder, VGTok clears every published VisDrone-val AP and AP$_S$ at every budget from 40% to 100% of
tokens. At 40% it records 44.22 AP with three fifths of the sequence discarded before the first
transformer block; dense, it reaches 48.38 AP, $6.08$ above the strongest published entry. VGTok transfers to AI-TOD-v2 untouched, same scorer and same rank, and sets a new state of the art at 37.27 AP and 19.51 AP$_{vt}$. As a pure drop-in into a frozen checkpoint it reaches 36.29 AP at 78.5% of
tokens, above every published entry, where our 376.3M-parameter detector clears a 3.0B multi-expert model. We show that a
token budget fixed before the backbone, from local separability and structure geometry alone, holds accuracy on the tiny-object regimes that dominate aerial detection, at $3.1\times$ less encoder compute and $1.9\times$ less encoder
memory. Code and models are available at \href{https://github.com/khayrulbuet13/vgtok}{github.com/khayrulbuet13/vgtok} and \href{https://huggingface.co/khayrulbuet13/vgtok}{huggingface.co/khayrulbuet13/vgtok}.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Is Deformable Image Registration Ready for Brain Metastasis Reirradiation Dose Accumulation? A Longitudinal MRI Benchmark of Registration Accuracy
Hengjie Liu, Manju Sharma, Xinyi Fu et al. (+2 more)
Dose accumulation is increasingly important in adaptive radiation therapy and reirradiation, but its clinical validity depends on the performance of deformable image registration (DIR). Reirradiation of
brain metastases (BMs) with stereotactic radiosurgery (SRS) provides a controlled but clinically meaningful DIR test case: intra-subject
brain deformation is usually limited after rigid
alignment, yet recurrent lesions can undergo substantial local shape and volume changes that rigid registration cannot capture and can affect dose accumulation. We benchmarked a wide range of learning-based and optimization-based DIR methods on 87 manually screened longitudinal contrast-enhanced T1-weighted MRI lesion pairs from an institutional BM SRS retreatment cohort. Learning-based methods pretrained on healthy-
brain MRI were evaluated zero-shot and after instance-specific optimization (ISO) or tumor-proximity target-specific optimization (TSO). Registration was assessed using lesion overlap (Dice), surface distance metrics (HD95 and sASD), target-volume recovery, and runtime and
memory. Pretrained learning-based methods showed variable zero-shot performance, while ISO/TSO improved all tested learning-based families. However, optimization-based methods remained the best-performing approach while maintaining reasonable runtime. These findings suggest that even state-of-the-art DIR methods do not yet provide sufficiently accurate and consistent registration for unmonitored use in
brain metastasis reirradiation dose accumulation. Because accurate registration is a prerequisite for deformable dose accumulation, clinical application will require case-level quality control and direct assessment of how registration uncertainty affects downstream dose metrics.
Read the full abstract ▾
All authors Hengjie Liu, Manju Sharma, Xinyi Fu, Di Xu, Ke Sheng
Computer Vision
Stochastic Liquid Deformation Fields: An SDE Generalisation of Closed-Form Continuous-Time Cells for Dynamic 3D Gaussian Splatting
Mingzhao Li and Arghya Pal
Deformable 3D Gaussian Splatting (D-3DGS) reconstructs dynamic scenes by deforming a canonical set of 3D Gaussians through a deformation field of frame time. Replacing its MLP with a stack of Closed-form Continuous-time (CfC)
cells-a
Liquid Neural Network that solves the
Liquid Timeconstant ODE in closed form-gives the field continuous-time behaviour at feed-forward cost. That closed form, however, is only the deterministic limit of a noise-driven system, and drops the
stochastic term usually credited for the robustness of
liquid networks. We put it back: a small Gaussian perturbation is added to the time gate of every CfC
cell, turning the deterministic field into a simple
stochastic (SDE) one. The noise is used only during training, needs no solver, and reduces exactly to the CfC when switched off. On the synthetic D-NeRF scenes the
stochastic field is on par with the deterministic CfC and beats the MLP baseline on most scenes; on the real-world NeRF-DS scenes the deterministic limit is already best and adding noise does not help. The study thus gives both a clean way to read the CfC field as an SDE and an honest account of when a plain noise term helps and when it does not.
Read the full abstract ▾
Computer Vision
TopoAgent: A Structure-Aware Perception-to-Reasoning Framework for Diagram-to-Graph Topology Extraction with Large Vision-Language Models
Bangwei Guo, Xujiang Zhao, Yanchi Liu et al. (+7 more)
Diagram-to-graph
topology extraction aims to extract a graph of entities and their connections from a structural diagram. This task remains challenging for current
vision-
language models because it requires both fine-grained perceptual grounding and
topology-aware
reasoning with global consistency. We present TopoBench-180, a human-verified
benchmark for diagram-to-graph
topology extraction, and TopoAgent, a structure-aware perception-to-
reasoning framework for reliable
topology extraction using large
vision-
language models. TopoBench-180 contains 180 structural diagrams spanning Web-style and Network-style categories, paired with canonical graph annotations. TopoAgent progressively extracts the target graph by combining grounded perception, global structural priors, canonical node inventory construction, node-centric local-to-global relation
reasoning, and topological consistency enforcement. Experiments on TopoBench-180 show that TopoAgent outperforms strong
vision-
language model baselines and recent visual
reasoning frameworks, especially on edge extraction. More broadly, this work fills an important gap in
multimodal structured understanding by establishing a
benchmark and framework for diagram-to-graph
topology extraction. The
benchmark and associated resources will be publicly released at https://huggingface.co/
datasets/WayneGuo0011/TopoBench-180.
Read the full abstract ▾
All authors Bangwei Guo, Xujiang Zhao, Yanchi Liu, Wei Cheng, Shengyu Chen, Dongyue Li, Masaharu Morimoto, Takayuki Kuroda, Dimitris Metaxas, Haifeng Chen
Computer Vision
Beyond Visual Boundaries: Rethinking Scene Segmentation for Movie RAG
Dong-Hee Kim, Seonwoo Choi, Changbeen Kim et al. (+6 more)
Understanding long-form
video remains a fundamental challenge for
multimodal large
language models (MLLMs). Sparse frame sampling fails to capture fine-grained visual details, while dense sampling quickly exceeds context length limits.
Retrieval-augmented generation (RAG) offers a promising middle ground by selectively retrieving relevant
video segments for grounded generation, yet its effectiveness critically depends on the quality of the
video segments used as
retrieval units. In this paper, we investigate RAG for movie understanding, which demands story-level
reasoning over characters, events, and narrative arcs spanning hours of content. Scene
segmentation, a long-studied problem that partitions movies into semantically coherent units, is a natural candidate for defining such
retrieval units. We reexamine whether existing methods actually serve this role through comprehensive evaluation on downstream movie understanding tasks, and find that they consistently fail to outperform naive uniform temporal chunking. Our audit of the most standard scene
segmentation benchmarks reveals why: current annotations prioritize visually salient transitions over narrative event structure. Motivated by this mismatch, we introduce NarraScene, a narrative-centric scene
segmentation dataset annotated with a three-level cognitive taxonomy spanning physical, character, and narrative change, where every valid boundary requires a narrative-level shift. When used as
retrieval units, these narrative-grounded segments outperform uniform chunking on downstream movie understanding tasks, suggesting that the central challenge for scene
segmentation in movie RAG is not detecting boundaries, but identifying the narrative event units that matter for movie understanding.
Read the full abstract ▾
All authors Dong-Hee Kim, Seonwoo Choi, Changbeen Kim, Jungmyung Wi, Juyeon Ko, Youngju Choi, Il Hyeon Mun, Hyunwoo J. Kim, Donghyun Kim
Computer Vision
State-Conditioned Visual Evidence Retrieval for Fine-Grained Perception in Document Vision-Language Models
Mingxu Chai, Chenyu Liu, Ziyu Shen et al. (+7 more)
Compared with typical
vision-
language tasks, document parsing places stronger demands on fine-grained visual perception. Existing
vision-
language model (VLM)-based parsing approaches rely on globally compressed visual
tokens, where fine-grained details are entangled within a single representation and repeatedly accessed during decoding. However, we observe that the visual evidence for each prediction is typically localized and conditioned on the current decoding state, whereas such representations must be accessed in full at every decoding step, resulting in inefficient computation. To address this mismatch, we formulate perception as state-conditioned visual evidence
retrieval (SCVER) during autoregressive decoding. The model operates on a compact global representation for coarse structure and retrieves a small set of relevant high-resolution regions conditioned on the current
token state. This coarse-to-fine design enables on-demand access to fine-grained visual cues, relieving globally shared representations from encoding all fine-grained details. We further find that learning such state-conditioned
retrieval in VLMs is challenging and unstable. To stabilize this process, we introduce a Spatially-Guided Learning Objective (SGLO) to guide the
retrieval process. Experiments on document parsing
benchmarks show that SCVER improves robustness under reduced input resolution and achieves a better accuracy-efficiency trade-off, demonstrating the effectiveness of on-demand visual evidence
retrieval for fine-grained perception.
Read the full abstract ▾
All authors Mingxu Chai, Chenyu Liu, Ziyu Shen, Jiazheng Zhang, Kaidi Zhang, Ruoyu Chen, Jun Long, Jihua Kang, Tao Gui, Qi Zhang
Computer Vision
Instruction Distillation: Text Instructions as Visual Examples
Hardik Jindal, Soumyabrata Pal and Sayak Ray Chowdhury
Visual in-context learning (ICL) with
multimodal large
language models (MLLMs) is effective for fine-grained visual classification, but each retrieved image example consumes several hundred context
tokens, making large-$K$ settings prohibitively expensive at inference scale. We propose Instruction
Distillation: an offline procedure in which the MLLM itself generates, for each individual training image, a structured identification instruction encoding general appearance cues, features that differentiate the class from visually similar ones, and a common confusion point. Unlike prior work that produces a single description per class, our instructions are generated per training image, preserving the intra-class visual diversity that per-class descriptions collapse. At inference time, we study five configurations sharing a single CLIP
retrieval index: zero-shot, image ICL, instruction-only ICL, and two hybrid variants in which retrieved neighbors are split between images and instructions. Across seven fine-grained
benchmarks and two MLLM backbones, instruction based pipelines match, or exceeds image ICL at $K{=}1$ and reduces per-query
tokens by $2.9\times$ and inference latency by $3.3\times$ at $K{=}5$. Hybrid configurations further show that visual and textual ICL signals are complementary, images give visual patterns to learn and see, while instructions give explicit rules and logic. When both of these are provided, the quality of context improves, which is noticeable in the performance.
Read the full abstract ▾
Computer Vision
Weaving Visual Narratives: Agentic Image Bundle Composition Beyond Atomic Visual Matching
Rong Shan, Tianyi Xu, Congmin Zheng et al. (+9 more)
Image
retrieval has traditionally been formulated as a point-wise matching problem, where each candidate image is scored in isolation. However, this atomic paradigm fails to capture the complexity of human search intent within personal photo collections, where users often seek compact visual stories bound by structural relations rather than isolated snapshots. To address this limitation, we introduce **Image Bundle Composition (IBC)**, a novel paradigm that shifts the objective from ranking individual images to dynamically composing cohesive image bundles from a massive, unstructured photo pool. Since target bundles are not predefined, IBC presents a severe combinatorial explosion challenge and demands modeling non-decomposable joint relevance. To establish this paradigm, we construct **IBCBench**, the first IBC
benchmark dataset containing 109,467 images and 667 verified queries, built via a semi-automated verification pipeline. Furthermore, we propose **BundleWeaver**, an agentic framework that reformulates IBC as query-conditioned incremental hyperedge discovery. By employing a Large
Language Model to adaptively search for missing relational roles and utilizing a
Vision-
Language Model for whole-bundle verification, BundleWeaver effectively navigates the combinatorial space. Extensive experiments demonstrate that while state-of-the-art
embedding models and static decompose-and-rerank paradigms suffer from relational blindness, BundleWeaver achieves substantial performance gains, highlighting the necessity of shifting from atomic scoring to dynamic relational composition. Our
dataset and code are available.
Read the full abstract ▾
All authors Rong Shan, Tianyi Xu, Congmin Zheng, Wenteng Chen, Jiachen Zhu, Junjie Wu, Teng Wang, Weiwen Liu, Changwang Zhang, Weinan Zhang, Jun Wang, Jianghao Lin
Computer Vision
SNF-Bench: Separating Static Drift from Natural Flow in Long-Horizon Fixed-Camera Video Generation
Matiur Rahman Minar, Seunghun Oh, Ganghyeon Jeong et al. (+1 more)
Long-horizon
video generation is evaluated with whole-frame metrics that reward motion and temporal consistency. For fixed-camera nature scenes this creates an ambiguity: motion of water, fire, smoke, or rain is desirable, whereas motion of the background is an error. A system can therefore score well on motion while its scene drifts, or on consistency while its flow stagnates. We introduce SNF-Bench, an evaluation framework for long-horizon fixed-camera generation that partitions each scene into static support and dynamic flow and reports static fidelity, flow persistence with absolute magnitude, and drift leakage separately, never as one score. Drift leakage is interpretive context rather than a headline measurement. Each factor is validated mechanistically rather than by correlation with preference: we inject global translation, rotation, and scale drift and progressive late freezing at known severity into real generations, and require each factor to respond in its stated direction and to remain selective against corruptions it does not target.
Auditing publicly released long-horizon text-conditioned checkpoints under one recorded common inference configuration, plus an image-conditioned track with released-pipeline references and a deployment-sensitivity panel, we find that whole-frame motion and static-region drift induce near-opposite orderings of the same outputs. At maximum controlled translation, fBD and NBF rise to $1.86\times$ and $1.32\times$ baseline, but whole-frame Dynamic Degree reaches only $1.07\times$—rewarding the corruption. SNF-Bench measures where motion occurs and whether it persists; it does not measure physical realism. Project page: https://minar09.github.io/snfbench/.
Read the full abstract ▾
All authors Matiur Rahman Minar, Seunghun Oh, Ganghyeon Jeong, Unsang Park
Computer Vision · Cross-listed from cs.RO
RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction
Zifan Wang, Ziang Ren, Pengyang Shi et al. (+7 more)
Enabling
humanoid robots to respond to human
speech with synchronized and semantically meaningful gestures is fundamental to natural human-
robot interaction. However, this task faces three critical barriers: the scarcity of semantically rich
datasets, the "modality eclipse" where models ignore audio cues in favor of kinematic inertia, and the sim-to-real gap regarding physical safety. We propose RoboGesture, a
robot-centric framework that co-designs data, modeling, and control to power a complete interactive human-
humanoid system in which the
robot listens, responds, and gestures in real time. We first establish the RoboGesture
dataset featuring over 300 gesture categories and develop an automated pipeline to synthesize large-scale collision-free,
robot-specific audio-motion pairs. Our architecture features a Hierarchical Semantic-Acoustic Aligner that extracts multi-granular prosodic and semantic cues directly from raw audio
tokens. These cues drive a Streaming Conditional Motion Generator based on a
diffusion transformer with conditional flow matching. To ensure high responsiveness, we introduce Anti-Inertia CFG Masking, which prevents the model from collapsing into repetitive historical patterns by compelling it to proactively mine control signals from the audio modality. Finally, an MPC-based safety filter ensures real-time, collision-free execution on physical hardware. Experiments on a Unitree G1
humanoid demonstrate that RoboGesture generates safer, more rhythmic, and more semantically appropriate responses compared to state-of-the-art baselines.
Read the full abstract ▾
All authors Zifan Wang, Ziang Ren, Pengyang Shi, Zirui Wang, Chenghuai Lin, Tianze Wang, Zekun Qi, Liangliang Zhao, He Wang, Li Yi
Computer Vision
Growing a Stand, Not a Tree: Joint Canopy Generation Reproduces Crown Shyness
Guang Yang and Fengchen Liu
In closed forests, neighboring tree crowns often stop short of touching, leaving a network of narrow gaps known as crown shyness. The pattern belongs to the stand rather than to any single tree, which makes it a natural probe of a question in generative modeling: can a learned model produce a set of objects whose defining structure exists only between them? We formulate stand-level canopy generation as set generation with a flow-matching model, in which
attention between trees is the only channel through which coupling can arise. Trained on stands grown by a resource-competition simulation that is provably not reducible to per-tree geometry, the joint model halves the clearance distribution error of an identical-capacity model that generates each tree alone, and the advantage persists at stem densities outside the training range. Against field measurements of a tropical oak forest, a single calibrated scalar yields held-out agreement in gap magnitude and crown asymmetry. The directional statistics of the gaps are controlled by stem placement rather than by the growth rule, and match the field once stem jitter is calibrated. Crown shyness, in both the simulation and the learned model, is a property of the stand and not of the tree.
Read the full abstract ▾
Computer Vision
What Can Low Resource Languages Learn From Each Other?
Achyuth P, Kahaan Shah and Chetan Arora
Despite the rapid advancement of
Vision-
Language Models (VLMs), their linguistic reach remains largely confined to high-resource
languages, leaving the majority of the world's 7,000+ living
languages on the wrong side of a growing digital divide. This disparity is especially pronounced in Optical Character Recognition (OCR), where low-resource scripts lack the massive
datasets required for traditional scaling laws. We investigate OCR adaptation in extreme data-scarce regimes (<10K real and <250K synthetic images), demonstrating that conventional fine-tuning strategies often reach a performance ceiling. Our key finding reveals a structural inefficiency in
language-specific adaptation: while higher layers of specialized models diverge to capture unique script nuances, the lower layers learn redundant, highly similar features. Motivated by this observation, we propose PSMC (Pre-train, Specialize, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect". Our approach first derives
language-specific experts from a high-resource base model, then employs task arithmetic to fuse these experts into a unified, high-performance
multilingual back- bone. Extensive evaluation across 10 Indian scripts (supporting 20+
languages) shows that PSMC achieves a ~2% average improvement in Word Recognition Rate (WRR) over individual specialist models without increasing parameter count. Our results indicate that joint training in the merged latent space facilitates a constructive knowledge transfer that benefits all constituent scripts, providing a scalable pathway for inclusive VLM development. Source code and
datasets will be released post publication.
Read the full abstract ▾
Computer Vision
ABCD: Alpha-Composited Block Coordinate Descent: Constant-VRAM Training for Large Radiance Fields
Ka Heng Shiu and Kartic Subr
We present ABCD (Alpha-Composited Block Coordinate Descent), an out-of-core training framework for alpha-composited radiance fields, instantiated here for 3D Gaussian Splatting. Our method reformulates training as block coordinate descent over spatial partitions: only one block of parameters is active at a time, while all others are frozen. By exploiting the associativity of alpha blending, these inactive regions can be pre-rendered and collapsed into foreground and background RGBA images. As a result, for fixed partition size and image resolution, peak VRAM becomes O(1) with respect to total scene extent, rather than growing with full scene size. This enables GPUs with limited
memory to train scenes that would otherwise not fit in core. In experiments, our method closely preserves the reconstruction quality of 3DGS, with less than 5% PSNR degradation, while ABCD with compositing ablated suffers roughly 40% degradation. Our code can be found at https://github.com/shiukaheng/abcd
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
CARDINAL Predicts Cardiovascular Risk From Non-contrast Cardiac CT
Roy Gabriel, Nattakorn Kittisut, Jamshid Hassanpour et al. (+6 more)
Cardiovascular risk prediction remains limited by incomplete clinical data and imaging biomarkers that reduce computed
tomography (CT) to a small number of handcrafted features. We developed CARDINAL (Cardiovascular Assessment via Representation learning from Deep Imaging with Nested Anatomical Latent
embeddings), a clinically grounded framework that learns compact representations from routine non-contrast cardiac CT for major adverse cardiovascular event (MACE) prediction. In 17,659 patients, CARDINAL was evaluated for 1-, 3-, 5-, and 10-year MACE prediction against American Heart Association (AHA) pooled cohort equations (PCE), AHA predicting risk of cardiovascular disease events (PREVENT), coronary artery calcium (CAC),
segmentation-derived CT biomarkers, and 70-feature structural radiomics. Gains were largest at longer horizons. At 10 years, CARDINAL (joint) achieved an area under the receiver operating characteristic curve (AUROC) of 0.866 $\pm$ 0.020 and an area under the precision-recall curve (AUPRC) of 0.890 $\pm$ 0.015, compared with an AUROC of 0.826 $\pm$ 0.023 and an AUPRC of 0.826 $\pm$ 0.022 for structural radiomics, the strongest baseline. CARDINAL also achieved the highest survival concordance index (C-index), 0.753 $\pm$ 0.015, and high-versus-low risk-tertile hazard ratio, 10.78 $\pm$ 3.16, with favorable reclassification and exploratory calibration. These findings suggest that non-contrast cardiac CT contains prognostic information beyond conventional risk equations, CAC scoring, and engineered imaging biomarkers.
Read the full abstract ▾
All authors Roy Gabriel, Nattakorn Kittisut, Jamshid Hassanpour, Michael Galarnyk, Abanoub Abdelmalak, Marly van Assen, Carlo N. De Cecco, Arshed Quyyumi, Ali Adibi
Computer Vision
Report Supervision
Pedro R. A. S. Bassia, Wenxuan Li, Jakob Wasserthal et al. (+10 more)
Segmentation models can surpass radiologists, classification models, and
vision-
language models in tumor detection. Importantly,
segmentation models outline tumors, allowing radiologists to better verify and trust the AI output. Their main limitation is the scarcity of tumor masks: creating one 3D tumor mask takes up to 30 minutes, so most public CT
datasets contain only a few hundred masks, and even the largest private
datasets contain only a couple of thousand. Tumor masks are not produced in clinical routine, but radiology reports are. Public
datasets contain tens of thousands of CT-Report pairs, and hospitals contain hundreds of thousands. These reports describe tumors in detail, providing large-scale, informative training data. Here, we introduce Report Supervision (R-Super), a training framework that uses reports to directly supervise and improve tumor
segmentation. R-Super introduces new loss functions that teach
segmentation models to segment tumors that match report descriptions of tumor count, sizes, and locations. Reports are only used for training. We evaluated R-Super on kidney and pancreatic tumor
segmentation, exploring diverse training data sizes, up to 41,418 CT-Report plus 3,488 pancreatic tumor CT-Mask pairs. On external validation, R-Super increased tumor detection F1-Score and
segmentation DSC by up to +15% with respect to mask-only training. It also surpassed alternative methods such as CLIP and multi-task learning. Leveraging numerous readily available reports to supplement scarce masks, R-Super strongly improves AI performance when very few training masks are available (e.g., 50), and when many masks are available (e.g., 3,488), unlocking scale in tumor
segmentation.
Read the full abstract ▾
All authors Pedro R. A. S. Bassia, Wenxuan Li, Jakob Wasserthal, Jieneng Chen, Xinze Zhou, Zheren Zhu, Chuntung Zhuanga, Sergio Decherchi, Andrea Cavalli, Kang Wang, Yang Yang, Alan Yuille, Zongwei Zhou
Computer Vision · Cross-listed from cs.LG
Curvature-Aware Radius Shrinkage for Adaptive Nearest Neighbor Classification
Alexandre L. M. Levada
Nearest neighbor classification relies fundamentally on how locality is defined, yet conventional $k$-NN imposes the same neighborhood cardinality throughout the feature space. This assumption can be inadequate for data whose local geometry varies substantially across the underlying
manifold. We introduce
Curvature-Aware Radius Shrinkage for Adaptive Nearest Neighbor Classification (CARSANN), a geometry-driven framework that adapts the spatial support of each neighborhood according to local
geometric complexity. CARSANN first estimates intrinsic dimensionality using TwoNN and constructs an intrinsic representation through principal component analysis. Local mean
curvature is then estimated using a shape-operator-based formulation and controls neighborhood scale: highly curved regions receive stronger radius shrinkage, whereas approximately flat regions retain broader spatial support. Unlike methods that modify only the number of neighbors or the local metric, CARSANN explicitly adapts the spatial extent of local evidence. Experiments on more than 70 real-world OpenML
datasets show that CARSANN consistently improves upon standard $k$-NN and is competitive with adaptive nearest-neighbor methods. In a controlled comparison using the same base neighborhood size, CARSANN achieves higher balanced accuracy on 40 of 45
datasets, increasing mean balanced accuracy from 0.6506 to 0.7528. The advantage also persists against $k$-NN with fixed $k=5$. Friedman and Nemenyi tests confirm that the improvements are statistically significant. These results indicate that local
manifold curvature can serve as an effective
geometric control variable for adapting neighborhood support, providing a complementary paradigm to cardinality-based nearest-neighbor adaptation.
Read the full abstract ▾
Computer Vision
Depth-Aware Pothole Detection Using YOLO and RT-DETR at the Edge
Md Monjurul Ahsan Prodhan and Md Nour Hossain
Pothole detection and its severity measurement is still an important challenges in urban infrastructure management, where late maintenance directly contributes to vehicle damage, road accidents, and escalating repair costs. Existing automated approaches depend on 2D RGB images and cannot measure physical depth of potholes. In this paper, we present a depthaware pothole detection framework and then compare five architectures: YOLOv8n, YOLOv8nSeg, YOLOv9t, RTDETRL, and RTDETRX for RGB-D sensor fusion-based detection and automated depth measurement. A custom offline augmentation pipeline is used here to simulate adverse road monitoring conditions. All models are trained on the PothRGBD
dataset with an 80% training and 20% validation split and evaluated using Precision, Recall, mAP@50, and mAP@50_95. Before measuring the depth data, all depth maps are corrected for camera tilt using RANSAC ground-plane orthorectification and all zero-valued sensor pixels are cast to NaN before any statistic is computed. YOLOv8nSeg achieves the highest mAP@50 of 0.9556 and mAP@50_95 of 0.6758 with the most accurate depth estimate of 2.96 cm with the pixel-precise Dseg
algorithm. YOLOv8n achieves the fastest inference at 3.6ms. RTDETRX achieves the highest detection confidence at 92.70%. An important finding is that even after full RANSAC orthorectification, bounding box models overestimate pothole depth by 0.16 to 0.21 cm compared to pixel precise
segmentation masks. This confirms that the pavement inclusion bias is structural rather than a calibration artifact.
Read the full abstract ▾
Computer Vision
ShiftSplit-AD: Separating Domain Shift from Defects in Foundation-Feature Visual Anomaly Detection
Muhamathu Ameer Ali Aacaas Muhamath
Visual anomaly detectors based on frozen foundation-model features commonly score distances from test patches to a
memory of normal features. Benign acquisition changes can also enlarge these distances, confounding domain variation with defects. We investigate whether structured decomposition of nearest-normal DINOv2 residuals can suppress shift-induced evidence while retaining unseen defects. ShiftSplit-AD decomposes the patch residual
matrix into low-rank and row-sparse components and scores the sparse component, with an optional low-rank/sparse fusion. The experiments expose a central trade-off rather than a universal separation: genuine defects can contain correlated, low-dimensional structure, so filtering broad residual activity may also remove defect information. On AeBAD-S, using settings fixed after Bottle development, sparse-only scoring improves image AUROC from 0.6780 to 0.7294 and AUPRC from 0.8052 to 0.8465. Paired bootstrap 95% intervals for the improvements are [0.0238, 0.0808] and [0.0170, 0.0650], respectively. However, sparse-only scoring reduces mean clean AUROC from 0.9890 to 0.9133 on four held-out MVTec categories and degrades Bottle localization. These findings show that residual decomposition can help when domain shift strongly contaminates anomaly evidence, but preserving defect structure remains the limiting problem.
Read the full abstract ▾
Computer Vision
Quanta Perception as Probabilistic Events
Varun Sundar, Pavan Thodima, Sacha Jungerman et al. (+1 more)
Autonomous systems rely on extracting information from light, yet remain brittle in extreme environments, from nighttime
navigation to high-speed robotics. Conventional sensors aggregate
photons over fixed exposures, imposing trade-offs between sensitivity, dynamic range, and temporal resolution that degrade perception when
photons are scarce or dynamics are rapid. Quanta sensors detect individual
photons, but their streams exceed real-time compute and latency budgets by orders of magnitude. Here we introduce $\textit{probabilistic events}$, a computational primitive for real-time quanta perception from individual
photon detections. By computing the posterior over the time since the last intensity change, we represent
photon streams as recursive belief states. Rather than fixed-threshold event-camera triggers, this recursive
Bayesian formulation yields three low-latency signals: motion-adaptive scene flux, high-fidelity activity maps, and
entropy-based perceptual uncertainty. This representation enables perception in extreme conditions, including pose estimation of a running person at $\sim$0.05 lux—without retraining
vision models. Our approach processes input streams exceeding 50{,}000 quanta frames per second on commodity GPU hardware—yielding kilohertz-scale outputs up to four orders of magnitude faster than state-of-the-art quanta reconstruction baselines, even for megapixel arrays. By replacing frame reconstruction with direct probabilistic inference over
photon streams, this work bridges
photon-counting quanta sensing with robotic
vision.
Read the full abstract ▾
All authors Varun Sundar, Pavan Thodima, Sacha Jungerman, Mohit Gupta
Computer Vision
VidParse: Online Parsing of Egocentric Procedures Like a Pro
Anubhav Gupta, Archit Kambhamettu, Vatsal Agarwal et al. (+2 more)
Translating continuous, noisy
egocentric video streams into discrete, temporally ordered action steps is fraught with visual challenges. Heavy ego-motion, transient occlusions, and the high intra-class variability of unscripted human-object interactions cause standard frame-level online temporal models to struggle, often resulting in severe over-
segmentation and structural collapse. To bridge the gap between unstable low-level perception and high-level procedural logic, we present VidParse, an online, training-free framework that treats activity understanding as a graph-constrained inference problem. Rather than relying on learned temporal filters, we dynamically identify semantic transitions using a temporal similarity
matrix over manipulation-anchored features, which are extracted from frozen foundation models to prioritize foreground hand-object interactions. A beam search decoder then leverages an induced procedural task graph to explicitly enforce valid action transitions and prune impossible trajectories. By anchoring robust visual segments to hard procedural constraints, our approach preserves long-range state transitions and achieves up to a 10x improvement in complex multi-step parsing accuracy over strong online baselines, all without requiring a single gradient update.
Read the full abstract ▾
All authors Anubhav Gupta, Archit Kambhamettu, Vatsal Agarwal, Pulkit Kumar, Abhinav Shrivastava