Computer Vision · Cross-listed from cs.LG
Layer-Wise Gate-Controlled Prompt Truncation in a Multimodal Chest X-Ray Classifier
Jingtao Lei, Hongji Li and Dexiang Shu
Mixture of
Prompt Experts (MoPE) adapts
multimodal transformers through input-dependent
prompt composition, while retaining a fixed
prompt length. We investigate a layer-wise gating extension in a binary chest X-ray classification pilot study. The controller predicts a retention ratio for each sample, averages these ratios within a mini-batch, and uses the resulting integer length to truncate the static and mixed visual
prompts. Retained mixed
prompts are also scaled by the individual ratios. In one recorded run per configuration, the gated model reached a best validation accuracy of 0.8996, compared with 0.8969 for the fixed-length baseline; the corresponding final values were 0.8963 and 0.8802. The exported gate statistics imply a retained length of one at all recorded training points, relative to a configured maximum of six. This reduces the complete visual sequence from 210 to 200
tokens, but no direct runtime measurements establish an acceleration benefit. Report-derived labels, report text as input, sequential data partitioning, and the absence of repeated controlled experiments limit interpretation. The findings document
prompt shortening under the configured gate penalty; they do not establish sample-specific length allocation, superiority over fixed short
prompts, or clinical utility. Code is available at: https://github.com/jingtaolei/mope-dynamic-
prompt-truncation.
Read the full abstract ▾
Computer Vision
Reading Decoder Trajectories: Training-Free Counterfactual Query-Trajectory Reliability for Small-Object Detection
Zhaoning Shi and Bo Ma
Small-object detection remains challenging because limited pixels cause information loss and suppress the scale knowledge encoded in pretrained detectors. Existing approaches mainly improve representations through multiscale training, architecture redesign, or parameter adaptation, implicitly assuming that frozen models lack the required capability. We challenge this assumption and hypothesize that small-object knowledge already exists in frozen detectors but remains underactivated and unstable during query
evolution. To test this hypothesis, we propose Counterfactual Query-Trajectory Reliability (CQTR), a training-free framework that elicits latent responses through counterfactual scale interventions and interprets candidate reliability from decoder-internal spatial convergence, semantic persistence, and cross-scale conflicts. A small unlabeled training subset selects the appropriate correction mechanism for each model-data stream, without parameter updates or target-domain annotations. Across 27 combinations of nine frozen detectors and three
datasets, CQTR consistently improves average precision (AP) and average precision for small objects (APs). Closed-loop analyses further show that scale intervention activates latent responses, trajectory evidence predicts ground-truth support, and unlabeled routing selects the more effective branch. CQTR therefore reframes small-object detection from external scale augmentation to the activation and reliability assessment of latent scale knowledge.
Read the full abstract ▾
Computer Vision
Learning to Use Imagination: Progress-Conditioned Future Utilization for World Action Models
Yijie Zhu, Zitong Yu, Wei Li et al. (+4 more)
World Action Models (WAMs) extend
Vision-
Language-Action (VLA) models by incorporating future visual dynamics into action generation. However, existing WAMs often utilize imagined futures with limited adaptation to evolving execution progress, potentially introducing distracting or unreliable predictive cues. This limitation arises from two empirically identified forms of non-uniformity in future utility: (i) at the inter-progress level, the utility of imagined futures varies across execution stages as control demands change; and (ii) at the intra-progress level, individual future latents exhibit heterogeneous relevance within the same progress state. To address these limitations, we propose ProWAM, a Progress-Conditioned World Action Model that introduces execution progress as an explicit intermediate representation for adaptive imagination utilization. ProWAM comprises two tightly coupled components: (1) To obtain a reliable representation of execution progress, we propose the Self-Supervised Dual-Temporal Progress Encoder (SS-DTPE). SS-DTPE couples short-term action-observation interaction modeling with long-term recurrent progress aggregation to capture recent execution feedback and accumulated task history. (2) Conditioned on the progress representation from SS-DTPE, we propose the Hierarchical Progress-Conditioned Imagination Modulation (HPIM) to adapt imagination utilization to execution progress. HPIM operates at two complementary levels: an inter-progress global modulation mechanism adapts future utilization across execution stages, while an intra-progress relevance mechanism differentiates individual future latents within each progress state. Extensive experiments demonstrate consistent gains over strong VLA and WAM baselines.
Read the full abstract ▾
All authors Yijie Zhu, Zitong Yu, Wei Li, Hui Ma, Wen Li, Rui Shao, Liqiang Nie
Computer Vision
3DHarnessBench: Probing Agentic 3D-to-Code Capabilities of Frontier Vision-Language Models
Ling Liu, Bingchen Gong, Amal Dev Parakkat et al. (+1 more)
We introduce 3DHarnessBench, a
benchmark that evaluates the agentic ability of frontier
vision-
language models (VLMs) to recover 3D geometry as Blender Python code from a variety of inputs. Unlike previous frameworks that
prompt the VLMs with a fixed input (e.g., a single rendering or a text description), 3DHarnessBench evaluates four separate harness settings that progressively enable active agentic exploration, facilitated by recent Blender MCP functionality. Our hierarchy from Single-view, Multi-view, Active Visual (arbitrary viewpoint access), and Full 3D Interaction (complete access to the target object through Blender function calls) probes the models' abilities in both visual perception and active inference, tool calling, and self-correction. We observe that the ability of all frontier models to recover 3D geometry improves significantly with richer function call access, although the improvements are strongly model-dependent, revealing highly uneven agentic 3D-to-code capabilities. We will release the
benchmark, code, outputs, and
agent trajectories for reproducible 3D evaluation.
Read the full abstract ▾
All authors Ling Liu, Bingchen Gong, Amal Dev Parakkat, Maks Ovsjanikov
Computer Vision
Selective Knowledge Control for Continual GUI Agent Learning over Application Streams
Zirui Shang, Xin Shu, Yang Liu et al. (+3 more)
Continual learning is a crucial capability for Graphical User Interface (GUI)
agents to adapt to evolving applications while retaining knowledge acquired from previous applications. Such application streams pose a challenging knowledge modeling problem: new applications often share underlying knowledge with past ones, yet also introduce specific knowledge that must not interfere with historical knowledge. In this paper, we propose activation-conditioned selective knowledge control, a lightweight method that achieves selective knowledge retention via
neuron-level gradient manipulation. Our method maintains a compact historical knowledge state to protect highly activated MLP
neurons that preserve previous knowledge. When a new application arrives, it performs real-time gradient surgery conditioned on forward activation. Concretely, the protected
neurons are categorized into two types: unactivated
neurons holding specific knowledge, whose gradients are truncated to prevent interference; and activated
neurons holding shared knowledge, whose gradients are orthogonally projected to preserve stability while enabling adaptation. After each application stage, newly identified critical
neurons are merged into the historical state for future learning. Empirical evaluations on multi-app sequential
benchmark demonstrate that our method effectively mitigates catastrophic forgetting on prior applications while sustaining robust adaptation to new ones.
Read the full abstract ▾
All authors Zirui Shang, Xin Shu, Yang Liu, Zhi Gao, Xinxiao Wu, Lifeng Fan
Computer Vision
A Cloud-Based Hybrid Model for Real-Time Detection of BRTA-Approved Licence Plates Using YOLO Tiny and Haar Cascade
Debashis Kar Suvra and Tahsina Farah Sanam
Accurate vehicle license plate detection is essential for applications such as intelligent transportation systems, toll collection, parking management, and law enforcement. In Bangladesh, this task presents distinct challenges due to the complexity of localized license plates and environmental factors like lighting, occlusion, motion blur, and obstructions such as dirt or mud. These challenges often render conventional methods ineffective. This paper introduces a novel hybrid approach, combining the YOLO Tiny deep learning model with the Haar-Cascade classifier, for enhanced detection and localization of Bengali license plates. A key innovation of our system is the integration of a dynamic retraining pipeline, which allows the model to adapt to evolving real-world conditions. This retraining mechanism significantly boosts performance in low-confidence scenarios by continuously improving the model's accuracy as new data is encountered. Additionally, a publicly accessible
dataset of BRTA-compliant license plates, captured under diverse and challenging conditions, has been developed to support this approach. Experimental results demonstrate that our approach not only achieves superior detection accuracy and computational efficiency over conventional models but also ensures consistent performance in resource-constrained environments, particularly in Bangladesh.
Read the full abstract ▾
Computer Vision
CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying
Yizhou Tian, Zizhe Chen, Shiyuan Deng et al. (+7 more)
Memory facilitates question answering over long
videos by extracting and retrieving facts to fit within the limited context windows of
multimodal LLMs (MLLMs). Existing solutions typically extract independent
memory entries from fixed-length
video clips and thus cannot capture high-level semantics that need to be summarized over extended time periods, such as character traits and relations. Moreover, they rely solely on similarity-based
retrieval and may fail to retrieve the fine-grained details required for question answering. To tackle these problems, we propose CAM, featuring continuous extraction for high-level semantics and adaptive querying for fine-grained details. In particular, CAM stores the entities and relations extracted from
video clips in a knowledge graph. To capture the high-level semantics of each entity or relation, CAM summarizes the local subgraph of the target entity or relation once the subgraph reaches a predefined size. To retrieve the fine-grained details required for question answering, CAM supports multiple search methods, including knowledge graph traversal,
video re-watching, and audio listening. It utilizes a planner-executor-verifier pipeline to adaptively compose these search methods according to question intent. Evaluations on three
benchmarks show that CAM outperforms SOTA baselines and improves their accuracy by up to 23 percentage points. Code is available at https://github.com/Jake-Tian/CAM.
Read the full abstract ▾
All authors Yizhou Tian, Zizhe Chen, Shiyuan Deng, Garry Yang, Zijie Dai, Luohao Pan, Hao Lin, Peiqi Yin, Xiao Yan, James Cheng
Computer Vision · Cross-listed from cs.MM
Xiaoran Kou, Jingyi Wu, Peng Sun et al. (+2 more)
Multimodal sentiment analysis requires effective modeling of both verbal semantics and non-verbal affective cues. A central challenge is to calibrate text-centered sentiment understanding with visual facial evidence in a controlled, adaptive, and parameter-efficient manner. Text usually serves as the semantic anchor, whereas visual cues provide complementary evidence for ambiguous or implicit expressions; however, indiscriminate fusion may introduce visual noise and distort textual semantics. Moreover, fully fine-tuning large visual and textual encoders is costly and prone to overfitting on limited and scenario-dependent MSA
benchmarks. To address these issues, we propose
Vision-Guided Text
Prompt Tuning (VG-TPT), which formulates visual-text sentiment modeling as controllable visual calibration of frozen text representations. VG-TPT injects visual affective cues into a frozen BERT encoder through layer-wise adaptive
prompts, rather than relying on late-stage feature fusion or full backbone tuning. A co-guided router composes
prompts from a trainable
prompt bank according to both the evolving text state and the visual guidance feature, enabling sample-specific and layer-specific modulation. Experiments on CMU-MOSEI and CMU-MOSI show that VG-TPT consistently improves over text-only baselines and achieves competitive or superior performance compared with several full-modality methods, while updating only 2.4M trainable parameters. The code is available at https://github.com/ma-tubu/VG-TPT.
Read the full abstract ▾
All authors Xiaoran Kou, Jingyi Wu, Peng Sun, Yang Liu, Hong Chen
Computer Vision
Diffuse2Seg: Diffusion Models Can Segment Anything Without Supervision
Christoph Hümmer, Joachim Sicking, Fabian Hüger et al. (+1 more)
Open-world entity
segmentation aims to predict masks for arbitrary objects across domains and at multiple granularities, from parts to whole objects. In this setting, SAM sets a strong standard: trained on SA-1B, comprising 11M images and over 1B carefully annotated masks, it achieves remarkable zero-shot performance. Collecting such labels is expensive and time-consuming, however, which limits how far this recipe can scale. Text-to-image
diffusion models offer a way around this. Their intermediate features transfer well across perception tasks, and since object structure emerges as the model denoises a noise sample into an image conditioned on a text
prompt, that structure is already encoded in these representations. They can therefore be exploited for open-world entity
segmentation without retraining or supervision. Building on this observation, we introduce Diffuse2Seg, which repurposes generative
diffusion models for automatic mask generation by propagating a grid of point
prompts through their self-
attention representations in an edge-preserving manner. Diffuse2Seg produces multi-granular instance masks and outperforms prior state-of-the-art label generators by 4.3-7.1 p.p. in AR_1000 across five domains. Training an instance
segmentation model on these generated masks advances detector-free open-world
segmentation by 7.4 and 7.7 p.p. on "things" and "stuff+things"
datasets and surpasses the detector-based UnSAM on "stuff+things" by 2.1 p.p. in AR_1000. Finally, we show that a model trained on Diffuse2Seg labels provides a strong initialization for semi-supervised learning, outperforming its fully supervised counterpart with already 5k labeled images.
Read the full abstract ▾
All authors Christoph Hümmer, Joachim Sicking, Fabian Hüger, Hanno Gottschalk
Computer Vision
OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolution
Shubhashis Roy Dipta, Sourajit Saha, Shaswati Saha et al. (+1 more)
Recursive Super-Resolution (SR) extends fixed-scale SR to extreme magnification by repeatedly feeding predictions back into the same model, analogous to zooming an image repeatedly. However, ground truth availability at every scale, especially at depth, remains challenging as the required source resolution grows geometrically, leaving deeper predictions unsupervised. We present OracleZoom, an on-policy
distillation-inspired, reference-constrained framework that trains on its trajectory while carrying the last ground-truth evidence beyond the supervision boundary. Direct and cross-scale supervision constrain verifiable content, while a no-reference quality objective guides unresolved fine-scale detail. A KL-constrained pretrained latent prior limits quality-driven drift, while EMA consistency stabilizes the supervision boundary. Across seven
datasets, OracleZoom achieves the state-of-the-art SR quality across zooming scales, averaging 0.713 CLIPIQA, with larger gains on deeper scales, while significantly reducing
hallucinations. Code, data, and models are available at https://dipta007.github.io/OracleZoom/ .
Read the full abstract ▾
All authors Shubhashis Roy Dipta, Sourajit Saha, Shaswati Saha, Nobin Sarwar
Computer Vision
One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints
Shiqi Pan, Qi Zheng, Hanqin Sun et al. (+3 more)
Vision-and-
Language Navigation in Continuous Environments (VLN-CE) requires an
embodied agent to navigate unseen environments by following natural
language instructions. Current zero-shot VLN-CE methods either rely on pre-trained waypoint predictors or require multiple queries to large models per step. To address prohibitive inference latency and computational overhead, we propose O2C-Nav, an efficient zero-shot
navigation framework that calls only a single large model once per decision step. Our approach introduces a training-free structured waypoint generator and a novel abstract representation that projects sparse, history-aware candidate waypoints directly onto RGB images as visual markers. The MLLM selects a waypoint or generates a fallback target bounding box at each step, while a low-level Fast Marching Method (FMM) planner converts the selected target into an executable collision-free path. This paradigm provides the model with concrete spatial perception and explicit
memory while significantly reducing the visual processing load. Extensive evaluations on the R2R-CE and RxR-CE
benchmarks demonstrate that O2C-Nav outperforms current state-of-the-art zero-shot methods, highlighting its great potential for real-time robotic deployment. Code is available at https://github.com/kkpsq/O2C-Nav-Code.
Read the full abstract ▾
All authors Shiqi Pan, Qi Zheng, Hanqin Sun, Youjian Zhang, Daquan Feng, Xu Wang
Computer Vision
Thinking with Cameras: Active Visual Reasoning via Dynamic Viewpoint Control for Surveillance Video Understanding
Xiao Zhang, Wang Zeng, Sheng Jin et al. (+3 more)
Large
vision-
language models (LVLMs) have recently achieved remarkable progress in general-purpose
video understanding. However, their application to real-world surveillance remains challenging due to the lack of large-scale domain-specific
datasets and the limitation of passive observation from fixed viewpoints. In surveillance scenarios, critical visual evidence can be easily missed when targets are distant, small, occluded, or move beyond the current camera view. In this work, we introduce CamVLM, a new framework for Thinking with Cameras, which enables LVLMs to actively acquire visual evidence in real-world surveillance by continuously controlling camera viewpoints. We first construct CCTV-Anomaly, a large-scale surveillance
video understanding
dataset containing 14,133
videos across 10 anomaly categories, with detailed captions and event annotations. We further formulate viewpoint control as an active visual perception problem and build CamTrack-53K, an object-centric viewpoint trajectory
dataset for learning camera actions. Moreover, we propose a reinforcement learning based viewpoint policy optimization framework, which models camera control as a sequential decision-making process and learns long-horizon observation strategies beyond supervised trajectory imitation. Extensive experiments demonstrate that CamVLM achieves state-of-the-art performance under both passive observation and dynamic viewpoint settings, validating the effectiveness of active camera-based
reasoning for surveillance
video understanding. Our
datasets, model, and code will be available at https://github.com/xiaozhang79/CamVLM.
Read the full abstract ▾
All authors Xiao Zhang, Wang Zeng, Sheng Jin, Wentao Liu, Chen Qian, Shichao Kan
Computer Vision
Large-Scale Pretraining for Improving Deep Learning-Based Geometric Distortion Correction of Diffusion-Weighted Imaging
Saroj Khanal, Yashawant Kumar Yadav, Kritam Bhattarai et al. (+12 more)
Diffusion-weighted imaging (DWI) is widely used in clinical settings but remains vulnerable to
geometric distortion. Conventional correction methods often require additional acquisitions or vendor-specific solutions, limiting their feasibility in high-throughput, resource-constrained settings. This study investigates whether large-scale pretraining strategies can improve deep learning-based distortion correction for single-phase-encoding DWI. We formulate the task as image reconstruction, and compare a non-pretrained baseline against a self-supervised and a generative pretrained model, evaluated using both quantitative image-similarity metrics and qualitative expert assessment. The best-performing model was further tested for transferability on data collected in an LMIC setting with acquisition shift. Pretrained models outperformed the non-pretrained baseline, with cWDM achieving the strongest results across both quantitative and qualitative evaluation. However, application to LMIC data revealed transferability challenges, including contrast alteration and over-reliance on T1-weighted anatomical structure. Registering images to a common standard space improved predictions, suggesting that harmonized preprocessing may enhance cross-domain deployment.
Read the full abstract ▾
All authors Saroj Khanal, Yashawant Kumar Yadav, Kritam Bhattarai, Jeevan Neupane, Shristi Subedi, Saship Gwachha, Manish Kumar Tiwari, Dong Zhang, Confidence Raymond, Aondona Moses Iorumbur, Udunna Anazodo, Surendra Maharjan, Bishesh Khanal, Mahesh Shakya, Pralhad Kumar Shrestha
Computer Vision
PLSR: Progressive and Localized Super-Resolution of 3D Objects via Localized Latent Voxel Diffusion
Yuxin Liu, Minshan Xie, Jiawen Liang et al. (+3 more)
High-resolution 3D asset generation is vital in various 3D applications. Existing state-of-the-art
diffusion-based models remain constrained by fixed resolutions, limiting their ability to produce details. In this paper, we tackle the challenge of generating more detailed, higher-resolution 3D objects by introducing a 3D super-resolution (SR) framework built on existing 3D generative foundation models. To this end, we design PLSR, a progressive and localized super-resolution solution to achieve this goal effectively and
memory efficiently. Technically, given a coarse geometry from a pretrained 3D generator, we decompose the global SR task into localized sub-tasks via an associative input decomposition scheme, adapt a flow-based 3D generator into a localized super-resolution model through low-cost finetuning, and unify them in an iterative patch-wise denoising pipeline for seamless high-resolution output. Experiments on challenging objects show that our approach is able to generate 3D details with new strong fine-detail fidelity while significantly reducing the computational cost, offering a new and practical solution for high-resolution 3D asset generation.
Read the full abstract ▾
All authors Yuxin Liu, Minshan Xie, Jiawen Liang, Runsong Zhu, Chi-Wing Fu, Tien-Tsin Wong
Computer Vision
Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Models
Yangyang Xie, Ke Hao, Jiaqi Liu et al. (+2 more)
Medical
vision-
language models (VLMs) require confidence that reflects both answer correctness and patient-specific visual evidence. Recent GRPO-based methods optimize verbalized confidence together with answer generation. However, this joint optimization may interfere with answer learning and drive confidence toward near-binary values. Verbalized confidence also provides no explicit assessment of visual support. We therefore separate capability learning from confidence estimation and propose DualRead. DualRead builds on the insight that reliability can be read from the actor's internal states at critical moments in the answering process. It freezes the GRPO-trained actor and combines pre-answer solvability with a post-answer assessment of the generated answer and its visual support. To further assess whether confidence reflects visual grounding, we introduce Counterfactual Confidence Grounding AUC (CCG-AUC). It measures whether confidence decreases when real-image substitution changes the actor from correct to incorrect. Across two VLM backbones and both in- and out-of-distribution medical VQA
benchmarks, DualRead improves correctness discrimination and calibration over verbalized confidence while preserving answer accuracy. CCG-AUC reveals whether confidence responds to answer-relevant visual evidence rather than primarily to non-visual cues.
Read the full abstract ▾
All authors Yangyang Xie, Ke Hao, Jiaqi Liu, Yun Gu, Xinglin Zhang
Computer Vision · Cross-listed from cs.CL
Visual Search Augmented Chain-of-Thought Reasoning for Attribute Value Extraction from Product Videos
Tong Wu, Ming Cheng, Jiazhen Hu et al. (+2 more)
Existing approaches to visual attribute value extraction (AVE) primarily rely on static product images, failing to capture temporal cues, multi-angle views and fine-grained visual details. Directly applying
video vision-
language models (VLMs) to product AVE results in limited performance due to the lack of domain knowledge, and fine-tuning them requires extensive high-quality data and substantial computational resources. Thus, we propose visual search augmented chain-of-thought
reasoning (ViS-CoT), a training-free, plug-and-play pipeline that can be easily applied to any open-source
video VLM for
video-to-text AVE in e-Commerce. Specifically, ViS-CoT employs visual clustering to identify representative frames, followed by visual search to retrieve semantically similar product knowledge that can enrich attribute cues. Next, an interleaved CoT
reasoning module iteratively refines
reasoning through visually-aligned auxiliary texts derived from captioning and automatic
speech recognition. Finally, the integrated information guides the model toward accurate and fine-grained attribute predictions. Extensive experiments across 14 product categories on the VideoAVE
dataset show that ViS-CoT consistently enhances multiple state-of-the-art
video VLMs, achieving an average improvement of 17.91 percentage points in micro-F1.
Read the full abstract ▾
All authors Tong Wu, Ming Cheng, Jiazhen Hu, Jiaying Gong, Hoda Eldardiry
Computer Vision
Scaling 3D Generative Priors to Large-Scale Scene Meshes from Multi-View Images
SangEun Lee, Wonseok Chae, Hoyoung Yoo et al. (+3 more)
Pretrained 3D generative models produce detailed geometry and appearance but are primarily designed for object-centric generation within a limited spatial extent. Recent approaches address this limitation by partitioning large scenes into smaller spatial regions and applying pretrained 3D generative priors to each region. However, scaling tiled generation to large multi-view scenes makes it challenging to maintain local
geometric continuity and global appearance consistency. We present a training-free framework for large-scale textured mesh generation from multi-view images. Our key idea is to scale tiled generation to large scenes with increased spatial detail while coordinating generation both locally and globally. We introduce local context tiled generation to improve
geometric continuity between neighboring regions and global appearance
alignment to reduce appearance discrepancies across distant regions. An adaptive scene decomposition further determines the number of tiles according to the input scene geometry. Experiments demonstrate improved
geometric and appearance fidelity over existing approaches while enabling fine-grained generation of large-scale scenes.
Read the full abstract ▾
All authors SangEun Lee, Wonseok Chae, Hoyoung Yoo, Geunyong Kim, NackWoo Kim, Hyeonjin Kim
Computer Vision
An Integrated Video-AI Platform for Action-Level Microanastomosis Training and Performance Feedback
Yan Meng and Daniel A. Donoho
Developing microanastomosis skill requires repeated practice with timely, action-specific feedback, yet expert review of lengthy microscope
videos does not scale to frequent or distributed training. We present an integrated
video-AI platform that turns a complete simulated procedure into inspectable, interactive feedback through three connected modules. First, a proposed
transformer segments the
video into six
surgical actions. Second, object detection and tracking localize instrument tips within each action; the resulting kinematic features and action statistics drive supervised classification of five NOMAT-aligned performance dimensions. Third, a grounded large
language model (LLM) uses these structured outputs to answer user questions about the current scene, actions, motion, and predicted performance through a unified interface. In a two-site study, 17 participants completed 72 procedures comprising 576 suture placements. The action-
segmentation module achieved 87.66% accuracy and 82.86% F1, increasing to 93.62% and 88.32% after workflow-aware refinement. The five performance classifiers achieved 76.0% mean accuracy, with Cohen's $\kappa$ from 0.63 to 0.93. Although the
language interface and educational benefit require prospective evaluation, these results establish the technical basis for an expert-supervised platform that can shorten review, expose the evidence behind performance estimates, and support scalable formative microsurgical training.
Read the full abstract ▾
Computer Vision
Multi-Grid Post-Training for Long-Form Multi-Shot Video Generation
Jiawei Mao, Haoqin Tu, Hardy Chen et al. (+8 more)
Generating long-form multi-shot
videos requires coherent within-shot motion and visually consistent narratives across shots. Existing
video generators favor continuous motion and struggle to present complete shot sets when an entire narrative is packed along one temporal axis. We propose MovieGrid, a Multi-Grid Post-Training paradigm that decomposes a long
video into shorter, temporally ordered chunks and arranges them on a spatial grid for joint modeling. This design reduces the number of shots handled by each temporal axis while enabling global information exchange across chunks. We construct the Multi-Grid Long
Video (MGLV)
dataset from 1,000 long-form
videos using source
video collection, hierarchical
segmentation, grid
video construction, and character-aware story annotation, producing 54K grid
videos paired with story
prompts. Our Noise-Free Random-Grid Training retains a random subset of chunks as clean visual context for denoising the remaining chunks. Grid
Embedding encodes grid structure, character-aware Story
Prompts link recurring entities, and Grid Boundary Loss stabilizes layouts. Under the same
token budget, MovieGrid generates 6.05 times more shots than Temporal Packing in a 1,616-frame
video. On a
benchmark spanning five real-world categories, it achieves state-of-the-art intra-shot consistency (0.9131 versus 0.8086 for HoloCine) and inter-shot consistency (0.5914 versus 0.5384 for StoryMem). MovieGrid can further scale
video length with minimal compromise through single or multiple generations.
Read the full abstract ▾
All authors Jiawei Mao, Haoqin Tu, Hardy Chen, Yuhan Wang, Keyang Xu, Jieru Mei, Hongliang Fei, Ruogu Fang, Wei Shao, Cihang Xie, Yuyin Zhou
Computer Vision
DualPathOcc: Dual-Resolution BEV Encoder for 3D Occupancy Prediction
Lihao Qiu, Jian Chen, Ruihao Wang et al. (+3 more)
Predicting 3D occupancy from multi-view images requires preserving
geometric detail during 2D-to-3D lifting while
reasoning over sparse, volumetric scene representations. We present DualPathOcc, a camera-based framework that combines a Spatial Enhancer for high-resolution feature aggregation before BEV compression, a SENet-augmented dual-path BEV encoder for local-global context modeling, and height-aware weighted cross-
entropy for near-ground occupancy. The final model is optimized with occupancy supervision and no explicit depth loss. On single-frame Occ3D-nuScenes, DualPathOcc achieves 37.37 mIoU. We further analyze how surface-centered depth targets interact with volumetric occupancy learning.
Read the full abstract ▾
All authors Lihao Qiu, Jian Chen, Ruihao Wang, Ramu Gautam, Mei Yang, Yingtao Jiang
Computer Vision
NOVA: Normal-Side Modeling for Training-Free Zero-Shot Video Anomaly Detection
Wei-Chih Yin, Yun-Ching Kao, Cheng-Kuan Lin et al. (+1 more)
Training-free zero-shot
video anomaly detection (ZS-VAD) leverages
vision-
language models (VLMs) to localize anomaly instances from a predefined anomaly vocabulary, without providing any
video. Existing CLIP-based methods often emphasize anomaly-side semantics, while the competing normality side remains less carefully formulated. We identify two key limitations in existing solutions: (i) blurred decision boundary: normal
prompts may contain ambiguous verbs, such as running, that are semantically close to anomalies, reducing normal and abnormal separation in the VLM
embedding space; and (ii) modality gap: poor
alignment between features of textual normal anchors and visual frames. We propose NOVA, a training-free ZS-VAD framework that strengthens the normal side at both linguistic and visual levels. NOVA introduces Normality-Aware
Prompt Construction (NA), which excludes anomaly-adjacent verbs and biases normal descriptions toward static, low-motion scenes. To overcome the text-
vision modality gap, NOVA constructs a Visual Normality Anchor (VNA), which creates a weighted visual normal anchor from the initial frames of each test
video, providing a
video-specific normal reference without task-specific training or annotations. NOVA achieves 89.86 percent AUC on UCF-Crime and 95.07 percent AUC and 84.82 percent AP on XD-Violence, reaching state-of-the-art performance among comparable training-free zero-shot methods.
Read the full abstract ▾
All authors Wei-Chih Yin, Yun-Ching Kao, Cheng-Kuan Lin, Yu-Chee Tseng
Computer Vision
AGSA-Net: Abundance-Guided Self-Attention Network for Spectral Unmixing-Aware Hyperspectral Remote Sensing Image Classification
Nafisa Anjum, Satavisa Dey Borno, Ananna Saha et al. (+4 more)
Hyperspectral image (HSI) classification plays a vital role in remote sensing applications, including agriculture, environmental monitoring, and urban analysis. However, its performance remains challenged by high spectral redundancy, noise sensitivity, and the difficulty of jointly modeling local material composition and long-range spectral dependencies. To address this, we propose AGSA-Net, an abundance-guided self-
attention network that explicitly integrates spectral unmixing priors into the classification process. AGSA Net first estimates physically meaningful subpixel abundance maps subject to non-negativity and sum-to-one constraints, regularized by hybrid linear-nonlinear reconstruction decoder. The learned abundances are then used to construct an abundance affinity prior that guides a spectral
transformer to emphasize class-discriminative interactions, and the resulting
transformer features are fused with compact abundance descriptors for final prediction; in contrast to existing approaches that use abundance as auxiliary or concatenated features. Experiments on Indian Pines, Augsburg, and Berlin demonstrate the benefit of incorporating abundance- guided contextual modeling, particularly in heterogeneous urban scenes. The source code and trained models are available at: https://github.com/nnuvi/AGSA-Net
Read the full abstract ▾
All authors Nafisa Anjum, Satavisa Dey Borno, Ananna Saha, Mir Faiyaz Hossain, Sifat Momen, Nabeel Mohammed, Shafin Rahman
Computer Vision
MSCA-UNet: Multi-Scale Context and Attention U-Net for Image Segmentation
Sheng-Wei Chan
U-Net remains a practical baseline for image
segmentation because of its simple encoder-decoder structure and skip connections. However, the bottleneck representation is still dominated by a limited set of receptive fields, while decoder features are propagated without explicitly emphasizing the most informative channels and spatial locations. This paper presents MSCA-UNet, a U-Net-based
segmentation architecture that combines multi-scale contextual aggregation at the bottleneck with channel-spatial
attention refinement in the decoder. The multi-scale module uses parallel atrous convolutions to capture contextual features at different receptive fields, while Convolutional Block
Attention Modules (CBAMs) progressively recalibrate decoder features. Under identical experimental settings, the baseline U-Net achieves 96.9% mIoU on a held-out test set. Adding multi-scale context improves mIoU to 97.5%, while
attention alone reaches 98.4%. Combining both mechanisms yields 99.1% mIoU, a 2.2 percentage-point improvement over the baseline. Parameter analysis further shows that the
attention-only variant adds approximately 0.044M parameters, whereas the multi-scale module contributes most of the additional model capacity. The results support the view that multi-scale context enrichment and
attention-based feature refinement provide complementary benefits within a U-Net framework.
Read the full abstract ▾
Computer Vision
ChildGaze: A Benchmark Dataset for Collaborative Behavior Understanding in Children
Sindhuja Penchala, Saketh Reddy Kontham, Prachi Bhattacharjee et al. (+7 more)
Understanding collaborative behavior in children is important for analyzing social participation, peer interaction, shared
attention, and engagement during play and learning activities. Reliable recognition of these cues can support research in child development, educational analysis, and human-centered computer
vision. However, estimating where a child is looking does not necessarily reveal whether the child is actively participating in a shared activity. To support this higher-level analysis, we introduce ChildGaze, a child-centered behavioral annotation
dataset built on the ChildPlay
video collection [1]. ChildGaze introduces two behavioral labels, collaborative and non-collaborative, assigned independently to each child within a frame. The
dataset provides face, left-hand, and right-hand bounding boxes for children and adults and organizes the annotations at the row, person, and frame levels. The current release contains 27 annotated
video files, 10,641 frames, and 73,268 body-part annotation rows. Annotation reliability was evaluated on 1,187 frames using independent annotations from two annotators. The collaboration labels achieved 93.16% raw agreement and a Cohen's kappa of 0.8631, while bounding-box annotations achieved an overall mean IoU of 0.808. Baseline experiments with pretrained ViT and Swin
Transformer models achieved up to 97.44% child-person-level accuracy and 96.80% frame-level accuracy, respectively. These results show that ChildGaze provides a reliable
benchmark for studying collaborative behavior in naturalistic child-adult and peer interactions.
Read the full abstract ▾
All authors Sindhuja Penchala, Saketh Reddy Kontham, Prachi Bhattacharjee, S. Nima Mahmoodi, Daniel Fonseca, Sareh Karami, Mehdi Garemani, Sudip Mittal, Shahram Rahimi, Noorbakhsh Amiri Golilarz
Computer Vision
Radiation, Rotation and Scale Invariant Feature Descriptor for Multimodal Image Matching
Yuanxin Ye, Tengfeng Tang, Tao Peng et al. (+3 more)
Multimodal image matching is a fundamental task for multi-source information fusion. However,
geometric distortions and nonlinear radiometric differences (NRD) severely limit performance, especially under radiometric, rotation, and scale variations. To address this issue, we propose a radiation, rotation, and scale invariant (RRSI) feature descriptor. First, a dual-head regional sampling (DHRS) module simultaneously performs Cartesian and Log-Polar sampling on keypoint neighborhoods, retaining spatial structural properties while enhancing robustness to rotation and scale variations. We then jointly encode
geometric and radiometric relations between
multimodal images in a unified deep feature space, enabling feature encoding, interaction, and fusion across intra-modal, dual-head sampled, and inter-modal regions. Furthermore, we introduce a bidirectional cross-modal generative reconstruction constraint during training. By decoding implicit features into structural patches of the counterpart modality, this mechanism anchors modality-invariant
geometric topologies without additional inference overhead. Experiments on optical-infrared and optical-SAR
datasets demonstrate highly competitive matching performance and strong robustness to rotation and scale variations. RRSI supports the full rotation range from 0 to 360 degrees and scale factors up to four. Its generalization ability is further validated on
multimodal images from computer
vision, remote sensing, and medical imaging. The implementation will be made publicly available at https://github.com/yeyuanxin110/RRSI .
Read the full abstract ▾
All authors Yuanxin Ye, Tengfeng Tang, Tao Peng, Zhiqiang Han, Jiayuan Li, Mi Wang
Computer Vision
One Shared LoRA Weight for MRI Reconstruction across Acceleration Factors
Zhiwei Zhao, Weikang Gong, Zhongnian Li et al. (+1 more)
Accelerated MRI reconstruction recovers images from undersampled k-space. However, different acceleration factors produce distinct artifact patterns. Existing methods often train separate models for each factor, leading to poor cross-factor generalization and high training and storage costs. We propose Shared LoRA, a parameter-efficient framework that freezes the pretrained SHFormer backbone and trains a single shared set of LoRA adapters together with a lightweight gating network. During training, undersampled inputs are generated by randomly sampling acceleration factors and their corresponding sampling masks, enabling the shared adapters to learn reconstruction knowledge across factors. Given the acceleration factor, GateNet generates layer-wise coefficients to dynamically modulate the residual strength of each adapter. Experiments show that Shared LoRA achieves the best or competitive PSNR and SSIM across acceleration factors, while its trainable parameters account for only about 5.3% of the total model parameters. Its performance at lower acceleration factors remains largely unaffected as the jointly trained factor set expands, and it generalizes stably to unseen neighboring factors.
Read the full abstract ▾
All authors Zhiwei Zhao, Weikang Gong, Zhongnian Li, Xinzheng Xu
Computer Vision · Cross-listed from astro-ph.GA
Deep learning from the crowd Fundamentals of morphological galaxy classification
Luis Enrique Sucar, Carlos del Burgo and Jonathan Serrano-Pérez
Aims. The objective of this work is to adapt a deep neural network model to perform
galaxy morphological classification trained from crowd annotations, considering the training scheme, the agreement between the annotators, and the hierarchy. Methods. We use
Galaxy Zoo 1 as our experimental testbed and trained a convolutional neural network (CNN) for the automatic classification of
galaxies' morphologies. We analyze the impact of the following aspects on the classification accuracy and training efficiency: (i) Training only the last layer vs. training all the network; (ii) Classification with only the CNN vs. considering the hierarchy; (iii) Comparing the models trained with different amounts of data and levels of agreement between the annotators; (iv) Training by stages, transferring knowledge from one model to another; and (v) Combining several models as an ensemble. Results From the experiments, we derive the following results: (i) Training all the layers in the network significantly improves the accuracy (10% increase in exact match), compared to training only the last layer; (ii) There is a tradeoff between the amount of data and the level of agreement between the annotators used for training; (iii) Using the hierarchy can improve accuracy when the amount of training data is reduced; (iv) Training by stages through transfer learning (curriculum learning) produces higher accuracy for limited data; (v) Ensembles can improve accuracy; (vi) Models achieve a low accuracy for the most difficult cases, but, if we consider hierarchical measures, we can derive useful results for upper levels in the hierarchy. An accuracy above 99% is achieved when training all layers of the network and considering a high agreement between the annotators. Conclusions. Training deep learning models from crowd annotations involves additional challenges than learning from hard annotations.
Read the full abstract ▾