Computer Vision
Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget
Guoxuan Chen, Chufeng Xiao, Haoran Yang et al. (+30 more)
We introduce Boogu-Image-0.1, an open-source unified
multimodal understanding and generation model family, comprising Base, Turbo, Edit, and Edit-Turbo variants. It delivers competitive performance in high-quality text-to-image generation, fast inference, instruction-based editing, and bilingual (Chinese-English) text rendering. Closed-source
multimodal systems like Nano-Banana-Pro and GPT-Image-2 achieve strong performance through system-level integration rather than a single model, yet their internal practices remain largely undisclosed. In this work, we demonstrate that strengthening the understanding capability of the system, through a stronger
multimodal encoder, agentic
prompt rewriting, and related techniques, together with improvements in data quality, training pipelines, and agentic inference-time scaling, can substantially enhance generation and editing performance even under highly constrained compute budgets. Comprehensive evaluations show that Boogu-Image-0.1 consistently matches or surpasses other open-source models across standard
benchmarks, and achieves results approaching leading closed-source systems. Notably, this is accomplished with only 208.62 million unique images. The base model's theoretical training cost is only approximately \$400K. We share practical discussions that we believe are valuable to the broader research community, and release weights, code, and recipes under Apache 2.0 to advance the open ecosystem for unified
multimodal understanding and generation. Our code is available here: https://github.com/Boogu-Project/Boogu-Image.
Read the full abstract ▾
All authors Guoxuan Chen, Chufeng Xiao, Haoran Yang, Siyue Xie, Binxiao Huang, Ming Zhang, Cheuk Him Chau, Xinyu Fu, Yingzhao Lian, Tom S.Y. Li, Jintao Lin, Bowen Dong, Zian Qian, Yuhao Liu, Yuxuan Hu, Weikang Shi, Bin Zou, Bowen Zheng, Haoxuan Che, Chang Chen, Yuyang He, Heyang Sun, Tianyu Huang, Chong Hou Choi, Cheng Gong, Han Shi, Haoli Bai, Xihui Liu, Hongsheng Li, Qifeng Chen, Chao Huang, Rui Liu, Chenyang Lei
Computer Vision
C-Norm: Cell-Distribution Normalization Enables Precision Recognition of Medical-Cell Image
Yang Qianl, Liu Xiany, Dai Daw et al. (+5 more)
ThinPrep Cytologic Test (TCT) enables early cervical cancer screening, but manual reading is time-consuming and yields inconsistent diagnostic results among cytopathologists. Existing AI detection models perform poorly under real clinical conditions, primarily restricted by two key constraints: unbalanced spatial distribution of
cell populations in TCT slides, and limited high-quality annotated cytology data relying on professional pathologist labeling. To address these limitations, we propose a
Cell-Distribution Normalization (C-Norm) method. By decoupling abnormal and normal
cells from the original TCT images and re-synthesizing them, this method ensures a uniform distribution of
cell populations, thereby mitigating generalization degradation caused by distribution bias. Building upon this, we integrate the YOLOv12 framework with a DINOv3 module. This hybrid architecture leverages the advanced detection capability of YOLO models and the superior feature representations of DINOv3 to capture subtle morphological nuances essential for precise recognition of TCT images. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance, significantly outperforming mainstream detection
algorithms. The complete implementation is available at: https://github.com/ddw2AIGROUP2CQUPT/
Cell-Norm
Read the full abstract ▾
All authors Yang Qianl, Liu Xiany, Dai Daw, Chen Jing, Shen Xiaoj, Fu Kaiw, Tang Ming, Zou Dongl
Computer Vision · Cross-listed from cs.LG
The Seriality Gap in Video Diffusion Models
Jorge Diaz Chao, Konpat Preechakul, Yuxi Liu et al. (+1 more)
When one ball strikes another, then another,
video models should predict the consequences of each bounce. In controlled experiments on multi-ball hard-sphere dynamics, we find that the performance of standard bidirectional
video diffusion degrades as the causal chain lengthens, even when provided more denoising steps. In a length-matched single-ball control, where ball-ball interactions are absent, the degradation largely disappears, isolating dependent-event structure rather than
video length as the cause. Across intervention studies, methods that increase effective serial computation improve performance disproportionately, including autoregressive/blockwise generation and architectural depth. We identify this pattern as the seriality gap: a mismatch between tasks requiring growing serial computation and
video diffusion models whose denoising loop does not provide scalable serial compute. We then prove that, for deterministic
video prediction, denoising steps do not add serial computation beyond the backbone, indicating a structural obstacle for
video diffusion on serial
reasoning and simulation tasks.
Read the full abstract ▾
All authors Jorge Diaz Chao, Konpat Preechakul, Yuxi Liu, Yutong Bai
Computer Vision · Cross-listed from cs.RO
FlowWAM: Optical Flow as a Unified Action Representation for World Action Models
Yixiang Chen, Peiyan Li, Yuan Xu et al. (+13 more)
World Action Models (WAMs) are able to leverage pretrained
video generators for both world modeling and action prediction. However, directly leveraging such
video generators for control raises a new challenge: how to represent actions in a suitable form that aligns with pretrained
video generators while carrying enough motion cues for accurate control. Existing numerical actions fail to satisfy the former, and prior visual action representations overlook the temporal motion structure across frames. We address this issue with FlowWAM, a dual-stream
diffusion framework that adopts optical flow as a unified,
video-native action representation. Flow
videos share the same format as RGB
videos and encode rich per-pixel displacement. By jointly modeling them within a shared pretrained
video generator, FlowWAM can naturally implement two modes of WAMs. In policy mode, FlowWAM generates flow for action prediction, while in world-model mode, it uses target flow sequences to guide future
video generation. Moreover, since flow can be easily extracted from raw
videos without action labels, FlowWAM can leverage large-scale action-unlabeled
video datasets for pretraining. We empirically find that our flow-based action representation delivers gains across both modes. On RoboTwin manipulation, FlowWAM raises the success rate to 92.94% on the Clean setting and 92.14% on Random, outperforming both VLA and WAM baselines. On WorldArena world modeling, it achieves the best overall EWMScore (63.71) with an 18.4% relative improvement in trajectory accuracy. More results can be found on our project website: https://flow-wam.github.io .
Read the full abstract ▾
All authors Yixiang Chen, Peiyan Li, Yuan Xu, Qisen Ma, Jiabing Yang, Kai Wang, Jianhua Yang, Dong An, He Guan, Gaoteng Liu, Jianlou Si, Jun Huang, Jing Liu, Nianfeng Liu, Yan Huang, Liang Wang
Computer Vision
DermDepth: Toward Monocular Metric Scale 3D Reconstruction Models for Dermatology
Héctor Carrión and Narges Norouzi
Dermatological practice routinely involves measuring and tracking lesion size, morphology and texture, as critical components of wound or skin cancer screening, monitoring and diagnosis. To accomplish this task, practitioners often image the skin surface with commonly available off-the-shelf camera sensors. This has led to an overwhelming research focus on 2D methods while these objectives naturally benefit from 3D information. In this paper, we demonstrate that dense
monocular 3D reconstructions, metric scale measurements and rich surface normal texture estimates are achievable for both dermoscopic and macroscopic cases without the need for additional hardware or multiple captures. We present DermDepth, the first single-view metric scale 3D model for the dermatological domain and D-Synth, the first synthetic dermoscopic
dataset with pixel-perfect 3D information. Our experiments show training DermDepth on D-Synth corrects metric scale error from over 16x to under 1.1x for real dermoscopic data, while preserving
geometric quality and increasing texture richness. Fine-tuning on a small amount of real clinical samples generalizes our method across three real-world
benchmarks spanning the few mm to hundred cm range, diverse skin-tones, chronic wound cases and produces measurements broadly consistent with disease size reported in medical literature. All code, data and models are available at https://github.com/hectorcarrion/dermdepth.
Read the full abstract ▾
Computer Vision
X-Lens: Real-Time Metric Depth Estimation with Heterogeneous Cameras
Heng Zhou, Shuhong Liu, Yonghao He et al. (+6 more)
We present X-lens, a compact feed-forward model for metric depth estimation from a variable number of calibrated fisheye and pinhole views. To support real-time downstream perception, X-lens is built around a geometry-aware heterogeneous camera formulation with two key components. Learnable calibration
tokens provide a coarse
alignment between fisheye and pinhole projective spaces, while a Jacobian-parameterized distortion bias injected into cross-
attention models local projection changes and promotes cross-camera consistency, enabling robust generalization with only 0.04B parameters and up to 41 FPS. The model predicts dense depth together with a global metric scale, avoiding auxiliary reconstruction targets that increase computation and optimization complexity. To learn such cross-camera generalization at scale and depth, X-lens is trained on multiple public
datasets and OmniScene, our newly released large-scale synthetic
dataset containing approximately 266K synchronized six-view frames, 1.7M individual images, and 103 indoor and outdoor scenes. Extensive experiments on both real-world and synthetic indoor and outdoor
datasets demonstrate superior heterogeneous-camera metric depth accuracy, reducing AbsRel by 25.4% on OmniScene-Full over the strongest baseline while using 88.9% fewer parameters, with competitive performance on conventional fisheye-only and pinhole-only settings.
Read the full abstract ▾
All authors Heng Zhou, Shuhong Liu, Yonghao He, Bohao Zhang, Fa Fu, Chenhui Hou, Xianbao Hou, Lijun Han, Wei Sui
Computer Vision
Controllable Generation of Diverse Dermatological Imagery for Fair and Efficient Malignancy Classification
Héctor Carrión and Narges Norouzi
Accurate dermatological diagnosis naturally necessitates equitable performance across diverse populations, yet a systematic lack of expertly annotated images, especially for underrepresented skin tones and rare diseases, impedes progress toward measurably fair methods. We introduce cgDDI (Controllable Generation of Diverse Dermatological Imagery), a hybrid framework that (1) synthesizes realistic healthy skin samples without disturbing other input properties, (2) maps single-sample rare lesions onto novel skin-tones and locations non-parametrically, and (3) allows for efficient parametric generation with as few as 10 training samples. The framework supports both human and automated
segmentation masking, enabling scalability to
datasets without pre-made lesion masks. We grow a 656-image
dataset by more than 400x and validate across two
datasets: biopsy-confirmed Diverse Dermatology Images (DDI) and expert-verified Fitzpatrick17k (F17k). On the DDI
benchmark, we achieve malignancy classification accuracy of 86.4% under synthetic-only training and 90.9% state-of-the-art performance with real data fine-tuning, alongside leading
fairness metrics. Cross-
dataset experiments show +13.9% accuracy improvements on unseen F17k data despite minimal disease overlap. We openly release 266k+ synthetic images, code, and generative models to further support
fairness research at https://github.com/hectorcarrion/ControllableGenDDI.
Read the full abstract ▾
Computer Vision
ViCo3D: Empowering LiDAR-based Collaborative 3D Object Detection with Vision Foundation Models
Haojie Ren, Songrui Luo, Lingfeng Wang et al. (+6 more)
LiDAR-based collaborative 3D perception in Vehicle-to-Everything (V2X) systems typically relies on fusing bird's-eye-view (BEV) features across
agents. However, current BEV representations, typically extracted by
LiDAR backbones trained from scratch, are geometry-dominated and lack general semantic priors, inherently limiting the efficacy of feature-level collaboration. Meanwhile,
vision foundation models (VFMs) pretrained on large-scale image data have demonstrated strong capability in learning general-purpose and informative visual representations for 2D tasks, and have the potential to enhance
agent-wise
LiDAR BEV representations for collaboration. Despite this potential, adapting VFMs to
LiDAR-based 3D detection remains challenging due to the substantial image-point cloud modality gap. To bridge this gap, we propose ViCo3D, a collaborative 3D object detection framework powered by VFMs. Specifically, ViCo3D adapts VFMs to
LiDAR-based collaborative perception from three aspects: First, ViCo3D projects point clouds onto the BEV plane as three-channel images, enabling DINOv2 to extract BEV-space visual features from
LiDAR inputs. Besides, to effectively integrate these DINOv2-derived features with
LiDAR geometric features, ViCo3D introduces a multi-scale BEV fusion module within the single-
agent encoder. In addition, ViCo3D adopts an ego-centric cross-
agent fusion strategy to aggregate complementary information from multiple
agents. Experiments on DAIR-V2X and V2XSet demonstrate that ViCo3D achieves state-of-the-art 3D detection performance. Remarkably, it delivers up to 1.8x greater collaborative gains than prior methods on DAIR-V2X. The code will be made public available for future investigation.
Read the full abstract ▾
All authors Haojie Ren, Songrui Luo, Lingfeng Wang, Yan Xia, Yao Li, Jing Li, Lu Zhang, Jiajun Deng, Yanyong Zhang
Computer Vision
Point Tracking in Surgery–The 2025 Surgical Tattoos in Infrared Challenge (STIRC2025)
Adam Schmidt, Mert Asim Karaoglu, Zijian Wu et al. (+32 more)
Point tracking in surgery is crucial to enable applications in downstream tasks such as
segmentation, 3D reconstruction, virtual tissue landmarking, autonomous probe-based scanning, and subtask autonomy. This paper introduces the 2025 iteration of a point tracking challenge to address this, wherein participants submit their
algorithms for quantification. Their
algorithms are evaluated using a
dataset named
surgical tattoos in infrared (STIR), with the challenge named the STIR Challenge 2025 (STIRC2025). The STIR Challenge 2025 comprises two quantitative components: accuracy and efficiency. The accuracy component tests the accuracy of
algorithms on in vivo and ex vivo sequences. The efficiency component tests
algorithm inference latency. The challenge was conducted as a part of MICCAI EndoVis 2025, and seven teams participated in this challenge. In this paper we summarize the challenge results and participant methods. The challenge
dataset is available at: https://zenodo.org/records/20191078, and the code for baseline models and metrics calculation is available here: https://github.com/athaddius/STIRMetrics
Read the full abstract ▾
All authors Adam Schmidt, Mert Asim Karaoglu, Zijian Wu, Jiaming Zhang, Yuxin Chen, Tim Salcudean, Ho-Gun Ha, Minkang Jang, Kyungmin Jung, Ihsan Ullah, Hyunki Lee, Suresh Guttikonda, Sarah Latus, Alexander Schlaefer, Xinkai Zhao, Yuichiro Hayashi, Masahiro Oda, Takayuki Kitasaka, Kensaku Mori, Peng Liu, Chenyang Li, Stefanie Speidel, Aoife Gardiner, Agostino Stilli, Danail Stoyanov, Francisco Vasconcelos, Anwesa Choudhuri, Meng Zheng, Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Terrence Chen, Ziyan Wu, Alexander Ladikos, Omid Mohareri
Computer Vision
Domain-Incremental Remote Sensing Change Detection via Difference-Guided Adaptation and Frequency-Decoupled Distillation
Daifeng Peng, Yaning Li and Haiyan Guan
Remote sensing change detection (RSCD) models are prone to catastrophic forgetting when incrementally adapted to new domains. Existing domain-incremental learning (DIL) methods mainly preserve image-level representations but often overlook bitemporal discrepancy cues, which are critical for robust change detection under domain shifts. To address this limitation, we propose DG-FDD, a domain-incremental change detection framework that integrates Difference-Guided Adaptation and Frequency-Decoupled
Distillation. Specifically, the Difference-Guided Dynamic Adapter (DGDA) models bitemporal feature discrepancies to promote change-aware feature adaptation and reduce domain-specific interference. Meanwhile, the Frequency-Decoupled Knowledge
Distillation strategy with Cross-domain Synthesis (FDKD-CS) separates structural information from domain style in the frequency domain, enabling stable knowledge transfer without historical data. Extensive experiments on three public high-resolution RSCD
datasets under two- and three-domain incremental protocols demonstrate that DG-FDD effectively mitigates catastrophic forgetting. Compared with independently trained single-task models, DG-FDD records mean relative changes in F1 and IoU of only -0.23% and -0.45%, respectively, across six two-domain sequences, and -0.69% and -1.31%, respectively, across the three evaluated three-domain sequences. These results indicate a favorable stability-plasticity balance between historical knowledge retention and new-domain adaptation in continual cross-domain change detection.
Read the full abstract ▾
Computer Vision
Open-KNEAD: Knowledge-grounded Nutrition Estimation via Agentic Decomposition
Bruce Coburn, Jingbo Yue, Jinge Ma et al. (+3 more)
Multimodal Large
Language Models (MLLMs) are increasingly used for dietary assessment from meal images, where
retrieval-augmented grounding was shown to sharpen nutrition estimates. However, we find this premise no longer holds for current MLLMs. A modern MLLM's direct estimate now matches or surpasses the full
retrieval pipeline. This raises a question: if
retrieval no longer improves the overall estimate, can it still deliver the two things clinicians value, accurate portions and a traceable, item-by-item record? We pursue this while preserving what matters for clinical adoption: minimal user burden (a single, unannotated meal image), explainability (an auditable record), and
privacy (locally hosted inference). We introduce Open-KNEAD, a knowledge-grounded agentic framework for meal nutrition estimation that is training-free and locally deployable. Each decomposed food item is grounded to a Food and Nutrient Database for Dietary Studies (FNDDS) code via selective, nutrient-aware
retrieval, composing an auditable per-item record. Across two open MLLM families and three cuisines, Open-KNEAD improves portion estimates over both prior grounding methods and direct estimation in most backbone-
dataset settings. An
agent-internal recipe-prior step further recovers the invisible cooking-added energy that biases estimates on non-US cuisine. The advantage is largest on the dietitian-verified ACETADA
dataset, where the local open
agent surpasses the direct portion estimates of two frontier closed models by roughly $30\%$ and $53\%$, all while keeping every meal image on local hardware. We release the Open-KNEAD framework and its
agent-ready FNDDS knowledge base.
Read the full abstract ▾
All authors Bruce Coburn, Jingbo Yue, Jinge Ma, Siddeshwar Raghavan, Gautham Vinod, Fengqing Zhu
Computer Vision · Cross-listed from q-bio.NC
Real-time fall detection based on vision for low-power edge platforms
Wenjun Xia, Zhicheng Peng, Haopeng Li et al. (+1 more)
Falling detection is vital for elderly care and intelligent surveillance; however, prevailing
vision-based approaches predominantly frame it as static pose classification or discrete temporal pattern matching, fundamentally overlooking the instability dynamics of the human support system. This paper proposes a physics-informed falling detection framework that recasts falling as a stability-loss event in a coupled dynamical system. We introduce a novel dual-LTC architecture comprising a Center-of-Mass (CoM) subsystem and a Base-of-Support (BoS) subsystem, both instantiated as
Liquid Time-Constant (LTC) neural networks to continuously model inertial trajectory
evolution and ground-contact adjustment through adaptive time constants, Physical interpretability of falling motion. A learnable coupling module emulates physical interaction between the two subsystems, while a Stability
Manifold classifier operates in the joint latent space to detect boundary crossing via Lyapunov-inspired stability metrics. Complementary counterfactual trajectory projection and Time-to-Collision (TTC) estimation further enable irreversibility assessment and early warning. The architecture is designed to support a three-state prediction paradigm (Normal, Falling, Fallen); in this preliminary study, we validate the core stability discrimination capability on a two-class
dataset (Normal vs. Falling), leaving the full three-state temporal transition to future work. Unlike conventional CNN–RNN pipelines, the proposed formulation encodes continuous-time mechanical inertia, yielding a sub-50K-parameter network capable of real-time inference on resource-constrained edge devices. Extensive experiments demonstrate competitive accuracy with superior physical interpretability, validating its efficacy for low-compute visual fall detection.
Read the full abstract ▾
All authors Wenjun Xia, Zhicheng Peng, Haopeng Li, Zhengdi Zhang
Computer Vision
Rank-1 Identity Consensus Predicts Gallery Enrollment in 1:N Face Matching More Accurately than Score Thresholding
Gabriella Pangelinan, Aman Bhatta, Michael C. King et al. (+1 more)
In operational 1:N face identification, a crucial question arises for each probe: is this person enrolled in the gallery or not? The stakes are high and asymmetric. Rejecting a mate-present (MP) probe loses a valid lead; accepting a mate-absent (MA) probe makes every returned candidate a false identification, at worst a wrongful arrest. Most approaches threshold match scores, but scores shift substantially with image quality and gallery size and composition, making thresholds fixed before deployment brittle under realistic conditions. Our prior work introduced 1-consistency, the only method based on rank consensus across multiple independently trained matchers: a probe is labeled MP if all matchers return the same rank-1 identity. This work stress-tests 1-consistency across 36 (gallery, probe quality) scenarios spanning four quality levels and two structural axes: images per identity and total enrolled identities. We
benchmark against two score-thresholding methods that bracket what any deployed threshold could achieve. Fixed Score-Thresholding (FST), calibrated once on baseline conditions, collapses asymmetrically as quality degrades: MP recall falls below 2% while MA recall holds near 100%. Oracle Score-Thresholding (OST), re-tuned per scenario, is the best any threshold could theoretically do, yet for degraded probes 1-consistency matches it with zero tuning. The two differ mainly in error type (OST favors MP recall, 1-consistency favors MA recall), but on one axis 1-consistency does not merely match the oracle: when it labels a probe MP, it returns the correct mate 97-100% of the time versus OST's 66-84% under severe degradation. In short, 1-consistency delivers oracle-level accuracy without the impossible requirement: it sets no threshold, so it needs no advance knowledge of the conditions a probe will arrive in, which is what makes it usable.
Read the full abstract ▾
All authors Gabriella Pangelinan, Aman Bhatta, Michael C. King, Kevin W. Bowyer
Computer Vision
UniMedSeg: Unified In-Context Learning for Multi-Paradigm 2D/3D Medical Image Segmentation
Yunzhou Li, Jiesi Hu, Yanwu Yang et al. (+5 more)
Medical image
segmentation foundation models are expected to generalize across diverse clinical scenarios, yet existing universal methods remain fragmented by
prompt paradigms and spatial dimensions. Visual in-context learning, interactive
segmentation, and
language-guided
segmentation are typically handled by paradigm-specific models, while 2D and 3D images are also modeled separately. Such isolation prevents heterogeneous annotations and data from being jointly absorbed by a single scalable model and limits cross-paradigm knowledge transfer. To address this bottleneck, we propose UniMedSeg, a
Transformer-centric universal
segmentation framework that maps visual examples,
geometric interactions,
language instructions, and 2D/3D images into a shared sequence space, enabling heterogeneous medical supervision to be jointly learned through a unified in-context interface without
prompt- or dimension-specific branches. To overcome the long-sequence
memory bottleneck caused by visual contexts, we introduce Decoupled Split
Attention, which reduces
attention complexity to linear while preserving hardware-friendly computation and focused context-target interaction. Extensively trained and evaluated on a large corpus curated from 27 public
datasets, UniMedSeg achieves state-of-the-art performance across visual in-context, interactive, and
language-guided
segmentation without task-specific fine-tuning, demonstrating strong generalization on diverse held-out tasks. The code and model weights are publicly available at https://github.com/Lii1228/UniMedSeg
Read the full abstract ▾
All authors Yunzhou Li, Jiesi Hu, Yanwu Yang, Hanyang Peng, Chenfei Ye, Jianfeng Cao, Yixuan Yuan, Ting Ma
Computer Vision
Hy-Embodied-VLM-1.0: Efficient Physical-World Agents
Ziyi Wang, Xumin Yu, Yongming Rao et al. (+19 more)
Building capable
embodied agents requires not only
multimodal perception and understanding, but also agentic capabilities for
reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy-
Embodied-VLM-1.0, an efficient and powerful
embodied foundation model specifically designed for
embodied agents operating in the physical world. To cultivate such capabilities from the pre-training stage onward, we define an action-centric capability taxonomy comprising three progressive dimensions: Action-Relevant State Understanding, Action-Transition
Reasoning, and Sequential and Adaptive
Reasoning. Guided by this taxonomy, we develop a systematic data pipeline and curate data mixtures spanning both pre-training and post-training. To deliver strong physical-world understanding and interaction capabilities while supporting latency-sensitive deployment, we build our model on the Hy3-A3B
language backbone and the Hy-ViT2
vision encoder. Its efficient Mixture-of-Experts architecture combines strong model capacity with high inference efficiency. We evaluate Hy-
Embodied-VLM-1.0 on a comprehensive suite of 38
benchmarks covering
embodied perception, physical-world understanding, and
embodied reasoning. The model achieves the best performance among similarly sized models on 19 of the 38
benchmarks and substantially outperforms strong competitors, including Qwen3.6-A3B and Cosmos 3. Compared with the previous-generation Hy-
Embodied-0.5 MoT-2B, Hy-
Embodied-VLM-1.0 improves average performance by 8.4%. Despite activating only 3B parameters, it achieves performance close to that of the previous-generation model with 32B activated parameters. Beyond static
benchmark evaluation, Hy-
Embodied-VLM-1.0 also demonstrates strong performance on
embodied agentic tasks requiring multi-turn interaction and long-horizon
reasoning.
Read the full abstract ▾
All authors Ziyi Wang, Xumin Yu, Yongming Rao, Yonggen Ling, Yunheng Li, Oran Wang, Mingqi Gao, Yuchen Zhou, Yves Liang, Zuyan Liu, Yani Zhang, Rui Huang, Xiaoran Xu, Bowen Yuan, Yifu Yuan, Xu Tan, He Zhang, Yufei Huang, Shenghao Zhang, Hongsheng Wu, Han Hu, Zhengyou Zhang
Computer Vision
Peter R.D. van der Wal, Nicola Strisciuglio and George Azzopardi
Vision Transformers (ViTs) have demonstrated remarkable performance in computer
vision tasks. However, their self-
attention mechanism often diffuses focus across background regions, relying on spurious correlations rather than object-relevant cues. Inspired by inhibitory mechanisms observed in biological
vision systems, we propose the Inhibited Self-
Attention (ISA), a novel self-
attention that integrates inhibitory signals to enhance feature selectivity and suppress spurious responses. In contrast to conventional self-
attention, which relies solely on positive
attention values due to softmax normalization, our approach retains and utilizes negative
attention scores to suppress irrelevant features and sharpen focus on objects of interest. Experiments across multiple
datasets, including ImageNet-1k and COCO, and several robustness
benchmarks demonstrate that ISA enhances object-centric selectivity, reduces shortcut reliance, and improves out-of-distribution generalization. Our analysis of relevance maps confirms that ViTs with ISA exhibit sharper, more localized focus on object-relevant regions while reducing distractions from non-relevant (background) features, enabling more reliable models. We release our code at https://github.com/prdvanderwal/inhibited-self-
attention
Read the full abstract ▾
Computer Vision
Metric-Guided Synthetic Image Data Rendering for Deep Learning compatible with Agentic AI
Martina Radoynova, Samuel Pantze, Trina De et al. (+2 more)
Deep learning computer
vision for scientific applications requires collecting and annotating large
datasets in a laborious, expensive and error-prone process. Synthetic data generation through 3D modelling and rendering may simplify this process and increase the accuracy of annotations by generating them programmatically. However, minimising the domain gap between real and synthetic images visually is subjective and lacks systematic quantitative guidance. We present GraNatPy, a Python package with metrics to guide improvement of the rendered scene. We show that quantifiable increase in realism, diversity and size of rendered
dataset correlates with improved visual perception of the scene and higher zero-shot performance of an object detection model. Furthermore, we demonstrated using photographs of virological plaque assays that gradient similarity affects performance on small object detection, which can be improved by mixing real and synthetic data. Finally, we turn procedural data rendering into an agentic skill (SynthClaw) to automate the procedural parameter optimisation.
Read the full abstract ▾
All authors Martina Radoynova, Samuel Pantze, Trina De, Ulrik Günther, Artur Yakimovich
Computer Vision
Statistical Non-linear Reconstruction Loss for Image Anomaly Detection
Nguyen Minh Tri, Hoang Khuong Duy and Huynh Cong Viet Ngu
Reconstruction-based methods are a cornerstone of unsupervised image anomaly detection, but they remain vulnerable to outlier leakage, where standard mean squared error (MSE) loss drives the model to faithfully reconstruct anomalous patterns. We propose a Non-linear Reconstruction Loss that applies a sigmoid-based squashing function to suppress high-magnitude features, preventing outliers from dominating optimization while preserving sensitivity to normal patterns. In addition, we introduce a statistical calibration scheme that selects the scaling factor $k$ from the confidence interval (CI) of the normal feature distribution, enabling data-driven control of the suppression strength. Our approach achieves competitive or superior anomaly detection performance compared to state-of-the-art methods, reaching 99.0% Image-AUROC and 97.3% Pixel-AUROC on MVTec-AD, and 95.3% Image-AUROC and 99.0% Pixel-AUROC on VisA. These results indicate that non-linear gradient suppression is an effective mechanism for mitigating outlier leakage and improving anomaly localization in unified industrial inspection settings. The implementation is available at https://github.com/mintii13/Statistical-Non-linear-Reconstruction-Loss.git.
Read the full abstract ▾
Computer Vision
LARAD: Layout-Aware Road Anomaly Detection via Spatial-Logic Reasoning
Shiyi Mu, Xujie Chen and Shugong Xu
Accurate open-world obstacle detection is critical for autonomous driving. Current anomaly
segmentation methods suffer from a fundamental blind spot: they over-rely on texture novelty to identify out-of-distribution (OoD) objects while ignoring contextual spatial logic. Furthermore, mitigating the resulting false positives often requires cascading massive
vision models, introducing unacceptable inference latency. To address these issues, we propose Layout-Aware Road Anomaly Detection (LARAD), shifting the paradigm from appearance matching to spatial-logic
reasoning. First, we introduce the Spatial-Logic Violation Synthesis (SLVS) pipeline, which generates training samples that are texture-consistent yet spatially invalid, forcing the model to learn contextual violations. Second, we augment a standard closed-set
segmentation network with a lightweight, OoD-guided
attention branch. Extensive experiments demonstrate that LARAD significantly enhances robustness against logical anomalies and establishes a new state-of-the-art, all while retaining the high efficiency of a single-model architecture.
Read the full abstract ▾
Computer Vision
AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning
Yanghai Wang, Jiahao Wang, Jiafu Tang et al. (+9 more)
Omni-modal
video captioning is not merely combining visual captioning with audio transcription: a useful caption must describe how visual actions,
speech, music, and sound effects co-evolve. Existing large
multimodal models often fail at this relational step, treating audio and visual streams as loosely coupled observations, relying on automatic
speech recognition, and under-specifying non-
speech sounds and their links to visual events. We present AVSCap, a framework for audio-visual captioning centered on explicit cross-modal event binding. First, we construct AVSCap-130K, a tri-modal training corpus generated by a decoupled-then-fused pipeline that anchors visual and acoustic evidence before composing grounded omni-modal captions. Second, we train AVSCap-7B, a 7B captioner with a two-stage strategy: supervised fine-tuning establishes baseline capabilities, while sample-efficient reinforcement learning uses hybrid rewards to optimize acoustic completeness and audio-visual synergy. Our scaling analysis shows that reinforcement learning brings larger gains than increasing SFT data. Third, we introduce AVSCapBench, a
benchmark that decomposes captions into visual, audio, and synergy events and evaluates them with fine-grained event recall. Experiments on AVSCapBench and external
benchmarks show that AVSCap-7B improves non-
speech audio coverage and cross-modal binding, delivering the best overall performance among evaluated open-source models.
Read the full abstract ▾
All authors Yanghai Wang, Jiahao Wang, Jiafu Tang, Yuanxing Zhang, Zhe Cao, Hanyan Bian, Zijie Zhang, Weiliang Luo, Zhiyu Pan, Zixuan Dong, Jiaheng Liu, Zhaoxiang Zhang
Computer Vision
Breaking Déjà Vu: Independent Auditing of Visual Place Recognition through Vision-Language Reasoning
Sania Waheed, Michael Milford, Sarvapali D. Ramchurn et al. (+1 more)
Visual place recognition (VPR) is a key enabler of accurate localization and long-term autonomous
navigation in robotics applications, such as loop closure detection for simultaneous localisation and mapping (SLAM). However, real-world VPR deployment relies on selecting an image matching threshold that balances precision and recall. These thresholds are typically tuned using labeled validation data and fixed during deployment, making them unreliable under environmental changes where ground truth is unavailable. This is particularly problematic in safety-critical robotics, where accepting a false loop closure can corrupt the estimated trajectory and map. In this work, we introduce Visual Place Recognition
Auditing, an independent post-
retrieval verification framework that leverages
Vision-
Language Models (VLMs) to assess retrieved matches by
reasoning jointly over query and candidate images. Unlike conventional verification methods, our approach performs instance-level verification without requiring architecture-specific confidence measures,
dataset-dependent thresholds, or prior knowledge of the deployment environment. We evaluate our method on six
benchmark datasets using five state-of-the-art VPR methods and four VLMs. Results show that VLM-based
auditing improves recall@1 by 13.6% on average as compared to state-of-the-art methods while reducing false acceptance rates to 12%, maintaining precision above 95% and coverage above 75%.
Read the full abstract ▾
All authors Sania Waheed, Michael Milford, Sarvapali D. Ramchurn, Shoaib Ehsan
Computer Vision
UniVR: Thinking in Visual Space for Unified Visual Reasoning
Zhongwei Ren, Yunchao Wei, Yao Zhao et al. (+5 more)
Learning broad world knowledge directly from raw visual data is a fundamental capability of intelligence. We introduce UniVR, the first investigation into simultaneously learning complex
reasoning, fine-grained physical dynamics, and long-term planning from pure visual demonstrations. At its core, UniVR features VR-GRPO, a reinforcement learning paradigm with complementary global and step-level rewards. This approach enforces logical coherence and physical consistency throughout the
reasoning process without requiring task-specific heuristics or image-text pairs. To train and evaluate UniVR, we construct VR-X, a large-scale
benchmark curated from 16 diverse sources spanning long-horizon manipulation, spatial puzzles, and physical
reasoning. It is the first comprehensive suite to assess these heterogeneous capabilities under a purely visual protocol. Remarkably, UniVR achieves up to a 25% improvement on VR-X, and its superior visual
reasoning also boosts performance on various
multimodal understanding
benchmarks. These findings underscore the vast potential of
reasoning within visual spaces, with all code, data, and models are open-sourced for further research.
Read the full abstract ▾
All authors Zhongwei Ren, Yunchao Wei, Yao Zhao, Weibo Gong, Xiao Liu, Anran Wang, Xiangtai Li, Xiaojie Jin
Computer Vision · Cross-listed from cs.LG
Winfried van den dool, Patrick Forré, Amir Habibian et al. (+2 more)
The $\mathcal{O}(N^2)$ complexity of
attention over $N$
tokens remains a computational bottleneck in
transformer models. Vector-Quantized (VQ)
attention reduces this to $\mathcal{O}(MN)$ by representing keys with $M$ codewords, but applies uniform codebook capacity regardless of where
attention mass concentrates: high-
attention regions of key space may be coarsely approximated while low-
attention regions waste representational capacity. We propose Adaptive Vector-Quantized (AVQ)
Attention, which adaptively allocates codebook capacity based on
attention importance. Starting from a small set of codewords, our method identifies the most important codes during the forward pass and refines them with pre-learned child codewords, achieving fine-grained quantization where it matters most while maintaining coarse quantization elsewhere. We develop an implementation using custom Triton kernels that enables the full adaptive refinement process, including importance scoring, child codeword insertion, and parent contribution replacement, to be carried out within the tiled computation paradigm of Flash
Attention with minimal overhead. Our approach maintains $\mathcal{O}(MN)$ complexity while achieving improved accuracy-efficiency trade-offs compared to fixed-codebook VQ-
attention.
Read the full abstract ▾
All authors Winfried van den dool, Patrick Forré, Amir Habibian, Yuki M. Asano, Max Welling
Computer Vision · Cross-listed from cs.AI
Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?
Kaiwen Zheng, Junchen Fu, Wenhao Deng et al. (+3 more)
Recent advances in
multimodal large
language models (MLLMs) have significantly improved the performance of
multimodal emotion recognition (MER) and enabled interpretable description generation by jointly modeling
video, audio, and
language, etc. However, these performance improvements are often accompanied by an increase in model parameter size (e.g, at least 7B), which simultaneously incurs high computational costs and reduces inference efficiency, thereby hindering real-time deployment on resource-constrained platforms such as
robots and mobile devices. This raises a fundamental question: do we really need the
multimodal MER model larger than 1B parameters for high-quality MER? In this paper, we challenge the assumption that larger models are inherently necessary and proposes a lightweight MER framework (called Light-MER), which achieves better and faster
multimodal sentiment understanding and recognition through knowledge
distillation. It can transfer knowledge from a strong, large-scale teacher model to a lightweight sub-billion-parameter student model, aiming to preserve rich
multimodal emotion reasoning and recognition while substantially improving deployment efficiency. Specifically, we introduce two new optimization strategies to enhance knowledge transfer: (1) a new optimal transport loss that combines Sliced Wasserstein Distance with hidden-state
alignment, and (2) a new multi-reward optimization strategy based on GRPO that balances MER performance and efficiency, aimed at further enhancing the learning capabilities of student models. Extensive experiments on nine
benchmark datasets demonstrate that Light-MER achieves state-of-the-art performance while significantly improving inference efficiency. This highlights the strong potential of small
multimodal emotion language models for future research. Code is available at https://github.com/GAIR-Lab/Light-MER.
Read the full abstract ▾
All authors Kaiwen Zheng, Junchen Fu, Wenhao Deng, Hu Han, Joemon M. Jose, Xuri Ge
Computer Vision
CoRe: A Comprehensive Framework for Cross-Image Comparative Reasoning in Vision-Language Models
Lin Peng, Cong Wan, Zeyu Guo et al. (+2 more)
Cross-image comparative
reasoning remains challenging for
vision-
language models (VLMs), especially when correct prediction requires fine-grained attribute grounding and globally consistent
reasoning. We present CoRe, a unified framework for this problem. CoRe includes: (i) CoRe-20K, a large-scale triplet-based training set automatically constructed from structured visual metadata through a multi-expert collaborative pipeline, covering counting, depth, distance, and spatial relations; (ii) TriSR, a structured reward framework that jointly supervises attribute grounding, judgment
alignment, and triplet consistency under GRPO optimization; and (iii) CoRe-Bench, the first
benchmark dedicated to fine-grained cross-image comparative
reasoning. Experiments show that CoRe substantially outperforms existing VLMs on CoRe-Bench while remaining competitive on standard
multimodal benchmarks, achieving a 28.2-point gain in partial accuracy over the strongest baseline.
Read the full abstract ▾
All authors Lin Peng, Cong Wan, Zeyu Guo, SongLin Dong, Yihong Gong
Computer Vision
ExtraGS: Enhancing Endoscopic View Extrapolation via Diffusion-Guided 3D Gaussian Splatting
Cheng-Tai Hsieh, Jiwei Shan, Han Fang et al. (+6 more)
Robot-assisted minimally invasive surgery (MIS) critically depends on reliable endoscopic perception for
navigation and safety. However, conventional endoscopes provide only a limited field of view, leaving large portions of the surrounding anatomy unobserved. Recent neural rendering approaches, such as Neural Radiance Fields and 3D Gaussian Splatting, enable novel view synthesis from endoscopic
videos, but their reliance on sparse observations often leads to severe artifacts when extrapolating beyond the training trajectory. In this work, we propose ExtraGS, a framework for enhancing endoscopic view extrapolation through
diffusion-guided 3D Gaussian Splatting. Starting from an initial reconstruction, we introduce an uncertainty-guided virtual camera sampling strategy to actively explore blind spots and maximize information gain. The rendered views from these sampled locations are refined using a
diffusion model to recover plausible anatomical structures, producing pseudo-observations that guide further optimization. To prevent the generated content from degrading reliable regions, we adopt a confidence-weighted fine-tuning strategy when incorporating these pseudo-observations. Extensive experiments on multiple public endoscopic
datasets demonstrate that ExtraGS significantly reduces extrapolation artifacts and achieves state-of-the-art performance in endoscopic novel view synthesis.
Read the full abstract ▾
All authors Cheng-Tai Hsieh, Jiwei Shan, Han Fang, Jianshu Hu, Tao Ni, Lijun Han, Yutong Ban, Shing Shin Cheng, Hesheng Wang
Computer Vision
MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models
Mingzhen Xu, Haonan Guo, Di Wang et al. (+9 more)
Hyperspectral foundation models learn transferable spectral-spatial representations from large-scale unlabeled data. They provide an effective paradigm for adapting to downstream hyperspectral image (HSI) classification tasks with limited labeled samples. However, spectral band configurations vary substantially across sensors, which makes direct model transfer difficult. Existing adaptation strategies often compress, select, or reshape the original spectra to match model-specific input requirements. These operations may discard useful spectral information and weaken local spectral continuity. To address this problem, we propose MBTI, a Multi-Branch efficient fine-tuning framework for Hyperspectral Image classification. MBTI adapts hyperspectral foundation models to downstream classification tasks while preserving full-band spectral information. First, we introduce a spectral-continuity-preserving multi-branch preprocessing strategy. The original HSI is divided into multiple continuous spectral subsets, and a band reuse mechanism is used when the remaining bands cannot form a complete branch. This avoids invalid padding and unnecessary spectral loss. Second, independent Low-Rank Adaptation (LoRA) modules are inserted into each branch. They enable different spectral intervals to learn task-specific discriminative features while keeping most pre-trained parameters frozen. Finally, a multi-branch channel
attention fusion module adaptively recalibrates and integrates features from all spectral branches. Experiments on three public hyperspectral
datasets show that MBTI achieves competitive and superior performance compared with representative classification methods. Under the final rank-8 configuration, only about 2.33%–2.36% of the parameters are trainable. The code will be available at https://github.com/Azhenmiddleblock/MBTI/tree/main.
Read the full abstract ▾
All authors Mingzhen Xu, Haonan Guo, Di Wang, Yinghua Qu, Zhiliang Zhou, Lei Zhang, Huiwen Yao, Rui Zhao, Fengxiang Wang, Gang Wan, Bo Du, Liangpei Zhang
Computer Vision
HSEmotion Team at the 11th ABAW Challenge: Multi-Task Learning and Ambivalence/Hesitancy Video Recognition
Aleksei Bakin and Andrey V. Savchenko
This article presents our results for the 11th Affective Behavior Analysis in-the-Wild (ABAW) competition. For multi-task learning with simultaneous prediction of valence, arousal, facial expressions, and action units on s-Aff-Wild2
dataset, we use frozen lightweight facial extractors, MT-EmotiDDAMFN and MT-EmotiEffNet-B0, with separate heads and systematic post-processing: temporal Gaussian smoothing, per-class expression bias, AffectNet blending, per-AU threshold tuning, and weighted backbone fusion. On the official validation set, our ensemble significantly exceeds the performance of the ConvNeXt baseline. For ambivalence/hesitancy
video recognition on the expanded BAH
dataset, we extend the audiovisual pipeline to
video-level Macro F1 by late fusion of face, HuBERT audio, and RoBERTa text classifiers, temporal aggregation, and a global-text gate. Frame-level Weighted F1 on validation set rises from 0.74 in ABAW-8 to 0.79, while the best public-test
video-level Macro F1 reaches 0.73. In both tasks, competitive performance is achieved without fine-tuning heavy backbones. These results indicate that systematic prediction calibration and lightweight
multimodal fusion can rival substantially heavier end-to-end approaches while offering improved efficiency and deployment flexibility.
Read the full abstract ▾
Computer Vision
EvoGraph-R1: Self-Evolving Multimodal Knowledge Hypergraphs for Agentic Retrieval
Jiashi Lin, Changhong Jiang, Xiangru Lin et al. (+12 more)
Retrieval-augmented generation (RAG) has emerged as a critical paradigm for grounding
Multimodal Large
Language Models (MLLMs) in external knowledge. Recent GraphRAG methods introduce structured entity-relation graphs to improve
retrieval and
reasoning. However, they remain limited by treating knowledge graphs as static data structures built offline and queried in a single pass. This static paradigm misaligns with the interactive, iterative nature of knowledge-intensive
reasoning, creating three bottlenecks: (i) text-centric fragmentation that impedes cross-modal
reasoning, (ii) frozen structures unable to incorporate new evidence or correct errors, and (iii) rigid single-pass
retrieval without adaptive refinement. To overcome these limitations, we introduce EvoGraph-R1, a self-evolving GraphRAG framework that reconceptualizes knowledge graphs as dynamic environments shaped through
agent interactions. We formulate
retrieval as a Markov Decision Process (MDP) where the
agent observes the graph state and executes actions to query (GraphRetrieve), expand (WebSearch), refine (GraphEdit), or terminate (Answer) the
reasoning. These actions reshape the hypergraph structure and generate feedback signals that guide subsequent
evolution. Through this closed loop, the hypergraph evolves by integrating new evidence, correcting errors, and refining structure to support multi-hop
reasoning. Experiments on
multimodal VQA and text QA
benchmarks demonstrate substantial improvements over existing RAG baselines in accuracy, coverage, and traceability, establishing self-evolving knowledge graphs as a fundamental paradigm across modalities.
Read the full abstract ▾
All authors Jiashi Lin, Changhong Jiang, Xiangru Lin, Ruifei Zhang, Xinyi Zhu, Jiyao Liu, Cheng Tang, Ye Du, Shujian Gao, Junzhi Ning, Lihao Liu, Ziyan Huang, Tianbin Li, Jin Ye, Junjun He
Computer Vision
VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression
Yupeng Zheng, Kai Zou, Bin Liu et al. (+1 more)
Vision-
language models (VLMs) process large numbers of visual
tokens, resulting in substantial inference latency and
memory overhead. This has motivated extensive research on visual
token compression. While training-free strategies rely on heuristic metrics and suffer significant performance degradation under high compression ratios, many training-based methods introduce external compression modules that force the VLM backbone to adapt, incurring substantial retraining cost and compromising VLMs' priors. Effective visual
token compression hinges on strong information encoding, a capability already present in pretrained VLMs but underutilized by existing approaches. Motivated by this, we propose VisCo, a training-efficient self-compression framework that reuses the pretrained VLM itself as an intrinsic compressor. VisCo is a parameter-sharing autoencoder that compresses visual information using a small set of
memory tokens and transfers hierarchical information from encoding to decoding. Experiments show that VisCo surpasses prior methods across all evaluated compression ratios, with larger gains under more aggressive compression, and remains stable even in the extreme single-
token setting. Moreover, when combined with the original visual
tokens, the learned
memory tokens can even improve the base model, suggesting that VisCo captures complementary representations beyond compression. Code is available at: \href{https://github.com/Zyvpeng/VisCo}{\textcolor{blue}
Read the full abstract ▾
All authors Yupeng Zheng, Kai Zou, Bin Liu, Nenghai Yu