Computer Vision
A Versatile Multimodal Agent for Multimedia Content Generation
Daoan Zhang, Wenlin Yao, Xiaoyang Wang et al. (+3 more)
With the advancement of AIGC (AI-generated content) technologies, an increasing number of generative models are revolutionizing fields such as
video editing, music generation, and even film production. However, due to the limitations of current AIGC models, most models can only serve as individual components within specific application scenarios and are not capable of completing tasks end-to-end in real-world applications. In real-world applications, editing experts often work with a wide variety of images and
video inputs, producing
multimodal outputs – a
video typically includes audio, text, and other elements. This level of integration across multiple modalities is something current models are unable to achieve effectively. However, the rise of
agent-based systems has made it possible to use AI tools to tackle complex content generation tasks. To deal with the complex scenarios, in this paper, we propose a MultiMedia-
Agent designed to automate complex content creation. Our
agent system includes a data generation pipeline, a tool library for content creation, and a set of metrics for evaluating preference
alignment. Notably, we introduce the skill acquisition theory to model the training data curation and
agent training. We designed a two-stage correlation strategy for plan optimization, including self-correlation and model preference correlation. Additionally, we utilized the generated plans to train the MultiMedia-
Agent via a three stage approach including base/success plan finetune and preference optimization. The comparison results demonstrate that the our approaches are effective and the MultiMedia-
Agent can generate better multimedia content compared to novel models.
Read the full abstract ▾
All authors Daoan Zhang, Wenlin Yao, Xiaoyang Wang, Yebowen Hu, Jiebo Luo, Dong Yu
Computer Vision
LTX-2: Efficient Joint Audio-Visual Foundation Model
Yoav HaCohen, Benny Brazowski, Nisan Chiprut et al. (+26 more)
Recent text-to-
video diffusion models can generate compelling
video sequences, yet they remain silent – missing the semantic, emotional, and atmospheric cues that audio provides. We introduce LTX-2, an open-source foundational model capable of generating high-quality, temporally synchronized audiovisual content in a unified manner. LTX-2 consists of an asymmetric dual-stream
transformer with a 14B-parameter
video stream and a 5B-parameter audio stream, coupled through bidirectional audio-
video cross-
attention layers with temporal positional
embeddings and cross-modality AdaLN for shared timestep conditioning. This architecture enables efficient training and inference of a unified audiovisual model while allocating more capacity for
video generation than audio generation. We employ a
multilingual text encoder for broader
prompt understanding and introduce a modality-aware classifier-free guidance (modality-CFG) mechanism for improved audiovisual
alignment and controllability. Beyond generating
speech, LTX-2 produces rich, coherent audio tracks that follow the characters, environment, style, and
emotion of each scene – complete with natural background and foley elements. In our evaluations, the model achieves state-of-the-art audiovisual quality and
prompt adherence among open-source systems, while delivering results comparable to proprietary models at a fraction of their computational cost and inference time. All model weights and code are publicly released.
Read the full abstract ▾
All authors Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, Andrew Kvochko, Avishai Berkowitz, Daniel Shalem, Daphna Lifschitz, Dudu Moshe, Eitan Porat, Eitan Richardson, Guy Shiran, Itay Chachy, Jonathan Chetboun, Michael Finkelson, Michael Kupchick, Nir Zabari, Nitzan Guetta, Noa Kotler, Ofir Bibi, Ori Gordon, Poriya Panet, Roi Benita, Shahar Armon, Victor Kulikov, Yaron Inger, Yonatan Shiftan, Zeev Melumian, Zeev Farbman
Computer Vision
UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision
Ruiyan Han, Zhen Fang, XinYu Sun et al. (+9 more)
While Unified
Multimodal Models (UMMs) have achieved remarkable success in cross-modal comprehension, a significant gap persists in their ability to leverage such internal knowledge for high-quality generation. We formalize this discrepancy as Conduction Aphasia, a phenomenon where models accurately interpret
multimodal inputs but struggle to translate that understanding into faithful and controllable synthesis. To address this, we propose UniCorn, a simple yet elegant self-improvement framework that eliminates the need for external data or teacher supervision. By partitioning a single UMM into three collaborative roles: Proposer, Solver, and Judge, UniCorn generates high-quality interactions via self-play and employs cognitive pattern reconstruction to distill latent understanding into explicit generative signals. To validate the restoration of
multimodal coherence, we introduce UniCycle, a cycle-consistency
benchmark based on a Text to Image to Text reconstruction loop. Extensive experiments demonstrate that UniCorn achieves comprehensive and substantial improvements over the base model across six general image generation
benchmarks. Notably, it achieves SOTA performance on TIIF(73.8), DPG(86.8), CompBench(88.5), and UniCycle while further delivering substantial gains of +5.0 on WISE and +6.5 on OneIG. These results highlight that our method significantly enhances T2I generation while maintaining robust comprehension, demonstrating the scalability of fully self-supervised refinement for unified
multimodal intelligence.
Read the full abstract ▾
All authors Ruiyan Han, Zhen Fang, XinYu Sun, Yuchen Ma, Ziheng Wang, Yu Zeng, Zehui Chen, Lin Chen, Wenxuan Huang, Wei-Jie Xu, Yi Cao, Feng Zhao
Computer Vision
AnatomiX, an Anatomy-Aware Grounded Multimodal Large Language Model for Chest X-Ray Interpretation
Anees Ur Rehman Hashmi, Numan Saeed and Christoph Lippert
Multimodal medical large
language models have shown substantial progress in chest X-ray interpretation but continue to face challenges in spatial
reasoning and anatomical understanding. Although existing grounding techniques improve overall performance, they often fail to establish a true anatomical correspondence, resulting in incorrect anatomical understanding in the medical domain. To address this gap, we introduce AnatomiX, a multitask
multimodal large
language model for anatomically grounded chest X-ray interpretation. Inspired by the radiological workflow, AnatomiX adopts a two stage approach: first, it identifies anatomical structures and extracts their features, and then leverages a large
language model to perform diverse downstream tasks such as phrase grounding, report generation, visual question answering, and image understanding. Extensive experiments across multiple
benchmarks demonstrate that AnatomiX achieves superior anatomical
reasoning and delivers over 25% improvement in performance on anatomy grounding, phrase grounding, grounded diagnosis and grounded captioning tasks compared to existing approaches. Code and pretrained model are available at https://aneesurhashmi.github.io/anatomix
Read the full abstract ▾
Computer Vision · Cross-listed from cs.NI
Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey
Han Zhang, Mohammad Farzanullah, Mohammad Ghassemi et al. (+3 more)
Foundation models (FMs) are recognized as a transformative breakthrough that has started to reshape the future of artificial intelligence (AI) across both academia and industry. The integration of FMs into wireless networks is expected to enable the development of general-purpose AI
agents capable of handling diverse network management requests and highly complex wireless-related tasks involving multi-modal data. Inspired by these ideas, this work discusses the utilization of FMs, especially multi-modal FMs in wireless networks. We focus on two important types of tasks in wireless network management: prediction tasks and control tasks. In particular, we first discuss FMs-enabled multi-modal contextual information understanding in wireless networks. Then, we explain how FMs can be applied to prediction and control tasks, respectively. Following this, we introduce the development of wireless-specific FMs from two perspectives: available
datasets for development and the methodologies used. Finally, we conclude with a discussion of the challenges and future directions for FM-enhanced wireless networks.
Read the full abstract ▾
All authors Han Zhang, Mohammad Farzanullah, Mohammad Ghassemi, Akram Bin Sediq, Ali Afana, Melike Erol-Kantarci
Computer Vision
DiffBench Meets DiffAgent: End-to-End LLM-Driven Diffusion Acceleration Code Generation
Jiajun jiao, Haowei Zhu, Puyuan Yang et al. (+8 more)
Diffusion models have achieved remarkable success in image and
video generation. However, their inherently multiple step inference process imposes substantial computational overhead, hindering real-world deployment. Accelerating
diffusion models is therefore essential, yet determining how to combine multiple model acceleration techniques remains a significant challenge. To address this issue, we introduce a framework driven by large
language models (LLMs) for automated acceleration code generation and evaluation. First, we present DiffBench, a comprehensive
benchmark that implements a three stage automated evaluation pipeline across diverse
diffusion architectures, optimization combinations and deployment scenarios. Second, we propose DiffAgent, an
agent that generates optimal acceleration strategies and codes for arbitrary
diffusion models. DiffAgent employs a closed-loop workflow in which a planning component and a debugging component iteratively refine the output of a code generation component, while a genetic
algorithm extracts performance feedback from the execution environment to guide subsequent code refinements. We provide a detailed explanation of the DiffBench construction and the design principles underlying DiffAgent. Extensive experiments show that DiffBench offers a thorough evaluation of generated codes and that DiffAgent significantly outperforms existing LLMs in producing effective
diffusion acceleration strategies.
Read the full abstract ▾
All authors Jiajun jiao, Haowei Zhu, Puyuan Yang, Jianghui Wang, Ji Liu, Ziqiong Liu, Dong Li, Yuejian Fang, Junhai Yong, Bin Wang, Emad Barsoum
Computer Vision
LSP-DETR: Efficient and Scalable Nuclei Segmentation in Whole-Slide Images
Matěj Pekár, Vít Musil, Rudolf Nenutil et al. (+2 more)
Background and Objective: Precise and scalable instance
segmentation of
cell nuclei is a fundamental prerequisite for computational pathology, yet gigapixel whole-slide images (WSIs) pose significant computational challenges. While patch-based processing is standard during training, existing methods are often limited to small tile sizes during inference due to architectural bottlenecks or reliance on computationally expensive post-processing for instance separation. We introduce a faster, scalable, and end-to-end framework capable of processing large-scale image tiles while accurately modeling biologically realistic overlapping
nuclei. Methods: We propose LSP-DETR (Local
Star Polygon DEtection
TRansformer). The model represents
nuclei as
star-convex polygons and employs a lightweight
transformer with linear complexity, enabling the processing of high-resolution images in a single forward pass. A novel radial distance loss accommodates annotation uncertainty, allowing the
segmentation of overlapping
nuclei to emerge naturally without explicit overlap labels. Results: LSP-DETR achieves state-of-the-art efficiency, with an inference time of 0.45 s/mm^2, a 3.2x speedup over StarDist, the next-fastest method. On PanNuke, the model achieves competitive accuracy (67.5 bPQ), while yielding an F$_1$-score of 0.964 in polygon overlap when evaluated against consensus annotations from two expert pathologists. Furthermore, it outperforms larger models such as LKCell in generalization robustness, reaching an F1-score of 85.0 on MoNuSeg. Conclusions: LSP-DETR bridges the gap between high-fidelity
segmentation and practical clinical requirements by eliminating heuristic post-processing. By providing a scalable, linear-complexity solution that naturally handles overlaps between
nuclei, this framework sets a new direction for efficient high-throughput WSI analysis in digital pathology.
Read the full abstract ▾
All authors Matěj Pekár, Vít Musil, Rudolf Nenutil, Petr Holub, Tomáš Brázdil
Computer Vision
Unified Thinker: A General Reasoning Modular Core for Image Generation
Sashuai Zhou, Qiang Zhou, Jijin Hu et al. (+9 more)
Despite impressive progress in high-fidelity image synthesis, generative models still struggle with logic-intensive instruction following, exposing a persistent
reasoning–execution gap. Meanwhile, closed-source systems (e.g., Nano Banana) have demonstrated strong
reasoning-driven image generation, highlighting a substantial gap to current open-source models. We argue that closing this gap requires not merely better visual generators, but executable
reasoning: decomposing high-level intents into grounded, verifiable plans that directly steer the generative process. To this end, we propose Unified Thinker, a task-agnostic
reasoning architecture for general image generation, designed as a unified planning core that can plug into diverse generators and workflows. Unified Thinker decouples a dedicated Thinker from the image Generator, enabling modular upgrades of
reasoning without retraining the entire generative model. We further introduce a two-stage training paradigm: we first build a structured planning interface for the Thinker, then apply reinforcement learning to ground its policy in pixel-level feedback, encouraging plans that optimize visual correctness over textual plausibility. Extensive experiments on text-to-image generation and image editing show that Unified Thinker substantially improves image
reasoning and generation quality.
Read the full abstract ▾
All authors Sashuai Zhou, Qiang Zhou, Jijin Hu, Hanqing Yang, Yue Cao, Junpeng Ma, Yinchao Ma, Jun Song, Tiezheng Ge, Cheng Yu, Bo Zheng, Zhou Zhao
Computer Vision
LeafLife: An Explainable Deep Learning Framework with Robustness for Grape Leaf Disease Recognition
B. M. Shahria Alam and Md. Nasim Ahmed
Plant disease diagnosis is essential to farmers' management choices because plant diseases frequently lower crop yield and product quality. For harvests to flourish and agricultural productivity to boost, grape leaf disease detection is important. The plant disease
dataset contains grape leaf diseases total of 9,032 images of four classes, among them three classes are leaf diseases, and the other one is healthy leaves. After rigorous pre-processing
dataset was split (70% training, 20% validation, 10% testing), and two pre-trained models were deployed: InceptionV3 and Xception. Xception shows a promising result of 96.23% accuracy, which is remarkable than InceptionV3.
Adversarial Training is used for robustness, along with more transparency. Grad-CAM is integrated to confirm the leaf disease. Finally deployed a web application using Streamlit with a heatmap visualization and prediction with confidence level for robust grape leaf disease classification.
Read the full abstract ▾
Computer Vision · Cross-listed from q-bio.NC
Transformers self-organize like newborn visual systems when trained in prenatal worlds
Lalit Pandey, Samantha M. W. Wood and Justin N. Wood
Do
transformers learn like
brains? A key challenge in addressing this question is that
transformers and
brains are trained on fundamentally different data.
Brains are initially "trained" on prenatal sensory experiences (e.g., retinal waves), whereas
transformers are typically trained on large
datasets that are not biologically plausible. We reasoned that if
transformers learn like
brains, then they should develop the same structure as newborn
brains when exposed to the same prenatal data. To test this prediction, we simulated prenatal visual input using a retinal wave generator. Then, using self-supervised temporal learning, we trained
transformers to adapt to those retinal waves. During training, the
transformers spontaneously developed the same structure as newborn visual systems: (1) early layers became sensitive to edges, (2) later layers became sensitive to shapes, and (3) the models developed larger receptive fields across layers. The organization of newborn visual systems emerges spontaneously when
transformers adapt to a prenatal visual world. This developmental convergence suggests that
brains and
transformers learn in common ways and follow the same general fitting principles.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
DiT-JSCC: Rethinking Deep JSCC with Diffusion Transformers and Semantic Representations
Kailin Tan, Jincheng Dai, Sixian Wang et al. (+5 more)
Generative joint source-channel coding (GJSCC) has emerged as a new Deep JSCC paradigm for achieving high-fidelity and robust image transmission under extreme wireless channel conditions, such as ultra-low bandwidth and low signal-to-noise ratio. Recent studies commonly adopt
diffusion models as generative decoders, but they frequently produce visually realistic results with limited semantic consistency. This limitation stems from a fundamental mismatch between reconstruction-oriented JSCC encoders and generative decoders, as the former lack explicit semantic discriminability and fail to provide reliable conditional cues. In this paper, we propose DiT-JSCC, a novel GJSCC backbone that can jointly learn a semantics-prioritized representation encoder and a
diffusion transformer (DiT) based generative decoder, our open-source project aims to promote the future research in GJSCC. Specifically, we design a semantics-detail dual-branch encoder that aligns naturally with a coarse-to-fine conditional DiT decoder, prioritizing semantic consistency under extreme channel conditions. Moreover, a training-free adaptive bandwidth allocation strategy inspired by Kolmogorov complexity is introduced to further improve the transmission efficiency, thereby indeed redefining the notion of information value in the era of generative decoding. Extensive experiments demonstrate that DiT-JSCC consistently outperforms existing JSCC methods in both semantic consistency and visual quality, particularly in extreme regimes.
Read the full abstract ▾
All authors Kailin Tan, Jincheng Dai, Sixian Wang, Guo Lu, Shuo Shao, Kai Niu, Wenjun Zhang, Ping Zhang
Computer Vision
TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs
Chenchen Lin, Sanbao Su, Rachel Luo et al. (+4 more)
Multimodal large
language models (MLLMs) typically rely on a single late-layer feature from a frozen
vision encoder, leaving the encoder's rich hierarchy of visual cues under-utilized. MLLMs still suffer from visually ungrounded
hallucinations, often relying on
language priors rather than image evidence. While many prior mitigation strategies operate on the text side, they leave the visual representation unchanged and do not exploit the rich hierarchy of features encoded across
vision layers. Existing multi-layer fusion methods partially address this limitation but remain static, applying the same layer mixture regardless of the query. In this work, we introduce TGIF (Text-Guided Inter-layer Fusion), a lightweight module that treats encoder layers as depth-wise "experts" and predicts a
prompt-dependent fusion of visual features. TGIF follows the principle of direct external fusion, requires no
vision-encoder updates, and adds minimal overhead. Integrated into LLaVA-1.5-7B, TGIF provides consistent improvements across
hallucination, OCR, and VQA
benchmarks, while preserving or improving performance on ScienceQA, GQA, and MMBench. These results suggest that query-conditioned, hierarchy-aware fusion is an effective way to strengthen visual grounding and reduce
hallucination in modern MLLMs. Code: https://github.com/Linchenchen/TGIF.
Read the full abstract ▾
All authors Chenchen Lin, Sanbao Su, Rachel Luo, Yuxiao Chen, Yan Wang, Marco Pavone, Fei Miao
Computer Vision
LesionTABE: Equitable AI for Skin Lesion Detection
Rocio Mexia Diaz, Yasmin Greenway and Petru Manescu
Bias remains a major barrier to the clinical adoption of AI in dermatology, as diagnostic models underperform on darker skin tones. We present LesionTABE, a
fairness-centric framework that couples
adversarial debiasing with dermatology-specific foundation model
embeddings. Evaluated across multiple
datasets covering both malignant and inflammatory conditions, LesionTABE achieves over a 25% improvement in
fairness metrics compared to a ResNet-152 baseline, outperforming existing debiasing methods while simultaneously enhancing overall diagnostic accuracy. These results highlight the potential of foundation model debiasing as a step towards equitable clinical AI adoption.
Read the full abstract ▾
Computer Vision
Understanding Multi-Agent Reasoning with Large Language Models for Cartoon VQA
Tong Wu and Thanet Markchom
Visual Question Answering (VQA) for stylised cartoon imagery presents challenges, such as interpreting exaggerated visual abstraction and narrative-driven context, which are not adequately addressed by standard large
language models (LLMs) trained on natural images. To investigate this issue, a multi-
agent LLM framework is introduced, specifically designed for VQA tasks in cartoon imagery. The proposed architecture consists of three specialised
agents: visual
agent,
language agent and critic
agent, which work collaboratively to support structured
reasoning by integrating visual cues and narrative context. The framework was systematically evaluated on two cartoon-based VQA
datasets: Pororo and Simpsons. Experimental results provide a detailed analysis of how each
agent contributes to the final prediction, offering a deeper understanding of LLM-based multi-
agent behaviour in cartoon VQA and
multimodal inference.
Read the full abstract ▾
Computer Vision
Fine-Grained Generalization via Structuralizing Concept and Feature Space into Commonality, Specificity and Confounding
Zhen Wang, Jiaojiao Zhao, Qilong Wang et al. (+2 more)
Fine-Grained Domain Generalization (FGDG) presents greater challenges than conventional domain generalization due to the subtle inter-class differences and relatively pronounced intra-class variations inherent in fine-grained recognition tasks. Under domain shifts, the model becomes overly sensitive to fine-grained cues, leading to the suppression of critical features and a significant drop in performance. Cognitive studies suggest that humans classify objects by leveraging both common and specific attributes, enabling accurate differentiation between fine-grained categories. However, current deep learning models have yet to incorporate this mechanism effectively. Inspired by this mechanism, we propose Concept-Feature Structuralized Generalization (CFSG). This model explicitly disentangles both the concept and feature spaces into three structured components: common, specific, and confounding segments. To mitigate the adverse effects of varying degrees of distribution shift, we introduce an adaptive mechanism that dynamically adjusts the proportions of common, specific, and confounding components. In the final prediction, explicit weights are assigned to each pair of components. Extensive experiments on three single-source
benchmark datasets demonstrate that CFSG achieves an average performance improvement of 9.87% over baseline models and outperforms existing state-of-the-art methods by an average of 3.08%. Additionally, explainability analysis validates that CFSG effectively integrates multi-granularity structured knowledge and confirms that feature structuralization facilitates the emergence of concept structuralization.
Read the full abstract ▾
All authors Zhen Wang, Jiaojiao Zhao, Qilong Wang, Yongfeng Dong, Wenlong Yu
Computer Vision
IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation
Yankai Jiang, Qiaoru Li, Binlu Xu et al. (+6 more)
Recent research on medical MLLMs has gradually shifted its focus from image-level understanding to fine-grained, pixel-level comprehension. Although
segmentation serves as the foundation for pixel-level understanding, existing approaches face two major challenges. First, they introduce implicit
segmentation tokens and require simultaneous fine-tuning of both the MLLM and external pixel decoders, which increases the risk of catastrophic forgetting and limits generalization to out-of-domain scenarios. Second, most methods rely on single-pass
reasoning and lack the capability to iteratively refine
segmentation results, leading to suboptimal performance. To overcome these limitations, we propose a novel agentic MLLM, named IBISAgent, that reformulates
segmentation as a
vision-centric, multi-step decision-making process. IBISAgent enables MLLMs to generate interleaved
reasoning and text-based click actions, invoke
segmentation tools, and produce high-quality masks without architectural modifications. By iteratively performing multi-step visual
reasoning on masked image features, IBISAgent naturally supports mask refinement and promotes the development of pixel-level visual
reasoning capabilities. We further design a two-stage training framework consisting of cold-start supervised fine-tuning and agentic reinforcement learning with tailored, fine-grained rewards, enhancing the model's robustness in complex medical referring and
reasoning segmentation tasks. Extensive experiments demonstrate that IBISAgent consistently outperforms both closed-source and open-source SOTA methods.
Read the full abstract ▾
All authors Yankai Jiang, Qiaoru Li, Binlu Xu, Haoran Sun, Chao Ding, Junting Dong, Yuxiang Cai, Xuhong Zhang, Jianwei Yin
Computer Vision
Siyi Lyu, Quan Liu and Feng Yan
Vision Transformers (ViTs) excel in semantic recognition but exhibit systematic failures in spatial
reasoning tasks such as mental rotation. While often attributed to data scale, this work argues that the limitation arises from the intrinsic
circuit complexity of the architecture. By formalizing spatial understanding as learning a Group Homomorphism Problem – where latent
embeddings preserve the algebraic structure of physical transformations acting on images – we identify a fundamental computational bottleneck. Specifically, for non-solvable groups (e.g., $\mathrm{SO}(3)$), maintaining such structure-preserving
embeddings is lowerbounded by the Word Problem, which is $\mathsf{NC^1}$-complete. In contrast, constant-depth ViTs with
polynomial precision are strictly bounded by the complexity class $\mathsf{TC^0}$. Under the standard
conjecture $\mathsf{TC^0} \subsetneq \mathsf{NC^1}$, a complexity boundary emerges: constant-depth architectures lack the logical depth required to capture non-solvable spatial structures in a single forward pass. To empirically validate this theoretical gap, we propose the Latent Space
Algebra (LSA)
benchmark, which reveals a significant degradation in ViT representations as the compositional depth of non-solvable tasks increases.
Read the full abstract ▾
Computer Vision
Motion Blur Robust Wheat Pest Damage Detection with Dynamic Fuzzy Feature Fusion
Han Zhang, Yanwei Wang, Fang Li et al. (+1 more)
Motion blur caused by camera shake produces ghosting artifacts that substantially degrade edge side object detection. Existing approaches either suppress blur as noise and lose discriminative structure, or apply full image restoration that increases latency and limits deployment on resource constrained devices. We propose DFRCP, a Dynamic Fuzzy Robust Convolutional Pyramid, as a plug in upgrade to YOLOv11 for blur robust detection. DFRCP enhances the YOLOv11 feature pyramid by combining large scale and medium scale features while preserving native representations, and by introducing Dynamic Robust Switch units that adaptively inject fuzzy features to strengthen global perception under jitter. Fuzzy features are synthesized by rotating and nonlinearly interpolating multiscale features, then merged through a transparency convolution that learns a content adaptive trade off between original and fuzzy cues. We further develop a CUDA parallel rotation and interpolation kernel that avoids boundary overflow and delivers more than 400 times speedup, making the design practical for edge deployment. We train with paired supervision on a private wheat pest damage
dataset of about 3,500 images, augmented threefold using two blur regimes, uniform image wide motion blur and bounding box confined rotational blur. On blurred test sets, YOLOv11 with DFRCP achieves about 10.4 percent higher accuracy than the YOLOv11 baseline with only a modest training time overhead, reducing the need for manual filtering after data collection.
Read the full abstract ▾
All authors Han Zhang, Yanwei Wang, Fang Li, Hongjun Wang
Computer Vision
Flow Matching and Diffusion Models via PointNet for Generating Fluid Fields on Irregular Geometries
Ali Kashefi
We present two novel generative
geometric deep learning frameworks, termed Flow Matching PointNet and
Diffusion PointNet, for predicting fluid flow variables on irregular geometries by incorporating PointNet into flow matching and
diffusion models, respectively. In these frameworks, a reverse generative process reconstructs physical fields from standard Gaussian noise conditioned on unseen geometries. The proposed approaches operate directly on point-cloud representations of computational domains (e.g., grid vertices of finite-volume meshes) and therefore avoid the limitations of pixelation used to project geometries onto uniform
lattices, as is common in U-Net-based flow matching and
diffusion models. In contrast to graph neural network-based
diffusion models, Flow Matching PointNet and
Diffusion PointNet do not exhibit high-frequency noise artifacts in the predicted fields. Moreover, unlike such approaches, which require auxiliary intermediate networks to condition geometry, the proposed frameworks rely solely on PointNet, resulting in a simple and unified architecture. The performance of the proposed frameworks is evaluated on steady incompressible flow past a cylinder, using a
geometric dataset constructed by varying the cylinder's cross-sectional shape and orientation across samples. The results demonstrate that Flow Matching PointNet and
Diffusion PointNet achieve more accurate predictions of velocity and pressure fields, as well as lift and drag forces, and exhibit greater robustness to incomplete geometries compared to a vanilla PointNet with the same number of trainable parameters.
Read the full abstract ▾
Computer Vision
SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection
Kim Jun-Seong, Tae-Hyun Oh, Eduardo Pérez-Pellitero et al. (+1 more)
We propose Self-Augmented Residual 3D Gaussian Splatting (SA-ResGS), a novel framework to stabilize uncertainty quantification and enhancing uncertainty-aware supervision in next-best-view (NBV) selection for active scene reconstruction. SA-ResGS improves both the reliability of uncertainty estimates and their effectiveness for supervision by generating Self-Augmented point clouds (SA-Points) via triangulation between a training view and a rasterized extrapolated view, enabling efficient scene coverage estimation. While improving scene coverage through physically guided view selection, SA-ResGS also addresses the challenge of under-supervised Gaussians, exacerbated by sparse and wide-baseline views, by introducing the first residual learning strategy tailored for 3D Gaussian Splatting. This targeted supervision enhances gradient flow in high-uncertainty Gaussians by combining uncertainty-driven filtering with dropout- and hard-negative-mining-inspired sampling. Our contributions are threefold: (1) a physically grounded view selection strategy that promotes efficient and uniform scene coverage; (2) an uncertainty-aware residual supervision scheme that amplifies learning signals for weakly contributing Gaussians, improving training stability and uncertainty estimation across scenes with diverse camera distributions; (3) an implicit unbiasing of uncertainty quantification as a consequence of constrained view selection and residual supervision, which together mitigate conflicting effects of wide-baseline exploration and sparse-view ambiguity in NBV planning. Experiments on active view selection demonstrate that SA-ResGS outperforms state-of-the-art baselines in both reconstruction quality and view selection robustness.
Read the full abstract ▾
All authors Kim Jun-Seong, Tae-Hyun Oh, Eduardo Pérez-Pellitero, Youngkyoon Jang
Computer Vision
ReCCur: A Recursive Corner-Case Curation Framework for Robust Vision-Language Understanding in Open and Edge Scenarios
Yihan Wei, Shenghai Yuan, Tianchen Deng et al. (+2 more)
Corner cases are rare or extreme scenarios that drive real-world failures, but they are difficult to curate at scale: web data are noisy, labels are brittle, and edge deployments preclude large retraining. We present ReCCur (Recursive Corner-Case Curation), a low-compute framework that converts noisy web imagery into auditable fine-grained labels via a multi-
agent recursive pipeline. First, large-scale data acquisition and filtering expands a domain vocabulary with a
vision-
language model (VLM), crawls the web, and enforces tri-modal (image, description, keyword) consistency with light human spot checks to yield refined candidates. Next, mixture-of-experts knowledge
distillation uses complementary encoders (e.g., CLIP, DINOv2, BEiT) for kNN voting with dual-confidence activation and uncertainty sampling, converging to a high-precision set. Finally, region-evidence VLM
adversarial labeling pairs a proposer (multi-granularity regions and semantic cues) with a validator (global and local chained consistency) to produce explainable labels and close the loop. On realistic corner-case scenarios (e.g., flooded-car inspection), ReCCur runs on consumer-grade GPUs, steadily improves purity and separability, and requires minimal human supervision, providing a practical substrate for downstream training and evaluation under resource constraints. Code and
dataset will be released.
Read the full abstract ▾
All authors Yihan Wei, Shenghai Yuan, Tianchen Deng, Boyang Lou, Enwen Hu
Computer Vision
Towards Efficient 3D Object Detection for Vehicle-Infrastructure Collaboration via Risk-Intent Selection
Li Wang, Boqi Li, Hang Chen et al. (+5 more)
Vehicle-Infrastructure Collaborative Perception (VICP) is pivotal for resolving occlusion in autonomous driving, yet the trade-off between communication bandwidth and feature redundancy remains a critical bottleneck. While intermediate fusion mitigates data volume compared to raw sharing, existing frameworks typically rely on spatial compression or static confidence maps, which inefficiently transmit spatially redundant features from non-critical background regions. To address this, we propose Risk-intent Selective detection (RiSe), an interaction-aware framework that shifts the paradigm from identifying visible regions to prioritizing risk-critical ones. Specifically, we introduce a Potential Field-Trajectory Correlation Model (PTCM) grounded in potential field theory to quantitatively assess kinematic risks. Complementing this, an Intention-Driven Area Prediction Module (IDAPM) leverages ego-motion priors to proactively predict and filter key Bird's-Eye-View (BEV) areas essential for decision-making. By integrating these components, RiSe implements a semantic-selective fusion scheme that transmits high-fidelity features only from high-interaction regions, effectively acting as a feature denoiser. Extensive experiments on the DeepAccident
dataset demonstrate that our method reduces communication volume to 0.71% of full feature sharing while maintaining state-of-the-art detection accuracy, establishing a competitive Pareto frontier between bandwidth efficiency and perception performance.
Read the full abstract ▾
All authors Li Wang, Boqi Li, Hang Chen, Xingjian Wu, Yichen Wang, Jiewen Tan, Xinyu Zhang, Huaping Liu
Computer Vision · Cross-listed from cs.LG
From Memorization to Creativity: LLM as a Designer of Novel Neural Architectures
Waleed Khalid, Dmitry Ignatov and Radu Timofte
Large
language models (LLMs) excel in program synthesis, yet their capacity for neural architecture design – balancing syntactic reliability, performance, and structural novelty – remains underexplored. We present a closed-loop architecture synthesis pipeline within the NNGPT framework, in which a code-oriented LLM evolves over 22 supervised fine-tuning cycles. At each cycle, the LLM synthesizes PyTorch convolutional networks, validated via low-fidelity performance signals and filtered via a MinHash–Jaccard criterion to prevent structural redundancy before being incorporated into the LEMUR
dataset. High-performing candidates with novel architectures are converted into
prompt–code pairs for parameter-efficient LoRA fine-tuning. This feedback loop drives a measurable distributional shift, progressively internalizing empirical architectural priors such that valid and high-performing outputs evolve from scarce to dominant across cycles. On CIFAR-10, the valid generation rate stabilizes at 50.6% (peaking at 74.5%), mean first-epoch accuracy rises from 28.1% to 51.0%, and candidates exceeding 40% accuracy grow from 2.0% to 96.8%. Cross-
dataset transfer to CIFAR-100 and SVHN confirms that improved validity, shifted accuracy distributions, and sustained novelty generalize across
benchmarks of varying difficulty and visual domain. Across 22 cycles, 455 unique architectures absent from the original corpus are admitted under the novelty filter. By grounding synthesis in execution feedback and novelty filtering, we demonstrate that iterative self-supervised fine-tuning reshapes an LLM into a task-specialized architectural prior – improving generation reliability, proxy performance, and structural diversity – offering a reproducible, annotation-free alternative to hand-crafted search spaces.
Read the full abstract ▾
Computer Vision
Towards Faithful Reasoning in Comics for Small MLLMs
Chengcheng Feng, Haojie Yin, Yucheng Jin et al. (+1 more)
Comic understanding presents a significant challenge for
Multimodal Large
Language Models (MLLMs), as the intended meaning of a comic often emerges from the joint interpretation of visual, textual, and social cues. This naturally motivates Chain-of-Thought (CoT) prompting, since explicit intermediate
reasoning appears promising for integrating such heterogeneous signals. However, existing CoT methods are poorly matched to this structure: they tend to force interpretation into a single
reasoning path before multiple cues have been jointly considered, often degrading performance, especially for small MLLMs. Our key idea is to explicitly preserve multi-cue interpretation during supervision construction, rather than collapsing comic understanding into a single
reasoning chain. To this end, we propose a two-stage framework for faithful comic
reasoning in small MLLMs. First, we introduce MoCoT, a modular supervision construction framework that preserves multi-cue interpretation and turns it into more faithful supervision. Second, we propose VERA, a structured reward mechanism that turns such supervision into faithful
reasoning behavior by aligning optimization with both
reasoning faithfulness and answer correctness. Extensive experiments on five
benchmarks spanning comic understanding and broader humor-centric and abstract visual
reasoning tasks demonstrate that our framework achieves strong results in the $\leq$ 4B regime, surpasses several 7B baselines, improves four small MLLMs by an average of $\mathbf{12.1%}$ as a plug-in, and consistently enhances
reasoning faithfulness while preserving inference efficiency.
Read the full abstract ▾
All authors Chengcheng Feng, Haojie Yin, Yucheng Jin, Kaizhu Huang
Computer Vision
ULS+: Data-driven Model Adaptation Enhances Lesion Segmentation
Rianne Weber, Niels Rocholl, Max de Grauw et al. (+3 more)
In this study, we present ULS+, an enhanced version of the Universal Lesion
Segmentation (ULS) model. The original ULS model segments lesions across the whole body in CT scans given volumes of interest (VOIs) centered around a click-point. Since its release, several new public
datasets have become available that can further improve model performance. ULS+ incorporates these additional
datasets and uses smaller input image sizes, resulting in higher accuracy and faster inference. We compared ULS and ULS+ using the Dice score and robustness to click-point location on the ULS23 Challenge test data and a subset of the Longitudinal-CT
dataset. In all comparisons, ULS+ significantly outperformed ULS. Additionally, ULS+ ranks first on the ULS23 Challenge test-phase leaderboard. By maintaining a cycle of data-driven updates and clinical validation, ULS+ establishes a foundation for robust and clinically relevant lesion
segmentation models.
Read the full abstract ▾
All authors Rianne Weber, Niels Rocholl, Max de Grauw, Mathias Prokop, Ewoud Smit, Alessa Hering
Computer Vision
LAMS-Edit: Latent and Attention Mixing with Schedulers for Improved Content Preservation in Diffusion-Based Image and Style Editing
Wingwa Fu and Takayuki Okatani
Text-to-Image editing using
diffusion models faces challenges in balancing content preservation with edit application and handling real-image editing. To address these, we propose LAMS-Edit, leveraging intermediate states from the inversion process–an essential step in real-image editing–during edited image generation. Specifically, latent representations and
attention maps from both processes are combined at each step using weighted interpolation, controlled by a scheduler. This technique, Latent and
Attention Mixing with Schedulers (LAMS), integrates with
Prompt-to-
Prompt (P2P) to form LAMS-Edit–an extensible framework that supports precise editing with region masks and enables style transfer via LoRA. Extensive experiments demonstrate that LAMS-Edit effectively balances content preservation and edit application.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Low-Resource Heuristics for Bahnaric Optical Character Recognition Improvement
Phat Tran, Phuoc Pham, Hung Trinh et al. (+1 more)
Bahnar, a minority
language spoken across Vietnam, Cambodia, and Laos, faces significant preservation challenges due to limited research and data availability. This study addresses the critical need for accurate digitization of Bahnar
language documents through optical character recognition (OCR) technology. Digitizing scanned paper documents poses significant challenges, as degraded image quality from broken or blurred areas introduces considerable OCR errors that compromise information
retrieval systems. We propose a comprehensive approach combining advanced table and non-table detection techniques with probability-based post-processing heuristics to enhance recognition accuracy. Our method first applies detection
algorithms to improve input data quality, then employs probabilistic error correction on OCR output. Experimental results indicate a substantial improvement, with recognition accuracy increasing from 72.86% to 79.26%. This work contributes valuable resources for Bahnar
language preservation and provides a framework applicable to other minority
language digitization efforts.
Read the full abstract ▾
All authors Phat Tran, Phuoc Pham, Hung Trinh, Tho Quan
Computer Vision
VTONQA: A Multi-Dimensional Quality Assessment Dataset for Virtual Try-on
Xinyi Wei, Sijing Wu, Zitong Xu et al. (+4 more)
With the rapid development of e-commerce and digital fashion, image-based virtual try-on (VTON) has attracted increasing
attention. However, existing VTON models often suffer from artifacts such as garment distortion and body inconsistency, highlighting the need for reliable quality evaluation of VTON-generated images. To this end, we construct VTONQA, the first multi-dimensional quality assessment
dataset specifically designed for VTON, which contains 8,132 images generated by 11 representative VTON models, along with 24,396 mean opinion scores (MOSs) across three evaluation dimensions (i.e., clothing fit, body compatibility, and overall quality). Based on VTONQA, we
benchmark both VTON models and a diverse set of image quality assessment (IQA) metrics, revealing the limitations of existing methods and highlighting the value of the proposed
dataset. We believe that the VTONQA
dataset and corresponding
benchmarks will provide a solid foundation for perceptually aligned evaluation, benefiting both the development of quality assessment methods and the advancement of VTON models. The
dataset we proposed in this paper is publicly available at: https://huggingface.co/
datasets/weixiny0408/vtonqa.
Read the full abstract ▾
All authors Xinyi Wei, Sijing Wu, Zitong Xu, Yunhao Li, Huiyu Duan, Jia Wang, Ning Liu
Computer Vision
HybridSolarNet: A Lightweight and Explainable EfficientNet-CBAM Architecture for Real-Time Solar Panel Fault Detection
Md. Asif Hossain, G M Mota-Tahrin Tayef and Nabil Subhan
Manual inspections for
solar panel systems are a tedious, costly, and error-prone task, making it desirable for Unmanned Aerial Vehicle (
UAV) based monitoring. Though deep learning models have excellent fault detection capabilities, almost all methods either are too large and heavy for edge computing devices or involve biased estimation of accuracy due to ineffective learning techniques. We propose a new
solar panel fault detection model called HybridSolarNet. It integrates EfficientNet-B0 with Convolutional Block
Attention Module (CBAM). We implemented it on the Kaggle
Solar Panel Images competition
dataset with a tight split-before-augmentation protocol. It avoids leakage in accuracy estimation. We introduced focal loss and cosine annealing. Ablation analysis validates that accuracy boosts due to added benefits from CBAM (+1.53%) and that there are benefits from recognition of classes with imbalanced samples via focal loss. Overall average accuracy on 5-fold stratified cross-validation experiments on the given competition
dataset topped 92.37% +/- 0.41 and an F1-score of 0.9226 +/- 0.39 compared to baselines like VGG19, requiring merely 16.3 MB storage, i.e., 32 times less. Its inference speed measured at 54.9 FPS with GPU support makes it a successful candidate for real-time
UAV implementation. Moreover, visualization obtained from Grad-CAM illustrates that HybridSolarNet focuses on actual locations instead of irrelevant ones.
Read the full abstract ▾
Computer Vision
PrismVAU: Prompt-Refined Inference System for Multimodal Video Anomaly Understanding
Iñaki Erregue, Kamal Nasrollahi and Sergio Escalera
Video Anomaly Understanding (VAU) extends traditional
Video Anomaly Detection (VAD) by not only localizing anomalies but also describing and
reasoning about their context. Existing VAU approaches often rely on fine-tuned
multimodal large
language models (MLLMs) or external modules such as
video captioners, which introduce costly annotations, complex training pipelines, and high inference overhead. In this work, we introduce PrismVAU, a lightweight yet effective system for real-time VAU that leverages a single off-the-shelf MLLM for anomaly scoring, explanation, and
prompt optimization. PrismVAU operates in two complementary stages: (1) a coarse anomaly scoring module that computes frame-level anomaly scores via similarity to textual anchors, and (2) an MLLM-based refinement module that contextualizes anomalies through system and user
prompts. Both textual anchors and
prompts are optimized with a weakly supervised Automatic
Prompt Engineering (APE) framework. Extensive experiments on standard VAD
benchmarks demonstrate that PrismVAU delivers competitive detection performance and interpretable anomaly explanations – without relying on instruction tuning, frame-level annotations, and external modules or dense processing – making it an efficient and practical solution for real-world applications.
Read the full abstract ▾