Computer Vision · Cross-listed from cs.CL
Detecting AI-Generated Content on Social Media with Multi-modal Language Models
Chenyang Yang, Shen Yan, Yibo Yang et al. (+13 more)
Generative AI has enabled the creation of photorealistic images and
videos that are increasingly disseminated on social media, often used for spam, misinformation, manipulation, and fraud. Existing AI-generated content (AIGC) detection methods face challenges including poor generalization to new generation models, reliance on single modalities, and lack of interpretable explanations. We present our pipeline that mitigates these issues by continuously curating diverse multi-modal social media data and training a compact
vision-
language model for detection and explanation. Our model achieves state-of-the-art detection performance on public
benchmarks and demonstrates robust detection and explanation capabilities on internal social media
datasets across multiple platforms. We deployed our model for post recommendation on social media platforms and observed positive downstream impacts on user engagement, demonstrating that it is feasible to perform effective AIGC detection in dynamic, real-world social media environments.
Read the full abstract ▾
All authors Chenyang Yang, Shen Yan, Yibo Yang, Litao Hu, Yuchen Liu, Yuan Zeng, Hanchao Yu, Yinan Zhu, Sumedha Singla, Brian Vanover, Huijun Qian, Zihao Wang, Fujun Liu, Aashu Singh, Jianyu Wang, Xuewen Zhang
Computer Vision · Cross-listed from cs.CL
CAPruner: Conceptual-Adjacent Scene Graph Pruner for Enhancing 3D Spatial Reasoning of Large Language Models
Shengli Zhou, Xiangchen Wang, Guanhua Chen et al. (+1 more)
Large
language models (LLMs) have recently been applied to 3D
vision-
language (3D-VL) tasks, which require spatial
reasoning to identify target objects relative to anchors. Scene graphs are commonly employed to represent such relations, but
reasoning over complete graphs incurs high
token costs and computational inefficiencies, motivating the need for
pruning. Existing
pruning methods primarily rely on spatial proximity and often remove task-relevant relations, thereby undermining reliable spatial
reasoning. To address these limitations, we derive a key requirement for scene graph
pruning: preserving spatial relations that are most pertinent to the specific 3D-VL task. Guided by this insight, we propose the Conceptual-Adjacent Scene Graph Pruner (CAPruner). CAPruner integrates fuzzy semantic relevance with spatial proximity to estimate the importance of relations, enabling the selection of critical relations in a task-specific context. Moreover, to avoid costly relation-level annotations, CAPruner is trained by supervising the aggregated scores of each node's incident edges. Extensive experiments demonstrate that CAPruner effectively preserves relations essential for spatial
reasoning, leading to substantial performance improvements of LLMs on 3D-VL tasks. Code is available at https://github.com/fz-zsl/CAPruner.
Read the full abstract ▾
All authors Shengli Zhou, Xiangchen Wang, Guanhua Chen, Feng Zheng
Computer Vision
Unified Multi-Foundation-Model Slide Representation for Pan-Cancer Recognition and Text-Guided Tumor Localization
Tianyang Wang, Ziyu Su, Abdul Rehman Akbar et al. (+6 more)
The expanding ecosystem of pathology foundation models has produced powerful but fragmented tile-level representations, limiting their use in clinical tasks that require unified slide-level
reasoning and interpretable linkage to clinically meaningful information. We present ASTRA, a pan-cancer framework that integrates heterogeneous foundation-model representations into a shared slide-level representation space and semantically grounds that space using structured pathology annotation fields, including classification category, cancer type, and anatomic site. ASTRA combines sparse mixture-of-experts contextualization, masked multi-model reconstruction, and contrastive
alignment to structured pathology
prompts to learn slide representations that support 4-category classification, 3-class solid tumor typing, 16-class cancer typing, and text-guided tumor localization without pixel-level supervision. Developed on a CHTN cohort of 10,359 whole-slide images (WSIs) spanning 16 tumor types, ASTRA consistently improves pan-cancer classification across four pathology foundation-model backbones, achieving up to 97.8% macro-AUC for 4-category classification, 99.7% for 3-class solid tumor typing, and 99.2% for 16-class cancer typing. For tumor localization, ASTRA achieves a mean Dice of 0.897 on an annotated in-domain CHTN subset (n = 380) spanning 16 cancer types and 0.738 on an external TCGA cohort (n = 1,686) spanning four cancer types. These results demonstrate that minimal structured pathology annotation fields derived from slide-level metadata can provide effective semantic supervision for unified slide representation learning, enabling both pan-cancer prediction and weakly supervised tumor localization within a single framework.
Read the full abstract ▾
All authors Tianyang Wang, Ziyu Su, Abdul Rehman Akbar, Usama Sajjad, Lina Gokhale, Charles Rabolli, Wei Chen, Anil Parwani, Muhammad Khalid Khan Niazi
Computer Vision
EX-FIQA: Leveraging Intermediate Early eXit Representations from Vision Transformers for Face Image Quality Assessment
Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira et al. (+4 more)
Face Image Quality Assessment is crucial for reliable face recognition systems, yet existing
Vision Transformer-based approaches rely exclusively on final-layer representations, ignoring quality-relevant information captured at intermediate network depths. This paper presents the first comprehensive investigation of how intermediate representations within ViTs contribute to face quality assessment through early exit mechanisms and score fusion strategies. We systematically analyze all twelve
transformer blocks of ViT-FIQA architectures, demonstrating that different depths capture distinct and complementary quality-relevant information, as evidenced by varying
attention patterns and performance characteristics across network layers. We propose a score fusion framework that combines quality predictions from multiple
transformer blocks without architectural modifications or additional training. Our early exit analysis reveals optimal performance-efficiency trade-offs, enabling significant computational savings while maintaining competitive performance. Through extensive evaluation across eight
benchmark datasets using four FR models, we demonstrate that our fusion strategy improves upon single-exit approaches. Our proposed quality fusion approach employs depth-weighted averaging that assigns progressively higher importance to deeper
transformer blocks, achieving the best quality assessment performance by effectively leveraging the hierarchical nature of feature learning in ViTs. Our work challenges the conventional wisdom that only deep features matter for face analysis, revealing that intermediate representations contain valuable information for quality assessment. The proposed framework offers practical benefits for real-world biometric systems by enabling adaptive computation based on resource constraints while maintaining competitive quality assessment capabilities.
Read the full abstract ▾
All authors Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Jan Niklas Kolf, Andrea Atzori, Fadi Boutros, Naser Damer
Computer Vision
ATTN-FIQA: Interpretable Attention-based Face Image Quality Assessment with Vision Transformers
Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira et al. (+5 more)
Face Image Quality Assessment (FIQA) aims to assess the recognition utility of face samples and is essential for reliable face recognition (FR) systems. Existing approaches require computationally expensive procedures such as multiple forward passes, backpropagation, or additional training, and only recent work has focused on the use of
Vision Transformers. Recent studies highlighted that these architectures inherently function as saliency learners with
attention patterns naturally encoding spatial importance. This work proposes ATTN-FIQA, a novel training-free approach that investigates whether pre-softmax
attention scores from pre-trained
Vision Transformer-based face recognition models can serve as quality indicators. We hypothesize that
attention magnitudes intrinsically encode quality: high-quality images with discriminative facial features enable strong query-key
alignments producing focused, high-magnitude
attention patterns, while degraded images generate diffuse, low-magnitude patterns. ATTN-FIQA extracts pre-softmax
attention matrices from the final
transformer block, aggregate multi-head
attention information across all patches, and compute image-level quality scores through simple averaging, requiring only a single forward pass through pre-trained models without architectural modifications, backpropagation, or additional training. Through comprehensive evaluation across eight
benchmark datasets and four FR models, this work demonstrates that
attention-based quality scores effectively correlate with face image quality and provide spatial interpretability, revealing which facial regions contribute most to quality determination.
Read the full abstract ▾
All authors Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Jan Niklas Kolf, Marco Huber, Andrea Atzori, Naser Damer, Fadi Boutros
Computer Vision
AeSlides: Incentivizing Aesthetic Layout in LLM-Based Slide Generation via Verifiable Rewards
Yiming Pan, Chengwei Hu, Xuancheng Huang et al. (+6 more)
Large
language models (LLMs) have demonstrated strong potential in agentic tasks, particularly in slide generation. However, slide generation poses a fundamental challenge: the generation process is text-centric, whereas its quality is governed by visual aesthetics. This modality gap leads current models to frequently produce slides with aesthetically suboptimal layouts. Existing solutions typically rely either on heavy visual reflection, which incurs high inference cost yet yields limited gains; or on fine-tuning with large-scale
datasets, which still provides weak and indirect aesthetic supervision. In contrast, the explicit use of aesthetic principles as supervision remains unexplored. In this work, we present AeSlides, a reinforcement learning framework with verifiable rewards for Aesthetic layout supervision in Slide generation. We introduce a suite of meticulously designed verifiable metrics to quantify slide layout quality, capturing key layout issues in an accurate, efficient, and low-cost manner. Leveraging these verifiable metrics, we develop a GRPO-based reinforcement learning method that directly optimizes slide generation models for aesthetically coherent layouts. With only 5K training
prompts on GLM-4.7-Flash, AeSlides improves aspect ratio compliance from 36% to 85%, while reducing whitespace by 44%, element collisions by 43%, and visual imbalance by 28%. Human evaluation further shows a substantial improvement in overall quality, increasing scores from 3.31 to 3.56 (+7.6%), outperforming both model-based reward optimization and reflection-based agentic approaches, and even edging out Claude-Sonnet-4.5. These results demonstrate that such a verifiable aesthetic paradigm provides an efficient and scalable approach to aligning slide generation with human aesthetic preferences. Our repository is available at https://github.com/ympan0508/aeslides.
Read the full abstract ▾
All authors Yiming Pan, Chengwei Hu, Xuancheng Huang, Can Huang, Mingming Zhao, Yuean Bi, Xiaohan Zhang, Aohan Zeng, Linmei Hu
Computer Vision
From Skeletons to Pixels: Few-Shot Precise Event Spotting via Representation and Prediction Distillation
Zhong Han Ervin Yeoh and Jiang Kan
Precise Event Spotting (PES) is essential in fast-paced sports such as tennis, where fine-grained events occur within very short temporal windows. Accurate frame-level localization is challenging because of motion blur, subtle action differences, and limited annotated data. We study two complementary
distillation strategies for few-shot PES: Adaptive Weight
Distillation (AWD), a prediction-level method that adaptively weights teacher supervision on unlabeled data, and Annealed
Multimodal Distillation for Few-Shot Event Detection (AMD-FED), a representation-level framework that transfers robust skeleton knowledge into visual modalities through annealed pseudo-labeling. Both methods use
multimodal distillation to improve generalization under limited supervision. We evaluate them on F3Set-Tennis(sub) under few-shot k-clip settings, where they consistently outperform single-modality baselines and prior PES approaches. After observing the stronger performance of representation-level
distillation on tennis, we further validate AMD-FED on a second sports
dataset, Figure Skating, where it also shows robust performance in the k-clip scenario. These results highlight the effectiveness of
multimodal distillation, especially representation-level transfer, for few-shot precise event spotting.
Read the full abstract ▾
Computer Vision
Neural Network Optimization Reimagined: Decoupled Techniques for Scratch and Fine-Tuning
Xin Ning, Qiankun Li, Xiaolong Huang et al. (+5 more)
With the accumulation of resources in the era of big data and the rise of pre-trained models in deep learning, optimizing neural networks for various tasks often involves different strategies for fine-tuning pre-trained models versus training from scratch. However, existing optimizers primarily focus on reducing the loss function by updating model parameters, without fully addressing the unique demands of these two major paradigms. In this paper, we propose DualOpt, a novel approach that decouples optimization techniques specifically tailored for these distinct training scenarios. For training from scratch, we introduce real-time layer-wise weight decay, designed to enhance both convergence and generalization by aligning with the characteristics of weight updates and network architecture. For more importantly fine-tuning, we integrate weight rollback with the optimizer, incorporating a rollback term into each weight update step. This ensures consistency in the weight distribution between upstream and downstream models, effectively mitigating knowledge forgetting and improving fine-tuning performance. Additionally, we extend the layer-wise weight decay to dynamically adjust the rollback levels across layers, adapting to the varying demands of different downstream tasks. Extensive experiments across diverse tasks, including image classification, object detection, semantic
segmentation, and instance
segmentation, demonstrate the broad applicability and state-of-the-art performance of DualOpt. Code is available at https://github.com/qklee-lz/OLOR-AAAI-2024.
Read the full abstract ▾
All authors Xin Ning, Qiankun Li, Xiaolong Huang, Qiupu Chen, Feng He, Weijun Li, Prayag Tiwari, Xinwang Liu
Computer Vision
PASTA: A Patch-Agnostic Twofold-Stealthy Backdoor Attack on Vision Transformers
Dazhuang Liu, Yanqi Qiao, Rui Wang et al. (+2 more)
Vision Transformers (ViTs) have achieved remarkable success across
vision tasks, yet recent studies show they remain vulnerable to
backdoor attacks. Existing patch-wise attacks typically assume a single fixed trigger location during inference to maximize trigger
attention. However, they overlook the self-
attention mechanism in ViTs, which captures long-range dependencies across patches. In this work, we observe that a patch-wise trigger can achieve high attack effectiveness when activating
backdoors across neighboring patches, a phenomenon we term the Trigger Radiating Effect (TRE). We further find that inter-patch trigger insertion during training can synergistically enhance TRE compared to single-patch insertion. Prior ViT-specific attacks that maximize trigger
attention often sacrifice visual and
attention stealthiness, making them detectable. Based on these insights, we propose PASTA, a twofold stealthy patch-wise
backdoor attack in both pixel and
attention domains. PASTA enables
backdoor activation when the trigger is placed at arbitrary patches during inference. To achieve this, we introduce a multi-location trigger insertion strategy to enhance TRE. However, preserving stealthiness while maintaining strong TRE is challenging, as TRE is weakened under stealthy constraints. We therefore formulate a bi-level optimization problem and propose an adaptive
backdoor learning framework, where the model and trigger iteratively adapt to each other to avoid local optima. Extensive experiments show that PASTA achieves 99.13% attack success rate across arbitrary patches on average, while significantly improving visual and
attention stealthiness (144.43x and 18.68x) and robustness (2.79x) against state-of-the-art ViT defenses across four
datasets, outperforming CNN- and ViT-based baselines.
Read the full abstract ▾
All authors Dazhuang Liu, Yanqi Qiao, Rui Wang, Kaitai Liang, Georgios Smaragdakis
Computer Vision
Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training
Yangming Zhang, Jian Xu, Chaojian Li et al. (+7 more)
3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis with high-quality rendering through continuous aggregations of millions of 3D Gaussian primitives. However, it suffers from a substantial
memory footprint, particularly during training due to uncontrolled densification, posing a critical bottleneck for deployment on
memory-constrained edge devices. While existing methods prune redundant Gaussians post-training, they fail to address the peak
memory spikes caused by the abrupt growth of Gaussians early in the training process. To solve the training
memory consumption problem, we propose a systematic
memory-bounded training framework that dynamically optimizes Gaussians through iterative growth and
pruning. In other words, the proposed framework alternates between incremental
pruning of low-impact Gaussians and strategic growing of new primitives with an adaptive Gaussian compensation, maintaining a near-constant low
memory usage while progressively refining rendering fidelity. We comprehensively evaluate the proposed training framework on various real-world
datasets under strict
memory constraints, showing significant improvements over existing state-of-the-art methods. Particularly, our proposed method practically enables
memory-efficient 3DGS training on NVIDIA Jetson AGX Xavier, achieving similar visual quality with up to 80% lower peak training
memory consumption than the original 3DGS.
Read the full abstract ▾
All authors Yangming Zhang, Jian Xu, Chaojian Li, Kunxiong Zhu, Wei Niu, Gagan Agrawal, Yang Katie Zhao, Jian Wang, Yingyan Celine Lin, Miao Yin
Computer Vision
Normalizing Flows with Iterative Denoising
Tianrong Chen, Jiatao Gu, David Berthelot et al. (+2 more)
Normalizing Flows (NFs) are a classical family of likelihood-based methods that have received revived
attention. Recent efforts such as TARFlow have shown that NFs are capable of achieving promising performance on image modeling tasks, making them viable alternatives to other methods such as
diffusion models. In this work, we further advance the state of Normalizing Flow generative models by introducing iterative TARFlow (iTARFlow). Unlike
diffusion models, iTARFlow maintains a fully end-to-end, likelihood-based objective during training. During sampling, it performs autoregressive generation followed by an iterative denoising procedure inspired by
diffusion-style methods. Through extensive experiments, we show that iTARFlow achieves competitive performance across ImageNet resolutions of 64, 128, and 256 pixels, demonstrating its potential as a strong generative model and advancing the frontier of Normalizing Flows. In addition, we analyze the characteristic artifacts produced by iTARFlow, offering insights that may shed light on future improvements. Code is available at https://github.com/apple/ml-itarflow.
Read the full abstract ▾
All authors Tianrong Chen, Jiatao Gu, David Berthelot, Joshua Susskind, Shuangfei Zhai
Computer Vision
FluSplat: Sparse-View 3D Editing without Test-Time Optimization
Haitao Huang, Shin-Fang Chng, Huangying Zhan et al. (+2 more)
Recent advances in text-guided image editing and 3D Gaussian Splatting (3DGS) have enabled high-quality 3D scene manipulation. However, existing pipelines rely on iterative edit-and-fit optimization at test time, alternating between 2D
diffusion editing and 3D reconstruction. This process is computationally expensive, scene-specific, and prone to cross-view inconsistencies. We propose a feed-forward framework for cross-view consistent 3D scene editing from sparse views. Instead of enforcing consistency through iterative 3D refinement, we introduce a cross-view regularization scheme in the image domain during training. By jointly supervising multi-view edits with
geometric alignment constraints, our model produces view-consistent results without per-scene optimization at inference. The edited views are then lifted into 3D via a feedforward 3DGS model, yielding a coherent 3DGS representation in a single forward pass. Experiments demonstrate competitive editing fidelity and substantially improved cross-view consistency compared to optimization-based methods, while reducing inference time by orders of magnitude.
Read the full abstract ▾
All authors Haitao Huang, Shin-Fang Chng, Huangying Zhan, Qingan Yan, Yi Xu
Computer Vision
Learning to count small and clustered objects with application to bacterial colonies
Minghua Zheng, Na Helian, Peter C. R. Lane et al. (+2 more)
Automated bacterial colony counting from images is an important technique to obtain data required for the development of vaccines and antibiotics. However, bacterial colonies present unique machine
vision challenges that affect counting, including (1) small physical size, (2) object clustering, (3) high data annotation cost, and (4) limited cross-species generalisation. While FamNet is an established object counting technique effective for clustered objects and costly data annotation, its effectiveness for small colony sizes and cross-species generalisation remains unknown. To address the first three challenges, we propose ACFamNet, an extension of FamNet that handles small and clustered objects using a novel region of interest pooling with
alignment and optimised feature engineering. To address all four challenges above, we introduce ACFamNet Pro, which augments ACFamNet with multi-head
attention and residual connections, enabling dynamic weighting of objects and improved gradient flow. Experiments show that ACFamNet Pro achieves a mean normalised absolute error (MNAE) of 9.64% under 5-fold cross-validation, outperforming ACFamNet and FamNet by 2.23% and 12.71%, respectively.
Read the full abstract ▾
All authors Minghua Zheng, Na Helian, Peter C. R. Lane, Yi Sun, Allen Donald
Computer Vision
Ethan Knights
For state-of-the-art image understanding,
Vision Transformers (ViTs) have become the standard architecture but their processing diverges substantially from human attentional characteristics. We investigate whether this cognitive gap can be shrunk by fine-tuning the self-
attention weights of Google's ViT-B/16 on human saliency fixation maps. To isolate the effects of semantically relevant signals from generic human supervision, the tuned model is compared against a shuffled control. Fine-tuning significantly improved
alignment across five saliency metrics and induced three hallmark human-like biases: tuning reversed the baseline's anti-human large-object bias toward small-objects, amplified the animacy preference and diminished extreme
attention entropy.
Bayesian parity analysis provides decisive to very-strong evidence that this cognitive
alignment comes at no cost to the model's original classification performance on in- (ImageNet), corrupted (ImageNet-C) and out-of-distribution (ObjectNet)
benchmarks. An equivalent procedure applied to a ResNet-50 Convolutional Neural Network (CNN) instead degraded both
alignment and accuracy, suggesting that the ViT's modular self-
attention mechanism is uniquely suited for dissociating spatial priority from representational logic. These findings demonstrate that biologically grounded priors can be instilled as a free emergent property of human-aligned
attention, to improve
transformer interpretability.
Read the full abstract ▾
Computer Vision
Investigation of cardinality classification for bacterial colony counting using explainable artificial intelligence
Minghua Zheng, Na Helian, Peter C. R. Lane et al. (+2 more)
Automatic bacterial colony counting is a highly sought-after technology in modern biological laboratories because it eliminates manual counting effort. Previous work has observed that MicrobiaNet, currently the best-performing cardinality classification model for colony counting, has difficulty distinguishing colonies of three or more individuals. However, it is unclear if this is due to properties of the data together with inherent characteristics of the MicrobiaNet model. By analysing MicrobiaNet with explainable artificial intelligence (XAI), we demonstrate that XAI can provide insights into how data properties constrain cardinality classification performance in colony counting. Our results show that high visual similarity across classes is the key issue hindering further performance improvement, revising prior assertions about MicrobiaNet. These findings suggest future work should focus on models that explicitly incorporate visual similarity or explore density estimation approaches, with broader implications for neural network classifiers trained on imbalanced
datasets.
Read the full abstract ▾
All authors Minghua Zheng, Na Helian, Peter C. R. Lane, Yi Sun, Allen Donald
Computer Vision
EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training
Yiyang Du, Zhanqiu Guo, Haochen Zhang et al. (+5 more)
Vision-
Language-Action Models (VLAs) inherit their visual and linguistic capabilities from
Vision-
Language Models (VLMs), yet most VLAs are built from off-the-shelf VLMs that are not adapted to the
embodied domain, limiting their downstream performance. In this work, we propose EmbodiedMidtrain to bridge the gap between VLMs and VLAs. We first characterize the data distribution gap between them, showing that VLA data occupy compact regions that are largely separated from the broader VLM distribution, while the degree of
alignment varies substantially both across and within VLM data sources. Then, we build a mid-training data engine that leverages a lightweight learnable proximity
estimator to select the most VLA-aligned candidates from a large VLM pool, and produce EmbodiedMidtrain-1M, a VLM
dataset that we further use to mid-train the VLM before downstream VLA fine-tuning. Experiments on both simulated
robot manipulation
benchmarks and real-world tasks show consistent VLA performance gains over off-the-shelf checkpoints across VLM backbones and VLA action heads. These gains extend to downstream tasks not used to guide data selection. Further analysis reveals that mid-training provides a stronger initialization for VLA fine-tuning, with gains emerging from the earliest steps and widening throughout training. Moreover, the data engine captures both
dataset-level and sample-level
alignment signals, favoring spatial
reasoning over text-centric tasks while preserving the diversity of the VLM data. We will release all code, data and models for future research.
Read the full abstract ▾
All authors Yiyang Du, Zhanqiu Guo, Haochen Zhang, Annabella Macaluso, Xin Ye, Liu Ren, Yonatan Bisk, Chenyan Xiong
Computer Vision
RareSpot+: A Benchmark, Model, and Active Learning Framework for Small and Rare Wildlife in Aerial Imagery
Bowen Zhang, Jesse T. Boulerice, Charvi Mendiratta et al. (+4 more)
Automated wildlife monitoring from aerial imagery is vital for conservation but remains limited by two persistent challenges: the difficulty of detecting small, rare species and the high cost of large-scale expert annotation. Prairie dogs exemplify this problem – they are ecologically important yet appear tiny, sparsely distributed, and visually indistinct from their surroundings, posing a severe challenge for conventional detection models. To overcome these limitations, we present RareSpot+, a detection framework that integrates multi-scale consistency learning, context-aware augmentation, and geospatially guided active learning to address these issues. A novel multi-scale consistency loss aligns intermediate feature maps across detection heads, enhancing localization of small (approx. 30 pixels wide) objects without architectural changes, while context-aware augmentation improves robustness by synthesizing hard, ecologically plausible examples. A geospatial active learning module exploits domain-specific spatial priors linking prairie dogs and burrows, together with test-time augmentation and a meta-uncertainty model, to reduce redundant labeling. On a 2 km^2 aerial
dataset, RareSpot+ improves detection over the baseline mAP@50 by +35.2% (absolute +0.13). Cross-
dataset tests on HerdNet, AED, and several other wildlife
benchmarks demonstrate robust detector-level transferability. The active learning module further boosts prairie dog AP by 14.5% using an annotation budget of just 1.7% of the unlabeled tiles. Beyond detection, RareSpot+ enables spatial ecological analyses such as clustering and co-occurrence, linking
vision-based detection with quantitative ecology.
Read the full abstract ▾
All authors Bowen Zhang, Jesse T. Boulerice, Charvi Mendiratta, Nikhil Kuniyil, Satish Kumar, Hila Shamon, B. S. Manjunath
Computer Vision
Optimizing Data Augmentation for Real-Time Small UAV Detection: A Lightweight Context-Aware Approach
Amir Zamani and Zeinab Abedini
Visual detection of Unmanned Aerial Vehicles (
UAVs) is a critical task in surveillance systems due to their small physical size and environmental challenges. Although deep learning models have achieved significant progress, deploying them on edge devices necessitates the use of lightweight models, such as YOLOv11 Nano, which possess limited learning capacity. In this research, an efficient and context-aware data augmentation pipeline, combining Mosaic strategies and HSV color-space adaptation, is proposed to enhance the performance of these models. Experimental results on four standard
datasets demonstrate that the proposed approach, compared to heavy and instance-level methods like Copy-Paste, not only prevents the generation of synthetic artifacts and overfitting but also significantly improves mean Average Precision (mAP) across all scenarios. Furthermore, the evaluation of generalization capability under foggy conditions revealed that the proposed method offers the optimal balance between Precision and stability for real-time systems, whereas alternative methods, such as MixUp, are effective only in specific applications.
Read the full abstract ▾
Computer Vision
A Computational Model of Message Sensation Value in Short Video Multimodal Features that Predicts Sensory and Behavioral Engagement
Haoning Xue, Jingwen Zhang, Xiaohui Wang et al. (+2 more)
The contemporary media landscape is characterized by sensational short
videos. While prior research examines the effects of individual
multimodal features, the collective impact of
multimodal features on viewer engagement with short
videos remains unknown. Grounded in the theoretical framework of Message Sensation Value (MSV), this study develops and tests a computational model of MSV with
multimodal feature analysis and human evaluation of 1,200 short
videos. This model that predicts sensory and behavioral engagement was further validated across two unseen
datasets from three short
video platforms (combined N = 14,492). While MSV is positively associated with sensory engagement, it shows an inverted U-shaped relationship with behavioral engagement: Higher MSV elicits stronger sensory stimulation, but moderate MSV optimizes behavioral engagement. This research advances the theoretical understanding of short
video engagement and introduces a robust computational tool for short
video research.
Read the full abstract ▾
All authors Haoning Xue, Jingwen Zhang, Xiaohui Wang, Diane Dagyong Kim, Yunya Song
Computer Vision
Online CS-based SAR Edge-Mapping
Conor Flynn, Radoslav Ivanov and Birsen Yazici
With modern defense applications increasingly relying on inexpensive, small Unmanned Aerial Vehicles (
UAVs), a major challenge lies in designing intelligent and computationally efficient onboard Automatic Target Recognition (ATR)
algorithms to carry out operational objectives. This is especially critical in Synthetic Aperture Radar (SAR), where processing techniques such as ATR are often carried out post data collection, requiring onboard systems to bear the
memory burden of storing the back-scattered signals. To alleviate this high cost, we propose an online, direct, edge-mapping technique which bypasses the image reconstruction step to classify scenes and targets. Furthermore, by reconstructing the scene as an edge-map we inherently promote sparsity, requiring fewer measurements and computational power than classic SAR reconstruction
algorithms such as backprojection.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Fast Amortized Fitting of Scientific Signals Across Time and Ensembles via Transferable Neural Fields
Sophia Zorek, Kushal Vyas, Yuhao Liu et al. (+3 more)
Neural fields, also known as implicit neural representations (INRs), offer a powerful framework for modeling continuous geometry, but their effectiveness in high-dimensional scientific settings is limited by slow convergence and scaling challenges. In this study, we extend INR models to handle spatiotemporal and multivariate signals and show how INR features can be transferred across scientific signals to enable efficient and scalable representation across time and ensemble runs in an amortized fashion. Across controlled transformation regimes (e.g.,
geometric transformations and localized perturbations of synthetic fields) and high-fidelity scientific domains-including turbulent flows, fluid-material impact dynamics, and astrophysical systems-we show that transferable features improve not only signal fidelity but also the accuracy of derived
geometric and physical quantities, including density gradients and vorticity. In particular, transferable features reduce iterations to reach target reconstruction quality by up to an order of magnitude, increase early-stage reconstruction quality by multiple dB (with gains exceeding 10 dB in some cases), and consistently improve gradient-based physical accuracy.
Read the full abstract ▾
All authors Sophia Zorek, Kushal Vyas, Yuhao Liu, David Lenz, Tom Peterka, Guha Balakrishnan
Computer Vision
Lucky High Dynamic Range Smartphone Imaging
Baiang Li, Ruyu Yan, Ethan Tseng et al. (+4 more)
While the human eye can perceive an impressive twenty stops of dynamic range, smartphone camera sensors remain limited to about twelve stops despite decades of research. A variety of high dynamic range (HDR) image capture and processing techniques have been proposed, and, in practice, they can extend the dynamic range by 3-5 stops for handheld photography. This paper proposes an approach that robustly captures dynamic range using a handheld smartphone camera and lightweight networks suitable for running on mobile devices. Our method operates indirectly on linear raw pixels in bracketed exposures. Every pixel in the final HDR image is a convex combination of input pixels in the neighborhood, adjusted for exposure, and thus avoids
hallucination artifacts typical of recent deep image synthesis networks. We validate our system on both synthetic imagery and unseen real bracketed images – we confirm zero-shot generalization of the method to smartphone camera captures. Our iterative inference architecture is capable of processing an arbitrary number of bracketed input photos, and we show examples from capture stacks containing 3–9 images. Our training process relies only on synthetic captures yet generalizes to unseen real photos from several cameras. Moreover, we show that this training scheme improves other SOTA methods over their pretrained counterparts.
Read the full abstract ▾
All authors Baiang Li, Ruyu Yan, Ethan Tseng, Zhoutong Zhang, Adam Finkelstein, Jiawen Chen, Felix Heide
Computer Vision
DistortBench: Benchmarking Vision Language Models on Image Distortion Identification
Divyanshu Goyal, Akhil Eppa and Vanya Bannihatti Kumar
Vision-
language models (VLMs) are increasingly used in settings where sensitivity to low-level image degradations matters, including content moderation, image restoration, and quality monitoring. Yet their ability to recognize distortion type and severity remains poorly understood. We present DistortBench, a diagnostic
benchmark for no-reference distortion perception in VLMs. DistortBench contains 13,500 four-choice questions covering 27 distortion types, six perceptual categories, and five severity levels: 25 distortions inherit KADID-10k calibrations, while two added rotation distortions use monotonic angle-based levels. We evaluate 18 VLMs, including 17 open-weight models from five families and one proprietary model. Despite strong performance on high-level
vision-
language tasks, the best model reaches only 61.9% accuracy, just below the human majority-vote baseline of 65.7% (average individual: 60.2%), indicating that low-level perceptual understanding remains a major weakness of current VLMs. Our analysis further reveals weak and non-monotonic scaling with model size, performance drops in most base–thinking pairs, and distinct severity-response patterns across model families. We hope DistortBench will serve as a useful
benchmark for measuring and improving low-level visual perception in VLMs.
Read the full abstract ▾
Computer Vision
Camera Control for Text-to-Image Generation via Learning Viewpoint Tokens
Xinxuan Lu, Charless Fowlkes and Alexander C. Berg
Current text-to-image models struggle to provide precise camera control using natural
language alone. In this work, we present a framework for precise camera control with global scene understanding in text-to-image generation by learning parametric camera
tokens. We fine-tune image generation models for viewpoint-conditioned text-to-image generation on a curated
dataset that combines 3D-rendered images for
geometric supervision and photorealistic augmentations for appearance and background diversity. Qualitative and quantitative experiments demonstrate that our method achieves state-of-the-art accuracy while preserving image quality and
prompt fidelity. Unlike prior methods that overfit to object-specific appearance correlations, our viewpoint
tokens learn factorized
geometric representations that transfer to unseen object categories. Our work shows that text-
vision latent spaces can be endowed with explicit 3D camera structure, offering a pathway toward geometrically-aware
prompts for text-to-image generation. Project page: https://randdl.github.io/viewtoken_control/
Read the full abstract ▾