Computer Vision · Cross-listed from cs.LG
Remote Sensing and Machine Learning-Based Analysis of Land Use and Vegetation Change in Dhaka District, Bangladesh
Muhammad Masud Tarek, Md. Alamgir Hossain, Md. Samiul Islam et al. (+1 more)
Rapid urbanization in Dhaka District, Bangladesh has triggered substantial alterations in land use and environmental conditions, necessitating systematic monitoring for informed urban planning and ecological sustainability. This study employs remote sensing data and machine learning techniques to analyze spatiotemporal changes in land cover and vegetation dynamics between 2019 and 2024. High-resolution satellite imagery from Sentinel-2 MSI and Landsat 8 was utilized to classify land cover types and compute spectral indices including the Normalized Difference Vegetation Index (NDVI), Normalized Difference Built-up Index (NDBI), and Normalized Difference Water Index (NDWI). A supervised machine learning approach incorporating Decision Tree, K-Nearest Neighbors (KNN), and Random Forest classifiers was applied using labeled geospatial training points within Google Earth Engine. Accuracy assessments were conducted using confusion
matrices and kappa statistics. Results indicate a 59.5% increase in urban built-up areas and a significant decline in vegetation (-8.46%) and water bodies (-7.77%) over the five-year period. Land conversion from vegetated and aquatic areas to urban infrastructure was identified as a dominant trend. Among the models, Random Forest demonstrated the highest classification accuracy. These findings underscore the growing environmental pressures driven by unregulated urban expansion in Dhaka. The study highlights the potential of remote sensing and machine learning tools in providing timely, actionable data to support sustainable urban development, land-use regulation, and ecosystem conservation policies.
Read the full abstract ▾
All authors Muhammad Masud Tarek, Md. Alamgir Hossain, Md. Samiul Islam, Muntasir Hasan Kanchan
Computer Vision
A Remote Approach to Cashew Orchard Detection: Leveraging Active Learning with Satellite Imagery in Guinea-Bissau
Miguel Pereira, Sofia C. Pereira, Maria J.P. Vasconcelos et al. (+3 more)
Cashew production is a widespread economic activity in Guinea-Bissau, as well as other countries in West Africa. However, unregulated cashew production can be directly associated with increasing regionwide deforestation rates, biodiversity losses, and a fragile economic structure. There is no nationwide database for listing or georeferencing cashew orchards, so there is a clear need to remotely map their locations. In recent years, multiple methods for detecting orchards have been developed, though they have only been applied on a regional level. This work expands regional analyses to a nationwide scale. It develops a scalable and cost-effective remote approach, based on Sentinel-2 satellite imagery, using Machine Learning techniques to detect cashew orchards automatically. Margin-based Active Learning techniques were employed to develop an optimal training set in terms of the number of points and their informativeness, leading to a cashew map with 94.0% balanced accuracy obtained entirely off-site. We created two
datasets and a 2021 cashew map with 10m spatial resolution that are openly accessible through GitHub. The results demonstrate the possibility of a broader cashew orchard mapping, creating a new stepping stone for this environmental application.
Read the full abstract ▾
All authors Miguel Pereira, Sofia C. Pereira, Maria J.P. Vasconcelos, Patrícia Guedes, Luke L. Powell, João P. Pedroso
Computer Vision
Auditing Frame-Level AUC in Weakly Supervised Video Anomaly Detection: Granularity, Resolution, and Scene Bias
Sara Abdulaziz and Egor Bondarev
Frame-level area under the ROC curve (AUC) is the dominant evaluation metric for weakly supervised
video anomaly detection (WSVAD). Its standard form measures whether an anomalous frame outranks a normal frame drawn from anywhere in the test set. We refer to this comparison as pooled AUC, since it aggregates frame pairs across test
videos regardless of source. Pooled AUC therefore credits both event localization and differences between
video sources. We audit this protocol on UCF-Crime across recent state-of-the-art models spanning different backbone families. Holding each model's frame scores fixed, we read them under three pairing granularities: global, per anomaly category, and within each
video, then repeat the same three-granularity readout on zero-shot scores computed from the models' internal representations. We assess ranking reliability with a paired
video bootstrap. Three findings follow. First, pooled AUC does not reliably predict within-
video anomaly localization: models with similar pooled scores exhibit large localization differences and rank reversals under stricter granularities. Second, at the
benchmark's test-split size, pooled AUC lacks the resolution to support state-of-the-art margins reported in the field. Within each backbone family, it resolves no comparison at those margins, while within-
video AUC resolves several over identical predictions. Learned representations further reveal that within-
video anomaly structure and detector localization are decoupled. Third, on normal footage alone, every model we examine separates
videos by recording properties, such as resolution and color encoding, indicating that scene sensitivity is shared across the setting rather than specific to any architecture. We publicly release a granularity-aware protocol computable from existing predictions and scene-factor annotations for UCF-Crime.
Read the full abstract ▾
Computer Vision
Evaluating and Calibrating Diffusion Model-derived Uncertainty for Quantitative MRI Mapping
Shishuai Wang, Stefan Klein, Juan A. Hernandez-Tamames et al. (+1 more)
Quantitative MRI (qMRI) provides standardised tissue parameter maps, but the reliability of deep learning-based qMRI mapping methods is often not explicitly characterised. In this work we systematically evaluate uncertainty maps for quantitative MRI derived from multiple inferences of a data-consistent
diffusion model-based qMRI framework. Evaluation on synthetic test data assessed error-awareness, high-error detection, selective prediction, and Gaussian interval calibration.
Diffusion model-derived uncertainty was positively associated with the mapping error, while risk-coverage analysis showed that excluding high-uncertainty voxels reduced the retained error. However, the raw uncertainty was poorly calibrated for quantitative interval interpretation. Calibration was substantially improved using a post-hoc procedure combining prediction-value-dependent bias correction with scalar uncertainty scaling. Qualitative evaluation on a healthy volunteer showed spatially meaningful uncertainty patterns. These results indicate that
diffusion model-derived uncertainty is informative for reliability assessment and selective prediction, but requires calibration for quantitative interval interpretation.
Read the full abstract ▾
All authors Shishuai Wang, Stefan Klein, Juan A. Hernandez-Tamames, Dirk H.J. Poot
Computer Vision
Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction
Yan Di, Chengxi Li, Yaoxing Wang et al. (+7 more)
Sparse voxel reconstruction offers an efficient representation for high-fidelity 3D modeling, yet its geometry is commonly optimized from local photometric evidence and discrete visibility statistics. This often leads to fragmented surfaces, excessive subdivision, and floating artifacts, particularly in weakly textured or sparsely observed regions. We introduce SurfSVR, a novel sparse voxel reconstruction paradigm that treats 2D surface priors as explicit 3D
geometric regularizers. Instead of directly lifting noisy pixel-wise depth predictions, SurfSVR first organizes each image into coherent surface regions by jointly
reasoning over appearance,
monocular depth, normals and cross-view geometry. Each region is then represented by an adaptively selected planar or quadratic surface model based on fitting reliability and
geometric complexity, while cross-model agreement distinguishes reliable geometry from ambiguous predictions. These structured 2D priors are lifted into 3D and integrated throughout the reconstruction pipeline. They guide surface-adaptive voxel subdivision, provide region-level depth and normal supervision during optimization, enhance geometrically reliable sparse-observed surfaces in voxel
pruning, and suppress off-surface floaters during post-refinement training. This unified design converts semantic and
geometric coherence in image space into persistent structural constraints in 3D. Extensive experiments on 3 public
benchmarks demonstrate that SurfSVR consistently improves sparse voxel reconstruction across scenes with substantially different visibility and geometry characteristics, achieving state-of-the-art reconstruction quality. Codes and models will be released soon.
Read the full abstract ▾
All authors Yan Di, Chengxi Li, Yaoxing Wang, Mengge Liu, Zhigang Li, Ruida Zhang, Mingyang Li, Pengyuan Wang, Shan Gao, Xiangyang Ji
Computer Vision
Dual Anchors, Do It Better: Hierarchical Group Merging for Zero-Shot Anomaly Detection
Jimin Roh, DongKyu Kim and Suk-Ju Kang
Zero-shot anomaly detection (ZSAD) aims to identify anomalies in unseen domains, a setting that is particularly critical for industrial and medical applications where domain shifts are prevalent. However, most CLIP-based ZSAD methods anchor semantics solely on the text modality, making performance highly sensitive to
prompt design and leading to weak visual grounding. To mitigate these limitations, we propose a Dual-Anchor framework that complements conventional text anchors with hierarchical image anchors constructed via a top-down grouping mechanism. This mechanism progressively aggregates local-to-global image features to form normal and abnormal group
tokens, which serve as image anchors and act as gating signals in a Group-Gated
Token Refiner to enhance the global representation. The refined image anchors are then fused with text
prompts to construct dynamic state
prompts. By jointly reinforcing visual and textual semantics, our framework stabilizes image-text
alignment, reduces
prompt dependency, and achieves strong generalization across 8 industrial and 6 medical
benchmarks.
Read the full abstract ▾
Computer Vision
Seed2GS: Camera-Free, Training-Free Object Extraction from 3D Gaussian Scenes via a Single Reference-View Grounding
Zongjian Ding, Yudong Gao, Jiale Liu et al. (+7 more)
Extracting a target object from a pre-built 3D Gaussian Splatting (3DGS) scene enables interactive 3D editing. Existing methods either train for tens of minutes per scene, sacrifice accuracy, or require original reconstruction cameras that pre-built assets may not include. We present Seed2GS, which achieves the highest reported LERF-MASK accuracy without original reconstruction cameras or scene-specific representation training. Its key insight is to separate target identity from 3D coverage. QD-SAM3 selects one reliable reference mask from several open-vocabulary candidates, fixing identity once. Seed lift and visibility-adaptive virtual orbits then expose the object from new viewpoints, while tracking propagates the seed without repeated detection. Because the scene remains frozen, these masks supervise only one temporary foreground logit per Gaussian. On LERF-MASK, Seed2GS reaches 92.1% mean intersection over union (mIoU) with a measured compute-only latency of 9.3 seconds, 3.7 points above the strongest scene-trained baseline and 7.6 points above the closest camera-free baseline. With one fixed test reference per scene, the complete pipeline retains 91.1% mIoU; replacing its predicted seed with a ground-truth mask improves mIoU by only 0.72 points. On 3D-OVS, Seed2GS reaches 95.7% mIoU.
Read the full abstract ▾
All authors Zongjian Ding, Yudong Gao, Jiale Liu, Xinglin Yu, Junxing Ren, Dong Wei, Yajing Chen, Shan Huang, Mingjun Cheng, Min Li
Computer Vision
HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion
Wenshuo Peng and Kaipeng Zhang
Video-to-audio (V2A) generation faces significant challenges in achieving precise temporal synchronization and high perceptual quality due to the complex, ambiguous relationship between visual and auditory cues. Existing methods typically compress
video inputs into single feature representations, leading to significant loss of temporal dynamics and fine-grained visual information. These approaches also rely on reconstruction-based training objectives that poorly correlate with human perceptual judgments of audio quality and appropriateness. We propose HarmoniDPO, a novel framework that integrates preference-based optimization into
diffusion-based V2A generation to address these limitations. (1) Our approach leverages a dual
video representation: combining global context with frame-wise features to preserve temporal dynamics and semantic detail. (2) Inspired by reinforcement learning from human feedback (RLHF), HarmoniDPO employs online Direct Preference Optimization (online-DPO) to fine-tune a
diffusion-based V2A model from preference judgments, enhancing perceptual quality and
alignment. (3) Additionally, we introduce Dual-scale
Diffusion Search (DDS), a test time scaling
algorithm that adaptively optimizes output fidelity during inference. Experiments demonstrate that HarmoniDPO outperforms state-of-the-art methods in audio-
video synchronization and subjective audio quality, offering a robust solution for generating realistic, human-preferred audio from
video.
Read the full abstract ▾
Computer Vision
A Model-Internal Protocol for Assessing Multimodal Models as Integrated Systems
Hao Zhang, Jiaxin Qi, Zhijiang Tang et al. (+1 more)
As Large
Vision-
Language Models increasingly aim to integrate visual generation and understanding within a single parameter space, evaluating such structural unification in a cohesive manner remains a critical challenge. Current evaluation protocols predominantly treat generative and discriminative capabilities as separate tasks, leaving a gap in system-level evaluation for unified
multimodal models (UMMs). In this work, we propose Self-Generative-Understanding (SGU), a novel, annotation-free evaluation framework that probes the integrated capabilities of unified models through a semantic closed-loop challenge. Without requiring new annotations, SGU leverages the dual understanding-and-generation abilities of UMMs by asking them to first perceive an image and produce a textual description, subsequently reconstruct a visual context based on that description, and finally perform
reasoning over the self-generated output. This pipeline provides a zero-cost testbed that yields an integrated performance score specifically tailored for evaluating UMMs as unified systems. Extensive experiments show that even high-performing UMMs often struggle to reason over their own generated contexts, revealing limitations that are not captured by separate evaluations of understanding or generation alone. Our work provides a complementary holistic evaluation framework and offers a foundation for benchmarking the development of next-generation unified
multimodal models.
Read the full abstract ▾
All authors Hao Zhang, Jiaxin Qi, Zhijiang Tang, Jianqiang Huang
Computer Vision · Cross-listed from quant-ph
CoQui: A Coordinate-Conditioned Quantum Implicit Generative Adversarial Network for End-to-End Image Generation
Xue Yang, Rigui Zhou, ShiZheng Jia et al. (+7 more)
Quantum generative
adversarial networks (QGANs) have attracted increasing
attention for image generation using parameterized
quantum circuits. Existing amplitude-based approaches face two key limitations: pixel locations are typically encoded by computational-basis indices or address
qubits, causing
quantum resources to grow with image resolution; meanwhile, jointly decoding many pixels from normalized
quantum states introduces probability competition among pixels and limits precise pixel-wise control. To address these issues, we reformulate
quantum image generation as coordinate-conditioned implicit function learning. Our method takes spatial coordinates and latent variables as inputs, uses a classical
embedding network to generate input-dependent
circuit parameters, and evaluates a variational
quantum circuit at each coordinate. Pixel intensities are directly obtained from the expectation value of a dedicated color
qubit, and a complete image is generated by querying all spatial coordinates. This design decouples image resolution from address-
qubit requirements and avoids shared probability-normalization constraints across pixels. We further design a specialized variational
quantum circuit to provide structural inductive bias for coordinate-conditioned generation. Simulated experiments on two
benchmark datasets show that our method outperforms FRQI-based generation and PQWGAN in visual and quantitative quality while using fewer
qubits, and also achieves better generation quality than the corresponding classical baseline.
Read the full abstract ▾
All authors Xue Yang, Rigui Zhou, ShiZheng Jia, Dax Enshan Koh, Siong Thye Goh, Young-Wook Cho, YaoChong Li, Xuezhi Ma, Hongyu Chen, Xin Wang
Computer Vision
Warping Earth Observations for better ice labeling in the Marginal Marginal Ice Zone
Tom Kelly and Martin S. J. Rogers
Multimodal satellite imagery provides complementary information for Earth Observation, but accurately combining heterogeneous sensors remains challenging in dynamic environments. Fast-changing regions, such as the Antarctic marginal ice zone, cannot fully exploit
multimodal information from different satellite sensors because surface features move between image acquisitions. This spatial and temporal mismatch challenges effective perceptual grounding, violating the assumption of pixel-level correspondence that underpins most
multimodal reasoning and downstream classification pipelines. Antarctic sea ice provides a challenging
benchmark due to the rapid, heterogeneous drift of individual ice floes and the differing responses of sea ice to radar, visible and thermal sensing modalities. Accurate, dense supervision of sea ice remains scarce because generating pixel-wise labels requires time-consuming expert interpretation of noisy data, leading to historical reliance on coarse-resolution maritime ice charts for model training. This paper presents a novel architecture based on mutual information warping to align multi-satellite (Sentinel-1 and MODIS platforms)
multimodal (visible, thermal, radar) satellite scenes. To demonstrate the approach, we introduce a sparse expert-labeled
dataset of 2,088 pixel-wise annotations (7,046 expert point classifications) located at the ice-water margin interface across 43 scenes. Our results demonstrate that spatially grounding and aligning modalities prior to
segmentation improves classification accuracy, and enables accurate, dense sea ice
segmentation from sparse point-wise supervision.
Read the full abstract ▾
Computer Vision · Cross-listed from physics.optics
Two-Stage Deformable-Convolutional Inverse Design of Nanophotonic Absorbers from Optical Spectra
Waleed Waseer, Muhammad Shahid Jabbar, Muhammad Sohail Ibrahim et al. (+1 more)
Data-driven inverse design enables efficient generation of nanophotonic structures with prescribed optical responses, but spectrum-to-geometry mapping remains challenging due to non-uniqueness and fine
geometric features. This work presents a two-stage deformable-convolutional framework for reconstructing metal–
insulator–metal resonator geometries from 80-dimensional absorption spectra. The spectrum is projected to a $150\times4\times4$ latent representation and decoded into a $64\times64$ resonator mask. Training combines supervised reconstruction with least-squares
adversarial refinement initialized from the best supervised checkpoint. A three-run ablation compares deformable convolution with plain convolution, involution, Dynamic Conv, and ODConv under the same architecture. The proposed model achieves $20.79\pm0.31$~dB PSNR and $0.8501\pm0.0082$ SSIM, improving over plain convolution by 2.16~dB and 0.0831, respectively. It further achieves Dice $0.9623\pm0.0027$, IoU $0.9342\pm0.0038$, and boundary F-score $0.9550\pm0.0027$. Spectral consistency evaluated using a frozen forward surrogate yields RMSE $0.0805\pm0.0013$ and $R^2=0.7923\pm0.0065$. Learned offsets show stronger adaptive sampling at coarse and intermediate decoder stages. Overall, deformable sampling with supervised initialization and
adversarial refinement improves spectrum-conditioned geometry reconstruction.
Read the full abstract ▾
All authors Waleed Waseer, Muhammad Shahid Jabbar, Muhammad Sohail Ibrahim, Shujaat Khan
Computer Vision
LookBack: Where and How to Score LVLM Responses via Visual Reference Usage
Beomsik Cho, Jinhyeong Kim, Dongseok Lee et al. (+1 more)
Large
Vision-
Language Models (LVLMs) integrate visual perception with
language generation, enabling responses that span image understanding and complex
reasoning. However, LVLMs do not just inherit the text-level
hallucinations; they also hallucinate against the image, producing fluent responses ungrounded in what they see. This makes LVLM response scoring inherently harder, and our diagnostics show that existing confidence-based metrics adopted from LLMs are insufficient for LVLMs. Specifically, removing the input image barely changes confidence-based selection, suggesting that output-space confidence primarily captures textual plausibility rather than agreement with the image. To address this gap, we propose LookBack, a training-free LVLM response scoring method that augments
token likelihood with visual lookback score, a lightweight measure of how strongly each response
token refers to image
tokens. Across four
benchmarks and three models, LookBack consistently improves Best-of-$N$ selection over existing baselines with negligible additional overhead.
Read the full abstract ▾
All authors Beomsik Cho, Jinhyeong Kim, Dongseok Lee, Jaehyung Kim
Computer Vision
BoltNet: An Ultra-Lightweight Convolutional Network for On-Device Plant Species Identification
Daniel Rossi, Guido Borghi and Roberto Vezzani
Automated plant species identification from citizen-science imagery is an established, demanding fine-grained recognition problem: large taxonomic label spaces, visually similar species, and long-tailed observations require real model capacity, while field use constrains
memory, latency, and power. Model size is only part of the deployment cost: intermediate activations held in
memory during inference and platformdependent execution behavior matter too, so compact recognition must be assessed on target hardware rather than through complexity metrics alone. We present BoltNet, an ultra-lightweight fully convolutional architecture combining a Spatial Redistribution Bottleneck and Logit PreSampling to improve the tradeoff between predictive performance and model size in high-cardinality classification, and report the AccuracyCompression Tradeoff as a complementary diagnostic. On Pl@ntNet300K, BoltNet reaches 0.682 F1-score with 341K parameters (1.37 MB), the highest F1-score among evaluated models below 2 MB and close to substantially larger convolutional backbones. Model-only measurements on a Raspberry Pi 5, Jetson Orin Nano, and Hailo-8 characterize execution across CPU, GPU, and NPU platforms, where BoltNet is the most consistently efficient model, with the best FPS/W on the GPU and NPU and second-best on the CPU. Results on AIDERv2 and CLRS provide secondary evidence of transfer across environmental image-classification tasks. Code available at: https://codeberg.org/danielrossi/BoltNet
Read the full abstract ▾
Computer Vision
GeoBridge: Decoupled Semantic Conditioning for Generative Image Geolocalization
Zhiyang Dou, Xumeng Han, Fengde Peng et al. (+4 more)
Multimodal large
language models (MLLMs) have advanced image geolocalization mainly by improving how they reason about geographic cues. How that
reasoning isdecoded into coordinates, however, has lagged behind. Predicting a place name for a geocoding API is discrete and lossy: it ignores image evidence and collapses multi-granular semantics into a coarse lookup. We argue that the bottleneck has shifted from what a model reasons to how that
reasoning is represented for a continuous, geometry-aware decoder. We present GeoBridge, a role-decoupled conditioning mechanism that connects a frozen semantic MLLM to a frozen Riemannian flow-matching head that generates coordinates on the sphere. The central obstacle is arole conflict: supervising the condition with discrete semantic labels biases its representation toward class-discriminative geometry, at odds with the smooth
manifold the generative head requires. GeoBridge keeps the semantic supervision decoupled from the condition interface: a separate projection forms the continuous condition the frozen head expects, injecting geographic priors without disturbing the spherical decoder. On IM2GPS3K, GeoBridge reaches 38.67/52.89/70.37 at the 25/200/750 km thresholds, improving over a place-name-to-API pipeline and
reasoning-augmented direct prediction at these precision-relevant scales. GeoBridge is a decode-side algorithmic contribution, orthogonal and complementary to chain-of-thought
reasoning. Code will be made publicly available.
Read the full abstract ▾
All authors Zhiyang Dou, Xumeng Han, Fengde Peng, Zipeng Wang, Moxuan Zhao, Zhipei Huang, Zhenjun Han
Computer Vision
Andreas Robinson, Abdelrahman Eldesokey and Michael Felsberg
Semi-supervised
video object
segmentation is a challenging task that aims to segment a target throughout a
video sequence given an initial mask at the first frame. Discriminative approaches have demonstrated competitive performance on this task at a sensible complexity. These approaches typically formulate the problem as a one-versus-one classification between the target and the background. However, in reality, a
video sequence usually encompasses a target, background, and possibly other distracting objects. Those objects increase the risk of introducing false positives, especially if they share visual similarities with the target. Therefore, it is more effective to separate distractors from the background, and handle them independently. We propose a one-versus-many scheme to address this situation by separating distractors into their own class. This separation allows imposing special
attention to challenging regions that are most likely to degrade the performance. We demonstrate the prominence of this formulation by modifying the learning-what-to-learn (LWL) method to be distractor-aware. Our proposed approach sets a new state-of-the-art on the DAVIS 2017 val
dataset, and improves over the baseline on the DAVIS 2017 test-dev
benchmark by 4.6 percentage points.
Read the full abstract ▾
Computer Vision
TD-VAD: Breaking Visual Dependence in Video Anomaly Detection with Text-Driven Learning
Shuangqing Zhang, Lei-Lei Ma, Zhao Wang et al. (+5 more)
Visual data is typically a prerequisite for training existing
video anomaly detection (VAD) methods. However, obtaining sufficient annotated anomaly data for training is challenging and not scalable due to the rarity of anomaly data and the wide variety of abnormal events. In this work, we advocate that the effectiveness of treating texts as
video sequences for the VAD model and propose a novel Text-Driven
Video Anomaly Detection (TD-VAD) approach to break visual dependence. In contrast to the anomaly
video data, text descriptions of abnormal events are easy to collect, and their class labels can be directly derived. Specifically, our method utilizes
video-like text descriptions with temporal characteristics generated by LLM to train a VAD model, without any reliance on target-domain anomaly data. To capture the long- and short-range temporal logic of events, we design the event
evolution causal
attention module to model contextual dependencies across time. During inference, considering the domain gap between the texts and
video sequences, we use the frozen CLIP encoder to extract
embeddings of
video frames to align the text modality while retaining crucial visual information. Comprehensive experiments on two large-scale VAD
datasets, XD-Violence and UCF-Crime, demonstrate that our method outperforms prior one-class and unsupervised VAD methods by a large margin.
Read the full abstract ▾
All authors Shuangqing Zhang, Lei-Lei Ma, Zhao Wang, Wen Dong, Xinyi Xu, Guo-Sen Xie, Caifeng Shan, Fang Zhao
Computer Vision · Cross-listed from cs.LG
Learning with Bilevel-Minimax Optimization for Efficient and Reliable Transfer Attacks
Yaohua Liu, Yifan Guo and Jiaxin Gao
Transfer-based
adversarial attacks craft
adversarial examples using surrogate models to mislead black-box victim models. Beyond perturbation generation, transferability is fundamentally governed by the coupling of initialization, surrogate adaptation, and gradient dynamics. We revisit this challenge from a bilevel-minimax perspective and propose BMAT (Bilevel-Minimax
Adversarial Transfer). The bilevel formulation captures the dependency between initialization and perturbation, while the inner minimax problem promotes surrogate robustness for cross-architecture generalization. Algorithmically, we develop an integrated bottom-up solver that combines a Soft Weight Modulator and an Implicit Gradient Approximator to enable ternary coupling among initialization, surrogate adaptation, and perturbation optimization. We further provide theoretical insights into the optimization dynamics of the proposed bilevel-minimax framework. Extensive experiments on classification and
segmentation benchmarks show that BMAT outperforms more than 10 strong baselines across more than 30 victim models, improving both intra- and cross-architecture transfer and yielding up to a 2x reduction in mIoU. Code is available at https://github.com/callous-youth/BMAT.
Read the full abstract ▾
Computer Vision
Can Vision Models Read the Radar Display? On the Feasibility of Radar Imagery for Air Traffic Complexity Estimation
Hyewook Kim, Byul Kang, Seokbin Yoon et al. (+1 more)
Air traffic controllers perceive traffic complexity through the radar display, suggesting that a computer
vision model operating on the same imagery may provide a natural architecture for modeling controller-perceived complexity; however, whether radar imagery is a viable input format for deep learning
vision models remains unclear. Unlike natural images, radar images are extremely sparse and self-similar, consisting primarily of a black background and a few visually identical aircraft blobs, while small changes in aircraft positions can substantially alter sector-level complexity. To test whether a
vision model can capture these operationally important differences, we encode each traffic situation as a position image supplemented by five channels representing aircraft state variables, including heading, speed, and altitude, and train a
Vision Transformer (ViT) to regress four intrinsic complexity components derived from pairwise
geometric relations among aircraft. The model achieves $R^2 > 0.96$ for all four components, and a one-aircraft-removal perturbation study shows that its response changes proportionally to how much the removed aircraft contributed to sector complexity rather than treating every removal as equivalent. These results demonstrate that, despite its atypical visual characteristics, radar imagery is a viable input format for air traffic complexity modeling.
Read the full abstract ▾
All authors Hyewook Kim, Byul Kang, Seokbin Yoon, Keumjin Lee
Computer Vision
CoDiR: Confidence-Guided Diffusion Refinement for Semi-Supervised Histopathology Segmentation
Hoai Nhan Pham, Dang-Nguyen Bui, Le-Van Thai et al. (+7 more)
Semi-supervised histopathology
segmentation is challenging due to scarce annotations and unreliable pseudo-labels in ambiguous gland regions. To address this problem, we propose Confidence-Guided
Diffusion Refinement (CoDiR), a semi-supervised framework that combines a Mean Teacher
segmentation model with
diffusion-based pseudo-label refinement. Given an unlabeled image, the teacher first produces a soft prediction, and only low-confidence regions are refined by a conditional
diffusion model trained to capture plausible mask structures from labeled data. The refined mask is then fused with reliable teacher predictions and used to train the student with confidence weighting and consistency regularization. On the GlaS and CRAG
datasets CoDiR reaches 88.09% and 89.83% mDice with 10% labeled data, and 89.19% and 90.29% mDice with 20%, matching or exceeding the strongest published method on seven of the eight
benchmark metrics. Ablations attribute the largest single contribution to the refinement module, which adds +6.36% mDice over the Mean Teacher baseline. The implementation code is publicly available at: https://github.com/vongla345/codir
Read the full abstract ▾
All authors Hoai Nhan Pham, Dang-Nguyen Bui, Le-Van Thai, Thanh-Hiep Vo, Lan Anh Dinh Thi, Tien Dat Nguyen, Duy-Dong Nguyen, Ngoc Lam Quang Bui, Tam Tran, Zhi Huang
Computer Vision
PolarSym: Polar Geometry-aware Attention for CAD Floorplan Parsing
Kerui Chen, Yiqing Wang, Kangzhou Xin et al. (+2 more)
CAD plan parsing is a fundamental task in Building Information Modeling (BIM), aiming to automatically extract architectural elements including walls, doors, windows, and furniture from 2D engineering drawings. Existing
Transformer-based methods capture global semantic dependencies via self-
attention, yet they infer spatial relationships merely from semantic features without explicitly characterizing the intrinsic
geometric symmetry of building layouts. Such methods tend to produce mismatched correspondences in long-range matching and complex symmetric spatial layouts. To tackle this limitation, we propose PolarSym, a polar-coordinate geometry-aware
attention framework for CAD plan parsing. The framework decouples
geometric relationships of buildings into two complementary components, direction and distance, which are modeled independently. Structural consistency is strengthened by directional constraints, while long-range symmetric correspondences are built with distance constraints. A dynamic gating mechanism is adopted to synergistically fuse the two
geometric information branches while maintaining the vanilla
Transformer architecture. This design boosts
geometric modeling capacity with negligible extra computation. Experiments on a public CAD plan parsing
dataset show that PolarSym surpasses the reproduced SymPoint V2 baseline by 1.73% PQ, 1.54% RQ and 4.31% mIoU under identical training settings. PolarSym also converges faster and yields more stable optimization. Ablation experiments verify the complementary effects of direction and distance modeling. Our results reveal that PolarSym improves the
geometric awareness of
Transformers at low computational cost, offering an effective
geometric modeling paradigm for CAD plan parsing.
Read the full abstract ▾
All authors Kerui Chen, Yiqing Wang, Kangzhou Xin, Qinghan Zhang, Songyang Ding
Computer Vision
Anti-Shortcut Distillation via Temporal Negative Knowledge Transfer
Syed Muhammad Raza, Omer Tariq and Jeongbae Son
Knowledge
distillation (KD) trains a compact student by attracting it towards a converged teacher. It is silent about which directions the teacher itself learned to suppress: repulsive and bias-aware objectives exist, but none exploits the teacher's own trajectory to identify what the student should avoid. We observe that the missing signal is already encoded in the teacher's optimization trajectory: features that an early-stage teacher emphasizes but that a converged teacher attenuates are precisely the shortcut directions worth pushing the student away from. We instantiate this observation as Anti-Shortcut
Distillation (ASD), a push–pull KD framework that treats the converged teacher $\Tfinal$ as a positive semantic anchor and an early-checkpoint teacher $\Tearly$ as a temporal negative reference. ASD couples two losses: a temporal contrastive loss ($\Ltc$) that places the early-teacher feature as a same-sample negative against in-batch and
memory-bank final-teacher features in an InfoNCE objective; and a shortcut suppression loss ($\Lss$) that penalizes student projection onto the top eigenvectors of $\E[\Dh\Dh^{\top}]$, the uncentered second-moment
matrix of early-to-final feature displacements. Across 13 teacher–student pairs on CIFAR-100, ImageNet-100, and TinyImageNet, ASD attains the highest clean top-1 accuracy on more than 10 pairs and outperforms standard KD on 12. On CIFAR-100-C corruption robustness, ASD obtains the lowest mean Corruption Error ($86.1$\,mCE) on the most challenging cross-architecture pair (WRN-40-2$\to$ShuffleNet-V2). Mechanistic diagnostics confirm the intended geometry: the ASD student is systematically anti-aligned with the shortcut direction, while its projection onto the robust subspace is substantially larger ($0.45$ vs.\ $0.12$).
Read the full abstract ▾
Computer Vision
VOLA: Improving Open-World Driving by VLM-Based Semantic Attribute Prediction
Yuchen Zhang, Yuan Gao, Sebastian Schmidt et al. (+1 more)
Driving in the real world is open-world: a car may encounter a fallen mattress, a deer, or other objects outside its training data. Naming them is not enough. The system must know how to treat each region: can it drive over it, and how severe would a collision be? We therefore shift scene perception from category labels to dense action-relevant attributes, where each pixel is labeled by how it should affect motion rather than by object name. We instantiate this general formulation with two ordered attributes: 7-rank drivability and 5-rank vulnerability. We read Qwen3.5 image-
token hidden states directly as a spatial semantic representation. A lightweight boundary-aware decoder then turns this coarse
token grid into sharp full-resolution attribute maps. The whole process requires neither autoregressive text generation nor an external mask model such as SAM. We train on dense attribute labels built in CARLA and test transfer to real scenes and to novel obstacles never seen in training. We compare with
vision-only segmenters trained on the same attributes and prompted VLM segmenters. Our model matches strong
vision-only segmenters on familiar categories and improves transfer to real open-world anomalies, reaching 69.4% mean vulnerability-rank recall versus 57.1% for the best
vision-only baseline and 53.9% for the best prompted VLM baseline. These results show that VLM image
tokens provide useful semantic cues for transferring driving attributes to objects outside the training vocabulary.
Read the full abstract ▾
All authors Yuchen Zhang, Yuan Gao, Sebastian Schmidt, Johannes Betz
Computer Vision
Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines
Vaishnav Raju
Edge-deployed
vision systems in target recognition, surveillance, autonomous vehicles, and drone domains require hierarchical inference pipelines where a detection model identifies objects of interest and downstream classifiers provide fine-grained attribute analysis. Running all models on the GPU creates a serial bottleneck that limits real-time throughput as pipeline stages grow. Modern edge SoCs pair GPUs with dedicated neural accelerators (NPUs, DLAs) capable of concurrent execution, yet deploying custom models on these accelerators remains impractical due to strict operator constraints, quantization incompatibilities, and an undocumented end-to-end pipeline. We target NVIDIA Jetson DLA cores as the representative platform. We present a five-step methodology for zero GPU fallback DLA INT8 deployment of classification backbones, comprising architecture adaptation, manual dynamic range workaround to rescue TensorRT's implicit quantization (recovering 94.0% accuracy from implicit quantization's 75%) for rapid pipeline validation before explicit quantization, quantization-aware training, ONNX graph surgery for DLA compilation, and a concurrent GPU-detection/DLA-classification inference pipeline. We document nine engineering constraints with root-cause analysis and generalizable solutions. Validation on a dual-head person attribute classifier running on DLA alongside a GPU object detector on a Jetson Orin NX demonstrates near-zero pipeline overhead (12.5 vs. 13.3~FPS detector-only at 1080p), with dual-DLA scaling at no additional cost. The methodology is backbone-agnostic and generalizes to any detection-classification edge pipeline.
Read the full abstract ▾
Computer Vision
ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation
Duy-Dong Nguyen, Le-Van Thai, Hoai Nhan Pham et al. (+3 more)
Weakly supervised semantic
segmentation enables histopathology tissue
segmentation from image-level annotations, avoiding costly pixel-level labeling by expert pathologists. However, CAM-based methods often localize only highly discriminative regions and remain unreliable near tissue interfaces. We propose ProBAG, a stage-1 pseudo-mask generator that combines
dataset-specific visual prototypes with pathology-aligned CONCH text prototypes over multi-scale frozen UNI features. ProBAG introduces two complementary mechanisms: class-wise power recalibration that reshapes inter-class competition while preserving the total foreground activation mass at each pixel, and one-step graph
diffusion in which feature affinities are penalized by a late-
transformer attention-context discrepancy used as a soft structural boundary cue. The resulting stage-1 pseudo-masks require neither CRF nor an external
segmentation model; for complete two-stage comparison, they additionally supervise a downstream Phikon-FPN segmenter. Experiments on BCSS-WSSS and LUAD-HistoSeg show consistent gains over recent WSSS approaches, while ablations indicate that pathology-aligned text semantics provide the largest improvement and graph refinement provides a smaller complementary gain. The code is available at: https://github.com/wterrr/WSSS
Read the full abstract ▾
All authors Duy-Dong Nguyen, Le-Van Thai, Hoai Nhan Pham, Ngoc Lam Quang Bui, Tam Tran, Zhi Huang
Computer Vision · Cross-listed from eess.IV
A comparison of CNN architectures for Alzheimer's disease detection in single-view MRI scans
Hiram Zuniga, Ulises Orozco-Rosas and Kenia Picos
Alzheimer's disease is a leading cause of death with no cure. Therefore, early detection is critical to slow progression and preserve quality of life. Diagnosis relies on medical history, cognitive tests, physical exams, and MRI
brain scans, making deep learning suitable for Alzheimer's classification. This work proposes a
benchmark that evaluates ten different convolutional neural network (CNN) architectures (including ResNet, DenseNet, MobileNet, EfficientNet, and VGG family models) under the same held-out test split protocol. A two-stage transfer learning and full fine-tuning pipeline is introduced to perform training using a class-balanced subset (3,900 images) derived from the OASIS medical imaging
dataset, comprising 86,437 single-view MRI
brain scans labeled into four classifications of Alzheimer's disease: Non-Demented, Very Mild Dementia, Mild Dementia, and Moderate Dementia. The best results were achieved by VGG16, with a 0.9637 validation accuracy and a 0.9533 test accuracy score. A key finding documented in this work is the difficulty of classifying the transition from Non-Demented to Very Mild Demented stages, observed consistently across all ten architectures.
Read the full abstract ▾
Computer Vision
Automated binary classification of hazelnut X-ray images: A deep-learning benchmark for quality assessment
Giancarlo Sportelli, Nicola Belcari, Roberta Pace et al. (+4 more)
Non-destructive X-ray imaging can reveal internal hazelnut defects that are difficult to detect by external inspection alone; however, automated interpretation remains challenging because of subtle radiographic differences among classes, marked class imbalance, and limited annotated data. Here, we present a
benchmark for binary hazelnut quality classification (healthy versus defective) based on 799 segmented single-kernel X-ray images (224 x 224 pixels, grayscale), grouped into 101 acquisition units. Seven single-model configurations and ten probability-aggregation ensembles were evaluated using a group-wise split-rotation protocol across five data splits generated using different random seeds. Decision thresholds were selected on the validation set, and performance was assessed deterministically on validation and test sets. Under the expert-reassessed annotation condition, the average-probability ensemble of the binary cross-
entropy-trained convolutional neural network and frozen Swin
Transformer achieved the highest mean balanced accuracy (86.3% +/- 1.8%, five seeds), with several other ensembles providing comparable performance. Across methods, substantial split-to-split variability was observed, indicating that multi-split evaluation is essential for reliable model comparison at this
dataset scale. Expert reassessment of ambiguous samples improved the performance of all 17 evaluated methods by 2.8-8.1 percentage points, while having only a limited effect on cross-split variance. The results highlight both the potential of deep learning for automated X-ray-based hazelnut quality assessment and the importance of rigorous evaluation and label curation in small, imbalanced agricultural imaging
datasets.
Read the full abstract ▾
All authors Giancarlo Sportelli, Nicola Belcari, Roberta Pace, Umberto Bernardo, Sharmin Sultana, Alessandra Toncelli, Matteo Giaccone
Computer Vision
UniSwap: Streaming Audio-Visual Identity Swapping for Talking Videos
Yuxuan Zhang, Haozhong Xiong, Jiayi Song et al. (+5 more)
Talking-
video character replacement requires coordinated transfer of appearance and voice while preserving the source motion, scene, linguistic content, and audio-
video timing. Existing methods use separately optimized models for the two modalities, making audio-visual consistency difficult to enforce. We present UniSwap, the first framework for streaming joint audio-visual identity replacement in talking
videos. Given a source
video, a reference image, and a reference voice clip, UniSwap transfers the reference appearance and vocal timbre within a single audio-visual
diffusion transformer while preserving the source content and dynamics. To address the scarcity of aligned cross-identity training pairs, we introduce a swap-and-reconstruct pipeline that removes visual and vocal identity from real clips and uses the original clips as reconstruction targets. Starting from a bidirectional backbone, we progressively adapt the model through In-context Pretraining for joint replacement, Conditional Streaming Adaptation for block-causal KV-cached generation, and Efficient Self-forcing DMD for mitigating exposure bias and reducing sampling from 30 to 3 denoising steps per block. Efficient Multi-LoRA Switching enables the three DMD roles to share a single frozen backbone. Feature-RoPE Decomposition keeps cached positions within the training range, supporting stable long-form inference. Experiments demonstrate strong audio-visual synchronization, competitive identity preservation, efficient streaming, and stable long-form generation.
Read the full abstract ▾
All authors Yuxuan Zhang, Haozhong Xiong, Jiayi Song, Jinpeng Yu, Yang Shi, Jiaming Liu, Ruihua Huang, Liwei Wang
Computer Vision
Dual Modality Prompted Diffusion Priors for Zero Shot Hyperspectral Pansharpening
Pengwei Xie, Fei Zhu, Jiajun Li et al. (+4 more)
Hyperspectral pansharpening aims to reconstruct a high resolution hyperspectral (HRHS) image from a panchromatic (PAN) image and a low resolution hyperspectral (LRHS) image while preserving both spatial details and spectral fidelity. Recent
diffusion based methods exploit pretrained image priors by generating a low dimensional representation and subsequently mapping it to the hyperspectral domain. However, the observed panchromatic and hyperspectral images are typically imposed only through external reconstruction objectives, limiting their direct interaction with the
diffusion prior. To address this issue, we propose dual-modality image-prompted
diffusion model (DIDM) for zero shot hyperspectral pansharpening. DIDM encodes the low resolution hyperspectral and panchromatic observations into spectral and spatial
prompt tokens, respectively, and injects them into intermediate features of a frozen remote sensing
diffusion model through cross
attention, allowing complementary spectral and spatial information to directly guide
diffusion feature
evolution. In addition, we introduce a panchromatic guided weighted pixel aware total variation regularizer that combines low resolution hyperspectral degradation fidelity and panchromatic response fidelity with gradient adaptive structural regularization, thereby preserving structural discontinuities while suppressing spurious variations in homogeneous regions. Extensive experiments on Pavia, Chikusei, and Houston under reduced resolution protocols show that DIDM achieves the best performance across all evaluated metrics, while full resolution evaluation on FR1 yields the highest HQNR among the compared methods. These results demonstrate that internal dual modality prompting and panchromatic guided structural regularization provide an effective balance between spatial detail enhancement and spectral preservation.
Read the full abstract ▾
All authors Pengwei Xie, Fei Zhu, Jiajun Li, Xiangyuan Liu, Xiangyuan Liu, Kangqing Shen, Gemine Vivone
Computer Vision
Making Every Step Count: Spatio-Temporal Information Allocation for Imaging Inverse Problems
Yi Cao, Xiangyong Cao, Pei Liu et al. (+2 more)
Flow-based generative models have emerged as powerful image priors for training-free inverse problem solving, capturing coherent semantics and fine-grained structure. Despite these strengths, existing flow-based inverse solvers primarily focus on the design of individual updates, largely overlooking spatio-temporal information allocation under a fixed number of function evaluations (NFEs). Temporally, insufficient early exploration can trap the flow trajectory in an incorrect semantic basin, whereas excessive allocation of NFEs to early stages leaves little budget for late-stage refinement. Spatially, data consistency provides direct constraints only within observed regions, whereas the recovery of missing regions relies mainly on the generative prior. To address these two issues, we introduce two complementary and training-free components, i.e., Spectrum-Adaptive Scheduling (SAS) and Measurement-Prioritized
Attention (MPA). For temporal allocation, SAS distributes the available NFEs over flow time according to the degradation spectrum and logSNR geometry, thus better balancing semantic exploration and detail refinement. For spatial propagation, MPA exploits data-prior conflicts to guide information toward weakly constrained regions, thereby enhancing semantic and structural fidelity. Extensive experiments on standard image inverse problems, e.g., super-resolution, motion deblurring, and inpainting, demonstrate that the proposed components can be integrated into existing flow-based inverse solvers in a plug-and-play manner without retraining or additional flow-model evaluations, and can also significantly improve the restoration quality of existing solvers.
Read the full abstract ▾
All authors Yi Cao, Xiangyong Cao, Pei Liu, Yong-Jin Liu, Deyu Meng