Computer Vision
Do CNNs Internally Represent Real and Fake Images Differently? A Hidden-Layer Analysis
Moumita Sen Sarma, Pascal Hitzler and Eugene Y. Vasserman
Fake/synthetic images are increasingly prevalent, but it remains unclear whether Convolutional Neural Networks (CNNs) process real and fake images in the same internal manner. This work examines the hypothesis that CNNs represent real and fake images differently, such that fake images induce different hidden-layer activation patterns even when semantic content is preserved. The hypothesis is evaluated in scene recognition settings using trained CNN models. Dense-layer activations are extracted, and neurosymbolic methods assign semantic labels to selected
neurons. For each real test image, corresponding fake images are generated with similar semantic content using object-label-guided text-to-image and image-to-image generation based on Stable
Diffusion variants. Paired real-fake activation patterns are then compared statistically. Additional experiments with another
dataset, CNN architecture, generative model, and JPEG/blur degradation analysis assess robustness. Results suggest that fake images evoke different hidden-
neuron activations, and these differences are not explained only by simple image degradation. Overall, the findings indicate that real and fake images differ in CNN hidden-layer activation behavior at least in some settings, which opens the door for follow-up work on making use of this different behavior to improve fake image detection.
Read the full abstract ▾
Computer Vision
Low Cost Two-Stage Fabric Defect Detection at the Edge
Rasel Hossen, Diptajoy Mistry and Mosaddek Hossain Kamal
Fabric inspection in the garment industries of low-income economies remains largely manual, and commercial
vision systems are priced beyond most small and medium mills. Because defects are sparse under controlled production, a natural response is a cascade: screen every frame with a cheap anomaly detector and invoke a full detector only on suspicious frames. We build such a cascade for four knit-fabric defect classes and deploy it end-to-end on an NVIDIA Jetson Nano with TensorRT FP16. Stage 1 is a compact convolutional autoencoder with decoder
attention gates, an edge-weighted reconstruction loss, and feature-level
distillation from a frozen YOLOv5n teacher; Stage 2 is YOLOv5n, invoked only on flagged frames. On a 249-image
benchmark disjoint from detector training (20 defective, 229 non-defective), Stage 1 at a recall-prioritised threshold flags all 20 defective images (95% CI 0.83-1.00) at a false-positive rate of 49.3% (113/229), reducing false positives by 19.3% relative to a plain autoencoder (p=0.011). The parallel pipeline reaches 13.45 FPS against 9.86 FPS for a sequential YOLO-only loop. Our central finding comes from decomposing that 1.36x: 91% of it is attributable to overlapping JPEG decode with inference rather than to the cascade, which contributes only a 5.1% inference reduction at the measured forwarding rate p = 0.534. We further show that forwarding here is false-positive-limited rather than prevalence-limited - 85% of forwarded frames are false alarms - and quantify the 29-45% inference reduction attainable under tighter calibration. We report this as a caution for cascade speedups measured without controlling the data path, and position the system as AI-assisted triage rather than autonomous acceptance.
Read the full abstract ▾
Computer Vision
Spatial Attention Noise Masking for Causally Sufficient Interpretability
Benjamin Formby, Kuang-Ching Wang and D Hudson Smith
We present a novel causal approach to interpretability for computer
vision models that dynamically masks the input image prior to classification. The interpretability of deep learning predictions is critical in high-stakes fields such as medical imaging, security, and autonomous driving. Most interpretability methods are applied passively to already trained models, which typically result in correlational rather than causal explanations. Existing causal interpretability methods are limited to post hoc analysis, weakening the causal claims. Additionally, existing active methods generally lack explanations that explicitly assign responsibility to input features. This work proposes a spatial
attention noise masking framework that provides causal explanations about the features sufficient for the prediction. The proposed framework consists of: 1) a UNet-style mask generator, and 2) a Resnet18 encoder and linear classifier that classifies both masked and unmasked versions of an input image. The generated masks are regularized to be sparse and spatially smooth, while masked image
embeddings are constrained to remain consistent with
embeddings from the corresponding unmasked images. The resulting masks can be interpreted as feature attribution maps that are competitive with related interpretability methods while additionally providing strong causal explanations of model predictions. Quantitative evaluations demonstrate mask faithfulness, near-baseline classification performance across five classification tasks despite substantial masking of image information, and robustness to distribution shifts such as background swapping and natural
adversarial examples. Qualitative comparisons further demonstrate mask behavior and competitive interpretability relative to state-of-the-art feature attribution methods.
Read the full abstract ▾
Computer Vision
Privacy-Preserving Dataset Curation for Kuala Lumpur Urban Traffic: Grounded Vision-Language Detection with Spatial Vehicle-Context Filtering
Mohammed Abdul Al Arafat Tanzin and Rudzidatul Akmam Dziyauddin
The rapid advancement of intelligent transportation systems and autonomous driving relies heavily on multi-modal urban traffic
datasets. However, curating high-fidelity
video imagery in complex tropical urban environments—specifically Kuala Lumpur, Malaysia—presents severe challenges for Personally Identifiable Information (PII) anonymization due to high motorcycle density, dark acrylic license plates, dynamic camera tilt, and extreme tropical glare. We propose an automated anonymization framework tailored for the Kuala Lumpur Road
Dataset, captured via a mobile cycling platform at 2 FPS. We document how legacy Haar cascades and YOLOv8 fail under these conditions—generating false positives on background elements while missing rotated or occluded targets. Our architecture resolves this by integrating Grounding DINO—a zero-shot open-set
vision-
language transformer—with a novel Spatial Vehicle Region of Interest (ROI) Containment Engine. By requiring license plate centroids to reside within validated vehicle boundaries, the pipeline suppresses environmental false positives while automatically obfuscating faces, heads, and license plates. An initial evaluation on 1,266 frames demonstrates a $\sim$95% success rate, with remaining failures restricted to small, heavily occluded, oblique, or ambiguous targets. Coupled with temporal persistence mechanisms and an automated quality-control auditor, the framework minimizes
privacy-related false negatives while preserving scene context for downstream
vision tasks. While formal legal compliance depends on broader governance procedures, this publicly available pipeline and demonstration notebook provide an auditable preprocessing stage for
privacy-aware
dataset curation.
Read the full abstract ▾
Computer Vision
A Vision Transformer for ECG-Based Detection of Left Ventricular Systolic Dysfunction Across Multiple Clinical Sites
Burcu Ozek, Aruna Mohan, David Vorchheimer et al. (+5 more)
Reduced left ventricular ejection fraction (LVEF) is frequently asymptomatic and often detected only after advanced heart failure develops. Electrocardiograms are recorded routinely yet underused for this condition, because reduced LVEF has no single diagnostic waveform. We trained an ensemble of
vision transformers from scratch to detect reduced LVEF ($\leq$40%) from 12-lead ECGs, analyzing each heartbeat individually, using 10,142 patients across seven sites in three US health systems. In a held-out external cohort of 4,092 patients from three geographically independent US clinical sites at a real-world reduced-LVEF prevalence of 8.72%, the model achieved an AUROC of 0.88 (95% CI 0.86-0.89), sensitivity 81.2%, specificity 81.0%, and negative predictive value 97.8%. Sensitivity remained high across sex, race, ethnicity, and comorbidity subgroups, while specificity was lower in older patients and those with atrial fibrillation or cardiomyopathy. Beat-level
attention maps provided interpretability into the model's predictions, showing consistent focus on the QRS complex rather than the P wave. These findings support the potential of routine ECGs as a scalable first-pass triage step to identify patients who should undergo echocardiography for reduced ejection fraction across diverse patient populations.
Read the full abstract ▾
All authors Burcu Ozek, Aruna Mohan, David Vorchheimer, Daniel Weiss, Eyal Kedar, Tamar Sobol, Or Zilbershot, Fatemeh Afghah
Computer Vision
Braided Vision Transformer for Stroke Detection in Multi-view Retinal Fundus Imaging
Aysen Degerli and Mika Hilvo
Stroke remains a leading cause of mortality and morbidity worldwide, emphasizing the importance of its accurate and immediate assessment. Retinal fundus imaging has emerged as a promising modality for stroke assessment, as the retina reflects cerebrovascular and neurological risk factors. Contrary to conventional neuroimaging techniques, retinal fundus imaging offers a non-invasive, cost-effective, and portable alternative for rapid screening. This paper explores the feasibility of retinal fundus imaging for stroke and transient ischemic attack (TIA) detection using macula-centric and optic nerve head-centric views captured from both eyes. Our study introduces, to the best of our knowledge, the first
vision transformer model for retinal fundus imaging in stroke assessment, offering a novel approach for capturing retinal patterns. Thereby, we propose the Braided
Vision Transformer (BViT) model, which extracts representative features from the given multi-view images while simultaneously capturing inter-view relationships across both eyes, enabling a more informative understanding of retinal biomarkers associated with cerebrovascular events. Experiments conducted on our collected Stroke-Data
dataset demonstrate that BViT achieves an AUC score of 0.75 for stroke detection, outperforming regular
vision transformers.
Read the full abstract ▾
Computer Vision
AeroGround: A Comprehensive Benchmark for Aerial-Ground Collaborative Reasoning
Shenghong Yi, Lin Zhang, Muzian Li et al. (+6 more)
Vision-
language models (VLMs) have been widely employed in understanding and
reasoning tasks for unmanned aerial vehicles (
UAVs). Existing
UAV benchmarks primarily focus on aerial-view scenarios. However, whether current VLMs can perform well on understanding and
reasoning tasks in aerial-ground collaborative scenarios which are practical in real-world applications like rescue and infrastructure inspection remains underexplored. To address this gap, we introduce AeroGround, a comprehensive
benchmark for evaluating VLMs in aerial-ground collaborative
reasoning. AeroGround is built upon a simulated aerial-ground
dataset containing approximately 29,000
multimodal observation groups from diverse open environments, and provides 2,250 high-quality question-answering instances covering cross-view correspondence, spatial understanding, and
reasoning. Experiments on 16 pretrained VLMs, together with two domain-adapted variants, reveal a substantial gap between current models and human performance: the best model achieves an average accuracy of 54.4%, whereas humans reach 93.3%. By systematically revealing the strengths and limitations of existing models in aerial-ground collaborative
reasoning, AeroGround provides a foundation for developing more capable aerial-ground collaborative
embodied intelligence systems.
Read the full abstract ▾
All authors Shenghong Yi, Lin Zhang, Muzian Li, Jiakang Yuan, Haoyu Zhang, Peng Ye, Jiayuan Fan, Huafeng Qin, Tao Chen
Computer Vision
DeCo-MIL: Debiased Counterfactual Reasoning for Long-Tailed Whole Slide Image Analysis
Xiaoxiao Li, Xitong Ling, Jiawen Li et al. (+6 more)
Multiple instance learning (MIL) is widely used for weakly supervised whole slide image (WSI) analysis. However, under long-tailed distributions, MIL-based WSI analysis faces a nested dual long-tail: an inter-slide class long tail and an intra-slide long tail of instance-level discriminative evidence. The two long tails are coupled: tail classes have few training slides, while their limited diagnostic evidence is concentrated in a few patches and obscured by abundant within-bag redundancy. This coupling biases models toward head classes and degrades rare-class recognition. To address this, we propose DeCo-MIL for long-tailed WSI analysis, which jointly alleviates the nested dual long-tail through frequency-debiased counterfactual
reasoning. For the inner long tail, DeCo-MIL clusters patches into tissue-morphology anchors, replaces each anchor with its matched normal prototype to perform a counterfactual intervention, and estimates its counterfactual contribution to the ground-truth class using class-frequency-corrected predictions. These contributions guide redundancy masking to preserve scarce discriminative instances. For the outer long tail, DeCo-MIL constructs anchor-stratified pseudo-bags from redundancy-reduced bags and combines tail-aware oversampling with consistency regularization, increasing effective supervision for tail classes while preserving tissue-morphology composition. Extensive experiments on three long-tailed WSI
benchmarks demonstrate that DeCo-MIL achieves state-of-the-art performance in both tail-class recognition and overall classification.
Read the full abstract ▾
All authors Xiaoxiao Li, Xitong Ling, Jiawen Li, Weiming Chen, Zhenyang Cai, Xidong Wang, Tian Guan, Benyou Wang, Yonghong He
Computer Vision
VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understanding
Fan Zhang, Guangming Yao, Jinyang Wu et al. (+6 more)
Video understanding is a fundamental task for evaluating the capabilities of
multimodal large
language models (MLLMs). However, existing leading models have already achieved approximately 90% accuracy on the
Video-MME leaderboard, suggesting that conventional single-turn
video understanding tasks are becoming increasingly saturated and insufficient for assessing the intelligence of advanced MLLMs. Towards this end, we introduce VideoGAIA, an agentic
video understanding
benchmark for general artificial intelligence (AI) assistants. Moving beyond one-shot
video question answering, VideoGAIA formulates
video understanding as a multi-turn, tool-augmented interaction process, where models must iteratively perceive
videos, invoke external tools, gather complementary information, and integrate
multimodal evidence across turns. VideoGAIA contains 271 model-human co-designed tasks covering diverse and complex real-world scenarios. Each
video-question-answer instance is independently verified by three human experts to ensure both correctness and appropriate difficulty. All evaluated MLLMs, including frontier models such as GPT-5.5 and Kimi-K3, achieve less than 60% accuracy on VideoGAIA, highlighting its value as a high-quality and timely
benchmark for evaluating next-generation MLLMs. We hope that VideoGAIA will facilitate the transition from conventional
video understanding toward agentic
video understanding.
Read the full abstract ▾
All authors Fan Zhang, Guangming Yao, Jinyang Wu, Hao Wu, Zheng Lian, Xinyu Geng, Jingdong Chen, Yi Yuan, Pheng-Ann Heng
Computer Vision
Local Gains and Fixed-Assignment Set Losses in Shared Set Decoders
Ze Zhang and Yang Zhang
A query-relation deletion can improve the edited slot while reducing the utility of the prediction set that contains it. We study this tension in two related ResNet-50 DETR-family checkpoints using recorded, selection-conditional evidence from 710 paired image-relation units per checkpoint. The primary comparison subtracts a matched active control, which deletes the same leader source at a different recorded recipient, from the selected target deletion. It is therefore a composite contrast rather than a same-recipient placebo. The target-minus-control contrast is locally positive and fixed-assignment negative in both checkpoints. The opposite-sign pattern occurs within 302/710 DETR units and 460/710 DINO units. After rematching, the corresponding counts are 285/710 and 433/710. Rematching and native selection absorb enough of the mean loss for DETR intervals to cross zero, whereas DINO intervals remain negative, so persistence across readouts differs by checkpoint. A fixed-map comparison between hard deletion and a mass-preserving edit also differs before rematching. That comparison is conditional on the outcome-blind map and does not establish same-dose transport. Local intervention success therefore does not determine the consequence for a jointly decoded set. The supported conclusion is selection-conditional deletion sensitivity whose persistence depends on the readout and intervention operator. We do not identify an intervention-invariant edge mechanism, detector-level degradation, population prevalence, or the value of a training-time regularizer.
Read the full abstract ▾
Computer Vision
Inference-Time Orthogonal Seeding Enables Geometry-Aligned 3D Organ Segmentation for Slice-Propagation Methods
Md Rakibul Haque, Tushar Kataria and Shireen Y. Elhabian
Dense voxel-level annotation remains a major bottleneck in 3D medical image
segmentation. Single-slice propagation methods such as Sli2Vol reduce this burden by propagating one annotated seed slice through a volume using label-free registration. However, axial-only propagation accumulates errors with distance from the seed, especially in surface-distance metrics, because it ignores coronal and sagittal evidence and therefore underuses the 3D information available in CT/MRI volumes. To better leverage volumetric geometry, we study how key training and inference choices affect slice-propagation models, including single-axis versus multi-axis label-free registration, single-seed versus multi-seed propagation, and orthogonal seed configurations. Instead of propagating from a single axial seed, we use three orthogonal seeds—one axial, one coronal, and one sagittal—and fuse their propagated labels with a simple label-free rule. Our results show that the training paradigm has limited impact: an axially trained network applied to off-axis seeds captures nearly all the improvement, while explicit three-axis training adds little. Instead, performance is driven by inference-time seed geometry, especially orthogonality rather than the number of annotated slices, as a budget-matched three-axial control provides no benefit and can even degrade performance. On a multi-organ CT cohort, orthogonal seeding with the axial Sli2Vol backbone improves Dice by 21.9%, Normalized Surface Dice by 25.5%, and reduces Average Hausdorff Distance by 53.5% over the single-axis baseline.
Read the full abstract ▾
Computer Vision
EgoCITE: Context-Augmented Indexing and Time-Aware Retrieval for Long-Horizon Egocentric Memory
Le Zhang, Hao Chen, Vlad Roznyatovskiy et al. (+2 more)
Long-horizon
egocentric memory transforms continuous first-person
video and audio into a searchable record of past experiences. We demonstrate two bottlenecks in existing systems: indices built from context-poor captions are unreliable for agentic search, while
retrieval ignores a question's temporal intent. To address both bottlenecks, we introduce EgoCITE (
Egocentric Context-augmented Indexing and Time-aware Evidence
retrieval), a long-horizon agentic
memory framework for
egocentric QA. EgoCITE comprises three components. EgoScheme uses local
multimodal context to turn fragmentary
video captions and
speech transcripts into self-contained atomic
memory indices. EgoIndex organizes complementary action, activity, utterance, and conversation representations into searchable multi-view
memory indices at multiple granularities. EgoRetrv combines semantic search with question-conditioned temporal relevance scoring and curation of retrieved evidence. We evaluate EgoCITE on EgoLifeQA, EgoMem, and EgoR1-Bench in terms of answer accuracy and target-event
retrieval alignment. EgoCITE improves accuracy over agentic
memory baselines by at least 4.4–14.2% while achieving 36$\times$ lower cost than long-context LLM
agents.
Read the full abstract ▾
All authors Le Zhang, Hao Chen, Vlad Roznyatovskiy, Jianzhong Zhang, Ke Sun
Computer Vision
From Visual Widgets to UI Code: Efficient Tool-Grounded Generation
Houston H. Zhang, Tao Zhang, Li Gu et al. (+5 more)
Existing screenshot-to-code systems face a trade-off between flexibility and controllability. Direct
multimodal generation can hallucinate visible details, whereas structured pipelines reduce such errors through component-wise decomposition, predefined templates, and customized intermediate representations. These structures, however, introduce additional generative orchestration and restrict outputs to designs covered by the representation. We investigate whether selective tool grounding can improve the fidelity–efficiency trade-off of direct widget-to-code generation. We introduce WidgetGen, a lightweight tool-grounded framework that extracts observable text and color evidence, performs high-level layout and optional chart
reasoning, and directly generates executable JavaScript XML (JSX). This design reduces reliance on component-wise generation while avoiding a fixed UI schema. Across six
multimodal models and \(1{,}000\) held-out widgets, WidgetGen outperforms direct prompting and the structured Widget2Code pipeline on most visual reconstruction metrics, with consistent gains in area, legibility, and style. Finally, reconstruction-derived image-code pairs improve six Qwen-family open-weight models across every reported metric through supervised fine-tuning. These results establish WidgetGen as a strong lightweight baseline and show that selective evidence grounding offers an effective alternative to extensive representation constraints.
Read the full abstract ▾
All authors Houston H. Zhang, Tao Zhang, Li Gu, Linfeng Ye, Yuanhao Yu, Xinxin Zuo, Yang Wang, Zhixiang Chi
Computer Vision
PseudoMapLabeler: Confidence-Aware Pseudo-Label Generation for Semi-Supervised Online Mapping
Chikao Tsuchiya, Dhaval Bhanderi, David Ilstrup et al. (+2 more)
A critical challenge in deploying online HD map construction systems to real-world scenarios is the scarcity of labeled training data, which limits model generalization in diverse environments. To address this limitation, we propose a teacher-student semi-supervised learning (SSL) framework that generates high-quality pseudo-labels from unlabeled data through confidence-aware map refinement. Our approach first trains a teacher model on limited labeled data, then leverages Beta-distribution-based confidence maps to assess the reliability of predicted map elements across temporal observations. Unlike conventional filtering methods that discard entire elements, we introduce a spatial clipping technique that selectively preserves high-confidence regions while removing unreliable segments. The refined map elements serve as map priors that improve the teacher model's prediction accuracy on unlabeled data in a second pass. These enhanced predictions become pseudo-labels for training a student model from scratch, followed by fine-tuning on the original labeled data. Experimental results on the nuScenes
dataset demonstrate that our teacher-student framework with refined pseudo-labels improves performance by +6.1 mAP under a low-label regime compared to training on labeled data alone, offering a practical solution to the labeled data scarcity problem in online HD map construction.
Read the full abstract ▾
All authors Chikao Tsuchiya, Dhaval Bhanderi, David Ilstrup, Hsinmin Cheng, Christopher Ostafew
Computer Vision · Cross-listed from cs.AI
Auditable agentic AI for evidence-grounded thyroid ultrasound diagnosis and reporting
Haifan Gong, Shiyu Chen, Bodong Wang et al. (+11 more)
Thyroid ultrasound diagnosis requires coordinated lesion localization, measurement, risk stratification and reporting, yet most AI systems address these tasks in isolation and provide limited support for clinical review. We present ThyroidXAgent, a clinician-interactive agentic AI system that coordinates specialized diagnostic tools and stores their outputs as an auditable case-level evidence record. The system was developed using OpenThyroidDB, a multicentre, multitask resource integrating approximately 0.3 million ultrasound images and 24,000 paired reports, and was evaluated on 28,458 non-overlapping test cases, including 8,721 cases from 35 centres in the private NHC-MISD-TUS cohort. Across heterogeneous
datasets, ThyroidXAgent achieved a mean Dice score of 87.21 percent for nodule
segmentation and a mean AUROC of 0.9466 for benign-malignant classification. The same workflow supported lymph-node metastasis prediction and follicular versus papillary thyroid carcinoma classification, with AUROCs of 0.864 and 0.805, respectively. For report generation, evidence-grounded assembly outperformed
multimodal language-model baselines across three cohorts. ThyClinScore, a lesion-level clinical semantic metric introduced here, showed the strongest correlation with a location-aware
language-model judge. ThyroidXAgent improved physician classification accuracy, increased report diagnostic consistency from 70.3 percent to 86.2 percent, and reduced
segmentation and reporting time by 35.9 percent and 27.4 percent, respectively. These findings support auditable, clinician-correctable agentic AI for thyroid ultrasound diagnosis and reporting.
Read the full abstract ▾
All authors Haifan Gong, Shiyu Chen, Bodong Wang, Yuqi Wang, Shijie Wang, Guoliang You, Xinyu Xiong, Haowei Wang, Mingzhi Mao, Dexing Kong, Qinghua Liu, Wei Lou, Fei Chen, Guanbin Li
Computer Vision
Attribute-Conditioned Multimodal Slot Factorization for Controllable Fashion Retrieval
Najmeh Forouzandehmehr, Topojoy Biswas, Evren Korpeoglu et al. (+1 more)
Fashion
retrieval often requires satisfying multiple attributes at once, such as category, color, pattern, and demographic. Monolithic
embeddings mix these signals into a single vector, making attribute-specific control difficult at
retrieval time. Many existing semantic-ID methods provide discrete item codes, but these codes are typically optimized as item-level or residual addresses and do not expose named, independently controllable attribute slots. We introduce MM-slotgate, a
multimodal slot encoder that factorizes Fashion-CLIP text and image
embeddings into four named attribute slots. Each slot learns its own text-image gate, so visually grounded attributes such as color and pattern can rely more on image evidence, while taxonomy-oriented attributes such as category and demographic can remain more text-driven. On H&M, using a combined slot-similarity and slot-logit
retrieval score, MM-slotgate achieves 0.7566 macro ConstraintSatisfied@10, outperforming equal-weight
multimodal fusion (0.7142) and fCLIP text-only
retrieval (0.4755). The largest gain is on color, which improves from 0.321 to 0.889 (+0.568 absolute), as the learned color gate assigns 57.4% weight to image evidence. The learned gates are interpretable without modality supervision: color is image-leaning, category is text-leaning, and pattern and demographic lie near the middle. The resulting slots also remain controllable: linear probes show no measured excess leakage beyond the label-correlation baseline, and quantized slot codes support targeted intervention, including a 15.3x lift for color. These results suggest that controllable fashion
retrieval benefits from typed, attribute-conditioned
multimodal slots rather than either a single global
embedding or opaque item-level semantic IDs.
Read the full abstract ▾
All authors Najmeh Forouzandehmehr, Topojoy Biswas, Evren Korpeoglu, Kannan Achan
Computer Vision
StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos
Julian Spravil, Sebastian Houben and Sven Behnke
Visual content is the dominant medium of communication, yet without audio descriptions (ADs), it remains inaccessible to blind and low-
vision people. ADs narrate context-relevant visual events during natural audio pauses. Manually creating ADs is expensive, limiting coverage to a small fraction of available content. Most existing automatic AD generation methods frame the task as
video clip captioning, requiring ground-truth timestamps and additional context cues such as character databases. Current
benchmarks reinforce this framing, consisting of short
video segments paired with automatic or task-mismatched annotations. We introduce StrAD, a
benchmark for long-form AD generation on full-length
videos spanning diverse genres such as movies, documentaries, short films, performances, and
video games. We reformulate AD generation as streaming dense
video captioning. Our approach processes full-length
videos with a sliding window, inserting ADs into existing transcripts without ground-truth timestamps, and supports both fine-tuned models and zero-shot prompting of
vision-
language models. On the segment-level task with given timestamps, our fine-tuned StrAD-FT sets the state of the art on CMD-AD with 36.3 CIDEr (+10.0 over Shot-by-shot), establishes a reference point on StrAD (51.0 CIDEr), and remains competitive on MAD-Eval at 24.9 CIDEr. On the full-
video streaming task, StrAD-FT reaches a SODA score of 2.4 against 1.1 for our zero-shot baseline StrAD-Zero, though both exhibit limitations in temporal localization and narrative coherence. While prior work has tackled full-
video AD generation in an offline, multi-stage fashion, ours is the first streaming approach, generating ADs on the fly without ground-truth timestamps. StrAD makes progress on full-
video AD generation measurable, a prerequisite for scaling accessibility.
Read the full abstract ▾
Computer Vision
Surface-to-Skeleton 3D Cephalometry: Estimating Hidden Skeletal Landmarks from CT-Derived External Soft-Tissue Surfaces
Tomoki Abe, Taiki Kanaya, Kazuki Saita et al. (+4 more)
Existing 3D facial-landmark methods localize points on visible skin, but whether CT-defined internal skeletal landmarks can be inferred from external soft-tissue geometry remains unclear. We formulate a coordinate-consistent surface-to-skeleton task using same-acquisition CT-derived surfaces, separating estimation from optical-to-CT registration, scanner-domain, and acquisition-state effects, with coverage analyzed separately. From 240 clinical CT scans from two hospitals, we construct a locked retrospective protocol pairing CT-derived external soft-tissue point clouds with 21 skeletal landmarks and three visible soft-tissue landmarks. An integrated hierarchical point-cloud model achieves 2.97 mm mean radial error on skeletal landmarks and 3.03 mm on deep or surface-invisible landmarks in 40 held-out patients. Patient-mismatch controls support patient-specific signal beyond a fixed population configuration or global similarity alone, while coverage ablations indicate dependence on non-anterior geometry. Optical-transfer diagnostics reveal substantial coverage-related and global-configuration components, although deployable optical inference remains unresolved. These results answer the controlled feasibility question affirmatively and provide a basis for hidden skeletal landmark inference.
Read the full abstract ▾
All authors Tomoki Abe, Taiki Kanaya, Kazuki Saita, Mao Noda, Chie Tachiki, Yasushi Nishii, Hideo Saito
Computer Vision · Cross-listed from cs.MM
MASCOT: Model-Aware Submodular Coverage for Composite-Attribute Text-to-Image Retrieval
Aaryan Sharma, Vishak Prasad C, Virendra Singh et al. (+1 more)
Vision-
Language Models (VLMs) are highly effective in retrieving semantically relevant images. However, in practice, relevance alone is often insufficient. Systems must also achieve Result Diversification (RD) across composite attributes such as geography and time, a task for which precise control remains challenging. Current re-ranking methods, such as Multi-Source Determinantal Point Processes (MS-DPP), address this using
manifold-based repulsion over similarity representations. Although this strategy is effective for broad exploration, it exposes a key limitation in
manifold-based models: when subjected to diversity-decrease tasks on discrete metadata, they suffer substantial degradation in early-rank recall. To bridge this gap, we introduce MASCOT (Model-Aware Submodular Coverage for Composite-Attribute Text-to-Image
Retrieval). Instead of relying on
manifold repulsion, MASCOT formulates multi-attribute diversity as a resource allocation problem, projecting attributes into a soft-binning space weighted by query-driven importance. Averaged across the three PixelProse diversity-decrease tasks, MASCOT preserves an early-rank recall (R@10) of 88.58%, while MS-DPP retains 67.63%. The margin widens under composite constraints: on PP_geo_hour, where temporal and geographic diversity must be suppressed simultaneously, MS-DPP's recall collapses from 0.9737 to 0.4931 and its top-ranked result degrades to R@1 = 0.23, while MASCOT holds R@10 = 0.9410 and R@1 = 0.7202 at a diversity metric above the unconstrained baseline. We do not claim uniform superiority: on aggregate diversity-relevance scores our own simpler ablations attain higher harmonic means on all three decrease tasks, and MASCOT's advantage is specific to recall beyond rank 1 under composite constraints.
Read the full abstract ▾
All authors Aaryan Sharma, Vishak Prasad C, Virendra Singh, Ganesh Ramakrishnan
Computer Vision
Vladyslava Rudas and Dmytro Kuzmenko
Assessing proxemic danger from a
robot's
egocentric perspective is critical for safe
embodied navigation in human environments and requires both visual and contextual
reasoning. We evaluate three opensource
vision-
language models (VLMs) (InternVL, Qwen-VL, and SmolVLM) on the classification of
egocentric robot images into four danger levels, comparing three prompting strategies and two rounds of QLoRA fine-tuning against a stratified random baseline. Without fine-tuning, all models perform near the baseline, while fine-tuning yields only modest overall improvements. However, Qwen-VL with an advanced
prompt achieves substantially higher recall for high-danger cases than the other models. An analysis of person localization further shows that correct danger classification does not correspond to better spatial grounding, indicating that a model may produce a useful safety label without attending to the relevant region of the scene. These results show that current VLMs remain limited in fine-grained proxemic
reasoning and spatial grounding, although targeted prompting and fine-tuning can improve high-danger detection in selected models.
Read the full abstract ▾
Computer Vision
HIMEC: Directional Change Representation and Fixed-Interface Decoding for Remote Sensing Image Change Captioning
Aysha Ashraf, Shaina Ashraf, Wafaa I. M. Hussin et al. (+3 more)
Remote sensing image change captioning (RSICC) converts bitemporal imagery into a sentence describing semantic changes. Most RSICC methods condition caption decoders directly on fused visual features, leaving intermediate change structure and decoder-interface consistency less studied. We present HIMEC, combining Directional Change Representation (DCR) with fixed-interface decoding. DCR separates signed differences into appearance-oriented, disappearance-oriented, and shared-context streams before fusion. A learned-query encoder converts the fused representation into visually conditioned change-query
tokens that form the scene decoder's only sample-dependent
memory. A training-only auxiliary phrase decoder supplies caption-derived supervision. With a fixed zero input, the scene decoder maintains the same interface during training and inference. Separately, we evaluate a local-to-scene cascade conditioned on teacher-forced local states during training and autoregressive states at inference. On changed LEVIR-CC validation pairs, these states have a mean cosine distance of 0.69. Regime-matched conditioning recovers most of the associated deficit, whereas permuting state correspondence causes no detectable penalty. These findings are limited to the evaluated cascade. In a matched three-seed comparison, HIMEC reaches a Consensus-based Image Description Evaluation (CIDEr) score of $142.81\pm0.60$ on LEVIR-CC, versus $139.51\pm3.40$ for direct fused-feature
memory. On SECOND-CC, fixed-zero and regime-matched diagnostic conditioning reach 75.67 and 76.99 CIDEr, respectively, versus 60.77 for the mismatched cascade. The source code will be made publicly available at https://github.com/ayshaashra/HIMEC upon publication.
Read the full abstract ▾
All authors Aysha Ashraf, Shaina Ashraf, Wafaa I. M. Hussin, Ali Haider, Zhi Lu, Zhenming Peng
Computer Vision
MV2: Multi-View Multi-Vehicle Driving Dataset for Novel View Synthesis
Sanjay Bhargav Dharavath, Hanvitha Saraswathi Mukkamala, Faizan Farooq Khan et al. (+4 more)
Differentiable rendering has advanced novel view synthesis (NVS), yet applying it to real-world driving remains difficult due to sparse capture viewpoints, dynamic objects, and limited multi-trajectory data. We introduce the Multi-View Multi-Vehicle (MV2)
dataset and
benchmark for evaluating NVS models under large viewpoint changes in dynamic urban scenes. MV2 features synchronized captures from a car, scooter, and drone, each following distinct yet synchronized trajectories. Training NVS methods on one vehicle's camera stream and testing on another enables evaluation under substantially larger viewpoint variations than existing single-trajectory
datasets. All sequences are registered via Structure-from-Motion and camera poses verified using manual pixel-level correspondence annotations, yielding 50 high-quality scenes with 12000 images. Benchmarking recent NVS and camera pose estimation methods shows that NVS performance degrades with increasing viewpoint disparity, and that feed-forward pose
estimators notably lag behind optimization-based approaches, highlighting MV2 as a rigorous testbed for NVS in driving. The
dataset,
benchmark protocol, and project resources are available at https://mv2-
dataset.github.io/.
Read the full abstract ▾
All authors Sanjay Bhargav Dharavath, Hanvitha Saraswathi Mukkamala, Faizan Farooq Khan, Ioannis Kakogeorgiou, Aditya Arun, C V Jawahar, Zakaria Laskar
Computer Vision
StateFlow: Building, Evolving, and Accessing 3D World States for Previsualization
Yuyang Yin, Zixiang Li, Longxuan Deng et al. (+11 more)
Previsualization is an intermediate layer between ideas and production in film, games, architecture, and urban design. It lets creators iteratively refine scenes, actions, cameras, and spatial-temporal dynamics. Yet existing generative methods rely on simple
prompts to jointly control all of these factors through one-shot image or
video synthesis, offering weak controllability and limited support for iterative editing. Fundamentally, a world comprises multiple elements with geometry, appearance, and other attributes, together with cameras. Different frames are produced through local modifications or recombinations of this shared state, which is otherwise largely reused. Therefore, we argue that the missing component is an explicit and persistent working state. To address this, we present StateFlow, a state-centric framework for generative previsualization. Rather than generating
videos in one shot, StateFlow uses an editable 3D world to organize scene structure,
evolution, and cameras, while off-the-shelf
video models enhance visual quality when higher fidelity is desired. This world is maintained as a persistent structured 3D state of scene elements and camera configurations, serving as the core working representation for previsualization. Built on this insight, StateFlow has three stages to construct, evolve, and access the world state. State construction lifts generated 2D content into a coherent 3D world through prior-guided, conflict-aware dual-view initialization, while State
evolution translates user intent into structured state transitions while preserving world
memory, avoiding full-scene regeneration for each edit. State access uses render-feedback reflection to refine camera plans into visually feasible trajectories, avoiding reliance on VLM semantics alone. Experiments show that StateFlow produces high-quality 3D worlds for
video creation and game-like prototyping.
Read the full abstract ▾
All authors Yuyang Yin, Zixiang Li, Longxuan Deng, Hongkai Li, Shifang Zhao, Junnan Liu, Weirong Huang, Mengyu Wang, Tianxiao Fu, Yikai Wang, Peng-Shuai Wang, Xiaojie Jin, Yao Zhao, Yunchao Wei
Computer Vision
AVA-Encoder: Towards Agent-Native Video Representation Learning
Chuyue Li, Jinpeng Yu, Haozhe Wang et al. (+7 more)
Agentic
video creation systems remain far from cinematic-grade quality: their underlying models struggle to plan how story, characters, visuals, and sound fit together, and high-quality creation trajectories to learn such planning abilities are scarce. Finished human films embody this knowledge, but in a tightly coupled audiovisual form that
agents cannot directly read or manipulate. We propose the Agentic
Video Auto-Encoder (AVA-Encoder), which works backward from finished films, turning them into
agent-native representations with reconstruction itself as supervision. It encodes a film into a Film Knowledge Graph (Film KG) of structured-text nodes, typed dependency edges, and linked
multimodal assets, then regenerates the film from this graph with a fixed decoder. Reconstruction residuals are distilled into textual gradients that drive dual-loop self-
evolution: an outer loop learns a shared encoding policy across
videos, and an inner loop refines each
video's Film KG at test time. AVA-Encoder improves reconstruction fidelity by 20.7 percentage points over the strongest baseline; its learned policy outperforms a finely human-tuned policy with over 70 percent fewer system-
prompt tokens; and Film KGs support coordinated editing and improve all tested downstream creation systems. We release the framework, a reconstruction
benchmark, and a Film KG
dataset.
Read the full abstract ▾
All authors Chuyue Li, Jinpeng Yu, Haozhe Wang, Tian Xueyun, Zhijing Zhang, Bingnan Li, Shuqi Gu, Kan Ren, Jiaming Liu, Ruihua Huang
Computer Vision
Yan Deng and Fei Xu
Aerial
vision-
language navigation (VLN) requires an
embodied agent to integrate visual evidence over time, plan future actions, and determine when it has reached a
navigation goal under partial observability. Although recent VLA models offer a promising perception-to-action paradigm, adapting them to aerial
navigation remains challenging due to limited historical context, short planning horizons, and unreliable implicit termination. To address these challenges, we propose DreamFly, a
diffusion-based aerial VLN framework built on Dream-VLA. DreamFly introduces a causally aligned historical
memory that augments the current visual representation using only observations preceding the current decision step, enabling temporal
reasoning without future information leakage. We further formulate
navigation as receding-horizon
diffusion planning, where the policy predicts a $K$-step action chunk but executes only the first action before replanning. This plan-$K$, execute-one strategy uses future actions as auxiliary planning targets while preserving closed-loop visual feedback. Finally, LiteStop estimates the stop probability directly from action logits at the initial all-mask state, decoupling explicit termination from action generation. Experiments on the OpenFly
benchmark demonstrate consistent improvements in seen and unseen environments. DreamFly achieves 32.04%/29.46% SR and 28.22%/23.54% SPL on the test-seen/test-unseen splits, respectively, outperforming all compared methods on both metrics while attaining the lowest
navigation error. These results demonstrate the effectiveness of jointly modeling historical context, future action structure, and explicit termination for aerial VLN.
Read the full abstract ▾
Computer Vision
Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations
AmirHossein Eshghi, Hamid Saadatfar, Seyyed Ali Hoseini et al. (+2 more)
Class activation mapping (CAM) is one of the most widely used visual explanation families in explainable artificial intelligence. Its purpose is intuitive: it converts internal model evidence into a heatmap that highlights the image regions, convolutional channels,
tokens, or patches that support a target class or concept. Since the first CAM formulation in 2016, the field has moved far beyond global-average-pooled CNN classifiers. CAM-style methods now include gradient-based post-hoc explanations, gradient-free score and ablation methods, high-resolution upscaling, weakly supervised localization and
segmentation,
transformer token attribution, causal and debiasing methods, and foundation-model-era approaches that use CLIP, DINO, SAM, or feature-distribution comparisons. This review synthesizes a strict corpus of 57 method-centered papers published from 2016 onward. The paper develops a taxonomy that separates methods by attribution mechanism, architectural dependence, and evaluation objective. It then reviews gradient-based CAMs, recent and hybrid CAM-style methods, and model-based or architecture-aware methods. Across the corpus, the main trend is clear: the field is shifting from explaining one class score in one low-resolution CNN layer toward comparative, multi-layer, probabilistic,
token-aware, and foundation-model-aware explanations. At the same time, evaluation remains fragmented. Faithfulness, localization, robustness, computational cost, and human trust are often measured with different protocols. The review therefore emphasizes not only what each method contributes, but also which gap it leaves open and which later methods attempt to close that gap.
Read the full abstract ▾
All authors AmirHossein Eshghi, Hamid Saadatfar, Seyyed Ali Hoseini, AmirMohsen Eshghi, Siavash Arjomand Bigdeli
Computer Vision
Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence
Aman Tyagi, Hemanth Boinpally, Jonathan Chen et al. (+2 more)
Modern black-box Image-to-
Video (I2V) models offer powerful capabilities in automated content creation, yet their lack of fine-grained control and reliability presents significant challenges in professional workflows. Their inherent stochasticity causes minor variations in textual
prompts or hyperparameters to yield drastically different outputs often necessitating inefficient, brute-force trial-and-error processes. To address these limitations, we introduce the “Agentic Self-Improvement" framework, which reframes
video synthesis into a closed-loop, goal-directed optimization. Our framework systematically navigates the generation parameter space using a novel two-stage approach. In the first stage, an iterative
prompt optimization loop uses a
multimodal Large
Language Model (mLLM) to refine the input
prompt. This refinement implements two automated evaluations: Davidsonian Scene Graph (DSG) queries ensure semantic adherence, and Common Mistake Questions (CMQ) for artifact detection. At the second stage, we use
Bayesian optimization to efficiently co-optimize
stochastic seeds and CFG scales. This search is guided by a suite of quality metrics, including the novel
Video-Text Adherence (VTA) score derived from the DSG and CMQ evaluations. Our framework significantly outperforms unguided search methods: in human preference studies,
videos generated via our agentic approach were strongly preferred over baseline outputs, achieving win rates up to 69%. This work provides a practical and extensible methodology for enhancing the predictability and control of state-of-the-art
video generation models, moving the field beyond speculative curiosities toward reliable, production-ready tools.
Read the full abstract ▾
All authors Aman Tyagi, Hemanth Boinpally, Jonathan Chen, Douglas Gebert, Steven Hickson
Computer Vision
Curvature-Aware Zeroth-Order Optimization for Memory-Efficient Test-Time Adaptation
Junming Zhang, Shuyu Yin, Peilin Liu et al. (+2 more)
Test-time adaptation (TTA) aims to enhance the cross-domain performance of pre-trained models by adapting to unlabeled test data. While most existing TTA methods rely on backpropagation (BP) for finetuning, BP-free methods such as zeroth-order (ZO) methods are more desired in practical on-device scenarios. ZO methods rely only on forward computation, which can largely reduce the complexity and
memory overhead of on-device deployment. However, ZO methods suffer from much higher variance compared with first-order methods in estimating the gradient. To address this, we propose an improved ZO method to substantially boost the performance of ZO optimization based TTA. First, we provide an observation to reveal the persistent low-rank Hessian structure of the loss during the adaptation process. Based on this insight, we then propose a loss-landscape
curvature-aware zeroth-order (CAZO) method, which leverages a sliding-average estimation of the diagonal Hessian to construct a covariance
matrix for anisotropic perturbation sampling. CAZO operates by freezing pretrained weights and optimizing minimal adapter parameters via forward-only passes based gradient estimation, which can substantially reduce the
memory overhead compared to BP-based methods. Extensive experiments demonstrate that CAZO significantly outperforms existing TTA methods, achieving state-of-the-art performance while maintaining an excellent balance between accuracy and
memory efficiency. Code is available at https://github.com/Hollyming/CAZO.
Read the full abstract ▾
All authors Junming Zhang, Shuyu Yin, Peilin Liu, Rendong Ying, Fei Wen
Computer Vision
XYZFlow:Scaling Multi dimensional Shortcut Flows for Efficient Generative Modeling
Jinxiu Liu, Xuanming Liu, Kangfu Mei et al. (+2 more)
High-fidelity image generation faces a trade-off between speed and quality.
Diffusion models produce strong visuals but require costly iterative sampling. Existing efficient methods mainly distill pretrained models into few-step samplers, a challenging process that depends heavily on teacher-model quality. In this paper, we introduce XYZFlow, a framework that rethinks efficient generation through multidimensional scaling of flow matching. Unlike single-step mappings, XYZFlow enhances expressivity by making probability paths more identifiable and learnable through structured multidimensional conditioning. We view autoregressive modeling as implicit flow straightening, where richer context reduces trajectory ambiguity. XYZFlow realizes this idea through two orthogonal dimensions: temporal scaling, which uses non-Markovian conditioning on the full denoising history; and spatial scaling, enabled by Next Shortcut Prediction, which sequentially generates patches using preceding patches' denoising trajectories as priors. Experiments show that XYZFlow achieves state-of-the-art performance, with 7.2-8.5X teacher speedups and competitive FID, while Next Shortcut Prediction delivers superior quality-latency trade-offs over model scaling or step reduction.
Read the full abstract ▾
All authors Jinxiu Liu, Xuanming Liu, Kangfu Mei, Yandong Wen, Weiyang Liu
Computer Vision
A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery
Rafi Ibn Sultan, Chengyin Li, Yiannos Demetriou et al. (+6 more)
Background: Accurate
segmentation of the Left Anterior Descending (LAD) artery in 3D free-breathing, non-contrast CT is critical for cardiac dose sparing in thoracic radiotherapy. The LAD is extremely small, has poor soft-tissue contrast, and varies substantially across patients; even manual contours show limited inter-observer agreement, underscoring the ambiguity of the vessel boundaries. Purpose: To develop a
transformer-based framework that improves LAD delineation in low-contrast, imbalanced CT through local-global context modeling and uncertainty-guided optimization. Methods: We propose NA-UNETR, a 3D
transformer-based
segmentation model whose Neighborhood
Attention (NA) and Dilated NA (DiNA) blocks jointly capture fine structural detail and long-range context. Given the scarcity of annotated LAD data, the model is pretrained on 1,000 CTA volumes of general coronary anatomy and fine-tuned with LoRA-based parameter-efficient adaptation on 20 free-breathing institutional CT scans. A composite Dice-Focal and Hausdorff loss, dynamically balanced via homoscedastic uncertainty, improves overlap and boundary accuracy. Results: NA-UNETR reached 45.64% Dice, 38.16 mm HD95, and 10.01 mm ASD, improving Dice by 3.10 percentage points over nnU-Net and reducing HD95 by 2.96 mm relative to Swin UNETR, with the strongest boundary accuracy among all models and improved centerline stability. On ImageCAS it achieved 79.49% Dice, 8.89 mm HD95, and 1.02 mm ASD. Ablations confirmed that residual blocks, variable kernels, and uncertainty-weighted loss each contributed. Conclusions: NA-UNETR balances local precision and global context for thin, low-contrast LAD structures, offering a computationally efficient framework for substructure-level cardiac
segmentation in radiotherapy planning.
Read the full abstract ▾
All authors Rafi Ibn Sultan, Chengyin Li, Yiannos Demetriou, Ahmed I. Ghanem, Joshua P. Kim, Justine Cunningham, Hassan Bagher-Ebadian, Dongxiao Zhu, Kundan S. Thind