Computer Vision
Adapting Depth Anything to Adverse Imaging Conditions with Events
Shihan Peng, Yuyang Xiong, Hanyu Zhou et al. (+5 more)
Robust depth estimation under dynamic and adverse lighting conditions is essential for robotic systems. Currently, depth foundation models, such as Depth Anything, achieve great success in ideal scenes but remain challenging under adverse imaging conditions such as extreme illumination and motion blur. These degradations corrupt the visual signals of frame cameras, weakening the discriminative features of frame-based depths across the spatial and temporal dimensions. Typically, existing approaches incorporate event cameras to leverage their high dynamic range and temporal resolution, aiming to compensate for corrupted frame features. However, such specialized fusion models are predominantly trained from scratch on domain-specific
datasets, thereby failing to inherit the open-world knowledge and robust generalization inherent to foundation models. In this work, we propose ADAE, an event-guided spatiotemporal fusion framework for Depth Anything in degraded scenes. Our design is guided by two key insights: 1)
Entropy-Aware Spatial Fusion. We adaptively merge frame-based and event-based features using an information
entropy strategy to indicate illumination-induced degradation. 2) Motion-Guided Temporal Correction. We resort to the event-based motion cue to recalibrate ambiguous features in blurred regions. Under our unified framework, the two components are complementary to each other and jointly enhance Depth Anything under adverse imaging conditions. Extensive experiments have been performed to verify the superiority of the proposed method. Our code will be released upon acceptance.
Read the full abstract ▾
All authors Shihan Peng, Yuyang Xiong, Hanyu Zhou, Zhiwei Shi, Haoyue Liu, Gang Chen, Luxin Yan, Yi Chang
Computer Vision
Towards Any-Quality Image Segmentation via Generative and Adaptive Latent Space Enhancement
Guangqian Guo, Aixi Ren, Yong Guo et al. (+6 more)
Segment Anything Models (SAMs), known for their exceptional zero-shot
segmentation performance, have garnered significant
attention in the research community. Nevertheless, their performance drops significantly on severely degraded, low-quality images, limiting their effectiveness in real-world scenarios. To address this, we propose GleSAM++, which utilizes Generative Latent space Enhancement to boost robustness on low-quality images, thus enabling generalization across various image qualities. Additionally, to improve compatibility between the pre-trained
diffusion model and the
segmentation framework, we introduce two techniques, i.e., Feature Distribution
Alignment (FDA) and Channel Replication and Expansion (CRE). However, the above components lack explicit guidance regarding the degree of degradation. The model is forced to implicitly fit a complex noise distribution that spans conditions from mild noise to severe artifacts, which substantially increases the learning burden and leads to suboptimal reconstructions. To address this issue, we further introduce a Degradation-aware Adaptive Enhancement (DAE) mechanism. The key principle of DAE is to decouple the reconstruction process for arbitrary-quality features into two stages: degradation-level prediction and degradation-aware reconstruction. Our method can be applied to pre-trained SAM and SAM2 with only minimal additional learnable parameters, allowing for efficient optimization. Extensive experiments demonstrate that GleSAM++ significantly improves
segmentation robustness on complex degradations while maintaining generalization to clear images. Furthermore, GleSAM++ also performs well on unseen degradations, underscoring the versatility of our approach and
dataset.
Read the full abstract ▾
All authors Guangqian Guo, Aixi Ren, Yong Guo, Xuehui Yu, Jiacheng Tian, Wenli Li, Chaowei Wang, Yaoxing Wang, Shan Gao
Computer Vision
Enhancing Object Detection with Privileged Information: A Model-Agnostic Teacher-Student Approach
Matthias Bartolo, Dylan Seychell, Gabriel Hili et al. (+4 more)
This paper investigates the integration of the Learning Using Privileged Information (LUPI) paradigm in object detection to exploit fine-grained, descriptive information available during training but not at inference. We introduce a general, model-agnostic methodology for injecting privileged information-such as bounding box masks, saliency maps, and depth cues-into deep learning-based object detectors through a teacher-student architecture. Experiments are conducted across five state-of-the-art object detection models and multiple public
benchmarks, including
UAV-based litter detection
datasets and Pascal VOC 2012, to assess the impact on accuracy, generalization, and computational efficiency. Our results demonstrate that LUPI-trained students consistently outperform their baseline counterparts, achieving significant boosts in detection accuracy with no increase in inference complexity or model size. Performance improvements are especially marked for medium and large objects, while ablation studies reveal that intermediate weighting of teacher guidance optimally balances learning from privileged and standard inputs. The findings affirm that the LUPI framework provides an effective and practical strategy for advancing object detection systems in both resource-constrained and real-world settings.
Read the full abstract ▾
All authors Matthias Bartolo, Dylan Seychell, Gabriel Hili, Matthew Montebello, Carl James Debono, Saviour Formosa, Konstantinos Makantasis
Computer Vision · Cross-listed from cs.AI
XAI-MeD: Explainable Knowledge Guided Neuro-Symbolic Framework for Domain Generalization and Rare Class Detection in Medical Imaging
Midhat Urooj, Ayan Banerjee and Sandeep Gupta
Explainability domain generalization and rare class reliability are critical challenges in medical AI where deep models often fail under real world distribution shifts and exhibit bias against infrequent clinical conditions This paper introduces XAIMeD an explainable medical AI framework that integrates clinically accurate expert knowledge into deep learning through a unified neuro symbolic architecture XAIMeD is designed to improve robustness under distribution shift enhance rare class sensitivity and deliver transparent clinically aligned interpretations The framework encodes clinical expertise as logical connectives over atomic medical propositions transforming them into machine checkable class specific rules Their diagnostic utility is quantified through weighted feature satisfaction scores enabling a symbolic
reasoning branch that complements neural predictions A confidence weighted fusion integrates symbolic and deep outputs while a Hunt inspired adaptive routing mechanism guided by
Entropy Imbalance Gain EIG and Rare Class Gini mitigates class imbalance high intra class variability and uncertainty We evaluate XAIMeD across diverse modalities on four challenging tasks i Seizure Onset Zone SOZ localization from rs fMRI ii Diabetic Retinopathy grading across 6 multicenter
datasets demonstrate substantial performance improvements including 6 percent gains in cross domain generalization and a 10 percent improved rare class F1 score far outperforming state of the art deep learning baselines Ablation studies confirm that the clinically grounded symbolic components act as effective regularizers ensuring robustness to distribution shifts XAIMeD thus provides a principled clinically faithful and interpretable approach to
multimodal medical AI.
Read the full abstract ▾
Computer Vision
Nighttime Hazy Image Enhancement via Progressively and Mutually Reinforcing Night-Haze Priors
Chen Zhu, Huiwen Zhang, Mu He et al. (+2 more)
Enhancing the visibility of nighttime hazy images is challenging due to the complex degradation distributions. Existing methods mainly address a single type of degradation (e.g., haze or low-light) at a time, ignoring the interplay of different degradation types and resulting in limited visibility improvement. We observe that the domain knowledge shared between low-light and haze priors can be reinforced mutually for better visibility. Based on this key insight, in this paper, we propose a novel framework that enhances visibility in nighttime hazy images by reinforcing the intrinsic consistency between haze and low-light priors mutually and progressively. In particular, our model utilizes image-, patch-, and pixel-level experts that operate across visual and frequency domains to recover global scene structure, regional patterns, and fine-grained details progressively. A frequency-aware router is further introduced to adaptively guide the contribution of each expert, ensuring robust image restoration. Extensive experiments demonstrate the superior performance of our model on nighttime dehazing
benchmarks both quantitatively and qualitatively. Moreover, we showcase the generalizability of our model in daytime dehazing and low-light enhancement tasks.
Read the full abstract ▾
All authors Chen Zhu, Huiwen Zhang, Mu He, Yujie Li, Xiaotian Qiao
Computer Vision
API: Empowering Generalizable Real-World Image Dehazing via Adaptive Patch Importance Learning
Chen Zhu, Huiwen Zhang, Yujie Li et al. (+2 more)
Real-world image dehazing is a fundamental yet challenging task in low-level
vision. Existing learning-based methods often suffer from significant performance degradation when applied to complex real-world hazy scenes, primarily due to limited training data and the intrinsic complexity of haze density distributions.To address these challenges, we introduce a novel Adaptive Patch Importance-aware (API) framework for generalizable real-world image dehazing. Specifically, our framework consists of an Automatic Haze Generation (AHG) module and a Density-aware Haze Removal (DHR) module. AHG provides a hybrid data augmentation strategy by generating realistic and diverse hazy images as additional high-quality training data. DHR considers hazy regions with varying haze density distributions for generalizable real-world image dehazing in an adaptive patch importance-aware manner. To alleviate the ambiguity of the dehazed image details, we further introduce a new Multi-Negative Contrastive Dehazing (MNCD) loss, which fully utilizes information from multiple negative samples across both spatial and frequency domains. Extensive experiments demonstrate that our framework achieves state-of-the-art performance across multiple real-world
benchmarks, delivering strong results in both quantitative metrics and qualitative visual quality, and exhibiting robust generalization across diverse haze distributions.
Read the full abstract ▾
All authors Chen Zhu, Huiwen Zhang, Yujie Li, Mu He, Xiaotian Qiao
Computer Vision
VIT-Ped: Visionary Intention Transformer for Pedestrian Behavior Analysis
Aly R. Elkammar, Karim M. Gamaleldin and Catherine M. Elias
Pedestrian Intention prediction is one of the key technologies in the transition from level 3 to level 4 autonomous driving. To understand pedestrian crossing behaviour, several elements and features should be taken into consideration to make the roads of tomorrow safer for everybody. We introduce a
transformer /
video vision transformer based
algorithm of different sizes which uses different data modalities .We evaluated our
algorithms on popular pedestrian behaviour
dataset, JAAD, and have reached SOTA performance and passed the SOTA in metrics like Accuracy, AUC and F1-score. The advantages brought by different model design choices are investigated via extensive ablation studies.
Read the full abstract ▾
Computer Vision
Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representation
Weijian Ma, Shizhao Sun, Tianyu Yu et al. (+3 more)
Spatial
reasoning – the ability to perceive and reason about relationships in space – advances
vision-
language models (VLMs) from visual perception toward spatial semantic understanding. Existing approaches either revisit local image patches, improving fine-grained perception but weakening global spatial awareness, or mark isolated coordinates, which capture object locations but overlook their overall organization. In this work, we integrate the cognitive concept of an object-centric blueprint into VLMs to enhance spatial
reasoning. Given an image and a question, the model first constructs a JSON-style blueprint that records the positions, sizes, and attributes of relevant objects, and then reasons over this structured representation to produce the final answer. To achieve this, we introduce three key techniques: (1) blueprint-embedded
reasoning traces for supervised fine-tuning to elicit basic
reasoning skills; (2) blueprint-aware rewards in reinforcement learning to encourage the blueprint to include an appropriate number of objects and to align final answers with this causal
reasoning; and (3) anti-shortcut data augmentation that applies targeted perturbations to images and questions, discouraging reliance on superficial visual or linguistic cues. Experiments show that our method consistently outperforms existing VLMs and specialized spatial
reasoning models.
Read the full abstract ▾
All authors Weijian Ma, Shizhao Sun, Tianyu Yu, Ruiyu Wang, Tat-Seng Chua, Jiang Bian
Computer Vision
Forget Less by Learning Together through Concept Consolidation
Arjun Ramesh Kaushik, Naresh Kumar Devulapally, Vishnu Suresh Lokhande et al. (+2 more)
Custom
Diffusion Models (CDMs) have gained significant
attention due to their remarkable ability to personalize generative processes. However, existing CDMs suffer from catastrophic forgetting when continuously learning new concepts. Most prior works attempt to mitigate this issue under the sequential learning setting with a fixed order of concept inflow and neglect inter-concept interactions. In this paper, we propose a novel framework - Forget Less by Learning Together (FL2T) - that enables concurrent and order-agnostic concept learning while addressing catastrophic forgetting. Specifically, we introduce a set-invariant inter-concept learning module where proxies guide feature selection across concepts, facilitating improved knowledge retention and transfer. By leveraging inter-concept guidance, our approach preserves old concepts while efficiently incorporating new ones. Extensive experiments, across three
datasets, demonstrates that our method significantly improves concept retention and mitigates catastrophic forgetting, highlighting the effectiveness of inter-concept catalytic behavior in incremental concept learning of ten tasks with at least 2% gain on average CLIP Image
Alignment scores.
Read the full abstract ▾
All authors Arjun Ramesh Kaushik, Naresh Kumar Devulapally, Vishnu Suresh Lokhande, Nalini Ratha, Venu Govindaraju
Computer Vision
AFTER: Mitigating the Object Hallucination of LVLM via Adaptive Factual-Guided Activation Editing
Tianbo Wang, Yuqing Ma, Kewei Liao et al. (+4 more)
Large
Vision-
Language Models (LVLMs) have achieved substantial progress in cross-modal tasks. However, due to
language bias, LVLMs are susceptible to object
hallucination, which can be primarily divided into category, attribute, and relation
hallucination, significantly impeding the trustworthy AI applications. Editing the internal activations of LVLMs has shown promising effectiveness in mitigating
hallucinations with minimal cost. However, previous editing approaches neglect the effective guidance offered by factual textual semantics, thereby struggling to explicitly mitigate
language bias. To address these issues, we propose Adaptive Factual-guided Visual-Textual Editing for
hallucination mitigation (AFTER), which comprises Factual-Augmented Activation Steering (FAS) and Query-Adaptive Offset Optimization (QAO), to adaptively guides the original biased activations towards factual semantics. Specifically, FAS is proposed to provide factual and general guidance for activation editing, thereby explicitly modeling the precise visual-textual associations. Subsequently, QAO introduces a query-aware offset
estimator to establish query-specific editing from the general steering vector, enhancing the diversity and granularity of editing. Extensive experiments on standard
hallucination benchmarks across three widely adopted LVLMs validate the efficacy of the proposed AFTER, notably achieving up to a 16.3% reduction of
hallucination over baseline on the AMBER
benchmark. Our code and data will be released for reproducibility.
Read the full abstract ▾
All authors Tianbo Wang, Yuqing Ma, Kewei Liao, Zhange Zhang, Simin Li, Jinyang Guo, Xianglong Liu
Computer Vision
MotionAdapter: Video Motion Transfer via Content-Aware Attention Customization
Zhexin Zhang, Yangyang Xu, Yifeng Zhu et al. (+4 more)
Recent advances in
diffusion-based text-to-
video models, particularly those built on the
diffusion transformer architecture, have achieved remarkable progress in generating high-quality and temporally coherent
videos. However, transferring complex motions between
videos remains challenging. In this work, we present MotionAdapter, a content-aware motion transfer framework that enables robust and semantically aligned motion transfer within DiT-based
video diffusion models. Our key insight is that effective motion transfer requires 1) explicit disentanglement of motion from appearance and 2) adaptive customization of motion to target content. MotionAdapter first isolates motion by analyzing cross-frame
attention within 3D full-
attention modules to extract
attention-derived motion fields. To bridge the semantic gap between reference and target
videos, we further introduce a DINO-guided motion customization module that rearranges and refines motion fields based on content correspondences. The customized motion field is then used to guide the DiT denoising process, ensuring that the synthesized
video inherits the reference motion while preserving target appearance and semantics. Extensive experiments demonstrate that MotionAdapter outperforms state-of-the-art methods in both qualitative and quantitative evaluations. Moreover, MotionAdapter naturely support complex motion transfer and motion editing tasks such as zooming in/out and composition.
Read the full abstract ▾
All authors Zhexin Zhang, Yangyang Xu, Yifeng Zhu, Long Chen, Yong Du, Shengfeng He, Jun Yu
Computer Vision
Face Normal Estimation from Rags to Riches
Meng Wang, Wenjing Dai, Jiawan Zhang et al. (+1 more)
Although recent approaches to face normal estimation have achieved promising results, their effectiveness heavily depends on large-scale paired data for training. This paper concentrates on relieving this requirement via developing a coarse-to-fine normal
estimator. Concretely, our method first trains a neat model from a small
dataset to produce coarse face normals that perform as guidance (called exemplars) for the following refinement. A self-
attention mechanism is employed to capture long-range dependencies, thus remedying severe local artifacts left in estimated coarse facial normals. Then, a refinement network is customized for the sake of mapping input face images together with corresponding exemplars to fine-grained high-quality facial normals. Such a logical function split can significantly cut the requirement of massive paired data and computational resource. Extensive experiments and ablation studies are conducted to demonstrate the efficacy of our design and reveal its superiority over state-of-the-art methods in terms of both training expense as well as estimation quality. Our code and models are open-sourced at: https://github.com/AutoHDR/FNR2R.git.
Read the full abstract ▾
All authors Meng Wang, Wenjing Dai, Jiawan Zhang, Xiaojie Guo
Computer Vision
MacVQA: Adaptive Memory Allocation and Global Noise Filtering for Continual Visual Question Answering
Zhifei Li, Yiran Wang, Chenyi Xiong et al. (+6 more)
Visual Question Answering (VQA) requires models to reason over
multimodal information, combining visual and textual data. With the development of continual learning, significant progress has been made in retaining knowledge and adapting to new information in the VQA domain. However, current methods often struggle with balancing knowledge retention, adaptation, and robust feature representation. To address these challenges, we propose a novel framework with adaptive
memory allocation and global noise filtering called MacVQA for visual question answering. MacVQA fuses visual and question information while filtering noise to ensure robust representations, and employs prototype-based
memory allocation to optimize feature quality and
memory usage. These designs enable MacVQA to balance knowledge acquisition, retention, and compositional generalization in continual VQA learning. Experiments on ten continual VQA tasks show that MacVQA outperforms existing baselines, achieving 43.38% average accuracy and 2.32% average forgetting on standard tasks, and 42.53% average accuracy and 3.60% average forgetting on novel composition tasks.
Read the full abstract ▾
All authors Zhifei Li, Yiran Wang, Chenyi Xiong, Yujing Xia, Xiaoju Hou, Yue Zhao, Miao Zhang, Kui Xiao, Bing Yang
Computer Vision
AR-MOT: Autoregressive Multi-object Tracking
Lianjie Jia, Yuhan Wu, Binghao Ran et al. (+3 more)
As multi-object tracking (MOT) tasks continue to evolve toward more general and multi-modal scenarios, the rigid and task-specific architectures of existing MOT methods increasingly hinder their applicability across diverse tasks and limit flexibility in adapting to new tracking formulations. Most approaches rely on fixed output heads and bespoke tracking pipelines, making them difficult to extend to more complex or instruction-driven tasks. To address these limitations, we propose AR-MOT, a novel autoregressive paradigm that formulates MOT as a sequence generation task within a large
language model (LLM) framework. This design enables the model to output structured results through flexible sequence construction, without requiring any task-specific heads. To enhance region-level visual perception, we introduce an Object Tokenizer based on a pretrained detector. To mitigate the misalignment between global and regional features, we propose a Region-Aware
Alignment (RAA) module, and to support long-term tracking, we design a Temporal
Memory Fusion (TMF) module that caches historical object
tokens. AR-MOT offers strong potential for extensibility, as new modalities or instructions can be integrated by simply modifying the output sequence format without altering the model architecture. Extensive experiments on MOT17 and DanceTrack validate the feasibility of our approach, achieving performance comparable to state-of-the-art methods while laying the foundation for more general and flexible MOT systems.
Read the full abstract ▾
All authors Lianjie Jia, Yuhan Wu, Binghao Ran, Yifan Wang, Lijun Wang, Huchuan Lu
Computer Vision
TalkPhoto: A Versatile Training-Free Conversational Assistant for Intelligent Image Editing
Yujie Hu, Zecheng Tang, Xu Jiang et al. (+2 more)
Thanks to the powerful
language comprehension capabilities of Large
Language Models (LLMs), existing instruction-based image editing methods have introduced
Multimodal Large
Language Models (MLLMs) to promote information exchange between instructions and images, ensuring the controllability and flexibility of image editing. However, these frameworks often build a multi-instruction
dataset to train the model to handle multiple editing tasks, which is not only time-consuming and labor-intensive but also fails to achieve satisfactory results. In this paper, we present TalkPhoto, a versatile training-free image editing framework that facilitates precise image manipulation through conversational interaction. We instruct the open-source LLM with a specially designed
prompt template to analyze user needs after receiving instructions and hierarchically invoke existing advanced editing methods, all without additional training. Moreover, we implement a plug-and-play and efficient invocation of image editing methods, allowing complex and unseen editing tasks to be integrated into the current framework, achieving stable and high-quality editing results. Extensive experiments demonstrate that our method not only provides more accurate invocation with fewer
token consumption but also achieves higher editing quality across various image editing tasks.
Read the full abstract ▾
All authors Yujie Hu, Zecheng Tang, Xu Jiang, Weiqi Li, Jian Zhang
Computer Vision
Learning Action Hierarchies via Hybrid Geometric Diffusion
Arjun Ramesh Kaushik, Nalini K. Ratha and Venu Govindaraju
Temporal action
segmentation is a critical task in
video understanding, where the goal is to assign action labels to each frame in a
video. While recent advances leverage iterative refinement-based strategies, they fail to explicitly utilize the hierarchical nature of human actions. In this work, we propose HybridTAS - a novel framework that incorporates a hybrid of Euclidean and
hyperbolic geometries into the denoising process of
diffusion models to exploit the hierarchical structure of actions.
Hyperbolic geometry naturally provides tree-like relationships between
embeddings, enabling us to guide the action label denoising process in a coarse-to-fine manner: higher
diffusion timesteps are influenced by abstract, high-level action categories (root nodes), while lower timesteps are refined using fine-grained action classes (leaf nodes). Extensive experiments on three
benchmark datasets, GTEA, 50Salads, and Breakfast, demonstrate that our method achieves state-of-the-art performance, validating the effectiveness of
hyperbolic-guided denoising for the temporal action
segmentation task.
Read the full abstract ▾
Computer Vision
Nodule-DETR: A Novel DETR Architecture with Frequency-Channel Attention for Ultrasound Thyroid Nodule Detection
Jingjing Wang, Qianglin Liu, Zhuo Xiao et al. (+5 more)
Thyroid cancer is the most common endocrine malignancy, and its incidence is rising globally. While ultrasound is the preferred imaging modality for detecting thyroid nodules, its diagnostic accuracy is often limited by challenges such as low image contrast and blurred nodule boundaries. To address these issues, we propose Nodule-DETR, a novel detection
transformer (DETR) architecture designed for robust thyroid nodule detection in ultrasound images. Nodule-DETR introduces three key innovations: a Multi-Spectral Frequency-domain Channel
Attention (MSFCA) module that leverages frequency analysis to enhance features of low-contrast nodules; a Hierarchical Feature Fusion (HFF) module for efficient multi-scale integration; and Multi-Scale Deformable
Attention (MSDA) to flexibly capture small and irregularly shaped nodules. We conducted extensive experiments on a clinical
dataset of real-world thyroid ultrasound images. The results demonstrate that Nodule-DETR achieves state-of-the-art performance, outperforming the baseline model by a significant margin of 0.149 in mAP@0.5:0.95. The superior accuracy of Nodule-DETR highlights its significant potential for clinical application as an effective tool in computer-aided thyroid diagnosis. The code of work is available at https://github.com/wjj1wjj/Nodule-DETR.
Read the full abstract ▾
All authors Jingjing Wang, Qianglin Liu, Zhuo Xiao, Xinning Yao, Bo Liu, Lu Li, Lijuan Niu, Fugen Zhou
Computer Vision
Forget Less by Learning from Parents Through Hierarchical Relationships
Arjun Ramesh Kaushik, Naresh Kumar Devulapally, Vishnu Suresh Lokhande et al. (+2 more)
Custom
Diffusion Models (CDMs) offer impressive capabilities for personalization in generative modeling, yet they remain vulnerable to catastrophic forgetting when learning new concepts sequentially. Existing approaches primarily focus on minimizing interference between concepts, often neglecting the potential for positive inter-concept interactions. In this work, we present Forget Less by Learning from Parents (FLLP), a novel framework that introduces a parent-child inter-concept learning mechanism in
hyperbolic space to mitigate forgetting. By
embedding concept representations within a Lorentzian
manifold, naturally suited to modeling tree-like hierarchies, we define parent-child relationships in which previously learned concepts serve as guidance for adapting to new ones. Our method not only preserves prior knowledge but also supports continual integration of new concepts. We validate FLLP on three public
datasets and one synthetic
benchmark, showing consistent improvements in both robustness and generalization.
Read the full abstract ▾
All authors Arjun Ramesh Kaushik, Naresh Kumar Devulapally, Vishnu Suresh Lokhande, Nalini K. Ratha, Venu Govindaraju
Computer Vision
Agentic AI in Remote Sensing: Foundations, Taxonomy, and Emerging Systems
Niloufar Alipour Talemi, Julia Boone and Fatemeh Afghah
The paradigm of Earth Observation analysis is shifting from static deep learning models to autonomous agentic AI. Although recent
vision foundation models and
multimodal large
language models advance representation learning, they often lack the sequential planning and active tool orchestration required for complex geospatial workflows. This survey presents the first comprehensive review of agentic AI in remote sensing. We introduce a unified taxonomy distinguishing between single-
agent copilots and multi-
agent systems while analyzing architectural foundations such as planning mechanisms,
retrieval-augmented generation, and
memory structures. Furthermore, we review emerging
benchmarks that move the evaluation from pixel-level accuracy to trajectory-aware
reasoning correctness. By critically examining limitations in grounding, safety, and orchestration, this work outlines a strategic roadmap for the development of robust, autonomous geospatial intelligence.
Read the full abstract ▾
Computer Vision
CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solving
Shuhang Chen, Yunqiu Xu, Junjie Xie et al. (+7 more)
Despite significant progress,
multimodal large
language models continue to struggle with visual mathematical problem solving. Some recent works recognize that visual perception is a bottleneck in visual mathematical
reasoning, but their solutions are limited to improving the extraction and interpretation of visual inputs. Notably, they all ignore the key issue of whether the extracted visual cues are faithfully integrated and properly utilized in subsequent
reasoning. Motivated by this, we present CogFlow, a novel cognitive-inspired three-stage framework that incorporates a knowledge internalization stage, explicitly simulating the hierarchical flow of human
reasoning: perception$\Rightarrow$internalization$\Rightarrow$
reasoning. In line with this hierarchical flow, we holistically enhance all its stages. We devise Synergistic Visual Rewards to boost perception capabilities in parametric and semantic spaces, jointly improving visual information extraction from symbols and diagrams. To guarantee faithful integration of extracted visual cues into subsequent
reasoning, we introduce a Knowledge Internalization Reward model in the internalization stage, bridging perception and
reasoning. Moreover, we design a Visual-Gated Policy Optimization
algorithm to further enforce the
reasoning is grounded with the visual knowledge, preventing models seeking shortcuts that appear coherent but are visually ungrounded
reasoning chains. Moreover, we contribute a new
dataset MathCog for model training, which contains samples with over 120K high-quality perception-
reasoning aligned annotations. Comprehensive experiments and analysis on commonly used visual mathematical
reasoning benchmarks validate the superiority of the proposed CogFlow. Project page: https://shchen233.github.io/cogflow.
Read the full abstract ▾
All authors Shuhang Chen, Yunqiu Xu, Junjie Xie, Aojun Lu, Tao Feng, Zeying Huang, Ning Zhang, Yi Sun, Yi Yang, Hangjie Yuan
Computer Vision
Entity-Guided Multi-Task Learning for Infrared and Visible Image Fusion
Wenyu Shao, Hongbo Liu, Yunchuan Ma et al. (+1 more)
Existing text-driven infrared and visible image fusion approaches often rely on textual information at the sentence level, which can lead to semantic noise from redundant text and fail to fully exploit the deeper semantic value of textual information. To address these issues, we propose a novel fusion approach named Entity-Guided Multi-Task learning for infrared and visible image fusion (EGMT). Our approach includes three key innovative components: (i) A principled method is proposed to extract entity-level textual information from image captions generated by large
vision-
language models, eliminating semantic noise from raw text while preserving critical semantic information; (ii) A parallel multi-task learning architecture is constructed, which integrates image fusion with a multi-label classification task. By using entities as pseudo-labels, the multi-label classification task provides semantic supervision, enabling the model to achieve a deeper understanding of image content and significantly improving the quality and semantic density of the fused image; (iii) An entity-guided cross-modal interactive module is also developed to facilitate the fine-grained interaction between visual and entity-level textual features, which enhances feature representation by capturing cross-modal dependencies at both inter-visual and visual-entity levels. To promote the wide application of the entity-guided image fusion framework, we release the entity-annotated version of four public
datasets (i.e., TNO, RoadScene, M3FD, and MSRS). Extensive experiments demonstrate that EGMT achieves superior performance in preserving salient targets, texture details, and semantic consistency, compared to the state-of-the-art methods. The code and
dataset will be publicly available at https://github.com/wyshao-01/EGMT.
Read the full abstract ▾
All authors Wenyu Shao, Hongbo Liu, Yunchuan Ma, Ruili Wang
Computer Vision
RRNet: Configurable Real-Time Video Enhancement with Arbitrary Local Lighting Variations
Wenlong Yang, Canran Jin, Weihang Yuan et al. (+2 more)
With the growing demand for real-time
video enhancement in live applications, existing methods often struggle to balance speed and effective exposure control, particularly under uneven lighting. We introduce RRNet (Rendering Relighting Network), a lightweight and configurable framework that achieves a state-of-the-art tradeoff between visual quality and efficiency. By estimating parameters for a minimal set of virtual light sources, RRNet enables localized relighting through a depth-aware rendering module without requiring pixel-aligned training data. This object-aware formulation preserves facial identity and supports real-time, high-resolution performance using a streamlined encoder and lightweight prediction head. To facilitate training, we propose a generative AI-based
dataset creation pipeline that synthesizes diverse lighting conditions at low cost. With its interpretable lighting control and efficient architecture, RRNet is well suited for practical applications such as
video conferencing, AR-based portrait enhancement, and mobile photography. Experiments show that RRNet consistently achieves the best overall result in low-light enhancement, localized illumination adjustment, and glare removal.
Read the full abstract ▾
All authors Wenlong Yang, Canran Jin, Weihang Yuan, Chao Wang, Lifeng Sun
Computer Vision
GCR: Geometry-Consistent Routing for Task-Agnostic Continual Anomaly Detection
Joongwon Chae, Lihui Luo, Yang Liu et al. (+8 more)
Feature-based anomaly detection is widely adopted in industrial inspection due to the strong representational power of large pre-trained
vision encoders. While most existing methods focus on improving within-category anomaly scoring, practical deployments increasingly require task-agnostic operation under continual category expansion, where the category identity is unknown at test time. In this setting, overall performance is often dominated by expert selection, namely routing an input to an appropriate normality model before any head-specific scoring is applied. However, routing rules that compare head-specific anomaly scores across independently constructed heads are unreliable in practice, as score distributions can differ substantially across categories in scale and tail behavior. We propose GCR, a lightweight mixture-of-experts framework for stabilizing task-agnostic continual anomaly detection through geometry-consistent routing. GCR routes each test image directly in a shared frozen patch-
embedding space by minimizing an accumulated nearest-prototype distance to category-specific prototype banks, and then computes anomaly maps only within the routed expert using a standard prototype-based scoring rule. By separating cross-head decision making from within-head anomaly scoring, GCR avoids cross-head score comparability issues without requiring end-to-end representation learning. Experiments on MVTec AD and VisA show that geometry-consistent routing substantially improves routing stability and mitigates continual performance collapse, achieving near-zero forgetting while maintaining competitive detection and localization performance. These results indicate that many failures previously attributed to representation forgetting can instead be explained by decision-rule instability in cross-head routing. Code is available at https://github.com/jw-chae/GCR
Read the full abstract ▾
All authors Joongwon Chae, Lihui Luo, Yang Liu, Runming Wang, Dongmei Yu, Zeming Liang, Xi Yuan, Dayan Zhang, Zhenglin Chen, Peiwu Qin, Ilmoon Chae
Computer Vision
ESGaussianFace: Emotional and Stylized Audio-Driven Facial Animation via 3D Gaussian Splatting
Chuhang Ma, Shuai Tan, Ye Pan et al. (+2 more)
Most current audio-driven facial animation research primarily focuses on generating
videos with neutral
emotions. While some studies have addressed the generation of facial
videos driven by emotional audio, efficiently generating high-quality talking head
videos that integrate both emotional expressions and style features remains a significant challenge. In this paper, we propose ESGaussianFace, an innovative framework for emotional and stylized audio-driven facial animation. Our approach leverages 3D Gaussian Splatting to reconstruct 3D scenes and render
videos, ensuring efficient generation of 3D consistent results. We propose an
emotion-audio-guided spatial
attention method that effectively integrates
emotion features with audio content features. Through
emotion-guided
attention, the model is able to reconstruct facial details across different emotional states more accurately. To achieve emotional and stylized deformations of the 3D Gaussian points through
emotion and style features, we introduce two 3D Gaussian deformation predictors. Futhermore, we propose a multi-stage training strategy, enabling the step-by-step learning of the character's lip movements, emotional variations, and style features. Our generated results exhibit high efficiency, high quality, and 3D consistency. Extensive experimental results demonstrate that our method outperforms existing state-of-the-art techniques in terms of lip movement accuracy, expression variation, and style feature expressiveness.
Read the full abstract ▾
All authors Chuhang Ma, Shuai Tan, Ye Pan, Jiaolong Yang, Xin Tong
Computer Vision
RSwinV2-MD: An Enhanced Residual SwinV2 Transformer for Monkeypox Detection from Skin Images
Rashid Iqbal and Saddam Hussain Khan
In this paper, a deep learning approach for Mpox diagnosis named Customized Residual SwinTransformerV2 (RSwinV2) has been proposed, trying to enhance the capability of lesion classification by employing the RSwinV2 tool-assisted
vision approach. In the RSwinV2 method, a hierarchical structure of the
transformer has been customized based on the input dimensionality,
embedding structure, and output targeted by the method. In this RSwinV2 approach, the input image has been split into non-overlapping patches and processed using shifted windows and
attention in these patches. This process has helped the method link all the windows efficiently by avoiding the locality issues of non-overlapping regions in
attention, while being computationally efficient. RSwinV2 has further developed based on SwinTransformer and has included patch and position
embeddings to take advantage of the
transformer global-linking capability by employing multi-head
attention in these
embeddings. Furthermore, RSwinV2 has developed and incorporated the Inverse Residual Block (IRB) into this method, which utilizes convolutional skip connections with these inclusive designs to address the vanishing gradient issues during processing. RSwinV2 inclusion of IRB has therefore facilitated this method to link global patterns as well as local patterns; hence, its integrity has helped improve lesion classification capability by minimizing variability of Mpox and increasing differences of Mpox, chickenpox, measles, and cowpox. In testing SwinV2, its accuracy of 96.51 and an F1score of 96.13 have been achieved on the Kaggle public
dataset, which has outperformed standard CNN models and SwinTransformers; the RSwinV2 vector has thus proved its validity as a computer-assisted tool for Mpox lesion observation interpretation.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
DisCo-FLoc: Semantic-Free Floorplan Localization via $SE(2)$-Aware Contrastive Disambiguation
Ping Zhong, Shiyong Meng, Bolei Chen et al. (+3 more)
Visual Floorplan Localization (FLoc) struggles with severe structural aliasing caused by repetitive minimalist layouts. This occurs because physically distant poses share highly similar visual-
geometric features, which degrades spatial separability and angular discriminability. While existing methods attempt to mitigate these ambiguities by relying on costly semantic annotations, the resulting performance gains remain inherently limited. To address the above issues, we propose DisCo-FLoc, a semantic-free method for visual-
geometric Contrastive Disambiguation. First, we introduce a depth-aware Ray Regression Predictor (RRP) that serves as a dense-to-ray
geometric projector. By explicitly suppressing visual clutter along the vertical dimension, RRP projects
monocular RGB images into 2D ray primitives, which are matched with floorplans to produce geometry-aware FLoc candidates. Second, to resolve the remaining ambiguity among these candidates, we propose a spatially perturbed contrastive objective to align RGB images with local floorplan structures and formulate a visual-
geometric compatibility function. In particular, we meticulously construct positive and negative samples at both positional and directional levels through $SE(2)$ pose perturbations for contrastive learning, effectively achieving pose smoothness, spatial separability, and angular discriminability. The compatibility function enables DisCo-FLoc to disambiguate FLoc by using richer visual context beyond pure
geometric layouts, without requiring any semantic annotations. Extensive experiments on two challenging visual FLoc
benchmarks demonstrate that DisCo-FLoc significantly outperforms state-of-the-art semantic-based methods, especially narrowing the performance gap between positional and directional FLoc accuracy.
Read the full abstract ▾
All authors Ping Zhong, Shiyong Meng, Bolei Chen, Tao Zou, Chaoxu Mu, Jianxin Wang
Computer Vision
Robust Egocentric Visual Attention Prediction Through Language-guided Scene Context-aware Learning
Sungjune Park, Hongda Mao, Qingshuang Chen et al. (+2 more)
As the demand for analyzing
egocentric videos grows,
egocentric visual
attention prediction, anticipating where a camera wearer will attend, has garnered increasing
attention. However, it remains challenging due to the inherent complexity and ambiguity of dynamic
egocentric scenes. Motivated by evidence that scene contextual information plays a crucial role in modulating human
attention, in this paper, we present a
language-guided scene context-aware learning framework for robust
egocentric visual
attention prediction. We first design a context perceiver which is guided to summarize the
egocentric video based on a
language-based scene description, generating context-aware
video representations. We then introduce two training objectives that: 1) encourage the framework to focus on the target point-of-interest regions and 2) suppress distractions from irrelevant regions which are less likely to attract first-person
attention. Extensive experiments on Ego4D and Aria Everyday Activities (AEA)
datasets demonstrate the effectiveness of our approach, achieving state-of-the-art performance and enhanced robustness across diverse, dynamic
egocentric scenarios.
Read the full abstract ▾
All authors Sungjune Park, Hongda Mao, Qingshuang Chen, Yong Man Ro, Yelin Kim
Computer Vision
Adaptive Hybrid Optimizer based Framework for Lumpy Skin Disease Identification
Ubaidullah, Muhammad Abid Hussain, Mohsin Raza Jafri et al. (+4 more)
Lumpy Skin Disease (LSD) is a contagious viral infection that significantly deteriorates livestock health, thereby posing a serious threat to the global economy and food security. Owing to its rapid spread characteristics, early and precise identification is crucial to prevent outbreaks and ensure timely intervention. In this paper, we propose a hybrid deep learning-based approach called LUMPNet for the early detection of LSD. LUMPNet utilizes image data to detect and classify skin nodules – the primary indicator of LSD. To this end, LUMPNet uses YOLOv11, EfficientNet-based CNN classifier with compound scaling, and a novel adaptive hybrid optimizer. More precisely, LUMPNet detects and localizes LSD skin nodules and lesions on cattle images. It exploits EfficientNet to classify the localized cattle images into LSD-affected or healthy categories. To stabilize and accelerate the training of YOLOv11 and EfficientNet hybrid model, a novel adaptive hybrid optimizer is proposed and utilized. We evaluate LUMPNet at various stages of LSD using a publicly available
dataset. Results indicate that the proposed scheme achieves 99% LSD detection training accuracy, and outperforms existing schemes. The model also achieves validation accuracy of 98%. Moreover, for further evaluation, we conduct a case study using an optimized EfficientNet-B0 model trained with the AdamW optimizer, and compare its performance with LUMPNet. The results show that LUMPNet achieves superior performance.
Read the full abstract ▾
All authors Ubaidullah, Muhammad Abid Hussain, Mohsin Raza Jafri, Rozi Khan, Moid Sandhu, Abd Ullah Khan, Hyundong Shin
Computer Vision
V-CORE: Temporally Consistent Video Understanding for Video-LLM
Zhengjian Kang, Qi Chen, Rui Liu et al. (+4 more)
Recent
Video Large
Language Models (
Video-LLMs) have shown strong
multimodal reasoning capabilities, yet remain challenged by
video understanding tasks that require consistent temporal ordering and causal coherence. Many parameter-efficient
Video-LLMs rely on unconstrained bidirectional projectors to model inter-frame interactions, which can blur temporal ordering by allowing later frames to influence earlier representations, without explicit architectural mechanisms to respect the directional nature of
video reasoning. To address this limitation, we propose V-CORE, a parameter-efficient framework that introduces explicit temporal ordering constraints for
video understanding. V-CORE consists of two key components: (1) Learnable Spatial Aggregation (LSA), which adaptively selects salient spatial
tokens to reduce redundancy, and (2) a Causality-Aware Temporal Projector (CATP), which enforces structured unidirectional information flow via block-causal
attention and a terminal dynamic summary
token acting as a causal sink. This design preserves intra-frame spatial interactions while ensuring that temporal information is aggregated in a strictly ordered manner. With 4-bit QLoRA and a frozen LLM backbone, V-CORE can be trained efficiently on a single consumer GPU. Experiments show that V-CORE achieves strong performance on the challenging NExT-QA
benchmark, reaching 61.2% accuracy, and remains competitive across MSVD-QA, MSRVTT-QA, and TGIF-QA, with gains concentrated in temporal and causal
reasoning subcategories (+3.5% and +5.2% respectively), directly validating the importance of explicit temporal ordering constraints.
Read the full abstract ▾
All authors Zhengjian Kang, Qi Chen, Rui Liu, Kangtong Mo, Xingyu Zhang, Xiaoyu Deng, Ye Zhang
Computer Vision
VerLM: Explaining Face Verification Using Natural Language
Syed Abdul Hannan, Hazim Bukhari, Thomas Cantalapiedra et al. (+4 more)
Face verification systems have seen substantial advancements; however, they often lack transparency in their decision-making processes. In this paper, we introduce an innovative
Vision-
Language Model (VLM) for Face Verification, which not only accurately determines if two face images depict the same individual but also explicitly explains the rationale behind its decisions. Our model is uniquely trained using two complementary explanation styles: (1) concise explanations that summarize the key factors influencing its decision, and (2) comprehensive explanations detailing the specific differences observed between the images. We adapt and enhance a state-of-the-art modeling approach originally designed for audio-based differentiation to suit visual inputs effectively. This cross-modal transfer significantly improves our model's accuracy and interpretability. The proposed VLM integrates sophisticated feature extraction techniques with advanced
reasoning capabilities, enabling clear articulation of its verification process. Our approach demonstrates superior performance, surpassing baseline methods and existing models. These findings highlight the immense potential of
vision language models in face verification set up, contributing to more transparent, reliable, and explainable face verification systems.
Read the full abstract ▾
All authors Syed Abdul Hannan, Hazim Bukhari, Thomas Cantalapiedra, Eman Ansar, Massa Baali, Rita Singh, Bhiksha Raj