Computer Vision · Cross-listed from cs.RO
ReWorld: Multi-Dimensional Reward Modeling for Embodied Video Generation
Baorui Peng, Wenyao Zhang, Liang Xu et al. (+7 more)
Current
embodied video generation models primarily emphasize visual generative quality while overlooking physical fidelity, dynamic consistency, and task logic, especially for contact-rich manipulation tasks, which limits their applicability to downstream tasks. In this paper, we introduce ReWorld, a framework aimed to employ reinforcement learning to align the
video generation with physical realism, task completion capability, embodiment plausibility and visual quality. Specifically, we first construct a large-scale (~235K)
video preference
dataset and employ it to train a hierarchical reward model designed to capture multi-dimensional reward consistent with human preferences. We further propose a practical
alignment algorithm that post-trains flow-based
video generators using this reward through a computationally efficient PPO-style
algorithm. Comprehensive experiments and theoretical analysis demonstrate that ReWorld significantly improves the physical fidelity, logical coherence, embodiment and visual quality of generated rollouts, outperforming previous methods.
Read the full abstract ▾
All authors Baorui Peng, Wenyao Zhang, Liang Xu, Bohan Li, Hu Zhu, Zekun Qi, Hongsi Liu, Jiazhao Zhang, Wenjun Zeng, Xin Jin
Computer Vision
HOT-POT: Optimal Transport for Sparse Stereo Matching
Antonin Clerc, Michael Quellmalz, Moritz Piening et al. (+3 more)
Stereo
vision between images faces a range of challenges, including occlusions, motion, and camera distortions, across applications in autonomous driving, robotics, and face analysis. Due to parameter sensitivity, further complications arise for stereo matching with sparse features, such as facial landmarks. To overcome this ill-posedness and enable unsupervised sparse matching, we consider line constraints of the camera geometry from an optimal transport (OT) viewpoint. Formulating camera-projected points as (half)lines, we propose the use of the classical epipolar distance as well as a 3D ray distance to quantify matching quality. Employing these distances as a cost function of a (partial) OT problem, we arrive at efficiently solvable assignment problems. Moreover, we extend our approach to unsupervised object matching by formulating it as a hierarchical OT problem. The resulting
algorithms allow for efficient feature and object matching, as demonstrated in our numerical experiments. Here, we focus on applications in facial analysis, where we aim to match distinct landmarking conventions.
Read the full abstract ▾
All authors Antonin Clerc, Michael Quellmalz, Moritz Piening, Philipp Flotho, Gregor Kornhardt, Gabriele Steidl
Computer Vision
Weaknesses of Facial Emotion Recognition Systems
Aleksandra Jamróz, Patrycja Wysocka and Piotr Garbat
Emotion detection from faces is one of the machine learning problems needed for human-computer interaction. The variety of methods used is enormous, which motivated an in-depth review of articles and scientific studies. Three of the most interesting and best solutions are selected, followed by the selection of three
datasets that stood out for the diversity and number of images in them. The selected neural networks are trained, and then a series of experiments are performed to compare their performance, including testing on different
datasets than a model was trained on. This reveals weaknesses in existing solutions, including differences between
datasets, unequal levels of difficulty in recognizing certain
emotions and the challenges in differentiating between closely related
emotions.
Read the full abstract ▾
Computer Vision
Class-Partitioned VQ-VAE and Latent Flow Matching for Point Cloud Scene Generation
Dasith de Silva Edirimuni and Ajmal Saeed Mian
Most 3D scene generation methods are limited to only generating object bounding box parameters while newer
diffusion methods also generate class labels and latent features. Using object size or latent feature, they then retrieve objects from a predefined database. For complex scenes of varied, multi-categorical objects,
diffusion-based latents cannot be effectively decoded by current autoencoders into the correct point cloud objects which agree with target classes. We introduce a Class-Partitioned Vector Quantized Variational Autoencoder (CPVQ-VAE) that is trained to effectively decode object latent features, by employing a pioneering $\textit{class-partitioned codebook}$ where codevectors are labeled by class. To address the problem of $\textit{codebook collapse}$, we propose a $\textit{class-aware}$ running average update which reinitializes dead codevectors within each partition. During inference, object features and class labels, both generated by a Latent-space Flow Matching Model (LFMM) designed specifically for scene generation, are consumed by the CPVQ-VAE. The CPVQ-VAE's class-aware inverse look-up then maps generated latents to codebook entries that are decoded to class-specific point cloud shapes. Thereby, we achieve pure point cloud generation without relying on an external objects database for
retrieval. Extensive experiments reveal that our method reliably recovers plausible point cloud scenes, with up to 70.4% and 72.3% reduction in Chamfer and Point2Mesh errors on complex living room scenes.
Read the full abstract ▾
Computer Vision
A Hierarchical Benchmark of Foundation Models for Dermatology
Furkan Yuceyalcin, Abdurrahim Yilmaz and Burak Temelkuran
Foundation models have transformed medical image analysis by providing robust feature representations that reduce the need for large-scale task-specific training. However, current
benchmarks in dermatology often reduce the complex diagnostic taxonomy to flat, binary classification tasks, such as distinguishing melanoma from benign nevi. This oversimplification obscures a model's ability to perform fine-grained differential diagnoses, which is critical for clinical workflow integration. This study evaluates the utility of
embeddings derived from ten foundation models, spanning general computer
vision, general medical imaging, and dermatology-specific domains, for hierarchical skin lesion classification. Using the DERM12345
dataset, which comprises 40 lesion subclasses, we calculated frozen
embeddings and trained lightweight adapter models using a five-fold cross-validation. We introduce a hierarchical evaluation framework that assesses performance across four levels of clinical granularity: 40 Subclasses, 15 Main Classes, 2 and 4 Superclasses, and Binary Malignancy. Our results reveal a "granularity gap" in model capabilities: MedImageInsights achieved the strongest overall performance (97.52% weighted F1-Score on Binary Malignancy detection) but declined to 65.50% on fine-grained 40-class subtype classification. Conversely, MedSigLip (69.79%) and dermatology-specific models (Derm Foundation and MONET) excelled at fine-grained 40-class subtype discrimination while achieving lower overall performance than MedImageInsights on broader classification tasks. Our findings suggest that while general medical foundation models are highly effective for high-level screening, specialized modeling strategies are necessary for the granular distinctions required in diagnostic support systems.
Read the full abstract ▾
Computer Vision
Utilizing the Score of Data Distribution for Hyperspectral Anomaly Detection
Jiahui Sheng, Yidan Shi, Shu Xiang et al. (+2 more)
Hyperspectral images (HSIs) are a type of image that contains abundant spectral information. As a type of real-world data, the high-dimensional spectra in hyperspectral images are actually determined by only a few factors, such as chemical composition and illumination. Thus, spectra in hyperspectral images are highly likely to satisfy the
manifold hypothesis. Based on the hyperspectral
manifold hypothesis, we propose a novel hyperspectral anomaly detection method (named ScoreAD) that leverages the time-dependent gradient field of the data distribution (i.e., the score), as learned by a score-based generative model (SGM). Our method first trains the SGM on the entire set of spectra from the hyperspectral image. At test time, each spectrum is passed through a perturbation kernel, and the resulting perturbed spectrum is fed into the trained SGM to obtain the estimated score. The
manifold hypothesis of HSIs posits that background spectra reside on one or more low-dimensional
manifolds. Conversely, anomalous spectra, owing to their unique spectral signatures, are considered outliers that do not conform to the background
manifold. Based on this fundamental discrepancy in their
manifold distributions, we leverage a generative SGM to achieve hyperspectral anomaly detection. Experiments on the four hyperspectral
datasets demonstrate the effectiveness of the proposed method. The code is available at https://github.com/jiahuisheng/ScoreAD.
Read the full abstract ▾
All authors Jiahui Sheng, Yidan Shi, Shu Xiang, Xiaorun Li, Shuhan Chen
Computer Vision
CD-TWINSAFE: A ROS-enabled Digital Twin for Scene Understanding and Safety Emerging V2I Technology
Amro Khaled, Farah Khaled, Omar Riad et al. (+1 more)
In this paper, the CD-TWINSAFE is introduced, a V2I-based digital twin for Autonomous Vehicles. The proposed architecture is composed of two stacks running simultaneously, an on-board driving stack that includes a stereo camera for scene understanding, and a digital twin stack that runs an Unreal Engine 5 replica of the scene viewed by the camera as well as returning safety alerts to the cockpit. The on-board stack is implemented on the vehicle side including 2 main autonomous modules; localization and perception. The position and orientation of the ego vehicle are obtained using on-board sensors. Furthermore, the perception module is responsible for processing 20-fps images from stereo camera and understands the scene through two complementary pipelines. The pipeline are working on object detection and feature extraction including object velocity, yaw and the safety metrics time-to-collision and time-headway. The collected data form the driving stack are sent to the infrastructure side through the ROS-enabled architecture in the form of custom ROS2 messages and sent over UDP links that ride a 4G modem for V2I communication. The environment is monitored via the digital twin through the shared messages which update the information of the spawned ego vehicle and detected objects based on the real-time localization and perception data. Several tests with different driving scenarios to confirm the validity and real-time response of the proposed architecture.
Read the full abstract ▾
All authors Amro Khaled, Farah Khaled, Omar Riad, Catherine M. Elias
Computer Vision
DepthCropSeg++: Scaling a Crop Segmentation Foundation Model With Depth-Labeled Data
Jiafei Zhang, Songliang Cao, Binghui Xu et al. (+6 more)
DepthCropSeg++: a foundation model for crop
segmentation, capable of segmenting different crop species under open in-field environment. Crop
segmentation is a fundamental task for modern agriculture, which closely relates to many downstream tasks such as plant phenotyping, density estimation, and weed control. In the era of foundation models, a number of generic large
language and
vision models have been developed. These models have demonstrated remarkable real world generalization due to significant model capacity and largescale
datasets. However, current crop
segmentation models mostly learn from limited data due to expensive pixel-level labelling cost, often performing well only under specific crop types or controlled environment. In this work, we follow the vein of our previous work DepthCropSeg, an almost unsupervised approach to crop
segmentation, to scale up a cross-species and crossscene crop
segmentation dataset, with 28,406 images across 30+ species and 15 environmental conditions. We also build upon a state-of-the-art semantic
segmentation architecture ViT-Adapter architecture, enhance it with dynamic upsampling for improved detail awareness, and train the model with a two-stage selftraining pipeline. To systematically validate model performance, we conduct comprehensive experiments to justify the effectiveness and generalization capabilities across multiple crop
datasets. Results demonstrate that DepthCropSeg++ achieves 93.11% mIoU on a comprehensive testing set, outperforming both supervised baselines and general-purpose
vision foundation models like
Segmentation Anything Model (SAM) by significant margins (+0.36% and +48.57% respectively). The model particularly excels in challenging scenarios including night-time environment (86.90% mIoU), high-density canopies (90.09% mIoU), and unseen crop varieties (90.09% mIoU), indicating a new state of the art for crop
segmentation.
Read the full abstract ▾
All authors Jiafei Zhang, Songliang Cao, Binghui Xu, Yanan Li, Weiwei Jia, Tingting Wu, Hao Lu, Weijuan Hu, Zhiguo Han
Computer Vision
From Prompts to Pavement: LMMs-based Agentic Behavior-Tree Generation Framework for Autonomous Vehicles
Omar Y. Goba, Ahmed Y. Gado, Catherine M. Elias et al. (+1 more)
Autonomous vehicles (AVs) require adaptive behavior planners to navigate unpredictable, real-world environments safely. Traditional behavior trees (BTs) offer structured decision logic but are inherently static and demand labor-intensive manual tuning, limiting their applicability at SAE Level 5 autonomy. This paper presents an agentic framework that leverages large
language models (LLMs) and multi-modal
vision models (LVMs) to generate and adapt BTs on the fly. A specialized Descriptor
agent applies chain-of-symbols prompting to assess scene criticality, a Planner
agent constructs high-level sub-goals via in-context learning, and a Generator
agent synthesizes executable BT sub-trees in XML format. Integrated into a CARLA+Nav2 simulation, our system triggers only upon baseline BT failure, demonstrating successful
navigation around unexpected obstacles (e.g., street blockage) with no human intervention. Compared to a static BT baseline, this approach is a proof-of-concept that extends to diverse driving scenarios.
Read the full abstract ▾
All authors Omar Y. Goba, Ahmed Y. Gado, Catherine M. Elias, Ahmed Hussein
Computer Vision
SimpleMatch: A Simple and Strong Baseline for Semantic Correspondence
Hailing Jin and Huiying Li
Recent advances in semantic correspondence have been largely driven by the use of pre-trained large-scale models. However, a limitation of these approaches is their dependence on high-resolution input images to achieve optimal performance, which results in considerable computational overhead. In this work, we address a fundamental limitation in current methods: the irreversible fusion of adjacent keypoint features caused by deep downsampling operations. This issue is triggered when semantically distinct keypoints fall within the same downsampled receptive field (e.g., 16x16 patches). To address this issue, we present SimpleMatch, a simple yet effective framework for semantic correspondence that delivers strong performance even at low resolutions. We propose a lightweight upsample decoder that progressively recovers spatial detail by upsampling deep features to 1/4 resolution, and a multi-scale supervised loss that ensures the upsampled features retain discriminative features across different spatial scales. In addition, we introduce sparse matching and window-based localization to optimize training
memory usage and reduce it by 51%. At a resolution of 252x252 (3.3x smaller than current SOTA methods), SimpleMatch achieves superior performance with 84.1% PCK@0.1 on the SPair-71k
benchmark. We believe this framework provides a practical and efficient baseline for future research in semantic correspondence. Code is available at: https://github.com/hailong23-jin/SimpleMatch.
Read the full abstract ▾
Computer Vision
Peizhou Huang, Zixuan Zhong, Zhongwei Wan et al. (+12 more)
Deep Research
Agents (DRAs) generate citation-rich reports via multi-step search and synthesis, yet existing
benchmarks mainly target text-only settings or short-form
multimodal QA, missing end-to-end
multimodal evidence use. We introduce MMDeepResearch-Bench (MMDR-Bench), a
benchmark of 140 expert-crafted tasks across 21 domains, where each task provides an image-text bundle to evaluate
multimodal understanding and citation-grounded report generation. Compared to prior setups, MMDR-Bench emphasizes report-style synthesis with explicit evidence use, where models must connect visual artifacts to sourced claims and maintain consistency across narrative, citations, and visual references. We further propose a unified, interpretable evaluation pipeline: Formula-LLM Adaptive Evaluation (FLAE) for report quality, Trustworthy
Retrieval-Aligned Citation Evaluation (TRACE) for citation-grounded evidence
alignment, and
Multimodal Support-Aligned Integrity Check (MOSAIC) for text-visual integrity, each producing fine-grained signals that support error diagnosis beyond a single overall score. Experiments across 25 state-of-the-art models reveal systematic trade-offs between generation quality, citation discipline, and
multimodal grounding, highlighting that strong prose alone does not guarantee faithful evidence use and that
multimodal integrity remains a key bottleneck for deep research
agents.
Read the full abstract ▾
All authors Peizhou Huang, Zixuan Zhong, Zhongwei Wan, Donghao Zhou, Samiul Alam, Xin Wang, Zexin Li, Zhihao Dou, Li Zhu, Jing Xiong, Chaofan Tao, Yan Xu, Dimitrios Dimitriadis, Tuo Zhang, Mi Zhang
Computer Vision
Turbo-GoDec: Exploiting the Cluster Sparsity Prior for Hyperspectral Anomaly Detection
Jiahui Sheng, Xiaorun Li and Shuhan Chen
As a key task in hyperspectral image processing, hyperspectral anomaly detection has garnered significant
attention and undergone extensive research. Existing methods primarily relt on two prior assumption: low-rank background and sparse anomaly, along with additional spatial assumptions of the background. However, most methods only utilize the sparsity prior assumption for anomalies and rarely expand on this hypothesis. From observations of hyperspectral images, we find that anomalous pixels exhibit certain spatial distribution characteristics: they often manifest as small, clustered groups in space, which we refer to as cluster sparsity of anomalies. Then, we combined the cluster sparsity prior with the classical GoDec
algorithm, incorporating the cluster sparsity prior into the S-step of GoDec. This resulted in a new hyperspectral anomaly detection method, which we called Turbo-GoDec. In this approach, we modeled the cluster sparsity prior of anomalies using a Markov random field and computed the marginal probabilities of anomalies through message passing on a factor graph. Locations with high anomalous probabilities were treated as the sparse component in the Turbo-GoDec. Experiments are conducted on three real hyperspectral image (HSI)
datasets which demonstrate the superior performance of the proposed Turbo-GoDec method in detecting small-size anomalies comparing with the vanilla GoDec (LSMAD) and state-of-the-art anomaly detection methods. The code is available at https://github.com/jiahuisheng/Turbo-GoDec.
Read the full abstract ▾
Computer Vision
FlowIID: Single-Step Intrinsic Image Decomposition via Latent Flow Matching
Mithlesh Singla, Seema Kumari and Shanmuganathan Raman
Intrinsic Image Decomposition (IID) separates an image into albedo and shading components. It is a core step in many real-world applications, such as relighting and material editing. Existing IID models achieve good results, but often use a large number of parameters. This makes them costly to combine with other models in real-world settings. To address this problem, we propose a flow matching-based solution. For this, we design a novel architecture, FlowIID, based on latent flow matching. FlowIID combines a VAE-guided latent space with a flow matching module, enabling a stable decomposition of albedo and shading. FlowIID is not only parameter-efficient, but also produces results in a single inference step. Despite its compact design, FlowIID delivers competitive and superior results compared to existing models across various
benchmarks. This makes it well-suited for deployment in resource-constrained and real-time
vision applications.
Read the full abstract ▾
Computer Vision
EmoKGEdit: Training-free Affective Injection via Visual Cue Transformation
Jing Zhang and Bingjie Fan
Existing image
emotion editing methods struggle to disentangle emotional cues from latent content representations, often yielding weak emotional expression and distorted visual structures. To bridge this gap, we propose EmoKGEdit, a novel training-free framework for precise and structure-preserving image
emotion editing. Specifically, we construct a
Multimodal Sentiment Association Knowledge Graph (MSA-KG) to disentangle the intricate relationships among objects, scenes, attributes, visual clues and
emotion. MSA-KG explicitly encode the causal chain among object-attribute-
emotion, and as external knowledge to support chain of thought
reasoning, guiding the
multimodal large model to infer plausible
emotion-related visual cues and generate coherent instructions. In addition, based on MSA-KG, we design a disentangled structure-
emotion editing module that explicitly separates emotional attributes from layout features within the latent space, which ensures that the target
emotion is effectively injected while strictly maintaining visual spatial coherence. Extensive experiments demonstrate that EmoKGEdit achieves excellent performance in both
emotion fidelity and content preservation, and outperforms the state-of-the-art methods.
Read the full abstract ▾
Computer Vision
Multi-Sensor Matching with HyperNetworks
Eli Passov, Nathan S. Netanyahu and Yosi Keller
Hypernetworks are models that generate or modulate the weights of another network. They provide a flexible mechanism for injecting context and task conditioning and have proven broadly useful across diverse applications without significant increases in model size. We leverage hypernetworks to improve
multimodal patch matching by introducing a lightweight descriptor-learning architecture that augments a Siamese CNN with (i) hypernetwork modules that compute adaptive, per-channel scaling and shifting and (ii) conditional instance normalization that provides modality-specific adaptation (e.g., visible vs. infrared, VIS-IR) in shallow layers. This combination preserves the efficiency of descriptor-based methods during inference while increasing robustness to appearance shifts. Trained with a triplet loss and hard-negative mining, our approach achieves state-of-the-art results on VIS-NIR and other VIS-IR
benchmarks and matches or surpasses prior methods on additional
datasets, despite their higher inference cost. To spur progress on domain shift, we also release GAP-VIR, a cross-platform (ground/aerial) VIS-IR patch
dataset with 500K pairs, enabling rigorous evaluation of cross-domain generalization and adaptation.
Read the full abstract ▾
Computer Vision
GazeFormer-MoE: Context-Aware Gaze Estimation via CLIP and MoE Transformer
Xinyuan Zhao, Xianrui Chen and Ahmad Chaddad
We present a semantics modulated, multi scale
Transformer for 3D gaze estimation. Our model conditions CLIP global features with learnable prototype banks (illumination, head pose, background, direction), fuses these prototype-enriched global vectors with CLIP patch
tokens and high-resolution CNN
tokens in a unified
attention space, and replaces several FFN blocks with routed/shared Mixture of Experts to increase conditional capacity. Evaluated on MPIIFaceGaze, EYEDIAP, Gaze360 and ETH-XGaze, our model achieves new state of the art angular errors of 2.49{\deg}, 3.22{\deg}, 10.16{\deg}, and 1.44{\deg}, demonstrating up to a 64% relative improvement over previously reported results. ablations attribute gains to prototype conditioning, cross scale fusion, MoE and hyperparameter. Our code is publicly available at https://github. com/AIPMLab/Gazeformer.
Read the full abstract ▾
Computer Vision
S^2F-Net:A Robust Spatial-Spectral Fusion Framework for Cross-Model AIGC Detection
Xiangyu Hu, Yicheng Hong, Hongchuang Zheng et al. (+2 more)
The rapid development of generative models has imposed an urgent demand for detection schemes with strong generalization capabilities. However, existing detection methods generally suffer from overfitting to specific source models, leading to significant performance degradation when confronted with unseen generative architectures. To address these challenges, this paper proposes a cross-model detection framework called S 2 F-Net, whose core lies in exploring and leveraging the inherent spectral discrepancies between real and synthetic textures. Considering that upsampling operations leave unique and distinguishable frequency fingerprints in both texture-poor and texture-rich regions, we focus our research on the detection of frequency-domain artifacts, aiming to fundamentally improve the generalization performance of the model. Specifically, we introduce a learnable frequency
attention module that adaptively weights and enhances discriminative frequency bands by synergizing spatial texture analysis and spectral dependencies.On the AIGCDetectBenchmark, which includes 17 categories of generative models, S 2 F-Net achieves a detection accuracy of 90.49%, significantly outperforming various existing baseline methods in cross-domain detection scenarios.
Read the full abstract ▾
All authors Xiangyu Hu, Yicheng Hong, Hongchuang Zheng, Wenjun Zeng, Bingyao Liu
Computer Vision
CurConMix+: A Unified Spatio-Temporal Framework for Hierarchical Surgical Workflow Understanding
Yongjun Jeon, Jongmin Shin, Kanggil Park et al. (+8 more)
Surgical action triplet recognition aims to understand fine-grained
surgical behaviors by modeling the interactions among instruments, actions, and anatomical targets. Despite its clinical importance for workflow analysis and skill assessment, progress has been hindered by severe class imbalance, subtle visual variations, and the semantic interdependence among triplet components. Existing approaches often address only a subset of these challenges rather than tackling them jointly, which limits their ability to form a holistic understanding. This study builds upon CurConMix, a spatial representation framework. At its core, a curriculum-guided contrastive learning strategy learns discriminative and progressively correlated features, further enhanced by structured hard-pair sampling and feature-level mixup. Its temporal extension, CurConMix+, integrates a Multi-Resolution Temporal
Transformer (MRTT) that achieves robust, context-aware understanding by adaptively fusing multi-scale temporal features and dynamically balancing spatio-temporal cues. Furthermore, we introduce LLS48, a new, hierarchically annotated
benchmark for complex laparoscopic left lateral sectionectomy, providing step-, task-, and action-level annotations. Extensive experiments on CholecT45 and LLS48 demonstrate that CurConMix+ not only outperforms state-of-the-art approaches in triplet recognition, but also exhibits strong cross-level generalization, as its fine-grained features effectively transfer to higher-level phase and step recognition tasks. Together, the framework and
dataset provide a unified foundation for hierarchy-aware, reproducible, and interpretable
surgical workflow understanding. The code and
dataset will be publicly released on GitHub to facilitate reproducibility and further research.
Read the full abstract ▾
All authors Yongjun Jeon, Jongmin Shin, Kanggil Park, Seonmin Park, Soyoung Lim, Jung Yong Kim, Jinsoo Rhu, Jongman Kim, Gyu-Seong Choi, Namkee Oh, Kyu-Hwan Jung
Computer Vision
Adaptive Multi-Scale Correlation Meta-Network for Few-Shot Remote Sensing Image Classification
Anurag Kaushish, Ayan Sar, Sampurna Roy et al. (+4 more)
Few-shot learning in remote sensing remains challenging due to three factors: the scarcity of labeled data, substantial domain shifts, and the multi-scale nature of geospatial objects. To address these issues, we introduce Adaptive Multi-Scale Correlation Meta-Network (AMC-MetaNet), a lightweight yet powerful framework with three key innovations: (i) correlation-guided feature pyramids for capturing scale-invariant patterns, (ii) an adaptive channel correlation module (ACCM) for learning dynamic cross-scale relationships, and (iii) correlation-guided meta-learning that leverages correlation patterns instead of conventional prototype averaging. Unlike prior approaches that rely on heavy pre-trained models or
transformers, AMC-MetaNet is trained from scratch with only $\sim600K$ parameters, offering $20\times$ fewer parameters than ResNet-18 while maintaining high efficiency ($<50$ms per image inference). AMC-MetaNet achieves up to 86.65% accuracy in 5-way 5-shot classification on various remote sensing
datasets, including EuroSAT, NWPU-RESISC45, UC Merced Land Use, and AID. Our results establish AMC-MetaNet as a computationally efficient, scale-aware framework for real-world few-shot remote sensing.
Read the full abstract ▾
All authors Anurag Kaushish, Ayan Sar, Sampurna Roy, Sudeshna Chakraborty, Prashant Trivedi, Tanupriya Choudhury, Kanav Gupta
Computer Vision
A Two-Stage Globally-Diverse Adversarial Attack for Vision-Language Pre-training Models
Wutao Chen, Huaqin Zou, Chen Wan et al. (+1 more)
Vision-
language pre-training (VLP) models are vulnerable to
adversarial examples, particularly in black-box scenarios. Existing
multimodal attacks often suffer from limited perturbation diversity and unstable multi-stage pipelines. To address these challenges, we propose 2S-GDA, a two-stage globally-diverse attack framework. The proposed method first introduces textual perturbations through a globally-diverse strategy by combining candidate text expansion with globally-aware replacement. To enhance visual diversity, image-level perturbations are generated using multi-scale resizing and block-shuffle rotation. Extensive experiments on VLP models demonstrate that 2S-GDA consistently improves attack success rates over state-of-the-art methods, with gains of up to 11.17% in black-box settings. Our framework is modular and can be easily combined with existing methods to further enhance
adversarial transferability.
Read the full abstract ▾
All authors Wutao Chen, Huaqin Zou, Chen Wan, Lifeng Huang
Computer Vision
Concepts from Representations: Post-hoc Concept Bottleneck Models via Sparse Decomposition of Visual Representations
Shizhan Gong, Xiaofan Zhang and Qi Dou
Deep learning has achieved remarkable success in image recognition, yet their inherent opacity poses challenges for deployment in critical domains. Concept-based interpretations aim to address this by explaining model
reasoning through human-understandable concepts. However, existing post-hoc methods and ante-hoc concept bottleneck models (CBMs), suffer from limitations such as unreliable concept relevance, non-visual or labor-intensive concept definitions, and model or data-agnostic assumptions. This paper introduces Post-hoc Concept Bottleneck Model via Representation Decomposition (PCBM-ReD), a novel pipeline that retrofits interpretability onto pretrained opaque models. PCBM-ReD automatically extracts visual concepts from a pre-trained encoder, employs
multimodal large
language models (MLLMs) to label and filter concepts based on visual identifiability and task relevance, and selects an independent subset via reconstruction-guided optimization. Leveraging CLIP's visual-text
alignment, it decomposes image representations into linear combination of concept
embeddings to fit into the CBMs abstraction. Extensive experiments across 11 image classification tasks show PCBM-ReD achieves state-of-the-art accuracy, narrows the performance gap with end-to-end models, and exhibits better interpretability.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
OpenNavMap: Multi-Session Appearance-Based Topometric Mapping for Scalable Visual Navigation
Jianhao Jiao, Changkun Liu, Jingwen Yu et al. (+4 more)
Scalable and maintainable maps are fundamental to large-scale
navigation and the long-term deployment of
robots in real-world environments. However, conventional maps that explicitly maintain dense geometry or 3D landmarks incur high storage and maintenance costs, while the core challenge of scaling to multi-session mapping is visual localization under sparse viewpoint overlap, temporal appearance shifts, and cross-device variance. To address this, we propose OpenNavMap, a lightweight, landmark-free topometric mapping system that organizes image nodes into covisibility, odometry, and traversability graphs and delegates local geometry recovery to 3D
geometric foundation models (GFMs) on demand. For localization, dynamic-programming-based sequence matching narrows candidate correspondences for a GFM, reducing global estimation to a lightweight, pose-only optimization; for mapping, a lifelong pipeline fuses multi-session, multi-device data via cross-device merging and node culling. On a 19km
dataset across four real-world environments, \methodname attains a state-of-the-art $0.62$m translation error on the Map-Free
benchmark, bounds the absolute trajectory error below $3$m across 15.7km without depth sensors, and completes $12$ autonomous image-goal visual
navigation tasks on both simulated and physical
robots. Code and
datasets will be made publicly available at https://rpl-cs-ucl.github.io/OpenNavMap_page.
Read the full abstract ▾
All authors Jianhao Jiao, Changkun Liu, Jingwen Yu, Boyi Liu, Qianyi Zhang, Yue Wang, Dimitrios Kanoulas
Computer Vision
LegacyAvatars: Volumetric Face Avatars For Traditional Graphics Pipelines
Safa C. Medin, Gengyan Li, Ziqian Bai et al. (+8 more)
We introduce a novel representation for efficient classical rendering of photorealistic 3D face avatars. Leveraging recent advances in radiance fields anchored to parametric face models, our approach achieves controllable volumetric rendering of complex facial features, including hair, skin, and eyes. At enrollment time, we learn a set of radiance
manifolds in 3D space to extract an explicit layered mesh, along with appearance and warp textures. During deployment, this allows us to control and animate the face through simple linear blending and alpha compositing of textures over a static mesh. This explicit representation also enables the generated avatar to be efficiently streamed online and then rendered using classical mesh and shader-based rendering on legacy graphics platforms, eliminating the need for any custom engineering or integration.
Read the full abstract ▾
All authors Safa C. Medin, Gengyan Li, Ziqian Bai, Ruofei Du, Leonhard Helminger, Yinda Zhang, Stephan J. Garbin, Philip L. Davidson, Gregory W. Wornell, Thabo Beeler, Abhimitra Meka
Computer Vision
Bowen Lin, Fanjiang Ye, Yihua Liu et al. (+7 more)
Diffusion Transformers (DiTs) achieve state-of-the-art performance in text-to-image synthesis but remain computationally expensive due to the iterative nature of denoising and the quadratic cost of global
attention. In this work, we observe that denoising dynamics are spatially non-uniform-background regions converge rapidly while edges and textured areas evolve much more actively. Building on this insight, we propose SDiT, a Semantic Region-Adaptive
Diffusion Transformer that allocates computation according to regional complexity. SDiT introduces a training-free framework combining (1) semantic-aware clustering via fast Quickshift-based
segmentation, (2) complexity-driven regional scheduling to selectively update informative areas, and (3) boundary-aware refinement to maintain spatial coherence. Without any model retraining or architectural modification, SDiT achieves up to 3.0x acceleration while preserving nearly identical perceptual and semantic quality to full-
attention inference.
Read the full abstract ▾
All authors Bowen Lin, Fanjiang Ye, Yihua Liu, Zhenghui Guo, Boyuan Zhang, Weijian Zheng, Yufan Xu, Tiancheng Xing, Yuke Wang, Chengming Zhang
Computer Vision
CytoCLIP: Learning Cytoarchitectural Characteristics in Developing Human Brain Using Contrastive Language Image Pre-Training
Pralaypati Ta, Sriram Venkatesaperumal, Keerthi Ram et al. (+1 more)
The functions of different regions of the human
brain are closely linked to their distinct cytoarchitecture, which is defined by the spatial arrangement and morphology of the
cells. Identifying
brain regions by their cytoarchitecture enables various scientific analyses of the
brain. However, delineating these areas manually in
brain histological sections is time-consuming and requires specialized knowledge. An automated approach is necessary to minimize the effort needed from human experts. To address this, we propose CytoCLIP, a suite of
vision-
language models derived from pre-trained Contrastive
Language-Image Pre-Training (CLIP) frameworks to learn joint visual-text representations of
brain cytoarchitecture. CytoCLIP comprises two model variants: one is trained using low-resolution whole-region images to understand the overall cytoarchitectural pattern of an area, and the other is trained on high-resolution image tiles for detailed cellular-level representation. The training
dataset is created from NISSL-stained histological sections of developing fetal
brains of different gestational weeks. It includes 86 distinct regions for low-resolution images and 379
brain regions for high-resolution tiles. We evaluate the model's understanding of the cytoarchitecture and generalization ability using region classification and cross-modal
retrieval tasks. Multiple experiments are performed under various data setups, including data from samples of different ages and sectioning planes. Experimental results demonstrate that CytoCLIP outperforms existing methods. It achieves a weighted F1 score of 0.87 for whole-region classification and 0.91 for high-resolution image tile classification.
Read the full abstract ▾
All authors Pralaypati Ta, Sriram Venkatesaperumal, Keerthi Ram, Mohanasankar Sivaprakasam
Computer Vision
AgenticPruner: MAC-Constrained Neural Network Compression via LLM-Driven Strategy Search
Shahrzad Esmat, Mahdi Banisharif and Ali Jannesari
Neural network
pruning remains essential for deploying deep learning models on resource-constrained devices, yet existing approaches primarily target parameter reduction without directly controlling computational cost. This yields unpredictable inference latency in deployment scenarios where strict Multiply-Accumulate (MAC) operation budgets must be met. We propose AgenticPruner, a framework utilizing large
language models to achieve MAC-constrained optimization through iterative strategy learning. Our approach coordinates three specialized
agents: a Profiling
Agent that analyzes model architecture and MAC distributions, a Master
Agent that orchestrates the workflow with divergence monitoring, and an Analysis
Agent powered by Claude 3.5 Sonnet that learns optimal strategies from historical attempts. Through in-context learning, the Analysis
Agent improves convergence success rate from 48% to 71% compared to grid search. Building upon isomorphic
pruning's graph-based structural grouping, our method adds context-aware adaptation by analyzing patterns across
pruning iterations, enabling automatic convergence to target MAC budgets within user-defined tolerance bands. We validate our framework on ImageNet-1K across ResNet, ConvNeXt, and DeiT architectures. On CNNs, our approach achieves MAC targeting while maintaining or improving accuracy: ResNet-50 reaches 1.77G MACs with 77.04% accuracy (+0.91% vs baseline); ResNet-101 achieves 4.22G MACs with 78.94% accuracy (+1.56% vs baseline). For ConvNeXt-Small,
pruning to 8.17G MACs yields 1.41x GPU and 1.07x CPU speedup with 45% parameter reduction. On
Vision Transformers, we demonstrate MAC-budget compliance within user-defined tolerance bands (typically +1% to +5% overshoot, -5% to -15% undershoot), establishing feasibility for deployment scenarios requiring strict computational guarantees.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
DALD-PCAC: Density-Adaptive Learning Descriptor for Point Cloud Lossless Attribute Compression
Chunyang Fu, Ge Li, Wei Gao et al. (+3 more)
Recently, deep learning has significantly advanced the performance of point cloud geometry compression. However, the learning-based lossless attribute compression of point clouds with varying densities is under-explored. In this paper, we develop a learning-based framework, namely DALD-PCAC that leverages Levels of Detail (LoD) to tailor for point cloud lossless attribute compression. We develop a point-wise
attention model using a permutation-invariant
Transformer to tackle the challenges of sparsity and irregularity of point clouds during context modeling. We also propose a Density-Adaptive Learning Descriptor (DALD) capable of capturing structure and correlations among points across a large range of neighbors. In addition, we develop a prior-guided block partitioning to reduce the attribute variance within blocks and enhance the performance. Experiments on
LiDAR and object point clouds show that DALD-PCAC achieves the state-of-the-art performance on most data. Our method boosts the compression performance and is robust to the varying densities of point clouds. Moreover, it guarantees a good trade-off between performance and complexity, exhibiting great potential in real-world applications. The source code is available at https://github.com/zb12138/DALD_PCAC.
Read the full abstract ▾
All authors Chunyang Fu, Ge Li, Wei Gao, Shiqi Wang, Zhu Li, Shan Liu
Computer Vision
Soft Shadow Diffusion (SSD): Physics-inspired Learning for 3D Computational Periscopy
Fadlullah Raji and John Murray-Bruce
Conventional imaging requires a line of sight to create accurate visual representations of a scene. In certain circumstances, however, obtaining a suitable line of sight may be impractical, dangerous, or even impossible. Non-line-of-sight (NLOS) imaging addresses this challenge by reconstructing the scene from indirect measurements. Recently, passive NLOS methods that use an ordinary photograph of the subtle shadow cast onto a visible wall by the hidden scene have gained interest. These methods are currently limited to 1D or low-resolution 2D color imaging or to localizing a hidden object whose shape is approximately known. Here, we generalize this class of methods and demonstrate a 3D reconstruction of a hidden scene from an ordinary NLOS photograph. To achieve this, we propose a novel reformulation of the light transport model that conveniently decomposes the hidden scene into light-occluding and non-light-occluding components to yield a separable non-linear least squares (SNLLS) inverse problem. We develop two solutions: A gradient-based optimization method and a physics-inspired neural network approach, which we call Soft Shadow
diffusion (SSD). Despite the challenging ill-conditioned inverse problem encountered here, our approaches are effective on numerous 3D scenes in real experimental scenarios. Moreover, SSD is trained in simulation but generalizes well to unseen classes in simulation and real-world NLOS scenes. SSD also shows surprising robustness to noise and ambient illumination.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
DeepRAHT: Learning Predictive RAHT for Point Cloud Attribute Compression
Chunyang Fu, Tai Qin, Shiqi Wang et al. (+1 more)
Regional Adaptive Hierarchical Transform (RAHT) is an effective point cloud attribute compression (PCAC) method. However, its application in deep learning lacks research. In this paper, we propose an end-to-end RAHT framework for lossy PCAC based on the sparse tensor, called DeepRAHT. The RAHT transform is performed within the learning reconstruction process, without requiring manual RAHT for preprocessing. We also introduce the predictive RAHT to reduce bitrates and design a learning-based prediction model to enhance performance. Moreover, we devise a bitrate proxy that applies run-length coding to
entropy model, achieving seamless variable-rate coding and improving robustness. DeepRAHT is a reversible and distortion-controllable framework, ensuring its lower bound performance and offering significant application potential. The experiments demonstrate that DeepRAHT is a high-performance, faster, and more robust solution than the baseline methods. Project Page: https://github.com/zb12138/DeepRAHT.
Read the full abstract ▾
All authors Chunyang Fu, Tai Qin, Shiqi Wang, Zhu Li
Computer Vision
Federated Joint Learning for Domain and Class Generalization
Haoran Xu, Jiaze Li, Jianzhong Ju et al. (+1 more)
Efficient fine-tuning of visual-
language models like CLIP has become crucial due to their large-scale parameter size and extensive pretraining requirements. Existing methods typically address either the issue of unseen classes or unseen domains in isolation, without considering a joint framework for both. In this paper, we propose
Federated Joint Learning for Domain and Class Generalization, termed FedDCG, a novel approach that addresses both class and domain generalization in
federated learning settings. Our method introduces a domain grouping strategy where class-generalized networks are trained within each group to prevent decision boundary confusion. During inference, we aggregate class-generalized results based on domain similarity, effectively integrating knowledge from both class and domain generalization. Specifically, a learnable network is employed to enhance class generalization capabilities, and a decoupling mechanism separates general and domain-specific knowledge, improving generalization to unseen domains. Extensive experiments across various
datasets show that FedDCG outperforms state-of-the-art baselines in terms of accuracy and robustness.
Read the full abstract ▾
All authors Haoran Xu, Jiaze Li, Jianzhong Ju, Zhenbo Luo