Computer Vision
The Mechanics of CNN Filtering with Rectification
Liam Frija-Altarac and Matthew Toews
This paper proposes elementary information mechanics as a new model for understanding the mechanical properties of convolutional filtering with rectification, inspired by physical theories of special relativity and
quantum mechanics. We consider kernels decomposed into orthogonal even and odd components. Even components cause image content to diffuse isotropically while preserving the center of mass, analogously to rest or potential energy with zero net momentum. Odd kernels cause directional displacement of the center of mass, analogously to kinetic energy with non-zero momentum. The speed of information displacement is linearly related to the ratio of odd vs total kernel energy. Even-Odd properties are analyzed in the spectral domain via the discrete cosine transform (DCT), where the structure of small convolutional filters (e.g. $3 \times 3$ pixels) is dominated by low-frequency bases, specifically the DC $\Sigma$ and gradient components $\nabla$, which define the fundamental modes of information propagation. To our knowledge, this is the first work demonstrating the link between information processing in generic CNNs and the energy-momentum relation, a cornerstone of modern relativistic physics.
Read the full abstract ▾
Computer Vision
Spatial-aware Vision Language Model for Autonomous Driving
Weijie Wei, Zhipeng Luo, Ling Feng et al. (+1 more)
While
Vision-
Language Models (VLMs) show significant promise for end-to-end autonomous driving by leveraging the common sense embedded in
language models, their reliance on 2D image cues for complex scene understanding and decision-making presents a critical bottleneck for safety and reliability. Current image-based methods struggle with accurate metric spatial
reasoning and
geometric inference, leading to unreliable driving policies. To bridge this gap, we propose LVLDrive (
LiDAR-
Vision-
Language), a novel framework specifically designed to upgrade existing VLMs with robust 3D metric spatial understanding for autonomous driving by incoperating
LiDAR point cloud as an extra input modality. A key challenge lies in mitigating the catastrophic disturbance introduced by disparate 3D data to the pre-trained VLMs. To this end, we introduce a Gradual Fusion Q-Former that incrementally injects
LiDAR features, ensuring the stability and preservation of the VLM's existing knowledge base. Furthermore, we develop a spatial-aware question-answering (SA-QA)
dataset to explicitly teach the model advanced 3D perception and
reasoning capabilities. Extensive experiments on driving
benchmarks demonstrate that LVLDrive achieves superior performance compared to
vision-only counterparts across scene understanding, metric spatial perception, and reliable driving decision-making. Our work highlights the necessity of explicit 3D metric data for building trustworthy VLM-based autonomous systems.
Read the full abstract ▾
All authors Weijie Wei, Zhipeng Luo, Ling Feng, Venice Erin Liong
Computer Vision
SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning
Yong Xien Chng, Tao Hu, Wenwen Tong et al. (+10 more)
While
Vision-
Language Models (VLMs) can solve complex tasks through agentic
reasoning, their capabilities remain largely constrained to text-oriented chain-of-thought or isolated tool invocation. They fail to exhibit the human-like proficiency required to seamlessly interleave dynamic tool manipulation with continuous
reasoning, particularly in knowledge-intensive and visually complex scenarios that demand coordinated external tools such as search and image cropping. In this work, we introduce SenseNova-MARS, a novel
Multimodal Agentic
Reasoning and Search framework that empowers VLMs with interleaved visual
reasoning and tool-use capabilities via reinforcement learning (RL). Specifically, SenseNova-MARS dynamically integrates the image search, text search, and image crop tools to tackle fine-grained and knowledge-intensive visual understanding challenges. In the RL stage, we propose the Batch-Normalized Group Sequence Policy Optimization (BN-GSPO)
algorithm to improve the training stability and advance the model's ability to invoke tools and reason effectively. To comprehensively evaluate the agentic VLMs on complex visual tasks, we introduce the HR-MMSearch
benchmark, the first search-oriented
benchmark composed of high-resolution images with knowledge-intensive and search-driven questions. Experiments demonstrate that SenseNova-MARS achieves state-of-the-art performance on open-source search and fine-grained image understanding
benchmarks. Specifically, on search-oriented
benchmarks, SenseNova-MARS-32B scores 74.3 on MMSearch and 54.4 on HR-MMSearch, surpassing proprietary models such as Gemini-3-Pro and GPT-5.2. SenseNova-MARS represents a promising step toward agentic VLMs by providing effective and robust tool-use capabilities. To facilitate further research in this field, we will release all code, models, and
datasets.
Read the full abstract ▾
All authors Yong Xien Chng, Tao Hu, Wenwen Tong, Xueheng Li, Jiandong Chen, Haojia Yu, Jiefan Lu, Hewei Guo, Hanming Deng, Chengjun Xie, Gao Huang, Dahua Lin, Lewei Lu
Computer Vision
Robust Egocentric Referring Video Object Segmentation via Dual-Modal Causal Intervention
Haijing Liu, Zhiyuan Song, Hefeng Wu et al. (+3 more)
Egocentric Referring
Video Object
Segmentation (Ego-RVOS) aims to segment the specific object actively involved in a human action, as described by a
language query, within first-person
videos. This task is critical for understanding
egocentric human behavior. However, achieving such
segmentation robustly is challenging due to ambiguities inherent in
egocentric videos and biases present in training data. Consequently, existing methods often struggle, learning spurious correlations from skewed object-action pairings in
datasets and fundamental visual confounding factors of the
egocentric perspective, such as rapid motion and frequent occlusions. To address these limitations, we introduce Causal Ego-REferring
Segmentation (CERES), a plug-in causal framework that adapts strong, pre-trained RVOS backbones to the
egocentric domain. CERES implements dual-modal causal intervention: applying
backdoor adjustment principles to counteract
language representation biases learned from
dataset statistics, and leveraging front-door adjustment concepts to address visual confounding by intelligently integrating semantic visual features with
geometric depth information guided by causal principles, creating representations more robust to
egocentric distortions. Extensive experiments demonstrate that CERES achieves state-of-the-art performance on Ego-RVOS
benchmarks, highlighting the potential of applying causal
reasoning to build more reliable models for broader
egocentric video understanding.
Read the full abstract ▾
All authors Haijing Liu, Zhiyuan Song, Hefeng Wu, Tao Pu, Keze Wang, Liang Lin
Computer Vision
UniAct: Unified Motion Generation and Action Streaming for Humanoid Robots
Nan Jiang, Zimo He, Wanhe Yu et al. (+8 more)
A long-standing objective in
humanoid robotics is the realization of versatile
agents capable of following diverse
multimodal instructions with human-level flexibility. Despite advances in
humanoid control, bridging high-level
multimodal perception with whole-body execution remains a significant bottleneck. Existing methods often struggle to translate heterogeneous instructions – such as
language, music, and trajectories – into stable, real-time actions. Here we show that UniAct, a two-stage framework integrating a fine-tuned MLLM with a causal streaming pipeline, enables
humanoid robots to execute
multimodal instructions with sub-500 ms latency. By unifying inputs through a shared discrete codebook via FSQ, UniAct ensures cross-modal
alignment while constraining motions to a physically grounded
manifold. This approach yields a 19% improvement in the success rate of zero-shot tracking of imperfect reference motions. We validate UniAct on UniMoCap, our 20-hour
humanoid motion
benchmark, demonstrating robust generalization across diverse real-world scenarios. Our results mark a critical step toward responsive, general-purpose
humanoid assistants capable of seamless interaction through unified perception and control.
Read the full abstract ▾
All authors Nan Jiang, Zimo He, Wanhe Yu, Lexi Pang, Yunhao Li, Hongjie Li, Jieming Cui, Yuhan Li, Yizhou Wang, Yixin Zhu, Siyuan Huang
Computer Vision
Virtual-Eyes: Quantitative Validation of a Lung CT Quality-Control Pipeline for Foundation-Model Cancer Risk Prediction
Md. Enamul Hoq, Linda Larson-Prior and Fred Prior
Robust preprocessing is rarely quantified in deep-learning pipelines for low-dose CT (LDCT) lung cancer screening. We develop and validate Virtual-Eyes, a clinically motivated 16-bit CT quality-control pipeline, and measure its differential impact on generalist foundation models versus specialist models. Virtual-Eyes enforces strict 512x512 in-plane resolution, rejects short or non-diagnostic series, and extracts a contiguous lung block using Hounsfield-unit filtering and bilateral lung-coverage scoring while preserving the native 16-bit grid. Using 765 NLST patients (182 cancer, 583 non-cancer), we compute slice-level
embeddings from RAD-DINO and Merlin with frozen encoders and train leakage-free patient-level MLP heads; we also evaluate Sybil and a 2D ResNet-18 baseline under Raw versus Virtual-Eyes inputs without backbone retraining. Virtual-Eyes improves RAD-DINO slice-level AUC from 0.576 to 0.610 and patient-level AUC from 0.646 to 0.683 (mean pooling) and from 0.619 to 0.735 (max pooling), with improved calibration (Brier score 0.188 to 0.112). In contrast, Sybil and ResNet-18 degrade under Virtual-Eyes (Sybil AUC 0.886 to 0.837; ResNet-18 AUC 0.571 to 0.596) with evidence of context dependence and shortcut learning, and Merlin shows limited transferability (AUC approximately 0.507 to 0.567) regardless of preprocessing. These results demonstrate that anatomically targeted QC can stabilize and improve generalist foundation-model workflows but may disrupt specialist models adapted to raw clinical context.
Read the full abstract ▾
Computer Vision
One-shot synthesis of rare gastrointestinal lesions improves diagnostic accuracy and clinical training
Jia Yu, Yan Zhu, Peiyao Fu et al. (+7 more)
Rare gastrointestinal lesions are infrequently encountered in routine endoscopy, restricting the data available for developing reliable artificial intelligence (AI) models and training novice clinicians. Here we present EndoRare, a one-shot, retraining-free generative framework that synthesizes diverse, high-fidelity lesion exemplars from a single reference image. By leveraging
language-guided concept disentanglement, EndoRare separates pathognomonic lesion features from non-diagnostic attributes, encoding the former into a learnable prototype
embedding while varying the latter to ensure diversity. We validated the framework across four rare pathologies (calcifying fibrous tumor, juvenile polyposis syndrome, familial adenomatous polyposis, and Peutz-Jeghers syndrome). Synthetic images were judged clinically plausible by experts and, when used for data augmentation, significantly enhanced downstream AI classifiers, improving the true positive rate at low false-positive rates. Crucially, a blinded reader study demonstrated that novice endoscopists exposed to EndoRare-generated cases achieved a 0.400 increase in recall and a 0.267 increase in precision. These results establish a practical, data-efficient pathway to bridge the rare-disease gap in both computer-aided diagnostics and clinical education.
Read the full abstract ▾
All authors Jia Yu, Yan Zhu, Peiyao Fu, Tianyi Chen, Zhihua Wang, Fei Wu, Quanlin Li, Pinghong Zhou, Shuo Wang, Xian Yang
Computer Vision
LiftProj: Space Lifting and Projection-Based Panorama Stitching
Yuan Jia, Ruimin Wu, Rui Song et al. (+2 more)
Traditional image stitching techniques have predominantly utilized two-dimensional homography transformations and mesh warping to achieve
alignment on a planar surface. While effective for scenes that are approximately coplanar or exhibit minimal parallax, these approaches often result in ghosting, structural bending, and stretching distortions in non-overlapping regions when applied to real three-dimensional scenes characterized by multiple depth layers and occlusions. Such challenges are exacerbated in multi-view accumulations and 360{\deg} closed-loop stitching scenarios. In response, this study introduces a spatially lifted panoramic stitching framework that initially elevates each input image into a dense three-dimensional point representation within a unified coordinate system, facilitating global cross-view fusion augmented by confidence metrics. Subsequently, a unified projection center is established in three-dimensional space, and an equidistant cylindrical projection is employed to map the fused data onto a single panoramic
manifold, thereby producing a geometrically consistent 360{\deg} panoramic layout. Finally, hole filling is conducted within the canvas domain to address unknown regions revealed by viewpoint transitions, restoring continuous texture and semantic coherence. This framework reconceptualizes stitching from a two-dimensional warping paradigm to a three-dimensional consistency paradigm and is designed to flexibly incorporate various three-dimensional lifting and completion modules. Experimental evaluations demonstrate that the proposed method substantially mitigates
geometric distortions and ghosting artifacts in scenarios involving significant parallax and complex occlusions, yielding panoramic results that are more natural and consistent.
Read the full abstract ▾
All authors Yuan Jia, Ruimin Wu, Rui Song, Jiaojiao Li, Bin Song
Computer Vision
Taming Hallucinations: Boosting MLLMs' Video Understanding via Counterfactual Video Generation
Zhe Huang, Hao Wen, Aiming Hao et al. (+6 more)
Multimodal Large
Language Models (MLLMs) have made remarkable progress in
video understanding. However, they suffer from a critical vulnerability: an over-reliance on
language priors, which can lead to visual ungrounded
hallucinations, especially when processing counterfactual
videos that defy common sense. This limitation, stemming from the intrinsic data imbalance between text and
video, is challenging to address due to the substantial cost of collecting and annotating counterfactual data. To address this, we introduce DualityForge, a novel counterfactual data synthesis framework that employs controllable,
diffusion-based
video editing to transform real-world
videos into counterfactual scenarios. By
embedding structured contextual information into the
video editing and QA generation processes, the framework automatically produces high-quality QA pairs together with original-edited
video pairs for contrastive training. Based on this, we build DualityVidQA, a large-scale
video dataset designed to reduce MLLM
hallucinations. In addition, to fully exploit the contrastive nature of our paired data, we propose Duality-Normalized Advantage Training (DNA-Train), a two-stage SFT-RL training regime where the RL phase applies pair-wise $\ell_1$ advantage normalization, thereby enabling a more stable and efficient policy optimization. Experiments on DualityVidQA-Test demonstrate that our method substantially reduces model
hallucinations on counterfactual
videos, yielding a relative improvement of 24.0% over the Qwen2.5-VL-7B baseline. Moreover, our approach achieves significant gains across both
hallucination and general-purpose
benchmarks, indicating strong generalization capability. We will open-source our
dataset and code.
Read the full abstract ▾
All authors Zhe Huang, Hao Wen, Aiming Hao, Bingze Song, Meiqi Wu, Jiahong Wu, Xiangxiang Chu, Sheng Lu, Haoqian Wang
Computer Vision
Physically-Grounded Manifold Projection Model for Generalizable Metal Artifact Reduction in Dental CBCT
Zhi Li, Yaqi Wang, Bingtao Ma et al. (+3 more)
Metal artifacts in Dental CBCT severely obscure anatomical structures, hindering diagnosis. Current deep learning for Metal Artifact Reduction (MAR) faces limitations: supervised methods suffer from spectral blurring due to "regression-to-the-mean", while unsupervised ones risk structural
hallucinations. Denoising
Diffusion Models (DDPMs) offer realism but rely on slow,
stochastic iterative sampling, unsuitable for clinical use. To resolve this, we propose the Physically-Grounded
Manifold Projection (PGMP) framework. First, our Anatomically-Adaptive Physics Simulation (AAPS) pipeline synthesizes high-fidelity training pairs via Monte Carlo spectral modeling and patient-specific digital twins, bridging the synthetic-to-real gap. Second, our DMP-Former adapts the Direct x-Prediction paradigm, reformulating restoration as a deterministic
manifold projection to recover clean anatomy in a single forward pass, eliminating
stochastic sampling. Finally, a Semantic-Structural
Alignment (SSA) module anchors the solution using priors from medical foundation models (MedDINOv3), ensuring clinical plausibility. Experiments on synthetic and multi-center clinical
datasets show PGMP outperforms state-of-the-art methods on unseen anatomy, setting new
benchmarks in efficiency and diagnostic reliability. Code and data: https://github.com/ricoleehduu/PGMP.
Read the full abstract ▾
All authors Zhi Li, Yaqi Wang, Bingtao Ma, Yifan Zhang, Huiyu Zhou, Shuai Wang
Computer Vision
MambaSeg: Harnessing Mamba for Accurate and Efficient Image-Event Semantic Segmentation
Fuqiang Gu, Yuanke Li, Xianlei Long et al. (+4 more)
Semantic
segmentation is a fundamental task in computer
vision with wide-ranging applications, including autonomous driving and robotics. While RGB-based methods have achieved strong performance with CNNs and
Transformers, their effectiveness degrades under fast motion, low-light, or high dynamic range conditions due to limitations of frame cameras. Event cameras offer complementary advantages such as high temporal resolution and low latency, yet lack color and texture, making them insufficient on their own. To address this, recent research has explored
multimodal fusion of RGB and event data; however, many existing approaches are computationally expensive and focus primarily on spatial fusion, neglecting the temporal dynamics inherent in event streams. In this work, we propose MambaSeg, a novel dual-branch semantic
segmentation framework that employs parallel Mamba encoders to efficiently model RGB images and event streams. To reduce cross-modal ambiguity, we introduce the Dual-Dimensional Interaction Module (DDIM), comprising a Cross-Spatial Interaction Module (CSIM) and a Cross-Temporal Interaction Module (CTIM), which jointly perform fine-grained fusion along both spatial and temporal dimensions. This design improves cross-modal
alignment, reduces ambiguity, and leverages the complementary properties of each modality. Extensive experiments on the DDD17 and DSEC
datasets demonstrate that MambaSeg achieves state-of-the-art
segmentation performance while significantly reducing computational cost, showcasing its promise for efficient, scalable, and robust
multimodal perception.
Read the full abstract ▾
All authors Fuqiang Gu, Yuanke Li, Xianlei Long, Kangping Ji, Chao Chen, Qingyi Gu, Zhenliang Ni
Computer Vision
MotivNet: Evolving Meta-Sapiens into an Emotionally Intelligent Foundation Model
Rahul Medicharla and Alper Yilmaz
In this paper, we introduce MotivNet, a generalizable facial
emotion recognition model for robust real-world application. Current state-of-the-art FER models tend to have weak generalization when tested on diverse data, leading to deteriorated performance in the real world and hindering FER as a research domain. Though researchers have proposed complex architectures to address this generalization issue, they require training cross-domain to obtain generalizable results, which is inherently contradictory for real-world application. Our model, MotivNet, achieves competitive performance across
datasets without cross-domain training by using Meta-Sapiens as a backbone. Sapiens is a human
vision foundational model with state-of-the-art generalization in the real world through large-scale pretraining of a Masked Autoencoder. We propose MotivNet as an additional downstream task for Sapiens and define three criteria to evaluate MotivNet's viability as a Sapiens task:
benchmark performance, model similarity, and data similarity. Throughout this paper, we describe the components of MotivNet, our training approach, and our results showing MotivNet is generalizable across domains. We demonstrate that MotivNet can be benchmarked against existing SOTA models and meets the listed criteria, validating MotivNet as a Sapiens downstream task, and making FER more incentivizing for in-the-wild application. The code is available at https://github.com/OSUPCVLab/EmotionFromFaceImages.
Read the full abstract ▾
Computer Vision
Mirage: One-Step Video Diffusion for Photorealistic and Coherent Asset Editing in Driving Scenes
Shuyun Wang, Haiyang Sun, Bing Wang et al. (+2 more)
Vision-centric autonomous driving systems rely on diverse and scalable training data to achieve robust performance. While
video object editing offers a promising path for data augmentation, existing methods often struggle to maintain both high visual fidelity and temporal coherence. In this work, we propose Mirage, a one-step
video diffusion model for photorealistic and coherent asset editing in driving scenes. Mirage builds upon a text-to-
video diffusion prior to ensure temporal consistency across frames. However, 3D causal variational autoencoders often suffer from degraded spatial fidelity due to compression, and directly passing 3D encoder features to decoder layers breaks temporal causality. To address this, we inject temporally agnostic latents from a pretrained 2D encoder into the 3D decoder to restore detail while preserving causal structures. Furthermore, because scene objects and inserted assets are optimized under different objectives, their Gaussians exhibit a distribution mismatch that leads to pose misalignment. To mitigate this, we introduce a two-stage data
alignment strategy combining coarse 3D
alignment and fine 2D refinement, thereby improving
alignment and providing cleaner supervision. Extensive experiments demonstrate that Mirage achieves high realism and temporal consistency across diverse editing scenarios. Beyond asset editing, Mirage can also generalize to other
video-to-
video translation tasks, serving as a reliable baseline for future research. Our code is available at https://github.com/wm-research/mirage.
Read the full abstract ▾
All authors Shuyun Wang, Haiyang Sun, Bing Wang, Hangjun Ye, Xin Yu
Computer Vision
ARM: A Learnable, Plug-and-Play Module for CLIP-based Open-vocabulary Semantic Segmentation
Ziquan Liu, Zhewei Zhu and Xuyang Shi
Open-vocabulary semantic
segmentation (OVSS) is fundamentally hampered by the coarse, image-level representations of CLIP, which lack precise pixel-level details. Existing training-free methods attempt to resolve this by either importing priors from costly external foundation models (e.g., SAM, DINO) or by applying static, hand-crafted heuristics to CLIP's internal features. These approaches are either computationally expensive or sub-optimal. We propose the
Attention Refinement Module (ARM), a lightweight, learnable module that effectively unlocks and refines CLIP's internal potential. Unlike static-fusion methods, ARM learns to adaptively fuse hierarchical features. It employs a semantically-guided cross-
attention block, using robust deep features (K, V) to select and refine detail-rich shallow features (Q), followed by a self-
attention block. The key innovation lies in a “train once, use anywhere" paradigm. Trained once on a general-purpose
dataset (e.g., COCO-Stuff), ARM acts as a universal plug-and-play post-processor for diverse training-free frameworks. Extensive experiments show that ARM consistently boosts baseline performance on multiple
benchmarks with negligible inference overhead, establishing an efficient and effective paradigm for training-free OVSS.
Read the full abstract ▾
Computer Vision
Medical Image Classification on Imbalanced Data Using ProGAN and SMA-Optimized ResNet: Application to COVID-19
Sina Jahromi, Farshid Hajati, Alireza Rezaee et al. (+1 more)
The challenge of imbalanced data is prominent in medical image classification. This challenge arises when there is a significant disparity in the number of images belonging to a particular class, such as the presence or absence of a specific disease, as compared to the number of images belonging to other classes. This issue is especially notable during pandemics, which may result in an even more significant imbalance in the
dataset. Researchers have employed various approaches in recent years to detect COVID-19 infected individuals accurately and quickly, with artificial intelligence and machine learning
algorithms at the forefront. However, the lack of sufficient and balanced data remains a significant obstacle to these methods. This study addresses the challenge by proposing a progressive generative
adversarial network to generate synthetic data to supplement the real ones. The proposed method suggests a weighted approach to combine synthetic data with real ones before inputting it into a deep network classifier. A multi-objective meta-heuristic population-based optimization
algorithm is employed to optimize the hyper-parameters of the classifier. The proposed model exhibits superior cross-validated metrics compared to existing methods when applied to a large and imbalanced chest X-ray image
dataset of COVID-19. The proposed model achieves 95.5% and 98.5% accuracy for 4-class and 2-class imbalanced classification problems, respectively. The successful experimental outcomes demonstrate the effectiveness of the proposed model in classifying medical images using imbalanced data during pandemics.
Read the full abstract ▾
All authors Sina Jahromi, Farshid Hajati, Alireza Rezaee, Javaher Nourian
Computer Vision · Cross-listed from cs.RO
RANGER: A Monocular Zero-Shot Semantic Navigation Framework through Visual Contextual Adaptation
Ming-Ming Yu, Yi Chen, Börje F. Karlsson et al. (+1 more)
Efficient target localization and autonomous
navigation in complex environments are fundamental to real-world
embodied applications. While recent advances in
multimodal foundation models have enabled zero-shot object goal
navigation, allowing
robots to search for arbitrary objects without fine-tuning, existing methods face two key limitations: (1) heavy reliance on ground-truth depth and pose information, which restricts applicability in real-world scenarios; and (2) lack of visual in-context learning (VICL) capability to extract
geometric and semantic priors from environmental context, as in a short traversal
video. To address these challenges, we propose RANGER, a novel zero-shot, open-vocabulary semantic
navigation framework that operates using only a
monocular camera. Leveraging powerful 3D foundation models, RANGER eliminates the dependency on depth and pose while exhibiting strong VICL capability. By simply observing a short
video of the target environment, the system can also significantly improve task efficiency without requiring architectural modifications or task-specific retraining. The framework integrates several key components: keyframe-based 3D reconstruction, semantic point cloud generation,
vision-
language model (VLM)-driven exploration value estimation, high-level adaptive waypoint selection, and low-level action execution. Experiments on the HM3D
benchmark and real-world environments demonstrate that RANGER achieves competitive performance in terms of
navigation success rate and exploration efficiency, while showing superior VICL adaptability, with no previous 3D mapping of the environment required.
Read the full abstract ▾
All authors Ming-Ming Yu, Yi Chen, Börje F. Karlsson, Wenjun Wu
Computer Vision
CorGi: Contribution-Guided Block-Wise Interval Caching for Training-Free Acceleration of Diffusion Transformers
Yonglak Son, Suhyeok Kim, Seungryong Kim et al. (+1 more)
Diffusion transformer (DiT) achieves remarkable performance in visual generation, but its iterative denoising process combined with larger capacity leads to a high inference cost. Recent works have demonstrated that the iterative denoising process of DiT models involves substantial redundant computation across steps. To effectively reduce the redundant computation in DiT, we propose CorGi (Contribution-Guided Block-Wise Interval Caching), training-free DiT inference acceleration framework that selectively reuses the outputs of
transformer blocks in DiT across denoising steps. CorGi caches low-contribution blocks and reuses them in later steps within each interval to reduce redundant computation while preserving generation quality. For text-to-image tasks, we further propose CorGi+, which leverages per-block cross-
attention maps to identify salient
tokens and applies partial
attention updates to protect important object details. Evaluation on the state-of-the-art DiT models demonstrates that CorGi and CorGi+ achieve up to 2.0x speedup on average, while preserving high generation quality.
Read the full abstract ▾
All authors Yonglak Son, Suhyeok Kim, Seungryong Kim, Young Geun Kim
Computer Vision
PointRAFT: 3D deep learning for high-throughput prediction of potato tuber weight from partial point clouds
Pieter M. Blok, Haozhou Wang, Hyun Kwon Suh et al. (+3 more)
Key phenotypic traits, such as crop size, volume, and weight, can be estimated non-destructively from point clouds. In practice, however, point clouds of crops are typically incomplete due to self-occlusion, leading to systematic errors when estimating these traits. To overcome this limitation, crop traits can be inferred directly from incomplete point clouds using deep learning-based regression networks. These networks, however, may perform suboptimally, because their predictions do not incorporate prior
geometric knowledge of the crop. Such
geometric knowledge is often available in structured settings, such as on a potato harvester, where the fixed distance between the camera and the conveyor belt allows tuber height to be derived from the point cloud. To exploit this
geometric cue, we introduce PointRAFT, a high-throughput point cloud regression network that incorporates object height through a dedicated height
embedding. We hypothesized that this height
embedding improves tuber weight estimation from partial point clouds. To test this hypothesis, we collected 26,688 point clouds of 859 potato tubers on an operational harvester, spanning four cultivars and three growing seasons. On a test set of 5,254 point clouds representing 172 tubers, PointRAFT achieved a mean absolute error (MAE) of 11.7 g and a root mean squared error (RMSE) of 17.1 g, outperforming six point cloud regression networks without a height
embedding. Requiring only 6.3 ms per point cloud for preprocessing and inference, PointRAFT is fast enough for high-throughput weight estimation on a commercial potato harvester. Beyond potato weight estimation, PointRAFT provides a versatile regression network applicable to a wide range of 3D phenotyping and robotic perception tasks. The code, network weights, and a subset of the
dataset are publicly available at https://github.com/pieterblok/pointraft.
Read the full abstract ▾
All authors Pieter M. Blok, Haozhou Wang, Hyun Kwon Suh, Peicheng Wang, James Burridge, Wei Guo
Computer Vision
Guiding a Diffusion Transformer with the Internal Dynamics of Itself
Xingyu Zhou, Qifan Li, Xiaobin Hu et al. (+2 more)
The
diffusion model presents a powerful ability to capture the entire (conditional) data distribution. However, due to the lack of sufficient training and data to learn to cover low-probability areas, the model will be penalized for failing to generate high-quality images corresponding to these areas. To achieve better generation quality, guidance strategies such as classifier free guidance (CFG) can guide the samples to the high-probability areas during the sampling stage. However, the standard CFG often leads to over-simplified or distorted samples. On the other hand, the alternative line of guiding
diffusion model with its bad version is limited by carefully designed degradation strategies, extra training and additional sampling steps. In this paper, we proposed a simple yet effective strategy Internal Guidance (IG), which introduces an auxiliary supervision on the intermediate layer during training process and extrapolates the intermediate and deep layer's outputs to obtain generative results during sampling process. This simple strategy yields significant improvements in both training efficiency and generation quality on various baselines. On ImageNet 256x256, SiT-XL/2+IG achieves FID=5.31 and FID=1.75 at 80 and 800 epochs. More impressively, LightningDiT-XL/1+IG achieves FID=1.34 which achieves a large margin between all of these methods. Combined with CFG, LightningDiT-XL/1+IG achieves the current state-of-the-art FID of 1.19.
Read the full abstract ▾
All authors Xingyu Zhou, Qifan Li, Xiaobin Hu, Hai Chen, Shuhang Gu
Computer Vision
Deep Global Clustering for Hyperspectral Image Segmentation: Concepts, Applications, and Open Challenges
Yu-Tang Chang, Pin-Wei Chen and Shih-Fang Chen
Hyperspectral imaging (HSI) analysis faces computational bottlenecks due to massive data volumes that exceed available
memory. While foundation models pre-trained on large remote sensing
datasets show promise, their learned representations often fail to transfer to domain-specific applications like close-range agricultural monitoring where spectral signatures, spatial scales, and semantic targets differ fundamentally. This report presents Deep Global Clustering (DGC), a conceptual framework for
memory-efficient HSI
segmentation that learns global clustering structure from local patch observations without pre-training. DGC operates on small patches with overlapping regions to enforce consistency, enabling training in under 30 minutes on consumer hardware while maintaining constant
memory usage. On a leaf disease
dataset, DGC achieves background-tissue separation (mean IoU 0.925) and demonstrates unsupervised disease detection through navigable semantic granularity. However, the framework suffers from optimization instability rooted in multi-objective loss balancing: meaningful representations emerge rapidly but degrade due to cluster over-merging in feature space. We position this work as intellectual scaffolding - the design philosophy has merit, but stable implementation requires principled approaches to dynamic loss balancing. Code and data are available at https://github.com/b05611038/HSI_global_clustering.
Read the full abstract ▾
Computer Vision
Zefeng He, Xiaoye Qu, Yafu Li et al. (+3 more)
While recent
Multimodal Large
Language Models (MLLMs) have attained significant strides in
multimodal reasoning, their
reasoning processes remain predominantly text-centric, leading to suboptimal performance in complex long-horizon,
vision-centric tasks. In this paper, we establish a novel Generative
Multimodal Reasoning paradigm and introduce DiffThinker, a
diffusion-based
reasoning framework. Conceptually, DiffThinker reformulates
multimodal reasoning as a native generative image-to-image task, achieving superior logical consistency and spatial precision in
vision-centric tasks. We perform a systematic comparison between DiffThinker and MLLMs, providing the first in-depth investigation into the intrinsic characteristics of this paradigm, revealing four core properties: efficiency, controllability, native parallelism, and collaboration. Extensive experiments across four domains (sequential planning, combinatorial optimization, constraint satisfaction, and spatial configuration) demonstrate that DiffThinker significantly outperforms leading closed source models including GPT-5 (+314.2%) and Gemini-3-Flash (+111.6%), as well as the fine-tuned Qwen3-VL-32B baseline (+39.0%), highlighting generative
multimodal reasoning as a promising approach for
vision-centric
reasoning.
Read the full abstract ▾
All authors Zefeng He, Xiaoye Qu, Yafu Li, Tong Zhu, Siyuan Huang, Yu Cheng
Computer Vision
Deep Probabilistic Supervision for Image Classification
Anton Adelöw, Matteo Gamba and Atsuto Maki
Supervised training of deep neural networks for classification typically relies on hard targets, which promote overconfidence and can limit calibration, generalization, and robustness. Self-
distillation methods aim to mitigate this by leveraging inter-class and sample-specific information present in the model's own predictions, but often remain dependent on hard targets without explicitly modeling predictive uncertainty. With this in mind, we propose Deep Probabilistic Supervision (DPS), a principled learning framework constructing sample-specific target distributions via statistical inference on the model's own predictions, remaining independent of hard targets after initialization. We show that DPS consistently yields higher test accuracy (e.g., +2.0% for DenseNet-264 on ImageNet) and significantly lower Expected Calibration Error (ECE) (-40% ResNet-50, CIFAR-100) than existing self-
distillation methods. When combined with a contrastive loss, DPS achieves state-of-the-art robustness under label noise.
Read the full abstract ▾
Computer Vision
Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset
TsaiChing Ni, ZhenQi Chen and YuanFu Yang
We present IMDD-1M, the first large-scale Industrial
Multimodal Defect
Dataset comprising 1,000,000 aligned image-text pairs, designed to advance
multimodal learning for manufacturing and quality inspection. IMDD-1M contains high-resolution real-world defects spanning over 60 material categories and more than 400 defect types, each accompanied by expert-verified annotations and fine-grained textual descriptions detailing defect location, severity, and contextual attributes. This
dataset enables a wide spectrum of applications, including classification,
segmentation,
retrieval, captioning, and generative modeling. Building upon IMDD-1M, we train a
diffusion-based
vision-
language foundation model from scratch, specifically tailored for industrial scenarios. The model serves as a generalizable foundation that can be efficiently adapted to specialized domains through lightweight fine-tuning. With less than 5% of the task-specific data required by dedicated expert models, it achieves comparable performance, highlighting the potential of data-efficient foundation model adaptation for industrial inspection and generation, paving the way for scalable, domain-adaptive, and knowledge-grounded manufacturing intelligence. Additional details and resources can be found in this URL: https://ninaneon.github.io/projectpage/
Read the full abstract ▾
Computer Vision
Taming Preference Mode Collapse via Directional Decoupling Alignment in Diffusion Reinforcement Learning
Chubin Chen, Sujie Hu, Jiashu Zhu et al. (+8 more)
Recent studies have demonstrated significant progress in aligning text-to-image
diffusion models with human preference via Reinforcement Learning from Human Feedback. However, while existing methods achieve high scores on automated reward metrics, they often lead to Preference Mode Collapse (PMC)-a specific form of reward hacking where models converge on narrow, high-scoring outputs (e.g., images with monolithic styles or pervasive overexposure), severely degrading generative diversity. In this work, we introduce and quantify this phenomenon, proposing DivGenBench, a novel
benchmark designed to measure the extent of PMC. We posit that this collapse is driven by over-optimization along the reward model's inherent biases. Building on this analysis, we propose Directional Decoupling
Alignment (D$^2$-Align), a novel framework that mitigates PMC by directionally correcting the reward signal. Specifically, our method first learns a directional correction within the reward model's
embedding space while keeping the model frozen. This correction is then applied to the reward signal during the optimization process, preventing the model from collapsing into specific modes and thereby maintaining diversity. Our comprehensive evaluation, combining qualitative analysis with quantitative metrics for both quality and diversity, reveals that D$^2$-Align achieves superior
alignment with human preference.
Read the full abstract ▾
All authors Chubin Chen, Sujie Hu, Jiashu Zhu, Meiqi Wu, Jintao Chen, Yanxun Li, Nisha Huang, Chengyu Fang, Jiahong Wu, Xiangxiang Chu, Xiu Li
Computer Vision · Cross-listed from cs.LG
GARDO: Reinforcing Diffusion Models without Reward Hacking
Haoran He, Yuxiao Ye, Jie Liu et al. (+7 more)
Fine-tuning
diffusion models via online reinforcement learning (RL) has shown great potential for enhancing text-to-image
alignment. However, since precisely specifying a ground-truth objective for visual tasks remains challenging, the models are often optimized using a proxy reward that only partially captures the true goal. This mismatch often leads to reward hacking, where proxy scores increase while real image quality deteriorates and generation diversity collapses. While common solutions add regularization against the reference policy to prevent reward hacking, they compromise sample efficiency and impede the exploration of novel, high-reward regions, as the reference policy is usually sub-optimal. To address the competing demands of sample efficiency, effective exploration, and mitigation of reward hacking, we propose Gated and Adaptive Regularization with Diversity-aware Optimization (GARDO), a versatile framework compatible with various RL
algorithms. Our key insight is that regularization need not be applied universally; instead, it is highly effective to selectively penalize a subset of samples that exhibit high uncertainty. To address the exploration challenge, GARDO introduces an adaptive regularization mechanism wherein the reference model is periodically updated to match the capabilities of the online policy, ensuring a relevant regularization target. To address the mode collapse issue in RL, GARDO amplifies the rewards for high-quality samples that also exhibit high diversity, encouraging mode coverage without destabilizing the optimization process. Extensive experiments across diverse proxy rewards and hold-out unseen metrics consistently show that GARDO mitigates reward hacking and enhances generation diversity without sacrificing sample efficiency or exploration, highlighting its effectiveness and robustness.
Read the full abstract ▾
All authors Haoran He, Yuxiao Ye, Jie Liu, Jiajun Liang, Zhiyong Wang, Ziyang Yuan, Xintao Wang, Hangyu Mao, Pengfei Wan, Ling Pan
Computer Vision
Enhancing LLM-Based Neural Network Generation: Few-Shot Prompting and Efficient Validation for Automated Architecture Design
Raghuvir Duvvuri, Chandini Vysyaraju, Avi Goyal et al. (+2 more)
Automated neural network architecture design remains a significant challenge in computer
vision. Task diversity and computational constraints require both effective architectures and efficient search methods. Large
Language Models (LLMs) present a promising alternative to computationally intensive Neural Architecture Search (NAS), but their application to architecture generation in computer
vision has not been systematically studied, particularly regarding
prompt engineering and validation strategies. Building on the task-agnostic NNGPT/LEMUR framework, this work introduces and validates two key contributions for computer
vision. First, we present Few-Shot Architecture Prompting (FSAP), the first systematic study of the number of supporting examples (n = 1, 2, 3, 4, 5, 6) for LLM-based architecture generation. We find that using n = 3 examples best balances architectural diversity and context focus for
vision tasks. Second, we introduce Whitespace-Normalized Hash Validation, a lightweight deduplication method (less than 1 ms) that provides a 100x speedup over AST parsing and prevents redundant training of duplicate computer
vision architectures. In large-scale experiments across seven computer
vision benchmarks (MNIST, CIFAR-10, CIFAR-100, CelebA, ImageNette, SVHN, Places365), we generated 1,900 unique architectures. We also introduce a
dataset-balanced evaluation methodology to address the challenge of comparing architectures across heterogeneous
vision tasks. These contributions provide actionable guidelines for LLM-based architecture search in computer
vision and establish rigorous evaluation practices, making automated design more accessible to researchers with limited computational resources.
Read the full abstract ▾
All authors Raghuvir Duvvuri, Chandini Vysyaraju, Avi Goyal, Dmitry Ignatov, Radu Timofte
Computer Vision
GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation
Yuan Feng, Yue Yang, Xiaohan He et al. (+8 more)
Geometric problem solving constitutes a critical branch of mathematical
reasoning, requiring precise analysis of shapes and spatial relationships. Current evaluations of
geometric reasoning in
vision-
language models (VLMs) face limitations, including the risk of test data contamination from textbook-based
benchmarks, overemphasis on final answers over
reasoning processes, and insufficient diagnostic granularity. To address these issues, we present GeoBench, a hierarchical
benchmark featuring four
reasoning levels in
geometric problem-solving: Visual Perception, Goal-Oriented Planning, Rigorous
Theorem Application, and Self-Reflective Backtracking. Through six formally verified tasks generated via TrustGeoGen, we systematically assess capabilities ranging from attribute extraction to logical error correction. Experiments reveal that while
reasoning models like OpenAI-o3 outperform general MLLMs, performance declines significantly with increasing task complexity. Key findings demonstrate that sub-goal decomposition and irrelevant premise filtering critically influence final problem-solving accuracy, whereas Chain-of-Thought prompting unexpectedly degrades performance in some tasks. These findings establish GeoBench as a comprehensive
benchmark while offering actionable guidelines for developing
geometric problem-solving systems.
Read the full abstract ▾
All authors Yuan Feng, Yue Yang, Xiaohan He, Jiatong Zhao, Jianlong Chen, Zijun Chen, Daocheng Fu, Qi Liu, Renqiu Xia, Bo Zhang, Junchi Yan
Computer Vision · Cross-listed from eess.IV
Targeted Semantic Segmentation of Himalayan Glacial Lakes Using Time-Series SAR: Towards Automated GLOF Early Warning
Pawan Adhikari, Satish Raj Regmi and Hari Ram Shrestha
Glacial Lake Outburst Floods (GLOFs) are one of the most devastating
climate change induced hazards. Existing remote monitoring approaches often prioritise maximising spatial coverage to train generalistic models or rely on optical imagery hampered by persistent cloud coverage. This paper presents an end-to-end, automated deep learning pipeline for the targeted monitoring of high-risk Himalayan glacial lakes using time-series Sentinel-1 SAR. We introduce a "temporal-first" training strategy, utilising a U-Net with an EfficientNet-B3 backbone trained on a curated
dataset of a cohort of 4 lakes (Tsho Rolpa, Chamlang Tsho, Tilicho and Gokyo Lake). The model achieves an IoU of 0.9130 validating the success and efficacy of the "temporal-first" strategy required for transitioning to Early Warning Systems. Beyond the model, we propose an operational engineering architecture: a Dockerised pipeline that automates data ingestion via the ASF Search API and exposes inference results via a RESTful endpoint. This system shifts the paradigm from static mapping to dynamic and automated early warning, providing a scalable architectural foundation for future development in Early Warning Systems.
Read the full abstract ▾
Computer Vision
Guided Diffusion-based Generation of Adversarial Objects for Real-World Monocular Depth Estimation Attacks
Yongtao Chen, Yanbo Wang, Wentao Zhao et al. (+3 more)
Monocular Depth Estimation (MDE) serves as a core perception module in autonomous driving systems, but it remains highly susceptible to
adversarial attacks. Errors in depth estimation may propagate through downstream decision making and influence overall traffic safety. Existing physical attacks primarily rely on texture-based patches, which impose strict placement constraints and exhibit limited realism, thereby reducing their effectiveness in complex driving environments. To overcome these limitations, this work introduces a training-free generative
adversarial attack framework that generates naturalistic, scene-consistent
adversarial objects via a
diffusion-based conditional generation process. The framework incorporates a Salient Region Selection module that identifies regions most influential to MDE and a Jacobian Vector Product Guidance mechanism that steers
adversarial gradients toward update directions supported by the pre-trained
diffusion model. This formulation enables the generation of physically plausible
adversarial objects capable of inducing substantial
adversarial depth shifts. Extensive digital and physical experiments demonstrate that our method significantly outperforms existing attacks in effectiveness, stealthiness, and physical deployability, underscoring its strong practical implications for autonomous driving safety assessment.
Read the full abstract ▾
All authors Yongtao Chen, Yanbo Wang, Wentao Zhao, Guole Shen, Tianchen Deng, Jingchuan Wang
Computer Vision
Think Before You Move: Latent Motion Reasoning for Text-to-Motion Generation
Yijie Qian, Juncheng Wang, Yuxiang Feng et al. (+7 more)
Current state-of-the-art paradigms predominantly treat Text-to-Motion (T2M) generation as a direct translation problem, mapping symbolic
language directly to continuous poses. While effective for simple actions, this System 1 approach faces a fundamental theoretical bottleneck we identify as the Semantic-Kinematic Impedance Mismatch: the inherent difficulty of grounding semantically dense, discrete linguistic intent into kinematically dense, high-frequency motion data in a single shot. In this paper, we argue that the solution lies in an architectural shift towards Latent System 2
Reasoning. Drawing inspiration from Hierarchical Motor Control in cognitive science, we propose Latent Motion
Reasoning (LMR) that reformulates generation as a two-stage Think-then-Act decision process. Central to LMR is a novel Dual-Granularity Tokenizer that disentangles motion into two distinct
manifolds: a compressed, semantically rich
Reasoning Latent for planning global
topology, and a high-frequency Execution Latent for preserving physical fidelity. By forcing the model to autoregressively reason (plan the coarse trajectory) before it moves (instantiates the frames), we effectively bridge the ineffability gap between
language and physics. We demonstrate LMR's versatility by implementing it for two representative baselines: T2M-GPT (discrete) and MotionStreamer (continuous). Extensive experiments show that LMR yields non-trivial improvements in both semantic
alignment and physical plausibility, validating that the optimal substrate for motion planning is not natural
language, but a learned, motion-aligned concept space. Codes and demos can be found in \hyperlink{https://chenhaoqcdyq.github.io/LMR/}{https://chenhaoqcdyq.github.io/LMR/}
Read the full abstract ▾
All authors Yijie Qian, Juncheng Wang, Yuxiang Feng, Chao Xu, Wang Lu, Yang Liu, Baigui Sun, Yiqiang Chen, Yong Liu, Shujun Wang