Computer Vision · Cross-listed from cs.GR
Turbo4DGen: Ultra-Fast Acceleration for 4D Generation
Yuanbin Man, Ying Huang, Zhile Ren et al. (+1 more)
4D generation, or dynamic 3D content generation, integrates spatial, temporal, and view dimensions to model realistic dynamic scenes, playing a foundational role in advancing world models and physical AI. However, maintaining long-chain consistency across both frames and viewpoints through the unique spatio-camera-motion (SCM)
attention mechanism introduces substantial computational and
memory overhead, often leading to out-of-
memory (OOM) failures and prohibitive generation times. To address these challenges, we propose Turbo4DGen, an ultra-fast acceleration framework for
diffusion-based multi-view 4D content generation. Turbo4DGen introduces a spatiotemporal cache mechanism that persistently reuses intermediate
attention across denoising steps, combined with dynamically semantic-aware
attention pruning and an adaptive SCM chain bypass scheduler, to drastically reduce redundant SCM
attention computation. Our experimental results show that Turbo4DGen achieves an average 9.7$\times$ speedup without quality degradation on the ObjaverseDy and Consistent4D
datasets. To the best of our knowledge, Turbo4DGen is the first dedicated acceleration framework for 4D generation.
Read the full abstract ▾
All authors Yuanbin Man, Ying Huang, Zhile Ren, Miao Yin
Computer Vision
Lost in Translation: How Language Re-Aligns Vision for Cross-Species Pathology
Ekansh Arora
Foundation models are increasingly applied to computational pathology, yet their behavior under cross-cancer and cross-species transfer remains unspecified. This study investigated how fine-tuning CPath-CLIP affects cancer detection under same-cancer, cross-cancer, and cross-species conditions using whole-slide image patches from canine and human histopathology. Performance was measured using area under the receiver operating characteristic curve (AUC). Few-shot fine-tuning improved same-cancer (64.9% to 72.6% AUC) and cross-cancer performance (56.84% to 66.31% AUC). Cross-species evaluation revealed that while tissue matching enables meaningful transfer, performance remains below state-of-the-art
benchmarks (H-optimus-0: 84.97% AUC), indicating that standard
vision-
language alignment is suboptimal for cross-species generalization.
Embedding space analysis revealed extremely high cosine similarity (greater than 0.99) between tumor and normal prototypes. Grad-CAM shows prototype-based models remain domain-locked, while
language-guided models attend to conserved tumor morphology. To address this, we introduce Semantic Anchoring, which uses
language to provide a stable coordinate system for visual features. Ablation studies reveal that benefits stem from the text-
alignment mechanism itself, regardless of text encoder complexity. Benchmarking against H-optimus-0 shows that CPath-CLIP's failure stems from intrinsic
embedding collapse, which text
alignment effectively circumvents. Additional gains were observed in same-cancer (8.52%) and cross-cancer classification (5.67%). We identified a previously uncharacterized failure mode: semantic collapse driven by species-dominated
alignment rather than missing visual information. These results demonstrate that
language acts as a control mechanism, enabling semantic re-interpretation without retraining.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Frequent Pattern Mining approach to Image Compression
Avinash Kadimisetty, C. Oswald and B. Sivalselvan
The paper focuses on Image Compression, explaining efficient approaches based on Frequent Pattern Mining(FPM). The proposed compression mechanism is based on clustering similar pixels in the image and thus using cluster identifiers in image compression. Redundant data in the image is effectively handled by replacing the DCT phase of conventional JPEG through a mixture of k-means Clustering and Closed Frequent Sequence Mining. To optimize the cardinality of pattern(s) in encoding, efficient
pruning techniques have been used through the refinement of Conventional Generalized Sequential Pattern Mining(GSP)
algorithm. We have proposed a mechanism for finding the frequency of a sequence which will yield significant reduction in the code table size. The
algorithm is tested by compressing
benchmark datasets yielding an improvement of 45% in compression ratios, often outperforming the existing alternatives. PSNR and SSIM, which are the image quality metrics, have been tested which show a negligible loss in visual quality.
Read the full abstract ▾
Computer Vision
Mirage2Matter: A Physically Grounded Gaussian World Model from Video
Zhengqing Gao, Ziwen Li, Xin Wang et al. (+12 more)
The scalability of
embodied intelligence is fundamentally constrained by the scarcity of real-world interaction data. While simulation platforms provide a promising alternative, existing approaches often suffer from a substantial visual and physical gap to real environments and rely on expensive sensors, precise
robot calibration, or depth measurements, limiting their practicality at scale. We present Simulate Anything, a graphics-driven world modeling and simulation framework that enables efficient generation of high-fidelity
embodied training data using only multi-view environment
videos and off-the-shelf assets. Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS), seamlessly capturing fine-grained geometry and appearance from
video. We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target, enabling accurate scale
alignment between the reconstructed scene and the real world. Together, these components provide a unified, editable, and physically grounded world model.
Vision Language Action (VLA) models trained on our simulated data achieve strong zero-shot performance on downstream tasks, matching or even surpassing results obtained with real-world data, highlighting the potential of reconstruction-driven world modeling for scalable and practical
embodied intelligence training.
Read the full abstract ▾
All authors Zhengqing Gao, Ziwen Li, Xin Wang, Jiaxin Huang, Zhenyang Ren, Mingkai Shao, Hanlue Zhang, Tianyu Huang, Yongkang Cheng, Yandong Guo, Runqi Lin, Yuanyuan Wang, Tongliang Liu, Kun Zhang, Mingming Gong
Computer Vision · Cross-listed from cs.LG
NCSAM Noise-Compensated Sharpness-Aware Minimization for Noisy Label Learning
Jiayu Xu and Junbiao Pang
Learning from Noisy Labels (LNL) remains a fundamental challenge in deep learning because real-world
datasets often contain corrupted annotations. Most existing methods rely on label correction or sample selection mechanisms. In contrast, we study LNL from an optimization perspective by establishing a theoretical connection between label noise and the flatness-seeking behavior of Sharpness-Aware Minimization (SAM). Based on this analysis, we propose Noise-Compensated Sharpness-Aware Minimization (NCSAM), which uses a noise-compensated perturbation to counteract the optimization bias induced by noisy labels. By correcting distorted SAM perturbations, NCSAM mitigates the memorization of noisy labels during training while preserving the simplicity of optimization-based learning. Experiments on synthetic and real-world noisy-label
benchmarks show that NCSAM consistently improves over SAM-based optimization baselines and remains competitive with representative noisy-label learning methods.
Read the full abstract ▾
Computer Vision
Stylizing ViT: Anatomy-Preserving Instance Style Transfer for Domain Generalization
Sebastian Doerrich, Francesco Di Salvo, Jonas Alle et al. (+1 more)
Deep learning models in medical image analysis often struggle with generalizability across domains and demographic groups due to data heterogeneity and scarcity. Traditional augmentation improves robustness, but fails under substantial domain shifts. Recent advances in stylistic augmentation enhance domain generalization by varying image styles but fall short in terms of style diversity or by introducing artifacts into the generated images. To address these limitations, we propose Stylizing ViT, a novel
Vision Transformer encoder that utilizes weight-shared
attention blocks for both self- and cross-
attention. This design allows the same
attention block to maintain anatomical consistency through self-
attention while performing style transfer via cross-
attention. We assess the effectiveness of our method for domain generalization by employing it for data augmentation on three distinct image classification tasks in the context of histopathology and dermatology. Results demonstrate an improved robustness (up to +13% accuracy) over the state of the art while generating perceptually convincing images without artifacts. Additionally, we show that Stylizing ViT is effective beyond training, achieving a 17% performance improvement during inference when used for test-time augmentation. The source code is available at https://github.com/sdoerrich97/stylizing-vit .
Read the full abstract ▾
All authors Sebastian Doerrich, Francesco Di Salvo, Jonas Alle, Christian Ledig
Computer Vision · Cross-listed from eess.SP
ME-WARD: A multimodal ergonomic analysis tool for musculoskeletal risk assessment from inertial and video data in working plac
Javier González-Alonso, Paula Martín-Tapia, David González-Ortega et al. (+3 more)
This study presents ME-WARD (
Multimodal Ergonomic Workplace Assessment and Risk from Data), a novel system for ergonomic assessment and musculoskeletal risk evaluation that implements the Rapid Upper Limb Assessment (RULA) method. ME-WARD is designed to process joint angle data from motion capture systems, including inertial measurement unit (IMU)-based setups, and deep learning human body pose tracking models. The tool's flexibility enables ergonomic risk assessment using any system capable of reliably measuring joint angles, extending the applicability of RULA beyond proprietary setups. To validate its performance, the tool was tested in an industrial setting during the assembly of conveyor belts, which involved high-risk tasks such as inserting rods and pushing conveyor belt components. The experiments leveraged gold standard IMU systems alongside a state-of-the-art
monocular 3D pose estimation system. The results confirmed that ME-WARD produces reliable RULA scores that closely align with IMU-derived metrics for flexion-dominated movements and comparable performance with the
monocular system, despite limitations in tracking lateral and rotational motions. This work highlights the potential of integrating multiple motion capture technologies into a unified and accessible ergonomic assessment pipeline. By supporting diverse input sources, including low-cost
video-based systems, the proposed
multimodal approach offers a scalable, cost-effective solution for ergonomic assessments, paving the way for broader adoption in resource-constrained industrial environments.
Read the full abstract ▾
All authors Javier González-Alonso, Paula Martín-Tapia, David González-Ortega, Míriam Antón-Rodríguez, Francisco Javier Díaz-Pernas, Mario Martínez-Zarzuela
Computer Vision
Sponge Tool Attack: Stealthy Denial-of-Efficiency against Tool-Augmented Agentic Reasoning
Qi Li and Xinchao Wang
Enabling large
language models (LLMs) to solve complex
reasoning tasks is a key step toward artificial general intelligence. Recent work augments LLMs with external tools to enable agentic
reasoning, achieving high utility and efficiency in a plug-and-play manner. However, the inherent vulnerabilities of such methods to malicious manipulation of the tool-calling process remain largely unexplored. In this work, we identify a tool-specific attack surface and propose Sponge Tool Attack (STA), which disrupts agentic
reasoning solely by rewriting the input
prompt under a strict query-only access assumption. Without any modification on the underlying model or the external tools, STA converts originally concise and efficient
reasoning trajectories into unnecessarily verbose and convoluted ones before arriving at the final answer. This results in substantial computational overhead while remaining stealthy by preserving the original task semantics and user intent. To achieve this, we design STA as an iterative, multi-
agent collaborative framework with explicit rewritten policy control, and generates benign-looking
prompt rewrites from the original one with high semantic fidelity. Extensive experiments across 6 models (including both open-source models and closed-source APIs), 12 tools, 4 agentic frameworks, and 13
datasets spanning 5 domains validate the effectiveness of STA.
Read the full abstract ▾
Computer Vision
Saliency Driven Imagery Preprocessing for Efficient Compression – Industrial Paper
Justin Downes, Sam Saltwick and Anthony Chen
The compression of satellite imagery remains an important research area as hundreds of terabytes of images are collected every day, which drives up storage and bandwidth costs. Although progress has been made in increasing the resolution of these satellite images, many downstream tasks are only interested in small regions of any given image. These areas of interest vary by task but, once known, can be used to optimize how information within the image is encoded. Whereas standard image encoding methods, even those optimized for remote sensing, work on the whole image equally, there are emerging methods that can be guided by saliency maps to focus on important areas. In this work we show how imagery preprocessing techniques driven by saliency maps can be used with traditional lossy compression coding standards to create variable rate image compression within a single large satellite image. Specifically, we use variable sized smoothing kernels that map to different quantized saliency levels to process imagery pixels in order to optimize downstream compression and encoding schemes.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
In-situ On-demand Digital Image Correlation: A New Data-rich Characterization Paradigm for Deformation and Damage Development in Solids
Ravi Venkata Surya Sai Mogilisetti, Partha Pratim Das, Rassel Raihan et al. (+1 more)
Digital image correlation (DIC) has become one of the most popular methods for deformation characterization in experimental mechanics. DIC is based on optical images taken during experimentation and post-test image processing. Its advantages include the capability to capture full-field deformation in a non-contact manner, the robustness in characterizing excessive deformation induced by events such as yielding and cracking, and the versatility to integrate optical cameras with a variety of open-source and commercial codes. In this paper, we developed a new paradigm of DIC analysis by integrating camera control into the DIC process flow. The essential idea is to dynamically increase the camera imaging frame rate with excessive deformation or deformation rate, while maintaining a relatively low imaging frame rate with small and slow deformation. We refer to this new DIC paradigm as in-situ on-demand (ISOD) DIC. ISOD DIC enables real-time deformation analysis, visualization, and closed-loop camera control. ISOD DIC has captured approximately 178% more images than conventional DIC for samples undergoing crack growth due to its dynamically adjusted frame rate, with the potential to significantly enhance data richness for damage inspection without consuming excessive storage space and analysis time, thereby benefiting the characterization of intrinsic constitutive behaviors and damage mechanisms
Read the full abstract ▾
All authors Ravi Venkata Surya Sai Mogilisetti, Partha Pratim Das, Rassel Raihan, Shiyao Lin
Computer Vision
OTI: A Model-free and Visually Interpretable Measure of Image Attackability
Jiaming Liang, Haowei Liu and Chi-Man Pun
Despite the tremendous success of neural networks, benign images can be corrupted by
adversarial perturbations to deceive these models. Intriguingly, images differ in their attackability. Specifically, given an attack configuration, some images are easily corrupted, whereas others are more resistant. Evaluating image attackability has important applications in active learning,
adversarial training, and attack enhancement. This
prompts a growing interest in developing attackability measures. However, existing methods are scarce and suffer from two major limitations: (1) They rely on a model proxy to provide prior knowledge (e.g., gradients or minimal perturbation) to extract model-dependent image features. Unfortunately, in practice, many task-specific models are not readily accessible. (2) Extracted features characterizing image attackability lack visual interpretability, obscuring their direct relationship with the images. To address these, we propose a novel Object Texture Intensity (OTI), a model-free and visually interpretable measure of image attackability, which measures image attackability as the texture intensity of the image's semantic object. Theoretically, we describe the principles of OTI from the perspectives of decision boundaries as well as the mid- and high-frequency characteristics of
adversarial perturbations. Comprehensive experiments demonstrate that OTI is effective and computationally efficient. In addition, our OTI provides the
adversarial machine learning community with a visual understanding of attackability.
Read the full abstract ▾
Computer Vision
Will It Zero-Shot?: Predicting Zero-Shot Classification Performance For Arbitrary Queries
Kevin Robbins, Xiaotong Liu, Yu Wu et al. (+4 more)
Vision-
Language Models like CLIP create aligned
embedding spaces for text and images, making it possible for anyone to build a visual classifier by simply naming the classes they want to distinguish. However, a model that works well in one domain may fail in another, and non-expert users have no straightforward way to assess whether their chosen VLM will work on their problem. We build on prior work using text-only comparisons to evaluate how well a model works for a given natural
language task, and explore approaches that also generate synthetic images relevant to that task to evaluate and refine the prediction of zero-shot accuracy. We show that generated imagery to the baseline text-only scores substantially improves the quality of these predictions. Additionally, it gives a user feedback on the kinds of images that were used to make the assessment. Experiments on standard CLIP
benchmark datasets demonstrate that the image-based approach helps users predict, without any labeled examples, whether a VLM will be effective for their application.
Read the full abstract ▾
All authors Kevin Robbins, Xiaotong Liu, Yu Wu, Le Sun, Grady McPeak, Abby Stylianou, Robert Pless
Computer Vision
FMIR, a foundation model-based Image Registration Framework for Robust Image Registration
Fengting Zhang, Yue He, Qinghao Liu et al. (+3 more)
Deep learning has revolutionized medical image registration by achieving unprecedented speeds, yet its clinical application is hindered by a limited ability to generalize beyond the training domain, a critical weakness given the typically small scale of medical
datasets. In this paper, we introduce FMIR, a foundation model-based registration framework that overcomes this limitation.Combining a foundation model-based feature encoder for extracting anatomical structures with a general registration head, and trained with a channel regularization strategy on just a single
dataset, FMIR achieves state-of-the-art(SOTA) in-domain performance while maintaining robust registration on out-of-domain images.Our approach demonstrates a viable path toward building generalizable medical imaging foundation models with limited resources. The code is available at https://github.com/Monday0328/FMIR.git.
Read the full abstract ▾
All authors Fengting Zhang, Yue He, Qinghao Liu, Yaonan Wang, Xiang Chen, Hang Zhang
Computer Vision
BMDS-Net:Deployment-aware multi-modal brain tumor segmentation with adaptive fusion,decoder regularization,and Bayesian calibration
Yan Zhou, Suncheng Xiang, Zhen Huang et al. (+3 more)
Multi-modal MRI enables detailed
brain tumor sub-region
segmentation, but clinical deployment remains affected by missing sequences,boundary errors, and overconfident predictions. We present BMDS-Net, a two-stage Swin UNETR framework that combines adaptivemodality fusion, boundary-aware decoder regularization, and efficient
Bayesian calibration. The deterministic stage adds zero-initializedmultimodal contextual fusion (MMCF) and residual-gated deep decoder supervision (DDS) to modulate modality contributions andregularize hierarchical boundary reconstruction. The second stage replaces only the final convolutional layer with a
Bayesian layer,enabling Monte Carlo uncertainty estimates with limited additional training cost. On BraTS 2021 (1,251 cases), BMDS-Net achieved Dice scores of 0.9293, 0.9098, and 0.8675 for WT, TC, and ET, with HD95 values of 2.27, 2.22, and 3.27 mm. Configuration comparisons showed that auxiliary supervision favored complete-input
segmentation, whereas the complete MMCF+gated-DDS configuration was most robust under single-modality removal.
Bayesian BMDS-Net achieved an expected calibration error of 0.0037, close to a three-model ensemble (0.0035) at approximately 60% lower training cost. On BraTS 2020, the same protocol preserved the leading internal ranking of the complete configuration among five Swin UNETR-controlled variants. The implementation code is available at https://github.com/RyanZhou168/BMDS-Net.
Read the full abstract ▾
All authors Yan Zhou, Suncheng Xiang, Zhen Huang, Yue Ouyang, Yingqiu Li, Zehua Wang
Computer Vision · Cross-listed from cs.LG
SpatialMath: Spatial Comprehension-Infused Symbolic Reasoning for Mathematical Problem-Solving
Ashutosh Bajpai, Akshat Bhandari, Akshay Nambi et al. (+1 more)
Multimodal Small-to-Medium sized
Language Models (MSLMs) have demonstrated strong capabilities in integrating visual and textual information but still face significant limitations in visual comprehension and mathematical
reasoning, particularly in
geometric problems with diverse levels of visual infusion. Current models struggle to accurately decompose intricate visual inputs and connect perception with structured
reasoning, leading to suboptimal performance. To address these challenges, we propose SpatialMath, a novel Spatial Comprehension-Infused Symbolic
Reasoning Framework designed to integrate spatial representations into structured symbolic
reasoning chains. SpatialMath employs a specialized perception module to extract spatially-grounded representations from visual diagrams, capturing critical
geometric structures and spatial relationships. These representations are then methodically infused into symbolic
reasoning chains, facilitating visual comprehension-aware structured
reasoning. To this end, we introduce MATHVERSE-PLUS, a novel
dataset containing structured visual interpretations and step-by-step
reasoning paths for
vision-intensive mathematical problems. SpatialMath significantly outperforms strong
multimodal baselines, achieving up to 10 percentage points improvement over supervised fine-tuning with data augmentation in
vision-intensive settings. Robustness analysis reveals that enhanced spatial representations directly improve
reasoning accuracy, reinforcing the need for structured perception-to-
reasoning pipelines in MSLMs.
Read the full abstract ▾
All authors Ashutosh Bajpai, Akshat Bhandari, Akshay Nambi, Tanmoy Chakraborty
Computer Vision
PhaSR: Generalized Image Shadow Removal with Physically Aligned Priors
Chia-Ming Lee, Yu-Fan Lin, Yu-Jou Hsiao et al. (+3 more)
Shadow removal under diverse lighting conditions requires disentangling illumination from intrinsic reflectance, a challenge compounded when physical priors are not properly aligned. We propose PhaSR (Physically Aligned Shadow Removal), addressing this through dual-level prior
alignment to enable robust performance from single-light shadows to multi-source ambient lighting. First, Physically Aligned Normalization (PAN) performs closed-form illumination correction via Gray-world normalization, log-domain Retinex decomposition, and dynamic range recombination, suppressing chromatic bias. Second,
Geometric-Semantic Rectification
Attention (GSRA) extends differential
attention to cross-modal
alignment, harmonizing depth-derived geometry with DINO-v2 semantic
embeddings to resolve modal conflicts under varying illumination. Experiments show competitive performance in shadow removal with lower complexity and generalization to ambient lighting where traditional methods fail under multi-source illumination. Our source code is available at https://github.com/ming053l/PhaSR.
Read the full abstract ▾
All authors Chia-Ming Lee, Yu-Fan Lin, Yu-Jou Hsiao, Jin-Hui Jiang, Yu-Lun Liu, Chih-Chung Hsu
Computer Vision
ReflexSplit: Single Image Reflection Separation via Layer Fusion-Separation
Chia-Ming Lee, Yu-Fan Lin, Jin-Hui Jiang et al. (+3 more)
Single Image Reflection Separation (SIRS) disentangles mixed images into transmission and reflection layers. Existing methods suffer from transmission-reflection confusion under nonlinear mixing, particularly in deep decoder layers, due to implicit fusion mechanisms and inadequate multi-scale coordination. We propose ReflexSplit, a dual-stream framework with three key innovations. (1) Cross-scale Gated Fusion (CrGF) adaptively aggregates semantic priors, texture details, and decoder context across hierarchical depths, stabilizing gradient flow and maintaining feature consistency. (2) Layer Fusion-Separation Blocks (LFSB) alternate between fusion for shared structure extraction and differential separation for layer-specific disentanglement. Inspired by Differential
Transformer, we extend
attention cancellation to dual-stream separation via cross-stream subtraction. (3) Curriculum training progressively strengthens differential separation through depth-dependent initialization and epoch-wise warmup. Extensive experiments on synthetic and real-world
benchmarks demonstrate state-of-the-art performance with superior perceptual quality and robust generalization. Our code is available at https://github.com/wuw2135/ReflexSplit.
Read the full abstract ▾
All authors Chia-Ming Lee, Yu-Fan Lin, Jin-Hui Jiang, Yu-Jou Hsiao, Chih-Chung Hsu, Yu-Lun Liu