Computer Vision
DeepMine-Mamba: Mitigating Information Dilution in Mamba-Based State Space Models for Document Image Binarization
Sheng-Wei Chan, Yung-Che Wang, Hsin-Jui Pan et al. (+2 more)
Document image binarization must preserve thin, broken, and low-contrast strokes against degraded backgrounds; Mamba-based state space models have so far been explored only preliminarily for it. We measure what state-space propagation does to document features and find that it does not attenuate foreground responses: every region gains amplitude, but the background gain is 1.8 times the stroke gain, compressing the foreground-background amplitude contrast from 1.86 to 1.03. This compression scales monotonically with the effective receptive field, suggesting it is not specific to state space models. Guided by this, we propose DeepMine-Mamba, whose Anti-Dilution Gate computes its modulation from the propagation-induced feature difference and confines it to stroke regions by a learned stroke prior. Under a strict leave-one-year-out protocol on DIBCO/H-DIBCO with no external training data it is competitive with recent
transformer- and gated-convolution baselines; because published protocols differ and are often unstated, we report protocol-controlled evidence separately from cross-paper comparison. Ablations under a fixed objective show that the gate recovers stroke-background discriminability and improves all four metrics, including the two absent from the objective.
Read the full abstract ▾
All authors Sheng-Wei Chan, Yung-Che Wang, Hsin-Jui Pan, Chia-Min Lin, Jen-Shiun Chiang
Computer Vision
Beyond Consistency: Preserving Temporal Structure in Zero-Shot Video Editing
Deyin Liu, Yisheng Ding, Zhe Jin et al. (+3 more)
Existing zero-shot
video editing methods rely on pre-trained
diffusion models, successfully achieving spatial control and basic temporal consistency but fundamentally fail to preserve the
video's original temporal structure.This distinction is critical: temporal consistency ensures visual smoothness, but temporal structure dictates the
video's high-level narrative, rhythm, and semantic flow. Without this preservation, the edited output, especially for long
videos with complex semantic variations, becomes narratively incoherent and semantically ambiguous. To address this limitation, we introduce a novel zero-shot editing approach that, for the first time, explicitly focuses on preserving the source
video's temporal structure. We achieve this by adaptively partitioning the
video into semantically distinct clips based on feature similarity and selecting a representative anchor frame for each clip. To enhance both intra-clip fidelity and computational efficiency, we design a clip-adaptive
token merging strategy which leverages the anchor's semantic dominance to stabilize the editing. Furthermore, we employ an alternating combination strategy that ensures seamless inter-clip transitions while maintaining semantic distinction. Extensive experiments demonstrate that our method achieves state-of-the-art results, successfully balancing the preservation of original temporal structure with computational efficiency, and setting a new
benchmark for zero-shot
video editing fidelity.
Read the full abstract ▾
All authors Deyin Liu, Yisheng Ding, Zhe Jin, Xiatian Zhu, Anjan Dutta, Lin Wu
Computer Vision · Cross-listed from cs.CL
TeamHerald@CHIPSAL 2026: Hate Speech Detection and Sentiment Analysis of Nepali Memes using Transformer-based Architectures and Ensemble Learning
Ashish Acharya, Anish Khatiwada, Rohit Khadka et al. (+1 more)
The analysis of internet memes in the Nepali
language is complicated by frequent code-mixing and a lack of established baseline resources. While memes inherently combine visual and textual elements, this study focuses on a text-centric approach by extracting embedded text using an OCR layer and modeling it with
Transformer-based architectures. We evaluate six distinct models and investigate the comparative effectiveness of Hard and Soft Voting ensemble strategies across two tasks: binary hate
speech detection and three-class sentiment analysis. Experimental results show that a standalone decoder-only model achieved the highest performance for binary classification, whereas the Soft Voting ensemble performed best for the multi-class sentiment task, yielding a 15.8% relative improvement in Macro F1-score over the strongest standalone baseline. These findings suggest that ensemble strategies behave differently across binary and multi-class tasks, highlighting the importance of selecting aggregation methods suited to the classification objective.
Read the full abstract ▾
All authors Ashish Acharya, Anish Khatiwada, Rohit Khadka, Pragya Aryal
Computer Vision · Cross-listed from cs.RO
Tac2Pix: Image-Space Visuo-Tactile Fusion for Dexterous Manipulation
Shengcheng Luo, Kefei Wu, Xiaoying Zhou et al. (+3 more)
Vision and touch are naturally complementary for dexterous manipulation. Yet they are represented in fundamentally different forms: visual observations are dense and spatially structured, whereas tactile measurements are sparse, heterogeneous, and
robot-centric. This mismatch makes it challenging to establish explicit spatial correspondence between the two modalities, particularly when they are fused in latent space. However, each tactile measurement is associated with a known location on the
robot's kinematic chain, offering a localization prior that could bridge this representational gap. This raises a natural question: can this localization prior be used to construct a shared representation that explicitly aligns touch with
vision? We introduce Tac2Pix, an image-space visuo-tactile fusion framework that projects sparse tactile measurements onto the RGB image plane using forward kinematics and camera calibration, then renders them as force-aware saliency maps. By expressing touch in the same pixel space as
vision, Tac2Pix directly leverages pretrained 2D visual representations. We further employ zero-initialized input adaptation to preserve the original visual pathway at the start of policy learning. To assess whether this geometry-based interface yields consistent policy gains, we evaluate Tac2Pix on three simulated and three real-world dexterous manipulation tasks. In simulation, Tac2Pix consistently improves visuo-tactile policy learning across tasks, policy architectures, and visual backbones. In real-world experiments, it improves average success under visual occlusion by 26.7 percentage points over the strongest latent visuo-tactile fusion baseline, with gains persisting under both random and physical occlusions. Together, these results support image space as an effective interface for fusing sparse touch with pretrained visual representations.
Read the full abstract ▾
All authors Shengcheng Luo, Kefei Wu, Xiaoying Zhou, Wanlin Li, Ziyuan Jiao, Chenxi Xiao
Computer Vision
Less Is More: Training-Free Acceleration Framework of 3D Diffusion Models for Low-Count PET Denoising via Global-Local Trajectory Reduction
Yuhan Liu, Scott M. Leonard, Marlee Crews et al. (+5 more)
Accurate quantification and uptake measurement in PET are critical for assessing disease progression and supporting clinical decision-making. While high-count PET provides reliable image quality, the associated radiation dose and prolonged acquisition remain significant clinical concerns, motivating the adoption of low-count protocols.
Diffusion-model-based methods have demonstrated strong potential for restoring low-count PET to near high-count quality, but their iterative sampling procedure becomes prohibitively expensive when applied to high-resolution 3D PET volumes, introducing substantial inference latency that limits practical clinical deployment. To address these challenges, we propose a training-free Global-Local Skipping Strategy that accelerates
diffusion model-based 3D PET denoising while simultaneously improving reconstruction quality. The proposed method is plug-and-play and directly applicable to pre-trained
diffusion models without retraining or architectural modification. Specifically, we introduce: (i) a global denoising step skipping strategy that initializes the reverse
diffusion process from an intermediate denoising step using a noise-consistent transformation of the low-count input, substantially reducing the number of required denoising steps; and (ii) a local feature reuse shortcut that reuses slowly-varying high-level U-Net features across neighboring denoising steps, further reducing per-step computation while preserving image fidelity. We evaluate the proposed approach on multiple PET tracers from in-house and public
datasets, including 18F-FDG PET, 68Ga-DOTATATE PET, and 18F-PSMA PET, demonstrating consistent acceleration of over an order of magnitude alongside improved or comparable reconstruction performance relative to the full-step baseline. Blinded reader studies further confirm enhanced clinical confidence and perceived diagnostic quality.
Read the full abstract ▾
All authors Yuhan Liu, Scott M. Leonard, Marlee Crews, Muhannad Fadhel, Jinkui Hao, Tianqi Chen, Ryan J. Avery, Bo Zhou
Computer Vision
Stain-Aware Wavelet Regularization for Instant Adversarial Purification in Histopathology
Zhe Li and Bernhard Kainz
Deep learning has become prevalent in computational pathology pipelines that support tasks such as cancer screening and digital pathology analysis. However, the susceptibility of neural networks to
adversarial perturbations raises safety concerns for reliable deployment in clinical practice. In histopathological images, this challenge is exacerbated by the difficulty of distinguishing high-frequency
adversarial noise from subtle and diagnostically relevant tissue structures. To address this issue, we propose Stain-Aware Wavelet Regularization (SAWR), an
adversarial purification framework that leverages multi-level wavelet-domain regularization based on Haar transform to hierarchically disentangle
adversarial perturbations from diagnostic structural information. This spectral constraint is further extended to individual histological channels, enabling stain-specific frequency regulation consistent with the biological properties of Hematoxylin and Eosin. Extensive experiments demonstrate that SAWR improves
adversarial robustness by up to 10.69% over the baseline approach, while maintaining texture and spectral fidelity under
adversarial perturbations.
Read the full abstract ▾
Computer Vision
MB-Loc: Multi-planar Bird's-eye-view Localization in outdoor LiDAR scenes
Ayaan Choudhury, Preet Savalia, Anirudh Pydah et al. (+1 more)
Global
LiDAR localization is a fundamental task for autonomous
navigation systems. Recent methods perform Scene Coordinate Regression (SCR) and achieve superior accuracy over Absolute Pose Regression (APR) solutions by predicting dense 3D world coordinates. However, SCR approaches introduce two major bottlenecks: severe computational inefficiency from processing raw 3D geometries and significant performance degradation under varying sensor viewpoints. To address these limitations, we present MB-Loc, a lightweight and viewpoint-robust SCR framework. Instead of relying on heavy 3D convolutions, we project the input
LiDAR scan into a 2.5D Multi-planar Bird's-Eye View (BEV) representation. By slicing the point-cloud along the Z-axis and mapping signed depths into discrete 2D planes, MB-Loc retains essential 3D
geometric structures while exploiting the computational tractability of standard 2D CNNs. To handle the inherent sparsity of outdoor
LiDAR, we introduce a KL-regularized latent bottleneck that explicitly models spatial uncertainty without injecting
stochastic noise. Finally, to ensure rotation robustness, we apply 3D spatial augmentations prior to planar projection, forcing the network to implicitly learn viewpoint-invariant features. We perform extensive experiments on the publicly available NCLT
dataset and demonstrate that our proposed method outperforms the current state-of-the-art. Operating at real-time inference speeds, MB-Loc significantly outperforms traditional 3D-SCR architectures in computational efficiency.
Read the full abstract ▾
All authors Ayaan Choudhury, Preet Savalia, Anirudh Pydah, Avinash Sharma
Computer Vision
AUCp: Pseudo-AUC for Inference Model Selection with Unlabeled Validation Data in Abnormality Detection
Md Mahfuzur Rahman Siddiquee, Fazle Rafsani, Jay Shah et al. (+4 more)
Abnormality detection is a crucial yet challenging task in medical image analysis. Distinguishing abnormalities from normal data by learning to reconstruct normal-only data alleviates the reliance on labeled
datasets. However, many studies, even if unsupervised, rely on a labeled validation set to select the best model for inference from multiple training iterations. For many diseases labeled data are unavailable and substantially time consuming to obtain. To address this, AUCp - a novel metric that supports abnormality detection for unsupervised and self-supervised methods is proposed. Instead of evaluating the realism of reconstructed images to select the best of model for inference, it focuses on actual detection performance and without requiring an annotated test set. Assuming the pseudo ground truth of all unannotated samples in the test set as abnormal/positive and using traditional AUC calculation, AUCp scores are derived. Given a large and representative training set of normal samples, we show mathematical and empirical evidence that model selection using AUCp scores improves disease detection in terms of unsupervised and self-supervised methods over conventional metrics. Using two unsupervised methods for neurologic disease detection and self-supervised methods on diverse
datasets, our results demonstrate that the AUCp score effectively identifies the optimal model for inference, significantly enhancing abnormality and disease detection. The corresponding implementations are available in https://github.com/mahfuzmohammad/AUCp.
Read the full abstract ▾
All authors Md Mahfuzur Rahman Siddiquee, Fazle Rafsani, Jay Shah, Teresa Wu, Catherine D Chong, Todd J Schwedt, Baoxin Li
Computer Vision · Cross-listed from cs.AI
Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery
Syed Rifat Raiyan, Mohsinul Kabir, Hasan Mahmud et al. (+2 more)
Mathematical
reasoning has long served as a stringent test of machine intelligence; over the past decade, it has moved from a niche problem within NLP to one of the most consequential AI frontiers. This survey provides a unified account of the field's
evolution, from early rule-based math word problem (MWP) solvers and template-driven geometry systems, through neural expression generation and LLM prompting, to contemporary
reasoning models, multi-
agent systems, neuro-symbolic
theorem provers, and verified discovery workflows. We organize the landscape along four axes: (i) informal
reasoning over text and diagrams, spanning MWP solving,
multimodal geometry, and VLMs; (ii) formal
reasoning in proof assistants, including autoformalization, tactic prediction, compiler-guided repair, and proof search; (iii) mathematical discovery, where systems propose constructions, improve bounds, or assist attacks on open problems; and (iv) the inference and training-time techniques, including CoT prompting, tool use, process reward models, and RLVR, that increasingly connect generation with verification. We catalog major
benchmarks across grade-school arithmetic, competition mathematics, geometry, formal proving,
multimodal and
multilingual reasoning, and expert evaluation, and we examine
benchmark saturation, contamination, reporting mismatches, and the distinction between pass@1, majority voting, and verifier-assisted pass@$k$. We critically assess failure modes: brittleness under perturbation, reward hacking,
multimodal grounding failures, fragile formalization, and the energy cost of
reasoning-scale inference. Drawing on recent perspectives from working mathematicians, we identify future directions centered on verified-discovery workflows,
reasoning efficiency, and infrastructure to make AI-assisted formalization broadly usable. Companion materials: https://github.com/Starscream-11813/awesome-AI4Math.
Read the full abstract ▾
All authors Syed Rifat Raiyan, Mohsinul Kabir, Hasan Mahmud, Md Kamrul Hasan, Sophia Ananiadou
Computer Vision
Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillation
Yishuo Cai, Jiahui Liu, Yuanxin Liu et al. (+9 more)
”Thinking with Images” has emerged as an effective paradigm for fine-grained visual
reasoning: by explicitly zooming into relevant regions and
reasoning over crops, models can access local evidence that is difficult to recover from a single global image. However, this benefit comes with redundant tool invocations and longer inference traces. Moreover, when such behaviors are learned mainly from outcome reward, the resulting intermediate crops or visual cues can be noisy or fail to faithfully capture task-relevant visual evidence. In this work, we ask whether the
reasoning benefits of ”Thinking with Images” can be internalized through Thinking with Imagination: an internal process that decides where to look and imagines what visual cues closer inspection would reveal without actually invoking tools. We propose Imagine-OPD, an on-policy self-
distillation framework in which a teacher plays the role of a ”Thinking with Images” reasoner during training: it receives privileged zoomed evidence views derived from annotated regions, and supervises the model's own imagination
reasoning trajectories. Imagine-OPD does not require an external teacher or high-quality imagination demonstrations. Experiments on
vision-centric
benchmarks show that Imagine-OPD achieves the best average performance among compared models while significantly reducing inference overhead compared with ”Thinking with Images” methods.
Read the full abstract ▾
All authors Yishuo Cai, Jiahui Liu, Yuanxin Liu, Haobo Deng, Linli Yao, Yuhao Zheng, Kun Ouyang, Zhimo Li, Ziyue Wang, Xu Sun, Haoli Bai, Xiaohui Li
Computer Vision · Cross-listed from cs.LG
SNR-ST-Mix: Sample-specific Neighborhood Regression Mixup for Augmented Spatial Transcriptomics Imputation with Deep Neural Network
Hongyi Yu, Yaoyu Fang, Jiahe Qian et al. (+3 more)
Purpose: Spatial transcriptomics (ST) enables gene expression measurements within the tissue context. However, these measurements are often noisy, low-resolution, and sparsely sampled, which limits the recovery of fine spatial structure. Deep neural networks have become powerful tools for expression imputation from histology, but their performance remains constrained by limited sample sizes and a lack of biologically informed augmentation. Most of the existing augmentation strategies for learning are designed for classification tasks rather than regression, which neglect spatial and transcriptomic relationships, leading to biologically implausible interpolations that hinder prediction performance. Approach: To address these limitations, we propose SNR-ST-Mix, a geometry- and expression-aware data augmentation framework designed specifically for ST data. It constrains mixing to a spot's k-nearest spatial neighbors and adaptively weights interpolation coefficients based on expression similarity, generating augmented samples that preserve local biological structure while ensuring spatial smoothness. This dual conditioning yields synthetic examples that expand the effective training
manifold, promote generalization, and enhance prediction stability under sample-specific training. Results: Extensive experiments with various tissue types demonstrate that SNR-ST-Mix consistently outperforms conventional augmentation methods without requiring architectural changes or additional computation. Conclusions: SNR-ST-Mix provides an effective and biologically principled augmentation strategy for spatial transcriptomics regression tasks. By explicitly leveraging spatial geometry and transcriptomic similarity, it expands the effective training
manifold and improves predictive performance without increasing model complexity.
Read the full abstract ▾
All authors Hongyi Yu, Yaoyu Fang, Jiahe Qian, Xinkun Wang, Lee A. Cooper, Bo Zhou
Computer Vision
PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping
Qiming Li, Tianlun Li, Xiaolong Cheng et al. (+5 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective paradigm for improving the
reasoning capability of Large
Vision-
Language Models (LVLMs). However, existing RLVR methods primarily rely on trajectory-level outcome rewards, which assign identical learning signals across all generated
tokens. This coarse-grained credit assignment is fundamentally mismatched to
multimodal reasoning, where only a sparse subset of
tokens is causally grounded in visual evidence. Consequently, these pivotal perceptual
tokens receive weak supervision and are often overwhelmed by
language priors or
reasoning-template
tokens. To address this limitation, we propose Perception-Reinforced Policy Optimization (PRPO), a
token-level reinforcement learning framework that explicitly identifies and reinforces pivotal perceptual
tokens within long-horizon
multimodal reasoning trajectories. PRPO introduces Robust Visual Dependency (RVD), a principled metric that identifies
tokens whose predictions are both visually grounded and perturbation-stable, filtering out brittle or noisy visual
tokens. Based on RVD, we further propose Perceptual Advantage Reshaping (PAR), a
token-level credit assignment technique that amplifies perceptually informative
tokens while preserving stable gradients for non-perceptual
tokens. Extensive experiments on seven
multimodal reasoning benchmarks demonstrate that PRPO consistently outperforms strong LVLM baselines across both 3B and 7B model scales, achieving average gains of 23.3% and 21.1%, respectively. PRPO achieves state-of-the-art performance with improved training efficiency and stronger cross-task generalization. Our findings highlight the importance of fine-grained credit assignment for scalable
multimodal reinforcement learning.
Read the full abstract ▾
All authors Qiming Li, Tianlun Li, Xiaolong Cheng, Hangyu Li, Ruiyan Gong, Kangning Niu, Kaitao Jiang, Mu Xu
Computer Vision · Cross-listed from cs.RO
PhysAgent: Automating Physics-Based 4D Synthesis via Trajectory-Grounded Multi-Agent Feedback
Chunji Lv, Jiaxi Ye, Yuchen Jiang et al. (+2 more)
Achieving fully automated, physically plausible 3D motion synthesis is a core objective in graphics and generative AI. However, configuring complex environmental force fields still relies entirely on manual expert intervention, creating a severe bottleneck for large-scale simulation data generation. Existing automated methods primarily focus on material optimization and exhibit severe modality gaps and technical flaws when applied to the vastly more complex force field optimization space: naive Large
Language Models (LLMs) lack underlying simulation feedback, causing severe physical inaccuracies, while traditional Score
Distillation Sampling (SDS) suffers from sluggish gradients, local optima entrapment, and a mathematical inability to dynamically switch discrete force fields. To address this, we propose PhysAgent, the first simulator-in-the-loop multi-
agent framework that leverages
multimodal inputs for automated, physically grounded 4D synthesis. By decoupling intrinsic materials from extrinsic dynamics, PhysAgent utilizes a Semantic
Agent equipped with an externalized Force Field Skill module to master simulation rules and generate valid initializations. Subsequently, the Refine
Agents, driven by Trajectory-Grounded Multi-
Agent Feedback, leverage
vision foundation models to extract dense point trajectories from rendered frames. By converting these explicit motion trajectories into structured textual descriptors, the
agent harnesses LLM commonsense
reasoning to execute zero-shot macroscopic leaps, effectively escaping local optima and dynamically switching discrete force fields. Extensive experiments demonstrate that PhysAgent rapidly generates stable, diverse physical scenes from arbitrary
multimodal prompts, significantly outperforming existing baselines in both generation diversity and physical accuracy.
Read the full abstract ▾
All authors Chunji Lv, Jiaxi Ye, Yuchen Jiang, Rexar Lin, Changsheng Li
Computer Vision
Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation
Xingyue Zhao, Wenke Huang, Linghao Zhuang et al. (+7 more)
Low-Rank Adaptation (LoRA) enables efficient
federated fine-tuning of
segmentation foundation models for medical imaging. However, most
federated LoRA methods adopt a uniform aggregation rule, which breaks under the encoder-decoder asymmetry in medical
segmentation: the encoder is dominated by appearance shifts, while the decoder is dominated by supervision variations. This mismatch entangles shared anatomy with site-specific biases and harms generalization. To address this, we propose Inverse Asymmetric Tuning (IAT). IAT aligns adaptation with heterogeneity sources by personalizing module-specific components in the encoder to absorb appearance shifts and in the decoder to accommodate site-dependent supervision, while retaining a shared pathway for transferable consensus. However, structural separation alone is insufficient under LoRA's bilinear parameterization, where multiplicative coupling can still cause site-specific updates to leak into the shared direction. We therefore introduce a Subspace Orthogonality Regularizer that penalizes shared-local collinearity in the effective update space, mitigating leakage without extra communication. Experiments show consistent improvements over strong
federated LoRA and parameter-efficient FL baselines.
Read the full abstract ▾
All authors Xingyue Zhao, Wenke Huang, Linghao Zhuang, Haoran Wu, Anwen Jiang, Zhifeng Wang, Wenwen He, Ming Feng, Mang Ye, Bo Xu
Computer Vision
BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving
George Ling, Lijin Yang, Hao Yang et al. (+1 more)
We present BLUE, a minimal method for better
language use in
vision-
language-action (VLA) models for autonomous driving (AD). Through extensive analysis, we reveal that
language matters on only a small fraction of routes, but on those routes it can greatly improve or degrade performance. Generating
language at every frame is therefore inefficient, since most computation is spent on frames that do not benefit from
language. We further show that pretrained VLA hidden states potentially already encode whether
language will benefit a given frame, even though scene complexity and kinematic features alone struggle to predict this. Based on this finding, BLUE trains a lightweight gate on frozen VLA hidden states to decide per frame whether to activate
language generation or predict actions directly, without modifying the backbone or requiring additional human annotation. With just a 0.11M-parameter gate, BLUE sets a new state of the art on both
benchmarks, achieving 76.2% success rate on Bench2Drive and 36 driving score on Longest6 v2, while delivering 2.54x inference speedup and 8.9% success rate improvement over the backbone. BLUE provides a practical path toward efficient
language-augmented AD, showing that VLA models can retain the benefits of
language at a fraction of the cost. Our code, data, logs and checkpoints are fully available on https://github.com/George-Ling3/BLUE.
Read the full abstract ▾
All authors George Ling, Lijin Yang, Hao Yang, Zhongzhan Huang
Computer Vision
Distortion-Aware PETR for BEV Object Detection with Mixed Pinhole-Fisheye Cameras
Xiangzhong Liu
Fisheye cameras are widely deployed in autonomous driving perception suites for their low cost and full-coverage field of view (FOV), yet their potential remains underleveraged in 3D object detection. Severe radial distortion challenges most BEV detectors by violating the fundamental assumption of uniform sampling. To bridge this gap, we propose Distortion-Aware PETR (DAPETR), a projection-free detector tailored for mixed pinhole-fisheye camera setups. DAPETR incorporates two key learned-adaptive modules: a unified distortion-aware positional
embedding that harmonizes positional encodings for image representations with fisheye geometry, and a bidirectional feature-geometry co-modulation module that mutually adapts image features and 3D positional
embeddings. In our experiments on a converted KITTI-360
benchmark, we systematically compare our learned adaptive approach against PETR in polar coordinates (PolarPETR). We find that while both methods improve over the baseline, our learned modules achieve superior performance. Crucially, we uncover a negative interaction when combining both strategies, revealing that learned adaptation and explicit
geometric reparameterization can conflict. Our final DAPETR model significantly advances the research and
benchmark for fisheye BEV detection, providing critical insights into effective distortion-aware 3D perception design other than image rectification.
Read the full abstract ▾
Computer Vision
BioVid: Autoregressive Video Generation with Biological Behavior Semantic Comprehension
Tsung-Wei Pan and Jung-Hua Wang
Video generation for biological behavior requires more than visually plausible motion: the duration of an action is itself a semantic property. Existing models usually rely on fixed temporal windows, external continuation, or
prompt-driven stories, so length is specified externally rather than learned from behavior. To address this gap, we propose BioVid, a data-driven autoregressive framework for adaptive-length biological behavior generation. BioVid uses a 2D-encode/3D-decode tokenizer: a two-dimensional FSQ-R3GAN encoder converts each frame into discrete visual
tokens, preserving single-frame information suited for next-
token prediction and EOS-based termination, while a temporally inflated and
video-finetuned three-dimensional decoder reconstructs generated
tokens with temporal context to reduce flickering. A causal
Transformer then models the frame-wise
token sequence and, conditioned only on the first frame, stops generation when it emits an End-of-Sequence
token, allowing duration to emerge from the learned behavior distribution. We evaluate BioVid on the A001 drinking action from NTU RGB+D. On 94 held-out clips, BioVid achieves a Wasserstein-1 distance of 1.24 frames from the real duration distribution. In comparison, fixed-length baselines yield distances of approximately 6-7 frames even when configured to the available length closest to the
dataset mean, and approximately 15 frames when using the conventional 16-frame generation length. These results demonstrate the ability of BioVid to learn and reproduce the intrinsic duration distribution of biological behavior.
Read the full abstract ▾
Computer Vision
Learning to Solve Generative ODEs Beyond the Linear Span
Sihyeon Kim, Seunghun Lee, Vikas Singh et al. (+1 more)
Diffusion and flow generative models sample by integrating a learned ODE, but high quality still requires many sequential model evaluations. Solver learning reduces this cost by adapting scalar coefficients, timesteps, or both, while keeping the backbone model fixed. In this work, we identify a structural bottleneck in this update family: each step remains span-limited. Since the scalar-coefficient update lies in the span of buffered velocity evaluations, it can fit only the in-span component while leaving any out-of-span residual unreachable by scalar recombination alone. We propose SpanLift, a lightweight neural solver that augments scalar-coefficient updates with a spatial residual operator. SpanLift keeps a fixed base solver as an in-span prior and learns a spatial residual operator over the state and velocity buffer. The operator is trained by endpoint teacher matching, preserves the pretrained backbone, and adds no model NFEs. Empirically, the learned correction transfers across base solvers and is predominantly out-of-span. Across pixel-space
diffusion, latent flow matching, and precipitation nowcasting, SpanLift achieves state-of-the-art few-step sampling. With only 3 NFE, it improves CIFAR-10 FID from 8.16 to 5.69 and ImageNet FID from 17.37 to 11.83.
Read the full abstract ▾
All authors Sihyeon Kim, Seunghun Lee, Vikas Singh, Hyunwoo J. Kim
Computer Vision
WaveDiT: Distribution-Aware Wavelet Flow Matching for Efficient 3D Brain MRI Synthesis
Danilo Danese, Angela Lombardi, Giuseppe Fasano et al. (+2 more)
Large and demographically balanced
datasets are essential for reliable neuroimaging biomarkers. Full-resolution 3D
brain MRI synthesis can support data augmentation in this setting, but existing approaches either incur prohibitive computational cost at volumetric scale or rely on lossy latent compression that may compromise anatomical detail. As a result, practical 3D generative augmentation often requires specialized compute infrastructure. We propose WaveDiT, a conditional flow matching framework operating in the coefficient space of a 3D Haar Discrete Wavelet Transform. The model combines factorized spatio-depth
attention with band-wise heteroscedastic uncertainty modeling derived from higher-order wavelet statistics. Predicted log-variance is integrated directly into both the flow objective and conditioning pathway, enabling adaptive precision consistent with the heavy-tailed and input-dependent variance structure of anatomical detail. This formulation supports full-resolution 3D synthesis under practical
memory and time constraints on a single modern GPU. Evaluation on a multi-site cohort demonstrates improved
alignment between generated and real MRI distributions, together with enhanced downstream
brain age prediction and region-level anatomical agreement relative to
diffusion, latent, and wavelet-based baselines. Code is available at https://github.com/sisinflab/WaveDiT
Read the full abstract ▾
All authors Danilo Danese, Angela Lombardi, Giuseppe Fasano, Matteo Attimonelli, Tommaso Di Noia
Computer Vision · Cross-listed from cs.RO
PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning
Haoyu Li, Aaron Thomas, Shuyan Zhou et al. (+1 more)
To perform a wide range of daily tasks,
robots need to construct a 3D representation that is semantically rich, physically grounded, and structured enough to support task planning and affordance prediction. However, existing approaches primarily focus on semantic
retrieval, often overlooking physical and kinematic factors. Methods that attempt to model physical properties typically rely on narrow training sets or single-object modeling, limiting scalability and generalization across diverse object types. To address these challenges, we present PhysGraph, a framework that unifies symbolic
reasoning with structured 3D geometry to model kinematic and physical properties in cluttered scenes. Given RGB-D observations, PhysGraph reconstructs object-centric 3D geometry and associates object instances across views. It then decomposes objects into functional parts and infers materials and articulations through visual
reasoning. Evaluated on both synthetic and real-world
datasets, PhysGraph achieves state-of-the-art results in semantic
segmentation, multi-object mass estimation, and articulation prediction. With its simple yet effective design, PhysGraph produces physically consistent and semantically structured scene graphs, serving as a structured 3D representation for downstream tasks such as constraint-aware 3D affordance prediction and real-to-sim transfer, both of which are demonstrated in our experiments.
Read the full abstract ▾
All authors Haoyu Li, Aaron Thomas, Shuyan Zhou, Xianyi Cheng
Computer Vision
FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuning
Haihao Lin, Xiangsheng Huang, Xiao Yang et al. (+7 more)
Action-supervised fine-tuning of
vision-
language-action (VLA) policies can compress visual structure that is weakly constrained by the action objective. We formalize this as residual visual collapse along local action fibers and propose FiberTune, a training-time objective that preserves teacher-structured visual residuals without adding inference-time overhead. FiberTune uses an online action probe to filter action-predictive directions from intermediate visual-
token representations. It preserves the residuals through centered directional
alignment with a frozen visual teacher and effective-rank regularization. Under identical training conditions, FiberTune improves over task-loss-only fine-tuning in every one of six simulation settings spanning two
benchmarks and two architectures ($\pi_{0.5}$ and OpenVLA-OFT), as well as on physical SO-101 pick-and-place; representative gains include +10.7 pp SR(5) on long-horizon CALVIN ABC$\rightarrow$D and physical SO-101 task success rising from 72.7% to 78.1%. Representation analyses support the preservation of visual information in the residuals.
Read the full abstract ▾
All authors Haihao Lin, Xiangsheng Huang, Xiao Yang, Weibang Zhou, Yiqi Zhang, Bo Yang, Simin Zeng, Jiawei Yang, Zhengyang Wang, Jiahui Du
Computer Vision · Cross-listed from astro-ph.SR
Reconstructing Synthetic SDO/AIA 193 A EUV Images from He I 10830 A Observations with Diffusion Model Translator
Marco Marena, Qin Li, Haimin Wang et al. (+3 more)
Routine full-disk EUV imaging has been available only since the modern era, such as SOHO and SDO. To extend EUV coronal context into earlier periods, we leverage the multi-decade availability of full-disk \HeI{} observations, whose absorption is modulated by coronal irradiance and
magnetic topology and is widely used as a proxy for open-field regions. We present a
diffusion-based conditional image translation framework, Coronal Hole-aware
Diffusion Model Translator (CH-aware DMT), to reconstruct synthetic SDO/AIA 193 Å EUV images from \HeI{} inputs. The model is trained on temporally co-aligned SOLIS \HeI{} and AIA 193 Å pairs spanning 2011–2015 using a month-based split, where January–October are used for training, November is used for validation, and December for testing. On the held-out test set, the reconstructions preserve dominant full-disk EUV morphology (CC=0.92) and recover CH-related low-intensity structure (CC=0.84). We further assess historical applicability by (1) comparing reconstructed AIA 193 Å morphology with SOHO/EIT 195 Å over 2005–2015; (2) comparing reconstructed AIA 193 Å images generated from KPVT \HeI{} inputs against Yohkoh/SXT soft X-ray observations; and (3) evaluating long-term reconstructed disk-integrated emission statistics against observational EUV series and independent
solar activity proxies (sunspot number and F10.7
radio flux over 1974–2015). These results indicate that CH-aware DMT conditioned on \HeI{} can provide a physically plausible synthetic AIA 193 Å coronal proxy for historical studies, supporting multi-decade analyses of large-scale coronal
evolution before the direct EUV imaging was available.
Read the full abstract ▾
All authors Marco Marena, Qin Li, Haimin Wang, Haodi Jiang, Prajwal Shah, Bo Shen
Computer Vision
Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning
Jingzhi Chen, Landi He, Zhuo Chen et al. (+2 more)
The processing of gigapixel whole slide images within
vision language models faces a major difficulty due to an excessive number of visual
tokens. Existing solutions typically rely on spatial downsampling or heuristic
pruning strategies that operate without training, and these methods often discard subtle but clinically meaningful patterns because pathological evidence is scattered irregularly across the tissue. To overcome this limitation, we reformulate
token reduction in whole slide images as a trainable sparsification problem, allowing the model to learn an optimal selection strategy instead of following fixed heuristics. We propose a decoupled routing architecture. To enable gradient propagation through the nondifferentiable
pruning operation during training, we introduce a component called SparseLearn. This component uses a variance-preserving noise gate that regulates the information flow of each patch via a differentiable Soft Top-K operator, together with a diagonal
attention denoiser that recovers perturbed representations without leaking spatial information. At inference time, the SparseLearn module is entirely discarded, and the trained scorer applies a deterministic Hard Top-K operator to keep only the highest scoring 32
tokens, incurring no extra computation. By compressing the visual sequence down to a sparse set of just 32
tokens, which represents as little as 0.78% of the original length, our framework achieves 73.32% overall accuracy on SlideBench (TCGA), consistently surpassing sampling-based baselines and general-purpose
vision language models. It also demonstrates strong zero shot generalization on SlideBench (BCNB) and WSI VQA*. By resolving the visual context bottleneck and preventing the dilution of sparse diagnostic evidence, this work provides a highly efficient paradigm for end to end gigapixel whole slide image
reasoning.
Read the full abstract ▾
All authors Jingzhi Chen, Landi He, Zhuo Chen, Shawn Young, Lijian Xu
Computer Vision
SSAFE: Simple and Strong AI-Generated Image Detection via Frozen Vision Encoders
Seunghyun Lee, Byoungkwon Kim, Jaehyun Nam et al. (+2 more)
The rapid advancement of generative models has blurred the boundary between synthetic and real imagery, creating an urgent need for reliable deepfake detection. Yet most existing approaches rely on massive real–fake
datasets, which are increasingly difficult to maintain as new generators continue to emerge. In this work, we investigate how much information about image authenticity is already encoded in modern
multimodal vision representations. We find that frozen
multimodal encoders naturally separate real and synthetic images in their
embedding space, enabling a simple linear classifier to achieve strong performance without task-specific fine-tuning. Motivated by this observation, we develop a representation-aware data curation strategy that selects a compact set of representative generators for training. The resulting training set contains only 10K images, compared to 288K in AIGIBench and 4M in OpenFake, while improving robustness to unseen generators and distribution shifts. We additionally introduce RealWorldBench, a
benchmark consisting of modern camera photographs, contemporary stock images, and outputs from recent commercial generators. Experiments across multiple
benchmarks show that combining frozen
multimodal representations with carefully curated training data provides a simple and effective approach to AI-generated image detection.
Read the full abstract ▾
All authors Seunghyun Lee, Byoungkwon Kim, Jaehyun Nam, Kyungmin Lee, Jinwoo Shin
Computer Vision
Harnessing Streaming Video in the Wild
Dingyu Yao, Shuhuan Gu, Qingyi Si et al. (+8 more)
Vision-
Language Models (VLMs) are increasingly required to process unbounded
video streams in applications such as
video-call assistants, live commentary, and
embodied robots. An ideal streaming system should support proactive interaction, long-horizon
memory, and real-time processing, while resting on a VLM backbone capable of handling diverse in-the-wild streaming tasks. However, existing VLMs excel at offline
video understanding but fall short in streaming capabilities and lack dedicated infrastructure for streaming deployment. We address this gap on three fronts. (i) For backbone capability, we construct Streaming-Train-248K, a streaming
dataset paired with a novel training objective for adapting VLMs to streaming interaction and understanding. (ii) For real-world deployment, we introduce Streaming Harness, a plug-and-play system that endows any VLM with three core abilities: proactive interaction (per-second response decisions), long-term
memory (12-hour context retention), and real-time processing (sub-second latency). (iii) To drive continued community progress on streaming capabilities, we design Streaming-Eval, a
benchmark that reflects models' capabilities across diverse in-the-wild scenarios. Extensive experiments demonstrate consistent gains from our approach across all core capabilities required for streaming
video understanding. We will open-source our data, code, and
benchmark to advance the community's shift from offline
video understanding to deployable streaming intelligence.
Read the full abstract ▾
All authors Dingyu Yao, Shuhuan Gu, Qingyi Si, Junhao Zhou, Chenxu Yang, Chuanyu Qin, Naibin Gu, Zheng Lin, Weiping Wang, Nan Duan, Jiaqi Wang
Computer Vision
Facial Expression Recognition in the Deep Learning Era: A Systematic Multi-Criteria Review of Methods, Models, Datasets, Performance, Challenges, and Future Research Directions
Spyridon Georgiou, Aggelos Psiris, Spyridon Evangelatos et al. (+5 more)
Facial Expression Recognition (FER) has advanced rapidly over the last decade, driven by the shift from handcrafted descriptors and shallow classifiers to deep convolutional,
attention-based,
vision-
language, and foundation-model architectures, and by the parallel growth of large-scale in-the-wild
benchmarks spanning categorical, dimensional, compound, micro-expression, Action Unit (AU), and intensity-estimation tasks. Yet the deep learning-based FER landscape has so far been reviewed only along narrow task-, architecture-, or application-specific axes, leaving a holistic, systematically organized account of its recent advances missing. This survey addresses that gap with a comprehensive review of recent deep learning-based FER, explicitly linked to the wider Facial Affect Recognition (FAR) domain. Its main contributions are: a) A description of FER's
evolution into five distinct phases, from handcrafted features and classical machine learning to
attention-based,
vision-
language, and foundation-model approaches, with the key milestone works of each, b) A multi-criteria taxonomy analyzing the literature along seven complementary axes: recognition task, input modality, face pre-processing pipeline, network architecture, learning strategy, acquisition setting, and application domain, c) A per-criterion comparative analysis, with critical insights into the strengths and limitations of each category under in-the-wild conditions, d) A task-organized review of public FER
datasets, with their annotation schemes, modalities, and evaluation protocols, e) A compilation of performance metrics and a per-task quantitative comparison of representative state-of-the-art methods on widely adopted
benchmarks, and f) A discussion of current challenges and promising future directions.
Read the full abstract ▾
All authors Spyridon Georgiou, Aggelos Psiris, Spyridon Evangelatos, Thomas Lagkas, Vasileios Argyriou, Panagiotis Sarigiannidis, Iraklis Varlamis, Georgios Th. Papadopoulos
Computer Vision · Cross-listed from cs.LG
OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Framework
Chenhan Jin, Shengze Xu, Qingsong Wang et al. (+3 more)
Data
pruning (DP), as an oft-stated strategy to alleviate heavy training burdens, reduces the volume of training samples according to a well-defined
pruning method while striving for near-lossless performance. However, existing approaches, which commonly select highly informative samples, can lead to biased gradient estimation compared to full-
dataset training. Furthermore, the analysis of this bias and its impact on final performance remains ambiguous. To address these challenges, we propose OrderDP, a plug-and-play framework that aims to obtain stable, unbiased, and near-lossless training acceleration with theoretical guarantees. Specifically, OrderDP first randomly selects a subset and then chooses the top-$q$ samples, where unbiasedness is established with respect to a surrogate loss. This ensures that OrderDP conducts unbiased training in terms of the surrogate objective. We further establish convergence and generalization analyses, elucidating how OrderDP affects optimal performance and enables well-controlled acceleration while ensuring guaranteed final performance. Empirically, we evaluate OrderDP against comprehensive baselines on CIFAR-10, CIFAR-100, and ImageNet-1K, demonstrating competitive accuracy, stable convergence, and exact control – all with a simpler design and faster runtime, while reducing training cost by over 40%. Delivering both strong performance and computational efficiency, our method serves as a robust and easily adaptable tool for data-efficient learning. The code is publicly available at https://github.com/shengze-xu/OrderDP.
Read the full abstract ▾
All authors Chenhan Jin, Shengze Xu, Qingsong Wang, Fan Jia, Dingshuo Chen, Tieyong Zeng
Computer Vision
OmniCap-IF: Benchmarking and Improving Instruction Following Abilities for Omni-Video Captioning
Jiahao Wang, An Ping, Yanghai Wang et al. (+13 more)
While Omni-modal Large
Language Models (OLLMs) have demonstrated impressive capabilities in jointly processing audio and visual streams, their ability to strictly adhere to complex, multi-faceted user instructions remains largely unexplored. Existing
benchmarks primarily focus on holistic
video understanding or text-only instruction following, failing to capture the intricate interplay between modalities and user constraints. To bridge this gap, we introduce OmniCap-IF, the first comprehensive
benchmark specifically designed to evaluate instruction-following capabilities in omni-modal captioning. OmniCap-IF incorporates a systematic framework that assesses captions on two dimensions: format correctness and content correctness. Our
benchmark encompasses 50 distinct constraint types across pure visual, pure audio, and audio-visual modalities, while integrating Temporal Grounding to assess spatio-temporal precision. Extensive evaluations of prominent models on 1,920 high-quality samples reveal significant performance disparities. Furthermore, our analysis uncovers a critical "format-content tradeoff", demonstrating that increasing formatting complexity directly degrades models' omni-modal
reasoning abilities. Finally, to advance the field, we curate a 54K instruction-tuning
dataset, OmniCap-IF-54K and present OmniCaptioner-IF, which achieves notable improvements in both complex instruction adherence and general omni-modal captioning performance.
Read the full abstract ▾
All authors Jiahao Wang, An Ping, Yanghai Wang, Yuanxing Zhang, Shihao Li, Hanyan Bian, Yichi Ren, Yize Zhang, Han Wang, Haowen Chen, Junze Li, Jiaqi Wang, Yiyang Hu, Zhuze Xu, Zijie Zhang, Jiaheng Liu
Computer Vision
Towards Accurate Emotion-Attributed Video Captioning via Fine-grained Emotion-Cause Pair Extraction
Weidong Chen, Cheng Ye, Zhendong Mao et al. (+3 more)
Emotional
Video Captioning (EVC) is a challenging task that aims to generate factually accurate and emotionally rich descriptions for
videos. Existing EVC methods leverage holistic visual features to mine global emotional cues, and then aggregate
multimodal features to guide the emotional caption generation, which ignores the critical characteristic of the EVC task. Visual
emotions are evoked by specific motivational causes, which are usually only implied in core
video segments. The holistic mining brings significant information redundancy and inaccurate emotional cues. Thus, fine-grained visual cause extraction has a facilitative effect on both
emotion perception and
emotion-attributed caption generation. To this end, we propose a fine-grained
emotion-cause pair extraction framework for
emotion-attributed
video captioning. Specifically, we learn pair-wise
emotion and cause features in two rounds: 1) We propose a Concept-aware Visual Semantic Decomposition module to augment visual features by exploring scene, object, and motion concepts. Besides, to enhance emotional features, we propose a Visual-guided
Emotion Interpretable Learning module, which guides
emotion refinement with visual temporal dynamics, and augments the interpretable refinement process by reliable VAD-vector constraints. 2) We achieve
emotion-cause pair extraction by cross-coupling the visual and emotional features before and after refinement, and leverage contrastive loss to achieve semantic forced
alignment. Overall, our approach optimizes complex semantic understanding and
emotion perception of
videos, leading to a promising performance in emotional captioning. Extensive experiments on three challenging
datasets demonstrate the superiority of our approach and each proposed module, e.g., achieving the best performances with +4.4% and +5.4% w.r.t. BLEU-2 and ROUGE-L, respectively, on the EVC-MSVD
dataset.
Read the full abstract ▾
All authors Weidong Chen, Cheng Ye, Zhendong Mao, Liping Wang, Xinyan Liu, Yongdong Zhang
Computer Vision · Cross-listed from cs.RO
When Video Misreads: Closed-Loop Distillation of Reading Heuristics for Exploratory Manipulation Trace QA
Haizhou Ge, Yufei Jia, Yue Li et al. (+5 more)
Exploratory manipulation often turns an apparent failed attempt into the key evidence for what to do next. For example, a
robot pulls a locked cabinet drawer, fails, and only succeeds after opening the lock. The failed pull reveals a latent precondition (the drawer is locked) that determines the minimal-success action chain (the fewest actions that complete the task), here [lock-open, drawer-pull]. Correctly reading this trace is therefore the prerequisite for recovering that chain. We formalize this setting as Exploratory Manipulation Trace QA (EMT-QA): given synchronized
video and proprioception from an exploratory trace, predict the minimal-success action chain under the latent precondition revealed by the probe. However, even state-of-the-art VLMs and
embodied multimodal LLMs misread this evidence: they do not reliably recover the chain from raw
video, raw proprioception, or their combination. We introduce Closed-Loop Trace
Distillation, a pipeline that uses a per-task coding
agent to inspect labeled training traces and distill a one-line natural-
language prompt over the trace, which we call the Distilled Reading Heuristic (DRH). At inference, no
agent is invoked and no model weights are updated; a frozen VLM receives the raw trace plus the DRH as a
prompt entry. Across three simulator and two real-
robot tasks, the DRH improves chain accuracy by +0.38 to +0.47 over the best raw-modality baseline. The same DRH also serves as the sole specification for one-shot programmatic classifiers that match the prompted VLM.
Read the full abstract ▾
All authors Haizhou Ge, Yufei Jia, Yue Li, Zhixing Chen, Lu Shi, Lei Han, Guyue Zhou, Ruqi Huang