Computer Vision
Unlimited OCR Works
Youyang Yin, Huanhuan Liu, YY et al. (+14 more)
Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large
language model (LLM) as the decoder allows the model to leverage the prior distribution of
language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up
memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working
memory. Taking DeepSeek OCR as the baseline, we replace all
attention layers in the decoder with our proposed Reference Sliding Window
Attention (R-SWA), which reduces
attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing
attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.
Read the full abstract ▾
All authors Youyang Yin, Huanhuan Liu, YY, Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu, Mingxin Huang, Zhen Wu, Guibin Wang, Tengyu Du, Lei Jia
Computer Vision
UECP: Uncertainty-Enhanced Collaborative Perception
Kang Yang, Tianci Bu, Peng Wang et al. (+3 more)
Collaborative perception serves as a pivotal solution to enhance the perception capability of individual
agents in autonomous driving, where a core challenge lies in seeking reliable evidence to quantify and weight the contribution of each participating
agent. Existing methods typically rely on a confidence map, which is co-trained with the detection head, but it is inherently correlated with the detection results and thus fails to provide unbiased physical evidence. Furthermore, how to deeply integrate evidence into the cooperative fusion process remains an open question. To address these issues, this paper first proposes an uncertainty map, a physically grounded and unambiguous metric for evaluating perception quality. This map is directly supervised by real-time sensor signals, i.e.,
LiDAR point density, ensuring decoupling from detection noise and thereby providing physical scenario-aware evidence for weighting
agent contribution. Based on this map, we develop the Uncertainty-Enhanced Collaborative Perception (UECP) framework, centered on the Uncertainty-Aware Pyramid Fusion (UAPF) module. UAPF uses a coarse-to-fine strategy, with two key components: Uncertainty-Weighted Downsampling (UWD) for high-fidelity feature preservation, and Uncertainty-Guided Residual Fusion (UGRF) to reinforce ego features, suppressing noise and ensuring robust fusion. Extensive experiments on real-world
datasets show UECP outperforms state-of-the-art methods in effectiveness and robustness by
embedding the uncertainty map into fusion. Code will be publicly available.
Read the full abstract ▾
All authors Kang Yang, Tianci Bu, Peng Wang, Deying Li, Wen Jie, Yongcai Wang
Computer Vision
SPAR: Semantic-Pixel Self-Alignment and Adaptive Routing for Unified Multimodal Models
Hongxiang Li, Hongxu Chen, Chenyang Zhu et al. (+5 more)
Multimodal Large
Language Models (MLLMs) have achieved remarkable success in visual understanding but remain constrained in visual generation due to the fundamental feature discrepancy between semantic perception and pixel-level reconstruction. Bridging this gap requires overcoming two core challenges: endowing semantic encoders with high-fidelity reconstruction capabilities, and effectively aligning generative models with semantic spaces without relying on external teachers. To this end, we propose a novel unified
multimodal framework featuring Semantic-Pixel self-
alignment and Adaptive Routing (SPAR). First, to reconcile semantic perception with pixel-level reconstruction, we introduce an asymmetric dual-stream unified tokenizer. A lightweight semantic stream anchors discriminative features, while a
Transformer-augmented pixel stream recovers fine-grained visual details into a unified compact latent space. Second, to eliminate external dependencies, we propose a self-aligned generation paradigm that natively leverages this optimized tokenizer as an internal
alignment teacher for the
diffusion model. Furthermore, to facilitate flexible
multimodal interaction within this unified space, we introduce Dynamic
Token Routing, which enables each
token to adaptively aggregate multi-layer MLLM features based on its distinct semantic demands. Extensive experiments demonstrate that SPAR establishes the state-of-the-art for unified architectures, achieving exceptional generation and reconstruction quality while preserving foundational visual understanding capabilities.
Read the full abstract ▾
All authors Hongxiang Li, Hongxu Chen, Chenyang Zhu, Xiaoshuang Huang, Jiayin Cai, Xiaolong Jiang, Yao Hu, Long Chen
Computer Vision
DrivingVoxels: Compositional Sparse Voxel Rasterization for Dynamic Driving Scene Reconstruction
Tania Aguirre, Luis Roldão, Moussab Bennehar et al. (+4 more)
Reconstructing dynamic urban scenes remains challenging due to the unbounded nature of driving environments and the presence of multiple dynamic objects. Currently, potentially faster sparse voxel methods are mainly designed for static scenarios. On the other hand, dynamic approaches based on 3D Gaussian Splatting, despite their high-fidelity, are often time-consuming for driving scenarios and exhibit uncontrollable
memory growth in large scenes. To address these limitations, we present DrivingVoxels, a compositional sparse voxel rendering framework for dynamic driving scenes. Our method jointly rasterizes sparse voxels from multiple independent octrees within a single rendering pass. Each rigid dynamic object is represented by an octree defined in its local coordinate frame, while a separate static octree models the stationary background. DrivingVoxels adopts a fully explicit, neural-free representation together with a
LiDAR-guided structural initialization that efficiently captures scene geometry. We evaluate our framework on the PandaSet
benchmark, demonstrating that DrivingVoxels performs on par on perceptual metrics and better on structural metrics for NVS and reconstruction while requiring shorter training times than previous 3DGS-base methods to an efficient optimization workflow anchored by a strong
LiDAR prior.
Read the full abstract ▾
All authors Tania Aguirre, Luis Roldão, Moussab Bennehar, Nathan Piasco, Dzmitry Tsishkou, Simone Rossi, Pietro Michiardi
Computer Vision
Physics-Guided Spatiotemporal State Space Modeling for Lookahead Molten Pool Segmentation in Laser Wire-Feed Welding
Sen Li, Haichao Cui, Changhao Yin et al. (+4 more)
Real-time weld-pool perception is critical for closed-loop control in
laser wire-feed welding, where sensing, computation, and actuator response introduce unavoidable delay. This paper presents a physics-guided spatiotemporal state space network for lookahead weld-pool
segmentation. The model uses historical coaxial grayscale images, welding process parameters, and aligned wire-state electrical signals to predict the future semantic layout of three physically meaningful regions: keyhole, wire, and molten pool. It combines a visual encoder, process- and sensor-conditioned feature normalization, patch-level temporal state space modeling, horizon-conditioned latent prediction, dense future feature prediction, and a motion-aware mask decoder. Auxiliary signed-distance-function supervision, temporal consistency, feature
distillation, and fine-grained keyhole losses further constrain the predicted geometry and local motion. Experiments on a 43-sequence
laser welding
dataset show that the proposed WeldMamba reaches 74.63% mIoU at a 500 ms lookahead. Ablation studies further show that temporal history, patch-level state space modeling, and keyhole motion awareness are the main contributors to robust future
segmentation.
Read the full abstract ▾
All authors Sen Li, Haichao Cui, Changhao Yin, Chendong Shao, Yaqi Wang, Xinhua Tang, Fenggui Lu
Computer Vision
Learning Stable Canonical Worlds for Novel View Synthesis and Beyond
Xiaoyu Xu, Jian Zou, Sheyang Tang et al. (+3 more)
Feed-forward Gaussian splatting (FFGS) facilitates real-time novel view synthesis, yet current methods often remain tied to view-dependent predictions. As more input views are added, they may accumulate noisy or redundant evidence instead of converging to a stable scene representation. In this paper, we introduce CanonicalGS, a feed-forward pipeline that maps cluttered multi-view observations into a stable, scene-centric representation. CanonicalGS first extracts view-centric evidence from depth, semantic features, and uncertainty estimates, and then aggregates this evidence in a canonical latent world using uncertainty-aware fusion. By emphasizing reliable observations while suppressing uncertain or redundant ones, CanonicalGS produces representations that scale more effectively for novel view synthesis and transfer to downstream visual perception tasks. Experiments show up to a $2.5$ dB improvement in peak signal-to-noise ratio for synthesizing novel views and an $11\%$ gain in semantic
segmentation accuracy.
Read the full abstract ▾
All authors Xiaoyu Xu, Jian Zou, Sheyang Tang, Zhihua Wang, Jing Liao, Kede Ma
Computer Vision
Boosting Neural Video Codec via Scale-Driven Online Flow Refinement
Tiange Zhang, Rongqun Lin, Haocheng Tang et al. (+4 more)
Although state-of-the-art neural
video codecs (NVCs) have achieved remarkable performance, they suffer from limited generalization when encountering complex motion patterns unseen during training. To bridge this domain gap without the expensive cost of online fine-tuning, we propose a Training-Free Scale-Driven Online Flow Refinement (SOFR) method. Serving as a plug-and-play module, SOFR integrates motion information from coarse and fine scales and dynamically fuses them according to warping accuracy, effectively rectifying motion estimation errors with negligible computational overhead. Furthermore, we design a rate-aware strategy that selects different dynamic fusion strategies according to bitrate modes, and employs a reliability check based on warping error to ensure robustness. Extensive experiments on the USTC-TD
dataset verify the effectiveness and generalization of SOFR across various NVC frameworks, including DCVC-SDD, DCVC-FM, and EHVC. Notably, it brings an average of 2.84% and 4.05% bitrate savings in terms of PSNR and MS-SSIM, respectively, to DCVC-FM with negligible coding time increase. Our code is available at https://github.com/SunnyMass/SOFR.
Read the full abstract ▾
All authors Tiange Zhang, Rongqun Lin, Haocheng Tang, Xiandong Meng, Weijia Jiang, Zhimeng Huang, Siwei Ma
Computer Vision
Ruiliang Zhou, Xuecheng Wu, Kang He et al. (+6 more)
While
Diffusion Transformers (DiTs) have revolutionized high-fidelity
video generation, their reliance on 3D full
attention creates a quadratic computational bottleneck. Existing sparse methods face a dilemma: dynamic
pruning suffers from prohibitive runtime overhead and
memory fragmentation, while static heuristics fail to capture fine-grained dependencies. In this work, we propose ScalingAttention, a training-free framework grounded in a key inductive bias: while individual activations are input-dependent, the high-mass
attention regions for each head rapidly converge to a stable,
prompt-agnostic Intrinsic Sparse
Topology. This
topology is weight-encoded, scale-invariant, and efficient to extract. ScalingAttention decouples
topology discovery from sparsity control via: (1) WEST (Weight-Encoded Sparse
Topology), which extracts a robust block-sparse prior mask offline to eliminate runtime search; (2) FAST (Fidelity-Aware Sensitivity Tuning), which adaptively tunes head-wise sparsity based on
diffusion fidelity requirements. To ensure practical acceleration, we co-design a hardware-aligned bit-wise block-sparse kernel. Experiments on Wan2.1 show up to 1.90X end-to-end speedup with superior fidelity, establishing a new Pareto frontier over state-of-the-art baselines.
Read the full abstract ▾
All authors Ruiliang Zhou, Xuecheng Wu, Kang He, Guangyun Han, Bin Liu, Qinqin Chen, Wende Xu, Qingjie Zhao, Chengru Song
Computer Vision
From Point Estimates to Distributions: GMM Pooling for MIL in Preterm Birth Prediction
Hussain Alasmawi, Numan Saeed, Soha Said et al. (+1 more)
Preterm birth (PTB) prediction can enable targeted surveillance and timely intervention, yet most ultrasound-based models use a single selected transvaginal ultrasound (TVUS) frame per patient despite routine exams acquiring multiple cervical images. We formulate PTB prediction as a multiple instance learning (MIL) problem, representing each patient as a variable-sized bag of TVUS images with a single outcome label. To move beyond standard MIL aggregators that collapse a bag into a point estimate, we propose a Gaussian Mixture Model (GMM) pooling, which summarizes all images in a bag into a fixed-length representation by modeling their feature distribution. This design captures intra-patient variability. We evaluate the method on a private clinical cohort and on a public lymph node metastasis
benchmark. For PTB prediction, GMM pooling improves over the instance-based model PR-AUC from 0.44 to 0.56. On the lymph node
benchmark, it achieves state-of-the-art performance with 0.91 F1-score and 0.89 ROC-AUC for classification and 0.18 MAE for regression. The code is publicly available at https://github.com/HussainAlasmawi/GMM_Pooling.
Read the full abstract ▾
All authors Hussain Alasmawi, Numan Saeed, Soha Said, Mohammad Yaqub
Computer Vision
MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations
Yuhua Luo, Junsheng Zhang, Mengyin Liu et al. (+7 more)
Human motion follows a temporal hierarchical structure, transitioning from low-frequency global trajectories to high-frequency details. Inspired by the success of multi-level autoregressive models in computer
vision, we propose MotionMAR, a coarse-to-fine framework for motion reconstruction from sparse observations. It first estimates the global trajectory of human motion and then gradually refines the temporal details. This architecture consists of four integrated components. The Temporal Multi-scale Tokenization (TMT) VQ-VAE encodes the data at multiple temporal resolutions, separating semantic motion from minor jitters. The Motion Autoregressive Network (MAN) operates in this latent space, predicting motion across scales. It first establishes the global structure through coarse indices and then generates finer indices to recover specific details. Meanwhile, the Scale-Aware Control (SAC) module integrates sparse tracking data to ensure the generated output aligns with actual observations. The Motion Refinement Network (MRN) subsequently smooths consecutive poses and eliminates quantization artifacts. Experiments show that MotionMAR achieves state-of-the-art accuracy on the AMASS
dataset, providing a reliable and structure-aware approach for motion reconstruction. The source code is publicly available at http://www.lidarhumanmotion.net/motionmar/.
Read the full abstract ▾
All authors Yuhua Luo, Junsheng Zhang, Mengyin Liu, Xincheng Lin, Ming Yan, Zhudi Chen, Chenglu Wen, Lan Xu, Siqi Shen, Cheng Wang
Computer Vision
Black-Box Continual Learning for Vision-Language Models
Yuting Li, Weihang Fang, Haoyuan Gao et al. (+4 more)
The rapid deployment of
Vision-
Language Models (VLMs) in dynamic environments necessitates the ability to learn continuously without forgetting. However, traditional continual learning (CL) settings often rely on white-box paradigms, which is increasingly invalidated by the shift toward cloud-hosted models. In this paper, we introduce Black-CL, a more realistic
benchmark for VLMs that enforces three primary real-world challenges: weight and architecture inaccessibility, constrained computation, and task-agnostic inference. The learner can query only output
embeddings or logits, with no gradient flow through or structural modification of the backbone. Current CL methodologies, which rely on backbone backpropagation or complex parameter expansion, are fundamentally incompatible with these constraints. Under this setting, we propose BETA, a simple yet effective baseline built on the key insight that solely optimizing textual prototypes can navigate the complexities of CL. BETA integrates three core components: Semantic Projection Accumulation (SPA) for incremental knowledge acquisition, Latent Distribution Replay (LDR) for anchoring the
embedding space against catastrophic forgetting, and Test-Time Prototype Adaptation (TTPA) for dynamic, instance-aware boundary refinement. Extensive experiments across ten diverse
datasets and various backbones demonstrate that BETA significantly outperforms existing black-box tuners. Remarkably, with only 0.05 M trainable parameters, a 180–3000$\times$ reduction compared to competitive methods, BETA achieves performance on par with or even exceeding white-box CL methods. We believe Black-CL and BETA provide a foundational framework for future advancements in continual learning and accelerates the transition of continual learning from academia to real-world systems.
Read the full abstract ▾
All authors Yuting Li, Weihang Fang, Haoyuan Gao, Linghe Kong, Yexin Li, Lichao Sun, Weiran Huang
Computer Vision
Can Single-View Mesh Reconstruction Generalize to Robot Camera Rotation?
Yu Zhan, Guangcheng Chen, Hanjing Ye et al. (+4 more)
Single-view mesh reconstruction predicts object meshes and spatial layouts from a single observation, making it attractive for fast
robot spatial
reasoning and real-to-sim digital twins. However,
robot-mounted cameras naturally rotate during manipulation and
navigation, while learned single-view reconstruction models often rely on view-dependent priors and may generalize poorly to out-of-distribution camera rotations. Such rotations can introduce 3D inconsistencies, incorrect layouts, and violations of physical constraints, but this failure mode remains under-evaluated. We introduce an evaluation protocol with controlled axis-wise roll, pitch, and yaw sweeps to trace errors in
monocular depth estimation (MDE), canonical object meshes, camera-space layout, and physical plausibility within a representative SAM3D-style pipeline. On the Aria Digital Twin
dataset and a real Franka wrist-camera sequence, camera rotations induce MDE distortion, layout drift, and collision penetration, while canonical mesh predictions remain relatively stable. A two-stage SAM3D+FoundationPose pipeline is more robust than one-stage feed-forward layout prediction, and our
Gravity-Aware Refinement reduces one-stage pairwise ICP-based layout-orientation error by 47.1$\%$. Our evaluation reveals that current single-view mesh reconstruction methods generalize poorly to
robot camera rotation, and suggests that explicit
gravity cues are important for reliable robotic single-view mesh reconstruction.
Read the full abstract ▾
All authors Yu Zhan, Guangcheng Chen, Hanjing Ye, Zhiqin Cheng, Zanjia Tong, Wenjun Xu, Hong Zhang
Computer Vision
Subject-Level Unknown-Identity Identification from Leap Motion Controller 2 Hand Landmarks
Bahar Moharrer, Susanna Cifani, Marco Raoul Marini et al. (+2 more)
This work studies subject recognition from Leap Motion Controller 2 (LMC2) hand landmark data under a subject-level unknown-identity identification protocol on the Multi View Leap2 Hand Pose (ML2HP)
dataset. Using only the landmark modality, we retain the original
geometric representation and enrich it with fingertip-to-palm distances and palm-normalized inter-finger angular descriptors. Evaluation is performed under a Leave-One-Subject-Out (LOSO) protocol in which, for each outer fold, one subject is excluded from the enrolled set and treated as unknown at test time. To avoid tuning on the true outer unknown subject, the unknown-rejection threshold is selected in an inner validation step by temporarily withholding one enrolled subject from the inner gallery and using it only for threshold estimation. We compare a tree ensemble baseline with two neural alternatives: a learned
embedding baseline based on centroid matching and cosine-similarity-based rejection, and an MLP+OpenMax model, which represents a more established open-set recognition approach. Under this evaluation setup, Extra Trees remains the strongest overall method, indicating that the main challenge on this
benchmark is not enrolled-subject discrimination alone, but robust score separation between known and unknown probes. The results support the feasibility of compact, interpretable landmark-based descriptors for contactless hand-based unknown-subject rejection and identification on a small-cohort
dataset.
Read the full abstract ▾
All authors Bahar Moharrer, Susanna Cifani, Marco Raoul Marini, Luigi Cinque, Maria De Marsico
Computer Vision · Cross-listed from cs.RO
Humanoid-OmniOcc: Stereo-Based Full-View Occupancy Dataset for Embodied AI
Xianda Guo, Bohao Zhang, Chenwei Huang et al. (+6 more)
Occupancy prediction at voxel-level granularity is essential for safe robotic
navigation and interaction in complex environments. Existing occupancy
datasets, however, are predominantly designed for autonomous driving with vehicle-centric biases – forward-facing cameras, far-field geometry, and static road priors – limiting their applicability to
embodied humanoid perception. We present
Humanoid-OmniOcc, a large-scale panoramic stereo-based occupancy
dataset tailored for
humanoid robots. The
dataset encompasses 15 diverse simulated indoor scenes and 5 real-world environments, yielding over 155K samples with broad scene and style diversity. Importantly, the
dataset is designed around a Real2Sim2Real closed-loop paradigm: real sensor specifications drive physically accurate simulation, simulation produces large-scale annotated training data, and models trained in simulation are directly evaluated on real-world captures – enabling iterative refinement of the sim-to-real pipeline. We further propose
Humanoid Surround Stereo-guided Occupancy model (
Humanoid-OmniOcc) that exploits robust depth priors for accurate 2D-to-3D lifting. Extensive experiments show that
Humanoid-OmniOcc consistently outperforms
monocular baselines and generalizes well to both unseen simulated test scenes and real-world environments, validating the effectiveness of the Real2Sim2Real design. Code and data will be available upon acceptance at https://d-robotics-ai-lab.github.io/
humanoid-omniocc.
Read the full abstract ▾
All authors Xianda Guo, Bohao Zhang, Chenwei Huang, Shiyuan Chen, Ruilin Wang, Yiqun Duan, Cong Yang, Qin Zou, Wei Sui
Computer Vision
Concept Alignment Contrast and Long-Short Prompt Memory for Test-Time Adaptation of SAM3 in Medical Image Segmentation
Yubo Zhou, Jianghao Wu, Ping Ye et al. (+2 more)
Concept
segmentation models like Segment Anything Model 3 (SAM3) show strong generalization on natural images, yet their performance degrades in medical imaging due to the domain gap caused by different imaging principles and styles. Test-Time Adaptation (TTA) is essential for improving the testing performance by updating the model on the fly without annotations. However, existing
vision-
language TTA methods are mainly driven by image-level uncertainty minimization, which does not necessarily reflect region-level semantic correctness in medical
segmentation. Moreover, they often lack mechanisms to maintain stability in continual one-pass adaptation, leading to limited performance when reliable dense supervision is missing for
segmentation. To address these issues, we propose Concept
Alignment Contrast and LongShort
Prompt Memory for Test-Time Adaptation (CM-TTA) of SAM3 for medical images. First, for a test sample with multiple augmentations, we introduce a novel Concept
Alignment Contrast (CAC) metric, which leverages textual-visual semantic consistency to robustly evaluate prediction quality to select the best augmented view as the supervision. Second, to balance rapid and stable adaptation, we design a Long-Short
Prompt Memory (LSPM) module. The short
memory dynamically fuses recent
prompts based on CAC scores for agile local adaptation, while the long
memory maintains a stable global
prompt to generate enhanced pseudo-labels. Finally, a Densely Supervised
Prompt Update (DSPU) strategy is proposed to optimize the
prompt embeddings with enhanced pseudo labels as dense supervision. Extensive experiments on prostate and skin lesion
segmentation demonstrate that our CM-TTA framework significantly outperforms existing methods for TTA of SAM3. The code is available at https://github.com/SherlockZYB/CM-TTA.
Read the full abstract ▾
All authors Yubo Zhou, Jianghao Wu, Ping Ye, Shaoting Zhang, Guotai Wang
Computer Vision · Cross-listed from cs.AI
The Impact of VAE Design on Latent Pose Representations for Diffusion-based Sign Language Production
Guilhem Fauré, Mostafa Sadeghi, Sam Bigeard et al. (+1 more)
Latent
diffusion approaches to sign
language production (SLP) rely on an initial stage that learns an encoding of sign pose sequences, enabling generative modeling in the resulting latent space. The autoencoder used in this stage is typically evaluated in terms of reconstruction quality using
geometric metrics common in SLP. While informative, these metrics do not fully capture latent space properties that may influence the training and performance of the downstream generative model. In this work, we investigate how architectural and training objective design choices in a variational autoencoder (VAE) for sign pose encoding affect latent space structure, and how these differences translate into the performance of a latent
diffusion model for text-to-sign generation. Our experiments on Phoenix14T
dataset show that variations in generative performance, measured through back-translation BLEU scores, can sometimes be better explained by differences in latent space properties than by VAE reconstruction accuracy alone.
Read the full abstract ▾
All authors Guilhem Fauré, Mostafa Sadeghi, Sam Bigeard, Slim Ouni
Computer Vision · Cross-listed from cs.LG
PG-MAP: Joint MAP Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models
Ruolan Sun and Pawel Polak
Inference-time
alignment of pretrained text-to-image models is typically performed along a single control axis, such as classifier-free guidance,
attention editing, or reward-based latent perturbations. This limitation prevents modeling joint dependencies between conditioning and latent variables and hinders transfer across generative transports. We propose PG-MAP, a training-free framework that formulates inference-time
alignment as a trajectory-level Gibbs-MAP / proximal energy optimization over the conditioning $c$ and latent state $z_t$ via a forward-consistency coupling, optionally guided by a frozen preference reward. This joint formulation enables coordinated updates across modalities while remaining compatible with both
diffusion and flow-matching models through transport-specific adaptations. Across
diffusion backbones (SD~1.5, SDXL), PG-MAP consistently improves
alignment metrics such as PickScore and Aesthetic, and can be effectively combined with tuned classifier-free guidance to achieve the strongest overall performance. On flow-matching models (SD3.5-medium), the framework reduces to a latent-only variant, achieving $\mathbf{91.9\%}$ PickScore and $75.7\%$ HPS win rates against a static baseline, with controlled experiments ruling out noise-related artifacts. Human evaluations further confirm consistent preference over strong baselines, including tuned CFG and compute-matched universal guidance. Finally, an oracle-routing analysis shows that the relative importance of conditioning and latent optimization depends on
prompt types, surfacing further headroom that a per-
prompt selector could exploit.
Read the full abstract ▾
Computer Vision
Evo-RAD: Navigating Rare Retinal Disease Diagnosis via Self-Evolving Agentic Retrieval
Wangding Xia, Ye Du, Jiashi Lin et al. (+3 more)
Large-scale pretrained foundation models have revolutionized general medical screening, but often falter on rare diseases because such conditions are underrepresented in real-world clinical
datasets. While
retrieval-augmented diagnosis attempts to mitigate this, conventional static methods frequently succumb to the hubness problem, retrieving visually similar but semantically incorrect common diseases. To address this, we propose Evo-RAD, a self-evolving agentic framework that transforms evidence acquisition into a dynamic decision-making task. We formulate
retrieval as a Markov Decision Process (MDP) where a graphbased
agent observes the reference set state and executes actions to purge discordant evidence (DELETE), acquire pathologically consistent samples (INSERT), or conclude the
evolution (TERMINATE). Optimized via Group Relative Policy Optimization (GRPO) with a homogeneityaware reward, the
agent learns to maximize the diagnostic homogeneity of the support reference set. Experiments on retinal disease
benchmarks show that Evo-RAD substantially improves rare-disease diagnosis, outperforming retinal foundation models by +21.04%, while also surpassing
retrieval-based and parameter-efficient fine-tuning methods by +3.56%. Code is available at https://github.com/SDH-Lab/Evo-RAD.
Read the full abstract ▾
All authors Wangding Xia, Ye Du, Jiashi Lin, Meng Wang, Danli Shi, Shujun Wang
Computer Vision · Cross-listed from cs.AI
ENVS: Environment-Native Verified Search for Long-Horizon GUI Agents
Yincheng Zhou, Athena Zhuoming Zhong, Shijie Zhang et al. (+3 more)
As
multimodal agents move from interface understanding to real software control, successful trajectory discovery in live desktop environments becomes a key challenge. GUI tasks require long-horizon sequences of precise mouse and keyboard actions, while feedback is sparse, delayed, and costly to obtain through VM rollouts. We propose Environment-Native Verified Search (ENVS), a training-time search-and-filter pipeline that uses the environment to construct verified supervision before policy optimization: it branches over behaviorally distinct GUI actions in live OSWorld VMs, verifies successful leaves, and trains from globally balanced step-level supervision. To evaluate robustness under realistic desktop interruptions, we also introduce OSWorld-Noisy, a dynamic
benchmark for recoverable desktop interruptions that preserves the original tasks while testing whether
agents can refocus, dismiss, wait, or recover under live perturbations. On the 300-task OSWorld pool, ENVS reaches 30.3 pass@8 on original evaluations and 29.0 on OSWorld-Noisy, outperforming matched ARPO-style online RL while reducing compute from 184-192 to 138-153 GPU-hours; even with only 30% of its search data, ENVS reaches 27.0 pass@8, exceeding ARPO from the base model. Training from noisy environments also better preserves visual-
reasoning abilities on auxiliary
benchmarks, including OSWorld-G Refusal (16.7 vs. 1.9) and BLINK Functional Correspondence (26.2 vs. 23.1).
Read the full abstract ▾
All authors Yincheng Zhou, Athena Zhuoming Zhong, Shijie Zhang, Kevin Zhang, Teresa Xiaotao Shang, Shanghang Zhang
Computer Vision · Cross-listed from cs.GR
Controllable Texture Tiling with Transformed RoPE-Enhanced Diffusion Models
Junrong Huang, Zhiyuan Zhang, Rui Tang et al. (+2 more)
Realistic integration of user-specified textures into scene images is a fundamental task in computer graphics and image editing. While existing material transfer and reference-guided inpainting methods can edit surface appearances, they often fail to address the specific requirements of texture tiling. This task necessitates precisely repeating a reference pattern according to user-defined parameters such as frequency, orientation, and scale. Furthermore, current generative approaches often struggle to maintain the structural fidelity of the reference texture, limited by either destructive pixel-level resampling or the lack of fine-grained spatial information in semantic image encoders, and they frequently fail to preserve the coherent lighting and geometry of the original scene. In this paper, we propose a novel framework for controllable and high-fidelity texture tiling based on
Diffusion Transformers. Our approach introduces two key technical innovations to decouple spatial manipulation from content generation. First, we propose a Coordinate-Transformed Rotary
Embedding mechanism. By applying 2D affine transformations directly to the relative positional
embeddings between the target latent and the image condition, we achieve precise control over tiling patterns without explicit pixel warping, thereby utilizing the full information of the reference condition without degradation. Second, a Disjoint
Attention Mask is employed to shield reference features from semantic leakage. This preserves structural integrity while seamlessly blending the synthesized texture with the scene's original lighting and geometry. Extensive experiments demonstrate that our method outperforms state-of-the-art baselines in both control accuracy and texture fidelity.
Read the full abstract ▾
All authors Junrong Huang, Zhiyuan Zhang, Rui Tang, Hongbo Fu, Jnig Liao
Computer Vision
Evaluating self-supervised echocardiographic representations across downstream extraction strategies for left-ventricular segmentation and ejection fraction estimation
Sylwia Majchrowska and Philip Teare
Self-supervised learning (SSL) is increasingly used in medical imaging to reduce annotation requirements, but representation quality is often judged using a single downstream evaluation setting. For dense clinical tasks, this can confound representation quality with the capacity of the downstream model used to recover task-relevant information. We present a systematic evaluation of self-supervised representations for left-ventricular
segmentation and ejection fraction (EF) estimation from apical four-chamber echocardiography on EchoNet-Dynamic. Rather than relying on a single downstream probe, we compare a hierarchy of extraction strategies with increasing expressivity: heuristic extraction without mask-supervised training, frozen linear probes, frozen lightweight decoder probes, and partial fine-tuning. We apply this framework to two complementary representation families: generic frozen self-
DIstillation with NO labels (DINOv3) features and a task-adapted dense self-supervised representation, Bootstrap Your Own
Segmentation (BYOS). In both families, heuristic extraction substantially understated what was recoverable from the frozen representation. For DINOv3, performance improved from Dice 0.684 and EF mean absolute error (MAE) 13.01 under heuristic extraction to Dice 0.906 and EF MAE 9.65 with a frozen lightweight decoder, approaching a supervised U-Net baseline (Dice 0.915, EF MAE 9.72). For BYOS, performance improved from Dice 0.687 and EF MAE 17.83 under heuristic extraction to Dice 0.902 and EF MAE 8.74 with a frozen lightweight decoder. These results show that conclusions about self-supervised representation quality in dense echocardiographic analysis depend strongly on the downstream extraction strategy used for evaluation. We therefore argue that multi-strategy evaluation is an important methodological consideration for SSL in dense medical image analysis.
Read the full abstract ▾
Computer Vision
Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks
Minh-Loi Nguyen, Long-Bao Nguyen, Van-Hieu Huynh et al. (+2 more)
Deep neural networks have witnessed remarkable advancements in recent years and have become integral to various applications. However, alongside these developments, training and deployment of neural network models on
embedding and edge devices face significant challenges due to limited
memory and computational resources. These problems can be addressed with deep neural network compression, which involves a trade-off between model size and performance. In this paper, we propose a novel method for model compression through two phases. First, we utilize model compression techniques, such as
pruning and quantization, to significantly reduce the model size. Then, we use Mixture of Experts to route the previously compressed models to enhance performance while maintaining a balance in inference efficiency. MoEs consist of multiple expert models (i.e., compressed models) that are moderately sized and deliver stable performance. Experimental results on several
benchmark datasets show that our method successfully compresses CNN models which achieves substantial reductions in FLOPs and parameters with a negligible accuracy drop.
Read the full abstract ▾
All authors Minh-Loi Nguyen, Long-Bao Nguyen, Van-Hieu Huynh, Minh-Triet Tran, Trung-Nghia Le
Computer Vision
BEV-Denoise: Learning Intrinsic Noise for Accurate Bird's-Eye-View Semantic Segmentation
Dooseop Choi, Kyounghwan An and Kyoung-Wook Min
In this paper, we present a framework dubbed BEV-Denoise that estimates and removes intrinsic noise from learned Bird's-Eye-View (BEV) features to achieve accurate BEV semantic
segmentation. Inspired by the noise estimation capability of Denoising
Diffusion Probabilistic Models (DDPM), we design a UNet-based noise estimation module that learns to estimate the noise from the learned BEV features. The estimated noise is then subtracted from the BEV features and fed to BEV map decoders for the final prediction results. To facilitate supervision for the noise estimation module, we follow a sequential learning paradigm called Task Decomposition (TD) where a pre-trained BEV map autoencoder is employed to train a view transformation (VT) encoder. We share three key insights learned from our intensive experiments that are critical for improved performance. We apply our framework to four existing models, encompassing the three major VT paradigms. Experimental results on a large-scale real-world
dataset, nuScenes, demonstrate the effectiveness of our framework.
Read the full abstract ▾
Computer Vision
MythraGen: Two-Stage Retrieval Augmented Art Generation Framework
Quang-Khai Le, Cong-Long Nguyen, Minh-Triet Tran et al. (+1 more)
Text-to-image generation has seen rapid advancements, especially with the development of generative models. However, challenges remain in achieving high-quality, contextually accurate image outputs that faithfully match the provided textual descriptions, especially in artistic generation. In this paper, we present a simple yet efficient
retrieval augmented generation framework, namely MythraGen, for text-to-artistic image generation by integrating an art
retrieval mechanism with LoRA-based model fine-tuning. Our method extracts features from a large-scale art
dataset, optimizing the generation process by combining artist-specific styles and content. Particularly, retrieved images from an external art database that have the highest similarity to the query
prompt are used to finetune Stable
Diffusion using LoRA for desired art generation. Experimental results and user studies on the WikiArt
dataset show that our proposed method can generate artworks that closely match the user's input, significantly outperforming existing solutions.
Read the full abstract ▾
All authors Quang-Khai Le, Cong-Long Nguyen, Minh-Triet Tran, Trung-Nghia Le
Computer Vision
Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation
Yu Cao, Ziquan Liu, Zhensong Zhang et al. (+3 more)
Maintaining physical consistency in
video generators and world models increasingly relies on
vision-
language models (VLMs) as automated judges that provide reward signals, ranking decisions, and data-filtering criteria. Yet VLMs differ substantially in training data and architecture, encoding physical phenomena through distinct internal representations. A single global evaluation schema therefore gives every VLM the same axes of competence, regardless of what each can actually perceive. We propose JudgeFit, an iterative refinement procedure that discovers a per-VLM evaluation taxonomy. An initial taxonomy is constructed by prompting the target VLM to enumerate physics errors on a small set of
videos and clustering the resulting descriptions. The taxonomy is then refined through a diagnostic step: we calibrate the VLM's per-dimension scores to human physical-commonsense ratings, diagnose which dimensions it scores unreliably or redundantly, and
prompt an LLM to repair them, iterating until convergence. We further instantiate this procedure as a
benchmark and apply it to 16 VLMs spanning eight model families. The refined taxonomy outperforms the global-schema baseline on held-out
videos for every VLM tested, with a mean relative improvement of approximately 32%. Beyond aggregate accuracy, the per-VLM profiles expose model-specific blind spots that overall rankings cannot anticipate, with reliability patterns differing markedly across model families.
Read the full abstract ▾
All authors Yu Cao, Ziquan Liu, Zhensong Zhang, Jiankang Deng, Shaogang Gong, Jifei Song
Computer Vision
Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving
Zisheng Chen, Yuping Qiu, Jianhua Han et al. (+6 more)
Recent
Vision-
Language-Action (VLA) models have advanced end-to-end autonomous driving by incorporating
reasoning for better interpretability and planning quality. However, most existing approaches directly generate the final trajectory without explicitly examining its future consequences, which limits their reliability in complex and dynamic environments. To address this limitation, we propose IRR-Drive (Intend, Reflect, Refine), an adaptive
multimodal reflection framework for autonomous driving. Specifically, to tightly couple high-level
reasoning with physical constraints, IRR-Drive first generates a preliminary textual intention and anticipates potential interactions by predicting future semantic bird's-eye view (BEV) representations. This dual-modality (Text + BEV) reflection space explicitly models anticipated scene
evolution, enabling the model to rigorously self-correct and refine its initial intent before generating the final trajectory. Furthermore, to balance planning performance and computational efficiency, we construct reflection-oriented training data and design an adaptive reflection reward, enabling the model to adaptively select its
reasoning mode according to scene complexity. Instead of using
reasoning primarily as an auxiliary interpretation, IRR-Drive directly integrates an adaptive reflection mechanism into the planning framework, enabling grounded, decision-aware trajectory correction that is driven by scene complexity. Our method achieves state-of-the-art performance on the NAVSIM
benchmark in both PDMS and EPDMS. Extensive experiments demonstrate the effectiveness of our
multimodal reflection framework and validate the efficacy of the proposed adaptive reflection strategy.
Read the full abstract ▾
All authors Zisheng Chen, Yuping Qiu, Jianhua Han, Tao Tang, Xiuwei Chen, Likui Zhang, Ying-Cong Chen, Hang Xu, Xiaodan Liang
Computer Vision · Cross-listed from cs.RO
Improving Robotic Imitation Learning via Trajectory Standardization
Licheng Yang, Lingfeng Qian, Fei Zheng et al. (+4 more)
Imitation learning for robotic manipulation relies on large sets of human demonstration trajectories, which are often noisy and temporally irregular due to variable operator speed, intermittent pauses, and inconsistent action density. A common preprocessing strategy is time-uniform downsampling to shorten sequences, but it cannot effectively remove speed-induced non-uniformity or redundant pauses. This mismatch degrades data quality and hinders policy learning. To address this issue, we propose Information-Standardized Trajectory Resampling (ISR), an offline preprocessing method for effective imitation learning. ISR resamples each trajectory by enforcing approximately equal information distance between adjacent points. Specifically, we map trajectories onto an information-modulated Riemannian
manifold and perform geodesic-equidistant parameterization. We construct an information-intensity field from velocity and acceleration norms: the velocity term removes small-motion redundancy, while the acceleration term preserves high-
curvature and fine-manipulation phases. We evaluate ISR on three real-world manipulation tasks with mainstream imitation learning policies. Compared with the baseline time-uniform 3x downsampling, ISR improves task success rates by about 25%, remains robust across
datasets collected from different operators, and reduces both
dataset size and training cost. The code and
videos are publicly available at https://d-robotics-ai-lab.github.io/isr.page.
Read the full abstract ▾
All authors Licheng Yang, Lingfeng Qian, Fei Zheng, Yonghao He, Wei Sui, Shuangshuang Li, Hu Su
Computer Vision
InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars
Quanyue Song, Yishan He, Yanfei Zhang et al. (+6 more)
Recent
diffusion-based models have enabled realistic audio-driven avatar generation in real-time streaming. However, existing approaches struggle to maintain visual temporal consistency and fail to explicitly perceive user intent in complex interactive streaming scenarios. To address these challenges, we propose InteractiveAvatar, a real-time infinite-streaming
video generation framework that supports visually consistent avatar
video generation and intent-aware interactions. With autoregressive
distillation, InteractiveAvatar achieves real-time str-eaming generation of human avatars over arbitrarily long durations. For visual consistency, we introduce a Long-Short Visual
Memory (LSVM) mechanism that flexibly compresses historical visual information into compact
tokens, preserving both short-range coherence and long-term consistency. To generate avatars with
speeches and actions aligned with user intent, we propose a
Reasoning-Reaction Module (RRM), which incorporates a State-Cycling strategy and a Cache-Switching mechanism. Extensive experimental results over diverse scenarios demonstrate that our method achieves state-of-the-art visual consistency in long-duration generation, while enabling complex user-avatar interaction in real time.
Read the full abstract ▾
All authors Quanyue Song, Yishan He, Yanfei Zhang, Shihao Cheng, Zhixiang He, Zhizhi Guo, Chi Zhang, Xuelong Li, Caigui Jiang
Computer Vision · Cross-listed from eess.IV
IViT: A Novel Interpretable Visual Transformer for Skin Disease Detection
Haibiao Li, Di Lin, Xue Jiang et al. (+3 more)
The clinical diagnosis of skin diseases is susceptible to interference from inter-class similarity of skin lesions, and over-reliance on clinicians'experience easily leads to subjective bias. Although existing deep learning aided diagnosis methods achieve competitive accuracy, they suffer from the black-box opacity of
Vision Transformer (ViT) and poor adaptability to medical few-shot scenarios. Moreover, mainstream explainable
algorithms generally face the bottleneck of significant accuracy degradation when improving interpretability. This paper proposes an interpretable ViT (IViT) constrained by Quadratic Programming (QP). The introduced pre-trained transfer learning adapts to few-shot feature extraction. A discrete QP feature selection framework is constructed to screen generic and discriminative features consistent with clinical diagnostic logic. A multi-objective loss function is designed to reduce feature redundancy and optimize activation distribution while preserving classification performance. Experimental results on six standard skin disease
datasets show that IViT achieves an accuracy of 93.80%, only 0.21% lower than the baseline, with feature redundancy reduced by 29.5%. Its core activation regions are consistent with clinically concerned lesion areas. The proposed model balances accuracy and interpretability, providing a reliable solution for the clinical deployment of few-shot intelligent skin disease diagnosis.
Read the full abstract ▾
All authors Haibiao Li, Di Lin, Xue Jiang, Weiwei Wu, Yanxi Li, Yugang Chi
Computer Vision
PHOEBI: An Open-World Benchmark for Multi-Label Bacterial Identification in Phase-Contrast Microscopy
Aaditya Baranwal, Md Jahid Hasan and Shruti Vyas
Optical microscopy lets us watch living bacteria in real time without stains or labels, and it remains the standard instrument for identifying species across clinical, environmental and industrial microbiology. Yet real samples rarely hold a single species. They arrive as mixtures and sometimes carry an organism no model has ever seen. What happens when a model trained on familiar mixtures meets one it has never encountered? No
benchmark has asked this of phase-contrast images, so we built one. PHOEBI brings together 120,000 phase-contrast images of 40 mixtures of six bacterial species that we cultured and imaged ourselves, and it holds whole mixtures out of training so that every test is a combination the model has never seen. The answer is sobering. Classifiers that are nearly perfect on the mixtures they know collapse on new ones, and no choice of backbone or pretraining rescues them. The features turn out to be sound and the failure lies in how the evidence within each image is pooled. A culture looks the same everywhere, and decoders built on this simple fact hold steady where large networks fail even though one of them is never trained by gradient descent. The same features go further still. They notice when a species outside the catalogue appears, and the simplest way of giving it a class of its own proves the most effective while leaving the known species almost untouched. PHOEBI turns the mixed culture, the everyday case at the bench, into a measurable target for open-world recognition.
Read the full abstract ▾