Computer Vision
RealDenseFace: Real-time Monocular 3D Face Reconstruction from Dense UV-space Priors
Linzhou Li, Tianjia Shao and Kun Zhou
Recent
monocular 3D face reconstruction methods achieve high fidelity by fitting a 3D Morphable Model (3DMM) to dense priors predicted by networks, but the optimization stage is computationally expensive, often taking tens of seconds per image. We present RealDenseFace, a real-time optimization-based 3D face reconstruction method with dense UV-space network predictions. Our key idea is to formulate 3DMM fitting as a nonlinear least-squares problem and solve it with a tailored Gauss-Newton solver that converges in only a few iterations. The reconstruction is conducted in two stages. In the first stage, the network predicts two dense UV-space maps from a single RGB image: a correspondence map for UV-to-image
alignment, and a relative-depth map for
geometric constraints along the viewing direction. In the second stage, the solver fits per-vertex targets sampled from these maps at the vertex UV coordinates. The solver supports all three reconstruction settings: single-image fitting, offline sequence reconstruction, and online tracking. Our method achieves state-of-the-art accuracy on the NeRSemble SVFR
benchmark. The online tracker runs at 80+ FPS, and the offline sequence reconstruction is over 20 times faster than previous optimization-based baselines.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
DreOPD: Degraded-Reference Extrapolative On-Policy Distillation for Flow-matching Models
Mingfeng Lin, Chengfei Cai, Lin Xu et al. (+2 more)
Flow-matching models are now a mainstream method to image generation, but its adaptation to diverse downstream scenarios typically relies on post-training, which may cause conflicts among task-specific optimization objectives. Reinforcement learning enables direct optimization of task-specific rewards beyond the original models, yet trajectory-level optimization may incur high-variance gradients and cross-task interference. On-policy
distillation (OPD) offers dense and stable supervision on student rollouts, but conventional teacher matching remains imitation-based. We propose DreOPD, a Degraded-reference extrapolative OPD method for flow-matching models that bridges these two paradigms. Our DreOPD converts implicit reward extrapolation into closed-form velocity regression, enabling extrapolative post-training with the stability of OPD. It further uses a mildly degraded reference to strengthen the teacher-reference contrast, yielding a clearer extrapolation direction. Experiments on single- and multi-teacher settings show that DreOPD outperforms OPD and multi-task RL baselines in average performance, while surpassing specialized teachers on most metrics.
Read the full abstract ▾
All authors Mingfeng Lin, Chengfei Cai, Lin Xu, Yuxiang Wei, Liang Han
Computer Vision
BAG: Budget-Aware Gating for Diffusion Caching
Tong Zhao, Mingkun Lei, Yucheng Han et al. (+1 more)
Diffusion caching is a lightweight strategy that accelerates
Diffusion Transformers (DiTs) by reusing intermediate features across denoising steps, but existing paradigms face a fundamental trade-off: online heuristics lack global budget awareness, whereas static schedules lack instance adaptivity and fail to flexibly adapt to varying runtime budget constraints. To bridge this gap, we present BAG (Budget-Aware Gating), a novel caching policy that unifies global budget pacing with dynamic, instance-adaptive feature reuse. Rather than relying on hand-crafted rules, BAG employs a lightweight gating network that dynamically decides whether to execute a full computation or reuse cached features at each step by jointly conditioning on the budget state and local trajectory feedback. We train this policy via offline-to-online schedule
distillation, transferring the decision-making of offline-searched schedules into a compact online gate. Extensive experiments on FLUX.1-dev, Wan2.1, and Qwen-Image-2512 demonstrate that BAG consistently outperforms state-of-the-art caching methods across various speedup tiers while remaining robust across different resolutions, seeds, and guidance scales. Code will be released.
Read the full abstract ▾
All authors Tong Zhao, Mingkun Lei, Yucheng Han, Chi Zhang
Computer Vision
RL-Native Distillation: Exploiting Scored Trajectories for Few-Step Image Generation
Yuhan Li, Fangao Zeng, Sicong Kang et al. (+7 more)
Efficient text-to-image generation requires both reinforcement-learning (RL)-based reward
alignment and few-step
distillation, yet these procedures are typically performed sequentially, increasing training cost and risking the loss of reward gains during compression. We take an RL-native perspective:
diffusion RL already generates reward-scored finite-step trajectories, whose intermediate states provide
distillation supervision. Based on this insight, we propose REST (Reward-Enhanced Scored-Trajectory
Distillation), a single-stage co-training framework in which a decoupled student learns from the evolving RL teacher's trajectories without changing teacher optimization. Advantage-Modulated
Distillation (AMD) transforms rollout advantages into signed weights, strengthening imitation of preferred trajectories and aligning
distillation priorities with task value. The resulting framework is general and lightweight, requires no extra image rollouts, no separate
distillation dataset, and no
adversarial training. Experiments on compositional generation, visual text rendering, and human-preference
alignment demonstrate competitive few-step, CFG-free generation with RAM or DiffusionNFT teachers. With only four sampling steps, REST-RAM achieves a DrawBench PickScore of 23.97, outperforming both the 40-step RAM teacher (23.95) and RTDMD (23.71).
Read the full abstract ▾
All authors Yuhan Li, Fangao Zeng, Sicong Kang, Mengfei Xu, Hao Zhou, Wenxiang Shang, Wei Li, Pipei Huang, Bo Zheng, Bingbing Ni
Computer Vision
PatchHead: Learning Spatial Patch Evidence for Generalizable AI-Generated Image Detection
Shengbo Qi, Hongyi Fang, Benjia Zhou et al. (+1 more)
AI-generated image detectors generalize poorly when their training and test images originate from different generators or
datasets. Despite the rich spatial representations produced by
vision foundation models like DINO, existing detectors typically classify images using only the globally aggregated CLS
token. We hypothesize that globally aggregating DINO features into a single CLS
token obscures spatially distributed generation traces. To test this hypothesis, we introduce PatchHead, a lightweight spatial aggregation head that preserves the two-dimensional organization of DINO patch
tokens and integrates evidence across neighboring regions. During training, we freeze the pretrained DINO backbone and optimize only the inserted LoRA adapters, PatchHead, and auxiliary projection head. Across nine cross-
dataset benchmarks spanning manually curated and in-the-wild settings, PatchHead ranks first on seven
datasets and second on the remaining two. It improves the strongest prior method from 91.6% to 94.6% in average balanced accuracy (+3.0 points) and raises the worst-case accuracy from 82.4% to 89.4% (+6.9 points), while introducing only 8.6% more trainable parameters and 0.08% additional FLOPs. Further qualitative analysis suggests that PatchHead (i) reduces class-conditional domain discrepancy, and (ii) redirects the representation from content-dominated saliency toward spatially distributed authenticity evidence. Together, these observations provide a representation-level account of why spatial patch aggregation transfers more reliably across generators and
datasets than a single CLS-based global representation. Our code and models will be made available upon acceptance.
Read the full abstract ▾
All authors Shengbo Qi, Hongyi Fang, Benjia Zhou, Rui Mao
Computer Vision · Cross-listed from cs.LG
FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning
Radwan Selo, Majid Kundroo and Taehong Kim
Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data
privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in
dataset sizes, which can lead to biased model updates and hinder convergence. To address this, we propose FedTVD, a novel FL
algorithm that weights client contributions during aggregation by considering both data quality and quantity. Unlike traditional FL approaches such as FedAvg, which rely solely on
dataset size for client weighting, FedTVD integrates Total Variation Distance (TVD) to measure the divergence between each client's local label distribution and a uniform global distribution. Clients with highly skewed distributions receive lower weights, preventing unbalanced
datasets with imbalances from disproportionately influencing the global model. At the same time,
dataset size is incorporated to ensure scalability and
fairness. This dual-weighting mechanism effectively mitigates the impact of data imbalance, leading to more stable and generalized global models. Experimental results show that FedTVD consistently outperforms state-of-the-art methods across all
datasets (FMNIST, CIFAR-10, and CIFAR-100) and all levels of data heterogeneity. Notably, it achieves up to 10.6% improvement over FedAvg on CIFAR-10 under highly skewed data, while maintaining top performance even under moderate and IID settings.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning
Van Truong Vo, Khoa Nguyen and Taehong Kim
Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized
federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs. Foundational layers are responsible for maintaining network consensus, while specialized layers adapt to local data characteristics, leading to conflicting gradients and degraded performance under non-IID conditions. To address this fundamental tension, this work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals. By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination. Extensive evaluations across DFL
algorithms, various model architectures, and
datasets demonstrate that FedA2L achieves up to 4.94 times faster convergence than vanilla DFL and reduces communication rounds by up to 59% compared to scheduler-based baselines. Furthermore, FedA2L exhibits resilience to severe data heterogeneity, larger network sizes, and sparse
topologies, reducing communication overhead and establishing it as a versatile optimization tool for resource-constrained or large-scale distributed learning in edge and IoT deployments. The code is released at https://github.com/nclabteam/FedA2L.
Read the full abstract ▾
Computer Vision
NBA_Streaming: A Large-Scale Benchmark for Fine-Grained Basketball Commentary Generation in Continuous Streams
Lifang Wu, Yuyang Wu, Yangdong Gao et al. (+3 more)
Live basketball commentary generation requires determining when an event is sufficiently observable and describing it before subsequent events unfold. However, existing methods are primarily designed for pre-segmented clips or complete
videos, making them unsuitable for continuous streams. Existing
datasets also provide limited supervision for player identities, fine-grained actions, event attributes, and coherent event chains, restricting the factual richness of generated commentary. To address these limitations, we introduce NBA_Streaming, a large-scale
benchmark for online fine-grained basketball commentary generation. It contains 307.5 hours of basketball broadcasts and approximately 35K temporally aligned events, with annotations of event boundaries, player identities, fine-grained actions, event chains, and natural-
language commentary. By moving from isolated clips to continuous streams, NBA_Streaming enables unified evaluation of event localization, response reliability, factual grounding, and commentary quality under causal constraints. We further propose a causal two-stage framework that combines completion-first localization with ball-centric semantic grounding, enabling the system to identify complete events from observed streams and organize scene, event, identity, and action cues for commentary generation. Extensive experiments reveal the difficulty of NBA_Streaming, where existing baselines struggle with online timing, factual grounding, and fine-grained description. Our framework consistently improves over strong alternatives, while the remaining gap highlights NBA_Streaming as a valuable
benchmark for streaming sports
video understanding and generation. The
dataset is publicly available at https://github.com/wyy-081/NBA_Streaming.
Read the full abstract ▾
All authors Lifang Wu, Yuyang Wu, Yangdong Gao, Fengyu Liu, Ya Jing, Liang Wang
Computer Vision
RAGMesh with FaME-G2E: Long-Form Text-Driven 3D Face Generation and Editing
Hao Li, Ju Dai, Feng Zhou et al. (+6 more)
Text-driven 3D face generation and editing remains challenging due to the difficulty of translating long-form descriptions into fine-grained facial geometry. Existing methods primarily align global textual semantics with facial structures but often struggle to capture subtle local deformations, such as eyebrow tension, cheek contraction, and asymmetric mouth motions, resulting in limited
geometric fidelity and editing precision. To facilitate fine-grained text-driven facial modeling, we first construct FaME-G2E, a large-scale
multimodal dataset containing detailed text–mesh annotations and paired text–blendshape samples for unified 3D facial generation and editing. Based on this
dataset, we propose RAGMesh, a
retrieval-augmented framework that leverages text-correlated
geometric priors to improve high-fidelity facial synthesis and editing. Specifically, the Multi-Scale
Retrieval Fusion (MSRF) module retrieves semantically consistent global and regional facial priors and fuses them in the blendshape space, suppressing conflicting local deformations while preserving coherent deformation patterns. Furthermore, we introduce Adaptive RAG-guided Supervision (AdaRAGS), a region-aware constraint that explicitly aligns textual semantics with corresponding facial regions, enhancing regional controllability and editing accuracy. Extensive experiments on FaME-G2E demonstrate that RAGMesh achieves superior performance over state-of-the-art methods in local
geometric accuracy, text-guided controllability, regional editing precision, and inference efficiency.
Video demo is available at https://youtu.be/Yr0_XkpWcNk, and the source code and
dataset will be released upon paper acceptance.
Read the full abstract ▾
All authors Hao Li, Ju Dai, Feng Zhou, Mengting Shi, Haofei Wang, Zhen Song, Wei Zhou, Lei Li, Junjun Pan
Computer Vision
Rethinking Medical Landmark Localization with Prototype Learning-based Progressive Offset Correction
Jingxian Xu, Yuhao Huang, Rusi Chen et al. (+2 more)
Accurate landmark localization in medical images is a fundamental step for quantitative clinical measurement and downstream analysis. Existing localization methods have advanced, among which multi-stage refinement is a superior solution. Although this strategy mitigates the anatomical ambiguity inherent in single-stage global predictions, its high computational cost limits practical applicability. In this work, we propose a parameter-economic model, PPOC-LL, which leverages Prototype learning-based Progressive Offset Correction for Landmark Localization. Our contribution is three-fold. First, to drive coarse-to-fine landmark optimization, we introduce a multi-scale dynamic perception strategy for patch-level feature pyramid modeling. Second, to effectively handle anatomically similar patterns, we design a similarity-driven prototype learning mechanism that captures informative local semantics for robust offset prediction. Last, to stabilize the model learning and improve the overall performance, we incorporate a novel error-aware reliability regularization via tolerance-based balancing. We collected a large validation cohort, including two public and one private
datasets spanning X-ray and ultrasound modalities, covering cephalometric, symphysis-fetal head, and fetal heart landmarks. Extensive experiments demonstrate that PPOC-LL achieves satisfactory performance with a favorable trade-off between accuracy and model complexity.
Read the full abstract ▾
All authors Jingxian Xu, Yuhao Huang, Rusi Chen, Yanfeng Zhou, Dong Ni
Computer Vision
Not All Visual Tokens Are Equally Safe to Remove:Consequence-Sensitive Visual Token Compression
Jingbo Wen, Liang He, Mingyu Cao et al. (+4 more)
Visual
token compression for
vision–
language models (VLMs) has largely relied on criteria such as
attention, redundancy, and uncertainty to maximize average accuracy under a fixed compute budget, implicitly assuming that all errors carry equal cost. However, the consequence of an incorrect prediction on downstream tasks is rarely symmetric: misreading an invoice amount can be far more costly than misclassifying a background color. Motivated by this, we introduce consequence-sensitive visual
token compression, which allocates visual computation across requests according to their potential error costs. Our method follows a calibrate-then-allocate procedure, estimating consequence-specific error-budget curves offline and applying the calibrated
token budgets online using consequence signals available from question or task information. On a controlled within-task
benchmark, high- and low-consequence questions are drawn from the same document images, so content alone cannot reveal which questions are costly to get wrong. In this setting, our method reduces high-stakes errors from 0.300 to 0.133 under the same total
token budget, whereas a content-driven allocator performs no better than uniform allocation. Measuring how error rates change with
token budget across different cost ratios, we derive an allocation frontier: uniform allocation is optimal when errors are equally costly, and
token transfer toward high-consequence questions becomes increasingly beneficial as the cost gap grows. This allocation principle generalizes well across three dense
vision-
language benchmarks, two budget realization mechanisms (
token deletion and resolution reallocation), two VLM architectures, and multiple
token selection strategies. On a realistic mixed workload, consequence-sensitive allocation reduces cost-weighted error by 38% while achieving approximately 21% lower latency than full-resolution inference.
Read the full abstract ▾
All authors Jingbo Wen, Liang He, Mingyu Cao, Haoyu Wang, Minxuan Hu, Kangning Cui, Xilu Wang
Computer Vision
LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search
Yulun Zhang, Zixu Li, Zhiwei Chen et al. (+6 more)
Traditional Text-based Person Search (TPS) is typically limited to matching static appearance attributes, severely neglecting dynamic action information. The Text-based Person Anomaly Search (TPAS) task bridges this gap, requiring models to locate micro-level specific abnormal behaviors while matching macro-level appearance of pedestrians. However, current TPAS methods face fundamental limitations: external explicit pose
estimators are fragile in unconstrained surveillance scenarios, and implicit learning encounters visual decoupling failure under pixel-level
entanglement, causing dominant appearance information to easily swallow and contaminate subtle action features. Furthermore, performing contrastive optimization on hard negative samples (“same appearance, different actions”) in conventional Euclidean spaces induces severe shortcut learning. To address these, we propose the Lightweight Action Inversion and Riemannian rectification network (LightAIR). First, it introduces textual semantic priors as anchors via a lightweight action inversion operator to extract pure action features, thereby overcoming visual-inherent coupling. Subsequently, it employs orthogonal null-space projection to constrain appearance features within the orthogonal complement space of action features, guaranteeing strict forward decoupling. Finally, we designed a gradient rectification module that computes the Riemannian gradient to constrain the backpropagation trajectory, forcing the gradient flow to update strictly along the tangent space that preserves decoupling properties, thereby cutting off harmful shortcuts. Extensive experiments on the widely used TPAS and TIPR
datasets demonstrate that LightAIR significantly outperforms existing state-of-the-art methods. Codes are available at https://github.com/rainy-london/LightAIR
Read the full abstract ▾
All authors Yulun Zhang, Zixu Li, Zhiwei Chen, Zhiheng Fu, Wenbo Wang, Zihang Qiu, Zhilin Wang, Ruxin Wang, Yupeng Hu
Computer Vision
OGG-FR: Orthogonal Gradient Gaming and Frequency Rectification for Unmanned Aerial Vehicle Infrared Image Super-Resolution
Yongsong Huang, Qingzhong Wang, Xiaofeng Liu et al. (+3 more)
Unmanned aerial vehicle (
UAV) infrared image super-resolution aims to recover weak thermal structures for deployment on resource-constrained platforms; lightweight models are therefore preferred, but multi-loss training can be unstable. A common strategy combines pixel-domain and frequency-domain objectives; however, low contrast, limited high-frequency content, and sensor-specific noise often make their gradients weakly aligned or conflicting. To address this optimization ambiguity, we propose Orthogonal Gradient Gaming and Frequency Rectification (OGG-FR), a plug-and-play optimization framework that decomposes the frequency gradient into a redundant parallel component and an orthogonal innovation component relative to the pixel gradient. In the conflict regime, OGG-FR computes a safe base gradient using the Multiple Gradient Descent
Algorithm (MGDA) and adds a variance-rectified orthogonal innovation; in the compatible regime, it discards redundant parallel information and injects the orthogonal innovation according to a confidence score estimated from the high-frequency residual. Experimental results on the
UAV thermal
benchmark show broad gains under BI and BD degradations at $\times 4$ and $\times 8$ scales, while gradient analyses support the effectiveness of the proposed conflict-aware update rule.
Read the full abstract ▾
All authors Yongsong Huang, Qingzhong Wang, Xiaofeng Liu, Tomo Miyazaki, Yaohou Fan, Shinichiro Omachi
Computer Vision
RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detection
Zhihao Zhang, Gengwei Zhang, Tianlong Chen et al. (+1 more)
Monocular 3D object detection spans two regimes: closed-set detectors operating within a fixed category vocabulary, and open-vocabulary detectors that localize arbitrary categories by leveraging depth foundation models for 3D geometry. We find that current depth foundation models, despite their strong zero-shot generalization, lack the object-level precision 3D detection demands: substituting a state-of-the-art depth foundation model for a strong detector's predicted depth degrades accuracy, even falling below the detector's own prediction. Rather than pushing detectors or depth models to be more accurate end-to-end, we treat object-level depth refinement as a stand-alone task and present RefineAny3D, a
vision-
language model that corrects depth without ever predicting a numerical value. Our key insight is that depth error has a direct visual signature in image space: when projected onto the image, a correctly placed box tightly encloses the object, while a too-far box projects too small and a too-close box projects too large. Depth refinement thus reduces to a visual
alignment problem rather than a metric regression problem, which we instantiate by extending the VLM's vocabulary with action
tokens that replace numerical depth output with categorical decisions, and by supervising the model on a large-scale chain-of-thought
dataset that grounds each decision in explicit visual evidence. Applied as a single post-hoc step, RefineAny3D delivers consistent gains across closed-set detectors, open-vocabulary detectors, and 3D auto-labeling tools, and generalizes to novel categories, scenes, and cameras without retraining.
Read the full abstract ▾
All authors Zhihao Zhang, Gengwei Zhang, Tianlong Chen, Xiaoming Liu
Computer Vision
Multi-Submap Implicit Neural SLAM with Local-to-Global Loop Closure for Large-Scale Scene Reconstruction
Tianchen Deng, Chongdi Wang, Nailin Wang et al. (+6 more)
Neural Radiance Fields (NeRF)-based SLAM has demonstrated impressive results in small-scale scene reconstruction, yet scaling these methods to extensive, complex environments remains challenging due to catastrophic forgetting and accumulated trajectory drift. This paper presents a robust, large-scale neural SLAM system featuring a multi-submap architecture and a dual-tier loop closure mechanism. Specifically, we propose a progressive mapping strategy that dynamically allocates neural submaps to maintain high-fidelity representations without
memory explosion. For robust pose estimation, an optical-flow-based tracking module is integrated to handle aggressive motions. To address global consistency, we introduce a local-to-global loop closure framework leveraging the foundation model for high-performance global descriptor extraction, significantly enhancing relocalization accuracy under varying viewpoints. Furthermore, an inter-submap online
distillation algorithm is designed during back-end optimization to enforce
geometric and appearance consistency across overlapping submap boundaries. To validate the system, we developed a customized handheld mechatronic platform and conducted extensive evaluations on both public
benchmarks and our large-scale indoor-outdoor
datasets. Experimental results, including direct deployment on an onboard computing unit, demonstrate that our approach outperforms state-of-the-art neural SLAM methods in reconstruction quality and localization robustness, providing a scalable solution for real-world robotic perception and digital twinning. We will release the code publicly on \href{https://github.com/dtc111111/MSN-SLAM}{https://github.com/dtc111111/MSN-SLAM} .
Read the full abstract ▾
All authors Tianchen Deng, Chongdi Wang, Nailin Wang, Lei Zhao, Ziqi Ma, Tianjun Zhang, Zhe Liu, Danwei Wang, Hesheng Wang
Computer Vision
Right Answer, Wrong Heat: Explanation-Aware Evaluation and Thermal-Grounded Feedback for MLLMs on Infrared Images
Yongsong Huang, Xiaofeng Liu, Tomo Miyazaki et al. (+2 more)
General-purpose
multimodal large
language models (MLLMs) are increasingly applied to infrared images, where they are commonly scored by answer accuracy alone. However, a correct answer does not ensure that the model's explanation is grounded in infrared thermal evidence. We introduce an explanation-aware evaluation framework that separates answer correctness, output-level explanation groundedness, and thermal grounding for infrared visual questions. Using a Dual-LLM Consensus Judge with a preliminary human-anchor calibration check, we find that correct answers can still rely on weak or visible-light evidence; withholding the original infrared image and showing only a visible-like rendering erodes thermal grounding with little accuracy change; and this erosion is observed most strongly for more capable models but disappears when infrared remains available. We further propose Thermal-Grounded Feedback (TGF), a training-free feedback loop that diagnoses explanation-side failures and revises the explanation while preserving the selected answer. On local paired-input validation, TGF improves explanation-side grounding without changing answers. These findings suggest that future trustworthy MLLMs for infrared scene understanding should be evaluated and developed to produce thermally grounded explanations rather than merely accurate answers.
Read the full abstract ▾
All authors Yongsong Huang, Xiaofeng Liu, Tomo Miyazaki, Yaohou Fan, Shinichiro Omachi
Computer Vision
UniMoFlow: Grounding Instruction-Driven 3D Human Motion Editing in Generation
Yilei Hua, Beibei Jing, Ce Zheng et al. (+3 more)
Instruction-driven editing of 3D human motion requires precise spatiotemporal localization, rich semantic grounding, and strict preservation of unmodified content. Existing methods either resort to training-free adaptation of generative models or rely solely on triplet supervision; however, adaptation often yields suboptimal control, and manually curated triplet
datasets remain severely limited in scale and semantic diversity. To overcome this bottleneck, we ground motion editing directly within text-to-motion generation across data, architecture, and inference. At the data level, we develop a closed-loop synthesis-and-verification pipeline that produces Omni-MoEdit, a large-scale
dataset spanning body-part, amplitude, temporal, action, and style edits. At the architectural level, we introduce UniMoFlow, a unified latent flow-matching model that shares broad semantic and kinematic knowledge between generation and editing. At the inference level, SAFE (Source-Anchored Flow Editing) complements UniMoFlow with controllable, source-anchored refinement. Furthermore, we augment standard evaluations with semantics-aware metrics to account for valid edits that inherently deviate from a single ground-truth reference. Extensive experiments demonstrate improved target-text
alignment, edit effectiveness, and cycle consistency, while maintaining competitive source fidelity and text-to-motion generation quality.
Read the full abstract ▾
All authors Yilei Hua, Beibei Jing, Ce Zheng, Hanyu Zhou, Yawei Luo, Wei Yang
Computer Vision
CodecArena: Codec Quality Assessment via Visual Reinforcement Learning
Jiaye Fu, Weiqi Li, Qiankun Gao et al. (+5 more)
Video coding is advancing into the low and ultra-low bitrate regime, driven by end-to-end codecs that replace the hand-crafted pipeline with jointly optimized neural networks and generative codecs that exploit the priors of
video generation models. Yet the dominant metrics, LPIPS and DISTS, measure feature and texture similarity rather than content fidelity: a reconstruction that hallucinates a wrong face or blurs text into convincing strokes can still score well, even when a human rejects it instantly. To address this, we propose CodecArena, the first
vision-
language framework for
video coding quality assessment, casting codec evaluation as source-conditioned comparative
reasoning between a reference and its reconstructions. We optimize CodecArena with Facet-GRPO, a visual reinforcement learning scheme that aligns pairwise codec preferences while grounding the verdict in five fidelity facets: identity, objects, text, texture, and temporal consistency. Its facet-anchored reward uses automatically derived facet directions as weak anchors, rather than human per-facet labels, to prevent any single sub-score from dominating the holistic preference and to yield interpretable fine-grained quality judgments. To support training and evaluation in this underexplored regime, we construct two complementary resources: CodecArena-1K, a fully automatic preference
dataset of 1,500 comparison groups built from traditional, neural, and generative codec reconstructions with fused
vision-
language and objective supervision; and CodecArena-Bench, a human-ranked
benchmark with source-disjoint
videos for fair out-of-domain evaluation. Extensive experiments demonstrate that CodecArena achieves state-of-the-art agreement with human judgments on source-disjoint content across diverse codecs and bitrates, surpassing perceptual metrics and prior
vision-
language evaluators.
Read the full abstract ▾
All authors Jiaye Fu, Weiqi Li, Qiankun Gao, Yanchen Zhao, Xiandong Meng, Jian Zhang, Siwei Ma, Jiaqi Zhang
Computer Vision
Bright-Channel Retinex Enhancement with a Conditional Overdispered-Noise Analysis
Jongpil Jeong
I present a training-free low-light enhancement method that combines local bright-channel illumination estimation, Retinex division, and edge-preserving denoising. For a fixed illumination estimate, a conditional Negative -Binominal psueduo-count method characterises the heteroscedastic noise amplified by division. The unconstrained reflectance ratio is the pixelwise maximum-likelihood estimate, with a boundary solution for zero-valued observations; the implemented estimate additionally applies illumination filtering and range clipping. The NB model is a diagnostic noise analysis rather than a calibrated sensor model, and the final fixed-bandwidth bilateral filter is an empirical approximation rather than the exact
Bayesian solution. On the LOL-v1
dataset, the methodobtains mean PSNR/SSIM of 17.74dB/0.739, the highest values among the evaluated with conventional methods. A 400X600 image is processed at approximately 43 FPS on an Apple M2 Pro CPU.
Read the full abstract ▾
Computer Vision
When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution
Yu Shi, Yuyao Zhang and Yu-wing Tai
Image super-resolution (SR) with large generative models has recently achieved remarkable perceptual quality, yet maintaining fidelity to the LR observation remains challenging. In particular, we observe that
diffusion transformers (DiTs) built on latent representations suffer from a critical limitation: the compression bottleneck of the VAE weakens fine-grained spatial information, leading to hallucinated details that are weakly grounded in the input image. In this work, we revisit generative SR from a representation perspective and propose a pixel-grounded super-resolution (PGSR) framework that preserves LR-observed pixel evidence before VAE compression and reuses it throughout restoration. Instead of relying solely on the compressed latent condition, PGSR extracts pre-VAE pixel evidence from the upsampled LR image and reuses it at two stages. First, Condition-Side Trajectory Guidance fuses LR-derived pixel evidence with the latent LR condition to guide the latent restoration trajectory. Second, Decoder-Side Pixel Grounding injects multi-scale pixel features into the frozen VAE decoder to ground the final rendering with LR-observed cues. To efficiently adapt large pretrained DiT models, we keep the latent autoencoder and main flow-matching backbone frozen, and train only lightweight restoration modules. We further study an efficient local-window
attention variant for improved high-resolution efficiency and scalability. Extensive experiments demonstrate that PGSR improves the realism–fidelity trade-off and produces more faithful, visually convincing results than existing latent generative SR approaches.
Read the full abstract ▾
Computer Vision
Visual Distortion Detection in UGC Images Using Large Multimodal Models
Ziheng Jia, Yingji Liang, Jiaying Qian et al. (+1 more)
The localized depiction of perceptual quality has long been a crucial, yet underexplored, challenge in image quality assessment (IQA). Existing approaches based on large
multimodal models (LMMs) predominantly rely on text-driven supervised fine-tuning (SFT). However, this training paradigm exhibits notable limitations in detection accuracy. Moreover, synthetically distorted images, which are often used as the primary training data source, show a significant generalization gap when deployed in real-world scenarios; thus, the \textbf{synthetic-to-authentic (S2A)} problem represents a critical challenge. Motivated by these issues, we propose \textbf{VIGIL}, which leverages the LMM architecture for precise visual distortion detection. From a candidate pool of over 1000K samples, we construct the \textbf{VIGIL-140K} training set, which consists of over 140K distorted images. These images are obtained through rigorous quality filtering and carefully crafted distortion injection, covering 8 major synthetic distortion categories. Our model leverages different layers of the large
language model (LLM) decoder, treating them as multiple detectors that perform synchronous distortion detection using multi-level features. Additionally, we retain distortion cues from predictions assigned to the non-distortion class, which helps mitigate the ambiguous foreground-background (FG-BG) separation commonly encountered in the S2A problem. After post-processing, our model consistently outperforms strong baselines on both in-domain synthetic distortion detection and S2A tasks.
Read the full abstract ▾
All authors Ziheng Jia, Yingji Liang, Jiaying Qian, Xiongkuo Min
Computer Vision
View-Adaptive Renderer for View-Consistent 2D-to-3D Generation
U-Chae Jun, Jaeeun Ko and Jiwoo Kang
Reconstructing 3D shapes from a single image remains a fundamental yet challenging problem in computer
vision. Traditional
monocular 3D generation pipelines typically synthesize multiple views from a single input image before applying Neural Radiance Field (NeRF)-based reconstruction. However, inherent projective ambiguities often produce visual discontinuities across generated viewpoints, leading to inaccuracies in reconstructed 3D models. Current solutions either incur significant additional computational burdens or fail to adequately resolve practical inconsistencies between synthesized views. To address these limitations, we propose a novel viewpoint-adaptive neural rendering framework that enables robust 3D reconstruction even when given partially inconsistent multi-view inputs. Our approach introduces view-adaptive neural renderers that independently correct viewpoint-dependent errors while simultaneously sharing a global feature backbone to preserve structural coherence. Furthermore, we propose a self-
attention fusion module that adaptively integrates multi-view information, ensuring
geometric consistency without relying heavily on indirect regularizations or computationally intensive methods. Through extensive experiments, we demonstrate that our method consistently improves 3D reconstruction fidelity. Importantly, our approach achieves near state-of-the-art performance without
diffusion-based SDS supervision, relying primarily on photometric rendering loss with lightweight
attention regularizers. This balance between accuracy and efficiency makes the proposed framework highly practical for real-world applications.
Read the full abstract ▾
Computer Vision
Contrastive Mask Fidelity: Reference-Free Auditing of Ground-Truth Masks in Remote Sensing Semantic Segmentation
Shuaishuai Cao, Shuwei Peng, Meng Tang et al. (+5 more)
Semantic
segmentation models are trained and evaluated against human-drawn masks, yet remote-sensing annotations are often coarse, incomplete, or misaligned; high overlap scores may then reflect agreement with imperfect labels rather than faithfulness to the image, creating an evaluation paradox. We introduce Contrastive Mask Fidelity (CMF), a training-free, reference-free metric that scores competing class masks directly against image evidence. CMF composites keep and erase counterfactual views of each mask and asks a frozen
vision-
language judge whether class evidence is concentrated inside the mask and absent outside. We validate CMF on controlled mask corruptions, then audit 10,731 image-class pairs across ten remote-sensing
benchmarks using candidate masks from Seg-Probe, a training-free open-vocabulary probe built on SegEarth-OV3 that outperforms prior baselines on nine of ten
datasets. The audit reveals systematic, class-dependent annotation distortion: man-made classes such as buildings, roads, and cars favor the candidate mask on 62-85% of pairs, whereas ambiguous land cover more often favors human annotations. On a blinded three-annotator consensus, CMF matches expert judgment on 81% of pairs, exceeding keep-only scoring, model confidence, and a trained label-quality baseline. Finally, conservative class-wise arbitration yields supervision that improves cross-domain transfer over raw annotations and matched replacement controls, positioning CMF as a scalable tool for
auditing ground truth rather than presuming it infallible.
Read the full abstract ▾
All authors Shuaishuai Cao, Shuwei Peng, Meng Tang, Min Huang, Youjin Wang, Jie Chen, Jing Ouyang, Zhiwei Zhai
Computer Vision · Cross-listed from cs.LG
Real Data Closes Synthetic-to-Real Gap in Optical Chemical Structure Recognition
Yani Guan, Dengpan Dong, Zi Wei et al. (+4 more)
Millions of chemical structures appear in patents and papers only as drawings, and using that information at scale requires reading the drawings. OCSR appears nearly solved on synthetic images yet remains difficult on real documents: the starting recognizer, Qwen2.5-VL-7B, exceeds 91% accuracy on synthetic renders but falls below 16% on three real-world
benchmarks (ACS, CLEF-IP, USPTO). To identify the main source of improvement, 21 recognizers were fine-tuned on mixtures of synthetically rendered structures and labeled real depictions from patents, journal figures, and hand-drawn collections, varying the
vision language model (VLM) base, the fraction of real training data, and the
vision-tower adaptation strategy. Labeled real training images make the largest difference. For Qwen2.5-VL, ACS exact match rises from 0.15 with no real data to 0.37 at 9.5% and 0.46 at 50.2%; a controlled experiment across three base models reproduces the trend. A
vision-tower LoRA, in contrast, does nothing for Qwen (+0.00, paired p=1.00), substantially helps InternVL3-8B (+22.8 to +34.6 pt), and modestly helps GLM-4.1V-9B (+1.0 to +9.6 pt), so its value depends on the base model. The best configuration reaches 0.96 exact match on clean renders and 0.49, 0.65, 0.84, and 0.76 on ACS, CLEF-IP, UOB, and USPTO, respectively. Gaps between base models are largest without real data (0.21), shrink to 0.06 at 70% real data, and reorder the ranking; base model and real-data mixture must therefore be selected together. Small-scale experiments on handwritten image-to-LaTeX recognition and chart-to-table conversion show that base-model rankings also vary beyond chemistry. More generally, model and adaptation choices for visual structure recognition should be evaluated on the target task.
Read the full abstract ▾
All authors Yani Guan, Dengpan Dong, Zi Wei, Shuang Luo, Dan Hannah, Yumin Zhang, Kang Xu
Computer Vision
SI-Edit: Toward Sketch-Instruction Guided Local Image Editing with Pixel-Level Precision
Weixin Ye, Wei Wang, Hongguang Zhu et al. (+1 more)
Despite rapid advances in generative models, achieving pixel-level precision in sketch-based image editing remains a persistent challenge, particularly for fine-grained local deformations. This gap stems primarily from the critical shortage of high-quality, publicly available
benchmark datasets that jointly provide
geometric constraints and semantic instructions. To address this issue, we first introduce **SI-Data**, a high-quality
dataset specifically designed for instruction-guided local sketch editing. We develop an automated pipeline leveraging
Multimodal Large
Language Models (MLLMs) to synthesize comprehensive quadruplets comprising original images, local
geometric sketches, semantic instructions, and corresponding edited images. By providing both reliable spatial anchors and explicit semantic intent, SI-Data uniquely enables collaborative spatial-semantic learning. Building upon this, we propose a collaborative framework called **SI-Edit** that integrates semantic instructions with precise
geometric constraints. Furthermore, to address the lack of standardized evaluation, we establish a comprehensive set of metrics designed to measure both structural fidelity (e.g., sketch-to-edge
alignment) and semantic adherence. Experimental results demonstrate that SI-Edit provides more reliable structural control than baselines for sketch-based image editing, and achieves precise, pixel-level local refinements aligned with user intent. The data and code are released on the [project page](https://github.com/ywxsuperstar/SIEdit).
Read the full abstract ▾
All authors Weixin Ye, Wei Wang, Hongguang Zhu, Xuecheng Nie
Computer Vision
TLDChoiceNet: Quantitatively Choosing a Transfer Learning Dataset
Jing Ning and James D. Braza
Transfer learning is particularly useful in settings with limited training data, and within image classification it is common to transfer learn upon massive
datasets like ImageNet , CIFAR-100, or COCO . Qualitatively, it seems a transfer learning
dataset should have both more classes and more examples per class than the fine tuning
dataset; however, a quantitative method to choose the best transfer learning
dataset does not currently exist. In this paper, we design TLDChoiceNet, a model to choose the best transfer learning
dataset given a fine tuning
dataset by predicting the test-set accuracy after fine-tuning. A simple version 1 achieves 0.154 MSE on the test
dataset, while a version 2 leveraging an ImageNet pre-trained ResNet50 v2
embedding with per-class information attains a 5X lower MSE of 0.031. We further design two metrics that enable an unsupervised method of choosing an optimal transfer learning
dataset: distribution distance (DD), which linearly regresses against fine-tune accuracy with an R2 of 0.89, and average class correlation (ACC), which improves the R2 to 0.97. Our results underscore that a
dataset's low-level statistics can explain the transfer learning effect, and that using a pre-trained ImageNet can embed different classes further apart in latent feature space.
Read the full abstract ▾
Computer Vision
Learning human joint torques from pixels
Chen Chen and Rui Cheng
Estimating human joint torques from visual observations is a key step toward bringing biomechanical analysis from controlled laboratories to real-world movement scenarios. Existing torque estimation methods typically depend on surface electromyography, motion-capture markers, force plates, or simulated imitation data, which limits their applicability to ordinary RGB images. In this work, we introduce VID, a
vision-based inverse dynamics
dataset and
benchmark for predicting human joint torques directly from real
monocular images. VID contains 63,369 synchronized frames with real human images, kinematic annotations, anthropometric attributes, and OpenSim-derived dynamic labels, providing paired visual and biomechanical supervision for real-image inverse dynamics. We further define a standardized evaluation protocol covering overall torque estimation, joint-specific analysis, and action-specific prediction. To establish a strong reference model, we propose VID-Network, which combines pose-pretrained spatial probabilistic features, marker regression, and temporal torque inference to recover joint torques from image sequences. Experiments on VID show that VID-Network achieves an overall mPJE of 1.7612 N$\cdot$m/kg, improving over the best compared baseline by 39.81%, and obtains the lowest error across all evaluated joint types and most action categories. VID establishes a first practical
benchmark for
vision-driven human inverse dynamics and provides a foundation for studying biomechanical inference in less constrained environments.
Read the full abstract ▾
Computer Vision
Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective
Hongyi Fang, Chuwen Xie, Benjia Zhou et al. (+6 more)
Next-scale visual autoregressive models (VARs) have emerged as a powerful generative paradigm, producing high-quality images through efficient coarse-to-fine prediction. However, their potential for text-guided image editing remains largely underexplored. Existing training-free VAR editing approaches often formulate editing as target-conditioned regeneration guided or constrained by the source image, and may rely on inversion, test-time optimization,
attention control, or user-provided masks. This generation-centric formulation does not fully exploit the multiscale source representations provided by VARs and may introduce additional computation or intervention. We instead take a source-centric perspective on VAR editing, in which the encoded source image
tokens serve as the primary visual state and the editing process focuses on condition-induced changes. Based on this perspective, we propose EditMod, which compares source- and target-conditioned predictions under a shared autoregressive context, treats their difference as a scale-wise editing direction, and applies it as a residual update to source
tokens at selected scales. Experiments show that EditMod achieves leading source-image fidelity while maintaining strong text
alignment, and completes end-to-end editing of a 1K image in only 1.57 seconds on a single A100 GPU without per-image preparation.
Read the full abstract ▾
All authors Hongyi Fang, Chuwen Xie, Benjia Zhou, Yu-Xuan Qiu, Chenggong Hu, Zhibin Wang, Chao Chen, Jianbin Qin, Rui Mao
Computer Vision · Cross-listed from eess.SP
Diagnosing as Cardiologists Do: ECG Agents with Doctor-Grounded Priors for Clinical Reasoning Across Diseases and Populations
Hongxiang Gao, He-yang Xu, Yuwen Li et al. (+6 more)
Cardiologists interpret electrocardiograms by localizing waveform components, measuring rhythm and interval patterns, and translating these structured observations into diagnostic evidence. Whether this expert reading process can serve as an effective prior for ECG
agents remains unclear. To address this question, we introduce LuminaECG, a clinically structured ECG
reasoning framework that reformulates ECG interpretation as measurement-grounded visual reading. ECG signals are rendered on standard electrocardiographic grid paper to preserve the spatial and scale cues used in clinical reading. P-wave, QRS-complex, and T-wave boundaries are explicitly delineated, and color-coded
segmentation decomposes the waveform into discrete visual measurement primitives. A general 2B
vision-
language backbone is then trained with low-rank supervised fine-tuning to associate these primitives with diagnostic
reasoning, without architectural modification. Across open, proprietary, and ECG-specialist zero-shot baselines, LuminaECG improves both waveform measurement and diagnostic recovery. It reaches a clinically meaningful reader tier on the CODE-test
benchmark, transfers across geographically diverse ECG
datasets without retraining, and generates reports whose structure contains an emergent prognostic signal. These findings suggest that effective ECG
agents require not only larger models, but supervision that preserves the
alignment between measurable waveform evidence and clinical knowledge.
Read the full abstract ▾
All authors Hongxiang Gao, He-yang Xu, Yuwen Li, Minghui Zhao, Zhipeng Cai, Xingyao Wang, Chenxi Yang, Jianqing Li, Chengyu Liu
Computer Vision
Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation
Xuanyu Liu, Zheng Fang, Hongyang He et al. (+2 more)
Semi-supervised adaptation of
vision foundation models (VFMs) commonly freezes the pretrained backbone and updates lightweight modules such as LoRA. However, pseudo-labels have mixed reliability, and a single LoRA adapter must absorb reliable, ambiguous, and noisy gradients in the same low-rank space. This can make VFM adaptation sensitive to pseudo-label noise. We propose TriNoL, a Triple-expert learning framework from Noisy Labels for semi-supervised VFM adaptation. TriNoL routes unlabeled samples into three confidence regions and assigns them to three LoRA experts: a Positive Expert for high-confidence pseudo-labels, an
Alignment Expert for medium-confidence ambiguous samples, and a Negative Expert for low-confidence noisy samples. The VFM backbone remains frozen, and only the LoRA experts and classifier head are updated. By separating different pseudo-label reliability regions into specialized adaptation paths, TriNoL improves robustness to noisy supervision while keeping the training cost low.
Read the full abstract ▾
All authors Xuanyu Liu, Zheng Fang, Hongyang He, Yundi Hong, Daizong Liu