Computer Vision
PICO: Projection-Informed Consistency Optimisation for 6DoF Surgical Tool Pose Estimation
Lucy Fothergill, Pietro Valdastri, Dominic Jones et al. (+1 more)
Purpose: Accurate 6 DoF pose estimation of
surgical tools is critical for automa- tion, robotic proprioception, and safe interaction with the tissue operated on. Kinematics-based approaches suffer from accumulated errors due to the cable- driven nature of robotic arms, while
vision-based methods often rely on external markers or trackers. Although more recent
vision-based advances have been pro- posed, these two-stage pose estimation methods often lack real-time robustness due to accumulated errors and computational overhead. Methods: We propose a novel end-to-end trainable model, PICO. Our model employs a multi-task learning architecture to predict
segmentation and depth maps, alongside regression of translation and rotation parameters. We define two proxy tasks that enforce
geometric consistency in both 2D and 3D spaces, improving accuracy and robustness. For this, we propose a projection loss, and a point-to-point loss. Results: We evaluate our method on the SurgRIPE
dataset, benchmarking its performance against state-of-the-art approaches using standard 6DoF pose esti- mation metrics. Our results demonstrate consistently strong performance across all four subsets, specifically in rotation, ranking second even under occlusion. It also demonstrates comparable translational performance, remaining competitive, especially in occluded cases. Conclusion: PICO demonstrates the effectiveness of multi-task learning and geometry-aware proxy tasks for robust and reliable
surgical tool pose estimation, especially in occluded scenarios, highlighting potential for future applications.
Read the full abstract ▾
All authors Lucy Fothergill, Pietro Valdastri, Dominic Jones, Duygu Sarikaya
Computer Vision
PhoenixSR: Generative Heterogeneous Distillation Unleashes Efficient Models for Real-World Super-Resolution
Xin Di, Mingyu Shi, Yuanfei Bao et al. (+7 more)
Real-world image super-resolution (SR) requires recovering perceptually realistic high-resolution images from complex low-resolution observations while preserving faithful content.
Diffusion-based SR benefits from strong generative priors but incurs substantial computational overhead, whereas feed-forward CNN and
Transformer SR models are efficient yet often struggle to recover realistic high-frequency details. This motivates a natural question: can
diffusion priors be transferred to existing
diffusion-free SR networks without introducing
diffusion components at inference time? To this end, we propose PhoenixSR, a generative heterogeneous
distillation framework that transfers
diffusion priors to independently designed feed-forward SR networks through score-based distribution matching. Rather than aligning heterogeneous features or imitating sampled
diffusion outputs, PhoenixSR uses the pretrained
diffusion model as distribution-level supervision, while paired SR supervision preserves reconstruction fidelity. To make distribution matching effective for fidelity-sensitive SR, we introduce Heterogeneous Distribution Adaptation, which adapts the target score to the SR domain, improves tracking of the evolving student distribution, and anchors training with paired supervision. We further employ Directional Reliability Weighting, a lightweight residual-consistency-based reweighting strategy that reduces unstable distributional guidance. All
diffusion-related components are removed after training, leaving the original student architecture and inference cost unchanged. Experiments on three SR
benchmarks and six feed-forward backbones, including SwinIR, HAT, Real-ESRGAN, and SeeMoRe, show consistent perceptual improvements with largely preserved reconstruction fidelity.
Read the full abstract ▾
All authors Xin Di, Mingyu Shi, Yuanfei Bao, Long Peng, Yue Zhao, Jiaming Guo, Renjing Pei, Xueyang Fu, Yang Cao, Zheng-Jun Zha
Computer Vision
Self-Supervised Perceptually Interpretable Monocular Depth Estimation
Zain Ul Abidin, George Dimas and Dimitris K. Iakovidis
Self-supervised
monocular depth estimation (MDE) enables depth prediction from
monocular images without requiring ground-truth supervision, making it attractive for large-scale and real-world applications. Despite steady improvements in accuracy, most existing methods remain difficult to interpret, as depth is inferred from RGB representations that obscure the impact of individual perceptual image components. This lack of transparency limits systematic analysis of failure cases and reduces confidence in safety-critical settings. This paper presents a self-supervised framework for perceptually interpretable
monocular depth estimation (PIMDE), designed to associate depth predictions with distinct perceptual components of the input image. Rather than operating directly on RGB inputs, the proposed method decomposes each image into a set of perceptual feature maps (PFMs), each encoding a specific visual cue. Distinct depth estimation branches process these PFMs independently to produce depth estimates (PIDEs), which are subsequently combined through an explicit fusion strategy. This formulation allows us to examine directly the contribution of each perceptual cue to the final depth prediction. Experiments conducted on the KITTI
benchmark dataset demonstrate that PIMDE achieves performance comparable to established self-supervised MDE methods while providing additional insight into how different perceptual cues influence depth estimation. These results indicate that perceptual decomposition can support interpretability without sacrificing depth estimation accuracy.
Read the full abstract ▾
Computer Vision
FARE: Forensic Acceptance Region Estimation for Catching Bait-and-Switch Image Generators
Kai Yao and Marc Juarez
Modern AI image generators are increasingly deployed as opaque APIs, where customers can query the deployed service, but cannot inspect model weights or architecture. This creates a practical challenge: a provider may pass governance certification with one generator and later silently switch to a cheaper and lower-quality one for deployment, compromising public trust or even safety in high-stakes domains. We study integrity
auditing at deployment time and propose FARE (Forensic Acceptance Region Estimation). A certified generator is enrolled by training FARE on images sampled from that generator. After deployment, FARE can determine whether a generated image is consistent with the enrolled generator—using only that image. FARE's features are based on image generator-specific artifacts that have been proposed for forensic applications. FARE amplifies these features during training by finding hard samples that tighten the acceptance region and increase sensitivity to subtle changes in the certified generator. Across generator swaps, including substitutions with similar model versions and model variants, FARE is effective at detecting swaps, consistently outperforming existing baselines at strict operating points, and remains effective under the exact-model and decision-only attacks evaluated in this work.
Read the full abstract ▾
Computer Vision
STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidence
Siru Zhong, Shenghan Tan, Rihong Yan et al. (+6 more)
Reliable online
video question answering requires tracking state transitions while selectively abstaining when visual evidence is insufficient. Existing
benchmarks focus on static recognition or long-range
retrieval, rarely evaluating these coupled capabilities under evolving and incomplete evidence. We present STORM-Bench, comprising 5,736 questions across 630 compact, change-dense episodes spanning five
egocentric domains (STORM-Real) and two controlled simulation subsets (STORM-Sim) at 1 FPS. Questions are stratified by a proxy for accumulated change intensity (Low, Medium, High) and query-time answerability (Known, Uncertain). To measure reliability, we introduce STORM-BR, a harmonic metric over joint answer-status correctness that exposes abstention failures masked by aggregate accuracy, alongside STORM-BR-ATTR for uncertainty attribution. Across 14
video LLMs, online accuracy peaks at 60.3% (mean 51.7%), whereas STORM-BR ranges from 5.7% to 35.6% (mean 18.8%), driven by pervasive overconfidence on uncertain queries. STORM-Bench shows that task accuracy masks these gaps in epistemic reliability and state tracking.
Benchmark and code are available at https://github.com/siruzhong/STORM-Bench.
Read the full abstract ▾
All authors Siru Zhong, Shenghan Tan, Rihong Yan, Xiaohui Lv, Yuzheng Zhuang, Shuai Tao, Wulong Liu, Haohuan Fu, Yuxuan Liang
Computer Vision · Cross-listed from cs.CR
FeatMark: Feature-level Watermark Protection against Mimicry Attacks with Diffusion Models
Haoyang Li, Ruoxi Sun, Qingqing Ye et al. (+4 more)
Text-to-image
diffusion models enable data-efficient "mimicry" attacks, wherein adversaries fine-tune the model on a handful of public photos to synthesize convincing forgeries of a target individual. A common countermeasure is to embed imperceptible, low-energy watermarks, yet recent studies show these signatures are brittle: modest post-processing or lightweight
adversarial perturbations readily suppress detection, exposing a fundamental tension between imperceptibility and robustness. We introduce FeatMark, a watermarking framework that shifts from pixel-level, energy-starved perturbations to inconspicuous semantic features: small, scene-consistent micro-features that remain natural to humans while providing a stronger, machine-verifiable provenance signal. FeatMark builds domain-specific feature banks that encode each watermark as a compact concept program, pairing open-vocabulary semantic cues with reliable edit regions and instruction templates. It then automatically selects features that are both feasible and executable and injects them through modular, mask-guided concept editing, yielding highly localized, scene-consistent micro-edits that are difficult to perceive. We conduct extensive experiments across VGGFace2, CelebA-HQ, and WikiArt, evaluating against 10 strong watermark removal/purification attacks (including regeneration-style purification) and several bespoke adaptive attacks tailored to FeatMark, to assess perceptual fidelity, watermark detection accuracy, and robustness. We further demonstrate FeatMark's extensibility to
video mimicry attacks. The results show FeatMark remains virtually impervious, withstanding all evaluated attacks with negligible bit-accuracy and fidelity degradation.
Read the full abstract ▾
All authors Haoyang Li, Ruoxi Sun, Qingqing Ye, Benjamin Zi Hao Zhao, Yaxin Xiao, Jason Xue, Haibo Hu
Computer Vision
CCRV-Bench: Constraint-Based Evaluation of Causal Reasoning in Vision-Language Models
Linyuan Gao, Yuan Wu and Yi Chang
Vision-
language models (VLMs) have demonstrated excellent performance in visual tasks, but their visual causal
reasoning capabilities still lack reliable evaluation. Existing evaluations struggle to distinguish whether a model is performing causal
reasoning based on visual evidence or relying on statistical correlations for shortcut learning, thereby potentially overestimating their actual capabilities. This paper proposes CCRV-Bench, a constraint-driven visual causal
reasoning benchmark for single-image physical scenarios. We construct an orthogonal framework that evaluates four causal task dimensions: causal relation discovery, state prediction, causal diagnosis, and intervention-outcome prediction. We further introduce entity symbolization, spatial grounding, the factual
adversarial constraint, and minimalist output constraints to reduce shortcut cues while preserving the physical commonsense required by the task. Experiments across 14
multimodal models show that constraint sensitivity is task- and model-dependent: intervention-outcome prediction has the largest average effective degradation among the four causal tasks, spatial grounding is the most damaging constraint on average, and the factual
adversarial constraint improves DCR for all evaluated models. These results show that unconstrained performance does not determine constrained robustness and that a single aggregate score can obscure distinct failures in causal identification, spatial grounding, and constraint-compliant expression. CCRV-Bench provides a standardized framework for diagnosing image-grounded causal
reasoning under controlled constraints. The code is available at https://github.com/0815linyuan/CCRVBench
Read the full abstract ▾
Computer Vision
Where and When to Force: Routed Forcing for Streaming Avatars
Zihan Su, Siwen Lu, Junhao Zhuang et al. (+6 more)
Audio-driven streaming avatar generation requires real-time synthesis of
speech-synchronized
videos with dynamic and diverse motion. Self Forcing uses Distribution Matching
Distillation (DMD) to distill bidirectional
video diffusion models into causal, few-step generators for real-time streaming. However, DMD minimizes a reverse KL divergence, which is inherently mode-seeking: it causes the student to discard high-dynamic modes and collapse onto static outputs, compressing both dynamics and diversity of generated
videos. We find that this collapse is region-heterogeneous: person regions involving pose and gesture variations suffer the largest diversity loss, the audio-driven mouth region shows a small loss, and the background remains nearly stable. Based on this observation, we propose Routed Forcing, which routes the
distillation objective by semantic region and noise stage to improve dynamics and diversity while preserving visual quality. Specifically, (1) Where to Force: Semantic-Region Routing applies Data-Forcing
Distillation (DFD), which supervises the student with real
videos, to the person region where diversity collapse is most severe, while retaining DMD for the mouth and background to preserve lip synchronization and scene stability. (2) When to Force: Noise-Stage Routing activates DFD at high noise stages, where real
video serves as effective supervision to inject diverse and dynamic motion patterns. At low noise stages, DMD is used to refine details, avoiding blur and artifacts from spatial differences between real
video and student-generated
video. Experiments show that Routed Forcing improves dynamics by up to 45% and diversity by 7-25% over Self Forcing, while preserving
video quality and lip synchronization.
Read the full abstract ▾
All authors Zihan Su, Siwen Lu, Junhao Zhuang, Zeyue Xue, Haoyang Huang, Guanghao Li, Xiaofeng Tan, Chun Yuan, Nan Duan
Computer Vision
IDM-Net: A Lightweight Illumination-Decoupled Modulation Network for Low-Light Image Enhancement
Cheng-Yen Hsiao and Jing-Ming Guo
Low-light image enhancement (LLIE) remains challenging for lightweight models because illumination restoration and color fidelity are difficult to optimize simultaneously in the RGB color space. Although recent color-decoupled methods separate luminance and chrominance representations, they primarily optimize luminance as an enhancement target, leaving its potential as an explicit guidance prior largely unexplored during feature reconstruction. To address this limitation, we propose IDM-Net, a lightweight Illumination-Decoupled Modulation Network for low-light image enhancement. IDM-Net adopts a dual-encoder architecture consisting of a structure encoder that extracts multi-scale appearance features from the RGB image and a lightweight illumination encoder that learns illumination priors from the decoupled luminance (Y) channel. To effectively exploit these priors, we introduce an Illumination-Guided Modulation (IGM) module that injects multi-scale illumination cues into the decoder through spatially adaptive affine modulation, enabling accurate brightness restoration while preserving natural color consistency. Furthermore, we design a lightweight Feature Refinement Block (FRB) to progressively suppress degradation artifacts and recover fine-grained image details during reconstruction. Extensive experiments on multiple standard low-light image enhancement
benchmarks demonstrate that IDM-Net achieves competitive performance among lightweight LLIE methods while maintaining an excellent balance between restoration quality and computational efficiency.
Read the full abstract ▾
Computer Vision
Hyungjin Chung, Byeongjun Park, Joonseok Lee et al. (+3 more)
Multi-view
video understanding requires integrating spatial and temporal evidence across multiple, often non-overlapping camera streams: tracking entities as they transition between viewpoints, aligning events across time, and
reasoning about latent 4D continuity rather than any single visible frame. We introduce MVVBench, a
benchmark for multi-view
video reasoning built from real world multi camera
datasets. Questions are curated to be
monocular-ambiguous along both the view and the temporal axis: each question is unanswerable from any single view in the designated input set, and the majority are further unanswerable from any single moment. Each question becomes uniquely solvable only by jointly
reasoning across views and across time. MVVBench spans diverse dynamic scenes and probes six capabilities: implicit/explicit attribute identification, implicit/explicit relative distance, relative camera pose, and compositional counting, with human-authored QA and rigorous verification. Beyond benchmarking, we provide an extensive analysis of when and why current
vision language models succeed or fail, characterizing errors due to temporal mis-localization, cross-view identity breaks, and brittle multi-hop
reasoning. We then study inference-time elicitation strategies that unlock latent multi-view competence—task-specific chain-of-thought scaffolds and structured cross-view evidence aggregation—yielding substantial gains without retraining. Finally, we present preliminary evidence that reinforcement learning with verifiable rewards can elicit some latent multi-view competence in the base model, pointing to training-time approaches as a promising direction for future work. Together, MVVBench offers a rigorous evaluation of 4D multi-view
reasoning and a foundation for future progress toward reliable
embodied perception.
Read the full abstract ▾
All authors Hyungjin Chung, Byeongjun Park, Joonseok Lee, Hojun Kim, Jaeho Choi, Byung-Hoon Kim
Computer Vision
DAPEVO: Deep Adaptive Patch Frame-Event Visual Odometry
Luca Gandolfi, Simone Nascivera, Roberto Pellerito et al. (+3 more)
Visual odometry is essential for autonomous
navigation in GPS-denied environments, yet RGB-based methods remain vulnerable to motion blur, challenging illumination, and dropped frames. Event cameras complement conventional cameras with high temporal resolution and dynamic range, but their asynchronous measurements complicate reliable correspondence estimation. We present DAPEVO, a learned visual odometry system that estimates image and event correspondences independently at shared patch locations and fuses their correlation evidence before motion refinement. Each tracked patch maintains image and event descriptors, and a learned scalar gate combines modality-specific correlation
embeddings for each patch–frame edge before a shared recurrent refinement and bundle-adjustment update. DAPEVO also supports event-only observations, enabling continued tracking when RGB frames are sparse or unavailable, while modality-aware keyframe culling preserves scarce frame constraints. On UZH-FPV, when retaining only one in six RGB frames, DAPEVO's mean absolute trajectory error (ATE) increases by only 36%, from 1.00 to 1.36m, whereas the ATE of DPVO and RAMP-VO rises by factors of $3.7\times$ and $3.1\times$, respectively. On TartanEvent, DAPEVO similarly remains below 1m ATE at 3Hz RGB input, while DPVO and RAMP-VO exceed 9m. Under degraded RGB input on TartanEvent, DAPEVO achieves an ATE of 0.60m, compared with more than 4m for both DPVO and RAMP-VO, while also outperforming event-only DEVO at 0.87m.
Read the full abstract ▾
All authors Luca Gandolfi, Simone Nascivera, Roberto Pellerito, Rong Zou, Chiara Plizzari, Davide Scaramuzza
Computer Vision
OneWorld: Learning Consistent Physics Across Actions in World Models
Ke He, Yichen Ding and Bin Yang
Action-conditioned
video world models aim to predict scene
evolution under different actions, a capability that is essential for reliable planning, decision-making, and interaction in dynamic environments. However, futures generated independently from the same initial scene may each appear plausible while implying incompatible physical properties, such as friction or mass. This inconsistency can lead to contradictory predictions across interventions, making it difficult for the model to maintain a coherent understanding of the underlying world and limiting its reliability for planning and decision-making. To address these issues, we propose OneWorld, a shared-mechanism counterfactual generation framework that jointly models multiple action-conditioned futures under a common latent physical mechanism. A physical mechanism interpreter first infers a distribution over latent mechanisms from each action-outcome branch. These distributions are then aggregated into shared-world evidence, which captures whether the branches admit a common physical explanation while accounting for uncertainty in less informative branches. This evidence constrains flow training and guides sampling, encouraging consistency in the underlying physical mechanism while preserving the distinct outcomes induced by different actions. We further introduce a multi-intervention evaluation protocol in controlled environments, following the interaction settings of ACWM-Phys, to assess whether generated futures can be jointly explained by the same physical parameters, alongside standard measures of single-rollout prediction quality. Experiments in these environments show that OneWorld improves cross-intervention physical consistency while maintaining competitive single-rollout prediction quality.
Read the full abstract ▾
Computer Vision
Spackle: Completing Large View Single Image NVS with Adaptive Gaussians
Xuanzhi Liu, Yuhe Zhou, Xinyi Wu et al. (+4 more)
Single-image novel view synthesis (NVS) enables photorealistic rendering of un- observed viewpoints from a single input. Practical NVS systems require two key capabilities: robust reconstruction of occluded regions and high inference effi- ciency. While hybrid decoupled frameworks combining feedforward 3D Gaussian Splatting (3DGS) and
diffusion models show promise for large-view-deviation NVS, they suffer from capacity competition: a fixed number of Gaussians forces resource shifts from visible to newly disoccluded areas, degrading original scene fidelity when the target view deviates significantly from the input. To address this, we propose Spackle, a lightweight residual learning framework that mit- igates capacity competition without sacrificing efficiency. Spackle operates in three stages: predicting base 3DGS attributes from given views, automatically identifying poorly reconstructed regions, and learning a residual 3DGS optimized exclusively for these areas. At inference, we combine the baseline and aug- mented Gaussians for NVS. We conduct comprehensive experiments and show that Spackle achieves state-of-the-art performance on large-view-deviation cases.
Read the full abstract ▾
All authors Xuanzhi Liu, Yuhe Zhou, Xinyi Wu, Zhenyao Wu, Jinghao Chen, Ruize Han, Song Wang
Computer Vision
ManiVid: Unified and Explainable Forensic Analysis of Manipulated Videos
Hengrui Kang, Zhonghao Yan, Yuxuan Yang et al. (+7 more)
Rapid advances in AI-generated
video (AIGV) have increased the risks posed by deceptive
video manipulation. Unlike fully synthetic
videos, manipulated
videos retain most source content and alter only localized regions, making forensic analysis particularly challenging. Existing
video forgery research faces two limitations in both data and methodology: (1) High-quality
datasets and
benchmarks tailored for manipulated
videos remain scarce. (2)
Multimodal large
language models (MLLMs) extend forgery analysis beyond binary classification but struggle to use low-level forensic cues and provide precise pixel-level grounding. Specifically, we introduce ManiVid, a unified forensic analysis task covering forgery detection, artifact grounding, and anomaly explanation for manipulated
videos. We construct ManiVid-38K, the first
dataset to combine paired, open-vocabulary localized manipulations of general
videos with authenticity labels, forgery masks, and anomaly explanations. It comprises about 19K manually verified real-fake
video pairs, mostly at 1080P resolution, generated under 2 paradigms with 15 powerful generation models. We sample 1K pairs for ManiVidBench, balanced across six manipulation types and generation models for fair evaluation. We further propose ManiVidLens, a unified framework for explainable
video forgery analysis. Its Forensic Evidence Router supplies shared low-level forensic evidence for
multimodal reasoning and
video segmentation. Its
Prompt Distill Module converts grounding states into semantic and
geometric prompts and distills spatial priors for mask decoding and full-
video propagation. ManiVidLens achieves relative gains over the strongest comparison methods in artifact grounding (+21.1% mIoU; +21.3% J&F) and anomaly explanation (+131.3% ROUGE-L; +9.9% CSS). Its forgery detection remains comparable to dedicated classifiers (0.914 Acc; 0.913 F1).
Read the full abstract ▾
All authors Hengrui Kang, Zhonghao Yan, Yuxuan Yang, Ruoyan Jing, Yuncheng Guo, Hao Chen, Kongming Liang, Zhanyu Ma, Conghui He, Weijia Li
Computer Vision
UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasound
Quanhao Zhu, Bo Xu, Rui Lin et al. (+7 more)
Ultrasound is one of the most widely used medical imaging modalities, and recent large
vision-
language models(VLMs) have shown increasing capabilities in ultrasound image understanding. However, these models fail to provide pixel-level visual evidence aligned with their semantic predictions, and their fine-grained grounding capability in ultrasound remains largely unclear. We introduce UltraG-Bench, a large-scale multi-task
benchmark for evaluating pixel-level evidence grounding in ultrasound. UltraG-Bench is built by annotating 40 public ultrasound
segmentation datasets spanning 13 anatomical categories, and comprises three progressive tasks: instruction-guided
segmentation, evidence-grounded VQA, and evidence-grounded report generation, with 331125, 666779, and 138832 annotations, respectively. Comprehensive evaluation of 14 state-of-the-art models reveals a substantial gap between semantic understanding and fine-grained pixel-level localization. We further propose UltraG-
Agent, which combines the semantic
reasoning capabilities of a VLM with the ultrasound-specific
segmentation capability of UltraSAM3. Experiments show that UltraG-
Agent substantially improves both semantic prediction and pixel-level visual grounding. Our
dataset and code are available at https://github.com/zhuqh19/UltraG-Bench.
Read the full abstract ▾
All authors Quanhao Zhu, Bo Xu, Rui Lin, Chenyuan Wang, Yu Shao, Boling Zhu, Jiuyan Sun, Liang Zhao, Hongfei Lin, Feng Xia
Computer Vision · Cross-listed from cond-mat.mtrl-sci
Universal Drift Correction for Multidimensional Scanning Microscopy
Sangjoon Lee, William Millsaps, Dasol Yoon et al. (+7 more)
In scanning microscopy, drift causes the specimen to be sampled at positions displaced from the nominal probe positions. This displacement alters the spatial assignment of the recorded signals and biases quantitative measurements across two-dimensional imaging, channel-resolved spectroscopic mapping, and scan-position-resolved diffraction analysis. Here, we extend orthogonal-scan drift correction from 2D images to spectrum images and diffraction
datasets. We demonstrate how to recover probe positions using either differently oriented multidimensional scans or structural reference images. The recovered positions are used either to resample the multidimensional data onto a regular grid or to assign each recorded signal to its corrected coordinate. Our method combines affine and non-rigid correction, requires no prior structural model, and is implemented as open-source, GPU-accelerated software that reduces processing times by two to three orders of magnitude, enabling routine and automated drift correction for quantitative multidimensional microscopy.
Read the full abstract ▾
All authors Sangjoon Lee, William Millsaps, Dasol Yoon, Caitlyn Obrero, Guoliang Hu, Corrie Barnes, Cedric Lim, Andrew Barnum, Arthur R. C. McCray, Colin Ophus
Computer Vision
Reliability-Regulated Trajectory Optimization for Progressive COLMAP-Free 3D Gaussian Splatting
Zijian Wu, Jinliang Wang, Zidian Lin et al. (+5 more)
COLMAP-free 3D Gaussian Splatting (3DGS) bypasses computationally expensive structure-from-motion (SfM) pipelines, yet progressive camera pose tracking remains fundamentally vulnerable to error compounding—early pairwise tracking inaccuracies both corrupt subsequent frame initializations and remain permanently frozen in the scene representation. Rather than relying on heavyweight external neural priors or treating progressive tracking through isolated heuristic fixes, we propose a unified reliability-regulated trajectory optimization framework for progressive COLMAP-free 3DGS. At its core, our framework establishes an intrinsic, self-supervised bidirectional cycle-consistency mechanism that systematically regulates progressive camera trajectory estimation across two complementary temporal horizons: (1) Forward Motion Propagation, where the online reliability signal adaptively gates first-order kinematic warm-starts of rigid motion into upcoming pairwise registrations, supplying informed directional search priors while safely intercepting untrusted transitions; and (2) Retrospective Trajectory Correction, where the same reliability signal dynamically weights relative-pose consistency constraints within a sliding window of neighboring camera poses. By governing both prospective state initialization and retrospective trajectory consolidation through a unified reliability regulator, our self-contained framework resolves progressive drift without external priors or offline preprocessing. Extensive evaluations on Tanks and Temples and CO3D-V2
benchmarks show that our method substantially improves camera trajectory accuracy and novel-view rendering quality, outperforming existing unposed baselines. Code is available at https://github.com/Zijian1026/RRTO-CF3DGS.
Read the full abstract ▾
All authors Zijian Wu, Jinliang Wang, Zidian Lin, Ying Song, Ziqian Lu, Hanjie Ma, Zhen Ye, Mingfeng Jiang
Computer Vision
MDSkin-Net: Multi-Task Skin Lesion Analysis Driven by Pattern Analysis Priors and Spatial Alignment Regularization
Yijian Li, Saad Bedros, Paul Bigliardi et al. (+3 more)
Reliable skin lesion
segmentation and classification are central to dermoscopic computer-aided diagnosis. Existing multi-task frameworks couple the two tasks architecturally without clinical knowledge, while knowledge-injecting approaches rely on the macroscopic ABCD rule, which was not designed for dermoscopy. Dermoscopic diagnosis is grounded in Pattern Analysis, a microscopic framework structured around dermoscopic features. We propose MDSkin-Net, which incorporates cue-level Pattern Analysis priors into a hybrid CNN-
Transformer architecture. At its core is a Pattern Analysis-Guided
Attention Module (PAGAM) comprising three priors motivated by distinct dermoscopic cues: an improved Efficient Channel
Attention (iECA), a Multi-Scale Spatial
Attention (MSSA), and a Biased Asymmetry
Attention (BAA). We further introduce a multi-scale spatial
alignment regularization (MSAR) that uses the
segmentation ground-truth mask as hierarchical soft supervision, confining the classification head to lesion-localized evidence and coupling both task pathways through a shared spatial prior. Trained exclusively on the ISIC 2017 training split without external dermoscopy data, the MDSkin-Net ensemble transfers robustly under zero-shot evaluation, reaching a Dice Similarity Coefficient (DSC) of 92.38% and a melanoma AUC of 97.84%on PH2, and a DSC of 88.92% on the ISIC 2018 Task 1 test set. On the in-domain ISIC 2017
benchmark, the ensemble attains a mean Area Under the Curve (AUC) of 91.60% across the two classification tasks (melanoma and seborrheic keratosis vs. rest), and a DSC of 84.72% for
segmentation. Classification remains competitive with baselines; in-domain
segmentation trails single-task specialists, yet the proposed priors and
alignment regularization yield representations that generalize consistently across cohorts of different scales.
Read the full abstract ▾
All authors Yijian Li, Saad Bedros, Paul Bigliardi, Mei Bigliardi Qi, Vassilios Morellas, Nikolaos Papanikolopoulos
Computer Vision · Cross-listed from cs.LG
Aligning One-Step Generative Models with Reward-Weighted Transport Distillation
Austin Wang, Ziheng Cheng and Lexing Ying
One-step generators enable high-quality visual generation with a single network evaluation, but their post-training is difficult: general implicit generators provide neither tractable likelihoods nor denoising trajectories, and many rewards are non-differentiable. We introduce Reward-Weighted Transport
Distillation (RWTD), a post-training method that requires only generated samples and scalar reward evaluations. Rather than aligning solely to the conventional reward-tilted reference distribution, RWTD constructs an adaptive target that mixes separately tilted current and reference distributions. The current component incorporates improvements discovered during training, while the reference component anchors the target to the pretrained generator. RWTD realizes this target through feature-space optimal transport and fixed-point regression. Theoretical analysis shows that the fixed-point distributions of RWTD interpolate between off-policy reward tilting of the reference and on-policy tilting of the current model, providing a principled approach to balancing reward adaptation with retention of prior knowledge. Empirically, RWTD substantially improves the GenEval score of the one-step SANA Sprint 1.6B backbone from 0.73 to 0.80, while separate preference
alignment experiments demonstrate strong cross-reward generalization that yields balanced improvements and preservation of compositional capabilities.
Read the full abstract ▾
Computer Vision
Motion Style Slider: Endpoint-Supervised Continuous Style Control for Human Motion Diffusion
Chen-Chieh Liao, Yichen Peng, Yiyi Cai et al. (+5 more)
Existing human motion
diffusion methods provide strong motion generation quality, and recent style transfer models can inject target style cues, but fine-grained continuous control of style intensity remains underexplored. In production, style intensity is subjective across artists and directors, so the practical requirement is not a universal absolute unit, but a reliable monotonic control axis. We propose Motion Style Slider, a motion-to-motion style transfer framework for endpoint-supervised continuous control. Given a content motion and a style motion, we construct a style direction in a learned motion-style
embedding space and condition
diffusion generation with a scalar intensity. The training objective combines
diffusion denoising with latent intensity regularization to encourage smooth and monotonic style scaling without requiring intermediate-intensity ground-truth motions. Our framework is compatible with pretrained motion
diffusion backbones and supports heterogeneous style
datasets, including the multi-actor style motion
dataset. To test out-of-range usability, we additionally introduce a small real-capture over-reaction extension and evaluate large-intensity behavior against these unseen targets. Experiments measure controllability, interpolation/extrapolation behavior, content preservation, and motion realism, with ablations on direction construction and loss design.
Read the full abstract ▾
All authors Chen-Chieh Liao, Yichen Peng, Yiyi Cai, Yûi Ono, Hiroki Hanaoka, Erwin Wu, Hideki Koike, Shuichi Kurabayashi
Computer Vision
Tianhang Guo, Yulin He, Wei Chen et al. (+3 more)
Reasoning segmentation aims to interpret implicit textual queries and enable fine-grained visual perception, which is critical for applications such as human-computer interaction and
embodied agents. Existing methods typically generate explicit Chain-of-Thought (CoT) by
multimodal large
language models (MLLMs) before localizing the target. Although intuitive, such explicit verbal
reasoning introduces substantial
attention interference: redundant textual
tokens disrupt
attention during perception-
token generation and also increase the effective distance between visual
tokens. To address this issue, we propose LIRSeg, which fully replaces explicit CoT with a compact set of learnable latent
tokens for
reasoning segmentation. LIRSeg is trained in two stages: spatial
alignment grounds the latent
tokens in object-relevant visual evidence, and GRPO further optimizes them with
segmentation rewards. To make these compact latent
tokens more informative, we introduce three complementary mechanisms from an information perspective: extreme-advantage sampling for selecting informative training signals, decoupled exploration-stability updates for learning complementary representations, and latent diversity amplification for preventing representational collapse. Extensive experiments on
benchmarks demonstrate that LIRSeg consistently improves both
segmentation accuracy and
reasoning efficiency. Compared with the VisionReasoner baseline, LIRSeg achieves absolute gIoU improvements of 4.9% on ReasonSeg, 7.1% on MUSE, and 4.7% on MMR, while achieving a approximately 16x reduction in
reasoning tokens. Code is available in supplementary materials.
Read the full abstract ▾
All authors Tianhang Guo, Yulin He, Wei Chen, Wenjuan Zhou, Yuhang Li, Xinbiao Gan
Computer Vision
Query-Conditioned Prototype Adaptation for Cross-Domain Few-Shot Learning: Single-Query Inference, Controlled Comparisons, and Failure Modes
Rushab Rasik Karania and Tomas Maul
Cross-domain few-shot learning requires adapting a classifier to a new visual domain from very few labelled examples without target-time parameter updates. We isolate one question: under a fixed global representation, what does joint query-support adaptation contribute to prototype construction? The Within-Instance Prototypical
Transformer (WIPT) implements single-query test-time prototype adaptation by jointly transforming one unlabelled query and the labelled support
embeddings, then forming query-specific class means. Using a shared frozen ViT-S/16 encoder, miniImageNet source training, and CUB, EuroSAT and ISIC targets, we replicate the key comparisons across five independent training seeds. In 1-shot evaluation, WIPT improves frozen ProtoNet in every run on CUB (+0.21 percentage points) and EuroSAT (+2.07), but decreases ISIC (-0.22). In 5-shot evaluation, ProtoNet remains strongest overall, while WIPT consistently improves a capacity-matched support-only
Transformer on ISIC (+0.99). Joint processing of up to five queries yields no reliable accuracy gain; in a head-only 5-shot
benchmark, g = 5 reduces analytical
attention-
token pairs by 73% and peak allocated
memory by 29% relative to g = 1, although latency is non-monotonic. Across all target/shot conditions, WIPT changes uncertain ProtoNet decisions far more than confident ones, and rescue/break decomposition accounts for the observed gains and losses. Source-shift and scorer controls further show that the benefit is not universal. Overall, WIPT provides a streaming-compatible form of test-time prototype adaptation that can improve difficult low-shot cross-domain decisions without target-time optimization.
Read the full abstract ▾
Computer Vision
Zhao Wang, Jiangtao Hu, Jack Yu et al. (+1 more)
Most existing human motion generation (HMG) methods use cross-
attention modules to inject text semantics, but ignore the importance of bidirectional modeling between motion and text
tokens, which limits text comprehension. A straightforward idea is introducing
multimodal diffusion transformers (MMDiT), which have shown effective joint text–visual modeling in
vision generation, into HMG. However, we find that articulated motion is temporally coherent but weakly correlated across joints, in which directly applying an MMDiT with flow matching produces poorly coordinated and jerky motion. In this work, we propose Timo, a novel kinematics-aware MMDiT framework tailored for HMG. Timo combines fully shared
multimodal attention for bidirectional text–motion modeling with flow matching,
geometric and rotational-kinematics supervision that compares actual rotations and their changes over time, and a two-stage curriculum progressing from broad motion learning to detailed caption
alignment. Further, we construct a
benchmark of $40{,}025$ held-out clips from six public
datasets spanning diverse actions, assessing six complementary dimensions under a common evaluator and scoring protocol. Our model substantially outperforms state-of-the-art methods in both quantitative and qualitative evaluations. Remarkably, Timo surpasses Kimodo on five of six dimensions, achieving a $40.8$% relative improvement in the average
benchmark score. Project page: https://kyfafyd.wang/projects/timo. Demo page: https://timo.kyfafyd.wang.
Read the full abstract ▾
All authors Zhao Wang, Jiangtao Hu, Jack Yu, Tao Yu
Computer Vision
LLPR: Location-aware learning and physics-based reconstruction for raindrop removal from a single image
Zewei He, Xingyu Liu, Xing Luo et al. (+5 more)
Raindrops can cause occlusion and distortion in the background scenes due to their adherence to windows or camera lenses. Existing raindrop removal methods concentrate on designing sophisticated CNN or
Transformer architectures to recover distorted and missing texture. In this paper, we try to integrate location information and physical model into off-the-shelf CNN or
Transformer architectures to help improve their performance. Specifically, we notice that existing methods deploy a preprocessing sub-network to generate a binary or soft mask to indicate the raindrop location, which will increase the network parameters and computational complexity. In contrast, a location-aware learning branch is embedded to teach the encoder in the training phase with the capability of perceiving the position of the raindrops. Note that this location-aware learning branch can be removed during the inference process (achieving performance improvements at no cost). Furthermore, instead of directly reconstructing the raindrop-free image (i.e., background scene), we devise a physics-based reconstruction scheme to first learn the transparency
matrix and the raindrop layer. The latent background layer is then reversely derived based on the physical model. By combining the above-mentioned components, we propose our location-aware learning and physics-based reconstruction (LLPR) framework for this challenging ill-posed problem. We also collect a real-world raindrop-degraded image
dataset, which is challenging for single-image raindrop removal (SIRR) methods. Extensive experimental results demonstrate the effectiveness and generality of our LLPR framework, achieving superior performance against state-of-the-art SIRR methods. The code will be made available upon acceptance.
Read the full abstract ▾
All authors Zewei He, Xingyu Liu, Xing Luo, Guizhong Fu, Zixuan Chen, Yu Chen, Jinlei Li, Zhe-Ming Lu
Computer Vision
Training-Free Bottleneck Width Planning for Convolutional Autoencoders
Guannan Guo
Multiscale Spectral Rate-Distortion (MS-SRD) estimates the bottleneck channels required at user-supplied spatial cuts from training images and a normalized mean-squared error (NMSE) bound, without fitting a neural network. Its covariance-tail rule is exact for shared linear block-convolutional autoencoders under squared error. A nested-scale dominance result motivates reporting the activation-parameter Pareto frontier alongside the minimum-latent candidate. At NMSE <= 0.01 on thirteen grayscale
datasets, its latent-size prediction has 0.84% mean absolute percentage error against nonlinear patch-autoencoder boundaries; ten predictions are exact and the remaining three differ by one channel. In a four-
dataset deployable comparison, MS-SRD matches all retrospective external widths and all four selected models pass, without training a selector; a 46-fit validation grid and four Least-Volume fits each pass on two
datasets. In a skip-closed U-shaped autoencoder at the same bound, five predictions are exact, nine are within one channel, and every failing prediction is one channel short. Experiments at looser bounds show progressively larger nonlinear savings.
Read the full abstract ▾
Computer Vision
From Mono to Stereo: Accelerating Binocular Gaussian Splatting via Reprojection and Selective Patching
Hongfei Zhu and Ling Zhou
Binocular rendering requires two nearby views of the same scene and therefore repeats substantial visibility and shading work. We present a 2D Gaussian Splatting (2DGS) pipeline that fully renders a dominant-eye RGB image and an alpha-weighted depth proxy, reprojects that image to the affiliated eye, and repairs uncovered pixels. Small interior gaps are interpolated, whereas larger disoccluded regions are identified as regions of interest (ROIs) and selectively re-rendered. The depth proxy reuses the alpha-blending weights computed during dominant-eye rasterization, avoiding a separate depth-rendering pass. An adaptive ROI generator localizes the required updates using reprojected image boundaries and optional connected center-hole detection. On DTU, Tanks and Temples, and MipNeRF-360, the method reduces the measured time of a sequential two-pass binocular reference by 15.5% to 28.8% and peak GPU
memory by 6% to 11%. The corresponding affiliated-eye quality degradation is at most 1.3 dB PSNR, 0.02 SSIM, and 0.02 LPIPS, representing a measurable trade-off that requires application-specific perceptual validation. These results establish a practical efficiency-quality trade-off for controlled static-scene stereo rendering and motivate future evaluation under continuous motion and on physical VR hardware.
Read the full abstract ▾
Computer Vision
Amplify What You Gaze At: Target Saliency Boosting in Text-to-Image Generation
Shengqi Dang, Zhengxi Yu, Feilin Han et al. (+2 more)
Text-to-image generation has advanced in controlling what, where, and how objects appear, yet how visual
attention is distributed among objects remains largely unexplored. In this paper, we introduce Target Saliency Boosting, a new task aimed at boosting the visual saliency of a specific object during text-to-image generation without requiring any visual priors. Our key insight is that visual saliency is inherently relative: boosting the saliency of a target object also depends on the global saliency distribution across all objects in the scene. Based on this insight, we propose GazeME, a lightweight framework that uses saliency-marked
prompts, inserting learnable marker
tokens around object descriptions to indicate which objects to visually emphasize or suppress. To learn these markers, we construct a saliency-semantics
dataset that associates objects in image–
prompt pairs with object-level saliency scores, and propose Saliency Prior Marker Activation (SPMA), a saliency-aware
stochastic marker activation strategy that exploits relative saliency relationships for robust training. During inference, GazeME automatically inserts appropriate markers into the
prompt, thereby directly enhancing the visual saliency of the target object. Extensive experiments demonstrate that GazeME effectively boosts target saliency while preserving both semantic
alignment and image quality.
Read the full abstract ▾
All authors Shengqi Dang, Zhengxi Yu, Feilin Han, Xingyu Lan, Nan Cao
Computer Vision
Learning Polarization Image Restoration with General Restoration Priors
Chenggong Li, Jinhao Liu, Caiyun Wu et al. (+3 more)
Polarization imaging captures distinctive surface and
geometric cues that benefit a wide range of
vision tasks. However, real-world polarization acquisition is often affected by multiple coupled degradations, making image restoration essential for practical polarization
vision. Existing methods are largely tailored to specific degradations and remain constrained by the limited scale and quality of polarization data. To address these limitations, we develop an all-in-one polarization restoration framework for diverse and composite degradations. We first study the impact of different polarization representations on restoration performance and identify the normalized Stokes representation as an effective choice for separating intensity and polarization information. Accordingly, we devise a dual-branch architecture that separates intensity and polarization modeling. To overcome the limitations of polarization-specific training, the intensity branch leverages pretrained general restoration priors and a mixture-of-experts extension for composite degradations, while its restoration knowledge is adaptively distilled into the symmetric polarization branch via a cross-domain feature transform. In addition, we establish a composite-degradation polarization
benchmark to support all-in-one restoration research. Extensive experiments on public
datasets and our proposed
benchmark demonstrate the effectiveness of the proposed method.
Read the full abstract ▾
All authors Chenggong Li, Jinhao Liu, Caiyun Wu, Yidong Luo, Junchao Zhang, Degui Yang
Computer Vision
EviDETR: Preserving Query-Relevant Temporal Evidence for Moment Retrieval and Highlight Detection
Haoran Sun, Yufan Li, Qichen Zhang et al. (+2 more)
Joint
video moment
retrieval and highlight detection requires identifying query-relevant temporal segments while estimating clip-level saliency, yet DETR-style pipelines do not explicitly preserve query-relevant evidence throughout encoding, decoding, and cross-task prediction. We propose EviDETR, an evidence-preserving framework with three components. Semantic-aware Feature Reweighting (SFR) enhances query-relevant clip representations through saliency estimation and cross-modal interaction. A Temporal Top-2 Mixture-of-Experts (TTop2MoE) decoder performs query-adaptive refinement via sparse expert routing. MR-to-HD (MR2HD) fusion transfers span-level
retrieval evidence to clip-level highlight prediction through confidence-weighted multi-scale aggregation. Using CLIP+SlowFast features, EviDETR achieves 69.29 R1@0.5, 54.77 R1@0.7, and 48.41 Avg. mAP for moment
retrieval on QVHighlights, together with 41.83 HD-mAP and 68.33 HIT@1. Strong results on TACoS and Charades-STA further demonstrate cross-
dataset transferability.
Read the full abstract ▾
All authors Haoran Sun, Yufan Li, Qichen Zhang, Haoran Zhao, Shuqi Wang
Computer Vision
TrafficImag: A Benchmark for Counterfactual Roadside Traffic Video Generation
Xiangyu Li, Tianyi Wang, Zhihao Dou et al. (+2 more)
Existing roadside traffic
datasets support perception,
forecasting, and visual question answering, but they do not evaluate counterfactual
video generation, in which a selected actor is modified and the generated future should remain consistent with road
topology and unrelated traffic. We introduce TrafficImag, the first
benchmark for counterfactual roadside traffic
video generation. TrafficImag combines a large-scale roadside
dataset (9,022 annotated images, 7,043 deduplicated
video clips, and 31,145 actor-centered history-future samples) with an executable protocol that supports behavior
reasoning, intervention-aware image editing, and conditional
video generation. Each intervention is represented as an actor-level program describing the target actor, intended behavior, legal route, interaction order, and temporal constraints, enabling a unified evaluation interface across heterogeneous foundation models. TrafficImag evaluates four complementary validity dimensions: initial-state correctness, route and behavior validity, interaction consistency, and non-target preservation, and considers an end-to-end counterfactual successful only when all four are satisfied. Across state-of-the-art foundation models, the strongest reasoner reaches 80.4% macro F1, the complete condition interface raises end-to-end success from 23.3% to 55.0% for the best generator. Oracle studies further show that conditional
video execution is the primary remaining bottleneck. TrafficImag provides a reproducible
benchmark for evaluating and diagnosing counterfactual traffic
video generation beyond perceptual
video quality.
Read the full abstract ▾
All authors Xiangyu Li, Tianyi Wang, Zhihao Dou, Christian Claudel, Zhaomiao Guo