Computer Vision
Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment
Yuchen Guo, Junli Gong, Yao Lu et al. (+3 more)
Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception. However, existing evaluation approaches tend to over-optimize both hand-crafted no-reference statistics and full-reference metrics that treat the source images as pseudo ground truths. Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores, neither leveraging the
reasoning of
Multimodal Large
Language Models (MLLMs) nor encoding per-image perceptual ambiguity in their supervision, but naively introducing MLLMs with discrete one-hot supervision likewise collapses fused images of similar quality into different rating levels. To address this, we introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score, rather than discrete level predictions, enabling fine-grained discrimination among fused images of similar quality. We exploit the agreement among four IVIF-specific sub-dimensions to construct a per-image soft label whose sharpness reflects how consensual the overall judgment is. We further introduce a tripartite objective combining per-image distributional supervision, within-source-pair Thurstone fidelity for method-level ordering, and cross-source-pair Thurstone fidelity for scene-level ordering across scenes. Extensive experiments demonstrate that FuScore achieves state-of-the-art correlation with human visual preferences.
Read the full abstract ▾
All authors Yuchen Guo, Junli Gong, Yao Lu, Xintong Xu, Yiuming Cheung, Weifeng Su
Computer Vision
XiYOLO: Energy-Aware Object Detection via Iterative Architecture Search and Scaling
Tony Tran, Richie R. Suganda and Bin Hu
Object detection on heterogeneous edge devices must satisfy strict energy, latency, and
memory constraints while still providing reliable perception for downstream autonomy. Existing energy-aware NAS methods often target limited deployment settings, while real energy remains difficult to optimize because it is highly device-dependent and costly to measure. We address these challenges with an energy-adaptive framework that combines an energy-aware XiResOFA search space, a two-stage energy
estimator, and iterative search to identify a single energy-efficient base architecture. We then apply compound scaling to transform this base design into the XiYOLO family across deployment budgets, enabling interpretable accuracy-energy tradeoffs under sparse hardware measurements. Experiments on PascalVOC, COCO, and real-device deployment show that XiYOLO achieves a stronger energy-accuracy tradeoff than YOLO baselines. On PascalVOC, the medium XiYOLO model reaches 86.15 mAP50 while reducing energy relative to YOLOv12m by 20.6% on GPU and 35.9% on NPU. On COCO, XiYOLO reduces energy relative to YOLOv12 by up to 53.7% on GPU and 51.6% on NPU at the small scale. The proposed two-stage
estimator also improves sample efficiency over a joint predictor under few-shot adaptation with only 2-20 target-device samples.
Read the full abstract ▾
Computer Vision
A$^2$RD: Agentic Autoregressive Diffusion for Long Video Consistency
Do Xuan Long, Yale Song, Min-Yen Kan et al. (+2 more)
Synthesizing consistent and coherent long
video remains a fundamental challenge. Existing methods suffer from semantic drift and narrative collapse over long horizons. We present A$^2$RD, an Agentic Auto-Regressive
Diffusion architecture that decouples creative synthesis from consistency enforcement. A$^2$RD formulates long
video synthesis as a closed-loop process that synthesizes and self-improves
video segment-by-segment through a Retrieve–Synthesize–Refine–Update cycle. It comprises three core components: (i)
Multimodal Video Memory that tracks
video progression across modalities; (ii) Adaptive Segment Generation that switches among generation modes for natural progression and visual consistency; and (iii) Hierarchical Test-Time Self-Improvement that self-improves each segment at frame and
video levels to prevent error propagation. We further introduce LVBench-C, a challenging
benchmark with non-linear entity and environment transitions to stress-test long-horizon consistency. Across public and LVBench-C
benchmarks spanning one- to ten-minute
videos, A$^2$RD outperforms state-of-the-art baselines by up to 30% in consistency and 20% in narrative coherence. Human evaluations corroborate these gains while also highlighting notable improvements in motion and transition smoothness.
Read the full abstract ▾
All authors Do Xuan Long, Yale Song, Min-Yen Kan, Tomas Pfister, Long T. Le
Computer Vision
Advancing Reliable Synthetic Video Detection: Insights from the SAFE Challenge
Kirill Trapeznikov, Gabriel Mancino-Ball, Jonathan Li et al. (+9 more)
The proliferation of generative
video technologies has intensified the need for reliable methods to detect and characterize synthetic media. To address this challenge, we organized the \href{https://safe-
video-2025.dsri.org}{SAFE: Synthetic
Video Detection Challenge}, co-located with the Authenticity and Provenance in the Age of Generative AI (APAI) Workshop at ICCV 2025. The competition invited participants to develop and evaluate
algorithms capable of distinguishing real from synthetic
videos under fully blind evaluation conditions with over 600 submissions from 12 teams over a 90 day span. Hosted on the Hugging Face platform, the challenge comprised two primary tasks: (1) detection of synthetic
video content generated by diverse state-of-the-art models, and (2) detection of synthetic content following common post-processing operations such as resizing, re-compression, motion blur and others. The challenge data consisted of 13 modern high quality synthetic
video models with generated content matched to real
videos from 21 diverse and challenge sources, all adding up to 20 hours of 6,000
video samples. This paper describes the challenge design,
dataset construction, evaluation methodology, and outcomes, offering insights into the generalization and robustness of contemporary synthetic
video detection methods. Our findings highlight measurable progress in cross-generator generalization but also persistent vulnerabilities to post-processing artifacts. https://safe-
video-2025.dsri.org
Read the full abstract ▾
All authors Kirill Trapeznikov, Gabriel Mancino-Ball, Jonathan Li, Paul Cummer, Jai Aslam, Danial Samadi Vahdati, Tai Nguyen, Matthew C. Stamm, Peter Bautista, Michael Davinroy, Laura Cassani, Jill Crisman
Computer Vision
Not All Tokens Need 40 Steps: Heterogeneous Step Allocation in Diffusion Transformers for Efficient Video Generation
Ernie Chu and Vishal M. Patel
Diffusion Transformers (DiTs) have achieved state-of-the-art
video generation quality, but they incur immense computational cost because standard inference applies the same number of denoising steps uniformly to every
token in the sequence. It is well known that human
vision ignores vast amounts of redundant motion. Why, then, do our densest models treat every spatiotemporal
token with equal priority? In this paper, we introduce Heterogeneous Step Allocation (HSA), a training-free inference
algorithm that assigns varying step budgets to different spatiotemporal
tokens based on their velocity dynamics. To resolve the resulting sequence-length mismatch without sacrificing global context, HSA introduces a KV-cache synchronization mechanism that allows active
tokens to attend to the full sequence while entirely bypassing inactive
tokens. Furthermore, we derive a cached Euler update that advances the latent states of skipped
tokens in a single operation without additional model evaluations. We evaluate HSA on the Wan-2 and LTX-2 models for both text-to-
video (T2V) and image-to-
video (I2V) generation. Our results demonstrate that HSA significantly outperforms previous state-of-the-art caching methods and the vanilla Flow Matching baseline, especially at aggressive acceleration regimes (e.g., 50% and 25% runtimes). Crucially, HSA achieves a superior quality-runtime Pareto frontier without the need for expensive offline profiling, robustly preserving structural integrity and generation quality even under tight computational budgets. Project page: https://ernestchu.github.io/hsa
Read the full abstract ▾
Computer Vision
Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation
Aditya Parikh, Stella Frank, Sneha Das et al. (+1 more)
Biases in annotation pipelines may introduce label bias: when annotation quality or style varies systematically across demographic subgroups, causing disparities in performance and auditability. Label bias in image
segmentation remains underexplored, as even detecting it typically requires clean, unbiased annotations, which are not readily available. We present a Confident Learning framework adapted to
segmentation for
auditing label bias directly in the training data without a clean, unbiased ground truth. By comparing the provided training labels to the model's confident predictions, we measure the prevalence and direction of suspected label errors; where standard overlap metrics like Dice fail. We further show that label bias influences subgroup separability in the encoder's feature space, an artifact we leverage for bias mitigation rather than suppressing it. We evaluate three
datasets spanning from synthetic to real-life bias and across experimental conditions, showing how our framework produces proxy audit signals without clean labels and mitigates disparities with a correctly identified cleaner reference group.
Read the full abstract ▾
All authors Aditya Parikh, Stella Frank, Sneha Das, Aasa Feragen
Computer Vision
TriDE: Triangle-Consistent Translation Directions for Global Camera Pose Estimation
Francisco Chen, Yiran Wang and Yunpeng Shi
Pairwise translation directions are a key input to camera location estimation in global structure-from-motion. Existing
estimators usually process each image pair independently, producing directions that may be locally plausible but inconsistent with the other relative directions in the viewing graph. To jointly estimate the direction, we propose TriDE, which exploits camera-triangle consistency as an efficient higher-order verification signal. Instead of solving a costly global nonlinear optimization problem that is sensitive to initialization, TriDE refines unreliable pairwise directions through message passing between directions and their incident weighted triangles. This information propagation strategy enables us to establish a strong phase-transition bound for exact recovery under a realistic random corruption model. Experiments on real image graphs show that TriDE improves direction accuracy by a large margin and yields better downstream camera locations, providing a practical link between local pairwise estimation and global camera pose geometry.
Read the full abstract ▾
Computer Vision
AdpSplit: Error-Driven Adaptive Splitting for Faster Geometry Discovery in 3D Gaussian Splatting
Yongjae Lee, Jingxing Li, Abhay Kumar Yadav et al. (+2 more)
Adaptive density control in 3D Gaussian Splatting (3DGS) repeatedly grows the Gaussian population through fixed-cardinality random splitting to discover useful scene structure. However, in vanilla 3DGS, its binary split operator requires many densification rounds to expose fine details, making it a bottleneck for efficient training schedules with fewer iterations. We introduce AdpSplit, an error-driven adaptive split operator that determines the number of split children and initializes the child parameters from L1-pixel-error region statistics, enabling fewer densification iterations, thus reduced training time, while preserving the rendering quality of full-schedule training. Across the MipNeRF360, Deep-Blending, and Tanks&Temples
datasets, AdpSplit reduces the training time of multiple accelerated 3DGS pipelines by 9.2%-22.3% as a simple drop-in replacement for the standard split operator. With FastGS, AdpSplit matches the full-schedule PSNR on MipNeRF360 while reducing training time by 16.4%, corresponding to a 12.6x acceleration over vanilla 3DGS.
Read the full abstract ▾
All authors Yongjae Lee, Jingxing Li, Abhay Kumar Yadav, Rama Chellappa, Deliang Fan
Computer Vision · Cross-listed from cs.AR
EULER-ADAS: Energy-Efficient & SIMD-Unified Logarithmic-Posit Engine for Precision-Reconfigurable Approximate ADAS Acceleration
Mukul Lokhande, Ratko Pilipovic, Omkar Kokane et al. (+2 more)
Advanced driver-assistance systems (ADAS) require neural compute engines that deliver low-latency inference under strict power and area constraints. Posit arithmetic is attractive for such accelerators because it provides high numerical fidelity at low precision, but its variable-length regime encoding increases encode/decode cost and exposes the datapath to large regime-field fault effects. This paper presents EULER-ADAS, a SIMD-enabled logarithmic bounded-Posit neural compute engine for energyefficient and reliability-aware ADAS acceleration. The proposed datapath combines bounded-regime Posit representation, stageadaptive logarithmic mantissa multiplication with bit truncation, and a SIMD-shared quire accumulation path supporting Posit- (8,0), Posit-(16,1), and Posit-(32,2) execution. The unified architecture enables 4xPosit-8, 2xPosit-16, or 1xPosit-32 operation without duplicating precision-specific hardware. FPGA implementation shows that the proposed configurations reduce LUT count by up to 41.4%, delay by up to 76.1%, and power by up to 71.9% relative to exact Posit neural compute engines, while achieving up to 10x lower energy-delay product than radix-4 Booth-based Posit multipliers. In 28-nm CMOS, the bounded variants occupy 0.013-0.016 mm2 , consume 19.8-22.1 mW, and operate at up to 1.84 GHz. Application-level evaluation across image-classification, ADAS, and edge-inference workloads shows that the evaluated Posit-16 and Posit-32 configurations remain within about 1.5 percentage points of FP32 accuracy. A TinyYOLOv3 prototype on Pynq-Z2 achieves 78 ms latency at 0.29 W and 22.6 mJ/frame, demonstrating the suitability of EULERADAS for low-power real-time ADAS inference.
Read the full abstract ▾
All authors Mukul Lokhande, Ratko Pilipovic, Omkar Kokane, Adam Teman, Santosh Kumar Vishvakarma
Computer Vision
Knowledge Transfer Scaling Laws for 3D Medical Imaging
Ho Hin Lee, Dongna Du, Chu Wang et al. (+4 more)
Vision foundation models are increasingly moving beyond 2D to volumetric domains such as 3D medical imaging, where unified pretraining across different imaging modalities (i.e. CT, MRI, and PET) could provide foundational models for diverse clinical tasks. However, training such models requires mixing heterogeneous imaging domains, and current mixture strategies remain largely heuristic. In this work, we observe that different medical imaging domains scale at variable rates during pretraining, and knowledge transfer between domains is strongly asymmetric: training on one domain can substantially improve another, but the reverse may be much weaker. Interestingly, both MAE reconstruction loss and cross-domain transfer follow predictable power-law trends with domain-specific behaviors. Motivated by these findings, we formulate data allocation as a scaling-law optimization problem. The derived allocations reveal an interpretable hub-and-island structure: highly transferable domains emerge as hubs that benefit many others and deserve strategic allocation, while isolated domains act as islands requiring direct investment. Empirically, transfer-aware allocation outperforms data-proportional sampling by up to 58% and generalizes well to unseen budgets with r=0.989. Downstream validation on disease classification and organ/lesion
segmentation further confirms that the derived transfer-aware mixtures provide stronger pretrained representations for clinical 3D medical imaging tasks.
Read the full abstract ▾
All authors Ho Hin Lee, Dongna Du, Chu Wang, Yuankai Huo, Shi Gu, James C. Gee, Yifan Wu
Computer Vision · Cross-listed from cs.LG
A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
Aditya Ranganath and Mukesh Singhal
We survey continuous-time generative modeling methods based on transporting a simple reference distribution to a data distribution via
stochastic or deterministic dynamics. We present a unified framework in which
diffusion models, score-based generative models, and flow matching are instances of learning a time-dependent vector field that induces a family of marginals $(\rho_t)_{t \in [0,1]}$ governed by continuity and Fokker-Planck equations. Such a unified theory is timely because these methods are converging methodologically, yet fragmented notation and competing derivations continue to obscure their shared structure and the practical tradeoffs governing sampling, stability, and computation. Within this framework, we (i) derive reverse-time sampling for
diffusion and score-based models as controlled
stochastic dynamics, (ii) show that the probability flow ODE yields identical marginals and connects
diffusion to likelihood-based normalizing flows, and (iii) interpret flow matching as direct regression of the velocity field under a chosen interpolation, clarifying when it coincides with or differs from score-based training. We compare objectives, sampling schemes, and discretization errors under unified notation, discuss connections to Schrodinger bridges and entropic optimal transport, and summarize theoretical guarantees and open problems on approximation, stability, and scalability.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Uneven Evolution of Cognition Across Generations of Generative AI Models
Isaac Galatzer-Levy, Daniel McDuff, Xin Liu et al. (+1 more)
The pursuit of artificial general intelligence necessitates robust methods for evaluating the cognitive capabilities of models beyond narrow task performance. Here, we introduce a psychometric framework to assess the cognitive profiles of generative AI, comparing them to human norms and tracking their
evolution across generations. Initial evaluation of leading
multimodal models using tasks adapted from the Wechsler Adult Intelligence Scale revealed a profoundly uneven cognitive architecture: near-ceiling performance in verbal comprehension and working
memory (>$98^{\text{th}}$ percentile) contrasted with near-floor performance in perceptual
reasoning (<$1^{\text{st}}$ percentile). To track developmental trajectories beyond human-normed limits, we developed the Artificial Intelligence Quotient (AIQ)
Benchmark and applied it to six generations and two model families, revealing significant but asymmetric performance gains. Notably, we uncovered a sharp dissociation between modalities; abstract quantitative
reasoning matured far more rapidly when presented linguistically compared to a visually analogous format, indicating an architectural bias towards
language-based symbolic manipulation. While abstract visual
reasoning improved, visual-perceptual organization remained largely stagnant. Collectively, these findings demonstrate that the cognitive abilities of generative models are evolving unevenly, suggesting that scaling and optimization approaches to AGI development alone may be insufficient to overcome fundamental architectural limitations in achieving balanced, human-like general intelligence.
Read the full abstract ▾
All authors Isaac Galatzer-Levy, Daniel McDuff, Xin Liu, Jed McGiffin
Computer Vision · Cross-listed from cs.HC
Enhancing Eye Movement Biometrics for User Authentication via Continuous Gaze Offset Score Fusion
Hashim Aziz, Mehedi Hasan Raju and Oleg V. Komogortsev
Eye movement biometrics (EMB) use subject-specific gaze dynamics for user authentication and identification. Recent deep learning-based EMB systems achieve strong performance by modeling temporal eye movement behavior. However, these systems typically overlook continuous gaze offset, despite prior evidence that it contains user-discriminative information. This work examines whether continuous gaze offset can improve biometric performance when combined with existing biometric features. We evaluate linear and nonlinear fusion methods on two publicly available
datasets, collected via the lab-grade eye tracker and virtual reality headset across multiple tasks and observation durations. Results indicate that fusion offers performance benefits on both
datasets, particularly when using nonlinear fusion. Additionally, fusing biometric information across multiple tasks further improves authentication performance. These findings support the hypothesis that continuous gaze offset may serve as useful auxiliary information under conditions of degraded or noisy eye tracking.
Read the full abstract ▾
Computer Vision
LookWhen? Fast Video Recognition by Learning When, Where, and What to Compute
Ali Salamatian, Anthony Fuller, Pritam Sarkar et al. (+3 more)
Transformers dominate
video recognition. They split
videos into
tokens, and processing them has expensive superlinear computational cost. Yet
videos are filled with redundancy, so we can question the need for this expense. We introduce LookWhen, a selector-extractor framework that factorizes
video recognition into learning when, where, and what to compute. Our shallow selector gets a scaled-down
video and quickly scores all
tokens across space-time, while our deep extractor gets the top-K selected
tokens to approximate full-
video representations without actually processing all the
tokens. A key challenge is defining effective supervision for selection and extraction. For selection pre-training, we introduce a score on representations that ranks
tokens by uniqueness using a simple nearest-neighbor distance. For extraction pre-training, we distill both a
video teacher and an image teacher, for which we normalize its frame-wise representations to learn what changes within
videos. Through these strategies, our selector-extractor learns general and efficient representations for feature extraction or fine-tuning to a task. Through experiments on Kinetics-400, SSv2, Epic-Kitchens, Diving48, Jester, and Charades, we show that LookWhen achieves a better accuracy-computation trade-off than efficient models and upgraded baselines of similar size. LookWhen Pareto-dominates in accuracy-FLOPs on 9 of 12 cases (6 tasks x 2 settings) and roughly matches on 3. In accuracy-throughput, measuring time in practice, LookWhen is more efficient still at 6.7x faster than InternVideo2-B at equal accuracy.
Read the full abstract ▾
All authors Ali Salamatian, Anthony Fuller, Pritam Sarkar, James R. Green, Leonid Sigal, Evan Shelhamer
Computer Vision · Cross-listed from cs.AI
Weblica: Scalable and Reproducible Training Environments for Visual Web Agents
Oğuzhan Fatih Kar, Roman Bachmann, Yuanzheng Gong et al. (+2 more)
The web is complex, open-ended, and constantly changing, making it challenging to scale training data for visual web
agents. Existing data collection attempts remain limited to offline trajectories for supervised fine-tuning or a handful of simulated environments for RL training, thus failing to capture web diversity. We propose Weblica (Web Replica), a framework for constructing reproducible and scalable web environments. Our framework leverages 1) HTTP-level caching to capture and replay stable visual states while preserving interactive behavior and 2) LLM-based environment synthesis grounded in real-world websites and core web
navigation skills. Using this framework, we scale RL training to thousands of diverse environments and tasks. Our best model, Weblica-8B, outperforms open-weight baselines of similar size across multiple web
navigation benchmarks while using fewer inference steps, scales favorably with additional test-time compute, and is competitive with API models.
Read the full abstract ▾
All authors Oğuzhan Fatih Kar, Roman Bachmann, Yuanzheng Gong, Anders Boesen Lindbo Larsen, Afshin Dehghan
Computer Vision
R$^3$L: Reasoning 3D Layouts from Relative Spatial Relations
Zhifeng Gu, Yuqi Wang and Bing Wang
Relative spatial relations provide a compact representation of spatial structure and are fundamental to relative spatial
reasoning in 3D layout generation. Recent works leverage
Multimodal Large
Language Models (MLLMs) to infer such relations, but the inferred relations are often unreliable and are typically handled with post-hoc heuristics. In this paper, we propose R$^3$L, a general framework that improves the reliability and consistency of relative spatial
reasoning for 3D layout generation. Our key motivation is that multi-hop
reasoning requires repeated reference-frame transformations, which accumulate errors in inferred relations and lead to semantic and metric drift. To mitigate this, we propose invariant spatial decomposition to break coupled relation chains, and consistent spatial imagination to promote self-consistency through an imagine-and-revise loop. We further introduce supportive spatial optimization to ease pose optimization via global-to-local coordinate re-parameterization. Extensive experiments across diverse scene types and instructions demonstrate that R$^3$L produces more physically feasible and semantically consistent layouts. Notably, our analysis shows that resolving frame-induced inconsistencies is crucial for reliable multi-hop relative spatial
reasoning. The code is available at https://github.com/Neal2020GitHub/R3L.
Read the full abstract ▾
Computer Vision
HumanNet: Scaling Human-centric Video Learning to One Million Hours
Yufan Deng and Daquan Zhou
Progress in
embodied intelligence increasingly depends on scalable data infrastructure. While
vision and
language have scaled with internet corpora, learning physical interaction remains constrained by the lack of large, diverse, and richly annotated human activity data. We present HumanNet, a one-million-hour human-centric
video corpus that captures how humans interact with the physical world at scale. HumanNet spans both first-person and third-person perspectives and covers fine-grained activities, human-object interactions, tool use, and long-horizon behaviors across diverse real-world environments. Beyond raw
video, the
dataset provides interaction-centric annotations, including captions, motion descriptions, and hand and body-related signals, enabling motion-aware and interaction-aware learning. Beyond scale, HumanNet introduces a systematic data curation paradigm for
embodied learning, where human-centric filtering, temporal structuring, viewpoint diversity, and annotation enrichment are treated as first-class design principles. This design transforms unstructured internet
video into a scalable substrate for representation learning, activity understanding, motion generation, and human-to-
robot transfer. We conduct a first-step validation on the value of this design through controlled
vision-
language-action ablation: under a fixed set of validation data, continued training from the Qwen VLM model with 1000 hours of
egocentric video drawn from HumanNet surpasses the continued training with 100 hours of real-
robot data from Magic Cobot, indicating that
egocentric human
video could be a scalable and cost-effective substitute for
robot data. By building this project, we aim to explore the opportunity to scale
embodied foundation models using human-centric
videos, rather than relying solely on
robot-specific data.
Read the full abstract ▾
Computer Vision
Edge Deep Learning in Computer Vision and Medical Diagnostics: A Comprehensive Survey
Yiwen Xu, Tariq M. Khan, Yang Song et al. (+1 more)
Edge deep learning, a paradigm change reconciling edge computing and deep learning, facilitates real-time decision making attuned to environmental factors through the close integration of computational resources and data sources. Here we provide a comprehensive review of the current state of the art in edge deep learning, focusing on computer
vision applications, in particular medical diagnostics. An overview of the foundational principles and technical advantages of edge deep learning is presented, emphasising the capacity of this technology to revolutionise a wide range of domains. Furthermore, we present a novel categorisation of edge hardware platforms based on performance and usage scenarios, facilitating platform selection and operational effectiveness. Following this, we dive into approaches to effectively implement deep neural networks on edge devices, encompassing methods such as lightweight design and model compression. Reviewing practical applications in the fields of computer
vision in general and medical diagnostics in particular, we demonstrate the profound impact edge-deployed deep learning models can have in real-life situations. Finally, we provide an analysis of potential future directions and obstacles to the adoption of edge deep learning, with the intention to stimulate further investigations and advancements of intelligent edge deep learning solutions. This survey provides researchers and practitioners with a comprehensive reference shedding light on the critical role deep learning plays in the advancement of edge computing applications.
Read the full abstract ▾
All authors Yiwen Xu, Tariq M. Khan, Yang Song, Erik Meijering
Computer Vision
ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation
Omar El Khalifi, Thomas Rossi, Oscar Fossey et al. (+6 more)
For artistic applications,
video generation requires fine-grained control over both performance and cinematography, i.e., the actor's motion and the camera trajectory. We present ActCam, a zero-shot method for
video generation that jointly transfers character motion from a driving
video into a new scene and enables per-frame control of intrinsic and extrinsic camera parameters. ActCam builds on any pretrained image-to-
video diffusion model that accepts conditioning in terms of scene depth and character pose. Given a source
video with a moving character and a target camera motion, ActCam generates pose and depth conditions that remain geometrically consistent across frames. We then run a single sampling process with a two-phase conditioning schedule: early denoising steps condition on both pose and sparse depth to enforce scene structure, after which depth is dropped and pose-only guidance refines high-frequency details without over-constraining the generation. We evaluate ActCam on multiple
benchmarks spanning diverse character motions and challenging viewpoint changes. We find that, compared to pose-only control and other pose and camera methods, ActCam improves camera adherence and motion fidelity, and is preferred in human evaluations, especially under large viewpoint changes. Our results highlight that careful camera-consistent conditioning and staged guidance can enable strong joint camera and motion control without training. Project page: https://elkhomar.github.io/actcam/.
Read the full abstract ▾
All authors Omar El Khalifi, Thomas Rossi, Oscar Fossey, Thibault Fouque, Ulysse Mizrahi, Philip Torr, Ivan Laptev, Fabio Pizzati, Baptiste Bellot-Gurlet
Computer Vision
BAMI: Training-Free Bias Mitigation in GUI Grounding
Borui Zhang, Bo Zhang, Bo Wang et al. (+6 more)
GUI grounding is a critical capability for enabling GUI
agents to execute tasks such as clicking and dragging. However, in complex scenarios like the ScreenSpot-Pro
benchmark, existing models often suffer from suboptimal performance. Utilizing the proposed Masked Prediction Distribution (MPD) attribution method, we identify that the primary sources of errors are twofold: high image resolution (leading to precision bias) and intricate interface elements (resulting in ambiguity bias). To address these challenges, we introduce Bias-Aware Manipulation Inference (BAMI), which incorporates two key manipulations, coarse-to-fine focus and candidate selection, to effectively mitigate these biases. Our extensive experimental results demonstrate that BAMI significantly enhances the accuracy of various GUI grounding models in a training-free setting. For instance, applying our method to the TianXi-Action-7B model boosts its accuracy on the ScreenSpot-Pro
benchmark from 51.9% to 57.8%. Furthermore, ablation studies confirm the robustness of the BAMI approach across diverse parameter configurations, highlighting its stability and effectiveness. Code is available at https://github.com/Neur-IO/BAMI.
Read the full abstract ▾
All authors Borui Zhang, Bo Zhang, Bo Wang, Wenzhao Zheng, Yuhao Cheng, Liang Tang, Yiqiang Yan, Jie Zhou, Jiwen Lu
Computer Vision
Relit-LiVE: Relight Video by Jointly Learning Environment Video
Weiqing Xiao, Hong Li, Xiuyu Yang et al. (+7 more)
Recent advances have shown that large-scale
video diffusion models can be repurposed as neural renderers by first decomposing
videos into intrinsic scene representations and then performing forward rendering under novel illumination. While promising, this paradigm fundamentally relies on accurate intrinsic decomposition, which remains highly unreliable for real-world
videos and often leads to distorted appearances, broken materials, and accumulated temporal artifacts during relighting. In this work, we present Relit-LiVE, a novel
video relighting framework that produces physically consistent, temporally stable results without requiring prior knowledge of camera pose. Our key insight is to explicitly introduce raw reference images into the rendering process, enabling the model to recover critical scene cues that are inevitably lost or corrupted in intrinsic representations. Furthermore, we propose a novel environment
video prediction formulation that simultaneously generates relit
videos and per-frame environment maps aligned with each camera viewpoint in a single
diffusion process. This joint prediction enforces strong
geometric-illumination
alignment and naturally supports dynamic lighting and camera motion, significantly improving physical consistency in
video relighting while easing the requirement of known per-frame camera pose. Extensive experiments demonstrate that Relit-LiVE consistently outperforms state-of-the-art
video relighting and neural rendering methods across synthetic and real-world
benchmarks. Beyond relighting, our framework naturally supports a wide range of downstream applications, including scene-level rendering, material editing, object insertion, and streaming
video relighting. The Project is available at https://github.com/zhuxing0/Relit-LiVE.
Read the full abstract ▾
All authors Weiqing Xiao, Hong Li, Xiuyu Yang, Houyuan Chen, Wenyi Li, Tianqi Liu, Shaocong Xu, Chongjie Ye, Hao Zhao, Beibei Wang
Computer Vision
Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study
Hao Dong, Hongzhao Li, Shupan Li et al. (+3 more)
Despite the growing popularity of
Multimodal Domain Generalization (MMDG) for enhancing model robustness, it remains unclear whether reported performance gains reflect genuine algorithmic progress or are artifacts of inconsistent evaluation protocols. Current research is fragmented, with studies varying significantly across
datasets, modality configurations, and experimental settings. Furthermore, existing
benchmarks focus predominantly on action recognition, often neglecting critical real-world challenges such as input corruptions, missing modalities, and model trustworthiness. This lack of standardization obscures a reliable assessment of the field's advancement. To address this issue, we introduce MMDG-Bench, the first unified and comprehensive
benchmark for MMDG, which standardizes evaluation across six
datasets spanning three diverse tasks: action recognition, mechanical fault diagnosis, and sentiment analysis. MMDG-Bench encompasses six modality combinations, nine representative methods, and multiple evaluation settings. Beyond standard accuracy, it systematically assesses corruption robustness, missing-modality generalization, misclassification detection, and out-of-distribution detection. With 7, 402 neural networks trained in total across 95 unique cross-domain tasks, MMDG-Bench yields five key findings: (1) under fair comparisons, recent specialized MMDG methods offer only marginal improvements over ERM baseline; (2) no single method consistently outperforms others across
datasets or modality combinations; (3) a substantial gap to upper-bound performance persists, indicating that MMDG remains far from solved; (4) trimodal fusion does not consistently outperform the strongest bimodal configurations; and (5) all evaluated methods exhibit significant degradation under corruption and missing-modality scenarios, with some methods further compromising model trustworthiness.
Read the full abstract ▾
All authors Hao Dong, Hongzhao Li, Shupan Li, Muhammad Haris Khan, Eleni Chatzi, Olga Fink
Computer Vision · Cross-listed from cs.AI
GlazyBench: A Benchmark for Ceramic Glaze Property Prediction and Image Generation
Ziyu Zhai, Siyou Li, Juexi Shao et al. (+1 more)
Developing ceramic glazes is a costly, time-consuming process of trial and error due to complex chemistry, placing a significant burden on independent artists. While recent advances in
multimodal AI offer a modern solution, the field lacks the large-scale
datasets required to train these models. We propose GlazyBench, the first
dataset for AI-assisted glaze design. Comprising 23,148 real glaze formulations, GlazyBench supports two primary tasks: predicting post-firing surface properties, such as color and transparency, from raw materials, and generating accurate visual representations of the glaze based on these properties. We establish comprehensive baselines for property prediction using traditional machine learning and large
language models, alongside image generation
benchmarks using deep generative and large
multimodal models. Our experiments demonstrate promising yet challenging results. GlazyBench pioneers a new research direction in AI-assisted material design, providing a standardized
benchmark for systematic evaluation.
Read the full abstract ▾
All authors Ziyu Zhai, Siyou Li, Juexi Shao, Juntao Yu
Computer Vision
DPM++: Dynamic Masked Metric Learning for Occluded Person Re-identification
Lei Tan, Yingshi Luan, Pincong Zou et al. (+2 more)
Although person re-identification has made impressive progress, occlusion caused by obstacles remains an unsettled issue in real applications. The difficulty lies in the mismatch between incomplete occluded samples and holistic identity representations. Severe occlusion removes discriminative body cues and introduces interference from background clutter and occluders, making global metric learning unreliable. Existing methods mainly rely on extra pre-trained models to estimate visible parts for
alignment or construct occluded samples via data augmentation, but still lack a unified framework that learns robust visibility-consistent matching under realistic occlusion patterns. In this paper, we propose DPM++, a Dynamic Masked Metric Learning framework for occluded person re-identification. DPM++ learns an input-adaptive masked metric that dynamically selects reliable identity subspaces for each occluded instance, enabling matching to emphasize visibility-consistent evidence while suppressing unreliable components. Built upon the classifier-prototype space, DPM++ introduces a CLIP-based two-stage supervision scheme, where ID-level semantic priors are learned from the text branch and transferred into the classifier-prototype space for dynamic masked matching. To strengthen the masked metric, we introduce a saliency-guided patch transfer strategy to synthesize controllable and photo-realistic occluded samples during training. Exploiting real scene priors, this strategy exposes the model to realistic partial observations and provides richer supervision than random erasing. In addition, occlusion-aware sample pairing and mask-guided optimization improve the stability and effectiveness of the framework. Experiments on occluded and holistic person re-identification
benchmarks show that DPM++ consistently outperforms previous state-of-the-art methods in both holistic and occlusion scenarios.
Read the full abstract ▾
All authors Lei Tan, Yingshi Luan, Pincong Zou, Pingyang Dai, Liujuan Cao
Computer Vision · Cross-listed from cs.LG
SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoders
Jakub Stępień, Marcin Mazur, Jacek Tabor et al. (+1 more)
Sparse Autoencoders (SAEs) have become an important tool in mechanistic interpretability, helping to analyze internal representations in both Large
Language Models (LLMs) and
Vision Transformers (ViTs). By decomposing polysemantic activations into sparse sets of monosemantic features, SAEs aim to translate neural network computations into human-understandable concepts. However, common architectures such as TopK SAEs rely on a fixed sparsity level. They enforce the same number of active features (K) across all inputs, ignoring the varying complexity of real-world data. Natural data often lies on
manifolds with varying local intrinsic dimensionality, meaning the number of relevant factors can change significantly across samples. This suggests that a fixed sparsity level is not optimal. Simple inputs may require only a few features, while more complex ones need more expressive representations. Using a constant K can therefore introduce noise in simple cases or miss important structure in more complex ones. To address this issue, we propose SoftSAE, a sparse autoencoder with a Dynamic Top-K selection mechanism. Our method uses a differentiable Soft Top-K operator to learn an input-dependent sparsity level k. This allows the model to adjust the number of active features based on the complexity of each input. As a result, the representation better matches the structure of the data, and the explanation length reflects the amount of information in the input. Experimental results confirm that SoftSAE not only finds meaningful features, but also selects the right number of features for each concept. The source code is available at: https://github.com/St0pien/SoftSAE.
Read the full abstract ▾
All authors Jakub Stępień, Marcin Mazur, Jacek Tabor, Przemysław Spurek
Computer Vision
DINORANKCLIP: DINOv3 Distillation and Injection for Vision-Language Pretraining with High-Order Ranking Consistency
Shuyang Jiang, Nan Yu, Yiming Zhang et al. (+2 more)
Contrastive
language-image pretraining (CLIP) suffers from two structural weaknesses: the symmetric InfoNCE loss discards the relative ordering among unmatched in-batch pairs, and global pooling collapses the visual representation into a semantic bottleneck that is poorly sensitive to fine-grained local structure. RANKCLIP partially addresses the first issue with a list-wise Plackett-Luce ranking-consistency loss, but its model is strictly first-order and inherits the second weakness untouched. We propose DINORANKCLIP, a pretraining framework that addresses both jointly. Our principal contribution is injecting a frozen DINOv3 teacher into the contrastive trunk through a dual-branch lightweight student and a multi-scale fusion module with channel-spatial
attention, a self-
attention refiner, and a conflict-aware gate that preserves the cross-modal
alignment up to first order. Complementarily, we introduce a high-order Plackett-Luce ranking model in which the per-position utility is augmented with
attention-parameterised pairwise and tuple-wise transition terms; the family contains CLIP and RANKCLIP as nested zero-order and first-order special cases, and the optimal order on every
benchmark is $R^*=3$. The full empirical study – order sweep, Fine-grained Probe on five
datasets, four-node Modality-Gap analysis, six-variant Fusion ablation – fits in 72 hours on a single eight-GPU H100 node and trains entirely on Conceptual Captions 3M. DINORANKCLIP consistently outperforms CLIP, CyCLIP, ALIP, and RANKCLIP under matched compute, with the largest relative gains on the fine-grained and out-of-distribution evaluations that most directly stress local structural
reasoning.
Read the full abstract ▾
All authors Shuyang Jiang, Nan Yu, Yiming Zhang, Zenghui Ding, Zhenyu Wu
Computer Vision
Solving Minimal Problems Without Matrix Inversion Using FFT-Based Interpolation
Haidong Wu, Snehal Bhayani and Janne Heikkilä
Estimating camera geometry typically involves solving minimal problems formulated as systems of multivariate
polynomial equations, which often pose computational challenges when using existing Gröbner-basis or resultant-based methods due to
matrix inversion needed in the online solver. Here we propose a sampling-based,
matrix inversion-free method that constructs the solvers using sparse hidden-variable resultants. The determinant
polynomial in the hidden variable is efficiently reconstructed via inverse fast
Fourier transform interpolation from sampled evaluations, avoiding symbolic expansion. Solving this
polynomial yields the hidden variable, and the remaining unknowns are recovered by identifying rank-1 deficient submatrices and applying Cramer's rule. A greatest common divisor-based criterion ensures robust submatrix identification under noise. Experiments on diverse minimal problems demonstrate that the proposed solver achieves strong numerical stability and competitive runtime, particularly for small-scale problems, providing a practical alternative to traditional Gröbner-basis and resultant-based solvers.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Hongcan Guo, Qinyu Zhao, Yian Zhao et al. (+8 more)
Large
language models have achieved remarkable success under the autoregressive paradigm, yet high-quality text generation need not be tied to a fixed left-to-right order. Existing alternatives still struggle to jointly achieve generation efficiency, scalable representation learning, and effective global semantic modeling. We propose Cola DLM, a hierarchical latent
diffusion language model that frames text generation through hierarchical information decomposition. Cola DLM first learns a stable text-to-latent mapping with a Text VAE, then models a global semantic prior in continuous latent space with a block-causal DiT, and finally generates text through conditional decoding. From a unified Markov-path perspective, its
diffusion process performs latent prior transport rather than
token-level observation recovery, thereby separating global semantic organization from local textual realization. This design yields a more flexible non-autoregressive inductive bias, supports semantic compression and prior fitting in continuous space, and naturally extends to other continuous modalities. Through experiments spanning 4 research questions, 8
benchmarks, strictly matched ~2B-parameter autoregressive and LLaDA baselines, and scaling curves up to about 2000 EFLOPs, we identify an effective overall configuration of Cola DLM and verify its strong scaling behavior for text generation. Taken together, the results establish hierarchical continuous latent prior modeling as a principled alternative to strictly
token-level
language modeling, where generation quality and scaling behavior may better reflect model capability than likelihood, while also suggesting a concrete path toward unified modeling across discrete text and continuous modalities.
Read the full abstract ▾
All authors Hongcan Guo, Qinyu Zhao, Yian Zhao, Shen Nie, Rui Zhu, Qiushan Guo, Feng Wang, Tao Yang, Hengshuang Zhao, Guoqiang Wei, Yan Zeng
Computer Vision
MedHorizon: Towards Long-context Medical Video Understanding in the Wild
Bodong Du, Bowen Liu, Yang Yu et al. (+8 more)
Medical
multimodal large
language models (MLLMs) have advanced image understanding and short-
video analysis, but real clinical review often requires full-procedure
video understanding. Unlike general long
videos, medical procedures contain highly redundant anatomical views, while decisive evidence is temporally sparse, spatially subtle, and context dependent. Existing
benchmarks often assume this evidence has already been localized through images, short clips, or pre-segmented
videos, leaving the
retrieval-before-
reasoning problem under-tested. We introduce MedHorizon, an in-the-wild
benchmark for long-context medical
video understanding. MedHorizon preserves 759 hours of full-length clinical procedures and provides 1,253 evidence-grounded multiple-choice questionsthat jointly evaluate sparse evidence understanding and multi-hop clinical
reasoning. Its evidence is extremely sparse, with only 0.166% evidence frames on average, requiring models to search noisy procedural streams before interpreting and aggregating findings. We evaluate representative general-domain, medical-domain, and long-
video MLLMs. The best model reaches only 41.1% accuracy, showing that current systems remain far from robust full-procedure understanding. Further analysis yields four key findings: performance does not scale reliably with more frames, evidence
retrieval and clinical interpretation remain primary bottlenecks; these bottlenecks are rooted in weak procedural
reasoning and
attention drift under redundancy, and generic sampling methods only partially balances local detail with global coverage. MedHorizon provides a rigorous testbed for MLLMs that retrieve sparse evidence and reason over complete clinical workflows.
Read the full abstract ▾
All authors Bodong Du, Bowen Liu, Yang Yu, Xinpeng Ding, Zhiheng Wu, Shuning Wang, Shuo Nie, Naiming Liu, Qifeng Chen, Yangqiu Song, Xiaomeng Li
Computer Vision
Sparkle: Realizing Lively Instruction-Guided Video Background Replacement via Decoupled Guidance
Ziyun Zeng, Yiqi Lin, Guoqiang Liang et al. (+1 more)
In recent years, open-source efforts like Senorita-2M have propelled
video editing toward natural
language instruction. However, current publicly available
datasets predominantly focus on local editing or style transfer, which largely preserve the original scene structure and are easier to scale. In contrast, Background Replacement, a task central to creative applications such as film production and advertising, requires synthesizing entirely new, temporally consistent scenes while maintaining accurate foreground-background interactions, making large-scale data generation significantly more challenging. Consequently, this complex task remains largely underexplored due to a scarcity of high-quality training data. This gap is evident in poorly performing state-of-the-art models, e.g., Kiwi-Edit, because the primary open-source
dataset that contains this task, i.e., OpenVE-3M, frequently produces static, unnatural backgrounds. In this paper, we trace this quality degradation to a lack of precise background guidance during data synthesis. Accordingly, we design a scalable pipeline that generates foreground and background guidance in a decoupled manner with strict quality filtering. Building on this pipeline, we introduce Sparkle, a
dataset of ~140K
video pairs spanning five common background-change themes, alongside Sparkle-Bench, the largest evaluation
benchmark tailored for background replacement to date. Experiments demonstrate that our
dataset and the model trained on it achieve substantially better performance than all existing baselines on both OpenVE-Bench and Sparkle-Bench. Our proposed
dataset,
benchmark, model, as well as the full data generation pipeline, are fully open-sourced at https://showlab.github.io/Sparkle/.
Read the full abstract ▾
All authors Ziyun Zeng, Yiqi Lin, Guoqiang Liang, Mike Zheng Shou