Computer Vision
Geometry-Aware Cross-Height Channel Knowledge Map Prediction for UAV-Assisted Communications With Uncertainty-Guided 3D Sensing
Zhihan Zeng, Amir Hussain, Yue Xiu et al. (+4 more)
Low-altitude Unmanned Aerial Vehicles (
UAVs) often need to infer channel knowledge across a range of heights from only sparse observations collected at a few altitude layers. To address this challenge, this paper studies height-conditioned cross-height channel knowledge map (CKM) prediction for
UAV-assisted communications in geometry-rich urban environments. We develop a geometry-aware conditional prediction framework that combines urban scene priors, sparse multi-altitude observations, and target-height descriptors to reconstruct dense CKMs at unobserved target heights. An uncertainty head is further introduced to characterize prediction confidence and to support cost-aware online
UAV sensing under motion and safety constraints. Experiments on a layered aerial CKM
benchmark show that the proposed Feature Pyramid Network (FPN)-
Transformer achieves the best overall performance under both unseen-scene zero-shot and legacy patch-random protocols, reducing the Root Mean Square Error (RMSE) to 5.347dB and 1.111dB, respectively, compared with 6.937dB and 1.221dB for the strongest baseline 3D-RadioDiff. Moreover, after applying our unseen-scene few-shot adaptation, the RMSE further decreases from 5.347dB in zero-shot prediction to 3.518dB with 10-shot two-height support, while the uncertainty-guided cost-aware sensing policy improves active reconstruction from 6.94dB at initialization to 4.79dB at sensing budget 40, outperforming uncertainty-only sensing at 5.08dB and random aerial sampling at 5.84dB.
Read the full abstract ▾
All authors Zhihan Zeng, Amir Hussain, Yue Xiu, Phee Lep Yeoh, Lu Chen, Zhongpei Zhang, Guan Gui
Computer Vision
Beyond Pixel Overlap: A Framework for Decomposing Segmentation Evaluation Metrics
Youwei Pang and Xiaoqi Zhao
Evaluation metrics are central to binary target
segmentation because they determine how progress is measured, compared, and interpreted. In this paper, target denotes the task-defined positive region to be segmented rather than a generic foreground object. It may be salient, camouflaged, transparent, glass-like, mirror-like, shadow-like, lesion-like, or defined by other application-specific semantics. We treat existing metrics as compositions of modular design choices rather than isolated formulas. The proposed framework decomposes each metric into five stages covering prediction representation, target extraction, target matching, score computation, and metric reporting. We use this framework to analyze representative metrics and show how newer metrics address specific limits in earlier protocols. The stage choices keep each metric's assumptions visible. We then discuss the design space opened by the framework and its implications for task-aware evaluation protocols. Reference code is available at https://github.com/lartpang/PySODMetrics.
Read the full abstract ▾
Computer Vision
Improving Sparse-View 3DGS Generalization via Flat Minima Optimization
Kangmin Seo, Sangeek Hyun, MinKyu Lee et al. (+1 more)
Recent advances in neural rendering have established 3D Gaussian Splatting (3DGS) as a highly efficient representation for novel view synthesis, enabling fast training and real-time rendering with strong fidelity. However, when supervision is limited to sparse input views, 3DGS tends to overfit to the observed images and generalize poorly to unseen viewpoints. We address this challenge from the perspective of flat minima (FM) optimization, which seeks solutions that remain stable under small parameter perturbations. Viewing Gaussian parameters as trainable weights, we adapt FM principles to the
geometric and dynamic nature of 3DGS with a lightweight training framework. Our method regularizes optimization with controlled Gaussian perturbations that account for each Gaussian's anisotropy and the training progress, preserving fine details while improving robustness to sparse-view overfitting. To further stabilize this flat minima optimization process, we introduce periodic reinitialization, which temporarily returns non-positional parameters to their initial states for a short window. Together, these techniques integrate seamlessly into existing 3DGS pipelines without architectural changes. Experiments on LLFF and Mip-NeRF360
datasets demonstrate improved quantitative metrics and perceptual quality under sparse-view supervision, producing reconstructions that are sharper, more stable, and better generalized to novel viewpoints.
Read the full abstract ▾
All authors Kangmin Seo, Sangeek Hyun, MinKyu Lee, Jae-Pil Heo
Computer Vision
OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping
Xudong Li, Mengdan Zhang, Peixian Chen et al. (+7 more)
Spatial intelligence remains a persistent challenge for
Multimodal Large
Language Models (MLLMs), as it requires coherent spatial scene representations beyond basic object recognition. Existing methods typically build such representations through textual
reasoning or 3D reconstruction. However, they often falter during multi-step
reasoning, particularly when required to dynamically re-anchor evidence to the specific camera-, object-, or direction-centric reference frames demanded by complex queries. To address this, we propose OmniView-Space, a framework designed to maintain spatial consistency through
multimodal egocentric evidence. Our approach consists of three core components: (1) Multi-Perspective Spatial Mapping (MPSM), which re-anchors reconstructed geometry into a query-aligned visual cognitive map and a textual spatial graph; (2) Tool-Guided
Egocentric Reasoning, an interleaved policy trained to actively select the ego anchor required by the query and request the corresponding MPSM evidence; and (3) Cognitive-Map
Distillation, which uses MPSM-generated trajectories and ego-frame rewards to train the model to reason with self-generated cognitive maps. Experiments on single- and multi-image spatial
reasoning benchmarks show that OmniView-Space achieves state-of-the-art performance. Furthermore, the distilled model maintains this performance while reducing reliance on external geometry pipelines.
Read the full abstract ▾
All authors Xudong Li, Mengdan Zhang, Peixian Chen, Jiaxi Tan, Zihao Huang, Jingyuan Zheng, Yan Zhang, Xiawu Zheng, Xing Sun, Rongrong Ji
Computer Vision
EFlow: Learning Evidence Flow for Long-Video Reasoning with Adaptive Reflection
Wenhao Zhang, Kuanwei Lin, Xuyi Yang et al. (+2 more)
Long-
video reasoning is fundamentally constrained by how models acquire and utilize visual evidence. Existing tool-augmented
video frameworks often interleave temporal grounding and answer
reasoning within a single trajectory, causing early semantic hypotheses to bias evidence localization. We term this failure mode premature semantic commitment, where biased grounding retrieves incomplete evidence and incomplete evidence further reinforces incorrect
reasoning. To address this issue, we propose EFlow, an evidence-first
video reasoning framework built upon Qwen3-VL. EFlow explicitly separates temporal grounding and logical
reasoning through CoT for Temporal Grounding and CoT for
Reasoning, enabling the model to retrieve relevant evidence before answer inference. In addition, EFlow introduces a confidence-aware reflection mechanism that re-evaluates the full
video when retrieved evidence is potentially insufficient. We further construct dedicated trajectory
datasets and train EFlow through supervised fine-tuning, reinforcement learning, and reinforcement fine-tuning. Extensive experiments across five
video understanding
benchmarks demonstrate that EFlow consistently improves long-
video reasoning performance.
Read the full abstract ▾
All authors Wenhao Zhang, Kuanwei Lin, Xuyi Yang, Wei Gao, Ge Li
Computer Vision
TrajLoc: Trajectory-Attention Localization for Multi-Object Motion Control
Omer Sela, Inbar Huberman-Spiegelglas, Michael Rotman et al. (+2 more)
Controlling the motion of multiple objects in image-to-
video (I2V) generation requires preserving object identities while enforcing adherence to distinct target trajectories. This becomes particularly challenging as the number of objects increases and their paths intersect or occlude one another. Existing approaches entangle multiple trajectories within a shared, dense conditioning signal, making object-level correspondence difficult to preserve in crowded scenes. We depart from this paradigm and enforce a strict, per object spatial constraint that isolates instances independently. Our method, TrajLoc, achieves this directly within the
attention layers by substituting the cross-
attention weights of each object
token with a Gaussian heatmap centered on its target location at every frame. The same per object
token interface carries trajectory and depth through a learned
embedding and preserves identity by encoding first frame appearance in place of an object
token. Evaluations across six
datasets, featuring up to 20 simultaneously controlled objects and out of distribution real world scenes, demonstrate that our method consistently improves both visual fidelity and trajectory adherence. Applied to two architecturally distinct backbones (CogVideoX 5B and WaN 2.1 14B), our approach achieves average gains of +4.3 dB PSNR and a 51% reduction in trajectory end point error compared to the strongest baselines. Project page: https://sela-omer.github.io/traj-loc/
Read the full abstract ▾
All authors Omer Sela, Inbar Huberman-Spiegelglas, Michael Rotman, Sagie Benaim, Avi Ben-Cohen
Computer Vision
MoVA: Learning Asymmetric Dual Projections for Modular Long Video-Text Alignment
Peiyuan Zhu, Shaoan Xie, Zijian Li et al. (+5 more)
Contrastive pre-training has propelled
video-text
alignment, yet models often inherit the critical limitations of their image-text predecessors like CLIP, resulting in entangled representations. These challenges are severely exacerbated by two fundamental properties in the
video domain: Temporal Misalignment, where textual descriptions often correlate only to specific, constrained temporal windows, leaving other frames text-irrelevant; and Semantic Asymmetry, which dictates a sparse, bidirectional, and non-equivalent relevance between frame-level visual details and caption-level concepts. This failure persists whether captions are short and temporally disjoint, creating ambiguity, or long and detailed, fostering
entanglement between static objects and their temporal
evolution. In this paper, we establish theoretical conditions that enable flexible
alignment between
video and text representations across the temporal dimension and at varying levels of granularity. Building on these theoretical insights, we introduce MoVA, Modular Long
Video-Text
Alignment, which learns dual asymmetric projections: a text-side projection that adaptively selects frame-aware subspaces of the caption, and a
video-side projection that disentangles text-relevant visual concepts. Our framework ensures that the model can preserve global cross-modal semantics while disentangling evolving, frame-specific concepts and scale naturally to long captions and
videos. Empirical evaluations show that MoVA outperforms existing methods in multiple
video-text
alignment tasks, demonstrating the effectiveness of our method.
Read the full abstract ▾
All authors Peiyuan Zhu, Shaoan Xie, Zijian Li, Yifan Shen, Namrata Deka, Harsh Shrivastava, Guangyi Chen, Kun Zhang
Computer Vision
Mirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learning
Ruixin Li, Jin Liu, Yuling Shi et al. (+1 more)
Most self-supervised learning (SSL) methods encourage invariance across augmentations, but strict flip invariance can suppress informative left–right correspondences in approximately bilateral data such as medical images and human faces. We propose Mirror-Fusion-Augmented Self-Supervised Learning (MFASSL), a
Vision Transformer framework that injects a soft reflection prior into standard SSL without redesigning the backbone. MFASSL constructs mirror-paired views aligned to an estimated
symmetry axis and introduces a lightweight Mirror-Fusion
Attention (MFA) module for adaptive
token-level interaction between mirrored regions while preserving asymmetric cues. The base SSL objective is further coupled with reflection-consistency and mid-layer
token-
alignment losses. Across CheXpert, BraTS, CelebA-HQ, and WFLW, MFASSL improves downstream performance, calibration, and reflection robustness over MoCo-v3, DINO, and MAE baselines under matched ViT-B/16 settings. It also achieves stronger and more consistent gains than recent equivariant SSL approaches with only approximately 2.7% additional parameters. These results show that lightweight geometry-aware priors can effectively complement invariance-based SSL.
Read the full abstract ▾
All authors Ruixin Li, Jin Liu, Yuling Shi, Stefano Lodi
Computer Vision
Rethinking Multi-Label Image Classification With Deep Learning: Taxonomy, Challenge, and Outlook
Xuelin Zhu, Xiu-Shen Wei, Jiawei Ge et al. (+2 more)
Multi-label image classification (MLIC), a fundamental task in computer
vision, focuses on identifying multiple objects or concepts within an image, underpinning numerous read-world applications, such as autonomous driving, disease diagnosis, recommendation system, and mobile service
robot. Over the past decade, deep learning paradigms based on convolutional neural networks, recurrent neural networks, and
Transformers have significantly advanced this field, owing to their powerful capability in visual representation and relationship modeling. These advances have markedly improved the robustness, scalability, and generalization ability of MLIC models across diverse
datasets and application domains. In this survey, we provide a comprehensive review of the deep learning-based literature on MLIC. Concretely, we first revisit the background, including problem definition,
datasets, backbones and evaluation metrics. Next, we develop a plausible taxonomy for the deep learning-based MLIC approaches, organizing them into six groups: region-oriented methods, label-oriented methods, architecture-oriented methods, representation-oriented methods, learning-oriented methods, and data-oriented methods. Finally, we provide an insightful exposition of the underlying learning game in MLIC and its implications for other
vision domains, and we empirically summarize the key challenges and research directions in MLIC while outlining promising avenues for future development. We believe this survey offers the research community a holistic and systematic perspective on MLIC, thereby facilitating subsequent exploration and innovation in this field and beyond.
Read the full abstract ▾
All authors Xuelin Zhu, Xiu-Shen Wei, Jiawei Ge, Shuai Xu, Bing Wang
Computer Vision
Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences
Zhenjia Li, Jinrang Jia and Yifeng Shi
A single panorama captures the full visual sphere from one camera center, yet confines users to looking around in place without enabling true scene exploration. Converting a single panorama into a persistent, renderable 3D representation for free-viewpoint
navigation has attracted growing interest; existing methods either adopt iterative per-view completion that propagates inpainting results to update the underlying geometry, leading to progressive error accumulation and cumbersome multi-step pipelines, or leverage the temporal consistency priors of
video generation models, yet the continuous-trajectory constraint intrinsic to such models limits their flexibility in covering scenes from multiple directions simultaneously. We present Pano2World, which takes a single indoor panorama as input and directly outputs a persistent, explorable 3D Gaussian scene. Given the source panorama, Pano2World first reconstructs a coarse 3D Gaussian proxy and renders it at adaptively sampled nearby poses to obtain geometrically aligned guidance panoramas; a panoramic
diffusion model then jointly denoises all target views via View-Aware
Attention Routing, where each target view simultaneously receives
geometric constraints from its corresponding guidance panorama and global semantic guidance from the source panorama, naturally enforcing cross-view consistency. To avoid the information loss incurred by decoding the multi-view hidden features formed during joint denoising back to the pixel domain via VAE, we introduce Latent Feature Adapter, a geometry-aware bridge module that directly distills these hidden features into a scene latent, subsequently decoded into the final 3D Gaussian scene. Experiments demonstrate that Pano2World significantly outperforms existing methods on the multi-position panoramic novel-view synthesis
benchmark.
Read the full abstract ▾
Computer Vision
Stitched Embeddings: A Unified Latent Space for 3D Garments and 2D Patterns
Andrea Sanchietti, Riccardo Marin, Bharat Lal Bhatnagar et al. (+2 more)
While garments are essential for realistic digital humans, their topological variety makes them much harder to model than parametric bodies. Traditional tailoring relies on 2D sewing patterns, yet bridging these patterns to 3D geometry currently requires physical simulations. We present Stitched
Embeddings, the first simulation-free framework to unify 3D garment reconstruction and sewing pattern inference within a single bidirectional latent space. By leveraging the
geometric priors of a pretrained 3D foundation model, our approach overcomes the data scarcity typically associated with high-quality garment modeling. We propose to use the BoxMesh as a critical intermediate representation to align 2D panels into 3D configurations without the computational overhead of a simulator. This architecture achieves state-of-the-art accuracy in pattern reconstruction while significantly improving efficiency. Furthermore, our differentiable pipeline enables novel applications, including pattern recovery from meshes and 3D editing from 2D patterns. Finally, this work provides a scalable link between neural 3D
vision and the physical garment manufacturing pipeline. Project Page: https://andreus00.github.io/stitchedembeddings
Read the full abstract ▾
All authors Andrea Sanchietti, Riccardo Marin, Bharat Lal Bhatnagar, Yuanlu Xu, Gerard Pons-Moll
Computer Vision
Training-Free Debiasing of Diffusion Models via CLIP-Guided Denoising Optimization
Dain Kim, Jinseo Kim and Sungyong Baik
Text-to-image
diffusion models achieve impressive visual quality, yet demographic bias remains a challenge, as neutral
prompts consistently produce stereotypical representations across gender and race. Existing approaches remain limited by costly retraining or by inference-time interventions that often degrade image quality and semantic
alignment. We propose Text
Embedding Steering (TES), a training-free framework that mitigates demographic bias by directly optimizing conditional text
embeddings during the
diffusion process. We show that a two-stage strategy - early-stage global
alignment followed by iterative denoising-time refinement with CLIP-based feedback - enables stable and controllable attribute steering without modifying model parameters. Extensive experiments on Stable
Diffusion demonstrate that TES outperforms existing training-free baselines in
fairness while maintaining competitive image quality. These results highlight that inference-time text
embedding optimization is a practical and scalable solution for
fairness-aware generation in
diffusion models.
Read the full abstract ▾
Computer Vision
Towards High-Resolution Visual Perception via Hierarchical Entity Exploration
Ziyu Ma, Shidong Yang, Yuxiang Ji et al. (+5 more)
High-resolution (HR) image perception remains a key challenge in
multimodal large
language models (MLLMs), as fine-grained details are often lost when the image is processed as a whole. Existing methods either require training to teach models where to look or heuristically divide the image into fixed regions, both of which struggle to generalize in complex HR scenes. In this work, we propose Hierarchical Entity Exploration (HEE), a training-free and model-agnostic framework that transforms static image understanding into dynamic, query-guided entity exploration. HEE first evaluates each region using a dual scoring mechanism to determine whether it already contains sufficient evidence to answer the question. If not, it applies object detection within the most promising region to extract fine-grained entities, clusters them into coherent subregions, and organizes them into a multi-level semantic hierarchy for deeper exploration. When deeper regions still fail to yield confident answers, a confidence-guided backtracking mechanism revisits alternative paths to ensure adaptive perception. Extensive results show that HEE outperforms training-free methods like ZoomEye and RAP in both accuracy and efficiency on two complex HR
benchmarks (Visual Probe and HR-Bench), across different MLLMs such as Qwen2.5-VL and LLaVA-OneVision. Moreover, HEE demonstrates generalization on the MME-RealWorld
benchmark.
Read the full abstract ▾
All authors Ziyu Ma, Shidong Yang, Yuxiang Ji, Yiming Hu, Tongwen Huang, Yong Wang, Jianfei Cai, Xiangxiang Chu
Computer Vision
Spotted: Location-informed Reidentification of Hyenas and Leopards in Camera Trap Surveys
Halil Sina Kelebek, Julia Hindel, Kobus Hoffman et al. (+9 more)
Animal re-identification (ReID) in camera-trap surveys remains challenging due to low image quality, strong variation in illumination and viewpoint, and highly imbalanced numbers of observations per individual. As a result, current ReID performance is often insufficient for fully automated use, and practical workflows typically depend on expert review of algorithmically proposed candidate matches. Moreover, most existing approaches focus almost exclusively on visual cues and overlook auxiliary information routinely available in field studies, such as image timestamps and camera-trap locations. We introduce Spotted, a location-informed, human-in-the-loop animal ReID framework that integrates visual similarity with spatio-temporal feasibility priors derived from camera locations, thereby reducing the amount of required expert review. Our method (i) computes an image-model-agnostic feasibility score based on the minimum travel speed required for two detections to correspond to the same individual, (ii) uses these feasibility cues as pseudo-supervision to train a lightweight head on top of a frozen visual foundation model, and (iii) fuses adapted visual similarity with spatio-temporal feasibility to obtain a robust pairwise matching score. We additionally integrate an active pair sampling strategy to accelerate annotation by initially prioritizing uncertain predictions. We evaluate Spotted on three challenging camera-trap ReID
datasets comprised of spotted hyenas and leopards, which we release as part of this work. Our model improves average top-5 identification accuracy by 9pp, 2pp and 9pp over the best baseline on our LeopardID102, SpottedHyenaID109 and SpottedHyenaID415
datasets, respectively. Further, we show that our human-in-the-loop strategy reduces the number of queried comparisons by up to 69pp while achieving equivalent positive matches.
Read the full abstract ▾
All authors Halil Sina Kelebek, Julia Hindel, Kobus Hoffman, Lauren Hoffman, Andrew Loveridge, Bob Mandinyenya, Kudakwashe Ncube, Justin Seymour-Smith, Andrea Sibanda, Abhinav Valada, Matthew Wijers, Daniele De Martini
Computer Vision
ClinRAG-GRAPH: Clinical-prior Retrieval-Augmented Graph Model with Domain Adversarial Learning for Breast pCR Prediction
Yaofei Duan, Yuhao Huang, Tianyu Zhang et al. (+12 more)
Neoadjuvant chemotherapy (NAC) response prediction is clinically important for treatment stratification in breast cancer. However, robust pre-treatment pathological complete response (pCR) prediction remains challenging due to insufficient cross-modal modeling, multicenter imaging heterogeneity, and weak evidence-grounded interpretability. We propose ClinRAG-GRAPH, a Clinically informed
Retrieval-Augmented Generation Graph framework, for pre-treatment pCR prediction from DCE-MRI, structured clinical variables, and biopsy-derived pathological biomarkers. ClinRAG-GRAPH constructs an intra-patient clinical-prior graph and applies a prior-guided relation-aware graph convolutional network for structured
multimodal representation learning. To improve cross-center robustness, we introduce a dual-branch domain-
adversarial learning strategy to suppress protocol-related MRI bias while preserving pCR-relevant features. To enhance interpretability, we further incorporate large
language model (LLM)-driven subgraph RAG module that retrieves clinically analogous historical cases and integrates retrieved evidence for pCR inference. We assemble a large-scale multicenter NAC breast cancer cohort for extensive validation, drawing from two public sources and three in-house centers.Results show that ClinRAG-GRAPH achieves AUCs of 0.815 on the internal test set and 0.774/0.712 on two external test sets, demonstrating robust pre-treatment pCR prediction across centers. The code is available at the anonymized https://github.com/miccai26-1181/ClinRAG-GRAPH.
Read the full abstract ▾
All authors Yaofei Duan, Yuhao Huang, Tianyu Zhang, Yuan Gao, Luyi Han, Xin Wang, Xinyu Xie, Xinglong Liang, Chunyao Lu, Muzhen He, Patrick Pang, Yue Sun, Ning Mao, Tao Tan, Ritse Mann
Computer Vision
LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives
Lukas Kuhn, Giuseppe Serra, Randall Balestriero et al. (+1 more)
Vision-
language pretraining remains dominated by contrastive objectives, whereas
vision-only self-supervised learning has largely adopted non-contrastive methods. At the same time, the role of
vision-
language encoders has shifted: they are increasingly deployed not as zero-shot classifiers but as the frozen visual backbone of
vision-
language models and dense prediction systems, which consume the full grid of patch
tokens rather than a single pooled
embedding. We introduce LeVLJEPA, the first fully non-contrastive end-to-end
vision-
language pretraining method. LeVLJEPA learns through cross-modal prediction with stop-gradient targets and per-modality distributional regularization, without negatives, temperature, momentum encoder, or teacher-student schedule, and trains stably at large scale. We find that the resulting encoder provides markedly stronger dense semantic features for downstream use: as a frozen
vision-
language-model backbone, LeVLJEPA is the strongest of the evaluated encoders across GQA, VQAv2, and POPE under two distinct
language models, and outperforms contrastive baselines on semantic
segmentation, while remaining on par on global readouts such as linear probing. These results establish non-contrastive pretraining as an effective means of producing dense semantic
vision features.
Read the full abstract ▾
All authors Lukas Kuhn, Giuseppe Serra, Randall Balestriero, Florian Buettner
Computer Vision
SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference
Kyan Mahajan and Mohammad Saqlain
Most adaptive-inference techniques for foundation models change what the model does - early exit, MoE routing, KV-cache compression, dynamic
attention sparsity. The input that hits the backbone, however, remains a fixed-grid tokenisation indifferent to image content. We argue that this is a missed lever. We present SpiralFovea, a parameter-free, input-adaptive tokeniser in which
token identity, location, scale, and count are all functions of local visual
entropy and selection completes before any backbone parameter is queried. Around content-driven hotspot anchors, multi-scale spiral rings produce <= 78 patches that replace the standard 196-patch ViT grid at the input stage. Across four canonical fine-grained
benchmarks, SpiralFovea yields +1.7-2.1 pp accuracy with a 60% reduction in input
tokens, an 84% reduction in self-
attention FLOPs at every
transformer layer, and 18-29% throughput gains over the matched static tokenisation baseline. A controlled ablation on CUB-200-2011 Genus across four backbones reveals a clean diagnostic: the gain magnitude tracks inversely with the strength of the backbone's whole-image positional prior, isolating self-supervised foundation models as the regime where input-adaptive tokenisation is most valuable.
Read the full abstract ▾
Computer Vision
Soft Mixture-of-Recursions: Going Deeper with Recursive Vision Transformers
Sang In Lee and Jihun Park
Vision Transformers are typically scaled by increasing depth or width, at the cost of more unique parameters. We explore recursion as an alternative scaling axis that increases effective computational depth through parameter reuse. Yet simply increasing recursion depth does not reliably improve performance because representations from different recursion stages are not fully utilized. We propose Soft Mixture-of-Recursions (SoftMoR), which learns
token-wise weights to combine representations across recursion steps, and instantiate it as the Soft Recursive
Vision Transformer (SR-ViT). Experiments on ImageNet-1K, COCO, and ADE20K show that SR-ViT can benefit from deeper recursion while adding few parameters. On ImageNet-1K, increasing recursion depth from T=1 to T=3 improves top-1 accuracy from 79.83% to 82.45% with only 1.1M additional parameters. At matched effective depth, SR-ViT also outperforms independently parameterized counterparts with substantially fewer parameters. These results support recursive deepening as an effective, parameter-efficient scaling axis for
Vision Transformers. The code is available at https://github.com/sangin-lee/sr-vit .
Read the full abstract ▾
Computer Vision
Decoupled Guidance: Disentangling Subject and Context Pathways in Text-to-Image Personalization
Seongmin Kim, Kyucheol Shin, Heesun Jung et al. (+2 more)
Text-to-image personalization aims to generate a user-provided subject in novel scenes described by text. However, most existing methods encode subject identity (fidelity) and context (editability) through the same conditioning pathway, forcing the two to compete for
attention-map resources. We refer to this phenomenon as conditioning
entanglement and show that it induces a fidelity-editability trade-off. We further provide causal evidence by replacing the target subject
token with a generic subject
token, which produces shifts in
attention allocation and corresponding changes in context adherence. To this end, we propose Decoupled Guidance (DeGu), a plug-and-play framework that routes subject identity and scene context through two independent guidance streams. We further introduce a spatial mixing mechanism that dynamically fuses these streams, ensuring each operates within its semantically relevant region without interference. Furthermore, DeGu can be readily applied to existing personalization methods without modifying the underlying backbone models, consistently improving the overall personalization performance while enabling inference-time control over the fidelity-editability balance, across diverse methods and backbones, including flow-matching
Diffusion Transformers (DiTs).
Read the full abstract ▾
All authors Seongmin Kim, Kyucheol Shin, Heesun Jung, Jinseo Kim, Sungyong Baik
Computer Vision
GKDT: General Keypoint Detection Transformer
Changsheng Lu, Yuxin Chen, Haokun Gui et al. (+5 more)
With the emergence of various pre-trained
vision and
language models, computer
vision is shifting from narrow-domain to open-domain recognition. The construction of a more powerful yet general keypoint detection (GKD) model to support diverse tasks has become increasingly important in the field. To this end, we firstly present a large-scale unified keypoint
dataset called MegaKPT. The
dataset is composed of over 1.3 million diverse object instances from twenty-nine existing
datasets, and enjoys high-quality unified annotations with keypoint text descriptions. Based on MegaKPT, we develop GKDT, a simple, flexible and powerful DINOv3 based
Transformer model for General Keypoint Detection. Our GKDT supports visual
prompts, text
prompts, or both. To enhance model training, we also propose a suite of useful strategies such as mix-modal prompted training and dynamic importance sampling. By testing over 22 test sets with seen or unseen objects, our single GKDT model shows strong performance and generality in detecting keypoints on broad categories, with most categories over 90% PCK@0.1 accuracy, offering high practical applicability to real-world problems. The
dataset, models, and codes will be released at https://github.com/AlanLuSun/General-Keypoint-Detection.
Read the full abstract ▾
All authors Changsheng Lu, Yuxin Chen, Haokun Gui, Rong Wang, Jie Yang, Harry Yang, Anton van den Hengel, Jiaya Jia
Computer Vision
FrameONE: Hierarchical Motion Modeling for Universal Multi-View Echocardiographic Keyframe Detection
Rusi Chen, Yuhao Huang, Hongyuan Zhang et al. (+4 more)
Accurate detection of end-systole (ES) and end-diastole (ED) frames is fundamental to echocardiographic assessment. Existing methods are typically developed in a view-specific manner, depend on auxiliary annotations or intensive visual modeling, which limits their generalizability. In multi-view modeling, keyframe detection is driven by shared cardiac motion, yet large appearance differences and motion patterns make unified modeling challenging. To address these issues, we propose FrameONE, a unified end-to-end framework for multi-view echocardiographic keyframe detection. FrameONE introduces a Hierarchical Motion Modeling strategy: an intra-view multi-task learning reduces appearance bias and promotes motion-focused representations within each view; an inter-view general motion learning module further separates view-agnostic dynamics from view-specific patterns, enabling shared yet flexible motion representation learning across views. Extensive experiments on 25,872
videos spanning four standard views demonstrate that FrameONE achieves state-of-the-art keyframe detection accuracy with strong cross-view generalization. Code is available at https://github.com/szuboy/FrameONE.
Read the full abstract ▾
All authors Rusi Chen, Yuhao Huang, Hongyuan Zhang, Chao Tian, Shunan Ji, Yuhan Zhang, Dong Ni
Computer Vision
Active Learning for Cascaded Object Detection: Balancing Coverage and Uncertainty in Table Extraction Pipelines
Eliott Thomas, Mickael Coustaty, Aurelie Joseph et al. (+3 more)
Table extraction from business documents relies on a cascaded pipeline where Table Detection (TD) first localizes tables and Table Structure Recognition (TSR) then recovers their internal layout. Building task-specific training sets for this pipeline is costly, particularly for TSR which requires fine-grained structural annotations. Active learning (AL) can reduce this annotation burden, yet most AL strategies are designed for single-model tasks and do not account for inter-stage dependencies in cascaded architectures. In this work, we present the first adaptation of Uncertainty Herding (UHerding), a hybrid coverage-uncertainty sampling method originally proposed for image classification, to cascaded object detection pipelines. We propose two pipeline-aware extensions that exploit the TD-to-TSR dependency: RankFusion adds dual-
manifold coverage over both detection and structure representation spaces, while CAPA further incorporates stage-dependent gating and per-task uncertainty calibration. Extensive experiments across two public (PubTables-1M and FinTabNet) and two private table extraction
datasets, with various annotation budgets (from 71 to 500 documents) show that UHerding generalizes well to table extraction, outperforming each baseline. Among pipeline-aware variants, RankFusion achieves higher expected gains but at the cost of greater variance, while CAPA emerges as the most consistent strategy, outperforming standard UHerding on three out of four
datasets.
Read the full abstract ▾
All authors Eliott Thomas, Mickael Coustaty, Aurelie Joseph, Gaspar Deloin, Vincent Poulain d'Andecy, Jean-Marc Ogier
Computer Vision
GaussianFusion: Unified 3D Gaussian Representation for Multi-Modal Fusion Perception
Xiao Zhao, Chang Liu, Mingxu Zhu et al. (+5 more)
The bird's-eye view (BEV) representation enables multi-sensor features to be fused within a unified space, serving as the primary approach for achieving comprehensive 3D perception. However, the discrete grid representation of BEV leads to significant detail loss and limits feature
alignment and cross-modal information interaction in
multimodal fusion perception. In this work, we break from the conventional BEV paradigm and propose a new universal framework for multi-modal fusion based on 3D Gaussian representation. This approach naturally unifies multi-modal features within a shared and continuous 3D Gaussian space, effectively preserving edge and fine texture details. To achieve this, we design a novel forward-projection-based multi-modal Gaussian initialization module and a shared cross-modal Gaussian encoder that iteratively updates Gaussian properties based on an
attention mechanism. GaussianFusion is inherently a task-agnostic model, with its unified Gaussian representation naturally supporting various 3D perception tasks. Extensive experiments demonstrate the generality and robustness of GaussianFusion. On the nuScenes
dataset, it outperforms the 3D object detection baseline BEVFusion by 2.6 NDS. Its variant surpasses GaussFormer on 3D semantic occupancy with 1.55 mIoU improvement while using only 30% of the Gaussians and achieving a 450% speedup.
Read the full abstract ▾
All authors Xiao Zhao, Chang Liu, Mingxu Zhu, Zheyuan Zhang, Linna Song, Qingliang Luo, Chufan Guo, Kuifeng Su
Computer Vision
Foundation Model-driven Key Anatomy Frame Selection for Blind-sweep Ultrasound Fetal Birth Weight Estimation
Le Ou, Xiliang Zhu, Huanwen Liang et al. (+8 more)
Accurate fetal birth weight (FBW) estimation shortly before delivery is clinically valuable yet challenging due to its reliance on operator expertise, particularly in low-resource settings. To reduce this reliance, we study near-term birth-weight regression from blind-sweep ultrasound (US)
videos acquired within 48 hours prior to delivery, with post-delivery weighing as ground truth. Accordingly, we propose a foundation model-driven key anatomy frame selection framework that enables accurate FBW regression despite the absence of plane constraints in blind sweeps. Our highlights are as follows: (1) We believe this is the first work to estimate FBW using blind-sweep US
videos, enabling operator-independent assessment. (2) An Anatomy-Guided Frame Selection module equipped with a
vision-
language foundation model is proposed for keyframe collection in unconstrained sweeps. (3) A Redundancy-Aware Feature Compression module is designed to compress frame features while preserving task-relevant information, alleviating temporal redundancy. Extensively validated on prospectively collected data from 839 patients, our method achieves an MAE of 161.3 g, with 90.23% and 100% of cases falling within 10% and 15% absolute percentage error, outperforming typical Hadlock estimation and strong competitors. Codes are available at https://github.com/ouleoule/BlindSweep-EBW.
Read the full abstract ▾
All authors Le Ou, Xiliang Zhu, Huanwen Liang, Wenxiong Pan, Yuhao Huang, Yuxiang Deng, Xuan Sheng, Hong Yin, Juhua Xiao, Xin Zhou, Dong Ni
Computer Vision
Prototype Memory-Guided Training-Free Anomaly Classification and Localization in Prenatal Ultrasound
Huanwen Liang, Yuhao Huang, Xiliang Zhu et al. (+6 more)
Prenatal anomaly classification and localization is of critical importance for fetal health and pregnancy management. Although ultrasound (US) is the primary modality for prenatal screening, accurate diagnosis remains challenging due to the low prevalence and high heterogeneity of anomalies. Existing deep learning methods for prenatal tasks rely on large-scale annotated
datasets, which are difficult to obtain in practice. Although few-shot learning alleviates data scarcity, it typically requires fine-tuning for new categories, limiting its practicality in resource-limited clinical settings. To address these challenges, we propose a training-free framework for multi-class prenatal US anomaly classification and localization that operates with only a few reference images per class, representing the first exploration of this setting. Our framework comprises three key components: (1) a
memory bank with multi-granular prototypes that explicitly models both class-level semantics and anomaly characteristics; (2) a prototype-driven soft merging mechanism that aggregates discriminative features to detect the anomaly region; and (3) a class-aware refinement strategy that leverages prototype consistency to improve category prediction. Extensively validated on a multi-center prenatal US
dataset containing 1,149 cases, with a total of 2,357 images and 9 categories, our proposed method outperforms the competitors.
Read the full abstract ▾
All authors Huanwen Liang, Yuhao Huang, Xiliang Zhu, Yuanji Zhang, Xuedong Deng, Xinru Gao, Guowei Tao, Yuhan Zhang, Dong Ni
Computer Vision
Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation
Zhaowen Zhu, Li Zhang, Yujie Chen et al. (+3 more)
Self-Supervised
Monocular Depth Estimation (MDE) has garnered
attention in recent years due to its independence from ground truth. However, most existing models are limited to a single scale and exhibit considerable performance degradation in complex driving environments. Networks specifically designed to handle dynamic traffic participants tend to be overly complex, hindering their deployment on resource-constrained automotive edge devices. To address these limitations and move towards robust driving perception, we propose FlexDepth, a scale-driven and flexible family of self-supervised MDE models tailored for challenging road scenarios. FlexDepth employs a two-stage static-dynamic decoupled training strategy, enabling the independent assessment of confidence for both static backgrounds and dynamic road objects. Furthermore, it introduces a meticulously designed Scale-Driven Decoder (SDD) to dynamically select components based on scale size, facilitating efficient feature fusion and the output of high-precision depth maps. Extensive experiments on standard driving
benchmarks demonstrate that without any auxiliary information, our model achieves state-of-the-art performance across arbitrary scales with minimal computational overhead. Our smallest model, Flex-Nano, requires only 0.7 GFLOPs and achieves 37.6 FPS on mobile platforms, ensuring reliable real-time perception while maintaining excellent zero-shot generalization. Our source code is available: https://github.com/startnew/flexdepth
Read the full abstract ▾
All authors Zhaowen Zhu, Li Zhang, Yujie Chen, Tian Zhang, Yingjie Wang, Mingxia Zhan
Computer Vision
ConRTF: Edge-Constrained Boundary Distribution Refinement for Realtime TransFormer Table Structure Recognition
Eliott Thomas, Tri-Cong Pham, Mickael Coustaty et al. (+5 more)
Table Structure Recognition (TSR) aims to recover the row and column layout of tables from document images, a key step in document understanding pipelines. Accurate TSR depends on precise boundary localization: small errors in row or column boundaries can propagate into incorrect
cell assignments and structural inconsistencies. Yet detection-based approaches treat table elements as generic objects, ignoring a fundamental property of table layout: rows and columns play structurally distinct roles and their boundaries carry unequal importance. We propose an Edge-constrained Fine-grained Localization loss (EFL) that formalizes this structural asymmetry by encoding table-specific
geometric priors into the training objective: row-like elements are supervised with emphasis on their horizontal boundaries, while column-like elements prioritize vertical boundaries. Implemented within a real-time detector with distribution-based boundary refinement (D-FINE), EFL operates during training only and guides boundary refinement toward structurally meaningful adjustments with no change to the inference pipeline. The proposed approach, ConRTF, is also data-efficient, maintaining robust accuracy with as few as 2k–3k annotated tables. Experiments on PubTables-1M and two private
datasets show consistent improvements over the optimized baseline and several real-time detectors including RT-DETRv2 and YOLOv10-11, with gains of up to +1.6 GriTS points at equal inference speed.
Read the full abstract ▾
All authors Eliott Thomas, Tri-Cong Pham, Mickael Coustaty, Aurelie Joseph, Gaspar Deloin, Vincent Poulain d'Andecy, Jean-Marc Ogier, Antoine Doucet
Computer Vision
AV-SyncBench: Decoupled Benchmarking of Temporal and Semantic Audio-Visual Synchronization
Tianhong Zhou, Mingyang Han, Boyu Li et al. (+8 more)
Audio-visual feature extraction is a fundamental component of
multimodal understanding and generation tasks. However, existing evaluation protocols for feature extraction models exhibit dimensional bias, typically focusing on either semantic matching or temporal offset detection. Moreover, their data construction remains coupled, preventing independent assessment of temporal and semantic consistency. We propose AV-SyncBench, the first
benchmark to fully separate temporal and semantic evaluation for audio-visual synchronization. Built from in-the-wild
videos, it spans Voice, Music, and Sound across 10 scenarios and 5 challenge tasks. Data are automatically filtered and manually verified to ensure on-screen sound sources. The
benchmark contains 3,269
videos and 38,390 samples, and we evaluate five representative models to quantify feature quality for
alignment and downstream tasks. The code and
dataset are available at: https://fgt7t6g.github.io/AV-SyncBench.
Read the full abstract ▾
All authors Tianhong Zhou, Mingyang Han, Boyu Li, Yuxuan Jiang, Jiaxin Ye, Dongxiao Wang, Haoxiang Shi, Kunpeng Wang, Jun Song, Cheng Yu, Bo Zheng
Computer Vision
Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach
Yingjie Dai, Tianyang Xu, Yanglin Deng et al. (+2 more)
Skeleton-based action recognition has achieved remarkable success by exploiting joint coordinates and their topological connections, yet prevailing methods overwhelmingly assume complete and clean skeleton inputs. In real-world deployments, such as
egocentric vision, crowded surveillance, wearable devices, or edge robotics, limited field-of-view (FoV) frequently causes substantial joint visibility dropout, leading to severe performance degradation that existing models are largely unprepared to handle. To bridge this critical yet underexplored gap, we introduce PartialVisGraph, a novel hypergraph framework tailored for robust skeleton action recognition under constrained FoV. We first construct highly expressive hypergraphs by introducing learnable virtual hyperedges that form a soft incidence
matrix, capturing flexible high-order dependencies beyond conventional pairwise graphs. We then propose the Single-Head Sample-Adaptive
Transformer, which adaptively aggregates joint features onto hyperedges while explicitly incorporating a visibility prior. This prior selectively gates information flow, preventing occluded or out-of-view joints from corrupting reliable feature propagation. We further establish rigorous evaluation protocols with realistic FoV simulation
benchmarks on NTU RGB+D 60 and 120. Extensive experiments demonstrate that PartialVisGraph consistently achieves state-of-the-art accuracy under partial visibility, with gains of up to 68.8% on subsets with severe FoV restrictions compared to recent strong baselines, while remaining superior on full-visibility settings. Our approach offers a principled and practical pathway toward deployable skeleton-based action understanding in unconstrained environments.
Read the full abstract ▾
All authors Yingjie Dai, Tianyang Xu, Yanglin Deng, Xiao-Jun Wu, Josef Kittler
Computer Vision
Towards Memory-Efficient Autoregressive Video Generation via Instance-Specific Parametric Absorption
Xiaomeng Fu, Jia Li, Yiming Hu et al. (+5 more)
Autoregressive (AR) streaming models have emerged as a powerful paradigm for long
video generation. However, the linearly growing Key-Value (KV) cache poses a significant bottleneck, leading to
memory overload and degraded inference throughput. A common compression method is to drop redundant KV
tokens, which often breaks long-range dependencies, resulting in temporal flickering and identity loss. In this paper, we propose Instance-Specific Parametric Absorption (ISPA), a novel framework that shifts the KV cache compression from discarding to distilling. The core idea is to transit a subset of layers from Full-
Attention (F-Layers) to
memory-efficient Local-
Attention (L-Layers) by "absorbing" historical context into the model's weights. Specifically, during a brief warmup phase, ISPA monitors the output discrepancy between global and local
attention. At the transition point, we solve a closed-form least-squares problem to compute an instance-specific weight modulation that compensates for the missing history. Experiments across architectures (1.3B to 14B) demonstrate that ISPA can remove up to 50% of the KV cache with near-lossless visual quality. We hope this perspective encourages future work to explore parametric
memory consolidation beyond external
token-level cache management for streaming generative models.
Read the full abstract ▾
All authors Xiaomeng Fu, Jia Li, Yiming Hu, Yong Wang, Hayden Kwok-Hay So, Jiao Dai, Xiangxiang Chu, Jizhong Han