Computer Vision
A Real-World CAD Segmentation Benchmark: Comparing Rule-Based and Learning-Based Approaches through Hole Detection
Kai Schütte, Edgar Heinert and Matthias Rottmann
CAD segmentation
datasets are predominantly synthetic or automatically labelled, leaving it unclear how well existing methods transfer to real-world engineering tasks. Fastener-relevant hole detection represents one such task, requiring human annotation and exhibiting greater
geometric complexity than commonly used
benchmarks. We address this gap through three main contributions. First, we develop two task-specific rule-based methods, one graph-wire-search and one multi-view based, and complement them with a human-annotated
benchmark dataset derived from engineering CAD data that also enables training of UV-Net and AAGNet, representative learning-based approaches for B-Rep segmentation. Second, we conduct an extensive experimental study including parameter optimisation, evaluation on our
benchmark dataset, cross-domain generalisation on a synthetic
dataset, and an assessment of the feasibility of the approaches at varying
dataset sizes and IoU thresholds. Third, we provide evidence-based insights into the strengths and limitations of different approaches: rule-based methods generalise well without training data, with the graph-wire-search method favouring recall and the multi-view method favouring precision, whereas learning-based methods achieve strong performance with sufficient annotations but exhibit reduced robustness under domain shift. These findings highlight trade-offs between data requirements, and robustness, and provide guidance not only for hole detection, but more broadly for real-world CAD segmentation tasks.
Read the full abstract ▾
Computer Vision
When MLLMs miss what humans read: diagnosing and learning to read squint text
Chen Peng, Shan Huang, Jianbo Huang et al. (+3 more)
Multimodal LLMs can fail to read text embedded in an image's coarse structure (text a human recovers by squinting) and report that no text is present. We name this class Squint Text and study it as a recognition deficiency, motivated by cases from a production moderation pipeline. We introduce SquintBench, comprising 1076 production-derived moderation images, and SquintCtrl, a controlled synthetic suite for mechanism diagnosis. Frozen-weight analysis shows that hidden text leaves a coarse signal while glyph identity is weakly represented; identity becomes substantially more accessible when the correct spatial regions are supplied, implicating spatial selection as a major bottleneck. Suppressing high-frequency content also improves reading at a fixed visual-
token budget, while visual
token count may also contribute. These findings motivate Squint OPSD: a frozen teacher receives a downscaled view and the ground-truth transcription as privileged context, and its
token-level distributions supervise a student along on-policy trajectories. At inference, the student reads the original image alone. Squint OPSD reaches 88.61 ANLS on SquintBench, versus 26.50 for the unadapted backbone, 24.44 for the best general-purpose model on the original view, and 68.60 at best for an off-the-shelf model under a downscaled view. These results substantially narrow the gap between available visual evidence and its generated read-out without auxiliary views at inference.
Read the full abstract ▾
All authors Chen Peng, Shan Huang, Jianbo Huang, Linrui Qin, Qibo Sun, Mengwei Xu
Computer Vision
SADL: A Size-Adaptive Decision Layer for Chronic Stroke Lesion Segmentation in ISLES 2026
Marek Wodzinski, Gniewosz Drwiega and Wojciech Szymanski
Chronic ischemic stroke lesions are difficult to segment on T1-weighted MRI. Their appearance overlaps with healthy tissue, and their volumes span four orders of magnitude, so voxel-level and lesion level metrics often disagree. We build an eleven-member nnU-Net ensemble that covers five design axes: a baseline dice + top-K loss, heavy augmentation, per-lesion volume loss re-weighting,
stochastic weight averaging, and a sampling-and-architecture block that pairs cohort-balanced sampling with a residual-encoder backbone. Weighted soft-map fusion of the members feeds a tuned decision layer, SADL, that estimates lesion volume, applies an adaptive per-lesion-size threshold, prunes components by voxel count and peak probability, and adds a never-empty rescue. All fusion weights and per-bucket parameters are fitted on out-of-fold (OOF) predictions, so tuning never touches the held-out test set. On a site-disjoint 91-case holdout, SADL reaches Dice 0.7322, lesion-F1 0.7461 and PR-AUC 0.8430 when scored online with the challenge's released evaluation code. A paired non-parametric bootstrap against a reference single ensemble member confirms the lesion-F1 gain at p=0.024, and the gain is largest in the small-lesion regime, where the rescue step recovers cases that a fixed threshold would silently discard.
Read the full abstract ▾
Computer Vision
TextVBench: Evaluating Text Rendering and Preservation in Video Generation
Ruiqiang Zhang, Hengyi Wang, Chang Liu et al. (+3 more)
Modern
video generators can synthesize convincing scenes while rendering a few characters incorrectly, omitting requested text, or changing it across frames. Measuring these failures is harder than applying OCR independently to each frame: when a scene contains multiple text targets and multiple noisy OCR regions, the evaluator must first determine which observation belongs to which intended text. We introduce TextVBench, a
benchmark and evaluation framework that formulates visual-text evaluation as a multi-target, multi-instance spatiotemporal correspondence problem. TextVBench contains two complementary tracks: TextVBench-T2V measures the creation and maintenance of
prompt-specified text, while TextVBench-Ref2V measures preservation of text from reference images. Both tracks use target-conditioned candidate selection rather than unconstrained VLM transcription, support one target mapping to multiple OCR regions, and separate frame-level spatial fidelity from cross-frame temporal fidelity. A length-weighted harmonic score summarizes these factors, while extra-text diagnostics expose failure modes hidden by a single OCR score. The
benchmark is organized around controlled text, layout, carrier, and motion attributes so that aggregate scores can be traced back to specific capability limits. Across the evaluated model runs, MiniMax-H3 reaches a T2V TextVScore of 68.32, while the strongest Ref2V score is 56.51; attribute slices reveal a pronounced drop as the number of simultaneous reference text regions increases.
Read the full abstract ▾
All authors Ruiqiang Zhang, Hengyi Wang, Chang Liu, Guanjie Wang, Zehua Ma, Weiming Zhang
Computer Vision · Cross-listed from eess.IV
Radiomics-Based Thyroid Nodule Classification with Label-Conditional Conformal Prediction
Laura Testa, Pierpaolo Brutti and Umberto Ferraro Petrillo
Thyroid nodule malignancy assessment relies on ultrasound imaging, yet current risk stratification systems suffer from substantial interobserver variability, contributing to a high rate of unnecessary biopsies. We propose a radiomics-based framework combining quantitative feature extraction via PyRadiomics with a Generalized Additive Model (GAM) classifier, selected among six candidates for its favorable trade-off between discriminative performance and interpretability. To complement point predictions with uncertainty estimates, Cross-Conformal Prediction with label-conditional guarantees is applied, flagging ambiguous cases for clinical review rather than forcing a binary decision. Experiments are conducted on TN5000, a publicly available ultrasound
dataset of thyroid nodules. After Spearman correlation filtering, 36 features are retained and the GAM achieves an AUC of $0.89$, a sensitivity of $0.83$, and a specificity of $0.82$ on the validation set. At a significance level $\epsilon=0.09$, selected to ensure empirical coverage guarantees on both classes, 63% of nodules on the test set receive a certain (i.e., single-label) prediction, with conditional precision of $0.94$ for certain malignant and conditional Negative Predictive Value (NPV) of $0.73$ for certain benign cases, while the remaining 37% are referred to the clinician.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.SP
ECGWmark: Deep Watermarking for Electrocardiogram Signals
Kuan-Yu Chen, Kuan-Chen Wang, Yu Tsao et al. (+1 more)
Secure transmission of electrocardiogram (ECG) signals remains a major challenge in telecardiology. While traditional mathematical watermarking can embed patient metadata directly into ECG signals, it is highly fragile against temporal desynchronization such as circular phase shifts. In contrast, existing deep watermarking approaches for temporal signals, such as audio, typically rely on lossy quantization, which compromises the high-frequency micro-morphological details critical for clinical diagnosis. To bridge this gap, we propose ECGWmark, a fully convolutional and quantization-free deep watermarking framework for medical time-series. ECGWmark operates in a continuous, full-resolution latent space and utilizes a localized frame-level mechanism to improve the robustness to severe desynchronization. Crucially, we introduce a differentiable Physiological Loss guided by a clinical classifier to preserve diagnostic semantics. Experiments on the
benchmark MIT-BIH database show that ECGWmark maintains channel-bit extraction accuracy above 90% under the evaluated circular temporal shifts. Furthermore, under a rigorous inter-patient diagnostic evaluation, Class-V sensitivity changes from 71.97% to 76.50%, whereas Macro-F1 changes from 35.25% to 34.71% and Class-S sensitivity from 8.89% to 1.39%; these results indicate mixed class-level effects.
Read the full abstract ▾
All authors Kuan-Yu Chen, Kuan-Chen Wang, Yu Tsao, Jian-Jiun Ding
Computer Vision
Why DiT Models Underperform as Representation Learners without Long Skip Connections
Benyuan Meng, Yiliang Zhang, Jin-Wen Wu et al. (+3 more)
Diffusion feature, i.e., intermediate activations extracted from
diffusion model backbones, has emerged as a promising approach for unifying generation and
vision understanding (discrimination). Despite the advancement of
diffusion backbones into
Diffusion Transformers (DiT), it has been found in previous literature that DiT models tend to yield weaker features despite their stronger generation capability. This poses a threat to good generation-understanding unification. In this work, we show via controlled from-scratch training that long skip connections (LSCs) affect feature quality: a DiT with LSCs can produce better features. Furthermore, through detailed diagnosis, we reveal the mechanism causing this effect. Specifically, the input to a
diffusion backbone contains both the latent image and the
diffusion noise, where the
diffusion noise, though appearing in the form of noises, actually reflects the information of the
diffusion trajectory. To make the final prediction of a noisy variable ($v$ or $\epsilon$), the
diffusion backbone needs both signals. However, carrying both signals through the main backbone effectively "pollutes" the image signal with noises, leading to poor feature quality. In comparison, LSCs can shortcut
diffusion noises directly to where they are needed, leaving much weaker noise signals in the main backbone, thus improving feature quality. To name the broader impact, this reveals that LSCs are not a mere trick but closely related to the
diffusion objective. The code is at https://github.com/Darkbblue/dit-lsc.
Read the full abstract ▾
All authors Benyuan Meng, Yiliang Zhang, Jin-Wen Wu, Qianqian Xu, Qingming Huang, Longtao Huang
Computer Vision
VideoAWR: Reinforcing Video Flow Models with Decoupled-Timestep Advantage-Weighted Rectification
Zheming Fu, Wenbin Ning, Ruizhe He et al. (+6 more)
Regression-based RL for flow models mandates sampling without classifier-free guidance (CFG) to align the rectification target with the reference marginal velocity. Extending Advantage-Weighted Rectification (AWR), which dynamically scales this regression target based on reward, to
video generation encounters a cold-start pathology: unguided rollouts collapse into low-fidelity regions where intra-group reward gaps remain wide but no longer track perceptual quality, rendering the optimization target arbitrary. Conversely, utilizing guided rollouts to restore fidelity, while keeping the regression target tied to the guided sampling distribution, provably triggers a
geometric guidance blow-up under exponential-moving-average (EMA) updates. We propose VideoAWR, which resolves this tension by untying sampling from the optimization anchor. An EMA reference policy generates guided rollouts to restore reward separability, while the trainable policy is rectified against the unguided conditional anchor. We prove this decoupled objective is mathematically exact: the centered advantage cancels the anchor in expectation, filtering out the reward-neutral guidance offset and retaining only the reward-correlated covariance increment. Consequently, inference runs single-branch and CFG-free, yet inference-time CFG preserves its original semantics. Furthermore, we introduce Decoupled Timestep Rectification, supervising each rollout at an independently sampled timestep without bias to improve sample efficiency. Experiments on Wan2.1-1.3B and 14B demonstrate stable optimization and superior visual and motion quality in both CFG-free and CFG-enabled regimes. Code is available at \pdfstartlink attr{/Border [0 0 0]} user{/Subtype /Link /A << /S /URI /URI (https://github.com/Fuzm7/FlowAWR) >>}% \textcolor{pearDark}{VideoAWR}\pdfendlink.
Read the full abstract ▾
All authors Zheming Fu, Wenbin Ning, Ruizhe He, Wei Shang, Xiaoxiao Ma, Lianrui Mu, Chang Liu, Lei Wang, Siming Fu
Computer Vision
Exploring Attention Sinks in Text-to-Image Transformers from Diffusion and Autoregressive Models
Binglei Li, Mengping Yang, Zhiyu Tan et al. (+3 more)
Attention sinks are
tokens or positions that consistently receive disproportionate
attention mass, often beyond what their semantic content would suggest. Despite extensive study in large
language models, their behavior and functional relevance in text-to-image (T2I)
Transformers remain largely unexplored. To fill this gap and better understand how text-to-image
Transformers allocate and route
attention, we explore this phenomenon of text-to-image
transformers across bidirectional
diffusion models and causal autoregressive models. Specifically, we explore whether
attention sinks exist, where they emerge, and how they functionally influence generation. Through systematic
attention analysis and targeted interventions, we identify that
attention sinks emerge in both bidirectional
diffusion and autoregressive models but exhibit distinct patterns. The autoregressive model concentrates
attention on the first and global-scale
tokens, whereas the bidirectional models develop sinks near modality boundaries, within text regions, and at spatial corners. Moreover, these patterns vary across generation stages and are localized to a sparse subset of layers and heads. To assess their functional relevance, we perturb sink-related heads and
token positions. It turns out that perturbing sink-ranked heads produces larger average changes than non-sink comparisons, while some sink-position perturbations have limited effects on bidirectional model scores. We further introduce register
tokens to redistribute
attention sinks and find that several configurations preserve or modestly improve generation performance. Overall,
attention sinks are architecture-, inference-stage-, and component-dependent phenomena, providing a basis for mechanistic analysis and principled
attention design in text-to-image
transformers.
Read the full abstract ▾
All authors Binglei Li, Mengping Yang, Zhiyu Tan, Wenxu Qian, Junping Zhang, Hao Li
Computer Vision
LETHE: Defending against Unauthorized Image-to-Video Generation via Value Diversity Collapse
Boheng Li, Yueming Xing, Yiming Li et al. (+2 more)
Image-to-
video (I2V)
diffusion models can animate a single reference image into high-quality, subject-consistent
videos, posing threats of misuse as anyone can turn personal photos into fabricated
videos without consent. A promising solution is to proactively add an
adversarial perturbation to the image before release, such that
videos generated from it would be unfaithful or visually corrupted. However, due to an insufficient understanding of what governs how subsequent frames read the reference image in modern I2V models, existing protections struggle to offer strong protection under common perturbation budgets. In this paper, we approach the problem by exploring this underlying reading mechanism. Through a convex
geometric analysis, we theoretically identify the spread of the reference value vectors as a key factor governing this reading process, and empirically find that even a mild contraction of this spread can severely degrade the generated
video, making it an efficient lever for protection. Guided by this insight, we propose LETHE, a novel protection method that directly disrupts this reading process. LETHE pairs two complementary objectives, where a value collapse objective drives the reference value vectors toward uniformity, while an
attention attraction objective draws subsequent frames toward these collapsed
tokens to amplify corruption. We further construct two human-centric I2V protection
benchmarks and conduct extensive experiments with multiple modern I2V models, which demonstrate the state-of-the-art protection performance and versatility of LETHE.
Read the full abstract ▾
All authors Boheng Li, Yueming Xing, Yiming Li, Jie Zhang, Tianwei Zhang
Computer Vision
CoVA: Composing VAE Representations for Generative Learning
Zhaowei Chen, Yikai Ge, Linze Li et al. (+2 more)
Representation
alignment with pretrained visual encoders improves
diffusion transformer training but requires auxiliary encoders and extra training overhead. Recent approaches instead exploit representations from the generative model's own variational autoencoder (VAE). However, reconstruction-oriented VAE features are not optimized for the semantic discrimination and spatial structure needed for generative learning. We introduce CoVA (Composed VAE Representations), a lightweight framework that composes VAE features online into stage-wise guidance for flow-matching training. A small latent composer is jointly trained with class supervision, masked latent reconstruction, and horizontal-flip consistency, yielding semantically discriminative, spatially structured dense features and sample-dependent class posteriors. CoVA aligns intermediate SiT features with the composed representations early in training, then removes the increasingly detrimental
alignment and reuses detached composer posteriors for sample-dependent soft label conditioning. We further uncover a hidden confound in soft label conditioning: averaging class
embeddings attenuates their magnitude, coupling semantic interpolation with conditioning strength. This attenuation alone can substantially improve FID while degrading Recall, revealing a non-semantic source of the apparent gain. CoVA corrects this confound by restoring the hard-label magnitude, isolating posterior-dependent directional semantics. CoVA adds only 8.5M training-only auxiliary parameters (1.3% of SiT-XL/2) and introduces no inference-time overhead. On ImageNet 256$\times$256, CoVA reduces SiT-XL/2 FID from 17.2 to 7.4 (57%) in 400K steps. With classifier-free guidance, it achieves an FID of 2.03 in only 400 epochs, versus 2.06 after 1,400 epochs for vanilla SiT-XL/2.
Read the full abstract ▾
All authors Zhaowei Chen, Yikai Ge, Linze Li, Yao Tang, Xiang Li
Computer Vision
ThinkDrag: Semantic Drag-Based Image Editing with Visual Reasoning
Armando Fortes, Tianyi Wei, Chindanai Trakantannarong et al. (+2 more)
Drag-based image editing provides an intuitive interface for spatially controlled image manipulation by allowing users to specify handle and target points. Existing methods have made substantial progress through optimization-based, one-step, and feed-forward formulations, but they often interpret drag instructions as
geometric displacements. This limits their effectiveness when the desired edit depends on interpreting image content and inferring a plausible semantic transformation from sparse point constraints. We introduce ThinkDrag, a unified
multimodal framework for drag-based image editing with visual
reasoning. ThinkDrag is trained to associate point constraints with meaningful object- and scene-level transformations, and can optionally produce an explicit
reasoning trace that interprets the intended edit before image generation. This
reasoning trace improves interpretability, particularly for ambiguous drags. Drag Endpoint Markers, motivated by an analysis of cross-image
attention, provide an explicit correspondence signal that grounds each drag in image space. To support this framework, we construct a supervised
dataset of semantic drag transformations paired with
reasoning traces and introduce DragBench++, a
benchmark targeting challenging drag-based editing scenarios with reference edit solutions. Experiments show that ThinkDrag achieves state-of-the-art performance, improving generation quality and plausibility while maintaining competitive point-following precision.
Read the full abstract ▾
All authors Armando Fortes, Tianyi Wei, Chindanai Trakantannarong, Shangchen Zhou, Xingang Pan
Computer Vision
NovaSplat: Model-Free 6D Object Pose Estimation via 3D Gaussian Splatting Onboarding and Refinement
Georgios Floros, Evangelos Sartinas, Orestis Vangelis et al. (+1 more)
6D pose estimation of novel objects without CAD models remains an open challenge: model-free methods must build the object representation from scratch during onboarding and have long trailed their model-based counterparts. We present NovaSplat, a model-free pipeline that closes this gap. From a short smartphone
video captured at onboarding, we train a 3D Gaussian Splatting (3DGS) model that serves both as a template source for detection and coarse pose estimation and as a differentiable backbone for pose refinement. Our refiner optimizes the object-to-camera pose with combined photometric and
geometric losses under a continuous rotation parametrization free of gradient singularities. On a household-object
benchmark, NovaSplat surpasses all model-based methods. On a challenging industrial
benchmark with textureless and specular surfaces, it outperforms all model-free competitors while substantially narrowing the gap to model-based ones. We further introduce NovaSplat-Lite, an inference variant that is $3.1\times$ faster at negligible accuracy cost. Code will be made available at \href{https://github.com/POSE-Lab/novasplat}{https://github.com/POSE-Lab/novasplat}.
Read the full abstract ▾
All authors Georgios Floros, Evangelos Sartinas, Orestis Vangelis, Maria Pateraki
Computer Vision
PAGE: Parts-Aware Guidance for Co-Speech Gesture Portrait Video Generation
Yabiao Wang, Jiafu Wu, Yicheng Xu et al. (+7 more)
Co-
speech gesture
video generation aims to synthesize realistic, audio-driven portrait
videos from a single reference image and a
speech signal. Despite advances in visual quality,
diffusion-based approaches still suffer from degraded local details and imperfect audio-visual
alignment. We present PAGE (Parts-Aware GuidancE), a general, backbone-agnostic optimization framework for
diffusion-based generation that enhances local spatial fidelity and global audio-visual consistency at both the training-objective and denoising-trajectory levels. During training, we propose Parts-Specific Reweighting (PSR), which adaptively increases supervision in semantically important regions such as lips, eyes, and hands, improving visual fidelity in perceptually critical areas. During inference, we introduce Consistency Classifier Guidance (CCG), an audio-conditioned latent classifier that identifies and corrects audio-visual inconsistencies in the generated
video. We further design Differential Guidance (DG), an efficient inference strategy that refines facial and body motion separately while avoiding cross-region interference. To enable large-scale evaluation, we construct the Chinese Announcer
Speech (CAS)
dataset, the first comprehensive Chinese
benchmark for co-
speech gesture portrait
video generation. Compared with the widely used PATS
dataset, CAS includes substantially more identities,
video clips, and scene diversity. Extensive experiments on both PATS and CAS demonstrate that PAGE achieves state-of-the-art performance in perceptual quality, gesture diversity, and audio-visual
alignment. Project page: https://aigc-explorer.github.io/PAGE/.
Read the full abstract ▾
All authors Yabiao Wang, Jiafu Wu, Yicheng Xu, Teng Hu, Shengzhe Zhou, Yanxiao Sun, Jiangning Zhang, Ying Tai, Yong Liu, Ming-Hsuan Yang
Computer Vision
Sonic-AMG: Streaming Human Motion Generation from Spatial Audio
Jingcheng Ni, Zekun Li, Zhiyang Dou et al. (+6 more)
Modeling human reactions to spatial audio requires understanding both what is heard and where it comes from. Prior work generates motion offline from complete acoustic and spatial input sequences. Humans, however, react to sounds without knowing what they will hear next. We present Sonic-AMG, to our knowledge the first framework for streaming human motion generation from spatial audio. The model encodes acoustic content and body-relative source geometry as separate
tokens. Conditioned on these
tokens, a causal
transformer with a flow head generates continuous motion latents, from which a causal decoder reconstructs motion segments. Standard teacher-forced training derives motion histories and body-relative source locations from ground-truth motion, whereas streaming inference relies on generated motion for both, resulting in a training-inference mismatch. To reduce this history discrepancy, we introduce rehearsal training, which combines full-sequence teacher-forced supervision with supervised prediction along short autoregressive self-rollouts. To keep spatial conditioning consistent with generated motion, we propose spatial feedback, which recomputes body-relative source locations from generated root poses during both rehearsal training and inference. On SAM, Sonic-AMG achieves state-of-the-art FID and R-precision. It further supports real-time online generation of coherent reactions across sound events and source locations. Project page: https://jingchengni.com/sonic-amg/.
Read the full abstract ▾
All authors Jingcheng Ni, Zekun Li, Zhiyang Dou, Xiaoyan Cong, Ziyao Zeng, Hongyang Du, Randall Balestriero, Taku Komura, Srinath Sridhar
Computer Vision
Decoding Register Tokens Preserves Global Scene Details
David Jacob Drexlin, Julius Hense and Jonas Dippel
Pretrained
vision transformers learn structured image representations spanning spatial patch
tokens and non-spatial
tokens. Recent analyses suggest that non-spatial register
tokens encode global scene properties such as illumination, blur, and style. Yet patch-based image decoders discard these outputs, leaving the decoder to recover image appearance from patches alone. We investigate whether retaining register features improves decoding when the complete spatial patch representation is already available. We study this question using representation autoencoders (RAEs) with frozen register-equipped encoders. Our decoder processes four register
tokens together with all patch
tokens through joint self-
attention. Appearance probes, paired reconstruction-error analysis, and fixed-decoder register substitutions examine which global image properties are accessible from registers and how their values affect reconstructed pixels. For generation, we train a multi-head
attention pooling (MHAP) predictor to recover register features from sampled patches, keeping the pretrained
diffusion model and its samples unchanged. On ImageNet-1K, our ViT-B decoders achieve mean reconstruction FIDs of 0.472 with DINOv2 and 0.442 with DINOv3, reductions of 9.5% and 5.2% relative to their respective patch-only baselines. The corresponding PSNRs are 19.73 and 20.54 dB. For generation, our DINOv2 models achieve a mean FID of 1.641, 9.8% below the patch-only baseline on identical sampled patches.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.DC
Multi-Source and Multi-Dimensional Information Fusion in Resource-Constrained Edge Networks
Yushi Wang, Zhengxin Yu, Guhan Zheng et al. (+2 more)
Multi-source semantic information fusion enables edge networks to support global inference using observations collected by distributed terminals. However, limited communication and computation resources restrict semantic uploading, leaving the edge server with incomplete
tokens that differ in modality, semantic granularity, arrival time, and channel condition. The value of these
tokens also depends on the complementary or redundant information provided by other terminals. In this paper, we propose a multi-source and multi-dimensional semantic information fusion framework that coordinates terminal uploading and edge-side fusion under resource constraints. We develop Hyper-Slot to combine semantic content with available
token attributes through iterative
token-to-slot assignment and slot-to-
token feedback, producing a compact representation for global inference. To encourage useful semantic uploading, we further design a contribution-based incentive mechanism that uses Hyper-Slot assignments to evaluate information coverage and cross-type complementarity. A bi-level Stackelberg-Nash game determines the server reward pool and terminal transmission-quality strategies while accounting for computation and communication energy costs. Experiments on a tri-modal extension of COCO2014 and image-text
retrieval on Flickr30K show that Hyper-Slot improves mAP by 3.66 percentage points and RSum by 15.9 points over the strongest evaluated baselines, respectively, while reducing the number of parameters by more than 80% compared with a naive concatenation-based fusion baseline.
Read the full abstract ▾
All authors Yushi Wang, Zhengxin Yu, Guhan Zheng, Wang Miao, Merouane Debbah
Computer Vision
QCLR: A Plug-in Robustification Module via Quantization for Contrastive Learning in Self-Supervised Skeleton-Based Human Action Recognition
Nanzhu Chen, Feixiang He, Yunfeng Diao et al. (+2 more)
Skeleton-based human action recognition has gained increasing
attention due to the stability of skeleton data across diverse environments and advances in motion capture technologies. Following the success of supervised learning approaches, self-supervised learning approaches, particularly contrastive learning ones, have achieved impressive performance. However, they remain vulnerable to
adversarial attacks. In this work, we propose a plug-in robustification module via quantization for contrastive learning in self-supervised skeleton-based human action recognition (QCLR), which enhances the robustness of representations against
adversarial attacks while preserving their representational capability. Specifically, we employ a dual-level quantization scheme across the encoding pipeline, which breaks the gradient propagation and absorbs small
adversarial perturbations within the quantization bins. To mitigate the impact of inevitable quantization error, we further design an
attention-guided multi-faceted quantization strategy to maintain the representation quality. Experiments on several
benchmark datasets demonstrate that integrating QCLR into a progressive lineage of mainstream contrastive learning models substantially improves robustness against both gradient-based white-box and gradient-free black-box
adversarial attacks, while preserving highly competitive classification accuracy.
Read the full abstract ▾
All authors Nanzhu Chen, Feixiang He, Yunfeng Diao, Duygu Sarikaya, He Wang
Computer Vision
OWLET: On-Policy Weighted Learning via Evidence Transfer for Video Large Language Model Post-Training
Qitan Lv, Ke Fan, Hong Wang et al. (+5 more)
Post-training has become a key stage for eliciting
reasoning capabilities in
Video Large
Language Models (
Video-LLMs). Existing post-training methods mainly follow two paradigms: reinforcement learning (RL) and on-policy
distillation (OPD). However, they do not fully exploit visual evidence for effective training. RL supervises the policy with a single trajectory-level reward, which is too coarse-grained for
video reasoning: the same response-level advantage is applied to every generated
token, disregarding that the
tokens of a
video response are grounded in the visual stream to very different degrees. OPD provides
token-level supervision through output-distribution matching, but lacks direct supervision of the spatiotemporal representations that produce those outputs. We observe across extensive experiments that the same model reasons more accurately when its
video view combines global temporal context with question-relevant details. Comparing the same student-generated response under this view and uniform sampling reveals changes in
token likelihoods and internal representations, providing a training signal for evidence use. Building on this observation, we propose OWLET, an evidence-privileged RL method that derives training signals from a richer teacher view. This view couples uniform global context with question-conditioned local windows, providing visual evidence without requiring a larger model. To better exploit this privileged visual evidence, OWLET introduces (i) evidence-aware credit redistribution, which compares
token likelihoods across the two views and redistributes positive response-level advantages according to their relative evidence support; and (ii) spatiotemporal state
alignment, which aligns the two views' hidden states on verifier-accepted responses.
Read the full abstract ▾
All authors Qitan Lv, Ke Fan, Hong Wang, Wen Wu, Yilin Wang, Bowen Zhou, Feng Wu, Chao Zhang
Computer Vision
ConClick: Element-Wise Contrastive Learning Improves GUI Grounding
Ruotian Zhang, Yuhui Zhang, Junzhe Huang et al. (+3 more)
GUI grounding, the task of locating the interface element specified by a user instruction, is a fundamental capability of GUI
agents. However, existing grounding models remain error-prone, often causing downstream task failures. We systematically evaluate 10 grounding models across five
datasets and identify a major error pattern: most incorrect clicks hit non-target interface elements rather than the background, with different models repeatedly selecting the same distractors. Motivated by these findings, we propose ConClick, a contrastive learning framework that teaches models to distinguish the target element from plausible distractors. We construct ConClick-Data, a
dataset containing 4,292 challenging target-distractor pairs, and train the model using complementary listwise ranking and pairwise margin objectives. ConClick improves the grounding accuracy of Qwen3-VL-8B from 54.71% to 66.03% on ScreenSpot-Pro, from 83.72% to 86.28% on MMBench-GUI L2, and from 63.33% to 70.20% on OSWorld-G, outperforming prior methods built on the same or similarly sized backbones. We further introduce ReClick, an inference-time strategy that revisits promising regions and compares predictions across competing candidates, yielding an additional 2.96 percentage-point improvement in average accuracy across the three
benchmarks. Together, our analysis,
dataset, and methods provide both empirical insights and practical tools for building more reliable GUI
agents. Data and code are available at https://github.com/zhangruotian/ConClick.
Read the full abstract ▾
All authors Ruotian Zhang, Yuhui Zhang, Junzhe Huang, Yu Xin, Hehe Fan, Xiaoxiao Sun
Computer Vision
Haowei Wang, Hao Lin, Qi Zhu et al. (+4 more)
On-policy
distillation (OPD) reveals versatility and convenience during VLM post-training, providing dense supervision for student from teacher. However, we observed an anomaly that the distilled VLM could still answer the questions accurately even when the related visual regions were masked or replaced with noise. Building on this observation, our quantitative and qualitative experiments demonstrate that OPD, which uses the answer as a hint, has failed to teach students to genuinely seek out and rely on visual evidence. Our theoretical analysis suggests that this failure arises from a shortcut enabled by the lack of visual constraints during
distillation. To address the issue, we introduce Contrastive
Vision On-policy
Distillation (CV-OPD), which compares teacher distributions induced by positive and negative crops while conditioning on the same student prefix and augments optimization with a margin-based contrastive constraint, which is modulated with discrepancy gating across
token and sample level. Specifically, CV-OPD first introduces a negative teacher unrelated to the answer and designs a Contrastive Divergence Loss (CDL) that constrains the feature distribution of student by contrasting the positive and negative teachers, correcting how the student exploits visual regions. The CV-OPD introduces a Discrepancy Aware Gate (DAG) that uses
token-level and sample-level discrepancies to select visually relevant
tokens for optimization, directing the student's
attention toward visual content. Extensive experiments across a broad range of fine-grained visual
benchmarks demonstrate that CV-OPD substantially alleviates shortcut and consistently improves the visual
reasoning capabilities of existing VLMs.
Read the full abstract ▾
All authors Haowei Wang, Hao Lin, Qi Zhu, Jiayi Ji, Deyi Ji, Liujuan Cao, Rongrong Ji
Computer Vision
SheafPrune: Sheaf-Guided Graph Propagation for Visual Token Pruning
Xiaoyang Ma, Chi Wang and Hong Sun
Multimodal large
language models process the complete visual
token sequence, increasing prefill cost even when many
tokens are irrelevant to the instruction.
Token-wise saliency captures limited inter-
token dependence; graph-based selection instead depends on how relations govern importance propagation and ranking. We study support-constrained propagation with node-degree correction. Inspired by sheaf geometry, SheafPrune uses modality-specific maps and a sparse text–visual graph to transfer relevance, then propagates scores on a separate visual graph and corrects them for degree. It retains a hard top-$k$ subset once before the frozen
language model. The selector has about $4.5$M parameters and is trained end-to-end through the frozen LLaVA-NeXT-8B backbone using only the
language-modeling objective. Explicit retention-ratio conditioning lets one checkpoint serve every budget. Across four backbones, ten
benchmarks and seven
pruning ratios, with all baselines re-implemented under one protocol, SheafPrune achieves the best retention at $22$ of $28$ backbone–budget settings, with monotonically growing margins in the moderate-to-aggressive regime. On LLaVA-NeXT, retaining $10\%$ of visual
tokens yields a $5.15\times$ net prefill speedup and an $84.0\%$ end-to-end FLOPs reduction. Together with component ablations, these results support graph-constrained importance propagation and degree correction as a design for visual
token selection across the evaluated backbones and tokenization schemes, improving the trade-off between accuracy retention and inference cost. code:https://github.com/Bob0817912/SheafPrune_ICLR_2027
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Yang Wang, Ping Liu, Ruiyu Wang et al. (+3 more)
Memory-dependent manipulation requires a policy to track execution state and recover visual evidence that is no longer observable. We propose dual-stream
memory for
vision-
language-action policies, assigning these requirements to complementary representations. A task-agnostic Gated DeltaNet compresses history into one temporal
token, supervised by progress prediction and action-conditioned feature reconstruction. A task-conditioned retriever retains frames above a fixed relevance threshold; linear multi-scale fusion encodes each frame into sixteen spatial
tokens. We then freeze the temporal compressor and jointly optimize
retrieval and policy learning. Across sixteen RoboMME tasks, the complete model achieves 49.22% mean success versus 44.51% for the published FrameSamp+Modul reference, improving fourteen tasks and all four suite means. On four real-
robot tasks, mean success reaches 62.5% versus 37.5%, with the largest gains on repeated actions requiring execution-stage discrimination. These results support combining execution context with selective visual access, while failures under cup swaps expose persistent identity tracking as an unresolved requirement.
Read the full abstract ▾
All authors Yang Wang, Ping Liu, Ruiyu Wang, Xuhua Chen, Lingfeng Zhang, Tao Zhang
Computer Vision
Beyond a Single Crop: Contextualized Embeddings for Player Re-Identification
Ge Luo, Changhuo Yang, Weisi Fan et al. (+2 more)
Person re-identification is typically addressed by learning identity-discriminative
embeddings from individual person crops. However, player re-identification in broadcast sports
videos presents a unique challenge: players from the same team often share similar appearances, while
video sequences provide repeated observations of the same players under different viewpoints and surrounding contexts. Existing crop-based approaches largely ignore this contextual information, which can be critical for resolving visually similar identities. Here, we propose ContextReID, a contextualized
embedding framework that represents a player by jointly considering other players in the same frame and observations from different frames of the broadcast sequence. By incorporating both co-occurring players and cross-frame observations, the model learns identity features relative to the surrounding visual context rather than relying only on isolated appearance cues. We implement this framework with a
vision-
language model that takes sets of player crops as input and generates context-aware
embeddings for individual players. A combined contextual learning and
retrieval objective encourages the model to capture cross-frame identity relationships while maintaining discriminative
embeddings for player matching. Experiments across three multi-object tracking
benchmarks show that ContextReID improves player
retrieval over conventional single-crop
embedding approaches in both Rank-1 and mAP. These results highlight the utility of spatial and temporal context for reliable player identification in broadcast sports
videos.
Read the full abstract ▾
All authors Ge Luo, Changhuo Yang, Weisi Fan, Rui Yang, Shaokai Wang
Computer Vision
Vision Transformers Learn Word Meaning from Images Alone
Anders Gjølbye, Sebastian Ray Mason, Stella Frank et al. (+1 more)
Can a
vision model learn what words mean from pixels alone? We show that the two largest DINOv3
vision transformers organize words rendered in images by word meaning, despite being pretrained using self-supervision on images only, without paired captions, class labels, or other
language supervision. We derive word representations from
embedding differences between word images and matched controls, then test whether semantically related words are neighbors within each model's
embedding space, while controlling for spelling similarity. We find that brand names in the same industry group together, as do nouns from the same semantic domain. Country-capital pairs exhibit consistent vector offsets, similarly to word
embeddings trained on text. These word representations also align with the representations of the visual object categories they name. DINOv3-H and DINOv3-7B are, to our knowledge, the first self-supervised
vision models to show substantial semantic structure for rendered words. Comparisons with text and
speech encoders reveal shared structure among representations of the same words. Contrastive
alignment with either encoder further strengthened DINOv3-H's word-level semantic structure, while leaving the other finetuned
vision models largely unchanged. Image-only training can therefore teach
vision models not just what words look like, but also how their meanings relate.
Read the full abstract ▾
All authors Anders Gjølbye, Sebastian Ray Mason, Stella Frank, Lars Kai Hansen
Computer Vision
Deep Learning Approach for RealTime Contraction Detection in Laboring Dairy Cattle
Sebastian Puchala, Dominik Gronkiewicz, Marcin Gołębiewski et al. (+2 more)
Currently, the animal production sector is undergoing rapid development of techniques that allow automatic monitoring of animal behaviour and health. This paper presents a computer
vision-based approach for the automatic recognition of labor contractions in cows. During the course of the study,
videos depicting 100 cow births were collected. From the obtained
videos, 8500 cow contraction actions and about 25 000 non-contraction sequences were extracted which constituted the
dataset. Leveraging image processing techniques and deep learning
algorithms like Convolutional Neural Networks (CNNs) and
Vision Transformers (ViTs), our methodology extracts relevant features from
video frames to detect and classify contractions. Our study demonstrates a high accuracy rate of over 95% in detecting labor contractions. This automated system has the potential to enhance productivity and animal welfare in the dairy industry by providing real-time monitoring and facilitating timely intervention during difficult calvings. The
video demonstration of the system can be found at the following URL: https://www.youtube.com/watch?v=8fTSG5D7dZ0
Read the full abstract ▾
All authors Sebastian Puchala, Dominik Gronkiewicz, Marcin Gołębiewski, Grzegorz Grodkowski, Grzegorz Gwardys
Computer Vision
Jag: Direct Box Prediction for Efficient Visual Grounding
Xiuyuan Zhu, Ke Lu, Hao Wu et al. (+3 more)
Visual grounding combines image–
language understanding with
geometric prediction, yet autoregressive
multimodal models often express a box as a coordinate sequence. Sequential decoding adds computation to the
multimodal processing needed for localization. Adapting these representations to box geometry offers a route to direct prediction. We present Jag, which adapts a compact pretrained
multimodal model for direct continuous box prediction. A lightweight regression head reads the last valid input-
token state, conditioned on the image and referring expression, and predicts all four box coordinates together.
Geometric supervision jointly adapts the
language backbone, visual merger, and regression head, allowing localization losses to shape both the
multimodal representations and the coordinate mapping. Inference retains image–
language processing and returns the box in one forward pass, without coordinate generation or text parsing. Across
benchmarks, Jag consistently improves grounding accuracy over its base model across the observed target sizes and expression groups. Component ablation show that jointly adapting the representations and head substantially improves accuracy over training the head alone. Against existing specialized grounding models, it achieves competitive accuracy with lower single-request latency and GPU
memory use. These results show how supervision tailored to the
geometric answer can specialize a compact pretrained
multimodal model for accurate grounding and efficient direct prediction. Project page: https://xyzzzh.github.io/Jag/
Read the full abstract ▾
All authors Xiuyuan Zhu, Ke Lu, Hao Wu, Zijin Du, Dongming Zhang, Jian Xue
Computer Vision · Cross-listed from eess.IV
The Predictability Bias of Joint-Embedding Predictive Learning in Medical Imaging
Fabian Drexel, Bernhard Kainz, Jakob N. Kather et al. (+4 more)
Joint-
embedding predictive architectures (JEPAs) learn by predicting teacher representations of masked regions from visible context. Point prediction summarizes the conditional target distribution and cannot reproduce variation that the context does not resolve. We call this constraint predictability bias. In computed
tomography (CT), recurrent anatomy is predictable from context, whereas the sporadic, localized pathology that determines clinical interpretation often is not. We introduce TAP-JEPA, a ViT-B/3D trained on 113,611 CT volumes that is competitive with state-of-the-art encoders (COLIPRI, Merlin, VoxelFM) in scan-level classification. We then audit the learned predictive pathway by replaying the exact masking, teacher, and predictor computations used during training. Prediction lowers the mean Skill (prevalence-normalized average precision) from 0.388 to 0.271 for pathology, but only from 0.921 to 0.873 for anatomy. Along the teacher direction, separating pathology from matched control regions in the same organ, prediction moves pathology 34.7% toward the controls and controls 13.6% toward pathology, with larger shifts farther from visible context. Replacing visible matched-control representations increases surrounding-target prediction loss more than fully replacing pathology representations across four pathology categories. Downstream, removing all final-layer pathology
tokens reduces the decision margin by at most 0.45% in four of seven scan-level probes. Together, these results show that contextual predictability is not a sufficient proxy for clinical relevance in medical representation learning. Training objectives should therefore retain diagnostic features regardless of their predictability from surrounding anatomy.
Read the full abstract ▾
All authors Fabian Drexel, Bernhard Kainz, Jakob N. Kather, Daniel Rueckert, Lisa C. Adams, Keno K. Bressem, Cosmin I. Bercea
Computer Vision
FPD-Drive: Future Policy Distillation for End-to-End Autonomous Driving
Li-Heng Chen, Qifeng Chen, Chengwei Wu et al. (+4 more)
End-to-end autonomous driving models are typically trained on real-world driving logs, which record only the trajectory actually executed in each scene. This provides limited supervision for learning alternative valid behaviors. Multi-target supervision can mitigate this limitation, but broader trajectory coverage makes safe candidate selection more challenging. We propose FPD-Drive, an end-to-end planner that distills privileged future knowledge into a visual driving policy to improve both multi-trajectory supervision and safety selection. Specifically, we introduce Future-Privileged Multi-Target
Distillation to transfer diverse, high-quality trajectory targets from a teacher with access to recorded future observations to the student. These targets extend supervision beyond the single recorded human trajectory. For candidate selection, we further introduce Safety-Aware Score Correction to correct ranking errors between nearby safe and unsafe candidates and raise underestimated safety scores of safe candidates. At deployment, the student requires neither future observations nor teacher inference. Our base model achieves 94.9 PDMS on NAVSIM v1 navtest and 55.9 EPDMS on NAVSIM v2 navhard. The scaled FPD-Drive-Pro improves these scores to 95.5 PDMS and 58.1 EPDMS, respectively, achieving SOTA performance among the compared methods on both
benchmarks. Zero-shot closed-loop evaluation on HUGSIM further yields an HD-Score of 45.1 and RC of 55.3.
Read the full abstract ▾
All authors Li-Heng Chen, Qifeng Chen, Chengwei Wu, Yingjie Zhang, Sheng Yang, Hongbo Fu, Shaoqing Ren
Computer Vision
Automated Detection and Encoding of Cartographic Signs in Historical Maps
Remi Petitpierre
Symbols and icons in archival maps represent an expansive source of evidence for historical geography. In this paper, we present a two-stage framework for (1) detecting cartographic signs in historical maps and (2) generating interpretable sign
embeddings that capture both semantics and stylistic variations. We also introduce CaSiDD, an open
benchmark dataset spanning multiple sign categories and diverse cartographic styles, designed for generic sign detection. Using YOLOv10-X, we demonstrate the practicality of automated sign detection on historical maps. We then present strategies for creating contrasted, informative
embeddings of cartographic signs. We approach
embedding design systematically, maximizing expressiveness while suppressing digitization or background material confounders. Our final method, based on an adapted RADIOv4 backbone, effectively encodes both sign form and semantics, while producing interpretable representations. This work lays a foundation for scalable evidence extraction for the comparative study of map signs within cartographic culture and geohistorical research.
Read the full abstract ▾