Computer Vision
Dual-Stream Semantic Guidance with Prototype Anchor Calibration for Source-Fully-Free Adaptation of Vision-Language Models
Weiwei Xiang, Shun Peng, Guangyi Xiao et al. (+2 more)
Source-Fully-Free Domain Adaptation (SFF-DA) has emerged as a strategic paradigm to adapt
Vision-
Language Models (VLMs) without any access to source data or task-specific source models. However, we identify a critical Dual Semantic Drift that hinders this process: static drift arising from the
rigidity of fixed class
embeddings, and dynamic drift stemming from the divergence of generated captions, causing severe semantic misalignment that intensifies the stability-plasticity dilemma. To address this, we propose DSSG (Dual-Stream Semantic Guidance), an end-to-end framework that reconciles fine-grained plasticity with global stability. Our core contribution is the Dual Semantic Guidance (DSG) module, which integrates a caption stream for domain-specific knowledge with a class-anchor stream to anchor global categorical consistency. Furthermore, a Dynamic Cross-Modal Knowledge
Distillation (CMKD) module is introduced to leverage the evolving teacher distribution for calibrating teacher-student consistency. Building upon DSSG, we further introduce Prototype Anchor Calibration (PAC), yielding DSSG-PAC, which periodically calibrates prototype anchors and caches them until the next calibration. This design reduces redundant text-side computation while preserving the adaptability of class guidance to the evolving text space. We further establish SFF-DA risk bounds that relate student risk to semantic-teacher quality and teacher–student discrepancy. Extensive experiments demonstrate that DSSG consistently outperforms current state-of-the-art methods across multiple
benchmarks, while DSSG-PAC largely preserves its adaptation performance with 18.9% lower total adaptation time. The code is available at https://github.com/mrmenand/DSSG.
Read the full abstract ▾
All authors Weiwei Xiang, Shun Peng, Guangyi Xiao, Hao Chen, Lei Yang
Computer Vision
Xiaoyang Guo, Guoping Luo, Jusheng Zhang et al. (+2 more)
Adapting
video vision-
language models (VLMs) is computationally expensive because
video inputs produce a large number of visual
tokens, making both fine-tuning and inference costly. Although visual
token compression can reduce this overhead, direct adaptation on compressed inputs often causes semantic drift and noticeable performance degradation. We present
Token-Budget
Distillation (TBD), a parameter-efficient fine-tuning framework for adapting
video VLMs under a fixed
token budget. TBD freezes the pretrained backbone, updates only LoRA adapters, and integrates FlashVID-based visual
token compression into the
video pathway. To preserve full-
token semantics under compression, TBD employs a dual-path teacher-student design, where a full-
token teacher provides stable supervision and a compressed student is optimized with task loss, answer-region KL
distillation, GT-anchored margin
distillation, and reliability-aware KD control. This design enables the student to recover the semantic behavior of the full-
token model while remaining efficient under aggressive
token reduction. We evaluate TBD on three
video VLM backbones, including LLaVA-
Video, LLaVA-OneVision, and Qwen3-VL-8B-Instruct, across four
video understanding
benchmarks. TBD consistently outperforms compression-only baselines under both moderate and aggressive compression. On LLaVA-
Video at retention ratio R = 10 percent, TBD preserves 97.0 percent of the Vanilla model's average accuracy; on LLaVA-OneVision at R = 10 percent, it achieves an average score of 58.4 and matches 100.0 percent relative accuracy.
Read the full abstract ▾
All authors Xiaoyang Guo, Guoping Luo, Jusheng Zhang, Keze Wang, Wenhao Wang
Computer Vision · Cross-listed from eess.IV
CheXtriev: Anatomy-Centered Representation for Case-Based Retrieval of Chest Radiographs
Naren Akash, Arihanth Tadanki and Jayanthi Sivaswamy
We present CheXtriev, a graph-based, anatomy-aware framework for chest radiograph
retrieval. Unlike prior methods focussed on global features, our method leverages graph
transformers to extract informative features from specific anatomical regions. Furthermore, it captures spatial context and the interplay between anatomical location and findings. This contextualization, grounded in evidence-based anatomy, results in a richer anatomy-aware representation and leads to more accurate, effective and efficient
retrieval, particularly for less prevalent findings. CheXtriv outperforms state-of-the-art global and local approaches by 18% to 26% in
retrieval accuracy and 11% to 23% in ranking quality. The code is available at https://github.com/cvit-mip/chextriev.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
DeicticVLA: Unifying Instruction Modes Based on Language and Deictic Gestures in a Single VLA
Kango Yanagida, Tatsuya Aoki, Yuichiro Yoshikawa et al. (+1 more)
Vision-
Language-Action models (VLAs) allow users to specify manipulation tasks in natural
language, but distinguishing a target or placement goal among objects of the same category or similar appearance requires detailed expressions that VLAs may not use reliably. We propose DeicticVLA, which canonicalizes
Language Instruction (LI),
Vision-
Language Instruction (VLI), and Visual Instruction (VI) into a text
prompt and deictic masks through text-
prompt completion and deictic gesture grounding, enabling a single pretrained VLA to handle all three instruction modes. With a shared backbone, demonstrations, and matched training steps, we compare two RGB visual prompting methods, two separate-channel mask prompting methods, and three training strategies in simulation. Under two-stage training, the four prompting methods achieve high in-distribution success but differ in their ability to use deictic masks in unseen layouts. Across methods, training-strategy ablations show that two-stage training improves such use, while retaining second-stage LI data mitigates forgetting without reducing VLI and VI performance. In three real-world tasks, one policy supports all modes. VLI and VI outperform LI under unseen expressions, appearance changes, and novel objects. For unseen categories, both achieve 100% success, compared with 16.7% for jointly trained LI. These results demonstrate the unified three-mode interface and guide DeicticVLA design.
Read the full abstract ▾
All authors Kango Yanagida, Tatsuya Aoki, Yuichiro Yoshikawa, Takato Horii
Computer Vision · Cross-listed from cs.GR
What Will This Copper Look Like Later? Forecasting Surface Appearance and Rendering It as a PBR Material
Teejuta Sriwaranon, Borworntat Dendumrongkul, Tanapat Chamted et al. (+1 more)
Digital design requires predicting how a metal surface will look later in its oxidation; this paper presents such a pipeline for copper. Given a fixed-camera observation, the system forecasts appearance 10 accelerated units ahead and converts it into the albedo, normal, roughness and metallic maps a renderer consumes.
Forecasting is evaluated as an authoring tool would use it, on a copper specimen the system has not observed: an entire recording is held out, so training and checkpoint selection use one specimen and the test set is the whole of a second, recorded on a different day and condition. Under this protocol a learned spatio-temporal model with a monotone oxidation state, the most accurate forecaster within a single recording, is less accurate than copying the last observed frame on an unseen specimen, in both directions, as are three further trained architectures. The only forecaster that transfers is a closed-form global color extrapolation with no trained parameters, improving on copy-last-frame by 13.4% and 50.6%, with a margin that increases with horizon to +16.7% and +55.5% at t+10. Two controls qualify this: correcting every frame for the photometric drift measured on a non-oxidizing reference region leaves both margins intact, ruling out uncontrolled exposure as their source, and a moving-block bootstrap over the 6 independent windows each recording contains separates the larger margin from zero but leaves the smaller one not individually significant. The mechanism is measured: a learned susceptibility map encodes where corrosion begins on the training specimen and misleads on a new one, whereas the global color trajectory is what specimens share. The pipeline therefore deploys the closed-form forecaster for unseen specimens and the learned model only for continuing one already observed. Code, splits, protocol and leakage audit are released.
Read the full abstract ▾
All authors Teejuta Sriwaranon, Borworntat Dendumrongkul, Tanapat Chamted, Pizzanu Kanongchaiyos
Computer Vision
Ex-Sim(3)-Reg: 2D-3D Correspondence Pruning via Extended Sim(3) Registration
Pei An, Muyao Peng, Junfeng Ding et al. (+2 more)
Learning-based image-to-point-cloud (I2P) registration has garnered increasing
attention in recent years. Nevertheless, existing methods still struggle with severe outliers under challenging scenarios with unseen, low-inlier, or distorted cases. A fast and robust 2D-3D correspondence
pruning method is therefore highly desirable. Recently, a promising scheme lifts 2D-3D correspondences to 3D-3D correspondences using depth priors, casting correspondence
pruning as a Sim(3) registration problem. However, depth priors estimated from
monocular images are inherently noisy, which undermines the reliability of this scheme. In this paper, to explicitly model non-negligible depth noise, we reformulate correspondence
pruning as an extended Sim(3) registration problem and propose a simple yet effective
pruning algorithm termed Ex-Sim(3)-Reg. We further provide a theoretical analysis to justify the effectiveness of our method. Extensive experiments on the 7-Scenes, RGBD-V2, ScanNet, and TUM
datasets demonstrate that Ex-Sim(3)-Reg achieves up to 24.7% improvement in registration recall over state-of-the-art baseline methods. Code is released at github.com/anpei96/ex-sim3-demo
Read the full abstract ▾
All authors Pei An, Muyao Peng, Junfeng Ding, Jiaqi Yang, Liangliang Nan
Computer Vision · Cross-listed from eess.IV
Do Medical Vision Models Reason About Anatomy? Probing the Spatial Inductive Biases of Learned Visual Representations
Naren Akash and Neeraja Ramanan
Interpreting a CT scan means comparing structures on either side, judging how far apart organs sit, and knowing where each one belongs. Medical
vision encoders are evaluated on diagnostic accuracy, or through assembled
multimodal systems where a failure is hard to attribute, so it remains unclear whether their representations support any of this. We construct SPAR-Bench, eight probes over multi-organ abdominal CT that separate coordinate localization, relational
reasoning, and spatial queries, and apply them to five architectural configurations and three medical foundation models, frozen and finetuned. Probes that ask for a comparison within the slice stay at chance, and neither pretraining scale, finetuning, nor architecture closes the gap. Probes that appear solved in domain fall to chance under zero-shot transfer, indicating that their accuracy reflects recall of canonical anatomy rather than computation over the image. Reading the same frozen features with a pooled head rather than the full set of
tokens moves relational recovery from 0.7% to 67.8%, so pooled probing understates what a representation holds. Questions the encoders answer well are answered at chance by four open-weight MLLMs. Our results suggest these encoders carry a map of where organs usually lie, and little of the machinery for comparing structures within a particular patient. Code and data will be available at https://spar-bench.github.io.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.IT
Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation
Zijun Zhang, Li Qiao, Mahdi Boloursaz Mashhadi et al. (+3 more)
Token Communications (TokenCom) has recently emerged as a new paradigm in which
tokens serve as unified units for communication and computation, enabling efficient
multimodal semantic and goal-oriented transmission. In this paper, we develop Ada-TokenCom, a rate-adaptive TokenCom framework based on large autoregressive models, which integrates next-
token prediction with arithmetic coding to achieve ultra-low bitrate semantic communication at the
token level. We propose a mixed reconstruction/generation scheme, where the transmitter encodes and transmits the highly informative
tokens at the beginning of the
token sequence leveraging a pre-trained autoregressive large model, while the receiver uses an identical model to predict the rest. Moreover, we design a Lyapunov-based
algorithm to dynamically optimize both the source compression rate and the modulation and coding scheme, adapting to time-varying network conditions. Simulation results demonstrate that our proposed Ada-TokenCom framework outperforms both digital and deep joint source-channel coding-based semantic communication baselines.
Read the full abstract ▾
All authors Zijun Zhang, Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao, Mehdi Bennis, Kaibin Huang
Computer Vision
Attribute Token Arithmetic: Disentangled and Continuous Semantic Control for Visual Autoregressive Models
Xindi Yang, Yicheng Wu, Cheng Zhang et al. (+2 more)
Autoregressive text-to-image generation has recently achieved remarkable progress, offering high-fidelity synthesis via a unified generative framework. However, fine-grained semantic control remains challenging due to the attribute
entanglement and the misalignment between textual and fine-grained visual representations. In this paper, we introduce Attribute
Token Arithmetic (ATA), a method that enables disentangled and continuous attribute control in visual autoregressive modelling. Inspired by the vector arithmetic property observed in word
embeddings, ATA identifies semantic directions corresponding to visual attributes (e.g., aging, fatness,
emotion) directly within the pretrained autoregressive latent space. These directions are learned from a single reference image, without model retraining or large-scale supervision. During generation, attributes can be continuously adjusted and compositionally combined through simple arithmetic operations with other attribute
tokens. Extensive experiments demonstrate that ATA achieves identity-preserving, fine-grained, and multi-attribute adjustment, outperforming existing autoregressive editing baselines in controllability, generality, and computational efficiency. Our code will be available at https://github.com/Madaoer/ATA.
Read the full abstract ▾
All authors Xindi Yang, Yicheng Wu, Cheng Zhang, Jianfei Cai, Tien-Tsin Wong
Computer Vision
Cyc3D: Evaluating Cyclic Structural Stability and Asset Usability in Image-to-3D Generation
Liwen Zhang
Image-conditioned 3D generation has advanced rapidly, yet existing evaluation protocols largely judge rendered-view plausibility and semantic
alignment, overlooking whether a generator forms a stable 3D interpretation and produces assets usable in graphics pipelines. We introduce Cyc3D, a multidimensional
benchmark that evaluates image-to-3D generation along two complementary axes: Cross-View Object Consistency and Representation Quality. At the asset level, Cyc3D measures whether object identity remains semantically coherent across rendered viewpoints. At the model level, we propose View-Cycle Structural Consistency, a closed-loop render-regenerate-align protocol that repeatedly re-observes a generated asset from novel views and quantifies
geometric, perceptual, and semantic drift across generations. To assess native asset usability beyond rendered appearance, Cyc3D further evaluates
geometric structure, reference-image fidelity, mesh discretization and efficiency, and UV parameterization quality. Together, these diagnostics expose failures obscured by a single perceptual score and provide interpretable evidence of both model instability and representation defects. Experiments on five representative image-to-3D systems show that closed-source feed-forward models consistently outperform open-source optimization-based baselines in
geometric fidelity, mesh quality, and cycle stability. Nevertheless, even the strongest methods achieve cycle-stability scores below 48, revealing a persistent gap between visually plausible generation and robust 3D object understanding.
Read the full abstract ▾
Computer Vision
Task-State Adaptation with Prototype Memory for Multi-Task Dense Prediction
Yangyang Xu, Haobo Yuan, Yuzhu Wang et al. (+4 more)
Vision foundation backbones provide strong representations for dense prediction, yet a single shared feature still needs to support tasks with different, image-dependent adaptation requirements. We propose MemMTL, a multi-task dense prediction framework that estimates a compact task state from global visual context and refines it through a learnable task-state prototype
memory. The refined state is converted into task-conditioned expert logits and combined with
token-level logits before sparse top-$k$ selection over a local expert bank shared by all tasks. A separate task-agnostic residual bank provides a common adaptation path, and both paths are added once to the backbone feature before task-specific prediction. We specify a matched evaluation protocol on NYUD-v2 and PASCAL-Context with SAM 3 and ViT-L backbones to measure predictive quality, computational cost, and the contributions of task-state conditioning, prototype
retrieval, and sparse routing. The numerical record in the present working draft predates this canonical implementation and must be regenerated before it can support empirical claims.
Read the full abstract ▾
All authors Yangyang Xu, Haobo Yuan, Yuzhu Wang, Duo Su, Xi Ye, Yibo Yang, Jun Zhu
Computer Vision
CF-YOLO: Context-Aware Feature Refinement for Camouflaged Industrial Micro-Defect Detection
Xinda Yu, Kunxin Zheng, Chunan Yu et al. (+4 more)
Automated detection of surface micro-defects on industrial components, such as copper tubes, is critically important for quality assurance but remains challenging due to the minute scale of anomalies and their visual camouflage against complex backgrounds. These factors lead to weak feature representations and high rates of false positives and missed detections. To address these issues, we propose a novel real-time detection framework designed for efficient context perception and feature refinement. Our method integrates a Context-Perception Aggregation Module (CPAM), which synergises large-kernel perception for macro-texture context and small-kernel aggregation for sharp boundary delineation, effectively breaking the background camouflage. Furthermore, a Feature Additive Refinement Module (FARM) employs a linear-complexity additive
token mixer to globally verify and refine the representation of fine-grained anomalies, suppressing noise-induced errors. To support research in this domain, we introduce the Copper Tube Defect
Dataset (CTDD), a manually annotated
benchmark containing 1,847 images and 4,898 boundingbox defect instances from copper-tube inspection scenarios. Extensive experiments demonstrate that our detector achieves strong and consistent performance on CTDD, outperforming representative baseline detectors, including YOLOv11, by 2.2% in mAP@50 and 3.9% in Precision while maintaining real-time inference speed. This work provides a robust and efficient solution for high-precision industrial inspection, bridging the gap between contextual understanding and detailed feature analysis. Our code and model are available at: https://github.com/Yu-Xinda/CFYOLO-Context-Aware-Feature-Refinement-for-Camouflaged-Industrial-Micro-Defect-Detection
Read the full abstract ▾
All authors Xinda Yu, Kunxin Zheng, Chunan Yu, Qingbo Song, Hao Xiao, Ying Zang, Jie Liu
Computer Vision
VersaGauss: A Versatile Framework for Generating Multiphase Dynamics with 3D Gaussians
Ruijie Su, Lingxiao Yang, Xiaohua Xie et al. (+1 more)
Recent progress has been made in 3D Gaussian representation for reconstruction, generation, and physical simulation. However, current approaches mainly concentrate on physics-based dynamic generation of solid objects and only handle single-phase collision interactions. We introduce VersaGauss, a unified framework for generation, simulation, and rendering that supports versatile physics-based dynamic generation, particularly for multiphase interactions. Our system takes a few images as input and produces a realistic, physics-driven 3D dynamic scene with multiple objects. To optimize the Gaussian kernel distribution, we develop a particle
pruning algorithm. We also propose the Coupled Multiphase Point Method (CMPM) to effectively model and generate multiphase interactions. Additionally, harmonic interpolation within CMPM and a Gaussian
evolution strategy are introduced to achieve realistic fluid rendering. Extensive experiments demonstrate that our framework can simulate interactions among various materials such as fluid, rubber, sand, snow, and others. Code is available at https://github.com/Elowen-surj/VersaGauss.
Read the full abstract ▾
All authors Ruijie Su, Lingxiao Yang, Xiaohua Xie, Jianhuang Lai
Computer Vision
A Controlled Audit of Architectural Complexity in Uncertainty-Aware Multi-Organ Ultrasound Classification
Yang Song, Pengbo Sun, Shichang Feng et al. (+3 more)
Multi-organ ultrasound classifiers increasingly combine
attention, mixture-of-experts routing, uncertainty gating, and evidential deep learning (EDL) objectives to address heterogeneous anatomy and acquisition. Yet a plausible design rationale does not by itself establish that an added component improves the trained system. We contribute a controlled complexity-audit framework, applied to the deployment decision between the maximal evidential candidate Full-EDL and simpler alternatives. Six candidates were evaluated on the primary
dataset and three in an internal replication, using ten matched seeds, frozen image-level partitions, capacity- and optimisation-aware comparisons, symmetric temperature scaling, paired decision rules, and a separate out-of-distribution (OOD) veto. Retaining Full-EDL did not establish a reliable macro-F1 gain on either
dataset, while the simplified alternatives remained inconclusive under the non-inferiority margin. Simple cross-
entropy with temperature scaling (Simple-CE+TS) met the calibrated negative log-likelihood criterion on both
datasets and showed favourable selective-risk ordering. The raw calibration advantage of evidential training disappeared after temperature scaling and did not recur on the second
dataset. The gate had negligible observable influence at the audited checkpoints, and deleting the Full-only chain revealed no stable task or calibrated-loss benefit. Simple-CE nevertheless triggered the OOD veto against the fetal probe but not the lung probe, precluding an unconditional OOD-safety claim. We therefore selected Simple-CE+TS for the evaluated in-distribution objective while retaining Full-EDL as the maximal reference. Components should earn retention through functional and retraining-based evidence, and calibration and distribution-shift reliability should be evaluated separately.
Read the full abstract ▾
All authors Yang Song, Pengbo Sun, Shichang Feng, Ye Zhu, Xin Xu, Ziran Wang
Computer Vision
Kairong Yu, Zixin Zhu, Le Yu et al. (+1 more)
Large
Vision-
Language Models (LVLMs) remain prone to
hallucinations, producing responses that are irrelevant or inconsistent with the
multimodal input. Existing mitigation methods mainly rely on external supervision, output calibration, or
attention regulation, leaving the internal representation dynamics of autoregressive generation underexplored. We identify an inference-time failure mode in which cross-modal representations degrade across decoder layers and drift across generation steps, destabilizing
token prediction and increasing
hallucination risk. We propose Dynamic
Alignment Compensation (DAC), a training-free inference-time method that detects representation divergence and selectively applies lightweight residual compensation. DAC combines Layer-wise Semantic Compensation to mitigate inter-layer degradation with Sequential Semantic Correction to constrain temporal drift. Experiments on nine
hallucination-focused and general-purpose
multimodal benchmarks across multiple LVLM backbones show that DAC consistently reduces
hallucinations while maintaining strong overall performance.
Read the full abstract ▾
All authors Kairong Yu, Zixin Zhu, Le Yu, Hongwei Wang
Computer Vision
ZipMVS: Multi-View Stereo with Compressed Cost Volumes
Guanglin Jin, Hongshan Yu, Javier Civera et al. (+1 more)
Multi-view stereo (MVS) methods typically deliver highly accurate 3D reconstructions from multiple registered RGB images, thanks to the highly informative,
geometric constraints between them. However, their substantial
memory requirements remain a major obstacle for deployment in domains such as aerospace and autonomous systems, where resource efficiency is critical. In this work, we introduce ZipMVS, an MVS method specifically designed for efficient high-quality reconstruction. We propose a novel depth-hypothesis strategy that enables substantial compression of the cost volume, hence greatly reducing GPU
memory consumption while preserving reconstruction accuracy. Experiments on the DTU and Tanks and Temples
datasets show that ZipMVS achieves competitive reconstruction quality compared with other efficiency-oriented MVS methods, while achieving a competitive balance between reconstruction quality and GPU
memory usage. The code is available at https://github.com/JihnGlyn/ZipMVS
Read the full abstract ▾
All authors Guanglin Jin, Hongshan Yu, Javier Civera, Zhaoxin Li
Computer Vision
3D-USE: From Image-Level to Scene-Level Underwater Enhancement
Jieyu Yuan, Yuanlin Zhang, Jihong Li et al. (+3 more)
Underwater 3D reconstruction faithfully reproduces the color shifts and visibility loss of captured views, while physical inversion may leave estimation errors in the recovered scene appearance. We formulate
Underwater Scene-level Enhancement (USE) as learning a persistent, visibility-enhanced 3D scene representation from degraded multi-view
underwater observations, enabling consistent enhanced rendering. Realizing USE requires both a reliable scene representation for enhancement and a consistent enhancement target without paired enhanced 3D data. Therefore, we present 3D-USE, a two-stage framework. First, the Medium Radial Basis Anchor Representation (MediumRBF) establishes a medium-aware Gaussian scene by representing water effects with shared radial-basis anchors and explicitly decomposing object and medium contributions. Based on this fixed scene representation, Appearance Transition Consensus (ATC) transfers paired 2D
underwater image enhancement (UIE) knowledge into scene-global and Gaussian-local targets, avoiding direct supervision from inconsistent enhanced views. An
Underwater Bilateral Appearance Field (U-BAF) then realizes these targets in Gaussian radiance and medium appearance. The scene directly renders enhanced novel views without a 2D UIE model at inference. Experiments on real
underwater scenes show improved visibility and cross-view consistency while preserving reconstruction quality.
Read the full abstract ▾
All authors Jieyu Yuan, Yuanlin Zhang, Jihong Li, Chunle Guo, Huimin Lu, Chongyi Li
Computer Vision
Chenxin Fang, Tao Chen, JunChao You et al. (+3 more)
In this paper, we propose a new
token compression paradigm for
video Multimodal Large
Language Models (MLLMs), termed Visual
Token Coding (VTC). Inspired by classical
video coding principles, e.g., HEVC, VTC performs structured compression by predicting the I/P frames of a
video and measuring their frame-wise residuals to estimate
token redundancy. Based on this baseline framework, we also enhance VTC with a set of novel dynamic designs, such as Dynamic Resolution Input (DyRSO), Dynamic
Token Allocation (DyTA), and Spatial Coverage Top-K (SC-TopK), and term this new approach $VTC_{Dy}$. To validate VTC, we apply it to three MLLMs and conduct experiments on multiple
video understanding
benchmarks. The experimental results show that VTC$_{\mathrm{Dy}}$ achieves an average performance retention of 100.1% with a 50%
token budget for Qwen3-VL, while still retaining 97.8% of the average performance when the
token budget is reduced to 25%. Moreover, as a plug-and-play design, VTC requires no additional tuning of MLLMs for
token coding. Our code is available at https://github.com/Msr233/VTC.
Read the full abstract ▾
All authors Chenxin Fang, Tao Chen, JunChao You, Jun Peng, Yiyi Zhou, Rongrong Ji
Computer Vision
A-PAIR: A Benchmark and Identity-Consistent Grounding Framework for Air-Ground Cross-View Referring Person Detection
Zhoupeng Guo, Xinjie Yao, Yunqi Zhu et al. (+6 more)
Air-ground cross-view referring person detection is a necessary component in the
language-to-perception-to-control chain of collective
embodied intelligence, grounding a
language command into the same physical target before ground and aerial
agents can coordinate downstream actions. Existing referring expression comprehension and open-vocabulary grounding methods do not jointly account for cross-view identity consistency, making them insufficient for Air-Ground Cross-View Referring Person Detection (AGCV-RPD), which involves similar pedestrian distractors, weak aerial appearance cues, and cross-view identity consistency. To study this problem, we introduce Air-Ground Paired Identity-Aware Referring (A-PAIR), the first comprehensive AGCV-RPD
benchmark, containing 22,137 cross-view referring samples. To construct A-PAIR efficiently, we propose Factorized Annotation and Referential
Alignment (FARA), a semi-automatic annotation framework that generates factorized referring descriptions and identity-consistency supervision at reduced cost. We propose Identity-Consistent Referring Grounding (ICRG), a framework that combines factorized referential grounding, candidate-completeness supervision, and cross-view consistency calibration for joint air-ground pair selection. ICRG improves ground, aerial, and pair-level detection over strong baselines, increasing pair F1 from 16.65% to 22.28%. These results show that AGCV-RPD requires paired detection and identity-consistent
reasoning.
Read the full abstract ▾
All authors Zhoupeng Guo, Xinjie Yao, Yunqi Zhu, Zhihe Fan, Siqi Zhao, Jianjun Chen, Yichen Dong, Yan Fan, Pengfei Zhu
Computer Vision
GAN-Based Semantic Communication for Image Transmission in IoV
Ruixing Ren, Shan Chen, Junhui Zhao et al. (+1 more)
For cooperative perception in the internet of vehicles, this paper proposes a generative
adversarial network-based semantic communication framework to address the efficiency and fidelity bottlenecks of traditional communication systems in visual data transmission under limited bandwidth and dynamic channel conditions. At the transmitter, the framework adopts a pyramid
attention network to extract semantic label maps and introduces a semantic priority preservation mechanism. It assigns differentiated weights to distinct semantic categories based on driving safety, guiding bit allocation and loss function design. At the receiver, an image reconstruction module integrating a coarse to-fine multi-resolution generator and multi-scale discriminator is designed. Combined with the temporal consistency branch, spatial pyramid pooling and class-aware convolutional layers, it achieves high-fidelity reconstruction of high-quality images from corrupted semantic labels. The model is trained with combined
adversarial, feature matching and perceptual losses, effectively improving semantic consistency and visual realism of generated images. Experimental results on the Cityscapes
dataset show that the proposed method outperforms existing counterparts in both semantic
segmentation accuracy and reconstructed image quality, and maintains stable reconstruction performance under AWGN and Rayleigh channels.
Read the full abstract ▾
All authors Ruixing Ren, Shan Chen, Junhui Zhao, Xiaoke Sun
Computer Vision
Jingpu Yang, Debin Tang, Yilin Sun et al. (+4 more)
Unmanned aerial vehicle (
UAV)
multimodal perception integrates visible (RGB), infrared (IR), synthetic aperture radar (SAR), and depth sensors for scene understanding under diverse conditions. However, differences in optics, resolution, and mounting often limit practical systems to global or image-center
alignment. After tokenization, parallax, platform motion, and lens distortion can shift corresponding patch centers across modalities, weakening the spatial correspondence assumed by dense contrastive learning and cross-modal fusion. We propose GAAT (Geometry-Aware
Alignment Transformer), an
alignment-first pretrained model that estimates local correspondence reliability before cross-modal interaction. GAAT introduces syncPATC, which learns patch-center consistency under synchronized view transformations without correspondence annotations. It emits
geometric priors, including
token and query confidence, query centers, and sub-
token offsets, that identify reliable local anchors across residual misalignment. Guided by these priors, MG-Sparse-MMA performs query-mediated sparse fusion over top-K_s reliable regions, replacing dense all-patch interaction with geometry-calibrated local updates. RA-QCGCL aligns pretraining supervision with this sparse query bottleneck through reliable patch-to-patch, patch-to-query, and query-to-query contrastive branches. We introduce UAVMeta and StateBench, which provide four acquisition-state scores derived from platform telemetry and image statistics: camera reliability, observation scale, viewpoint stability, and flight maneuver complexity. Extensive experiments across six downstream tasks demonstrate consistently superior transfer performance, establishing GAAT as a state-of-the-art
multimodal foundation model for
UAV perception. StateBench further enables a systematic diagnosis of real-world acquisition conditions.
Read the full abstract ▾
All authors Jingpu Yang, Debin Tang, Yilin Sun, Fengxian Ji, Jiahua Zhu, Wenrui Ding, Yufeng Wang
Computer Vision
Training-Free Temporal Abstraction for General Video Understanding
Etienne Casanova, Sevan Brodjian and Pietro Perona
Videos are expensive to analyze frame by frame, yet many
video understanding tasks depend on knowing where relevant moments occur. A system may need to find when an action changes, locate the segment described by a sentence, or choose a few frames for a
vision-
language model. Existing methods often solve these problems separately, using task-specific training data or specialized architectures. We study whether a pretrained
video-text model can provide enough temporal structure to support several of these tasks at once. We present STITCH, a training-free method that divides a
video into semantically meaningful temporal chunks. STITCH embeds short
video windows with a frozen
video-text backbone and detects changes in the resulting
embedding sequence. These chunks are computed once per
video and reused across tasks. We evaluate STITCH on generic event boundary detection,
language-based moment
retrieval, and frame selection for long-
video VLM
reasoning. Across all three settings, STITCH remains competitive with more specialized methods while requiring no task-specific training, with especially clear gains when only a small number of frames or
tokens can be processed. These results suggest that reusable temporal abstraction is a promising direction for general
video understanding, allowing dense
video streams to be converted once into semantic units that can be localized, retrieved, sampled, or reasoned over by downstream systems.
Read the full abstract ▾
Computer Vision
PCBnet: A Dataset and Automatic Construction of SPICE Netlists from Schematic Images
Zhen Huang, Yuhao Gao, Yuzhi Liu et al. (+15 more)
Printed
circuit boards (PCBs) are fundamental to modern electronic systems, yet AI-driven PCB design automation remains constrained by the lack of large-scale paired schematic-netlist
datasets. PCB schematics are particularly challenging due to diverse component types, complex wiring
topologies, and noisy textual annotations. To address this gap, we present PCBnet, a large-scale PCB schematic
dataset comprising over 300 real-world designs with annotated pins and paired SPICE netlists. It contains more than 50,000 component instances, 150,000 wires, 100,000 text regions, and 400,000 characters. We further develop an automated schematic-to-netlist pipeline that combines visual recognition,
topology construction, and domain-knowledge-guided multi-
agent correction. The proposed method achieves 94.54% component detection mAP, 98.57% text recognition accuracy, and 84.47% end-to-end connectivity accuracy. PCBnet provides a
benchmark and data foundation for future AI-driven PCB design automation.
Read the full abstract ▾
All authors Zhen Huang, Yuhao Gao, Yuzhi Liu, Daian Cheng, Chengyuan Shao, Yucheng Chen, Yongjian Jia, Futing Zhang, Yichen Shi, Wenhao Wang, Zuyan He, Yangbo Wei, Zhanfei Chen, Jinlong Yan, Yu Zhang, Haoying Wu, Ting-Jung Lin, Lei He
Computer Vision
DensityKV: Density-Guided KV Cache Compression for Long Video Generation
Wenqu Zhao, Xuemin Chi, Xin Zhang et al. (+6 more)
Autoregressive
video diffusion models enable streaming generation through sliding-window
attention, but each generated block is conditioned on previously generated content, causing appearance and motion errors to propagate recursively over time. Historical key-value (KV)
memory preserves earlier subject and scene states and helps maintain long-horizon consistency. However, retaining every generated state creates a historical archive that grows continuously with the rollout, while recurrent states repeatedly add redundant coverage. To address this problem, we propose DensityKV, a training-free historical KV bank management strategy. DensityKV maintains a separate
token-level KV bank for each
attention head and measures local redundancy among the post-RoPE keys that directly parameterize
attention routing using Soft-Riesz density. By constraining neighborhood-density growth after states enter the bank, DensityKV limits repeated historical accumulation while preserving coherent states from each completed generation block. Experiments across three autoregressive
video generation backbones and multiple generation lengths show that, at the same upper bound on historical KV capacity, DensityKV improves long-horizon consistency and generation stability while keeping persistent historical storage bounded independently of rollout length.
Read the full abstract ▾
All authors Wenqu Zhao, Xuemin Chi, Xin Zhang, Guoqing Ma, Baorun Li, Jianjie Fang, Peizhi Tang, Chen Gao, Wei Wu
Computer Vision
CommerceVibe: Learning to Design E-Commerce Creatives as Executable Visual Code via Dual-Feedback Reinforcement Learning
Yajiao Xu, Jin Zhang, Jiangbo Ai et al. (+4 more)
High-quality e-commerce creatives are essential for presenting products and conveying marketing messages. Recent
diffusion models enable scalable creative generation and produce visually compelling images, but their flattened raster outputs often contain distorted text and inconsistent product details, requiring refinement before deployment. Moreover, without explicit structure, the resulting creatives are difficult to edit and reuse, while complex design requirements remain challenging to encode as verifiable training signals. To address these challenges, we present CommerceVibe, which represents creatives as executable visual code and formulates generation as conditional HTML/CSS program synthesis. Given product images, design requirements, and product information, it produces renderable, editable, and reusable creatives. We further introduce dual-feedback reinforcement learning, in which rule-based feedback evaluates rendered programs for text readability, product visibility, and layout validity, while visual feedback from a
vision-
language model (VLM) assesses rendered creatives against input specifications across six perceptual and commercial dimensions. Together, these complementary feedback signals improve both constraint satisfaction and perception-dependent quality. We perform supervised fine-tuning (SFT) of Qwen3.5-9B on over 28,000 e-commerce examples, followed by dual-feedback reinforcement learning. On a 1,300-case
benchmark, the optimized CommerceVibe model achieves a weighted score of 94.0/100, compared with 87.3 for the SFT-only variant, and outperforms strong external models. Blind evaluations by five e-commerce design experts further validate these improvements. CommerceVibe supports controllable, editable, and scalable e-commerce creative production.
Read the full abstract ▾
All authors Yajiao Xu, Jin Zhang, Jiangbo Ai, Tao Jiang, Mo Xu, Lina Huang, Chengfu Huo
Computer Vision
Thread-Efficient Decoding for Neural Texture Compression
Janarbek Matai, Sho Ikeda, Lukasz Lipski et al. (+1 more)
Neural texture compression (NTC) achieves higher compression ratios than BCn formats but suffers from GPU thread divergence, which significantly reduces runtime performance. In this work, we propose a shared decoder MLP architecture – trained with a gradual decoder freezing schedule – combined with texture clustering to reduce thread divergence by 25%-52% while preserving rendering quality. We evaluate our method on over 500 textures and multiple real rendering scenes, demonstrating up to 8.48x speedup on the Radeon RX 9070 XT GPU compared to non-shared baselines. Our key contributions include: (1) a unified shared decoder architecture that reduces divergence by grouping textures; (2) a training recipe with gradual decoder freezing that improves stability and reconstruction accuracy; (3) a semantic clustering strategy using CLIP
embeddings that groups similar textures for effective decoder sharing; and (4) comprehensive performance and ablation studies validating our approach.
Read the full abstract ▾
All authors Janarbek Matai, Sho Ikeda, Lukasz Lipski, Takahiro Harada
Computer Vision
StreamEMS: Streaming Video Understanding with Self-Evolving Memory Scheme for Vision-Language Models
Yuxin Liu, Peiqin Zhuang and Yali Wang
Recently, many streaming
video understanding methods have been proposed by constructing an external
memory to store historical data for computational reduction. Most methods focus on optimizing the injection procedure of current data (write) and retrieving informative historical data (read) from
memory, while overlooking the opportunity to further enhancing the representational capability of
memory itself. In this work, we present StreamEMS, a general mechanism for improving streaming
video understanding by re-structuring the historical data stored in
memory through self-evolving
memory scheme, enabling more informative and robust
memory representations. Specifically, we first introduce a Semantic
Evolution Module to evolve the
memory into more information-dense representations by exploiting informative
memory entities discovered via progressively shrinking semantic scales from coarse to fine. In addition, we further introduce a Prior-informed
Evolution Module to evolve
memory into more robust representations by leveraging prior
memory distributions to refine the current
memory state. We validate the effectiveness of our proposed designs on widely-used streaming
video understanding
datasets, i.e., OVO-Bench and StreamingBench, and the results showcase that our method performs better than other methods. Moreover, the advantage of our method becomes consistently evident even under high
token usage drop rate settings, indicating the effectiveness and robustness of our method in unleashing the potential of the
memory itself.
Read the full abstract ▾
Computer Vision
What Do Interaction Representations Actually Measure? Pre-Event Separability in Weakly-Supervised Violence Detection
Parishruthi Ganesh
Articulated human pose provides detailed body-configuration information beyond coarse spatial relationships, but whether this detail yields greater discriminative information when the downstream pipeline is held fixed remains unclear. We examine this through early violence detection. Holding the tracker, temporal head, supervision, folds, and evaluation fixed, we compare five interaction representations spanning coarse bounding-box geometry, a matched handcrafted pose analogue, enriched pose descriptors, and a matched-capacity encoder learned from raw joints, under
video-level evaluation with cluster-bootstrap intervals. No pose-based representation outperforms coarse geometry, though with fifteen anomalous
videos this subset cannot rule out small effects. Extending the pipeline to frozen visual encoders, and repeating the comparison on XD-Violence (137 anomalous
videos, nine times our UCF-Crime sample), person-crop appearance and whole-frame context both exceed geometry by a wide margin, yet context matches appearance on UCF-Crime and exceeds it on the larger split: cropping to the interacting people yields no advantage over encoding the whole frame. This
prompts a direct test of what the
benchmark measures. Scoring anomalous
videos using only frames preceding the annotated onset, under a control removing sequence length as a cue, retains 39-91% of above-chance separation on both
benchmarks, including for seven hand-designed
geometric channels. Inspection of the tightest pre-onset windows identifies concrete provenance artifacts: editorial title cards and platform watermarks absent from the surveillance footage supplying the normal class.
Video-level AUC here is thus a composite of event evidence and pre-event source cues, a shared source of discrimination that can obscure differences between representations. The diagnostic requires only annotations these
benchmarks already ship.
Read the full abstract ▾
Computer Vision
Relational Knowledge Distillation Brings DNN Representations Close Enough to Humans to Be Aligned Without Supervision
Yuria Shimizu, Soh Takahashi, Takato Horii et al. (+1 more)
Linking the internal representations of deep neural networks (DNNs) to human mental representations is important for using DNNs as computational models of human
vision. Existing DNN representations remain insufficiently similar to human mental representations, which are not directly observable and are therefore commonly measured through large-scale similarity judgments of object images. A natural approach to narrowing this gap is to directly transfer the relational structure of human representations into DNNs, and previous studies have reported improved human-DNN representational similarity. However, whether this improvement holds under stricter evaluation remains untested in two respects: fine-grained
alignment at the individual-object level, and generalization to a human
embedding derived from a
dataset independent of the training data. Here, we employ an unsupervised comparison method, Gromov-Wasserstein optimal transport (GWOT), which estimates human-DNN correspondences from the internal distance structure alone and thereby tests fine-grained
alignment. We further assess generalization on a curated test set of concepts non-overlapping with the training data. We show that fine-tuning pre-trained DNNs with Relational Knowledge
Distillation (RKD), an established relational transfer method, brings DNNs close enough to humans to be aligned at the individual-object level on this test set. We also show that this improvement is driven by a more human-like global structure, as reflected in the ordering of distances among coarse categories, while the local human-DNN nearest-neighbor overlap rate remains largely unchanged. These findings indicate that relational transfer from humans brings the global structure of pre-trained DNNs close enough to the human structure to enable fine-grained human-DNN
alignment without supervision.
Read the full abstract ▾
All authors Yuria Shimizu, Soh Takahashi, Takato Horii, Masafumi Oizumi
Computer Vision
Temporal Tree of Thought: Reasoning-Guided Visual Cue Search for Long-Video Understanding
Ziling Huang and Shin'ichi Satoh
Long-
video understanding remains challenging for
Multimodal Large
Language Models (MLLMs) due to limited context length. Uniform sampling may miss crucial moments, while
agent-based frame
video understanding methods often evaluate frames independently, overlooking the temporal organization of
videos. Ideally, evidence selection should mimic how humans answer questions about long
videos: first locating the relevant segment from the global context, then zooming into local objects and details. We propose Temporal Tree of Thought T^3, a training-free framework for adaptive coarse-to-fine long-
video understanding. T^3 constructs a question-agnostic hierarchical temporal tree via recursive temporally constrained clustering, where each node represents a contiguous segment with an informative key frame. During inference, T^3 performs an answer-retrieve-explore loop: it reasons over coarse representative frames, generates a search statement when evidence is insufficient, and expands relevant branches for finer-grained evidence. This process adaptively shifts the search target from temporal regions to specific objects and visual details to help
video understanding. Experiments on VideoMME, LongVideoBench, and LVBench show that T^3 improves Qwen2.5-VL-7B by 0.5%, 4.6%, and 4.4%, respectively, under the same frame budget, demonstrating the effectiveness of structured temporal
reasoning.
Read the full abstract ▾