Computer Vision
Hiroaki Aizawa and Yuki Igaue
Transformers are strong baselines in both
vision and
language because self-
attention captures long-range dependencies across
tokens. However, the cost of self-
attention grows quadratically with the number of
tokens. Patch
pruning mitigates this cost by estimating per-patch importance and removing redundant patches. To identify informative patches for
pruning, we introduce a criterion based on the Shannon
entropy of the
attention distribution. Low-
entropy patches, which receive selective and concentrated
attention, are kept as important, while high-
entropy patches with
attention spread across many locations are treated as redundant. We also extend the criterion from Shannon to Rényi
entropy, which emphasizes sharp
attention peaks and supports
pruning strategies that adapt to task needs and computational limits. In experiments on fine-grained image recognition, where patch selection is critical, our method reduced computation while preserving accuracy. Moreover, adjusting the
pruning policy through the Rényi
entropy measure yields further gains and improves the trade-off between accuracy and computation.
Read the full abstract ▾
Computer Vision
HistoFusionNet: Histogram-Guided Fusion and Frequency-Adaptive Refinement for Nighttime Image Dehazing
Mohammad Heydari, Wei Dong, Shahram Shirani et al. (+2 more)
Nighttime image dehazing remains a challenging low-level
vision problem due to the joint presence of haze, glow, non-uniform illumination, color distortion, and sensor noise, which often invalidate assumptions commonly used in daytime dehazing. To address these challenges, we propose HistoFusionNet, a
transformer-enhanced architecture tailored for nighttime image dehazing by combining histogram-guided representation learning with frequency-adaptive feature refinement. Built upon a multi-scale encoder-decoder backbone, our method introduces histogram
transformer blocks that model long-range dependencies by grouping features according to their dynamic-range characteristics, enabling more effective aggregation of similarly degraded regions under complex nighttime lighting. To further improve restoration fidelity, we incorporate a frequency-aware refinement branch that adaptively exploits complementary low- and high-frequency cues, helping recover scene structures, suppress artifacts, and enhance local details. This design yields a unified framework that is particularly well suited to the heterogeneous degradations encountered in real nighttime hazy scenes. Extensive experiments and highly competitive performance of our method on the NTIRE 2026 Nighttime Image Dehazing Challenge
benchmark demonstrate the effectiveness of the proposed method. Our team ranked 1st among 22 participating teams, highlighting the robustness and competitive performance of HistoFusionNet. The code is available at: https://github.com/heydarimo/Night-Time-Dehazing
Read the full abstract ▾
All authors Mohammad Heydari, Wei Dong, Shahram Shirani, Jun Chen, Han Zhou
Computer Vision
Next-Scale Autoregressive Models for Text-to-Motion Generation
Zhiwei Zheng, Shibo Jin, Lingjie Liu et al. (+1 more)
Autoregressive (AR) models offer stable and efficient training, but standard next-
token prediction is not well aligned with the temporal structure required for text-conditioned motion generation. We introduce MoScale, a next-scale AR framework that generates motion hierarchically from coarse to fine temporal resolutions. By providing global semantics at the coarsest scale and refining them progressively, MoScale establishes a causal hierarchy better suited for long-range motion structure. To improve robustness under limited text-motion data, we further incorporate cross-scale hierarchical refinement for improving per-scale initial predictions and in-scale temporal refinement for selective bidirectional re-prediction. MoScale achieves SOTA text-to-motion performance with high training efficiency, scales effectively with model size, and generalizes zero-shot to diverse motion generation and editing tasks.
Read the full abstract ▾
All authors Zhiwei Zheng, Shibo Jin, Lingjie Liu, Mingmin Zhao
Computer Vision
Confidence-Driven Facade Refinement of 3D Building Models Using MLS Point Clouds
Xiaoyu Huang
Digital twins require continuous maintenance to meet the increasing demand for high-precision geospatial data. However, traditional coarse CityGML building models, typically derived from Airborne
Laser Scanning (ALS), often exhibit significant
geometric deficiencies, particularly regarding facade accuracy due to the nadir perspective of airborne sensors. Integrating these coarse models with high-precision Mobile
Laser Scanning (MLS) data is essential to recover detailed facade geometry. Unlike reconstruction-from-scratch approaches that discard existing semantic information and rely heavily on complete data coverage, this work presents an automated refinement framework that utilizes the coarse model as a
geometric prior. This method enables targeted updates to facade geometry even in complex urban environments. It integrates surface matching to identify outdated surfaces and employs a binary integer optimization to select optimal faces from candidate data. Crucially, hard constraints are enforced within the optimization to ensure the topological validity of the refined output. Experimental results demonstrate that the proposed approach effectively corrects facade misalignments, reducing the Cloud-to-Mesh RMSE by approximately 36% and achieving centimeter-level
alignment. Furthermore, the framework guarantees strictly watertight and
manifold geometry, providing a robust solution for upgrading ALS-derived city models.
Read the full abstract ▾
Computer Vision
When Does Multimodal AI Help? Diagnostic Complementarity of Vision-Language Models and CNNs for Spectrum Management in Satellite-Terrestrial Networks
Yuanhang Li
The adoption of
vision-
language models (VLMs) for wireless network management is accelerating, yet no systematic understanding exists of where these large foundation models outperform lightweight convolutional neural networks (CNNs) for spectrum-related tasks. This paper presents the first diagnostic comparison of VLMs and CNNs for spectrum heatmap understanding in non-terrestrial network and terrestrial network (NTN-TN) cooperative systems. We introduce SpectrumQA, a
benchmark comprising 108K visual question-answer pairs across four granularity levels: scene classification (L1), regional
reasoning (L2), spatial localization (L3), and semantic
reasoning (L4). Our experiments on three NTN-TN scenarios with a frozen Qwen2-VL-7B and a trained ResNet-18 reveal a clear taskdependent complementarity: CNN achieves 72.9% accuracy at severity classification (L1) and 0.552 IoU at spatial localization (L3), while VLM uniquely enables semantic
reasoning (L4) with F1=0.576 using only three in-context examples-a capability fundamentally absent in CNN architectures. Chain-of-thought (CoT) prompting further improves VLM
reasoning by 12.6% (F1: 0.209->0.233) while having zero effect on spatial tasks, confirming that the complementarity is rooted in architectural differences rather than prompting limitations. A deterministic task-type router that delegates supervised tasks to CNN and
reasoning tasks to VLM achieves a composite score of 0.616, a 39.1% improvement over CNN alone. We further show that VLM representations exhibit stronger cross-scenario robustness, with smaller performance degradation in 5 out of 6 transfer directions. These findings provide actionable guidelines: deploy CNNs for spatial localization and VLMs for semantic spectrum
reasoning, rather than treating them as substitutes.
Read the full abstract ▾
Computer Vision
M2StyleGS: Multi-Modality 3D Style Transfer with Gaussian Splatting
Xingyu Miao, Xueqi Qiu, Haoran Duan et al. (+4 more)
Conventional 3D style transfer methods rely on a fixed reference image to apply artistic patterns to 3D scenes. However, in practical applications such as virtual or augmented reality, users often prefer more flexible inputs, including textual descriptions and diverse imagery. In this work, we introduce a novel real-time styling technique M2StyleGS to generate a sequence of precisely color-mapped views. It utilizes 3D Gaussian Splatting (3DGS) as a 3D presentation and multi-modality knowledge refined by CLIP as a reference style. M2StyleGS resolves the abnormal transformation issue by employing a precise feature
alignment, namely subdivisive flow, it strengthens the projection of the mapped CLIP text-visual combination feature to the VGG style feature. In addition, we introduce observation loss, which assists in the stylized scene better matching the reference style during the generation, and suppression loss, which suppresses the offset of reference color information throughout the decoding process. By integrating these approaches, M2StyleGS can employ text or images as references to generate a set of style-enhanced novel views. Our experiments show that M2StyleGS achieves better visual quality and surpasses the previous work by up to 32.92% in terms of consistency.
Read the full abstract ▾
All authors Xingyu Miao, Xueqi Qiu, Haoran Duan, Yawen Huang, Xian Wu, Jingjing Deng, Yang Long
Computer Vision
ITIScore: An Image-to-Text-to-Image Rating Framework for the Image Captioning Ability of MLLMs
Zitong Xu, Huiyu Duan, Shengyao Qin et al. (+6 more)
Recent advances in
multimodal large
language models (MLLMs) have greatly improved image understanding and captioning capabilities. However, existing image captioning
benchmarks typically suffer from limited diversity in caption length, the absence of recent advanced MLLMs, and insufficient human annotations, which potentially introduces bias and limits the ability to comprehensively assess the performance of modern MLLMs. To address these limitations, we present a new large-scale image captioning
benchmark, termed, ICBench, which covers 12 content categories and consists of both short and long captions generated by 10 advanced MLLMs on 2K images, resulting in 40K captions in total. We conduct extensive human subjective studies to obtain mean opinion scores (MOSs) across fine-grained evaluation dimensions, where short captions are assessed in terms of fluency, relevance, and conciseness, while long captions are evaluated based on fluency, relevance, and completeness. Furthermore, we propose an automated evaluation metric, ITIScore, based on an image-to-text-to-image framework, which measures caption quality through reconstruction consistency. Experimental results demonstrate strong
alignment between our automatic metric and human judgments, as well as robust zero-shot generalization ability on other public captioning
datasets. Both the
dataset and model will be released upon publication.
Read the full abstract ▾
All authors Zitong Xu, Huiyu Duan, Shengyao Qin, Guangyu Yang, Guangji Ma, Xiongkuo Min, Ke Gu, Guangtao Zhai, Patrick Le Callet
Computer Vision · Cross-listed from cs.GR
Real-time Neural Six-way Lightmaps
Wei Li, Hanxiao Sun, Tao Huang et al. (+4 more)
Participating media are a pervasive and intriguing visual effect in virtual environments. Unfortunately, rendering such phenomena in real-time is notoriously difficult due to the computational expense of estimating the volume rendering equation. While the six-way lightmaps technique has been widely used in
video games to render smoke with a camera-oriented billboard and approximate lighting effects using six precomputed lightmaps, achieving a balance between realism and efficiency, it is limited to pre-simulated animation sequences and is ignorant of camera movement. In this work, we propose a neural six-way lightmaps method to strike a long-sought balance between dynamics and visual realism. Our approach first generates a guiding map from the camera view using ray marching with a large sampling distance to approximate smoke scattering and silhouette. Then, given a guiding map, we train a neural network to predict the corresponding six-way lightmaps. The resulting lightmaps can be seamlessly used in existing game engine pipelines. This approach supports visually appealing rendering effects while enabling real-time user interactivity, including smoke-obstacle interaction, camera movement, and light change. By conducting a series of comprehensive
benchmarks, we demonstrate that our method is well-suited for real-time applications, such as games and VR/AR.
Read the full abstract ▾
All authors Wei Li, Hanxiao Sun, Tao Huang, Haoxiang Wang, Tongtong Wang, Zherong Pan, Kui Wu
Computer Vision
Shower-Aware Dual-Stream Voxel Networks for Structural Defect Detection in Cosmic-Ray Muon Tomography
Parthiv Dasgupta, Sambhav Agarwal, Palash Dutta et al. (+2 more)
We present SA-DSVN, a 3D convolutional architecture for voxel-level
segmentation of structural defects in reinforced concrete using
cosmic-ray
muon tomography. Unlike conventional reconstruction methods (POCA, MLSD) that rely solely on
muon scattering angles, our approach jointly processes scattering kinematics (9 channels) and secondary electromagnetic shower multiplicities (40 channels) through independent encoder streams fused via cross-
attention. Training data were generated using Vega, a cloud-native Geant4 simulation framework, producing 4.5 million
muon events across 900 volumes containing four defect types - honeycombing, shear fracture, corrosion voids, and delamination - embedded within a dense 7x7 rebar cage. A five-variant ablation study demonstrates that the shower multiplicity stream alone accounts for the majority of discriminative power, raising defect-mean Dice from 0.535 (scattering only) to 0.685 (shower only). On 60 independently simulated validation volumes, the model achieves 96.3% voxel accuracy, per-defect Dice scores of 0.59-0.81, and 100% volume-level detection sensitivity at 10 ms inference per volume. These results establish secondary shower multiplicity as a previously unexploited but highly effective feature for learned
muon tomographic reconstruction.
Read the full abstract ▾
All authors Parthiv Dasgupta, Sambhav Agarwal, Palash Dutta, Raja Karmakar, Sudeshna Goswami
Computer Vision
Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation
Binyuan Huang, Yuning Lu, Weinan Jia et al. (+3 more)
Recent proprietary models such as Sora2 demonstrate promising progress in generating multi-shot
videos conditioned on multiple reference characters. However, academic research on this problem remains limited. We study this task and identify a core challenge: when reference images exhibit highly similar appearances, the model often suffers from reference confusion, where semantically similar
tokens degrade the model's ability to retrieve the correct context. To address this, we introduce PoCo (Position
Embedding as a Context Controller), which incorporates position encoding as additional context control beyond semantic
retrieval. By employing side information of
tokens, PoCo enables precise
token-level matching while preserving implicit semantic consistency modeling. Building on PoCo, we develop a multi-reference and multi-shot
video generation model capable of reliably controlling characters with extremely similar visual traits. Extensive experiments demonstrate that PoCo improves cross-shot consistency and reference fidelity compared with various baselines.
Read the full abstract ▾
All authors Binyuan Huang, Yuning Lu, Weinan Jia, Hualiang Wang, Mu Liu, Daiqing Yang
Computer Vision
SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation
Guiyu Zhang, Yabo Chen, Xunzhi Xiang et al. (+3 more)
Controlling both camera motion and object dynamics is essential for coherent and expressive
video generation, yet current methods typically handle only one motion type or rely on ambiguous 2D cues that entangle camera-induced parallax with true object movement. We present SymphoMotion, a unified motion-control framework that jointly governs camera trajectories and object dynamics within a single model. SymphoMotion features a Camera Trajectory Control mechanism that integrates explicit camera paths with geometry-aware cues to ensure stable, structurally consistent viewpoint transitions, and an Object Dynamics Control mechanism that combines 2D visual guidance with 3D trajectory
embeddings to enable depth-aware, spatially coherent object manipulation. To support large-scale training and evaluation, we further construct RealCOD-25K, a comprehensive real-world
dataset containing paired camera poses and object-level 3D trajectories across diverse indoor and outdoor scenes, addressing a key data gap in unified motion control. Extensive experiments and user studies show that SymphoMotion significantly outperforms existing methods in visual fidelity, camera controllability, and object-motion accuracy, establishing a new
benchmark for unified motion control in
video generation. Codes and data are publicly available at https://grenoble-zhang.github.io/SymphoMotion/.
Read the full abstract ▾
All authors Guiyu Zhang, Yabo Chen, Xunzhi Xiang, Junchao Huang, Zhongyu Wang, Li Jiang
Computer Vision
CGHair: Compact Gaussian Hair Reconstruction with Card Clustering
Haimin Luo, Srinjay Sarkar, Albert Mosella-Montoro et al. (+2 more)
We present a compact pipeline for high-fidelity hair reconstruction from multi-view images. While recent 3D Gaussian Splatting (3DGS) methods achieve realistic results, they often require millions of primitives, leading to high storage and rendering costs. Observing that hair exhibits structural and visual similarities across a hairstyle, we cluster strands into representative hair cards and group these into shared texture codebooks. Our approach integrates this structure with 3DGS rendering, significantly reducing reconstruction time and storage while maintaining comparable visual quality. In addition, we propose a generative prior accelerated method to reconstruct the initial strand geometry from a set of images. Our experiments demonstrate a 4-fold reduction in strand reconstruction time and achieve comparable rendering performance with over 200x lower
memory footprint.
Read the full abstract ▾
All authors Haimin Luo, Srinjay Sarkar, Albert Mosella-Montoro, Francisco Vicente Carrasco, Fernando De la Torre
Computer Vision
Learning Superpixel Ensemble and Hierarchy Graphs for Melanoma Detection
Asmaa M. Elwer, Muhammad A. Rushdi and Mahmoud H. Annaby
Graph signal processing (GSP) is becoming a major tool in biomedical signal and image analysis. In most GSP techniques, graph structures and edge weights have been typically set via statistical and computational methods. More recently, graph structure learning methods offered more reliable and flexible data representations. In this work, we introduce a graph learning approach for melanoma detection in dermoscopic images based on two graph-theoretic representations: superpixel ensemble graphs (SEG) and superpixel hierarchy graphs (SHG). For these two types of graphs, superpixel maps of a skin lesion image are respectively generated at multiple levels without and with parentchild constraints among superpixels at adjacent levels, where each level corresponds to a subgraph with a different number of nodes (20, 40, 60, 80, or 100 nodes). Two edge weight assignment techniques are explored: handcrafted Gaussian weights and learned weights based on optimization methods. The graph nodal signals are assigned based on texture,
geometric, and color superpixel features. In addition, the effect of graph edge thresholding is investigated by applying different thresholds (25%, 50%, and 75%) to prune the weakest edges and analyze the impact of
pruning on the melanoma detection performance. Experimental evaluation of the proposed method is performed with different classifiers trained and tested on the publicly available ISIC2017
dataset. Data augmentation is applied to alleviate class imbalance by adding more melanoma images from the ISIC archive. The results show that learned superpixel ensemble graphs with textural nodal signals give the highest performance reaching an accuracy of 99.00% and an AUC of 99.59%.
Read the full abstract ▾
Computer Vision
XSeg: A Large-scale X-ray Contraband Segmentation Benchmark For Real-World Security Screening
Hongxia Gao, Litao Li, Yixin Chen et al. (+3 more)
X-ray contraband detection is critical for public safety. However, current methods primarily rely on bounding box annotations, which limit model generalization and performance due to the lack of pixel-level supervision and real-world data. To address these limitations, we introduce XSeg. To the best of our knowledge, XSeg is the largest X-ray contraband
segmentation dataset to date, including 98,644 images and 295,932 instance masks, and contains the latest 30 common contraband categories. The images are sourced from public
datasets and our synthesized data, filtered through a custom data cleaning pipeline to remove low-quality samples. To enable accurate and efficient annotation and reduce manual labeling effort, we propose Adaptive Point SAM (APSAM), a specialized mask annotation model built upon the Segment Anything Model (SAM). We address SAM's poor cross-domain generalization and limited capability in detecting stacked objects by introducing an Energy-Aware Encoder that enhances the initialization of the mask decoder, significantly improving sensitivity to overlapping items. Additionally, we design an Adaptive Point Generator that allows users to obtain precise mask labels with only a single coarse point
prompt. Extensive experiments on XSeg demonstrate the superior performance of APSAM.
Read the full abstract ▾
All authors Hongxia Gao, Litao Li, Yixin Chen, Jiali Wen, Kaijie Zhang, Qianyun Liu
Computer Vision
VidNum: Diagnosing VLM Failure Modes in Video-Grounded Numerical Reasoning
Shaoyang Cui, Lingbei Meng, Yaodi Luo et al. (+1 more)
Video-grounded numerical
reasoning requires
Vision-
Language Models (VLMs) to identify, track, and combine quantitative evidence across frames, actions, and scene changes. Existing
benchmarks provide fragmented coverage: general VideoQA includes counting among broader tasks, while dedicated
benchmarks focus on repetition counting, ultra-long-
video enumeration, or instructional mathematics. We introduce VidNum, a manually curated and independently verified
benchmark containing 1,167 multiple-choice questions. Its three task groups distinguish Direct and Distinct Enumeration, Conditioned and Structured Enumeration, and Compositional Quantitative
Reasoning. Question-level annotations further identify the evidence target, counting structure, and required
reasoning operation. The best evaluated VLM reaches 59.8% accuracy, compared with 98.2% for human annotators, and no evaluated open-weight model exceeds 45%. Stratified analyses reveal that failures are not uniformly distributed: structured target construction and action-grounded compositional
reasoning form recurring bottlenecks across models. Zero-shot chain-of-thought prompting is not a reliable remedy: it recovers some errors but breaks previously correct answers, with effects that vary across models and task structures. VidNum therefore supports diagnostic analysis beyond a single aggregate score.
Read the full abstract ▾
All authors Shaoyang Cui, Lingbei Meng, Yaodi Luo, Peize He
Computer Vision
SGTA: Scene-Graph Based Multi-Modal Traffic Agent for Video Understanding
Xingcheng Zhou, Mingyu Liu, Walter Zimmer et al. (+2 more)
We present Scene-Graph Based Multi-Modal Traffic
Agent (SGTA), a modular framework for traffic
video understanding that combines structured scene graphs with multi-modal
reasoning. It constructs a traffic scene graph from roadside
videos using detection, tracking, and lane extraction, followed by tool-based
reasoning over both symbolic graph queries and visual inputs. SGTA adopts ReAct to process interleaved
reasoning traces from large
language models with tool invocations, enabling interpretable decision-making for complex
video questions. Experiments on selected TUMTraffic VideoQA
dataset sample demonstrate that SGTA achieves competitive accuracy across multiple question types while providing transparent
reasoning steps. These results highlight the potential of integrating structured scene representations with multi-modal
agents for traffic
video understanding.
Read the full abstract ▾
All authors Xingcheng Zhou, Mingyu Liu, Walter Zimmer, Jiajie Zhang, Alois Knoll
Computer Vision
FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning
Zhengyu Fu, René Zurbrügg, Kaixian Qu et al. (+4 more)
Recent work in 3D scene understanding is moving beyond purely spatial analysis toward functional scene understanding. However, existing methods often consider functional relationships between object pairs in isolation, failing to capture the scene-wide interdependence that humans use to resolve ambiguity. We introduce FunFact, a framework for constructing probabilistic open-vocabulary functional 3D scene graphs from posed RGB-D images. FunFact first builds an object- and part-centric 3D map and uses foundation models to propose semantically plausible functional relations. These candidates are converted into factor graph variables and constrained by both LLM-derived common-sense priors and
geometric priors. This formulation enables joint probabilistic inference over all functional edges and their marginals, yielding substantially better calibrated confidence scores. To
benchmark this setting, we introduce FunThor, a synthetic
dataset based on AI2-THOR with part-level geometry and rule-based functional annotations. Experiments on SceneFun3D, FunGraph3D, and FunThor show that FunFact improves node and relation discovery recall and significantly reduces calibration error for ambiguous relations, highlighting the benefits of holistic probabilistic modeling for functional scene understanding. See our project page at https://funfact-scenegraph.github.io/
Read the full abstract ▾
All authors Zhengyu Fu, René Zurbrügg, Kaixian Qu, Marc Pollefeys, Marco Hutter, Hermann Blum, Zuria Bauer
Computer Vision
ResGuard: Enhancing Robustness Against Known Original Attacks in Deep Watermarking
Hanyi Wang, Han Fang, Yupeng Qiu et al. (+2 more)
Deep learning-based image watermarking commonly adopts an "Encoder-Noise Layer-Decoder" (END) architecture to improve robustness against random channel distortions, yet it often overlooks intentional manipulations introduced by adversaries with additional knowledge. In this paper, we revisit this paradigm and expose a critical yet underexplored vulnerability: the Known Original Attack (KOA), where an adversary has access to multiple original-watermarked image pairs, enabling various targeted suppression strategies. We show that even a simple residual-based removal approach, namely estimating an
embedding residual from known pairs and subtracting it from unseen watermarked images, can almost completely remove the watermark while preserving visual quality. This vulnerability stems from the insufficient image dependency of residuals produced by END frameworks, which makes them transferable across images. To address this, we propose ResGuard, a plug-and-play module that enhances KOA robustness by enforcing image-dependent
embedding. Its core lies in a residual specificity enhancement loss, which encourages residuals to be tightly coupled with their host images and thus improves image dependency. Furthermore, an auxiliary KOA noise layer injects residual-style perturbations during training, allowing the decoder to remain reliable under stronger
embedding inconsistencies. Integrated into existing frameworks, ResGuard boosts KOA robustness, improving average watermark extraction accuracy from 59.87% to 99.81%.
Read the full abstract ▾
All authors Hanyi Wang, Han Fang, Yupeng Qiu, Shilin Wang, Ee-Chien Chang
Computer Vision
SciLT: Long-tailed Image Classification under Scientific Image Domains
Jiahao Chen and Bing Su
Long-tailed recognition has benefited from foundation models and fine-tuning paradigms, yet existing studies and
benchmarks are mainly confined to natural image domains, where pre-training and fine-tuning data share similar distributions. In contrast, scientific images exhibit distinct visual characteristics and supervision signals, raising questions about the effectiveness of fine-tuning foundation models in such settings. In this work, we investigate scientific long-tailed recognition under a purely visual and fine-tuning paradigm. Experiments on three scientific
benchmarks show that fine-tuning foundation models yields limited gains, and reveal that penultimate-layer features play an important role, particularly for tail classes. Motivated by these findings, we propose SciLT, a framework that exploits multi-level representations through adaptive feature fusion and dual-supervision learning. By jointly leveraging penultimate- and final-layer features, SciLT achieves balanced performance across head and tail classes. Extensive experiments demonstrate that SciLT consistently outperforms existing methods, establishing a strong and practical baseline for scientific long-tailed recognition and providing valuable guidance for adapting foundation models to scientific data with substantial domain shifts.
Read the full abstract ▾
Computer Vision
DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR
Hoonhee Cho, Jae-Young Kang, Yuhwan Jeong et al. (+4 more)
In this paper, we present DSERT-RoLL, a driving
dataset that incorporates stereo event, RGB, and thermal cameras together with 4D radar and dual
LiDAR, collected across diverse weather and illumination conditions. The
dataset provides precise 2D and 3D bounding boxes with track IDs and ego vehicle odometry, enabling fair comparisons within and across sensor combinations. It is designed to alleviate data scarcity for novel sensors such as event cameras and 4D radar and to support systematic studies of their behavior. We establish unified 3D and 2D
benchmarks that enable direct comparison of characteristics and strengths across sensor families and within each family. We report baselines for representative single modality and
multimodal methods and provide protocols that encourage research on different fusion strategies and sensor combinations. In addition, we propose a fusion framework that integrates sensor specific cues into a unified feature space and improves 3D detection robustness under varied weather and lighting.
Read the full abstract ▾
All authors Hoonhee Cho, Jae-Young Kang, Yuhwan Jeong, Yunseo Yang, Wonyoung Lee, Youngho Kim, Kuk-Jin Yoon
Computer Vision
Haowei Zhu, Ji Liu, Ziqiong Liu et al. (+4 more)
Diffusion models demonstrate outstanding performance in image generation, but their multi-step inference mechanism requires immense computational cost. Previous works accelerate inference by leveraging layer or
token cache techniques to reduce computational cost. However, these methods fail to achieve superior acceleration performance in few-step
diffusion transformer models due to inefficient feature caching strategies, manually designed sparsity allocation, and the practice of retaining complete forward computations in several steps in these
token cache methods. To tackle these challenges, we propose a differentiable layer-wise sparsity optimization framework for
diffusion transformer models, leveraging
token caching to reduce
token computation costs and enhance acceleration. Our method optimizes layer-wise sparsity allocation in an end-to-end manner through a learnable network combined with a dynamic programming solver. Additionally, our proposed two-stage training strategy eliminates the need for full-step processing in existing methods, further improving efficiency. We conducted extensive experiments on a range of
diffusion-
transformer models, including DiT-XL/2, PixArt-$\alpha$, FLUX, and Wan2.1. Across these architectures, our method consistently improves efficiency without degrading sample quality. For example, on PixArt-$\alpha$ with 20 sampling steps, we reduce computational cost by $54\%$ while achieving generation metrics that surpass those of the original model, substantially outperforming prior approaches. These results demonstrate that our method delivers large efficiency gains while often improving generation quality.
Read the full abstract ▾
All authors Haowei Zhu, Ji Liu, Ziqiong Liu, Dong Li, Junhai Yong, Bin Wang, Emad Barsoum
Computer Vision
Leveraging Gaze and Set-of-Mark in VLLMs for Human-Object Interaction Anticipation from Egocentric Videos
Daniele Materia, Francesco Ragusa and Giovanni Maria Farinella
The ability to anticipate human-object interactions is highly desirable in an intelligent assistive system in order to guide users during daily life activities and understand their short and long-term goals. Creating systems with such capabilities requires to approach several complex challenges. This work addresses the problem of human-object interaction anticipation in
Egocentric Vision using
Vision Large
Language Models (VLLMs). We tackle key limitations in existing approaches by improving visual grounding capabilities through Set-of-Mark prompting and understanding user intent via the trajectory formed by the user's most recent gaze fixations. To effectively capture the temporal dynamics immediately preceding the interaction, we further introduce a novel inverse exponential sampling strategy for input
video frames. Experiments conducted on the
egocentric dataset HD-EPIC demonstrate that our method surpasses state-of-the-art approaches for the considered task, showing its model-agnostic nature.
Read the full abstract ▾
Computer Vision
Love Me, Love My Label: Rethinking the Role of Labels in Prompt Retrieval for Visual In-Context Learning
Tianci Luo, Haohao Pan, Jinpeng Wang et al. (+5 more)
Visual in-context learning (VICL) enables visual foundation models to handle multiple tasks by steering them with demonstrative
prompts. The choice of such
prompts largely influences VICL performance, standing out as a key challenge. Prior work has made substantial progress on
prompt retrieval and reranking strategies, but mainly focuses on
prompt images while overlooking labels. We reveal these approaches sometimes get visually similar but label-inconsistent
prompts, which potentially degrade VICL performance. On the other hand, higher label consistency between query and
prompts preferably indicates stronger VICL results. Motivated by these findings, we develop a framework named LaPR (Label-aware
Prompt Retrieval), which highlights the role of labels in
prompt selection. Our framework first designs an image-label joint representation for
prompts to incorporate label cues explicitly. Besides, to handle unavailable query labels at test time, we introduce a mixture-of-expert mechanism to the dual encoders with query-adaptive routing. Each expert is expected to capture a specific label mode, while the router infers query-adaptive mixture weights and helps to learn label-aware representation. We carefully design alternative optimization for experts and router, with a VICL performance-guided contrastive loss and a label-guided contrastive loss, respectively. Extensive experiments show promising and consistent improvement of LaPR on in-context
segmentation, detection, and colorization tasks. Moreover, LaPR generalizes well across feature extractors and cross-fold scenarios, suggesting the importance of label utilization in
prompt retrieval for VICL. Code is available at https://github.com/luotc-why/CVPR26-LaPR.
Read the full abstract ▾
All authors Tianci Luo, Haohao Pan, Jinpeng Wang, Niu Lian, Xinrui Chen, Bin Chen, Shu-Tao Xia, Chun Yuan
Computer Vision
Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval
Jun Li, Xuhang Lou, Jinpeng Wang et al. (+4 more)
Partially Relevant
Video Retrieval (PRVR) aims to retrieve untrimmed
videos based on text queries that describe only partial events. Existing methods suffer from incomplete global contextual perception, struggling with query ambiguity and local noise induced by spurious responses. To address these issues, we propose DreamPRVR, which adopts a coarse-to-fine representation learning paradigm. The model first generates global contextual semantic registers as coarse-grained highlights spanning the entire
video and then concentrates on fine-grained similarity optimization for precise cross-modal matching. Concretely, these registers are generated by initializing from the
video-centric distribution produced by a probabilistic variational sampler and then iteratively refined via a text-supervised truncated
diffusion model. During this process, textual semantic structure learning constructs a well-formed textual latent space, enhancing the reliability of global perception. The registers are then adaptively fused with
video tokens through register-augmented Gaussian
attention blocks, enabling context-aware feature learning. Extensive experiments show that DreamPRVR outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/CVPR26-DreamPRVR.
Read the full abstract ▾
All authors Jun Li, Xuhang Lou, Jinpeng Wang, Yuting Wang, Yaowei Wang, Shu-Tao Xia, Bin Chen
Computer Vision
Motion-Adaptive Multi-Scale Temporal Modelling with Skeleton-Constrained Spatial Graphs for Efficient 3D Human Pose Estimation
Ruochen Li, Shuang Chen, Wenke E et al. (+2 more)
Accurate 3D human pose estimation from
monocular videos requires effective modelling of complex spatial and temporal dependencies. However, existing methods often face challenges in efficiency and adaptability when modelling spatial and temporal dependencies, particularly under dense
attention or fixed modelling schemes. In this work, we propose MASC-Pose, a Motion-Adaptive multi-scale temporal modelling framework with Skeleton-Constrained spatial graphs for efficient 3D human pose estimation. Specifically, it introduces an Adaptive Multi-scale Temporal Modelling (AMTM) module to adaptively capture heterogeneous motion dynamics at different temporal scales, together with a Skeleton-constrained Adaptive GCN (SAGCN) for joint-specific spatial interaction modelling. By jointly enabling adaptive temporal
reasoning and efficient spatial aggregation, our method achieves strong accuracy with high computational efficiency. Extensive experiments on Human3.6M and MPI-INF-3DHP
datasets demonstrate the effectiveness of our approach.
Read the full abstract ▾
All authors Ruochen Li, Shuang Chen, Wenke E, Farshad Arvin, Amir Atapour-Abarghouei
Computer Vision
ART: Adaptive Relational Transformer for Pedestrian Trajectory Prediction with Temporal-Aware Relations
Ruochen Li, Ziyi Chang, Junyan Hu et al. (+3 more)
Accurate prediction of real-world pedestrian trajectories is crucial for a wide range of
robot-related applications. Recent approaches typically adopt graph-based or
transformer-based frameworks to model interactions. Despite their effectiveness, these methods either introduce unnecessary computational overhead or struggle to represent the diverse and time-varying characteristics of human interactions. In this work, we present an Adaptive Relational
Transformer (ART), which introduces a Temporal-Aware Relation Graph (TARG) to explicitly capture the
evolution of pairwise interactions and an Adaptive Interaction
Pruning (AIP) mechanism to reduce redundant computations efficiently. Extensive evaluations on ETH/UCY and NBA
benchmarks show that ART delivers state-of-the-art accuracy with high computational efficiency.
Read the full abstract ▾
All authors Ruochen Li, Ziyi Chang, Junyan Hu, Jiannan Li, Amir Atapour-Abarghouei, Hubert P. H. Shum
Computer Vision
Stabilizing Unsupervised Self-Evolution of MLLMs via Continuous Softened Retracing reSampling
Yunyao Yu, Zhengxian Wu, Zhuohong Chen et al. (+6 more)
In the unsupervised self-
evolution of
Multimodal Large
Language Models, the quality of feedback signals during post-training is pivotal for stable and effective learning. However, existing self-
evolution methods predominantly rely on majority voting to select the most frequent output as the pseudo-golden answer, which may stem from the model's intrinsic biases rather than guaranteeing the objective correctness of the
reasoning paths. To counteract the degradation, we propose Continuous Softened Retracing reSampling (CSRS) in MLLM self-
evolution. Specifically, we introduce a Retracing Re-inference Mechanism (RRM) that the model re-inferences from anchor points to expand the exploration of long-tail
reasoning paths. Simultaneously, we propose Softened Frequency Reward (SFR), which replaces binary rewards with continuous signals, calibrating reward based on the answers' frequency across sampled
reasoning sets. Furthermore, incorporated with Visual Semantic Perturbation (VSP), CSRS ensures the model prioritizes mathematical logic over visual superficiality. Experimental results demonstrate that CSRS significantly enhances the
reasoning performance of Qwen2.5-VL-7B on
benchmarks such as MathVision. We achieve state-of-the-art (SOTA) results in unsupervised self-
evolution on
geometric tasks. Our code is avaible at https://github.com/yyy195/CSRS.
Read the full abstract ▾
All authors Yunyao Yu, Zhengxian Wu, Zhuohong Chen, Hangrui Xu, Zirui Liao, Xiangwen Deng, Zhifang Liu, Senyuan Shi, Haoqian Wang
Computer Vision · Cross-listed from eess.IV
UniSurgSAM: A Unified Promptable Model for Reliable Surgical Video Segmentation
Haofeng Liu, Ziyue Wang, Alex Y. W. Kong et al. (+6 more)
Surgical video segmentation is fundamental to computer-assisted surgery. In practice, surgeons need to dynamically specify targets throughout extended procedures, using heterogeneous cues such as visual selections, textual expressions, or audio instructions. However, existing Promptable
Video Object
Segmentation (PVOS) methods are typically restricted to a single
prompt modality and rely on coupled frameworks that cause optimization interference between target initialization and tracking. Moreover, these methods produce hallucinated predictions when the target is absent and suffer from accumulated mask drift without failure recovery. To address these challenges, we present UniSurgSAM, a unified PVOS model enabling reliable
surgical video segmentation through visual, textual, or audio
prompts. Specifically, UniSurgSAM employs a decoupled two-stage framework that independently optimizes initialization and tracking to resolve the optimization interference. Within this framework, we introduce three key designs for reliability: presence-aware decoding that models target absence to suppress
hallucinations; boundary-aware long-term tracking that prevents mask drift over extended sequences; and adaptive state transition that closes the loop between stages for failure recovery. Furthermore, we establish a multi-modal and multi-granular
benchmark from four public
surgical datasets with precise instance-level masklets. Extensive experiments demonstrate that UniSurgSAM achieves state-of-the-art performance in real time across all
prompt modalities and granularities, providing a practical foundation for computer-assisted surgery. Code and
datasets will be available at https://jinlab-imvr.github.io/UniSurgSAM.
Read the full abstract ▾
All authors Haofeng Liu, Ziyue Wang, Alex Y. W. Kong, Guanyi Qin, Yunqiu Xu, Chang Han Low, Mingqi Gao, Lap Yan Lennon Chan, Yueming Jin
Computer Vision
ComPrivDet: Efficient Privacy Object Detection in Compressed Domains Through Inference Reuse
Yunhao Yao, Zhiqiang Wang, Ruiqi Li et al. (+3 more)
As the Internet of Things (IoT) becomes deeply embedded in daily life, users are increasingly concerned about
privacy leakage, especially from
video data. Since frame-by-frame protection in large-scale
video analytics (e.g., smart communities) introduces significant latency, a more efficient solution is to selectively protect frames containing
privacy objects (e.g., faces). Existing object detectors require fully decoded
videos or per-frame processing in compressed
videos, leading to decoding overhead or reduced accuracy. Therefore, we propose ComPrivDet, an efficient method for detecting
privacy objects in compressed
video by reusing I-frame inference results. By identifying the presence of new objects through compressed-domain cues, ComPrivDet either skips P- and B-frame detections or efficiently refines them with a lightweight detector. ComPrivDet maintains 99.75% accuracy in private face detection and 96.83% in private license plate detection while skipping over 80% of inferences. It averages 9.84% higher accuracy with 75.95% lower latency than existing compressed-domain detection methods.
Read the full abstract ▾
All authors Yunhao Yao, Zhiqiang Wang, Ruiqi Li, Haoran Cheng, Puhan Luo, Xiangyang Li
Computer Vision
SAGE-GAN: Towards Realistic and Robust Segmentation of Spatially Ordered Nanoparticles via Attention-Guided GANs
Anindya Pal, Varun Ajith, Saumik Bhattacharya et al. (+1 more)
Precise analysis of nanoparticles for characterization in electron microscopy images is essential for advancing nanomaterial development. Yet it remains challenging due to the time-consuming nature of manual methods and the shortcomings of traditional automated
segmentation techniques, especially when dealing with complex shapes and imaging artifacts. While conventional methods yield promising results, they depend on a large volume of labeled training data, which is both difficult to acquire and highly time-consuming to generate. In order to overcome these challenges, we have developed a two-step solution: Firstly, our system learns to segment the key features of nanoparticles from a
dataset of real images using a self-
attention driven U-Net architecture that focuses on important physical and morphological details while ignoring background features and noise. Secondly, this trained
Attention U-Net is embedded in a cycle-consistent generative
adversarial network (CycleGAN) framework, inspired by the cGAN-Seg model introduced by Abzargar et al. This integration allows for the creation of highly realistic synthetic electron microscopy image-mask pairs that naturally reflect the structural patterns learned by the
Attention U-Net. Consequently, the model can accurately detect features in a diverse array of real-world nanoparticle images and autonomously augment the training
dataset without requiring human input. Cycle consistency enforces a direct correspondence between synthetic images and ground-truth masks, ensuring realistic features, which is crucial for accurate
segmentation training.
Read the full abstract ▾
All authors Anindya Pal, Varun Ajith, Saumik Bhattacharya, Sayantari Ghosh