Computer Vision
VC-Bench: Pioneering the Video Connecting Benchmark with a Dataset and Evaluation Metrics
Zhiyu Yin, Zhipeng Liu, Kehai Chen et al. (+5 more)
While current
video generation focuses on text or image conditions, practical applications like
video editing and vlogging often need to seamlessly connect separate clips. In our work, we introduce
Video Connecting, an innovative task that aims to generate smooth intermediate
video content between given start and end clips. However, the absence of standardized evaluation
benchmarks has hindered the development of this task. To bridge this gap, we proposed VC-Bench, a novel
benchmark specifically designed for
video connecting. It includes 1,579 high-quality
videos collected from public platforms, covering 15 main categories and 72 subcategories to ensure diversity and structure. VC-Bench focuses on three core aspects:
Video Quality Score VQS, Start-End Consistency Score SECS, and Transition Smoothness Score TSS. Together, they form a comprehensive framework that moves beyond conventional quality-only metrics. We evaluated multiple state-of-the-art
video generation models on VC-Bench. Experimental results reveal significant limitations in maintaining start-end consistency and transition smoothness, leading to lower overall coherence and fluidity. We expect that VC-Bench will serve as a pioneering
benchmark to inspire and guide future research in
video connecting. The evaluation metrics and
dataset are publicly available at: https://anonymous.4open.science/r/VC-Bench-1B67/.
Read the full abstract ▾
All authors Zhiyu Yin, Zhipeng Liu, Kehai Chen, Lemao Liu, Jin Liu, Hong-Dong Li, Yang Xiang, Min Zhang
Computer Vision
Towards Pixel-Level VLM Perception via Simple Points Prediction
Tianhui Song, Haoyu Lu, Hao Yang et al. (+8 more)
We present SimpleSeg, a strikingly simple yet highly effective approach to endow
Multimodal Large
Language Models (MLLMs) with native pixel-level perception. Our method reframes
segmentation as a simple sequence generation problem: the model directly predicts sequences of points (textual coordinates) delineating object boundaries, entirely within its
language space. To achieve high fidelity, we introduce a two-stage SF$\to$RL training pipeline, where Reinforcement Learning with an IoU-based reward refines the point sequences to accurately match ground-truth contours. We find that the standard MLLM architecture possesses a strong, inherent capacity for low-level perception that can be unlocked without any specialized architecture. On
segmentation benchmarks, SimpleSeg achieves performance that is comparable to, and often surpasses, methods relying on complex, task-specific designs. This work lays out that precise spatial understanding can emerge from simple point prediction, challenging the prevailing need for auxiliary components and paving the way for more unified and capable VLMs. Homepage: https://simpleseg.github.io/
Read the full abstract ▾
All authors Tianhui Song, Haoyu Lu, Hao Yang, Lin Sui, Haoning Wu, Zaida Zhou, Zhiqi Huang, Yiping Bao, Y.Charles, Xinyu Zhou, Limin Wang
Computer Vision
UniPCB: A Unified Vision-Language Benchmark for Open-Ended PCB Quality Inspection
Fuxiang Sun, Xi Jiang, Jiansheng Wu et al. (+3 more)
Multimodal Large
Language Models (MLLMs) show promise for general industrial quality inspection, but fall short in complex scenarios, such as Printed
Circuit Board (PCB) inspection. PCB inspection poses unique challenges due to densely packed components, complex wiring structures, and subtle defect patterns that require specialized domain expertise. However, a high-quality, unified
vision-
language benchmark for quantitatively evaluating MLLMs across PCB inspection tasks remains absent, stemming not only from limited data availability but also from fragmented
datasets and inconsistent standardization. To fill this gap, we propose UniPCB, the first unified
vision-
language benchmark for open-ended PCB quality inspection. UniPCB is built via a systematic pipeline that curates and standardizes data from disparate sources across three annotated scenarios. Furthermore, we introduce PCB-GPT, an MLLM trained on a new instruction
dataset generated by this pipeline, utilizing a novel progressive curriculum that mimics the learning process of human experts. Evaluations on the UniPCB
benchmark show that while existing MLLMs falter on domain-specific tasks, PCB-GPT establishes a new baseline. Notably, it more than doubles the performance on fine-grained defect localization compared to the strongest competitors, with significant advantages in localization and analysis. We will release the instruction data,
benchmark, and model to facilitate future research.
Read the full abstract ▾
All authors Fuxiang Sun, Xi Jiang, Jiansheng Wu, Haigang Zhang, Feng Zheng, Jinfeng Yang
Computer Vision · Cross-listed from cs.NI
Bridging Visual and Wireless Sensing via a Unified Radiation Field for 3D Radio Map Construction
Chaozheng Wen, Jingwen Tong, Zehong Lin et al. (+2 more)
The emerging applications of next-generation wireless networks demand high-fidelity environmental intelligence. 3D
radio maps bridge physical environments and electromagnetic propagation for spectrum planning and environment-aware sensing. However, most existing methods treat visual and wireless data as independent modalities and fail to leverage shared electromagnetic propagation principles. To bridge this gap, we propose URF-GS, a unified
radio-optical radiation field framework based on 3D Gaussian splatting and inverse rendering for 3D
radio map construction. By fusing cross-modal observations, our method recovers scene geometry and material properties to predict
radio signals under arbitrary transceiver configurations without retraining. Experiments demonstrate up to a 24.7% improvement in spatial spectrum accuracy and a 10x increase in sample efficiency compared with NeRF-based methods. We further showcase URF-GS in Wi-Fi AP deployment and
robot path planning tasks. This unified visual-wireless representation supports holistic radiation field modeling for future wireless communication systems.
Read the full abstract ▾
All authors Chaozheng Wen, Jingwen Tong, Zehong Lin, Chenghong Bian, Jun Zhang
Computer Vision
Contrastive Spectral Rectification: Test-Time Defense towards Zero-shot Adversarial Robustness of CLIP
Sen Nie, Jie Zhang, Zhuo Wang et al. (+2 more)
Vision-
language models (VLMs) such as CLIP have demonstrated remarkable zero-shot generalization, yet remain highly vulnerable to
adversarial examples (AEs). While test-time defenses are promising, existing methods fail to provide sufficient robustness against strong attacks and are often hampered by high inference latency and task-specific applicability. To address these limitations, we start by investigating the intrinsic properties of AEs, which reveals that AEs exhibit severe feature inconsistency under progressive frequency attenuation. We further attribute this to the model's inherent spectral bias. Leveraging this insight, we propose an efficient test-time defense named Contrastive Spectral Rectification (CSR). CSR optimizes a rectification perturbation to realign the input with the natural
manifold under a spectral-guided contrastive objective, which is applied input-adaptively. Extensive experiments across 16 classification
benchmarks demonstrate that CSR outperforms the SOTA by an average of 18.1% against strong APGD with modest inference overhead. Furthermore, CSR exhibits broad applicability across diverse visual tasks. Code is available at https://github.com/Summu77/CSR.
Read the full abstract ▾
All authors Sen Nie, Jie Zhang, Zhuo Wang, Shiguang Shan, Xilin Chen
Computer Vision · Cross-listed from cs.AI
MATA: A Trainable Hierarchical Automaton System for Multi-Agent Visual Reasoning
Zhixi Cai, Fucai Ke, Kevin Leo et al. (+4 more)
Recent
vision-
language models have strong perceptual ability but their implicit
reasoning is hard to explain and easily generates
hallucinations on complex queries. Compositional methods improve interpretability, but most rely on a single
agent or hand-crafted pipeline and cannot decide when to collaborate across complementary
agents or compete among overlapping ones. We introduce MATA (Multi-
Agent hierarchical Trainable Automaton), a multi-
agent system presented as a hierarchical finite-state automaton for visual
reasoning whose top-level transitions are chosen by a trainable hyper
agent. Each
agent corresponds to a state in the hyper automaton, and runs a small rule-based sub-automaton for reliable micro-control. All
agents read and write a shared
memory, yielding transparent execution history. To supervise the hyper
agent's transition policy, we build transition-trajectory trees and transform to
memory-to-next-state pairs, forming the MATA-SFT-90K
dataset for supervised finetuning (SFT). The finetuned LLM as the transition policy understands the query and the capacity of
agents, and it can efficiently choose the optimal
agent to solve the task. Across multiple visual
reasoning benchmarks, MATA achieves the state-of-the-art results compared with monolithic and compositional baselines. The code and
dataset are available at https://github.com/ControlNet/MATA.
Read the full abstract ▾
All authors Zhixi Cai, Fucai Ke, Kevin Leo, Sukai Huang, Maria Garcia de la Banda, Peter J. Stuckey, Hamid Rezatofighi
Computer Vision
SNR-Edit: Structure-Aware Noise Rectification for Inversion-Free Flow-Based Editing
Lifan Jiang, Boxi Wu, Yuhang Pei et al. (+5 more)
Inversion-free image editing using flow-based generative models challenges the prevailing inversion-based pipelines. However, existing approaches rely on fixed Gaussian noise to construct the source trajectory, leading to biased trajectory dynamics and causing structural degradation or quality loss. To address this, we introduce SNR-Edit, a training-free framework achieving faithful Latent Trajectory Correction via adaptive noise control. Mechanistically, SNR-Edit uses structure-aware noise rectification to inject
segmentation constraints into the initial noise, anchoring the
stochastic component of the source trajectory to the real image's implicit inversion position and reducing trajectory drift during source–target transport. This lightweight modification yields smoother latent trajectories and ensures high-fidelity structural preservation without requiring model tuning or inversion. Across SD3 and FLUX, evaluations on PIE-Bench and SNR-Bench show that SNR-Edit delivers performance on pixel-level metrics and VLM-based scoring, while adding only about 1s overhead per image.
Read the full abstract ▾
All authors Lifan Jiang, Boxi Wu, Yuhang Pei, Tianrun Wu, Yongyuan Chen, Yan Zhao, Shiyu Yu, Deng Cai
Computer Vision
GTFMN: Guided Texture and Feature Modulation Network for Low-Light Image Enhancement and Super-Resolution
Yongsong Huang, Tzu-Hsuan Peng, Tomo Miyazaki et al. (+4 more)
Low-light image super-resolution (LLSR) is a challenging task due to the coupled degradation of low resolution and poor illumination. To address this, we propose the Guided Texture and Feature Modulation Network (GTFMN), a novel framework that decouples the LLSR task into two sub-problems: illumination estimation and texture restoration. First, our network employs a dedicated Illumination Stream whose purpose is to predict a spatially varying illumination map that accurately captures lighting distribution. Further, this map is utilized as an explicit guide within our novel Illumination Guided Modulation Block (IGM Block) to dynamically modulate features in the Texture Stream. This mechanism achieves spatially adaptive restoration, enabling the network to intensify enhancement in poorly lit regions while preserving details in well-exposed areas. Extensive experiments demonstrate that GTFMN achieves the best performance among competing methods on the OmniNormal5 and OmniNormal15
datasets, outperforming them in both quantitative metrics and visual quality.
Read the full abstract ▾
All authors Yongsong Huang, Tzu-Hsuan Peng, Tomo Miyazaki, Xiaofeng Liu, Chun-Ting Chou, Ai-Chun Pang, Shinichiro Omachi
Computer Vision · Cross-listed from cs.AI
LocationAgent: A Hierarchical Agent for Image Geolocation via Decoupling Strategy and Evidence from Parametric Knowledge
Qiujun Li, Zijin Xiao, Xulin Wang et al. (+3 more)
Image geolocation aims to infer capture locations based on visual content. Fundamentally, this constitutes a
reasoning process composed of hypothesis-verification cycles, requiring models to possess both geospatial
reasoning capabilities and the ability to verify evidence against geographic facts. Existing methods typically internalize location knowledge and
reasoning patterns into static
memory via supervised training or trajectory-based reinforcement fine-tuning. Consequently, these methods are prone to factual
hallucinations and generalization bottlenecks in open-world settings or scenarios requiring dynamic knowledge. To address these challenges, we propose a Hierarchical Localization
Agent, called LocationAgent. Our core philosophy is to retain hierarchical
reasoning logic within the model while offloading the verification of geographic evidence to external tools. To implement hierarchical
reasoning, we design the RER architecture (Reasoner-Executor-Recorder), which employs role separation and context compression to prevent the drifting problem in multi-step
reasoning. For evidence verification, we construct a suite of clue exploration tools that provide diverse evidence to support location
reasoning. Furthermore, to address data leakage and the scarcity of Chinese data in existing
datasets, we introduce CCL-Bench (China City Location Bench), an image geolocation
benchmark encompassing various scene granularities and difficulty levels. Extensive experiments demonstrate that LocationAgent significantly outperforms existing methods by at least 30% in zero-shot settings.
Read the full abstract ▾
All authors Qiujun Li, Zijin Xiao, Xulin Wang, Zhidan Ma, Cheng Yang, Haifeng Li
Computer Vision
TFFM: Topology-Aware Feature Fusion Module via Latent Graph Reasoning for Retinal Vessel Segmentation
Iftekhar Ahmed, Shakib Absar, Aftar Ahmad Sami et al. (+3 more)
Precise
segmentation of retinal arteries and veins carries the diagnosis of systemic cardiovascular conditions. However, standard convolutional architectures often yield topologically disjointed
segmentations, characterized by gaps and discontinuities that render reliable graph-based clinical analysis impossible despite high pixel-level accuracy. To address this, we introduce a
topology-aware framework engineered to maintain vascular connectivity. Our architecture fuses a Topological Feature Fusion Module (TFFM) that maps local feature representations into a latent graph space, deploying Graph
Attention Networks to capture global structural dependencies often missed by fixed receptive fields. Furthermore, we drive the learning process with a hybrid objective function, coupling Tversky loss for class imbalance with soft clDice loss to explicitly penalize topological disconnects. Evaluation on the Fundus-AVSeg
dataset reveals state-of-the-art performance, achieving a combined Dice score of 90.97% and a 95% Hausdorff Distance of 3.50 pixels. Notably, our method decreases vessel fragmentation by approximately 38% relative to baselines, yielding topologically coherent vascular trees viable for automated biomarker quantification. We open-source our code at https://tffm-module.github.io/.
Read the full abstract ▾
All authors Iftekhar Ahmed, Shakib Absar, Aftar Ahmad Sami, Shadman Sakib, Debojyoti Biswas, Seraj Al Mahmud Mostafa
Computer Vision
QA-ReID: Quality-Aware Query-Adaptive Convolution Leveraging Fused Global and Structural Cues for Clothes-Changing ReID
Yuxiang Wang, Kunming Jiang, Tianxiang Zhang et al. (+2 more)
Unlike conventional person re-identification (ReID), clothes-changing ReID (CC-ReID) presents severe challenges due to substantial appearance variations introduced by clothing changes. In this work, we propose the Quality-Aware Dual-Branch Matching (QA-ReID), which jointly leverages RGB-based features and parsing-based representations to model both global appearance and clothing-invariant structural cues. These heterogeneous features are adaptively fused through a multi-modal
attention module. At the matching stage, we further design the Quality-Aware Query Adaptive Convolution (QAConv-QA), which incorporates pixel-level importance weighting and bidirectional consistency constraints to enhance robustness against clothing variations. Extensive experiments demonstrate that QA-ReID achieves state-of-the-art performance on multiple
benchmarks, including PRCC, LTCC, and VC-Clothes, and significantly outperforms existing approaches under cross-clothing scenarios.
Read the full abstract ▾
All authors Yuxiang Wang, Kunming Jiang, Tianxiang Zhang, Ke Tian, Gaozhe Jiang
Computer Vision
CLIP-Guided Unsupervised Semantic-Aware Exposure Correction
Puzhen Wu, Han Weng, Quan Zheng et al. (+5 more)
Improper exposure often leads to severe loss of details, color distortion, and reduced contrast. Exposure correction still faces two critical challenges: (1) the ignorance of object-wise regional semantic information causes the color shift artifacts; (2) real-world exposure images generally have no ground-truth labels, and its labeling entails massive manual editing. To tackle the challenges, we propose a new unsupervised semantic-aware exposure correction network. It contains an adaptive semantic-aware fusion module, which effectively fuses the semantic information extracted from a pre-trained Fast Segment Anything Model into a shared image feature space. Then the fused features are used by our multi-scale residual spatial mamba group to restore the details and adjust the exposure. To avoid manual editing, we propose a pseudo-ground truth generator guided by CLIP, which is fine-tuned to automatically identify exposure situations and instruct the tailored corrections. Also, we leverage the rich priors from the FastSAM and CLIP to develop a semantic-
prompt consistency loss to enforce semantic consistency and image-
prompt alignment for unsupervised training. Comprehensive experimental results illustrate the effectiveness of our method in correcting real-world exposure images and outperforms state-of-the-art unsupervised methods both numerically and visually.
Read the full abstract ▾
All authors Puzhen Wu, Han Weng, Quan Zheng, Yi Zhan, Hewei Wang, Yiming Li, Jiahui Han, Rui Xu
Computer Vision
Resolving Primitive-Sharing Ambiguity in Long-Tailed TLS-Based Industrial MEP Point Cloud Segmentation via Spatial Context Constraints
Chao Yin, Qing Han, Zhiwei Hou et al. (+5 more)
In terrestrial
laser scanning (TLS)-based mechanical, electrical, and plumbing (MEP) point cloud
segmentation, safety-critical components such as reducers and valves are persistently misclassified, blocking reliable engineering knowledge extraction. This stems from a dual crisis–extreme class imbalance (215:1) compounded by
geometric ambiguity, since most tail classes share cylindrical primitives with dominant head classes–that existing frequency-based re-weighting methods cannot resolve. We propose spatial context constraints that exploit neighborhood prediction consistency to disambiguate locally similar structures. Our approach extends Class-Balanced (CB) Loss with two architecture-agnostic mechanisms: Boundary-CB, an
entropy-based constraint that emphasizes ambiguous boundaries and encodes an MEP assembly-
topology prior, and Density-CB, a density-based constraint that compensates for scan-dependent variations and encodes TLS sensor-physics knowledge. Both operate at the loss level and require no backbone modifications; we evaluate them with ResPointNet++. On the Industrial3D
dataset (612.7M labelled points from water treatment facilities), our method achieves 55.74% mIoU on the evaluation split, which also served as the validation split, exceeding the strongest of three backbone baselines (39.83-52.48% mIoU), with a 21.7% relative improvement on tail-class performance (29.59% vs. 24.32%) while preserving head-class accuracy (88.14%). Reducer improves from 0% to 21.12% IoU and valve by 24.3% relative. Evaluated per area, the gain comes from the area rich in these primitive-sharing components (tail mIoU 21.4% to 26.5%), while the other area's mIoU is 0.6 points lower. Spatial context constraints thus reduce primitive-sharing errors in the target MEP setting, supporting more reliable identification of safety-critical components for Digital Twin and Scan-to-BIM applications.
Read the full abstract ▾
All authors Chao Yin, Qing Han, Zhiwei Hou, Yue Liu, Anjin Dai, Hongda Hu, Ji Yang, Wei Yao
Computer Vision
Implicit Non-Causal Factors are Out via Dataset Splitting for Domain Generalization Object Detection
Zhilong Zhang, Lei Zhang, Qing He et al. (+3 more)
Open world object detection faces a significant challenge in domain-invariant representation, i.e., implicit non-causal factors. Most domain generalization (DG) methods based on domain
adversarial learning (DAL) pay much
attention to learn domain-invariant information, but often overlook the potential non-causal factors. We unveil two critical causes: 1) The domain discriminator-based DAL method is subject to the extremely sparse domain label, i.e., assigning only one domain label to each
dataset, thus can only associate explicit non-causal factor, which is incredibly limited. 2) The non-causal factors, induced by unidentified data bias, are excessively implicit and cannot be solely discerned by conventional DAL paradigm. Based on these key findings, inspired by the Granular-Ball perspective, we propose an improved DAL method, i.e., GB-DAL. The proposed GB-DAL utilizes Prototype-based Granular Ball Splitting (PGBS) module to generate more dense domains from limited
datasets, akin to more fine-grained granular balls, indicating more potential non-causal factors. Inspired by
adversarial perturbations akin to non-causal factors, we propose a Simulated Non-causal Factors (SNF) module as a means of data augmentation to reduce the implicitness of non-causal factors, and facilitate the training of GB-DAL. Comparative experiments on numerous
benchmarks demonstrate that our method achieves better generalization performance in novel circumstances.
Read the full abstract ▾
All authors Zhilong Zhang, Lei Zhang, Qing He, Shuyin Xia, Guoyin Wang, Fuxiang Huang
Computer Vision · Cross-listed from eess.IV
Optimized $k$-means color quantization of digital images in machine-based and human perception-based colorspaces
Ranjan Maitra
Color quantization represents an image using a fraction of its original number of colors while only minimally losing its visual quality. The $k$-means
algorithm is commonly used in this context, but has mostly been applied in the machine-based RGB colorspace composed of the three primary colors. However, some recent studies have indicated its improved performance in human perception-based colorspaces. We investigated the performance of $k$-means color quantization at four quantization levels in the RGB, CIE-XYZ, and CIE-LUV/CIE-HCL colorspaces, on 148 varied digital images spanning a wide range of scenes, subjects and settings. The Visual Information Fidelity (VIF) measure numerically assessed the quality of the quantized images, and showed that in about half of the cases, $k$-means color quantization is best in the RGB space, while at other times, and especially for higher quantization levels ($k$), the CIE-XYZ colorspace is where it usually does better. There are also some cases, especially at lower $k$, where the best performance is obtained in the CIE-LUV colorspace. Further analysis of the performances in terms of the distributions of the hue, chromaticity and luminance in an image presents a nuanced perspective and characterization of the images for which each colorspace is better for $k$-means color quantization.
Read the full abstract ▾
Computer Vision
FBSDiff++: Improved Frequency Band Substitution of Diffusion Features for Efficient and Highly Controllable Text-Driven Image-to-Image Translation
Xiang Gao and Yunpeng Jia
With large-scale text-to-image (T2I)
diffusion models achieving significant advancements in open-domain image creation, increasing
attention has been focused on their natural extension to the realm of text-driven image-to-image (I2I) translation, where a source image acts as visual guidance to the generated image in addition to the textual guidance provided by the text
prompt. We propose FBSDiff, a novel framework adapting off-the-shelf T2I
diffusion model into the I2I paradigm from a fresh frequency-domain perspective. Through dynamic frequency band substitution of
diffusion features, FBSDiff realizes versatile and highly controllable text-driven I2I in a plug-and-play manner (without need for model training, fine-tuning, or online optimization), allowing appearance-guided, layout-guided, and contour-guided I2I translation by progressively substituting low-frequency band, mid-frequency band, and high-frequency band of latent
diffusion features, respectively. In addition, FBSDiff flexibly enables continuous control over I2I correlation intensity simply by tuning the bandwidth of the substituted frequency band. To further promote image translation efficiency, flexibility, and functionality, we propose FBSDiff++ which improves upon FBSDiff mainly in three aspects: (1) accelerate inference speed by a large margin (8.9$\times$ speedup in inference) with refined model architecture; (2) improve the Frequency Band Substitution module to allow for input source images of arbitrary resolution and aspect ratio; (3) extend model functionality to enable localized image manipulation and style-specific content creation with only subtle adjustments to the core method. Extensive qualitative and quantitative experiments verify superiority of FBSDiff++ in I2I translation visual quality, efficiency, versatility, and controllability compared to related advanced approaches.
Read the full abstract ▾
Computer Vision
Reg-TTR, Test-Time Refinement for Fast, Robust and Accurate Image Registration
Lin Chen, Yue He, Fengting Zhang et al. (+4 more)
Traditional image registration methods are robust but slow due to their iterative nature. While deep learning has accelerated inference, it often struggles with domain shifts. Emerging registration foundation models offer a balance of speed and robustness, yet typically cannot match the peak accuracy of specialized models trained on specific
datasets. To mitigate this limitation, we propose Reg-TTR, a test-time refinement framework that synergizes the complementary strengths of both deep learning and conventional registration techniques. By refining the predictions of pre-trained models at inference, our method delivers significantly improved registration accuracy at a modest computational cost, requiring only 21% additional inference time (0.56s). We evaluate Reg-TTR on two distinct tasks and show that it achieves state-of-the-art (SOTA) performance while maintaining inference speeds close to previous deep learning methods. As foundation models continue to emerge, our framework offers an efficient strategy to narrow the performance gap between registration foundation models and SOTA methods trained on specialized
datasets. The source code will be publicly available following the acceptance of this work.
Read the full abstract ▾
All authors Lin Chen, Yue He, Fengting Zhang, Yaonan Wang, Fengming Lin, Xiang Chen, Min Liu
Computer Vision
Glance and Focus Reinforcement for Pan-cancer Screening
Linshan Wu, Jiaxin Zhuang and Hao Chen
Pan-cancer screening in large-scale CT scans remains challenging for existing AI methods, primarily due to the difficulty of localizing diverse types of tiny lesions in large CT volumes. The extreme foreground-background imbalance significantly hinders models from focusing on diseased regions, while redundant focus on healthy regions not only decreases the efficiency but also increases false positives. Inspired by radiologists' glance and focus diagnostic strategy, we introduce GF-Screen, a Glance and Focus reinforcement learning framework for pan-cancer screening. GF-Screen employs a Glance model to localize the diseased regions and a Focus model to precisely segment the lesions, where
segmentation results of the Focus model are leveraged to reward the Glance model via Reinforcement Learning (RL). Specifically, the Glance model crops a group of sub-volumes from the entire CT volume and learns to select the sub-volumes with lesions for the Focus model to segment. Given that the selecting operation is non-differentiable for
segmentation training, we propose to employ the
segmentation results to reward the Glance model. To optimize the Glance model, we introduce a novel group relative learning paradigm, which employs group relative comparison to prioritize high-advantage predictions and discard low-advantage predictions within sub-volume groups, not only improving efficiency but also reducing false positives. In this way, for the first time, we effectively extend cutting-edge RL techniques to tackle the specific challenges in pan-cancer screening. Extensive experiments on 16 internal and 7 external
datasets across 9 lesion types demonstrated the effectiveness of GF-Screen. Notably, GF-Screen leads the public validation leaderboard of MICCAI
FLARE25 pan-cancer challenge, surpassing the
FLARE24 champion solution by a large margin (+25.6% DSC and +28.2% NSD).
Read the full abstract ▾
Computer Vision
m2sv: A Scalable Benchmark for Map-to-Street-View Spatial Reasoning
Yosub Shin, Michael Buriek and Igor Molybog
Vision–
language models (VLMs) achieve strong performance on many
multimodal benchmarks but remain brittle on spatial
reasoning tasks that require aligning abstract overhead representations with
egocentric views. We introduce m2sv, a scalable
benchmark for map-to-street-view spatial
reasoning that asks models to infer camera viewing direction by aligning a north-up overhead map with a Street View image captured at the same real-world intersection. We release m2sv-20k, a geographically diverse
benchmark with controlled ambiguity, along with m2sv-sft-11k, a curated set of structured
reasoning traces for supervised fine-tuning. Despite strong performance on existing
multimodal benchmarks, the best evaluated VLM achieves only 65.2% accuracy on m2sv, below human annotators who reach 72.0% on average (and 95% for an expert) with strong inter-annotator agreement ($\kappa$ up to 0.76). While supervised fine-tuning and reinforcement learning yield consistent gains, cross-
benchmark evaluations reveal limited transfer. Beyond aggregate accuracy, we systematically analyze difficulty in map-to-street-view
reasoning using both structural signals and human effort, and conduct an extensive failure analysis of adapted open models. Our findings highlight persistent gaps in
geometric alignment, evidence aggregation, and
reasoning consistency, motivating future work on grounded spatial
reasoning across viewpoints.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Privacy-Preserving Model Transcription with Differentially Private Synthetic Distillation
Bochao Liu, Shiming Ge, Pengju Wang et al. (+2 more)
While many deep learning models trained on private
datasets have been deployed in various practical tasks, they may pose a
privacy leakage risk as attackers could recover informative data or label knowledge from models. In this work, we present
privacy-preserving model transcription, a data-free model-to-model conversion solution to facilitate model deployment with a
privacy guarantee. To this end, we propose a cooperative-competitive learning approach termed differentially private synthetic
distillation that learns to convert a pretrained model (teacher) into its
privacy-preserving counterpart (student) via a trainable generator without access to private data. The learning collaborates with three players in a unified framework and performs alternate optimization: i)~the generator is learned to generate synthetic data, ii)~the teacher and student accept the synthetic data and compute differential private labels by flexible data or label noisy perturbation, and iii)~the student is updated with noisy labels and the generator is updated by taking the student as a discriminator for
adversarial training. We theoretically prove that our approach can guarantee differential
privacy and convergence. The transcribed student has good performance and
privacy protection, while the resulting generator can generate private synthetic data for downstream tasks. Extensive experiments clearly demonstrate that our approach outperforms 26 state-of-the-arts.
Read the full abstract ▾
All authors Bochao Liu, Shiming Ge, Pengju Wang, Shikun Li, Tongliang Liu
Computer Vision · Cross-listed from cs.LG
EPAS: Efficient Training with Progressive Activation Sharing
Rezaul Karim, Maryam Dialameh, Yang Liu et al. (+2 more)
We present a novel method for Efficient training with Progressive Activation Sharing (EPAS). This method bridges progressive training paradigm with the phenomenon of redundant QK (or KV ) activations across deeper layers of
transformers. EPAS gradually grows a sharing region during training by switching decoder layers to activation sharing mode. This results in throughput increase due to reduced compute. To utilize deeper layer redundancy, the sharing region starts from the deep end of the model and grows towards the shallow end. The EPAS trained models allow for variable region lengths of activation sharing for different compute budgets during inference. Empirical evaluations with QK activation sharing in LLaMA models ranging from 125M to 7B parameters show up to an 11.1% improvement in training throughput and up to a 29% improvement in inference throughput while maintaining similar loss curve to the baseline models. Furthermore, applying EPAS in continual pretraining to transform TinyLLaMA into an
attention-sharing model yields up to a 10% improvement in average accuracy over state-of-the-art methods, emphasizing the significance of progressive training in cross layer activation sharing models.
Read the full abstract ▾
All authors Rezaul Karim, Maryam Dialameh, Yang Liu, Boxing Chen, Walid Ahmed
Computer Vision
PixSearch: Region-Grounded Retrieval for Knowledgeable Large Multimodal Models
Jeonghwan Kim, Renjie Tao, Sanat Sharma et al. (+8 more)
Multimodal Retrieval-Augmented Generation (MM-RAG) augments visual question answering (VQA) with external knowledge, but many systems rely on modular pipelines that separate visual grounding, query construction, and answer generation. This separation can propagate errors from intermediate detections, masks, or captions into
retrieval while adding latency and limiting coordination with ongoing
reasoning. These limitations are especially consequential for
egocentric images, where questions often concern small, off-center, or long-tail entities that global
retrieval can overlook. We propose PixSearch, an end-to-end segmenting Large
Multimodal Model (LMM) that unifies region-level visual grounding with
retrieval-augmented
reasoning. During autoregressive generation, PixSearch learns when to retrieve, how to route queries across text, whole-image, and region-level
retrieval, and how to crop the target entity as a visual query using a jointly trained mask decoder. A two-stage supervised fine-tuning regimen with search-interleaved supervision teaches
retrieval triggering, query selection, and evidence-grounded answer generation while preserving
segmentation ability. On
egocentric and entity-centric VQA
benchmarks, PixSearch achieves an 18.3% relative accuracy gain on CRAG-MM over whole-image
retrieval, with particularly strong gains on
egocentric images, while remaining competitive on broader VQA and text-only QA tasks. It also reduces inference latency by 36.4% compared with a cached GroundedSAM-LLaVA pipeline.
Read the full abstract ▾
All authors Jeonghwan Kim, Renjie Tao, Sanat Sharma, Jiaqi Wang, Kai Sun, Zhaojiang Lin, Seungwhan Moon, Lambert Mathias, Anuj Kumar, Heng Ji, Xin Luna Dong
Computer Vision
NuiWorld: Exploring a Scalable Framework for End-to-End Controllable World Generation
Han-Hung Lee, Cheng-Yu Yang, Yu-Lun Liu et al. (+1 more)
World generation is a fundamental capability for applications like
video games, simulation, and robotics. However, existing approaches face three main obstacles: controllability, scalability, and efficiency. End-to-end scene generation models have been limited by data scarcity. While object-centric generation approaches rely on fixed resolution representations, degrading fidelity for larger scenes. Training-free approaches, while flexible, are often slow and computationally expensive at inference time. We present NuiWorld, a framework that attempts to address these challenges. To overcome data scarcity, we propose a generative bootstrapping strategy that starts from a few input images. Leveraging recent 3D reconstruction and expandable scene generation techniques, we synthesize scenes of varying sizes and layouts, producing enough data to train an end-to-end model. Furthermore, our framework enables controllability through pseudo sketch labels, and demonstrates a degree of generalization to previously unseen sketches. Our approach represents scenes as a collection of variable scene chunks, which are compressed into a flattened vector-set representation. This significantly reduces the
token length for large scenes, enabling consistent
geometric fidelity across scenes sizes while improving training and inference efficiency.
Read the full abstract ▾
All authors Han-Hung Lee, Cheng-Yu Yang, Yu-Lun Liu, Angel X. Chang