Computer Vision · Cross-listed from cs.LG
An accurate flatness measure to estimate the generalization performance of CNN models
Rahman Taleghani, Maryam Mohammadi and Francesco Marchetti
Flatness measures based on the spectrum or the trace of the Hessian of the loss are widely used as proxies for the generalization ability of deep networks. However, most existing definitions are either tailored to fully connected architectures, relying on
stochastic estimators of the Hessian trace, or ignore the specific
geometric structure of modern Convolutional Neural Networks (CNNs). In this work, we develop a flatness measure that is both exact and architecturally faithful for a broad and practically relevant class of CNNs. We first derive a closed-form expression for the trace of the Hessian of the cross-
entropy loss with respect to convolutional kernels in networks that use global average pooling followed by a linear classifier. Building on this result, we then specialize the notion of relative flatness to convolutional layers and obtain a parameterization-aware flatness measure that properly accounts for the scaling
symmetries and filter interactions induced by convolution and pooling. Finally, we empirically investigate the proposed measure on families of CNNs trained on standard image-classification
benchmarks. The results obtained suggest that the proposed measure can serve as a robust tool to assess and compare the generalization performance of CNN models, and to guide the design of architecture and training choices in practice.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
The Coupling Within: Flow Matching via Distilled Normalizing Flows
David Berthelot, Tianrong Chen, Jiatao Gu et al. (+6 more)
Flow models have rapidly become the go-to method for training and deploying large-scale generators, owing their success to inference-time flexibility via adjustable integration steps. A crucial ingredient in flow training is the choice of coupling measure for sampling noise/data pairs that define the flow matching (FM) regression loss. While FM training defaults usually to independent coupling, recent works show that adaptive couplings informed by noise/data distributions (e.g., via optimal transport, OT) improve both model training and inference. We radicalize this insight by shifting the paradigm: rather than computing adaptive couplings directly, we use distilled couplings from a different, pretrained model capable of placing noise and data spaces in bijection – a property intrinsic to normalizing flows (NF) through their maximum likelihood and invertibility requirements. Leveraging recent advances in NF image generation via auto-regressive (AR) blocks, we propose Normalized Flow Matching (NFM), a new method that distills the quasi-deterministic coupling of pretrained NF models to train student flow models. These students achieve the best of both worlds: significantly outperforming flow models trained with independent or even OT couplings, while also improving on the teacher AR-NF model.
Read the full abstract ▾
All authors David Berthelot, Tianrong Chen, Jiatao Gu, Marco Cuturi, Laurent Dinh, Bhavik Chandna, Michal Klein, Josh Susskind, Shuangfei Zhai
Computer Vision
Diffusion-Based Authentication of Copy Detection Patterns: A Multimodal Framework with Printer Signature Conditioning
Bolutife Atoki, Iuliia Tkachenko, Bertrand Kerautret et al. (+1 more)
Counterfeiting affects diverse industries, including pharmaceuticals, electronics, and food, posing serious health and economic risks. Printable unclonable codes, such as Copy Detection Patterns (CDPs), are widely used as an anti-counterfeiting measure and are applied to products and packaging. However, the increasing availability of high-resolution printing and scanning devices, along with advances in generative deep learning, undermines traditional authentication systems, which often fail to distinguish high-quality counterfeits from genuine prints. In this work, we propose a
diffusion-based authentication framework that jointly leverages the original binary template, the printed CDP, and a representation of printer identity that captures relevant semantic information. Formulating authentication as multi-class printer classification over printer signatures lets our model capture fine-grained, device-specific features via spatial and textual conditioning. We extend ControlNet by repurposing the denoising process for class-conditioned noise prediction, enabling effective printer classification. On the Indigo 1 x 1 Base
dataset, our method outperforms traditional similarity metrics and prior deep learning approaches. Results show the framework generalises to counterfeit types unseen during training.
Read the full abstract ▾
All authors Bolutife Atoki, Iuliia Tkachenko, Bertrand Kerautret, Carlos Crispim-Junior
Computer Vision
SkipGS: Post-Densification Backward Skipping for Efficient 3DGS Training
Jingxing Li, Yongjae Lee and Deliang Fan
3D Gaussian Splatting (3DGS) achieves real-time novel-view synthesis by optimizing millions of anisotropic Gaussians, yet its training remains expensive, with the backward pass dominating runtime in the post-densification refinement phase. We observe substantial update redundancy in this phase: many sampled views have near-plateaued losses and provide diminishing gradient benefits, but standard training still runs full backpropagation. We propose SkipGS with a novel view-adaptive backward gating mechanism for efficient post-densification training. SkipGS always performs the forward pass to update per-view loss statistics, and selectively skips backward passes when the sampled view's loss is consistent with its recent per-view baseline, while enforcing a minimum backward budget for stable optimization. On Mip-NeRF 360, compared to 3DGS, SkipGS reduces end-to-end training time by 23.1%, driven by a 42.0% reduction in post-densification time, with comparable reconstruction quality. Because it only changes when to backpropagate without modifying the renderer, representation, or loss, SkipGS is plug-and-play and compatible with other complementary efficiency strategies, enabling additive speedups. Code is available at https://github.com/ASU-ESIC-FAN-Lab/SkipGS.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
SurgCalib: Gaussian Splatting-Based Hand-Eye Calibration for Robot-Assisted Minimally Invasive Surgery
Zijian Wu, Shuojue Yang, Yu Chung Lee et al. (+3 more)
We present a Gaussian Splatting-based framework for hand-eye calibration of the da Vinci
surgical robot. In a
vision-guided robotic system, accurate estimation of the rigid transformation between the
robot base and the camera frame is essential for reliable closed-loop control. For cable-driven
surgical robots, this task faces unique challenges. The encoders of
surgical instruments often produce inaccurate proprioceptive measurements due to cable stretch and backlash. Conventional hand-eye calibration approaches typically rely on known fiducial patterns and solve the AX = XB formulation. While effective, introducing additional markers into the operating room (OR) environment can violate sterility protocols and disrupt
surgical workflows. In this study, we propose SurgCalib, an automatic, markerless framework that has the potential to be used in the OR. SurgCalib first initializes the pose of the
surgical instrument using raw kinematic measurements and subsequently refines this pose through a two-phase optimization procedure under the RCM constraint within a Gaussian Splatting-based differentiable rendering pipeline. We evaluate the proposed method on the public dVRK
benchmark, SurgPose. The results demonstrate average 2D tool-tip reprojection errors of 12.24 px (2.06 mm) and 11.33 px (1.9 mm), and 3D tool-tip Euclidean distance errors of 5.98 mm and 4.75 mm, for the left and right instruments, respectively.
Read the full abstract ▾
All authors Zijian Wu, Shuojue Yang, Yu Chung Lee, Eitan Prisman, Yueming Jin, Septimiu E. Salcudean
Computer Vision
SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing
Xuanyi Zhou, Qiuyang Mang, Shuo Yang et al. (+7 more)
Diffusion Transformers (DiTs) have become a leading backbone for
video generation, yet their quadratic
attention cost remains a major bottleneck. Sparse
attention reduces this cost by computing only a subset of
attention blocks. However, prior methods often either drop the remaining blocks, which incurs information loss, or rely on learned predictors to approximate them, introducing training overhead and potential output distribution shifting. In this paper, we show that the missing contributions can be recovered without training: after semantic clustering, keys and values within each block exhibit strong similarity and can be well summarized by a small set of cluster centroids. Based on this observation, we introduce SVG-EAR, a parameter-free linear compensation branch that uses the centroid to approximate skipped blocks and recover their contributions. While centroid compensation is accurate for most blocks, it can fail on a small subset. Standard sparsification typically selects blocks by
attention scores, which indicate where the model places its
attention mass, but not where the approximation error would be largest. SVG-EAR therefore performs error-aware routing: a lightweight probe estimates the compensation error for each block, and we compute exactly the blocks with the highest error-to-cost ratio while compensating for skipped blocks. We provide theoretical guarantees that relate
attention reconstruction error to clustering quality, and empirically show that SVG-EAR improves the quality-efficiency trade-off and increases throughput at the same generation fidelity on
video diffusion tasks. Overall, SVG-EAR establishes a clear Pareto frontier over prior approaches, achieving up to 1.77$\times$ and 1.93$\times$ speedups while maintaining PSNRs of up to 29.759 and 31.043 on Wan2.2 and HunyuanVideo, respectively.
Read the full abstract ▾
All authors Xuanyi Zhou, Qiuyang Mang, Shuo Yang, Haocheng Xi, Jintao Zhang, Huanzhi Mao, Joseph E. Gonzalez, Kurt Keutzer, Ion Stoica, Alvin Cheung
Computer Vision
Can You Hear, Localize, and Segment Continually? An Exemplar-Free Continual Learning Benchmark for Audio-Visual Segmentation
Siddeshwar Raghavan, Gautham Vinod, Bruce Coburn et al. (+1 more)
Audio-Visual
Segmentation (AVS) aims to produce pixel-level masks of sound producing objects in
videos, by jointly learning from audio and visual signals. However, real-world environments are inherently dynamic, causing audio and visual distributions to evolve over time, which challenge existing AVS systems that assume static training settings. To address this gap, we introduce the first exemplar-free continual learning
benchmark for Audio-Visual
Segmentation, comprising four learning protocols across single-source and multi-source AVS
datasets. We further propose a strong baseline, ATLAS, which uses audio-guided pre-fusion conditioning to modulate visual feature channels via projected audio context before cross-modal
attention. Finally, we mitigate catastrophic forgetting by introducing Low-Rank Anchoring (LRA), which stabilizes adapted weights based on loss sensitivity. Extensive experiments demonstrate competitive performance across diverse continual scenarios, establishing a foundation for lifelong audio-visual perception. Code is available at${}^{*}$\footnote{Paper under review} - \hyperlink{https://gitlab.com/viper-purdue/atlas}{https://gitlab.com/viper-purdue/atlas} \keywords{Continual Learning \and Audio-Visual
Segmentation \and Multi-Modal Learning}
Read the full abstract ▾
All authors Siddeshwar Raghavan, Gautham Vinod, Bruce Coburn, Fengqing Zhu
Computer Vision
BiCLIP: Domain Canonicalization via Structured Geometric Transformation
Pranav Mantini and Shishir K. Shah
Recent advances in
vision-
language models (VLMs) have demonstrated remarkable zero-shot capabilities, yet adapting these models to specialized domains remains a significant challenge. Building on recent theoretical insights suggesting that independently trained VLMs are related by a canonical transformation, we extend this understanding to the concept of domains. We hypothesize that image features across disparate domains are related by a canonicalized
geometric transformation that can be recovered using a small set of anchors. Few-shot classification provides a natural setting for this
alignment, as the limited labeled samples serve as the anchors required to estimate this transformation. Motivated by this hypothesis, we introduce BiCLIP, a framework that applies a targeted transformation to
multimodal features to enhance cross-modal
alignment. Our approach is characterized by its extreme simplicity and low parameter footprint. Extensive evaluations across 11 standard
benchmarks, including EuroSAT, DTD, and FGVCAircraft, demonstrate that BiCLIP consistently achieves state-of-the-art results. Furthermore, we provide empirical verification of existing
geometric findings by analyzing the orthogonality and angular distribution of the learned transformations, confirming that structured
alignment is the key to robust domain adaptation. Code is available at https://github.com/QuantitativeImagingLaboratory/BilinearCLIP
Read the full abstract ▾
Computer Vision
PathoScribe: Transforming Pathology Data into a Living Library with a Unified LLM-Driven Framework for Semantic Retrieval and Clinical Integration
Abdul Rehman Akbar, Samuel Wales-McGrath, Alejadro Levya et al. (+5 more)
Pathology underpins modern diagnosis and cancer care, yet its most valuable asset, the accumulated experience encoded in millions of narrative reports, remains largely inaccessible. Although institutions are rapidly digitizing pathology workflows, storing data without effective mechanisms for
retrieval and
reasoning risks transforming archives into a passive data repository, where institutional knowledge exists but cannot meaningfully inform patient care. True progress requires not only digitization, but the ability for pathologists to interrogate prior similar cases in real time while evaluating a new diagnostic dilemma. We present PathoScribe, a unified
retrieval-augmented large
language model (LLM) framework designed to transform static pathology archives into a searchable,
reasoning-enabled living library. PathoScribe enables natural
language case exploration, automated cohort construction, clinical question answering, immunohistochemistry (IHC) panel recommendation, and
prompt-controlled report transformation within a single architecture. Evaluated on 70,000 multi-institutional
surgical pathology reports, PathoScribe achieved perfect Recall@10 for natural
language case
retrieval and demonstrated high-quality
retrieval-grounded
reasoning (mean reviewer score 4.56/5). Critically, the system operationalized automated cohort construction from free-text eligibility criteria, assembling research-ready cohorts in minutes (mean 9.2 minutes) with 91.3% agreement to human reviewers and no eligible cases incorrectly excluded, representing orders-of-magnitude reductions in time and cost compared to traditional manual chart review. This work establishes a scalable foundation for converting digital pathology archives from passive storage systems into active clinical intelligence platforms.
Read the full abstract ▾
All authors Abdul Rehman Akbar, Samuel Wales-McGrath, Alejadro Levya, Lina Gokhale, Rajendra Singh, Wei Chen, Anil Parwani, Muhammad Khalid Khan Niazi
Computer Vision
Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning
Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario et al. (+4 more)
This paper introduces a
benchmark for evaluating whether
vision-
language models (VLMs) can generate plant simulation configurations from imagery using in-context learning. We study this
benchmark for cowpea plot reconstruction for plant simulations, where the VLM needs to generate structured JSON configurations that include field and plant information. Open-source
multimodal models from Gemma 4 and Qwen3.5 families are evaluated on a synthetic cowpea
dataset with known JSON ground truth and on a real drone orthophoto
dataset with field-collected JSON. Five in-context learning methods are used, from format restriction instruction to few-shot image examples with auxiliary grounding information. The results show that VLMs can generate valid JSON outputs, can generally estimate days after planting (DAP), plant counts, plant locations, sun angles, and leaf chlorophyll content, and can render approximate simulations of cowpea plots. Error metrics fluctuate across model families and often remain worse than
dataset baselines, particularly when VLMs' pretrained knowledge dominates over weak visual evidence. These results position image-to-simulation JSON generation as a promising but currently challenging task, and establish a
benchmark for studying how
multimodal reasoning,
prompt design, and the sim-to-real domain gap affect plant phenotyping tasks.
Read the full abstract ▾
All authors Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario, Lars Lundqvist, Christine H. Diepenbrock, Brian N. Bailey, J. Mason Earles
Computer Vision
TIDE: Text-Informed Dynamic Extrapolation with Step-Aware Temperature Control for Diffusion Transformers
Yihua Liu, Fanjiang Ye, Bowen Lin et al. (+2 more)
Diffusion Transformer (DiT) faces challenges when generating images with higher resolution compared at training resolution, causing especially structural degradation due to
attention dilution. Previous approaches attempt to mitigate this by sharpening
attention distributions, but fail to preserve fine-grained semantic details and introduce obvious artifacts. In this work, we analyze the characteristics of DiTs and propose TIDE, a training-free text-to-image (T2I) extrapolation method that enables generation with arbitrary resolution and aspect ratio without additional sampling overhead. We identify the core factor for
prompt information loss, and introduce a text anchoring mechanism to correct the imbalance between text and image
tokens. To further eliminate artifacts, we design a dynamic temperature control mechanism that leverages the pattern of spectral progression in the
diffusion process. Extensive evaluations demonstrate that TIDE delivers high-quality resolution extrapolation capability and integrates seamlessly with existing state-of-the-art methods.
Read the full abstract ▾
All authors Yihua Liu, Fanjiang Ye, Bowen Lin, Rongyu Fang, Chengming Zhang
Computer Vision
MEGC2026: Micro-Expression Grand Challenge on Visual Question Answering
Xinqi Fan, Jingting Li, John See et al. (+3 more)
Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an
emotion but attempts to suppress or repress the facial expression, typically found in a high-stakes environment. In recent years, substantial advancements have been made in the areas of ME recognition, spotting, and generation. The emergence of
multimodal large
language models (MLLMs) and large
vision-
language models (LVLMs) offers promising new avenues for enhancing ME analysis through their powerful
multimodal reasoning capabilities. The ME grand challenge (MEGC) 2026 introduces two tasks that reflect these evolving research directions: (1) ME
video question answering (ME-VQA), which explores ME understanding through visual question answering on relatively short
video sequences, leveraging MLLMs or LVLMs to address diverse question types related to MEs; and (2) ME long-
video question answering (ME-LVQA), which extends VQA to long-duration
video sequences in realistic settings, requiring models to handle temporal
reasoning and subtle micro-expression detection across extended time periods. All participating
algorithms are required to submit their results on a public leaderboard. More details are available at https://megc2026.github.io.
Read the full abstract ▾
All authors Xinqi Fan, Jingting Li, John See, Moi Hoon Yap, Su-Jing Wang, Adrian K. Davison
Computer Vision
Vision-Language Models Encode Clinical Guidelines for Concept-Based Medical Reasoning
Mohamed Harmanani, Bining Long, Zhuoxin Guo et al. (+6 more)
Concept Bottleneck Models (CBMs) are a prominent framework for interpretable AI that map learned visual features to a set of meaningful concepts for task-specific downstream predictions. Their sequential structure enhances transparency by connecting model predictions to the underlying concepts that support them. In medical imaging, where transparency is essential, CBMs offer an appealing foundation for explainable model design. However, discrete concept representations often overlook broader clinical context such as diagnostic guidelines and expert heuristics, reducing reliability in complex cases. We propose MedCBR, a concept-based
reasoning framework that integrates clinical guidelines with
vision-
language and
reasoning models. Labeled clinical descriptors are transformed into guideline-conformant text, and a concept-based model is trained with a multitask objective combining
multimodal contrastive
alignment, concept supervision, and diagnostic classification to jointly ground image features, concepts, and pathology. A
reasoning model then converts these predictions into structured clinical narratives that explain the diagnosis, emulating expert
reasoning based on established guidelines. MedCBR achieves superior diagnostic and concept-level performance, with AUROCs of 94.2% on ultrasound and 84.0% on mammography. Further experiments on non-medical
datasets achieve 86.1% accuracy. Our framework enhances interpretability and forms an end-to-end bridge from medical image analysis to decision-making.
Read the full abstract ▾
All authors Mohamed Harmanani, Bining Long, Zhuoxin Guo, Paul F.R. Wilson, Amirhossein Sabour, Minh Nguyen Nhat To, Gabor Fichtinger, Purang Abolmaesumi, Parvin Mousavi
Computer Vision
Multi-Kernel Gated Decoder Adapters for Robust Multi-Task Thyroid Ultrasound under Cross-Center Shift
Maziar Sabouri, Nourhan Bayasi and Arman Rahmim
Thyroid ultrasound (US) automation couples two competing requirements: global, geometry-driven
reasoning for nodule delineation and local, texture-driven
reasoning for malignancy risk assessment. Under cross-center domain shift, these cues degrade asymmetrically, yet most multi-task pipelines rely on a single shared backbone, often inducing negative transfer. In this paper, we characterize this interference across CNN (ResNet34) and medical ViT (MedSAM) backbones, and observe a consistent trend: ViTs transfer
geometric priors that benefit
segmentation, whereas CNNs more reliably preserve texture cues for malignancy discrimination under strong shift and artifacts. Motivated by this failure mode, we propose a lightweight family of decoder-side adapters, the Multi-Kernel Gated Adapter (MKGA) and a residual variant (ResMKGA), which refine multi-scale skip features using complementary receptive fields and apply semantic, context-conditioned gating to suppress artifact-prone content before fusion. Across two US
benchmarks, the proposed adapters improve cross-center robustness: they strengthen out-of-domain
segmentation and, in the CNN setting, yield clear gains in clinical TI-RADS diagnostic accuracy compared to standard multi-task baselines. Code and models will be released.
Read the full abstract ▾
Computer Vision
Towards Visual Query Segmentation in the Wild
Bing Fan, Minghao Li, Hanzhi Zhang et al. (+7 more)
In this paper, we introduce visual query
segmentation (VQS), a new paradigm of visual query localization (VQL) that aims to segment all pixel-level occurrences of an object of interest in an untrimmed
video, given a visual query. Compared with existing VQL locating only the last appearance of a target using bounding boxes, VQS enables more comprehensive (i.e., all object occurrences) and precise (i.e., pixel-level masks) localization, making it more practical for real-world scenarios. To foster research on this task, we introduce VQS-4K, a large-scale
benchmark dedicated to VQS. Specifically, VQS-4K consists of 4,111
videos with more than 1.3 million frames, and covers a diverse set of 222 object categories. Each
video is paired with a visual query defined by a frame outside search
video and its target mask, and annotated with spatial-temporal masklets corresponding to the queried target. To ensure high quality, all
videos in VQS-4K are manually labeled with meticulous inspection and iterative refinement. To our best knowledge, VQS-4K is the first
benchmark specifically designed for VQS. Furthermore, to stimulate future research, we present a simple yet effective method, named VQ-SAM, which extends SAM 2 by leveraging target-specific and background distractor cues from the
video to progressively evolve the
memory through a novel multi-stage framework with an adaptive
memory generation (AMG) module for VQS, significantly improving the performance. In our experiments on VQS-4K, VQ-SAM achieves promising results and surpasses existing methods, evidencing its effectiveness. With VQS-4K and VQ-SAM, we expect to inspire more future research and applications on VQS. Our
benchmark and code are released at https://github.com/VAIL-Fan/VQS.
Read the full abstract ▾
All authors Bing Fan, Minghao Li, Hanzhi Zhang, Shaohua Dong, Naga Prudhvi Mareedu, Xiaoqiong Liu, Weishi Shi, Yunhe Feng, Yan Huang, Heng Fan
Computer Vision
Comparative Analysis of Patch Attack on VLM-Based Autonomous Driving Architectures
David Fernandez, Pedram MohajerAnsari, Amir Salarpour et al. (+3 more)
Vision-
language models are emerging for autonomous driving, yet their robustness to physical
adversarial attacks remains unexplored. This paper presents a systematic framework for comparative
adversarial evaluation across three VLM architectures: Dolphins, OmniDrive (Omni-L), and LeapVAD. Using black-box optimization with semantic homogenization for fair comparison, we evaluate physically realizable patch attacks in CARLA simulation. Results reveal severe vulnerabilities across all architectures, sustained multi-frame failures, and critical object detection degradation. Our analysis exposes distinct architectural vulnerability patterns, demonstrating that current VLM designs inadequately address
adversarial threats in safety-critical autonomous driving applications.
Read the full abstract ▾
All authors David Fernandez, Pedram MohajerAnsari, Amir Salarpour, Long Cheng, Abolfazl Razi, Mert D. Pesé
Computer Vision
HECTOR: Hybrid Editable Compositional Object References for Video Generation
Guofeng Zhang, Angtian Wang, Jacob Zhiyuan Fang et al. (+4 more)
Real-world
videos naturally portray complex interactions among distinct physical objects, effectively forming dynamic compositions of visual elements. However, most current
video generation models synthesize scenes holistically and therefore lack mechanisms for explicit compositional manipulation. To address this limitation, we propose HECTOR, a generative pipeline that enables fine-grained compositional control. In contrast to prior methods,HECTOR supports hybrid reference conditioning, allowing generation to be simultaneously guided by static images and/or dynamic
videos. Moreover, users can explicitly specify the trajectory of each referenced element, precisely controlling its location, scale, and speed (see Figure1). This design allows the model to synthesize coherent
videos that satisfy complex spatiotemporal constraints while preserving high-fidelity adherence to references. Extensive experiments demonstrate that HECTOR achieves superior visual quality, stronger reference preservation, and improved motion controllability compared with existing approaches.
Read the full abstract ▾
All authors Guofeng Zhang, Angtian Wang, Jacob Zhiyuan Fang, Liming Jiang, Haotian Yang, Alan Yuille, Chongyang Ma
Computer Vision
A Lightweight Multi-Cancer Tumor Localization Framework for Deployable Digital Pathology
Brian Isett, Rebekah Dadey, Aofei Li et al. (+5 more)
Accurate localization of tumor regions from hematoxylin and eosin-stained whole-slide images is fundamental for translational research including spatial analysis, molecular profiling, and tissue architecture investigation. However, deep learning-based tumor detection trained within specific cancers may exhibit reduced robustness when applied across different tumor types. We investigated whether balanced training across cancers at modest scale can achieve high performance and generalize to unseen tumor types. A multi-cancer tumor localization model (MuCTaL) was trained on 79,984 non-overlapping tiles from four cancers (melanoma, hepatocellular carcinoma, colorectal cancer, and non-small
cell lung cancer) using transfer learning with DenseNet169. The model achieved a tile-level ROC-AUC of 0.97 in validation data from the four training cancers, and 0.71 on an independent pancreatic ductal adenocarcinoma cohort. A scalable inference workflow was built to generate spatial tumor probability heatmaps compatible with existing digital pathology tools. Code and models are publicly available at https://github.com/AivaraX-AI/MuCTaL.
Read the full abstract ▾
All authors Brian Isett, Rebekah Dadey, Aofei Li, Ryan C. Augustin, Kate Smith, Aatur D. Singhi, Qiangqiang Gu, Riyue Bao
Computer Vision
Computer Vision-Based Vehicle Allotment System using Perspective Mapping
Prachi Nandi, Sonakshi Satapathy and Suchismita Chinara
Smart city research envisions a future in which data-driven solutions and sustainable infrastructure work together to define urban living at the crossroads of urbanization and technology. Within this framework, smart parking systems play an important role in reducing urban congestion and supporting sustainable transportation. Automating parking solutions have considerable benefits, such as increased efficiency and less reliance on human involvement, but obstacles such as sensor limitations and integration complications remain. To overcome them, a more sophisticated car allotment system is required, particularly in heavily populated urban areas. Computer
vision, with its higher accuracy and adaptability, outperforms traditional sensor-based systems for recognizing vehicles and vacant parking spaces. Unlike fixed sensor technologies, computer
vision can dynamically assess a wide range of visual inputs while adjusting to changing parking layouts. This research presents a cost-effective, easy-to-implement smart parking system utilizing computer
vision and object detection models like YOLOv8. Using inverse perspective mapping (IPM) to merge images from four camera views, we extract data on vacant spaces. The system simulates a 3D parking environment, representing available spots with a 3D Cartesian plot to guide users.
Read the full abstract ▾
Computer Vision
VisionCreator-R1: A Reflection-Enhanced Native Visual-Generation Agentic Model
Jinxiang Lai, Wenzhe Zhao, Zexin Lu et al. (+7 more)
Visual content generation has advanced from single-image to multi-image workflows, yet existing
agents remain largely plan-driven and lack systematic reflection mechanisms to correct mid-trajectory visual errors. To address this limitation, we propose VisionCreator-R1, a native visual generation
agent with explicit reflection, together with a Reflection-Plan Co-Optimization (RPCO) training methodology. Through extensive experiments and trajectory-level analysis, we uncover reflection-plan optimization asymmetry in reinforcement learning (RL): planning can be reliably optimized via plan rewards, while reflection learning is hindered by noisy credit assignment. Guided by this insight, our RPCO first trains on the self-constructed VCR-SFT
dataset with reflection-strong single-image trajectories and planning-strong multi-image trajectories, then co-optimization on VCR-RL
dataset via RL. This yields our unified VisionCreator-R1
agent, which consistently outperforms Gemini2.5Pro on existing
benchmarks and our VCR-bench covering single-image and multi-image tasks.
Read the full abstract ▾
All authors Jinxiang Lai, Wenzhe Zhao, Zexin Lu, Hualei Zhang, Qinyu Yang, Rongwei Quan, Zhimin Li, Shuai Shao, Song Guo, Qinglin Lu
Computer Vision
Where, What, Why: Toward Explainable 3D-GS Watermarking
Mingshu Cai, Jiajun Li, Osamu Yoshie et al. (+2 more)
As 3D Gaussian Splatting becomes the de facto representation for interactive 3D assets, robust yet imperceptible watermarking is critical. We present a representation-native framework that separates where to write from how to preserve quality. A Trio-Experts module operates directly on Gaussian primitives to derive priors for carrier selection, while a Safety and Budget Aware Gate (SBAG) allocates Gaussians to watermark carriers, optimized for bit resilience under perturbation and bitrate budgets, and to visual compensators that are insulated from watermark loss. To maintain fidelity, we introduce a channel-wise group mask that controls gradient propagation for carriers and compensators, thereby limiting Gaussian parameter updates, repairing local artifacts, and preserving high-frequency details without increasing runtime. Our design yields view-consistent watermark persistence and strong robustness against common image distortions such as compression and noise, while achieving a favorable robustness-quality trade-off compared with prior methods. In addition, decoupled finetuning provides per-Gaussian attributions that reveal where the message is carried and why those carriers are selected, enabling auditable explainability. Compared with state-of-the-art methods, our approach achieves a PSNR improvement of +0.83 dB and a bit-accuracy gain of +1.24%.
Read the full abstract ▾
All authors Mingshu Cai, Jiajun Li, Osamu Yoshie, Yuya Ieiri, Yixuan Li
Computer Vision
Granulon: Awakening Pixel-Level Visual Encoders with Adaptive Multi-Granularity Semantics for MLLM
Junyuan Mao, Qiankun Li, Linghao Meng et al. (+5 more)
Recent advances in
multimodal large
language models largely rely on CLIP-based visual encoders, which emphasize global semantic
alignment but struggle with fine-grained visual understanding. In contrast, DINOv3 provides strong pixel-level perception yet lacks coarse-grained semantic abstraction, leading to limited multi-granularity
reasoning. To address this gap, we propose Granulon, a novel DINOv3-based MLLM with adaptive granularity augmentation. Granulon introduces a text-conditioned granularity Controller that dynamically adjusts the visual abstraction level according to the semantic scope of the textual input, and an Adaptive
Token Aggregation module that performs granularity-guided pooling and relation-aware clustering to produce compact, semantically rich visual
tokens. This design enables unified "pixel-to-fine-to-coarse"
reasoning within a single forward pass. Extensive and interpretable experiments demonstrate that Granulon improves accuracy by ~30% and reduces
hallucination by ~20%, outperforming all visual encoders under identical settings.
Read the full abstract ▾
All authors Junyuan Mao, Qiankun Li, Linghao Meng, Zhicheng He, Xinliang Zhou, Kun Wang, Yang Liu, Yueming Jin
Computer Vision
Soumik Mukhopadhyay, Prateksha Udhayanan and Abhinav Shrivastava
Diffusion models degrade images through noise, and reversing this process reveals an information hierarchy across timesteps. Scale-space theory exhibits a similar hierarchy via low-pass filtering. We formalize this connection and show that highly noisy
diffusion states contain no more information than small, downsampled images - raising the question of why they must be processed at full resolution. To address this, we fuse scale spaces into the
diffusion process by formulating a family of
diffusion models with generalized linear degradations and practical implementations. Using downsampling as the degradation yields our proposed Scale Space
Diffusion. To support Scale Space
Diffusion, we introduce Flexi-UNet, a UNet variant that performs resolution-preserving and resolution-increasing denoising using only the necessary parts of the network. We evaluate our framework on CelebA and ImageNet and analyze its scaling behavior across resolutions and network depths. Our project website ( https://prateksha.github.io/projects/scale-space-
diffusion/ ) is available publicly.
Read the full abstract ▾
Computer Vision
FVG-PT: Adaptive Foreground View-Guided Prompt Tuning for Vision-Language Models
Haoyang Li, Liang Wang, Siyu Zhou et al. (+5 more)
CLIP-based
prompt tuning enables pretrained
Vision-
Language Models (VLMs) to efficiently adapt to downstream tasks. Although existing studies have made significant progress, they pay limited
attention to changes in the internal
attention representations of VLMs during the tuning process. In this paper, we attribute the failure modes of
prompt tuning predictions to shifts in foreground
attention of the visual encoder, and propose Foreground View-Guided
Prompt Tuning (FVG-PT), an adaptive plug-and-play foreground
attention guidance module, to alleviate the shifts. Concretely, FVG-PT introduces a learnable Foreground Reliability Gate to automatically enhance the foreground view quality, applies a Foreground
Distillation Compensation module to guide visual
attention toward the foreground, and further introduces a Prior Calibration module to mitigate generalization degradation caused by excessive focus on the foreground. Experiments on multiple backbone models and
datasets show the effectiveness and compatibility of FVG-PT. Codes are available at: https://github.com/JREion/FVG-PT
Read the full abstract ▾
All authors Haoyang Li, Liang Wang, Siyu Zhou, Jiacheng Sun, Jing Jiang, Chao Wang, Guodong Long, Yan Peng
Computer Vision
HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising
Kai Zou, Dian Zheng, Hongbo Liu et al. (+3 more)
Autoregressive (AR)
diffusion offers a promising framework for generating
videos of theoretically infinite length. However, a major challenge is maintaining temporal continuity while preventing the progressive quality degradation caused by error accumulation. To ensure continuity, existing methods typically condition on highly denoised contexts; yet, this practice propagates prediction errors with high certainty, thereby exacerbating degradation. In this paper, we argue that a highly clean context is unnecessary. Drawing inspiration from bidirectional
diffusion models, which denoise frames at a shared noise level while maintaining coherence, we propose that conditioning on context at the same noise level as the current block provides sufficient signal for temporal consistency while effectively mitigating error propagation. Building on this insight, we propose HiAR, a hierarchical denoising framework that reverses the conventional generation order: instead of completing each block sequentially, it performs causal generation across all blocks at every denoising step, so that each block is always conditioned on context at the same noise level. This hierarchy naturally admits pipelined parallel inference, yielding a 1.8 wall-clock speedup in our 4-step setting. We further observe that self-rollout
distillation under this paradigm amplifies a low-motion shortcut inherent to the mode-seeking reverse-KL objective. To counteract this, we introduce a forward-KL regulariser in bidirectional-
attention mode, which preserves motion diversity for causal inference without interfering with the
distillation loss. On VBench (20s generation), HiAR achieves the best overall score and the lowest temporal drift among all compared methods.
Read the full abstract ▾
All authors Kai Zou, Dian Zheng, Hongbo Liu, Tiankai Hang, Bin Liu, Nenghai Yu
Computer Vision
ER-Pose: Rethinking Keypoint-Driven Representation Learning for Real-Time Human Pose Estimation
Nanjun Li, Pinqi Cheng, Zean Liu et al. (+2 more)
Single-stage multi-person pose estimation aims to jointly perform human localization and keypoint prediction within a unified framework, offering advantages in inference efficiency and architectural simplicity. Consequently, multi-scale real-time detection architectures, such as YOLO-like models, are widely adopted for real-time pose estimation. However, these approaches typically inherit a box-driven modeling paradigm from object detection, in which pose estimation is implicitly constrained by bounding-box supervision during training. This formulation introduces biases in sample assignment and feature representation, resulting in task misalignment and ultimately limiting pose estimation accuracy. In this work, we revisit box-driven single-stage pose estimation from a keypoint-driven perspective and identify semantic conflicts among parallel objectives as a key source of performance degradation. To address this issue, we propose a keypoint-driven learning paradigm that elevates pose estimation to a primary prediction objective. Specifically, we remove bounding-box prediction and redesign the prediction head to better accommodate the high-dimensional structured representations for pose estimation. We further introduce a keypoint-driven dynamic sample assignment strategy to align training objectives with pose evaluation metrics, enabling dense supervision during training and efficient NMS-free inference. In addition, we propose a smooth OKS-based loss function to stabilize optimization in regression-based pose estimation. Based on these designs, we develop a single-stage multi-person pose estimation framework, termed ER-Pose. On MS COCO and CrowdPose, ER-Pose-n achieves AP improvements of 3.2/6.7 without pre-training and 7.4/4.9 with pre-training respectively compared with the baseline YOLO-Pose. These improvements are achieved with fewer parameters and higher inference efficiency.
Read the full abstract ▾
All authors Nanjun Li, Pinqi Cheng, Zean Liu, Minghe Tian, Xuanyin Wang
Computer Vision
Talking Together: Synthesizing Co-Located 3D Conversations from Audio
Mengyi Shan, Shouchieh Chang, Ziqian Bai et al. (+6 more)
We tackle the challenging task of generating complete 3D facial animations for two interacting, co-located participants from a mixed audio stream. While existing methods often produce disembodied "talking heads" akin to a
video conference call, our work is the first to explicitly model the dynamic 3D spatial relationship – including relative position, orientation, and mutual gaze – that is crucial for realistic in-person dialogues. Our system synthesizes the full performance of both individuals, including precise lip-sync, and uniquely allows their relative head poses to be controlled via textual descriptions. To achieve this, we propose a dual-stream architecture where each stream is responsible for one participant's output. We employ speaker's role
embeddings and inter-speaker cross-
attention mechanisms designed to disentangle the mixed audio and model the interaction. Furthermore, we introduce a novel eye gaze loss to promote natural, mutual eye contact. To power our data-hungry approach, we introduce a novel pipeline to curate a large-scale conversational
dataset consisting of over 2 million dyadic pairs from in-the-wild
videos. Our method generates fluid, controllable, and spatially aware dyadic animations suitable for immersive applications in VR and telepresence, significantly outperforming existing baselines in perceived realism and interaction coherence.
Read the full abstract ▾
All authors Mengyi Shan, Shouchieh Chang, Ziqian Bai, Shichen Liu, Yinda Zhang, Luchuan Song, Rohit Pandey, Sean Fanello, Zeng Huang
Computer Vision
ImprovedGS+: A High-Performance C++/CUDA Re-Implementation Strategy for 3D Gaussian Splatting
Jordi Muñoz Vicente
Recent advancements in 3D Gaussian Splatting (3DGS) have shifted the focus toward balancing reconstruction fidelity with computational efficiency. In this work, we propose ImprovedGS+, a high-performance, low-level reinvention of the ImprovedGS strategy, implemented natively within the LichtFeld-Studio framework. By transitioning from high-level Python logic to hardware-optimized C++/CUDA kernels, we achieve a significant reduction in host-device synchronization and training latency. Our implementation introduces a Long-Axis-Split (LAS) CUDA kernel, custom
Laplacian-based importance kernels with Non-Maximum Suppression (NMS) for edge scores, and an adaptive Exponential Scale Scheduler. Experimental results on the Mip-NeRF360
dataset demonstrate that ImprovedGS+ establishes a new Pareto-optimal front for scene reconstruction. Our 1M-budget variant outperforms the state-of-the-art MCMC baseline by achieving a 26.8% reduction in training time (saving 17 minutes per session) and utilizing 13.3% fewer Gaussians while maintaining superior visual quality. Furthermore, our full variant demonstrates a 1.28 dB PSNR increase over the ADC baseline with a 38.4% reduction in parametric complexity. These results validate ImprovedGS+ as a scalable, high-speed solution that upholds the core pillars of Speed, Quality, and Usability within the LichtFeld-Studio ecosystem.
Read the full abstract ▾
Computer Vision
CAST: Modeling Visual State Transitions for Consistent Video Retrieval
Yanqing Liu, Yingcheng Liu, Fanghong Dong et al. (+3 more)
As
video content creation shifts toward long-form narratives, composing short clips into coherent storylines becomes increasingly important. However, prevailing
retrieval formulations remain context-agnostic at inference time, prioritizing local semantic
alignment while neglecting state and identity consistency. To address this structural limitation, we formalize the task of Consistent
Video Retrieval (CVR) and introduce a diagnostic
benchmark spanning YouCook2, COIN, and CrossTask. We propose CAST (Context-Aware State Transition), a lightweight, plug-and-play adapter compatible with diverse frozen
vision-
language embedding spaces. By predicting a state-conditioned residual update ($\Delta$) from visual history, CAST introduces an explicit inductive bias for latent state
evolution. Extensive experiments show that CAST improves performance on YouCook2 and CrossTask, remains competitive on COIN, and consistently outperforms zero-shot baselines across diverse foundation backbones. Furthermore, CAST provides a useful reranking signal for black-box
video generation candidates (e.g., from Veo), promoting more temporally coherent continuations.
Read the full abstract ▾
All authors Yanqing Liu, Yingcheng Liu, Fanghong Dong, Budianto Budianto, Cihang Xie, Yan Jiao
Computer Vision
Retrieval-Augmented Gaussian Avatars: Improving Expression Generalization
Matan Levy, Gavriel Habib, Issar Tzachor et al. (+5 more)
Template-free animatable head avatars can achieve high visual fidelity by learning expression-dependent facial deformation directly from a subject's capture, avoiding parametric face templates and hand-designed blendshape spaces. However, since learned deformation is supervised only by the expressions observed for a single identity, these models suffer from limited expression coverage and often struggle when driven by motions that deviate from the training distribution. We introduce RAF (
Retrieval-Augmented Faces), a simple training-time augmentation designed for template-free head avatars that learn deformation from data. RAF constructs a large unlabeled expression bank and, during training, replaces a subset of the subject's expression features with nearest-neighbor expressions retrieved from this bank while still reconstructing the subject's original frames. This exposes the deformation field to a broader range of expression conditions, encouraging stronger identity-expression decoupling and improving robustness to expression distribution shift without requiring paired cross-identity data, additional annotations, or architectural changes. We further analyze how
retrieval augmentation increases expression diversity and validate
retrieval quality with a user study showing that retrieved neighbors are perceptually closer in expression and pose. Experiments on the NeRSemble
benchmark demonstrate that RAF consistently improves expression fidelity over the baseline, in both self-driving and cross-driving scenarios.
Read the full abstract ▾
All authors Matan Levy, Gavriel Habib, Issar Tzachor, Dvir Samuel, Rami Ben-Ari, Nir Darshan, Or Litany, Dani Lischinski