Computer Vision
MDS-VQA: Model-Informed Data Selection for Video Quality Assessment
Jian Zou, Xiaoyu Xu, Zhihua Wang et al. (+3 more)
Learning-based
video quality assessment (VQA) has advanced rapidly, yet progress is increasingly constrained by a disconnect between model design and
dataset curation. Model-centric approaches often iterate on fixed
benchmarks, while data-centric efforts collect new human labels without systematically targeting the weaknesses of existing VQA models. Here, we describe MDS-VQA, a model-informed data selection mechanism for curating unlabeled
videos that are both difficult for the base VQA model and diverse in content. Difficulty is estimated by a failure predictor trained with a ranking objective, and diversity is measured using deep semantic
video features, with a greedy procedure balancing the two under a constrained labeling budget. Experiments across multiple VQA
datasets and models demonstrate that MDS-VQA identifies diverse, challenging samples that are particularly informative for active fine-tuning. With only a 5% selected subset per target domain, the fine-tuned model improves mean SRCC from 0.651 to 0.722 and achieves the top gMAD rank, indicating strong adaptation and generalization.
Read the full abstract ▾
All authors Jian Zou, Xiaoyu Xu, Zhihua Wang, Yilin Wang, Balu Adsumilli, Kede Ma
Computer Vision
EReCu: Pseudo-label Evolution Fusion and Refinement with Multi-Cue Learning for Unsupervised Camouflage Detection
Shuo Jiang, Gaojia Zhang, Min Tan et al. (+2 more)
Unsupervised Camouflaged Object Detection (UCOD) remains a challenging task due to the high intrinsic similarity between target objects and their surroundings, as well as the reliance on noisy pseudo-labels that hinder fine-grained texture learning. While existing refinement strategies aim to alleviate label noise, they often overlook intrinsic perceptual cues, leading to boundary overflow and structural ambiguity. In contrast, learning without pseudo-label guidance yields coarse features with significant detail loss. To address these issues, we propose a unified UCOD framework that enhances both the reliability of pseudo-labels and the fidelity of features. Our approach introduces the Multi-Cue Native Perception module, which extracts intrinsic visual priors by integrating low-level texture cues with mid-level semantics, enabling precise
alignment between masks and native object information. Additionally, Pseudo-Label
Evolution Fusion intelligently refines labels through teacher-student interaction and utilizes depthwise separable convolution for efficient semantic denoising. It also incorporates Spectral Tensor
Attention Fusion to effectively balance semantic and structural information through compact spectral aggregation across multi-layer
attention maps. Finally, Local Pseudo-Label Refinement plays a pivotal role in local detail optimization by leveraging
attention diversity to restore fine textures and enhance boundary fidelity. Extensive experiments on multiple UCOD
datasets demonstrate that our method achieves state-of-the-art performance, characterized by superior detail perception, robust boundary
alignment, and strong generalization under complex camouflage scenarios. Code is available at https://github.com/JSLiam94/EReCu.
Read the full abstract ▾
All authors Shuo Jiang, Gaojia Zhang, Min Tan, Yufei Yin, Gang Pan
Computer Vision
FBCIR: Balancing Cross-Modal Focuses in Composed Image Retrieval
Chenchen Zhao, Jianhuan Zhuo, Muxi Chen et al. (+6 more)
Composed image
retrieval (CIR) requires multi-modal models to jointly reason over visual content and semantic modifications presented in text-image input pairs. While current CIR models achieve strong performance on common
benchmark cases, their accuracies often degrades in more challenging scenarios where negative candidates are semantically aligned with the query image or text. In this paper, we attribute this degradation to focus imbalances, where models disproportionately attend to one modality while neglecting the other. To validate this claim, we propose FBCIR, a multi-modal focus interpretation method that identifies the most crucial visual and textual input components to a model's
retrieval decisions. Using FBCIR, we report that focus imbalances are prevalent in existing CIR models, especially under hard negative settings. Building on the analyses, we further propose a CIR data augmentation workflow that facilitates existing CIR
datasets with curated hard negatives designed to encourage balanced cross-modal
reasoning. Extensive experiments across multiple CIR models demonstrate that the proposed augmentation consistently improves performance in challenging cases, while maintaining their capabilities on standard
benchmarks. Together, our interpretation method and data augmentation workflow provide a new perspective on CIR model diagnosis and robustness improvements.
Read the full abstract ▾
All authors Chenchen Zhao, Jianhuan Zhuo, Muxi Chen, Zhaohua Zhang, Wenyu Jiang, Tianwen Jiang, Qiuyong Xiao, Jihong Zhang, Qiang Xu
Computer Vision · Cross-listed from cs.HC
Prediction of Grade, Gender, and Academic Performance of Children and Teenagers from Handwriting Using the Sigma-Lognormal Model
Adrian Iste, Kazuki Nishizawa, Chisa Tanaka et al. (+4 more)
Digital handwriting acquisition enables the capture of detailed temporal and kinematic signals reflecting the motor processes underlying writing behavior. While handwriting analysis has been extensively explored in clinical or adult populations, its potential for studying developmental and educational characteristics in children remains less investigated. In this work, we examine whether handwriting dynamics encode information related to student characteristics using a large-scale online
dataset collected from Japanese students from elementary school to junior high school. We systematically compare three families of handwriting-derived features: basic statistical descriptors of kinematic signals,
entropy-based measures of variability, and parameters obtained from the sigma-lognormal model. Although the
dataset contains dense stroke-level recordings, features are aggregated at the student level to enable a controlled comparison between representations. These features are evaluated across three prediction tasks: grade prediction, gender classification, and academic performance classification, using Linear or Logistic Regression and Random Forest models under consistent experimental settings. The results show that handwriting dynamics contain measurable signals related to developmental stage and individual differences, especially for the grade prediction task. These findings highlight the potential of kinematic handwriting analysis and confirm that through their development, children's handwriting evolves toward a lognormal motor organization.
Read the full abstract ▾
All authors Adrian Iste, Kazuki Nishizawa, Chisa Tanaka, Andrew Vargo, Anna Scius-Bertrand, Andreas Fischer, Koichi Kise
Computer Vision · Cross-listed from cs.HC
From Pen Strokes to Sleep States: Detecting Low-Recovery Days Using Sigma-Lognormal Handwriting Features
Chisa Tanaka, Andrew Vargo, Anna Scius-Bertrand et al. (+2 more)
While handwriting has traditionally been studied for character recognition and disease classification, its potential to reflect day-to-day physiological fluctuations in healthy individuals remains unexplored. This study examines whether daily variations in sleep-related recovery states can be inferred from online handwriting dynamics. % We propose a personalized binary classification framework that detects low-recovery days using features derived from the Sigma-Lognormal model, which captures the neuromotor generation process of pen strokes. In a 28-day in-the-wild study involving 13 university students, handwriting was recorded three times daily, and nocturnal cardiac indicators were measured using a wearable ring. For each participant, the lowest (or highest) quartile of four sleep-related metrics – HRV, lowest heart rate, average heart rate, and total sleep duration – defined the positive class. Leave-One-Day-Out cross-validation showed that PR-AUC significantly exceeded the baseline (0.25) for all four variables after FDR correction, with the strongest performance observed for cardiac-related variables. Importantly, classification performance did not differ significantly across task types or recording timings, indicating that recovery-related signals are embedded in general movement dynamics. These results demonstrate that subtle within-person autonomic recovery fluctuations can be detected from everyday handwriting, opening a new direction for non-invasive, device-independent health monitoring.
Read the full abstract ▾
All authors Chisa Tanaka, Andrew Vargo, Anna Scius-Bertrand, Andreas Fischer, Koichi Kise
Computer Vision
Manifold-Optimal Guidance: A Unified Riemannian Control View of Diffusion Guidance
Zexi Jia, Pengcheng Luo, Zhengyao Fang et al. (+2 more)
Classifier-Free Guidance (CFG) serves as the de facto control mechanism for conditional
diffusion, yet high guidance scales notoriously induce oversaturation, texture artifacts, and structural collapse. We attribute this failure to a
geometric mismatch: standard CFG performs Euclidean extrapolation in ambient space, inadvertently driving sampling trajectories off the high-density data
manifold. To resolve this, we present
Manifold-Optimal Guidance (MOG), a framework that reformulates guidance as a local optimal control problem. MOG yields a closed-form, geometry-aware Riemannian update that corrects off-
manifold drift without requiring retraining. Leveraging this perspective, we further introduce Auto-MOG, a dynamic energy-balancing schedule that adaptively calibrates guidance strength, effectively eliminating the need for manual hyperparameter tuning. Extensive validation demonstrates that MOG yields superior fidelity and
alignment compared to baselines, with virtually no added computational overhead.
Read the full abstract ▾
All authors Zexi Jia, Pengcheng Luo, Zhengyao Fang, Jinchao Zhang, Jie Zhou
Computer Vision
Gen-Fab: A Variation-Aware Generative Model for Predicting Fabrication Variations in Nanophotonic Devices
Rambod Azimi, Yuri Grinberg, Dan-Xia Xu et al. (+1 more)
Silicon photonic devices often exhibit fabrication-induced variations such as over-etching, underetching, and corner rounding, which can significantly alter device performance. These variations are non-uniform and are influenced by feature size and shape. Accurate digital twins are therefore needed to predict the range of possible fabricated outcomes for a given design. In this paper, we introduce Gen-Fab, a conditional generative
adversarial network (cGAN) based on Pix2Pix to predict and model uncertainty in photonic fabrication outcomes. The proposed method takes a design layout (in GDS format) as input and produces diverse high-resolution predictions similar to scanning electron microscope (SEM) images of fabricated devices, capturing the range of process variations at the nanometer scale. To enable one-to-many mapping, we inject a latent noise vector at the model bottleneck. We compare Gen-Fab against three baselines: (1) a deterministic U-Net predictor, (2) an inference-time Monte Carlo Dropout U-Net, and (3) an ensemble of varied U-Nets. Evaluations on an out-of-distribution
dataset of fabricated photonic test structures demonstrate that Gen-Fab outperforms all baselines in both accuracy and uncertainty modeling. An additional distribution shift analysis further confirms its strong generalization to unseen fabrication geometries. Gen-Fab achieves the highest intersection-over-union (IoU) score of 89.8%, outperforming the deterministic U-Net (85.3%), the MC-Dropout U-Net (83.4%), and varying U-Nets (85.8%). It also better aligns with the distribution of real fabrication outcomes, achieving lower Kullback-Leibler divergence and Wasserstein distance.
Read the full abstract ▾
All authors Rambod Azimi, Yuri Grinberg, Dan-Xia Xu, Odile Liboiron-Ladouceur
Computer Vision
ActiveFreq: Integrating Active Learning and Frequency Domain Analysis for Interactive Segmentation
Lijun Guo, Qian Zhou, Zidi Shi et al. (+2 more)
Interactive
segmentation is commonly used in medical image analysis to obtain precise, pixel-level labeling, typically involving iterative user input to correct mislabeled regions. However, existing approaches often fail to fully utilize user knowledge from interactive inputs and achieve comprehensive feature extraction. Specifically, these methods tend to treat all mislabeled regions equally, selecting them randomly for refinement without evaluating each region's potential impact on
segmentation quality. Additionally, most models rely solely on spatial domain features, overlooking frequency domain information that could enhance feature extraction and improve performance. To address these limitations, we propose ActiveFreq, a novel interactive
segmentation framework that integrates active learning and frequency domain analysis to minimize human intervention while achieving high-quality labeling. ActiveFreq introduces AcSelect, an autonomous module that prioritizes the most informative mislabeled regions, ensuring maximum performance gain from each click. Moreover, we develop FreqFormer, a
segmentation backbone incorporating a
Fourier transform module to map features from the spatial to the frequency domain, enabling richer feature extraction. Evaluations on the ISIC-2017 and OAI-ZIB
datasets demonstrate that ActiveFreq achieves high performance with reduced user interaction, achieving 3.74 NoC@90 on ISIC-2017 and 9.27 NoC@90 on OAI-ZIB, with 23.5% and 12.8% improvements over previous best results, respectively. Under minimal input conditions, such as two clicks, ActiveFreq reaches mIoU scores of 85.29% and 75.76% on ISIC-2017 and OAI-ZIB, highlighting its efficiency and accuracy in interactive medical
segmentation.
Read the full abstract ▾
All authors Lijun Guo, Qian Zhou, Zidi Shi, Hua Zou, Gang Ke
Computer Vision
OrthoEraser: Coupled-Neuron Orthogonal Projection for Concept Erasure
Chuancheng Shi, Wenhua Wu, Fei Shen et al. (+3 more)
Text-to-image (T2I) models face significant safety risks from
adversarial induction, yet current concept erasure methods often cause collateral damage to benign attributes when suppressing selected
neurons entirely. This occurs because sensitive and benign semantics exhibit non-orthogonal superposition, sharing activation subspaces where their respective vectors are inherently entangled. To address this issue, we propose OrthoEraser, which leverages sparse autoencoders (SAE) to achieve high-resolution feature disentanglement and subsequently redefines erasure as an analytical orthogonalization projection that preserves the benign
manifold's invariance. OrthoEraser first employs SAE to decompose dense activations and segregate sensitive
neurons. It then uses coupled
neuron detection to identify non-sensitive features vulnerable to intervention. The key novelty lies in an analytical gradient orthogonalization strategy that projects erasure vectors onto the null space of the coupled
neurons. This orthogonally decouples the sensitive concepts from the identified critical benign subspace, effectively preserving non-sensitive semantics. Experimental results on safety demonstrate that OrthoEraser achieves high erasure precision, effectively removing harmful content while preserving the integrity of the generative
manifold, and significantly outperforming SOTA baselines. This paper contains results of unsafe models.
Read the full abstract ▾
All authors Chuancheng Shi, Wenhua Wu, Fei Shen, Xiaogang Zhu, Kun Hu, Zhiyong Wang
Computer Vision
SPEGC: Continual Test-Time Adaptation via Semantic-Prompt-Enhanced Graph Clustering for Medical Image Segmentation
Xiaogang Du, Jiawei Zhang, Tongfei Liu et al. (+2 more)
In medical image
segmentation tasks, the domain gap caused by the difference in data collection between training and testing data seriously hinders the deployment of pre-trained models in clinical practice. Continual Test-Time Adaptation (CTTA) aims to enable pre-trained models to adapt to continuously changing unlabeled domains, providing an effective approach to solving this problem. However, existing CTTA methods often rely on unreliable supervisory signals, igniting a self-reinforcing cycle of error accumulation that culminates in catastrophic performance degradation. To overcome these challenges, we propose a CTTA via Semantic-
Prompt-Enhanced Graph Clustering (SPEGC) for medical image
segmentation. First, we design a semantic
prompt feature enhancement mechanism that utilizes decoupled commonality and heterogeneity
prompt pools to inject global contextual information into local features, alleviating their susceptibility to noise interference under domain shift. Second, based on these enhanced features, we design a differentiable graph clustering solver. This solver reframes global edge sparsification as an optimal transport problem, allowing it to distill a raw similarity
matrix into a refined and high-order structural representation in an end-to-end manner. Finally, this robust structural representation is used to guide model adaptation, ensuring predictions are consistent at a cluster-level and dynamically adjusting decision boundaries. Extensive experiments demonstrate that SPEGC outperforms other state-of-the-art CTTA methods on two medical image
segmentation benchmarks. The source code is available at https://github.com/Jwei-Z/SPEGC-for-MIS.
Read the full abstract ▾
All authors Xiaogang Du, Jiawei Zhang, Tongfei Liu, Tao Lei, Yingbo Wang
Computer Vision
INFACT: A Diagnostic Benchmark for Induced Faithfulness and Factuality Hallucinations in Video-LLMs
Junqi Yang, Yuecong Min, Jie Zhang et al. (+2 more)
Despite rapid progress,
Video Large
Language Models (
Video-LLMs) remain unreliable due to
hallucinations, which are outputs that contradict either
video evidence (faithfulness) or verifiable world knowledge (factuality). Existing
benchmarks provide limited coverage of factuality
hallucinations and predominantly evaluate models only in clean settings. We introduce INFACT, a diagnostic
benchmark comprising 9{,}800 QA instances with fine-grained taxonomies for faithfulness and factuality, spanning real and synthetic
videos. INFACT evaluates models in four modes: Base (clean), Visual Degradation, Evidence Corruption, and Temporal Intervention for order-sensitive items. Reliability under induced modes is quantified using Resist Rate (RR) and Temporal Sensitivity Score (TSS). Experiments on 14 representative
Video-LLMs reveal that higher Base-mode accuracy does not reliably translate to higher reliability in the induced modes, with evidence corruption reducing stability and temporal intervention yielding the largest degradation. Notably, many open-source baselines exhibit near-zero TSS on factuality, indicating pronounced temporal inertia on order-sensitive questions.
Read the full abstract ▾
All authors Junqi Yang, Yuecong Min, Jie Zhang, Shiguang Shan, Xilin Chen
Computer Vision
Follow the Saliency: Supervised Saliency for Retrieval-augmented Dense Video Captioning
Seung hee Choi, MinJu Jeon, Hyunwoo Oh et al. (+2 more)
Existing
retrieval-augmented approaches for Dense
Video Captioning (DVC) often fail to achieve accurate temporal
segmentation aligned with true event boundaries, as they rely on heuristic strategies that overlook ground truth event boundaries. The proposed framework, STaRC, overcomes this limitation by supervising frame-level saliency through a highlight detection module. Note that the highlight detection module is trained on binary labels derived directly from DVC ground truth annotations without the need for additional annotation. We also propose to utilize the saliency scores as a unified temporal signal that drives
retrieval via saliency-guided
segmentation and informs caption generation through explicit Saliency
Prompts injected into the decoder. By enforcing saliency-constrained
segmentation, our method produces temporally coherent segments that align closely with actual event transitions, leading to more accurate
retrieval and contextually grounded caption generation. We conduct comprehensive evaluations on the YouCook2 and ViTT
benchmarks, where STaRC achieves state-of-the-art performance across most of the metrics. Our code is available at https://github.com/ermitaju1/STaRC
Read the full abstract ▾
All authors Seung hee Choi, MinJu Jeon, Hyunwoo Oh, Jihwan Lee, Dong-Jin Kim
Computer Vision · Cross-listed from cs.AI
GPT4o-Receipt: A Dataset and Human Study for AI-Generated Document Forensics
Yan Zhang, Simiao Ren, Ankit Raj et al. (+6 more)
Can humans detect AI-generated financial documents better than machines? We present GPT4o-Receipt, a
benchmark of 1,235 receipt images pairing GPT-4o-generated receipts with authentic ones from established
datasets, evaluated by five state-of-the-art
multimodal LLMs and a 30-annotator crowdsourced perceptual study. Our findings reveal a striking paradox: humans are better at seeing AI artifacts, yet worse at detecting AI documents. Human annotators exhibit the largest visual discrimination gap of any evaluator, yet their binary detection F1 falls well below Claude Sonnet 4 and below Gemini 2.5 Flash. This paradox resolves once the mechanism is understood: the dominant forensic signals in AI-generated receipts are arithmetic errors – invisible to visual inspection but systematically verifiable by LLMs. Humans cannot perceive that a subtotal is incorrect; LLMs verify it in milliseconds. Beyond the human–LLM comparison, our five-model evaluation reveals dramatic performance disparities and calibration differences that render simple accuracy metrics insufficient for detector selection. GPT4o-Receipt, the evaluation framework, and all results are released publicly to support future research in AI document forensics.
Read the full abstract ▾
All authors Yan Zhang, Simiao Ren, Ankit Raj, En Wei, Dennis Ng, Alex Shen, Jiayu Xue, Yuxin Zhang, Evelyn Marotta
Computer Vision
Detect Anything in Real Time: From Single-Prompt Segmentation to Multi-Class Detection
Mehmet Kerem Turkcan
Recent advances in
vision-
language modeling have produced promptable detection and
segmentation systems that accept arbitrary natural
language queries at inference time. Among these, SAM3 achieves state-of-the-art accuracy by combining a ViT-H/14 backbone with cross-modal
transformer decoding and learned object queries. However, SAM3 processes a single text
prompt per forward pass. Detecting N categories requires N independent executions, each dominated by the 439M-parameter backbone. We present Detect Anything in Real Time (DART), a training-free framework that converts SAM3 into a real-time multi-class detector by exploiting a structural invariant: the visual backbone is class-agnostic, producing image features independent of the text
prompt. This allows the backbone computation to be shared between all classes, reducing its cost from O(N) to O(1). Combined with batched multi-class decoding, detection-only inference, and TensorRT FP16 deployment, these optimizations yield 5.6x cumulative speedup at 3 classes, scaling to 25x at 80 classes, without modifying any model weight. On COCO val2017 (5,000 images, 80 classes), DART achieves 55.8 AP at 15.8 FPS (4 classes, 1008x1008) on a single RTX 4080, surpassing purpose-built open-vocabulary detectors trained on millions of box annotations. For extreme latency targets, adapter
distillation with a frozen encoder-decoder achieves 38.7 AP with a 13.9 ms backbone. Code and models are available at https://github.com/mkturkcan/DART.
Read the full abstract ▾
Computer Vision
Stay in your Lane: Role Specific Queries with Overlap Suppression Loss for Dense Video Captioning
Seung Hyup Baek, Jimin Lee, Hyeongkeun Lee et al. (+1 more)
Dense
Video Captioning (DVC) is a challenging
multimodal task that involves temporally localizing multiple events within a
video and describing them with natural
language. While query-based frameworks enable the simultaneous, end-to-end processing of localization and captioning, their reliance on shared queries often leads to significant multi-task interference between the two tasks, as well as temporal redundancy in localization. In this paper, we propose utilizing role-specific queries that separate localization and captioning into independent components, allowing each to exclusively learn its role. We then employ contrastive
alignment to enforce semantic consistency between the corresponding outputs, ensuring coherent behavior across the separated queries. Furthermore, we design a novel suppression mechanism in which mutual temporal overlaps across queries are penalized to tackle temporal redundancy, supervising the model to learn distinct, non-overlapping event regions for more precise localization. Additionally, we introduce a lightweight module that captures core event concepts to further enhance semantic richness in captions through concept-level representations. We demonstrate the effectiveness of our method through extensive experiments on major DVC
benchmarks YouCook2 and ActivityNet Captions.
Read the full abstract ▾
All authors Seung Hyup Baek, Jimin Lee, Hyeongkeun Lee, Jae Won Cho
Computer Vision
Beyond Single-Sample: Reliable Multi-Sample Distillation for Video Understanding
Songlin Li, Xin Zhu, Zechao Guan et al. (+2 more)
Traditional black-box
distillation for Large
Vision-
Language Models (LVLMs) typically relies on a single teacher response per input, which often yields high-variance responses and format inconsistencies in
multimodal or temporal scenarios. To mitigate this unreliable supervision, we propose R-MSD (Reliable Multi-Sample
Distillation), a framework that explicitly models teacher sampling variance to enhance
distillation stability. Rather than relying on a single teacher response, our approach leverages a task-adaptive teacher pool to provide robust supervision tailored to both closed-ended and open-ended
reasoning. By integrating quality-aware signal matching with an
adversarial distillation objective, our approach effectively filters teacher noise while maximizing knowledge transfer. Extensive evaluations across comprehensive
video understanding
benchmarks demonstrate that R-MSD consistently outperforms single sample
distillation methods. We additionally include an original SFT+RL 4B baseline under the same training budget, which shows only marginal gains, while our method achieves significant improvements. With a 4B student model, our approach delivers gains on VideoMME (+1.5%),
Video-MMMU (+3.2%), and MathVerse (+3.6%).
Read the full abstract ▾
All authors Songlin Li, Xin Zhu, Zechao Guan, Peipeng Chen, Jian Yao
Computer Vision
ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation
Songlin Yang, Zhe Wang, Xuyi Yang et al. (+7 more)
Text-driven
video generation has democratized film creation, but camera control in cinematic multi-shot scenarios remains a significant block. Implicit textual
prompts lack precision, while explicit trajectory conditioning imposes prohibitive manual overhead and often triggers execution failures in current models. To overcome this bottleneck, we propose a data-centric paradigm shift, positing that aligned (Caption, Trajectory,
Video) triplets form an inherent joint distribution that can connect automated plotting and precise execution. Guided by this insight, we present ShotVerse, a “Plan-then-Control” framework that decouples generation into two collaborative
agents: a VLM (
Vision-
Language Model)-based Planner that leverages spatial priors to obtain cinematic, globally aligned trajectories from text, and a Controller that renders these trajectories into multi-shot
video content via a camera adapter. Central to our approach is the construction of a data foundation: we design an automated multi-shot camera calibration pipeline aligns disjoint single-shot trajectories into a unified global coordinate system. This facilitates the curation of ShotVerse-Bench, a high-fidelity cinematic
dataset with a three-track evaluation protocol that serves as the bedrock for our framework. Extensive experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting, achieving superior cinematic aesthetics and generating multi-shot
videos that are both camera-accurate and cross-shot consistent.
Read the full abstract ▾
All authors Songlin Yang, Zhe Wang, Xuyi Yang, Songchun Zhang, Xianghao Kong, Taiyi Wu, Xiaotong Zhao, Ran Zhang, Alan Zhao, Anyi Rao
Computer Vision
Zero-Shot Cross-City Generalization in End-to-End Autonomous Driving: Self-Supervised versus Supervised Representations
Fatemeh Naeinian, Ali Hamza, Haoran Zhu et al. (+1 more)
End-to-end autonomous driving models are typically trained on multi-city
datasets using supervised ImageNet-pretrained backbones, yet their ability to generalize to unseen cities remains largely unexamined. When training and evaluation data are geographically mixed, models may implicitly rely on city-specific cues, masking failure modes that would occur under real-world domain shifts when generalizing to new locations. In this work, we formulate zero-shot cross-city transfer as a controlled representation-level stress test for end-to-end autonomous driving and ask how visual pretraining affects transfer behavior under geographic domain shift. We conduct a comprehensive study by integrating self-supervised backbones I-JEPA, DINOv2, and MAE into planning frameworks. We evaluate performance under strict geographic splits on nuScenes in the open-loop setting and on NAVSIM in the closed-loop evaluation protocol. Our experiments reveal a substantial generalization gap when transferring models across cities with different road
topologies, traffic conventions, and visual environments. In open-loop evaluation, a supervised backbone exhibits severe degradation when transferring between cities, yet some domain-specific self-supervised methods can substantially reduce both displacement and collision degradation. In closed-loop evaluation, self-supervised pretraining improves average out-of-distribution PDMS in several single-city training settings. Our results provide empirical evidence that representation learning influences the robustness of cross-city planning and motivate zero-shot geographic transfer as an important stress test for evaluating end-to-end autonomous driving systems.
Read the full abstract ▾
All authors Fatemeh Naeinian, Ali Hamza, Haoran Zhu, Anna Choromanska
Computer Vision
Seeing Isn't Orienting: A Cognitively Grounded Benchmark Reveals Systematic Orientation Failures in MLLMs Supplementary
Nazia Tasnim, Keanu Nichols, Yuting Yang et al. (+4 more)
Humans learn object orientation progressively, from recognizing which way an object faces, to mentally rotating it, to
reasoning about orientations between objects. Current
vision-
language benchmarks largely conflate orientation with position and general scene understanding. We introduce Discriminative Orientation
Reasoning Intelligence (DORI), a cognitively grounded hierarchical
benchmark that makes object orientation the primary target. Inspired by stages of human orientation cognition, DORI decomposes orientation into four dimensions, each evaluated at coarse (categorical) and granular (metric) levels. Composed from 13,652 images across 14 sources, DORI provides 33,656 multiple-choice questions covering 67 object categories in real-world and synthetic settings. Its coarse-to-granular design isolates orientation from confounds such as object recognition difficulty, scene clutter, and linguistic ambiguity via bounding-box isolation, standardized spatial reference frames, and structured
prompts. Evaluating 24 state-of-the-art
vision-
language models shows a clear pattern: models that perform well on general spatial
benchmarks are near-random on object-centric orientation tasks. The best models reach only 54.2% on coarse and 45.0% on granular judgments, with largest failures on compound rotations and shifts in inter-object reference frames. Large coarse-to-granular gaps reveal reliance on categorical heuristics rather than
geometric reasoning, a limitation hidden by existing
benchmarks. These results identify orientation understanding as an unsolved challenge for
multimodal systems, with implications for robotic manipulation, 3D scene reconstruction, and human-AI interaction.
Read the full abstract ▾
All authors Nazia Tasnim, Keanu Nichols, Yuting Yang, Nicholas Ikechukwu, Elva Zou, Deepti Ghadiyaram, Bryan A. Plummer
Computer Vision · Cross-listed from cs.RO
Real-time Rendering-based Surgical Instrument Tracking via Evolutionary Optimization
Hanyang Hu, Zekai Liang, Florian Richter et al. (+1 more)
Accurate and efficient tracking of
surgical instruments is fundamental for
Robot-Assisted Minimally Invasive Surgery. Although
vision-based
robot pose estimation has enabled markerless calibration without tedious physical setups, reliable tool tracking for
surgical robots still remains challenging due to partial visibility and specialized articulation design of
surgical instruments. Previous works in the field are usually prone to unreliable feature detections under degraded visual quality and data scarcity, whereas rendering-based methods often struggle with computational costs and suboptimal convergence. In this work, we incorporate CMA-ES, an evolutionary optimization strategy, into a versatile tracking pipeline that jointly estimates
surgical instrument pose and joint configurations. Using batch rendering to efficiently evaluate multiple pose candidates in parallel, the method significantly reduces inference time and improves convergence robustness. The proposed framework further generalizes to joint angle-free and bi-manual tracking settings, making it suitable for both
vision feedback control and online surgery
video calibration. Extensive experiments on synthetic and real-world
datasets demonstrate that the proposed method significantly outperforms prior approaches in both accuracy and runtime. Source code and data are available at https://github.com/hanyang-hu/online_dvrk_tracking.
Read the full abstract ▾
All authors Hanyang Hu, Zekai Liang, Florian Richter, Michael C. Yip
Computer Vision
DeepHistoViT: An Interpretable Vision Transformer Framework for Histopathological Cancer Classification
Ravi Mosalpuri, Mohammed Abdelsamea and Ahmed Karam Eldaly
Histopathology remains the gold standard for cancer diagnosis because it provides detailed cellular-level assessment of tissue morphology. However, manual histopathological examination is time-consuming, labour-intensive, and subject to inter-observer variability, creating a demand for reliable computer-assisted diagnostic tools. Recent advances in deep learning, particularly
transformer-based architectures, have shown strong potential for modelling complex spatial dependencies in medical images. In this work, we propose DeepHistoViT, a
transformer-based framework for automated classification of histopathological images. The model employs a customized
Vision Transformer architecture with an integrated
attention mechanism designed to capture fine-grained cellular structures while improving interpretability through
attention-based localization of diagnostically relevant regions. The framework is evaluated on three publicly available histopathology
datasets covering lung cancer, colon cancer, and acute lymphoblastic leukaemia. Experimental results demonstrate state-of-the-art performance across all
datasets, with classification accuracy, precision, recall, F1-score, and ROC-AUC reaching 100 percent on the lung and colon cancer
datasets, and 99.85 percent, 99.84 percent, 99.86 percent, 99.85 percent, and 99.99 percent respectively on the acute lymphoblastic leukaemia
dataset. All performance metrics are reported with 95 percent confidence intervals. These results highlight the effectiveness of
transformer-based architectures for histopathological image analysis and demonstrate the potential of DeepHistoViT as an interpretable computer-assisted diagnostic tool to support pathologists in clinical decision-making.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Harnessing Data Asymmetry: Manifold Learning in the Finsler World
Thomas Dagès, Simon Weber, Daniel Cremers et al. (+1 more)
Manifold learning is a fundamental task at the core of data analysis and visualisation. It aims to capture the simple underlying structure of complex high-dimensional data by preserving pairwise dissimilarities in low-dimensional
embeddings. Traditional methods rely on symmetric Riemannian geometry, thus forcing symmetric dissimilarities and
embedding spaces, e.g. Euclidean. However, this discards in practice valuable asymmetric information inherent to the non-uniformity of data samples. We suggest to harness this asymmetry by switching to Finsler geometry, an asymmetric generalisation of Riemannian geometry, and propose a Finsler
manifold learning pipeline that constructs asymmetric dissimilarities and embeds in a Finsler space. This greatly broadens the applicability of existing asymmetric embedders beyond traditionally directed data to any data. We also modernise asymmetric embedders by generalising current reference methods to asymmetry, like Finsler t-SNE and Finsler Umap. On controlled synthetic and large real
datasets, we show that our asymmetric pipeline reveals valuable information lost in the traditional pipeline, e.g. density hierarchies, and consistently provides superior quality
embeddings than their Euclidean counterparts.
Read the full abstract ▾
All authors Thomas Dagès, Simon Weber, Daniel Cremers, Ron Kimmel
Computer Vision
High-Precision 6DOF Pose Estimation via Global Phase Retrieval in Fringe Projection Profilometry for 3D Mapping
Sehoon Tak, Keunhee Cho, Sangpil Kim et al. (+1 more)
Digital fringe projection (DFP) enables micrometer-level 3D reconstruction, yet extending it to large-scale mapping remains challenging because six-degree-of-freedom pose estimation often cannot match the reconstruction's precision. Conventional iterative closest point (ICP) registration becomes inefficient on multi-million-point clouds and typically relies on downsampling or feature-based selection, which can reduce local detail and degrade pose precision. Drift-correction methods improve long-term consistency but do not resolve sampling sensitivity in dense DFP point clouds.We propose a high-precision pose estimation method that augments a moving DFP system with a fixed, intrinsically calibrated global projector. Using the global projector's phase-derived pixel constraints and a PnP-style reprojection objective, the method estimates the DFP system pose in a fixed reference frame without relying on deterministic feature extraction, and we experimentally demonstrate sampling invariance under coordinate-preserving subsampling. Experiments demonstrate sub-millimeter pose accuracy against a reference with quantified uncertainty bounds, high repeatability under aggressive subsampling, robust operation on homogeneous surfaces and low-overlap views, and reduced error accumulation when used to correct ICP-based trajectories. The method extends DFP toward accurate 3D mapping in quasi-static scenarios such as inspection and metrology, with the trade-off of time-multiplexed acquisition for the additional projector measurements.
Read the full abstract ▾
All authors Sehoon Tak, Keunhee Cho, Sangpil Kim, Jae-Sang Hyun