Computer Vision
ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentation
Mohamed Farag, Genc Hoxha, Yahia Maleki et al. (+2 more)
Reliable decision support in digital agriculture requires not only accurate predictions but also well-calibrated uncertainty estimates, particularly for dense prediction tasks such as semantic
segmentation. Ensembles provide strong uncertainty quantification but are computationally and
memory demanding, while single-model approximations often sacrifice uncertainty quality. We propose ST-LoRA, a parameter-efficient ensemble that builds diverse members from a single training trajectory by combining Low-Rank Adaptation (LoRA) with snapshot ensembling. All members share a frozen pretrained backbone and differ only in lightweight low-rank adapters, which sharply reduces trainable parameters, checkpoint storage, and I/O overhead. We evaluate SegFormer, Mask2Former, and EoMT on GrowliFlower-L (cauliflower, open field) and BUP20 (sweet pepper, glasshouse), covering in-distribution performance, calibration under
covariate shift, and near- and far-out-of-distribution (OoD) detection, with BUTom21 (tomato) as near-OoD data. Extensive ablations show that feed-forward layers, not
attention projections, are the critical LoRA target for dense prediction, and that the scaling ratio $\alpha/r$ governs an accuracy–calibration trade-off. Against full-rank snapshot ensembles, ST-LoRA is competitive in
segmentation quality, with architecture-dependent training time and energy savings. Against MC Dropout, DDU, and six post-hoc calibrators, it achieves the strongest far-OoD image-level detection and near-OoD pixel-level localization with low cross-seed variance, although full-rank ensembles remain better calibrated. These results show that LoRA-based ensembling offers a compelling efficiency–performance trade-off for agricultural
vision systems.
Read the full abstract ▾
All authors Mohamed Farag, Genc Hoxha, Yahia Maleki, Chris McCool, Ribana Roscher
Computer Vision
UCBound-Net: Uncertainty-Guided Boundary-Aware Continual Learning for Domain-Incremental Ultrasound Segmentation
Mohammad Amanour Rahman
Continual learning in clinical imaging faces a dual challenge: a model must assimilate knowledge from new anatomical domains while retaining representations learned from prior tasks, a problem known as catastrophic forgetting. Existing mitigation strategies, including regularization and knowledge
distillation, treat all spatial regions equally, ignoring the fact that prediction uncertainty is strongly correlated with the propensity for forgetting. We introduce UCBound-Net, a continual
segmentation framework that exploits Monte Carlo (MC) Dropout uncertainty as a spatial proxy for forgetting risk. Our method contributes three synergistic components: (i) uncertainty-weighted boundary
distillation, which amplifies the knowledge transfer signal at high-
entropy regions of the frozen teacher; (ii) uncertainty-calibration regularization, which explicitly penalizes overconfident erroneous predictions; and (iii) uncertainty-guided exemplar selection, a
memory buffer that preferentially stores samples whose boundary regions exhibit the highest predictive
entropy. Evaluated on a sequential domain-incremental
benchmark comprising breast ultrasound (BUSI, Task 1) followed by thyroid ultrasound (TN3K, Task 2), UCBound-Net reduces forgetting relative to naive fine-tuning, achieving a backward transfer (BWT) of -0.098 compared with -0.173, while obtaining an average Dice Similarity Coefficient (DSC) of 0.755 across both tasks. The proposed framework outperforms baseline methods without requiring task-boundary supervision. An ablation study further demonstrates that each component contributes independently to forgetting mitigation, providing a practical pathway toward uncertainty-aware continual learning for clinical image
segmentation.
Read the full abstract ▾
Computer Vision
Rolling Shutter Camera Self-Calibration
Yongcong Zhang, Navid Rabbani, Bangyan Liao et al. (+3 more)
Rolling shutter (RS) cameras are widely used in consumer devices, but their row-wise exposure causes distortions under motion, making
geometric 3D
vision problems dependent on both camera intrinsics and readout time ratio. Existing RS calibration methods rely on calibration targets or specialised hardware, limiting their use in unconstrained settings. We present the first self-calibration method for RS cameras that directly estimates camera intrinsics and the readout time ratio from image sequences, without requiring calibration targets. The method is implemented as a self-calibrating bundle adjustment (BA), which critically depends on the RS imaging model. We combine two known complementary models. The first formulates RS imaging as continuous-time trajectory estimation under a row-wise pose representation. The second interprets RS images as temporally distorted global shutter (GS) images and requires to estimate correction fields. The combination is non-trivial and results in a unified dual-projection model, in which each 3D point is simultaneously constrained at both row-dependent and reference timestamps along a shared continuous trajectory, enforcing stronger
geometric and temporal consistency. Extensive simulations analyse the applicability of several implementations under varying conditions, and real data experiments demonstrate the accuracy, robustness, and practical effectiveness of the proposed approach.
Read the full abstract ▾
All authors Yongcong Zhang, Navid Rabbani, Bangyan Liao, Chengbo Wang, Yizhen Lao, Adrien Bartoli
Computer Vision · Cross-listed from cs.LG
Stochastic Sequential Search in Very-High-Dimensional Feature Selection
Petr Somol and Jiří Grim
Sequential subset search – forward selection with floating backtracking and its descendants – remains the quality reference in feature selection, but every member of the family sweeps the full pool of remaining candidate features at each step, which excludes it from very-high-dimensional problems; there, only individual-feature ranking remains practical, and it models feature interplay weakly or not at all. We introduce a budgeted sampled step operator pair that replaces the full sweeps by a fixed number of candidate evaluations per step. Candidates are drawn by temperature-controlled softmax sampling from dependency-aware per-feature statistics learned online from every criterion evaluation the search performs, guarded by a uniform exploration floor; per-step cost becomes independent of dimensionality. Substituting the operators turns any sequential method into its
stochastic counterpart, defining the
Stochastic Sequential Search (SSS) family; we study the
stochastic counterpart of floating search, sSFFS. On 500-dimensional madelon, sSFFS retains at least 97% of the full-SFFS criterion value at every subset size at about a quarter of its evaluations, while uniform sampling at the same budget collapses on madelon's synergistic features. On 5,000-dimensional gisette, far beyond full-SFFS reach, sSFFS exceeds the saturated criterion level of DAF and BIF ranking at matched budgets; holdout validation shows that at 500 training samples the binding constraint beyond the sequential frontier becomes the criterion, not the search. On 10,105-dimensional reuters, under a trustworthy multinomial filter criterion, sSFFS dominates BIF and DAF on the search objective and on holdout accuracy at every subset size, in about two minutes of single-core evaluation work. A verified standalone implementation accompanies the paper.
Read the full abstract ▾
Computer Vision
Probing the 3D Object-Level Understanding of Pre-Trained Detection Transformers
Robin Kim, Colin Samplawski and Benjamin M. Marlin
Detection
transformer models, including DETR and its extensions, learn to output a set of object-level
embeddings that can be simultaneously decoded into 2D bounding boxes and class distributions. In this paper, we investigate what pre-trained 2D detection
transformers understand about the 3D properties of objects. Specifically, we investigate the extent to which properties including the depth of objects from the camera and the 3D location of objects relative to the camera can be recovered from object-level
embeddings using linear and non-linear probes. Across a range of detection
transformer models, our results show a surprisingly strong and previously unknown ability of 2D DETR models to represent useful information about the 3D properties of objects, despite the complete lack of 3D supervision during model pre-training.
Read the full abstract ▾
Computer Vision
GaussianSelector: Lightweight Human-Guided Object Selection in 3D Gaussian Splatting with Graph Optimization
Baihan Yang, Tiexin Li, Yuheng Liu et al. (+4 more)
Selecting a complete 3D object from a reconstructed scene with minimal user effort is essential for practical scene editing and
embodied interaction. Existing 3DGS-based methods either retrain the Gaussian representation to embed per-object labels, or build dense multi-view SAM observations, both requiring heavy computation and dense viewpoint coverage that is rarely available in practice. We present GaussianSelector, a training-free framework for interactive 3D object selection from sparse views and sparse scribble guidance. Operating directly on native Gaussian primitives, we coarsen dense Gaussians into geometrically coherent superpoints and construct a continuity-weighted graph using appearance and spatial cues. Sparse user scribbles are lifted into 3D via visibility-aware transmittance coverage, and selection is solved as a global graph-cut energy minimization that propagates sparse evidence to a complete 3D object. This design naturally supports multi-round refinement, where users iteratively correct the selection from additional viewpoints to progressively improve the result. Experiments demonstrate that GaussianSelector achieves competitive selection quality against state-of-the-art multi-view SAM-based methods, while requiring significantly fewer interaction views and substantially lower computational overhead. These properties make it well suited for human-in-the-loop 3D scene editing and 3D asset extraction in real-world deployment scenarios.
Read the full abstract ▾
All authors Baihan Yang, Tiexin Li, Yuheng Liu, Xin Lin, Xinke Li, Xiaohui Xie, Truong Nguyen
Computer Vision
Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning
Yuqi Li, Yuedong Tan, Huiran Duan et al. (+7 more)
Unified
multimodal object tracking has achieved remarkable robustness by leveraging complementary sensor data (e.g., RGB, Thermal, Depth), yet the heavy computational burden of state-of-the-art models hinders their deployment on resource-constrained edge devices. In this work, we identify the prediction head as a critical but often overlooked efficiency bottleneck. By strategically streamlining the decoder architecture, we unlock the potential for real-time inference but simultaneously introduce a capacity gap between the lightweight student and the heavy teacher. To resolve this, we conduct a systematic analysis of 17
distillation strategies and introduce a Dual-
Alignment Distillation framework. Our key insight is that effective compression requires decoupling knowledge transfer into two complementary streams: (1) Spatial Representation
Alignment, which employs feature
distillation to sharpen the student's spatial focus on foreground targets ("Where to track"); and (2) Semantic Distribution
Alignment, which utilizes logit-based
distillation to align decision boundaries and transfer discriminative dark knowledge ("What to track"). Extensive experiments across five
benchmarks demonstrate that our approach significantly outperforms complex state-of-the-art methods. Notably, our distilled model achieves 91.5% MPR on RGBT234 and operates at 54 FPS on a single RTX 4090, representing a 5x speedup over the teacher model while maintaining superior accuracy.
Read the full abstract ▾
All authors Yuqi Li, Yuedong Tan, Huiran Duan, Weilun Feng, Chuanguang Yang, Zhulin An, Zongwei Wu, Shiping Wen, Tingwen Huang, Yingli Tian
Computer Vision
Slot2Text: Object-Centric Visual Tokenization for Efficient and Spatially Traceable Surgical MLLMs
Guiqiu Liao, Matjaz Jogan and Daniel A. Hashimoto
Multimodal large
language models (MLLM) for
surgical scene understanding typically inject hundreds of dense visual
tokens into a
language model, leading to costly inference and limited spatial traceability for generated answers. We present Slot2Text, a dual-mode
surgical MLLM that replaces dense representations of visual input with a compact set of regions encoded as slot latents. Instead of relying on contrastive
alignment of the visual encoder with
language, Slot2Text groups self-supervised
vision features into a few regions–slots that are consumed by the
language model as area-labeled visual
tokens. Slot2Text-Fast uses the slot prefix to answer
surgical questions. Slot2Text-Reason also identifies and locates areas relevant for
reasoning, linking
language outputs to corresponding slot
tokens, masks or regions. Experiments on multiple visual question answering and visual grounding
benchmarks show that Slot2Text-Fast is competitive with state-of-the-art baseline at a much lower cost, reducing the average total
token consumption by a 91.8% and the visual prefix from 1,295 to 47
tokens (a 96.4% reduction). Slot2Text-Reason trades additional
tokens and latency for explicit area identities, locations, and traceable spatial evidence. These results establish compact slot latents as an efficient default visual interface for
surgical MLLMs, with grounded
reasoning invoked when greater spatial traceability is required.
Read the full abstract ▾
Computer Vision
VGER: Voxel-Guided Global Event Ranking for Event Cloud Attribution
Youxin Jiang, Baoheng Fu, Hongwei Ren et al. (+1 more)
Event cameras produce sparse and asynchronous event streams that provide rich spatio-temporal information for efficient perception. Recent advances in event-based models have demonstrated strong performance by directly modeling asynchronous events without dense frame reconstruction. However, identifying the event-level evidence behind their predictions is crucial for improving model transparency and reliability. Directly adapting point-level saliency methods from point clouds provides fine-grained attribution but overlooks event-specific spatio-temporal structures. To address this limitation, we propose Voxel-Guided Global Event Ranking (VGER), a training-free attribution framework for point-based event cloud networks. VGER combines event-level gradient evidence with task-aware voxel perturbation evidence, transferring regional contribution into event-level attribution scores while preserving fine-grained resolution. Furthermore, VGER introduces a unified event ranking strategy, where high-ranked events are expected to be prediction-critical and low-ranked events are expected to have limited influence on predictions. We evaluate VGER on three event-based
benchmarks with PointNet, PointNet++, and EventMamba. Across nine
dataset-backbone settings, VGER consistently improves both high-tail and low-tail deletion performance over point-level saliency baselines.
Read the full abstract ▾
All authors Youxin Jiang, Baoheng Fu, Hongwei Ren, Xiangqian Wu
Computer Vision · Cross-listed from cs.NI
Clear-Weighted Bit Allocation for Satellite Downlinks
Alireza Furutanpey, Qiyang Zhang, Yujie Huang et al. (+2 more)
Earth-observation satellites capture more imagery than intermittent ground contacts can transmit. Onboard systems threshold a cloud detector, discard frames or tiles, and compress the survivors with a fixed codec. On expert-labeled imagery, these rules remove more than one-fifth of clear pixels, primarily through detector false positives. We train a neural codec with a clear-probability-weighted reconstruction loss, reallocating coded bytes from clouds to clear ground without requiring or transmitting a cloud map onboard. Each capture is encoded into a resumable base layer and a dependent refinement layer, while clear content is estimated from features produced by the encoder. At each contact, we causally rank arrived layers using estimated clear content, unfinished bytes, deadline slack, and aggregate deadline pressure. The scheduler serves base and computational deadlines, bounds stored residual bytes, and resumes interrupted packets. We evaluate the onboard-to-downlink pipeline using real
entropy-coded bytes, orbit-derived interruptible contact capacities, and measured service time and energy on resource-constrained embedded accelerators. Clear-weighted codecs require up to 47.8% fewer bytes than learned-compression baselines at matched clear-region quality. The optimized encoder consumes less time and energy than one pass of the cloud detector used by the frame-discard rules. Relative to fixed two-stage service on the same streams, our scheduler more than doubles deadline-full clear-content delivery for the interrupted combined cohort, reaches 83.6% of a certified clairvoyant upper bound, and exceeds replayed reference orders in deadline-usable delivery.
Read the full abstract ▾
All authors Alireza Furutanpey, Qiyang Zhang, Yujie Huang, Philipp Raith, Schahram Dustdar
Computer Vision
Bingxuan Li, Rui Yang, Cheng Qian et al. (+6 more)
Agents are increasingly expected to act not only as task executors, but also as decision-makers on behalf of human users. This shift requires
agents to accumulate evidence over long horizons, interpret implicit user preferences, and compare multiple candidates under partial observations. In this work, we propose DunphyBench, a new
benchmark for evaluating
agents on long-horizon human-centered
embodied decision-making, where the
agent must navigate through multiple
embodied housing environments and make decisions that align with multi-dimensional human preferences. Unlike standard
embodied reasoning tasks that often focus on procedural planning or immediate goal completion, our setting requires
agents to integrate
multimodal, multi-source input into coherent knowledge that supports complex
reasoning across long horizon. The evaluation results reveal that there is a substantial gap between current
agents and human performance. Furthermore, our diagnosis of state-of-the-art VLM-driven
agents reveals that
memory management is one of the bottlenecks, where raw
multimodal history introduces noise that hinders decision quality. Motivated by this finding, we design MeMento, a preference-conditioned
multimodal memory compressor that selectively compresses decision-relevant information from long-horizon history based on user preferences with a fixed set of
memory tokens. Experiments show that MeMento helps VLM-driven
agents improve accuracy by 7.18%, while reducing
memory usage by 85.38% compared to the strongest baseline.
Read the full abstract ▾
All authors Bingxuan Li, Rui Yang, Cheng Qian, Jiateng Liu, Jeonghwan Kim, Zhenhailong Wang, Manling Li, Tong Zhang, Heng Ji
Computer Vision · Cross-listed from cs.RO
DynamicManip: Enabling Dynamic Manipulation from a Single Static Demonstration
Haoran Liao, Pengyue Wang, Shuoyu Chen et al. (+10 more)
Dynamic manipulation is a critical capability for
robots operating in complex and dynamic environments, where
robots must interact with objects that are moving or require rapid adjustments. However, learning models for dynamic manipulation tasks face two major challenges: (1) the combinatorial complexity of dynamic scenarios leads to substantial data requirements, and (2) rapid variations in dynamics require real-time and accurate policy execution. In this paper, we propose DynamicManip to address these challenges through an efficient data augmentation pipeline and a low-latency imitation policy. We first propose a static-to-dynamic augmentation pipeline that synthesizes diverse dynamic manipulation demonstrations from a single static demonstration. Second, we introduce a dynamic-aware adaptive policy that adjusts its inference frequency according to task dynamics, enabling responsive and effective dynamic manipulation. Third, we build a dynamic manipulation
benchmark, which includes diverse dynamic tasks with an automatic evaluation system for scalable and consistent assessment. Extensive experiments in both simulation and the real world demonstrate that DynamicManip not only provides significant improvements in data efficiency but also achieves better performance in dynamic manipulation tasks, with a mean success rate 18.4 percentage points higher and policy-query latency 32.9% lower.
Read the full abstract ▾
All authors Haoran Liao, Pengyue Wang, Shuoyu Chen, Kehan Cheng, Xuhang Chen, Yuhao Lin, Mu Lin, Zhizhao Liang, Xiaoyi Fan, Chengyi Xing, Dan Niu, Yi-Lin Wei, Wei-Shi Zheng
Computer Vision
PackingGPT: 3D Packing Agent for Real Furniture in Last-Mile Delivery
Yi You and Hui Li
3D bin packing rectangular items into standardised containers to maximise space utilisation under
geometric shipping automation. Loading a furniture purchase into a personal vehicle is the same task, but under more complex conditions that standard container loading
algorithms ignore. This paper addresses the physically stable placement under these realistic conditions with heterogeneous boxes (e.g. varying dimensions and weights) and occupied containers (e.g. groceries). This paper provides a real-world
benchmark dataset and baseline model for the Heterogeneous furniture-in-vehicle packing task. The
dataset uses real furniture company flat-pack packaging data covering a large number of catalogue products via family-level extrapolation with diversity length, widths, heights, and weights. We also propose a PackingGPT framework for packing as a sequential placement inspired by the Lego assembly process, where heterogeneous boxes of varying dimensions (bricks) are placed step-by-step into the irregular remaining cargo space (creations). Five baseline packing methods were tested on our
dataset without considering the Centre-of- Mass (CoM) constraints. In sedan car simulations, 10-40% of placed boxes failed the stability check on average. When the LLP model was trained on packing sequences with CoM constraints enforced during placement, the failure rate dropped to 0.67% (SUV-500).
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
GenTrack: Physical Alignment for Robot-Native Motion Generation and Zero-Shot Humanoid Tracking
Zeyu Ling, Xinyao Yu, Renye Yan et al. (+4 more)
General-purpose
humanoid trackers can execute diverse references, but their zero-shot coverage depends on large
embodied corpora that are costly to extend. Text-to-motion generators offer scalable supervision, yet models trained on human motion or retargeted data inherit a gap between kinematic plausibility and
robot executability. Existing one-way pipelines fix either the generated corpus or the reward tracker. We introduce GenTrack, an online generator–tracker framework that alternates execution-grounded, group-relative generator
alignment with tracker training on newly generated references; anchoring and rehearsal constrain drift. On Unitree G1, we evaluate GenTrack with ProtoMotions and SONIC backbones across three zero-shot tracking splits including public AMASS and LAFAN
benchmarks, and a private out-of-distribution test set of 1,024
prompt-motion pairs in the wild. The online co-training strategy consistently produces generators that output more
robot-executable motions with strong semantic
alignment, and trackers with markedly broader zero-shot coverage and improved tracking accuracy, especially on out-of-distribution references. These results demonstrate that joint online post-training effectively narrows the executability gap between retargeted references and
robot-native motion, advancing zero-shot
humanoid control without additional data collection and beyond the limitations of a static reference pool.
Read the full abstract ▾
All authors Zeyu Ling, Xinyao Yu, Renye Yan, Jikang Cheng, Zhanke Wang, Qing Shuai, Changqing Zou
Computer Vision
Training-Free Out-of-Distribution Detection for Pathology Whole-Slide Images
Sabri Mustafa Kahya, Richard R. Chen, Muhammet Sami Yavuz et al. (+4 more)
Safe deployment of AI methods in medicine requires robust guardrails that detect when input data deviate from the training distribution to ensure that models provide predictions only within their scope of expertise and abstain otherwise. Out-of-distribution (OOD) detection can provide such safeguards and is extensively studied in general computer
vision. Yet, it remains underdeveloped in computational pathology, where gigapixel whole-slide images (WSIs), subtle differences between disease subtypes, and variability in tissue preparation pose unique challenges for conventional OOD methods. We propose ZIO, a training-free,
multimodal OOD detector for pathology WSIs that leverages
vision–
language pathology foundation models (FMs). ZIO constructs text and visual prototypes of in-distribution classes and integrates their complementary information through a prototype shrinkage mechanism to derive OOD scores. We provide the ZIO formulation for both slide- and patch-level FMs. We evaluate ZIO across diverse clinically relevant domain shifts, including rare diseases and near-OOD settings. Extensive evaluation of over 14,700 WSIs from five independent consortia shows that ZIO consistently outperforms both unimodal prototypes and 40 state-of-the-art OOD methods. These results demonstrate the benefits of
multimodal representation for OOD detection and pave the way towards safer AI deployment in clinical practice.
Read the full abstract ▾
All authors Sabri Mustafa Kahya, Richard R. Chen, Muhammet Sami Yavuz, Jerry Jierui Lou, Akanimoh Adeleye, Haci Ali Kahya, Jana Lipkova
Computer Vision · Cross-listed from cs.RO
SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
Ruiteng Zhao, Zhengshen Zhang, Yue Su et al. (+6 more)
World Action Models (WAMs) couple action generation with prediction of future states. Their effectiveness depends on whether future dynamics are modeled in a space that is both aligned with action generation and sufficiently geometry-aware to capture where and how actions change the scene. Existing WAMs typically satisfy only part of this requirement, relying on either perceptually heavy observation-space targets or auxiliary latent spaces that are not jointly structured for action relevance and geometry. We propose SG-WAM, a self-guided framework that learns geometry-aware action-conditioned dynamics directly in the policy-derived representation space. SG-WAM introduces learnable dynamics
tokens and a Self-Guided World Predictor that forecasts their future latent states conditioned on intervening
robot actions. Prediction targets are generated by an exponential moving average copy of the same policy backbone, providing stable supervision within the representation family used by the action expert.
Geometric supervision further structures the policy image-
token representations, providing spatially grounded context for the dynamics
tokens and yielding a future-
alignment space that is both action-relevant and geometry-aware. Latent future prediction,
geometric grounding, and flow-matching action generation are jointly optimized end-to-end in a unified framework. Built on a 0.9B model without large-scale
embodied pretraining, SG-WAM achieves 98.5% average success on LIBERO and 73% on LIBERO-Plus, while outperforming strong baselines in both in-distribution and out-of-distribution real-world evaluations.
Read the full abstract ▾
All authors Ruiteng Zhao, Zhengshen Zhang, Yue Su, Wenshuo Wang, Jiahui Li, Zhiyuan Yang, Francis E.H. Tay, Marcelo H. Ang Jr., Haiyue Zhu
Computer Vision
FineMoLA: Towards Fine-Grained Motion-Language Alignment from Clip-Level Supervision
Tongyan Wang, Zhengyuan Li, Muhan Lin et al. (+5 more)
Text-conditioned human motion generation has made rapid progress with the emergence of large-scale motion–
language datasets. However, even
datasets with rich long-form descriptions typically provide supervision only at the clip level, without explicit temporal correspondence between motion frames and
language. This limits fine-grained motion–text grounding and temporally precise generation. We propose FineMoLA, a weakly supervised framework that learns fine-grained frame–phrase correspondence directly from clip-level annotations. Our method first segments long-form descriptions into action-bearing phrases, and then formulates motion–
language alignment as an optimal transport problem, which naturally models many-to-many relations between motion frames and text under global constraints. With entropic regularization and Sinkhorn iterations, FineMoLA efficiently infers pseudo frame-level
alignments without human labeling. Experiments on SnapMoGen demonstrate that the learned
alignments outperform baselines in motion–text grounding.
Read the full abstract ▾
All authors Tongyan Wang, Zhengyuan Li, Muhan Lin, Shengyang Luo, Yifan Shen, Aniket Bera, Baijian Yang, Yingjie Victor Chen
Computer Vision
Understanding Synergistic Interactions among Pathology Foundation Models via Adaptive Fusion
Yuxiang Xiao, Yang Hu, Bin Li et al. (+5 more)
Pathology foundation models (PFMs) provide strong tile-level representations via self-supervised pre-training on large-scale pathology images. Yet, PFMs are developed under diverse and often opaque data, architecture, and objective choices, inducing latent representational biases that limit robustness and obscure what each model specialises in. We present AdaFusion, a lightweight adaptive fusion framework that integrates complementary signals from multiple frozen PFMs through (1) low-dimensional feature compression and (2) a sample-conditioned gating module that reweights model-wise (and optionally channel-wise) contributions. Beyond improving predictive accuracy, AdaFusion provides contribution-driven interpretation that offers evidence consistent with model-specific preferences and synergistic interactions across tissue phenotypes. We evaluate AdaFusion on three public
benchmarks spanning treatment response prediction, prostate cancer grading, and spatial gene expression inference. AdaFusion consistently outperforms individual PFMs and other fusion baselines, while providing interpretable tissue visualisation which aligns model preferences with morphological patterns. Code is available at: https://github.com/xyx-98/PathoOracle.
Read the full abstract ▾
All authors Yuxiang Xiao, Yang Hu, Bin Li, Tianyang Zhang, Zexi Li, Huazhu Fu, Jens Rittscher, Kaixiang Yang
Computer Vision
Harnessing Adversarial Distillation to Customise Debiased, Disease-Specific Pathology Foundation Models for Breast Cancer
Zhiwei Chen, Yang Hu, Yuxiang Xiao et al. (+7 more)
Pathology foundation models (PFMs) provide strong tissue representations and have become central to digital pathology. However, deployment in disease-specific settings is limited by 1) the high computational cost of billion-parameter PFMs and 2) distribution mismatch and non-biological bias inherited from pan-cancer, multi-centre pre-training, including site-specific signatures and imbalanced disease prevalence. These factors can encourage shortcut learning and under-emphasise subtle morphology required for reliable modelling of a specific cancer type. We present SmartStu (a Smart Student), a framework to customise compact, breast-cancer-specific PFMs via
distillation whilst mitigating confounding. SmartStu distils representations from multiple teacher PFMs into a lightweight student backbone. Crucially, we introduce
adversarial distillation that leverages a dedicated noise model trained to predict nuisance, edge-dominated cues on the
distillation set. Using this noise model as a counterexample, the
adversarial objective encourages the student to recognise, yet suppress, features predictive of nuisance targets. We further incorporate multi-teacher ensemble
distillation and an auxiliary self-supervised objective with artefact injection. We validate SmartStu on three external cohorts (Yale HER2, SLN-Breast, and BRACS) with multiple tiny backbones. SmartStu yields breast-cancer-specific PFMs that are over $30\times$ smaller than general PFMs whilst largely preserving, and sometimes improving, downstream performance measured by balanced accuracy (bAcc) and AUC. Code is available at https://github.com/zwchen03/advDistall.
Read the full abstract ▾
All authors Zhiwei Chen, Yang Hu, Yuxiang Xiao, Yakun Ju, Tianyang Zhang, Yingxue Xu, Wei Li, Hao Chen, Jens Rittscher, Kaixiang Yang
Computer Vision
CORTIVA: Candidate-Score Fusion of Complementary Visual Teachers for EEG- and MEG-to-Image Retrieval
Junhan Wang and Kani Chen
Decoding visual experience from non-invasive
brain activity is central to neuroscience and
brain-computer interfaces. Functional
magnetic resonance imaging (fMRI) offers fine spatial detail, but its slow hemodynamics and burdensome acquisition limit temporally resolved decoding. Electroencephalography (
EEG) and magnetoencephalography (MEG) provide millisecond resolution, making image
retrieval compelling: identify the viewed image from neural responses and a fixed candidate bank. Contrastive
alignment to pretrained visual representations enables zero-shot
retrieval from
EEG and MEG, but most systems collapse heterogeneous visual supervision into a single
embedding before ranking. This early consolidation imposes one similarity geometry on every candidate order and removes encoder-specific disagreements from the final ranking. We propose CORTIVA, a candidate-score fusion method that retains this complementary evidence. Three decoding routes are aligned to heterogeneous visual targets, score the same indexed candidates independently, and combine only their temperature-scaled score vectors before ranking. On the 200-way THINGS-
EEG2
benchmark, CORTIVA reaches 73.5% Top-1 and 95.3% Top-5 across ten participants, exceeding the strongest reported baseline by 10.3 and 5.4 percentage points. With a modality-specific neural encoder, the same fusion principle reaches 42.4% Top-1 on THINGS-MEG. Matched route-removal retraining and four weight controls show that the gain comes from combining complementary route scores and persists under uniform weighting, without depending on a specialized weighting rule. Analyses in an independent DINOv2 space recover the same local error-neighborhood enrichment and posterior neural-visual correspondence. These results support candidate-score fusion as a simple, testable alternative to
embedding-level consolidation for neural image
retrieval.
Read the full abstract ▾
Computer Vision
PixVL: Self-Supervised Training of Pixel-Level MLLMs via a Unified Mask–Text Consistency Cycle
Yicheng Xiao, Haoxuan Ma, Caorui Li et al. (+7 more)
Recent studies develop pixel-level
multimodal large
language models (MLLMs) that support both Region
Segmentation and Region Understanding, extending
multimodal interaction from whole images to specific objects and regions. However, these methods face two fundamental challenges. First, the scarcity of high-quality mask–text pairs leaves abundant mask annotations without corresponding
language supervision. Second, discrepancies in supervision formats and learning-signal densities induce optimization interference between Region
Segmentation and Region Understanding. To address these challenges, we propose PixVL, a self-supervised post-training framework that introduces a unified Mask–Text Consistency Cycle, enabling pixel-level MLLMs to generate and self-verify regional descriptions and learn from unlabeled data. We found that direct cycle based solely on
geometric reconstruction is unreliable because re-
segmentation IoU does not faithfully reflect the semantic quality and referring sufficiency. PixVL therefore introduces confuser-aware semantic verification, which uses the model's confidence when it correctly chooses the target among highly similar candidate regions, and assigns zero reward to an incorrect choice. Meanwhile, PixVL performs cross-view verification using temporally separated
video frames or geometrically transformed image views, preventing cyclic learning from collapsing to positional and shape shortcuts. Finally, a quality-coupled bidirectional learning strategy uses the highest-reward description to guide Text-to-Mask learning. This strategy transforms Region Understanding and Region
Segmentation from competing tasks into mutual generators and verifiers. Experiments demonstrate that PixVL improves both region understanding task and
segmentation task.
Read the full abstract ▾
All authors Yicheng Xiao, Haoxuan Ma, Caorui Li, Yucheng Wu, Weijie Wang, Haoxiao Wang, Shuang Chen, Fan Yang, Haiyun Guo, Jinqiao Wang
Computer Vision
Prompt-Driven Simulation with Feature Perturbation for Cross-Domain Few-Shot Object Detection
Linhai Zhuo, Junxi Cai, Tianwen Qian et al. (+2 more)
Data augmentation, which simulates diverse visual variations to expand the source distribution and induce synthetic domain shifts, is a simple yet effective strategy for mitigating severe domain shifts and limited labeled target data in cross-domain few-shot object detection (CD-FSOD). Existing approaches rely on conventional data augmentation, such as Color-Jitter, Mosaic, and background-centric adaptation (e.g., Domain-RAG), which are limited in modeling complex domain shifts and often lead to suboptimal performance. In this paper, we propose PSP-FSOD, a principled framework that integrates
prompt-driven domain simulation with feature perturbation regularization to improve generalization in CD-FSOD. To enable controllable domain synthesis, we design a
prompt-driven strategy that leverages the visual grounding capability of large VLMs to jointly model foreground and background variations, generating semantically consistent yet domain-diverse training samples. Moreover, we adopt a grounding-aware generation scheme that guides object placement and alleviates semantic-spatial misalignment, thereby improving foreground adaptation. To ensure training stability and robustness, we further introduce a noise-induced feature perturbation mechanism that injects Gaussian noise into multi-scale intermediate features with distribution correction, encouraging consistent predictions under perturbations and reducing reliance on domain-specific cues. Extensive experiments demonstrate that PSP-FSOD produces high-quality domain-diverse supervision and learns domain-invariant representations, consistently improving performance across CD-FSOD
benchmarks.
Read the full abstract ▾
All authors Linhai Zhuo, Junxi Cai, Tianwen Qian, Qingping Zheng, Yang Liu
Computer Vision
DeVIT: Low-Power Vision Transformer Acceleration Using Delta Computation
Reyhaneh Hosseinzadeh, Parham Zilouchian Moghaddam and Mehdi Modarressi
The emergence of
transformer-based deep learning models has brought unprecedented performance across various domains, particularly in natural
language processing and computer
vision. However, deploying these models, especially on resource-constrained devices, poses significant challenges due to their high computational complexity and large
memory size and bandwidth requirements. This complexity has led researchers to use low-bit model weights to reduce
memory usage and improve efficiency. In addition to reducing processing and
memory demands, quantization introduces another useful property: value locality, where the extremely large number of parameters are restricted to a limited range of values. To fully take advantage of this locality, this paper presents DeVIT, an acceleration method for
vision transformers that leverages differential computation to enable multiplier-less
matrix multiplication.
Read the full abstract ▾
Computer Vision
Driver2Map: Imitating Human Driving for Online High-Definition Map Construction
Pan Yin, Runtian Xia, Weisong Kuang et al. (+3 more)
High-definition (HD) maps are essential for autonomous driving systems. In constructing such maps, onboard multi-view camera images, standard-definition maps and satellite images provide crucial information. However, due to the modality and perspective differences among these data sources, existing methods often struggle to effectively align and fuse them, making online HD map construction still challenging. To address these issues, we propose Driver2Map, an online HD map construction model inspired by human drivers. Unlike existing HD map construction models that utilize only two modalities, our Driver2Map can simultaneously exploit three modalities. Specifically, we propose a "two-stage
alignment" strategy to reduce spatial misalignment across different modalities. Additionally, we introduce "Pose-Guided BEV Fusion", a BEV (bird's-eye-view) generation module that leverages camera pose information to adaptively weight multi-view features, thereby effectively suppressing cross-view feature overlap during BEV generation. Also, we design a "Pretrained Prior for Map Refinement" module to refine the initial prediction by learning map structure priors, thus improving the HD map prediction under dynamic occlusions. Extensive experiments demonstrate that Driver2Map outperforms existing methods on both IoU and AP metrics.
Read the full abstract ▾
All authors Pan Yin, Runtian Xia, Weisong Kuang, Kaiyu Li, Cong Zhao, Xiangyong Cao
Computer Vision
Asleep at the Wheel: JEPA's Limitations in Evaluating Novel Driving Data
Advait Pavuluri, Shamik Karkhanis and Uzma Mushtaque
Modern autonomous-driving fleets record far more
video than human reviewers can inspect. This motivates the need for an automatic clip triage mechanism, to surface rare and review-worthy clips, so that driving models can be fine-tuned to better handle unideal circumstances. We test a label-free approach that scores clips by the prediction-error "novelty" of a self-supervised joint-
embedding predictive architecture (JEPA); a frozen V-JEPA
video encoder is paired with a lightweight predictor head to reconstruct masked clip
embeddings, and clips whose
embeddings are hard to predict are flagged as interesting. Evaluated under a realistic protocol that trains on one
dataset and tests against footage from others, this approach appears highly effective. We show that this apparent success is actually a domain-shift consequence: on a fair
benchmark drawn from a single
dataset, this mechanism collapses to chance and is on par with simple no-training baselines. A lightly supervised probe on the same frozen
embeddings results in almost double the average precision, indicating that the bottleneck is indeed the self-supervised objective, rather than the representation. We present this as a study for evaluating the effectiveness of self-supervised learning, where cross-
dataset protocols can silently reward domain separation over novelty.
Read the full abstract ▾
Computer Vision
SphereVideo: Prototype-anchored Hyperspherical Boundary for Continual AI-generated Video Detection
Fei Li, Yue Yu, Yuran Wang et al. (+3 more)
AI-generated
video (AIGV) detection aims to distinguish real
videos from AI-generated ones. In practice, detectors trained on existing data often fail to generalize to newly emerging generative models, making this task challenging. Therefore, continual learning (CL) is essential for improving the adaptability. However, CL frameworks for this task remain underexplored. To this end, we propose SphereVideo, a novel CL framework for AIGV detection built on two key observations. First, real
videos exhibit a compact feature distribution. Based on this, we encourage real
video features to cluster around a real prototype on a hypersphere while repelling AI-generated samples, thereby establishing a decision boundary. This prototype serves as a stable anchor for CL, regulating boundary
evolution and mitigating catastrophic forgetting. Second, existing methods tend to rely solely on spatial artifacts as shortcuts. To enhance temporal modeling, we introduce a strategy that models the temporal dynamics of real data at both frame and clip levels. By strengthening real data modeling, this strategy further facilitates learning a real prototype and forming a stable decision boundary. Moreover, we construct a comprehensive and challenging
benchmark. Extensive experiments demonstrate that SphereVideo achieves an improved plasticity-stability trade-off, outperforming prior methods by 3.08% on seen data and 4.00% on unseen AI-generated data.
Read the full abstract ▾
All authors Fei Li, Yue Yu, Yuran Wang, Xinghan Li, Jingjing Chen, Yu-Gang Jiang
Computer Vision · Cross-listed from cs.CL
LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning
Ziyan Xiao, Yinghao Zhu, Wenting Zhang et al. (+2 more)
Multimodal large
language models (MLLMs) are rapidly evolving with expanded context windows and stronger
reasoning capabilities, enabling multi-chart understanding and multi-step inference. These abilities are increasingly important as MLLMs are adopted in complex agentic tasks. However, existing
benchmarks largely emphasize single-chart perception, while simple chart-to-chart connections are insufficient to evaluate these capabilities. To capture multi-chart complexity while ensuring consistency and validity, we design a synthesis pipeline supported by latent graphs. Building on this pipeline, we introduce LongChart, a
benchmark whose VQA sets contain an average of 6.5 images and 31.2 questions. We evaluate 10 state-of-the-art MLLMs and examine three factors that influence performance:
reasoning patterns, auxiliary tools, and robustness to image perturbations. Our results show that MLLM accuracy decreases and varies substantially as computational complexity increases, highlighting directions for future research in multi-chart
reasoning.
Read the full abstract ▾
All authors Ziyan Xiao, Yinghao Zhu, Wenting Zhang, Heaju Kim, Lequan Yu
Computer Vision
Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning
Jianmin Chen, Jiaqi Tang, Wei Wei et al. (+9 more)
Multimodal large
language models (MLLMs) increasingly rely on long chain-of-thought
reasoning for complex tasks. However, as
reasoning sequences lengthen, models may gradually rely less on visual evidence and more on accumulated textual context, leading to visual forgetting. Existing approaches do not directly constrain how visual evidence is used and maintained along the original
reasoning trajectory, leaving long-context visual forgetting insufficiently addressed. To address this issue, we propose Remember-R1, a reinforcement learning framework that mitigates long-context visual forgetting by applying process-level supervision directly on the original
reasoning trajectory. Specifically, Remember-R1 introduces rewards that encourage broader coverage of matched visual keywords, stronger persistence of visual dependence in later
reasoning steps, and greater focus on question-relevant image regions. Experiments across multiple model scales and diverse
multimodal benchmarks demonstrate that Remember-R1 consistently improves
reasoning performance. Additional analyses further show that it slows the decline of visual
attention during generation, supporting its effectiveness in mitigating long-context visual forgetting.
Read the full abstract ▾
All authors Jianmin Chen, Jiaqi Tang, Wei Wei, Xiaogang Xu, Jiafei Wu, Zhe Liu, Qianzhou Wang, Yingying Yan, Botong Geng, Yuyang Xia, Lei Zhang, Qifeng Chen
Computer Vision
SPAE: Spectrally Guided Autoencoder for Pretrained Visual Latents
Yibin Huang, Jixiang Hong, Zongzhao Li et al. (+8 more)
Latents from
vision foundation models (VFMs) are semantically rich and well suited for visual understanding. Recent representation autoencoder methods such as RAE have shown that they can provide promising latent spaces for image generation. However, VFM latents remain difficult to model directly: DiT-generated latents exhibit spectral mismatch with encoder latents, especially in high-frequency components. Our channel-wise spectral analysis further reveals that these high-frequency components are diffusely distributed across latent channels and entangled with semantic information, making the latent space difficult for DiT to model. To address these challenges, we propose SPAE, latent adaptation framework for generation. Specifically, SPAE employs a compact bottleneck to distill stable semantic information while suppressing high-frequency components, thereby improving the
alignment between DiT-generated latents and encoder latents. In addition, we apply a channel-wise masking strategy to promote the decoupling of semantic information and high-frequency details across bottleneck channels. Experiments show that SPAE achieves a favorable balance among visual understanding, generation quality, and reconstruction fidelity.
Read the full abstract ▾
All authors Yibin Huang, Jixiang Hong, Zongzhao Li, Yuhan Dai, Zhibin Wang, Chunwei Wang, Jun Song, Chen Wang, Xiaofei Sun, Xiaoxiao Xu, Conghui Zhu
Computer Vision
Beyond Symmetric Fusion: Exploiting Task-Dependent Modality Strengths for RGB-Event Small Object Detection
Ziheng Wang, Chaolang Li, Yutong Yang et al. (+5 more)
State-of-the-art RGB-Event detectors improve the detection of small, fast-moving objects by combining complementary features from RGB and Event data, yet they typically fuse the two modalities into a unified representation for both localization and classification. Such a task-symmetric design is inconsistent with the intuition that the two modalities should play different roles according to their task-specific strengths. To examine this issue, we conduct a modality-specific evaluation and find that the relative advantage of the two modalities reverses across tasks: Event data are substantially more effective for class-agnostic localization, whereas RGB data provide stronger category evidence within localized target regions. Motivated by this task-dependent asymmetry, we propose an Asymmetric Event-RGB Object Detection
Transformer (AERODet). During class-agnostic localization, Scale-wise Uncertainty-aware Reliability Estimation (SURE) calculates the relative reliability of the two modalities from their objectness response heatmaps and accordingly calibrates their contributions when the decoder aggregates
multimodal features. Once the candidate boxes are obtained, Task-Decoupled Semantic Refinement (TDSR) decouples classification from localization and uses RGB RoI features for fine-grained classification. Extensive experiments on FRED and NeRDD demonstrate that AERODet achieves state-of-the-art performance. In particular, it surpasses the strongest RGB-Event baseline by 10.7 mAP points on the FRED challenging split.
Read the full abstract ▾
All authors Ziheng Wang, Chaolang Li, Yutong Yang, Xiaohan Xu, Chongxiang Yang, Hengxuan Zhong, Zhen Liang, Pengwen Dai