Computer Vision
Predicting Viticulture Potential through an Ensemble of U-Net and a Geospatial Foundation Model
Jorge Ignacio Perez, Hwaai Kang Kee and Lucas Rassbach
Determining agricultural potential is fundamental to sustainable land management and agricultural planning. Remote sensing data is increasingly valuable as an avenue for agricultural potential due to the cost of traditional methods (surveys, in-situ measurements, soil testing, etc). ImageCLEF AI4Agri 2026: Subtask 1 is concerned with the prediction of viticulture potential in Southern France. The DS@GT ARC's submission for Subtask 1 introduces an ensemble of U-Net and a Geospatial Foundation Model (Prithvi-2.0). Our best model achieved a $\pm$1 accuracy of 68.32 on the leaderboard, ranking 2nd among 7 teams. The implementation for this work is publicly available at https://github.com/dsgt-arc/imageclef-ai4agri-2026 .
Read the full abstract ▾
Computer Vision
DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models
Pengjie Wang, Linger Deng, Zujia Zhang et al. (+6 more)
Current Unified Large
Multimodal Models (ULMMs) support interleaved
multimodal reasoning through textual
reasoning and intermediate visual states, but typically generate each visual state as a full image. This full-image generation paradigm introduces substantial visual-
token redundancy and dilutes supervision on sparse yet
reasoning-critical state transitions. We propose DeltaV, a ULMM that replaces full-image generation with visual updates. Conditioned on historical visual states, DeltaV incrementally predicts compact update
tokens that capture the visual changes across
reasoning steps, avoiding repeated modeling of unchanged content. To align the
token budget of each update with the magnitude of visual change, DeltaV introduces a temporal similarity (TSIM) Router, which stops allocating
tokens once the marginal reconstruction gain falls below a threshold. To support more diverse and generalizable
reasoning, we further construct StructCoT, a large-scale interleaved
multimodal reasoning dataset with 1.05M samples spanning 44 task domains. Experiments show that the visual-update paradigm reduces newly generated visual
tokens by 55.6% on average without compromising reconstruction fidelity, and improves
multimodal reasoning by 3.3% over full-image generation. Trained with StructCoT and large-scale
multimodal data, DeltaV-2B further outperforms substantially larger open-source models by 8.4% on in-domain
multimodal reasoning evaluations and surpasses the comparable-scale Qwen3-VL-2B by 5.9% on external
multimodal reasoning and understanding
benchmarks. Code, models, and StructCoT will be released at https://github.com/Pengjie-W/DeltaV.
Read the full abstract ▾
All authors Pengjie Wang, Linger Deng, Zujia Zhang, Shaojie Zhang, Zhenbo Luo, Pei Fu, Jian Luan, Xiang Bai, Yuliang Liu
Computer Vision
Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery
Tianyi Song, Sierra Bonilla, Xinwei Ju et al. (+6 more)
Gaussian splatting is the current state-of-the-art for dense, deformable 3D anatomy reconstruction in
robot-assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from
surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track-anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non-SLAM methods that utilize camera trajectory priors. The code is available at https://track2map.github.io/.
Read the full abstract ▾
All authors Tianyi Song, Sierra Bonilla, Xinwei Ju, Evangelos Mazomenos, Danail Stoyanov, Adam Schmidt, Omid Mohareri, Sophia Bano, Francisco Vasconcelos
Computer Vision
Swapping Faces, Saving Features: A Dual-Purpose Pipeline for Pedestrian Privacy in ITS
Roba H. Farouk and Catherine M. Elias
Large-scale and diverse
datasets are needed to train AI models to take real-time decisions for autonomous vehicles (AVs), an intelligent transportation system (ITS) application. Pedestrian intention and trajectory prediction are critical models used in AVs, requiring
datasets involving diverse pedestrian images. Unrestricted access to these
datasets imposes serious security risks, like identity theft and pedestrian tracking. The challenge is to apply
privacy preservation procedures while maintaining the image attributes needed to train the models. Existing
privacy methods may preserve the pedestrian's
privacy, but degrade the image usability, which hinders the models' effectiveness. This work's focus is to implement a five-stage pipeline to protect pedestrians'
privacy through face swapping while keeping the essential facial attributes intact. It should be tailored to satisfy the
privacy needs of the Egy-DRiVeS
dataset. Moreover, Roop and Ghost-v2 face-swapping models are evaluated. Provenly, Roop outperforms Ghost-v2 in various aspects, as will be discussed. Consequently, Roop is the face-swapping model to be used in the pipeline to strike the balance between pedestrian
privacy via identity concealment and data usability via facial attribute preservation.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
HoloTetSphere: Unified TetSphere Mesh Reconstruction for Physical Simulations
YaQiao Dai, Renjiao Yi, Zhirui Gao et al. (+3 more)
Standard pipelines for physics-ready 3D reconstruction rely on a decoupled two-stage paradigm: extracting surface geometry followed by an error-prone tetrahedralization process. While recent Lagrangian methods like TetSphere Splatting attempt to bypass this by directly optimizing volumetric primitives, their homeomorphic constraints prevent
topology-adaptive optimization. Consequently, they produce disjoint tetrahedra rather than a single connected mesh, rendering the structures unsuitable for further physical simulations. To address this, we propose a
topology-adaptive framework for holistic tetrahedral mesh reconstruction through end-to-end topological and
geometric optimization. First, by coupling Gaussian spheres to tetrahedral elements and leveraging edge connections, we estimate a continuous opacity field for differentiable element
pruning. Next, jointly minimizing mesh smoothing energy and multi-view Gaussian rendering error drives alternating
geometric refinement while preserving topological adaptivity. Consequently, our approach effectively constructs a unified and topologically coherent tetrahedral mesh. Extensive experiments demonstrate that our method outperforms state-of-the-art techniques by achieving superior
geometric accuracy and producing coherent, single-connected tetrahedral meshes, thereby effectively bypassing the error-prone conventional tetrahedralization step for reconstructed surface meshes and streamlining downstream physical simulation.
Read the full abstract ▾
All authors YaQiao Dai, Renjiao Yi, Zhirui Gao, Wei Chen, Kai Xu, Chenyang Zhu
Computer Vision
Attribute Retrieving for Open-Vocabulary Endoscopic Compositional Referring Segmentation
Shun Liu, Nan Xi, Yang Liu et al. (+3 more)
Referring Image
Segmentation (RIS) aims to segment image regions specified by natural
language, enabling fine-grained and controllable visual understanding. Extending RIS to endoscopic imagery, however, presents unique challenges, including scarce high-quality annotations and complex, domain-specific image-text relationships. Although recent
vision-
language models demonstrate strong cross-domain
alignment, they often fail to capture fine-grained textual cues in endoscopic settings, resulting in suboptimal performance and limited generalization. To address these challenges, we introduce ReferEndoscopy, a large-scale
benchmark for RIS in the endoscopy field. Building on this
dataset, we propose the Attribute
Retrieval-based Endoscopic-RIS (AR-ERIS) framework for open-vocabulary endoscopic compositional referring
segmentation. AR-ERIS leverages attribute
retrieval for open-vocabulary endoscopic compositional referring
segmentation and is pretrained on the curated ReferEndoscopy
dataset, achieving state-of-the-art performance with strong generalization across both simulated and real-world endoscopic data. The
dataset and code will be publicly released upon completion of the review process.
Read the full abstract ▾
All authors Shun Liu, Nan Xi, Yang Liu, Tianyu Luan, Xuan Gong, David Doermann
Computer Vision
Classical Versus Deep Mirror-Symmetry Scoring: A Benchmark of Thirteen Methods
Maximilian Woehrer
Quantifying how mirror-symmetric an image is about a given axis (
symmetry scoring) underpins applications from visual aesthetics to medical imaging, yet proposed scoring methods have never been compared on a common, statistically grounded protocol. We
benchmark 13 scoring methods (9 collected from the literature; 4 introduced here) spanning from classical features to frozen deep features, across four single-axis and five multi-axis
datasets under a reflection-exact protocol with a chance-anchored, significance-tested discrimination skill. Deep backbones perform best on single-axis and harder multi-axis protocols. However, a classical histogram-of-oriented-gradients (HOG) descriptor trails the best frozen-network readout by a small (but significant) margin, is not statistically separable from the runner-up (a CNN-filter measure), and runs $\sim$300$\times$ faster on CPU. Our results show that discrimination concentrates in mid-scale oriented features, where deep backbones peak at a low or mid stage, and HOG peaks at a mid
cell size. Among existing methods, frozen deep features thus offer little over a tuned classical descriptor for measuring
symmetry; whether task-trained deep scorers can do better remains open. We release the scorers and harness, in imgsym, an open toolkit for image
symmetry detection and measurement.
Read the full abstract ▾
Computer Vision
WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving
Xuerun Yan, Zhexi Lian, Nuoheng Zhang et al. (+5 more)
Vision-
Language-Action (VLA) models have advanced end-to-end autonomous driving. However, existing methods either lack comprehensive world cognition or suffer from fragmented world foresight, inherently confining these models to reactive driving. To address this limitation, we propose WCog-VLA, a novel dual-level World-Cognitive VLA framework that successfully bridges semantic world
forecasting with generative world
evolution to achieve proactive autonomous driving. At the semantic level, WCog-VLA unifies world cognition and
reasoning by incorporating 3D spatial perception and injecting
agent tokens to capture the world dynamics, while concurrently enabling Game-theoretic Chain-of-Thought (Game-CoT)
reasoning. At the generative level, we introduce the Aligned Decoupled
Diffusion Transformer (ADDT) as a powerful generative world model that synthesizes physically-plausible joint multi-
agent trajectories. Through scene representation
alignment, ADDT reduces the number of denoising steps required and thus significantly accelerates inference. To facilitate strategic
reasoning, we further construct a large-scale
dataset featuring 85k Game-CoT annotations. Extensive experiments on the NAVSIM
benchmark demonstrate that WCog-VLA achieves a State-Of-The-Art (SOTA) PDMS score of 92.9.
Read the full abstract ▾
All authors Xuerun Yan, Zhexi Lian, Nuoheng Zhang, Shiyu Fang, Haoran Wang, Chen Lv, Jia Hu, Binyang Song
Computer Vision
Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception
Ludovica de Paolis, Marco Baroni, Alessandro Laio et al. (+1 more)
In computational
vision science, Convolutional Neural Networks (CNNs) have emerged as a popular model of biological
vision because of the
alignment they can exhibit with neural and behavioral data in humans and animals. However, it remains unclear to what extent this
alignment persists for visual tasks that extend beyond the canonical object recognition paradigm based on well defined semantic content. In this study, we diverge from the common object-centric view by focusing on another aspect of
vision: texture perception. We consider textures of different complexity generated with three different
algorithms from the same source images. Using a rank-based statistic, we quantify the information encoded in the internal representations of a CNN and three
Vision Transformers (ViTs), and we compare the similarity of these representations to those inferred from human psychophysics data. We find that the representation of textures is aligned in different ViTs, but not between the ViTs and the CNN; that ViTs form similar representations for textures of different complexity; that human performance in recognizing textures can be better predicted from ViTs representations rather than CNN representations. Taken together, these results suggest that ViTs may capture more faithfully than CNNs how texture patterns are visually processed by humans, and that the representations of texture stimuli in computational models may be driven by the network architecture.
Read the full abstract ▾
All authors Ludovica de Paolis, Marco Baroni, Alessandro Laio, Eugenio Piasini
Computer Vision
ARGUS: Accelerated, Robust, General, and Unsupervised Cell Tracking Solutions
Noah Jaitner, Kandice Tanner, Ingolf Sack et al. (+1 more)
Background and Objective: Quantitative analysis of
cell dynamics is central to modern biological research, providing critical insights into immune
cell interactions, disease progression, and drug mechanisms. Automated
cell tracking in time-lapse microscopy remains challenging due to noise, morphological variations, overlapping
cells, and dynamic events such as divisions and fusions. Methods: We present ARGUS, a framework for Accelerated, Robust, General, and Unsupervised
Cell Tracking Solutions. ARGUS combines adaptive
cell detection, dense Farneback optical-flow prediction, frame-to-frame linear assignment, and a sequence-level tracklet-refinement step that reconnects trajectory fragments across short temporal gaps. Results: On publicly available
Cell Tracking Challenge
datasets, ARGUS achieved detection accuracy of 0.905-0.971 and tracking accuracy of 0.897-0.964, with runtimes within 1 minute (5-6 seconds for 3 frames). Conclusions: ARGUS is a modular, interpretable framework that can be adapted to different imaging modalities and biological applications without training data or GPU infrastructure. The implementation is publicly available at https://github.com/Gitinc/argus
Read the full abstract ▾
All authors Noah Jaitner, Kandice Tanner, Ingolf Sack, Hossein S. Aghamiry
Computer Vision
Enhancing the KidSat Model: Integrating Geographical Encoding and Data Quality Assessment for Childhood Poverty Prediction
Hou Hin Ip, Ka Nam Lam, Joshua Man Yu Ng et al. (+4 more)
Accurate poverty mapping using satellite imagery is often hindered by (i) noisy and sparse survey-derived supervision, (ii) image quality issues such as cloud cover and image corruption, and (iii) lack of explicit spatial structure in image-only models. Building on the KidSat framework, we develop an enhanced pipeline that improves predictive accuracy via refined data preprocessing, systematic image quality assessment, and mathematically defined geographic encoding. First, we refine the fine-tuning target
matrix by resolving high-cardinality sparsity and reducing one-hot dimensionality from 103 to 51 via DHS re-aggregation. Second, we introduce a simple two-stage quality-screening procedure to filter heavily clouded or corrupted observations. Third, we fuse DINOv2 visual
embeddings with Spherical Harmonics (SH) location features. Across extensive experiments, these changes reduce MAE from 0.2167 to 0.1759, corresponding to an 18.83% relative reduction on the cluster-level severe-deprivation proportion scale. When extended from 16 to 33 African countries, the best-performing configuration achieves an overall MAE of 0.1658. We find that SH features consistently improve performance over the image-only backbone, whereas higher-capacity coordinate Multi Layer Perception augmentation (SH+SIREN) can underperform without carefully designed objectives. Finally, gradient-boosted tree heads (XGBoost/LightGBM) most effectively exploit nonlinear interactions in the fused visual-geographic representation. These findings provide a scalable and principled recipe for improving satellite-based socioeconomic predictions using only publicly accessible data.
Read the full abstract ▾
All authors Hou Hin Ip, Ka Nam Lam, Joshua Man Yu Ng, Makkunda Sharma, Seth Flaxman, Codie Gerlach-Wood, H Juliette T Unwin
Computer Vision
Progression as Latent Drift: Generative Forecasting of Slow-Evolving Pathologies
Yuxiang Feng, Juncheng Wang, Chao Xu et al. (+7 more)
Forecasting the future anatomy of slow-evolving neurodegenerative diseases could enable earlier, more targeted intervention and improve clinical trial design, but it remains challenging because true progression signals are subtle in longitudinal MRI. In this low-signal regime, transferring modern generative sequence models directly is unreliable: training is dominated by stable baseline anatomy and confounded by dense, sample-specific nuisance variation. We first provide a theoretical analysis that explains these failures through two modes. Identity collapse occurs when optimization is driven toward reproducing the current anatomy, which prevents the model from learning faint temporal change. The continuous interpolation trap arises when standard smooth networks cannot separate localized biological drift from pervasive noise, which leads to spurious changes that diffuse across the volume. To address both issues, we propose Latent Drift, a progressive generative framework that learns change in a compressed semantic representation rather than synthesizing full-resolution anatomy. This design removes pixel-level identity from the prediction target and concentrates model capacity on progression-relevant dynamics. We further apply Finite Scalar Quantization to the learned change representation, which suppresses small, high-frequency nuisance fluctuations while preserving consistent structural drift. Experiments on longitudinal 3D
brain MRI show that Latent Drift improves patient-specific neuro-
forecasting over
diffusion and autoregressive
transformer baselines across generative fidelity and clinically relevant evaluation metrics. Project page: \href{https://cutepkq.github.io/latent-drift}{https://cutepkq.github.io/latent-drift}.
Read the full abstract ▾
All authors Yuxiang Feng, Juncheng Wang, Chao Xu, Wenlong Hou, Huihan Wang, Yijie Qian, Yang Liu, Baigui Sun, Yong Liu, Shujun Wang
Computer Vision
Haibin Tian, Huichao Xie, Xuelin Qian et al. (+3 more)
Unmanned aerial vehicles (
UAVs) increasingly rely on visual grounding capabilities to localize task-relevant targets from diverse instructions in complex aerial scenes. Existing referring expression comprehension (REC)
benchmarks and methods, however, are largely built around text-only queries and single-object outputs, which limits their applicability to practical
UAV scenarios involving reference images,
multimodal instructions, absent targets, and multiple valid target instances. To address this gap, we introduce Universal Referring, a generalized
UAV referring task that jointly expands the query modality and the output cardinality. We construct UniRef-
UAV, a
multimodal benchmark that supports text-only, image-only, and text+image queries with modality-dependent target cardinality, where text-only and text+image queries admit no-target, single-target, and multi-target grounding while image-only queries focus on existence-aware single-instance grounding. It also provides in-domain and cross-domain evaluation protocols for visual-query generalization. We further present
UAV-URNet, a detection-style baseline that maps heterogeneous queries into a shared query space and predicts variable-size target sets through set prediction. Extensive experiments show that
UAV-URNet provides a stable and reproducible baseline with more consistent no-target discrimination and a more lightweight, reproducible implementation than large general-purpose MLLMs. Additional domain analysis, query-representation analysis, and ablation studies demonstrate that
multimodal queries help reduce visual-query ambiguity and promote a more unified query–target
alignment space. The annotations, visual query crops/images, train/validation/test splits, evaluation scripts, and baseline code will be made publicly available to facilitate reproducible research.
Read the full abstract ▾
All authors Haibin Tian, Huichao Xie, Xuelin Qian, Ruitao Lu, Junwei Han, Dingwen Zhang
Computer Vision
On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting
In-Hwan Jin, Hyeongju Mun, Joonsoo Kim et al. (+2 more)
Dynamic scene reconstruction remains challenging due to the heterogeneous and spatially varying nature of real-world motion. Although recent 3D Gaussian Splatting methods have introduced diverse deformation formulations for dynamic novel view synthesis, each method typically relies on a single deformation model within its representation, which limits robustness across diverse dynamic scenarios. In this work, we study a fundamental problem-multi-deformation modeling for dynamic 3D Gaussian representations-under two distinct integration constraints that differ in when and how multiple deformation experts interact during training. From a Mixture-of-Experts (MoE) perspective, we view multi-deformation modeling as the problem of combining multiple specialized deformation models within a unified 3D representation. We first introduce Mixture of Deformation Experts (MoDE), which integrates multiple deformation experts directly into the deformable Gaussian Splatting pipeline through joint optimization. In MoDE, experts operate on a shared canonical Gaussian representation, enabling multi-deformation modeling without introducing additional training stages or modifying the original optimization schedule. In contrast, we further present Mixture of Experts for Dynamic Gaussian Splatting (MoE-GS) under a different integration constraint, where deformation experts are optimized independently and combined through a separate routing stage. As a result, expert interaction occurs over non-canonical Gaussian representations after individual optimization. Together, these two approaches provide alternative strategies for multi-deformation modeling, clarifying how integration constraints shape the design and behavior of deformation experts in dynamic 3D Gaussian representations. Our code is available at: https://github.com/cvsp-lab/MoE-GS-studio.
Read the full abstract ▾
All authors In-Hwan Jin, Hyeongju Mun, Joonsoo Kim, Kugjin Yun, Kyeongbo Kong
Computer Vision
HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition
Neelu Madan, Rongzhen Zhao, Andreas Mogelmose et al. (+4 more)
Slot
attention is a powerful framework for object-centric learning, decomposing visual scenes into latent slots through iterative competitive
attention. However, existing methods share two critical limitations: they decompose scenes into a flat set of slots at a single granularity, and this decomposition is based on appearance rather than semantics. Yet humans understand scenes through semantic hierarchies: separating foreground from background, recognizing object categories, and identifying individual instances. Crucially, such semantic hierarchies cannot emerge without supervision, because category names are human constructs, not visual patterns. We propose Hierarchical Slot
Attention (HSA), which learns multi-granularity semantic scene decomposition from a single model. HSA decomposes scenes at three levels: holistic (foreground/background), semantic (object categories), and panoptic (individual instances). Using only 10% labeled data, combined with hierarchical
alignment loss, HSA learns all three levels jointly. We further introduce grouping purity and containment to measure whether the hierarchy is encoded in representation space, not just output masks. Experiments on COCO and PASCAL VOC demonstrate that HSA outperforms the strongest flat baseline by up to \textbf{$+$41.5} ARI at holistic, \textbf{$+$14.6} at semantic, and \textbf{$+$10.4} at panoptic level on COCO, with even larger gains on Pascal VOC, while requiring a single model instead of three. Code will be made available upon acceptance.
Read the full abstract ▾
All authors Neelu Madan, Rongzhen Zhao, Andreas Mogelmose, Juho Kannala, Joni Pajarinen, Graham W. Taylor, Thomas B. Moeslund
Computer Vision
SkelGen4D: Weakly-Supervised Skeleton-Based 4D Generation for Text-Driven Mesh Animation
Hao Feng, Zhi Zuo, Jia-Hui Pan et al. (+6 more)
We study 4D generation to synthesize temporally coherent sequences of 3D geometry for animation and content creation. In contrast to existing SDS-based optimization methods and
video-driven animation approaches, we adopt a skeleton-driven animation framework aligned with standard industrial pipelines, which enables explicit control and editing. To this end, we propose SkelGen4D, a weakly supervised feed-forward framework for text-driven mesh animation that generates explicit skeleton motions without requiring per-frame skeleton annotations. SkelGen4D first recovers temporally consistent pseudo-skeletons from animated meshes via differentiable fitting, and then generates text-conditioned skeleton motion sequences in a feed-forward manner, further refined with Motion-GRPO to ensure temporally coherent, physically plausible, and articulated animation. We evaluate our method on two large-scale
benchmarks, Truebones Zoo and
Diffusion4D. Our results show that our weakly supervised skeleton modeling matches or surpasses fully supervised baselines while scaling to diverse object categories for high-quality text-driven mesh animation. Further, our method supports flexible motion editing and is aligned with standard animation production pipelines.
Read the full abstract ▾
All authors Hao Feng, Zhi Zuo, Jia-Hui Pan, Ka-Hei Hui, Zhengzhe Liu, Dian Zhang, Haoran Xie, Bin Sheng, Jingyu Hu
Computer Vision
Closing the Null Space: Guidance-Aware Quantization for Classifier-Free Diffusion
Abdullah Al Shafi and Sumaiya Rahim Suma
Deploying classifier-free guidance (CFG)
diffusion models under real-world compute budgets requires quantization, yet existing post-training quantization (PTQ) methods treat CFG models as single-branch networks, ignoring the paired conditional/unconditional structure that CFG inference fundamentally relies on. This structural blind spot has two consequences. At the system level, the two-pass CFG execution pattern imposes a latency overhead that parameter-count and bit-operation metrics conceal entirely, and commodity INT8 inference stacks fail to realize the theoretical efficiency gains that BOPs calculations promise. At the algorithmic level, calibrating against the guidance gap alone admits an exact null space: a quantized model can achieve perfect gap-fidelity diagnostics while the unconditional branch drifts arbitrarily, corrupting every guided prediction at inference time. This paper terms this the branch-drift trap, proves its existence analytically, and confirms it empirically through a false-positive result in which the best-calibrated model by standard diagnostics simultaneously produces the worst sample quality. To close the trap, Guidance-Aware Mixed Precision (GAMP) is proposed, which calibrates directly on the guided prediction, derives per-layer activation-bit sensitivity from guided-output degradation, and allocates bits via a greedy knapsack – provably preventing unconditional branch drift by construction.
Read the full abstract ▾
Computer Vision
TVTA: Trajectory-Aware Viseme-Guided Temporal Aggregation for Event-Based Lip Reading
Jingrong Zheng, Hongwei Ren and Xiangqian Wu
Event-based lip reading has recently emerged as a promising direction for visual
speech recognition, benefiting from the high temporal resolution and motion sensitivity of event cameras. However, existing methods typically perform spatial compression before sufficient temporal modeling, which may suppress sparse and localized motion trajectories that are crucial for distinguishing similar lip movements. Moreover, most current approaches optimize temporal representations mainly at the word-classification level, leaving the underlying articulatory structure weakly constrained. To address these limitations, we propose a temporally enhanced framework for event-based lip reading. First, we introduce Trajectory-Aware Differential Aggregation (TDA), which performs local temporal modeling at each spatial location before adaptive spatial aggregation. Second, we propose Viseme-Guided Aggregation (VGA), a unified temporal module composed of a CTC decoder and a viseme-guided gated aggregation branch, which injects viseme-aware sequence supervision and improves final temporal aggregation for word recognition. Third, we incorporate an EMA teacher–student training strategy to enhance robustness under strong event perturbations. Experiments on the DVS-Lip
benchmark verify the effectiveness of the proposed design, and extensive ablation studies further validate the contributions of TDA, VGA, and teacher–student consistency. Qualitative decoding results also demonstrate that the proposed CTC-based temporal modeling learns meaningful viseme-aware structure from event streams.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Induction
Sophia Koehler, Antonia Wüst, Inga Ibs et al. (+5 more)
A central challenge in building intelligent systems is enabling
agents to jointly perceive complex inputs, form hypotheses about hidden patterns, and design informative experiments to test them. To study this problem, we propose ZendoWorld, a controlled interactive environment in which
agents must infer a logical rule about visual game observations, acquire information by proposing new scenes, and refine their hypotheses based on feedback from the game environment. We evaluate several
agents spanning pure VLM
reasoning,
Bayesian particle filtering, dynamic concept discovery, and neuro-symbolic methods. Our main findings are: (1) high accuracy in predicting labels for observed examples does not imply recovery of the underlying rule; (2) perception and induction are distinct bottlenecks for different
agent classes; and (3) VLM-based
agents propose near-uninformative experiments, failing to actively reduce hypothesis uncertainty. To compare these results, we collect human data on the task, which reveals a gap in inductive
reasoning, particularly for more complex rules. Overall, ZENDOWORLD takes an important step toward evaluating intelligent
agents and identifies concrete avenues for improvement, particularly in domains like scientific discovery.
Read the full abstract ▾
All authors Sophia Koehler, Antonia Wüst, Inga Ibs, Wasu Top Piriyakulkij, Wolfgang Stammer, Constantin Rothkopf, Kevin Ellis, Kristian Kersting
Computer Vision
Multimodal 3D LUT Generation via StatLUT with Statistical Features for Photorealistic Style Transfer
Yifan Wang, Zhixiang Hao, Yu Wang et al. (+1 more)
Photorealistic Style Transfer (PST) aims to transfer the color and tonal style of a reference to a content image while strictly preserving its structural integrity. However, existing deep learning-based methods inherently suffer from semantic
entanglement caused by pre-trained image encoders, leading to unnatural spatial distortions. Moreover, current pixel-level mapping paradigms often ignore color gamut
topology, resulting in color banding, while also lacking the
multimodal capability for intuitive text-driven control. To address these bottlenecks, we propose StatLUT, an innovative
multimodal framework for 3D LUT generation. First, we bypass traditional encoders and introduce a Lab-Extractor to derive spatially-agnostic statistical features, fundamentally decoupling color distributions from structural semantics to ensure artifact-free rendering. Second, we formulate LUT generation as a
Transformer-based Seq2Seq translation task, utilizing a Multi-dimensional Residual Mapper (MR-Mapper) to predict topologically smooth 3D LUTs. Finally, to break the single-modal barrier, we propose the H-Diffuser, a lightweight
Diffusion Transformer that directly synthesizes statistical features from natural
language prompts, enabling flexible text-driven color grading. Extensive experiments on standard
benchmarks demonstrate that StatLUT significantly outperforms state-of-the-art methods in both visual quality and quantitative metrics, pioneering a highly robust and flexible paradigm for
multimodal photorealistic style transfer.
Read the full abstract ▾
All authors Yifan Wang, Zhixiang Hao, Yu Wang, Congchao Zhu
Computer Vision
LUMI: Tokenizer-Agnostic LLM-Based Lossless Image Compression
Chris Xing Tian, Chengkai Wu, Ziyu Wang et al. (+6 more)
Large
language model (LLM)-based lossless image compression methods typically represent pixel data through the native text interface of a pretrained model, converting pixel values into
token sequences that the LLM processes through its vocabulary head. This design shows that pretrained
language models can provide probability estimates for image coding, but it also couples compression to tokenizer behavior, vocabulary-specific numeric
tokens, and model-family-specific adaptation. In this paper, we present LUMI (LLM-based Unified Model-agnostic lossless Image compression), a tokenizer-agnostic framework for lossless RGB image compression with frozen LLM backbones. LUMI replaces pixel-as-text tokenization with a pixel
embedding module that maps raw intensity and channel information into the continuous
embedding space of the LLM. It further introduces intra-patch position encoding to retain two-dimensional spatial structure after flattening, and uses a 256-way prediction head to produce probabilities over the native pixel alphabet. Only the pixel
embedding, position encoding, soft-prefix parameters, and prediction head are trained, while the LLM backbone remains fixed. Experiments on natural, medical, and remote-sensing image
benchmarks with LLaMA, Qwen, and Gemma backbones show that LUMI provides a unified interface across tokenizer families, achieves competitive compression rates, and improves cross-domain robustness over tokenizer-based LLM compression baselines. These results formulate LLM-based lossless image compression as pixel-space adaptation of frozen foundation models rather than tokenizer-specific
language-symbol modeling.
Read the full abstract ▾
All authors Chris Xing Tian, Chengkai Wu, Ziyu Wang, Rongqun Lin, Kecheng Chen, Xiandong Meng, Haoliang Li, Shiqi Wang, Siwei Ma
Computer Vision
Benchmark Evaluation of Federated Learning on Multi-organ Images
Junbin Mao, Xu Tian, Jianchun Zhu et al. (+2 more)
The
privacy requirements of medical data and its substantial variations across organs and modalities hinder the clinical implementation of medical AI.
Federated learning (FL) is a feasible approach to overcome these challenges. Due to the continuous emergence of FL
algorithms and the highly heterogeneous nature of medical data, objectively evaluating their performance in real-world clinical settings remains difficult. Therefore, a comprehensive
federated medical imaging
benchmark, serving as a unified evaluation standard, is crucial for advancing the technology toward reliable clinical application. Existing
federated medical imaging
benchmarks have not yet adequately incorporated state-of-the-art
algorithms, are limited to data from single organs or modalities, and overly emphasize model accuracy, making it difficult to comprehensively assess the overall efficacy of FL in real-world medical environments. To address these challenges, we developed the MobenFL
benchmark. This
benchmark integrates 20 cutting-edge FL
algorithms and 22 medical imaging
datasets, covering 12 critical organs across the human body, surpassing existing
benchmark in breadth. In terms of evaluation dimensions, MobenFL not only assesses performance but also systematically incorporates key metrics such as algorithmic efficiency and
privacy protection capabilities. Additionally, it conducts specialized evaluations for complex real-world clinical scenarios involving different diseases, devices, and imaging modalities, thereby providing a comprehensive and in-depth evaluation framework for the clinical application of FL in the medical field.
Read the full abstract ▾
All authors Junbin Mao, Xu Tian, Jianchun Zhu, Ludi Li, Jin Liu
Computer Vision
Metrics or Mirage? An Audit of Evaluation Inconsistencies in Colonoscopy Polyp Segmentation Benchmarks
Aisha Urooj, Zain Ul Abdien and Neelu Madan
Progress in colonoscopy polyp
segmentation is routinely reported through leaderboard comparisons on a small set of public
benchmarks. We argue that this apparent progress is difficult to verify: a systematic audit of 27 papers published between 2015 and 2026 reveals three structural problems in how the community evaluates models. First, 25 of 27 papers omit the Hausdorff distance. Hausdorff distance is a boundary-accuracy metric with direct clinical relevance for detecting flat or small polyps, and is a standard in radiotherapy
segmentation. Second, at least five incompatible train/test split protocols co-exist across papers reporting results on the same two
datasets (Kvasir-SEG and CVC-ClinicDB), making published Dice scores non-comparable even when they appear in the same leaderboard column. Third, 26 of 27 papers make performance claims without any statistical significance test. Strikingly, four papers published after the Metrics Reloaded framework~\cite{metricsreloaded2024} (Maier-Hein et al., Nature Methods 2024) perpetuate these same problems, suggesting that general-purpose metric guidance has not yet reached the colonoscopy sub-community. To show these problems are not merely cosmetic, we re-evaluate five representative models under three controlled protocols with a single uniform scorer, and find that the reported metric conceals large boundary and recall failures, that the “best” model changes with the metric, and that near-tied rankings reverse across random splits. We propose a five-point Polyp
Segmentation Reporting Checklist~(PSRC) as a lightweight, domain-adapted corrective.
Read the full abstract ▾
Computer Vision
TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentation
Hyeonseop Song, Seokhun Choi and Hoseok Do
Large-vocabulary instance
segmentation is constrained by long-tailed category distributions and fine-grained inter-class ambiguity. While data synthesis offers a promising alternative, current paradigms have complementary limitations: text-to-image (T2I) methods inherit noisy pseudo-labels and struggle on rare classes, whereas copy-paste methods compromise contextual realism. To address these issues, we propose a hybrid pipeline coupling T2I generation with context-aware image-to-image (I2I) editing. The T2I branch provides broad category and scene diversity, while a teacher-student scheme ensures label reliability by selectively retaining only
prompt-specified categories. To strengthen supervision for rare classes, we introduce VRAIN (Verified Rare-class Augmentation via INstructed editing), a novel I2I editor. VRAIN inserts high-confidence instances at semantically appropriate locations within in-the-wild scenes, yielding semantically coherent and visually natural edits that reduce domain gaps and enable targeted augmentation. On the LVIS
benchmark, our method surpasses existing baselines, improving overall AP by up to +4.0 points and rare-class AP by up to +9.5 points, while scaling effectively with backbone capacity. Our project page is available at https://seokhunchoi.github.io/TMI
Read the full abstract ▾
Computer Vision
Unpaired Joint Distribution Modeling via Multi-Scale Image Representations
Yihang Zou, Hui Zhang, Zuowei Shen et al. (+1 more)
This paper studies the problem of learning a joint distribution from marginal observations, which is inherently ill-posed due to the ambiguity of feasible couplings. We propose LUD-MSR, a latent-variable probabilistic framework that models the joint distribution via auxiliary representations and optimizes evidence lower bounds using only marginal data. Under mild assumptions, we establish an upper bound on the distribution approximation error. This analysis reveals a trade-off in representation learning between domain consistency and information preservation. To address this trade-off, we introduce a Multi-Scale image Representation (MSR) mapping that exploits structural similarity at coarse scales while suppressing domain-specific variations. We show that MSR achieves a more favorable balance of this trade-off compared to existing approaches. Experiments on real-world denoising
benchmarks, including cryo-electron microscopy (cryo-EM), demonstrate the effectiveness of the proposed framework.
Read the full abstract ▾
All authors Yihang Zou, Hui Zhang, Zuowei Shen, Chenglong Bao
Computer Vision
Mingjia Shi, Shuo Wang, Xiaobo Wang et al. (+7 more)
Vision-
language alignment, the stage that bridges pretrained
vision encoders and large
language models, is widely treated as a form of pretraining requiring full-parameter updates. We challenge this view and investigate what happens when low-rank adaptation is applied to the LLM during this stage instead. We find that low-rank
alignment not only reduces computational costs but also outperforms full-parameter
alignment on most
benchmarks. To understand this phenomenon, we systematically characterize the implicit biases introduced by low-rank adaptation during
alignment. Empirically, we find that low-rank
alignment shifts model behavior from hallucinatory to conservative and preserves per-
token linear separability of visual features that full-parameter
alignment disrupts, a phenomenon we term LS-curse. Geometrically, low rank aligned models exhibit more homogeneous and structurally stable visual representations, maintaining modality-specific knowledge rather than prematurely fusing entity-level semantics. Theoretically, we establish two
theorems showing that low-rank
alignment induces preferences for parameter subspaces with flat gradients and feature subspaces robust to perturbations, providing a principled explanation for the observed structure-preserving behavior. Extensive experiments cover ablation over 100
alignment configurations, three families of low-rank operators, and various rank, encoder, and other settings.
Read the full abstract ▾
All authors Mingjia Shi, Shuo Wang, Xiaobo Wang, Sifan Zhou, Kai Wang, Tianyu Fu, Chenxu Zhao, Anyang Su, Ping Jiang, Minghui Wu
Computer Vision
Dual-Correlation Hypergraph Network for Unaligned RGBT Video Object Detection and A Large-scale Benchmark
Qishun Wang, Yapeng Li, Zhengzheng Tu et al. (+2 more)
RGB-Thermal (RGBT)
Video Object Detection (VOD) has gained significant
attention because of the limitations of conventional RGB-based VOD methods under challenging conditions, such as low light, heavy fog, and adverse weather, etc. However, spatial misalignment commonly exists between RGBT image pairs. To address this, we propose a Dual-Correlation Hypergraph Network (DCHNet) that captures high-dimensional complementary information by explicitly modeling two types of correlations: temporal correlation across consecutive frames and spatial correlation from cross-modal features. Specifically, we first design a Patch-based Spatial
Alignment Module (PSAM) to sequentially align the
multimodal features at the local region level. Subsequently, we propose a Dual Hypergraph Fusion Module (DHFM), which constructs temporal and
multimodal hypergraphs, respectively, to enhance object characteristic through dual-correlation learning. Furthermore, the field currently lacks a large-scale, scene-diverse
benchmark dataset for comprehensive evaluation. Therefore, we construct DVT-VOD1000, a large-scale RGBT VOD
dataset containing 1,000
video sequences with 103,464 RGBT image pairs. The
dataset covers diverse scenarios, including campuses, parks, traffics, rural areas, night scenes, rainy weather, and snowy weather. Comprehensive experiments on VT-VOD50 and our DVT-VOD1000 demonstrate that DCHNet achieves state-of-the-art detection accuracy. The
dataset and source code will be made publicly available on https://github.com/tzz-ahu/ to support academic research.
Read the full abstract ▾
All authors Qishun Wang, Yapeng Li, Zhengzheng Tu, Chenglong Li, Bin Luo
Computer Vision
Leveraging Color Naming for Image Enhancement
David Serrano-Lozano, Luis Herranz, Michael S. Brown et al. (+1 more)
Enhancing images to make them visually appealing is a persistent challenge in computer
vision. Many deep-learning methods train models on paired
datasets to replicate expert editing styles. However, these approaches struggle with two key issues: (1) interpretability and (2) a parametrization suitable for user adjustments. To address these challenges, we present NamedCurves+, an approach inspired by the concept of Color Naming, a universal set of familiar colors widely used in software tools for intuitive editing. Our method integrates color names into a learning-based framework, enabling global adjustments for each named color through tone curves. To address local image variations, we incorporate a
transformer block that captures spatial dependencies, enabling context-aware edits across the image. NamedCurves+ enhances the retouching process's interpretability and supports user interaction, allowing flexible modifications of individual tone curves to refine the retouched image according to personal preferences. Extensive experiments on tasks such as image retouching, tone mapping, and exposure correction demonstrate that NamedCurves+ outperforms state-of-the-art methods. Notably, our approach is both explainable, as the tone curves explicitly represent how each color name contributes to the enhancement, and interactive, allowing users to customize the retouching process and achieve results tailored to their liking.
Read the full abstract ▾
All authors David Serrano-Lozano, Luis Herranz, Michael S. Brown, Javier Vazquez-Corral
Computer Vision
LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Action
Qi Lyu, Baicheng Liu, Xudong Wang et al. (+3 more)
Vision-
language-action (VLA) models aim to map
multimodal inputs to
robot actions. However, most existing approaches struggle to cover complex dynamic scenarios due to treating all visual
tokens uniformly and
reasoning with human-selected factors, which lack mechanisms to emphasize task-critical evidence and ignore underlying factors. To address this issue, we propose LEEVLA, a VLA architecture for seeing what matters in Latent Environment
Evolution that explicitly guides the model toward informative regions while preserving the structured
evolution of latent world representations. To identify salient and instruction-relevant regions, we introduce drift-guided dynamic prioritization (DGDP), which combines dynamic position prioritization (DPP) with semantic drift guidance (SDG) to guide the VLA
agent where to attend during training. On top of this, we introduce structured feature flow generation (SFFG), which models how these prioritized features should evolve in latent space via prototype-to-periphery (P2P) prediction, and a mutual-neighborhood contrastive (MC) loss to maintain topological consistency among neighborhoods. Together, DGDP and SFFG form a task-aware "where-how" training framework. Extensive experiments on VLA
benchmarks show that LEEVLA consistently outperforms prior methods, confirming that explicit task-evidence guidance and structured latent
reasoning are both crucial for scalable VLA. Our code is available at https://github.com/LyuQi127/LEEVLA.
Read the full abstract ▾
All authors Qi Lyu, Baicheng Liu, Xudong Wang, Jiahua Dong, Lianqing Liu, Zhi Han
Computer Vision
Attention-Based Segmentation of WMHs and Differentiation of Vascular vs. Demyelinating Lesions
Aina Tur-Serrano, Gabriel Moyà-Alcover and Francisco J. Perales López
White Matter Hyperintensities (WMHs) are commonly observed in
brain Magnetic Resonance Imaging (MRI) scans. They are associated with various neurological conditions, including vascular and inflammatory demyelinating diseases. Despite differing in etiology, WMHs from these conditions often appear similar on Fluid Attenuated Inversion Recovery (FLAIR) images. This similarity makes differential diagnosis challenging. In this work, we highlight the potential of combining
attention-based
segmentation with feature-driven classification. This approach supports more accurate and efficient classification between vascular and demyelinating white matter pathologies. For
segmentation, we evaluate the effectiveness of
attention mechanisms, specifically the Bottleneck
Attention Module (BAM) and the Convolutional Block
Attention Module (CBAM). We also test different architectures, particularly
Attention U-Net. In addition, we explore advanced training strategies, such as patch-based learning and a 2.5D approach, to enhance lesion detection. After
segmentation, we extract morphological features from the lesion masks. We then use them to classify WMHs based on their underlying cause. Our experiments utilize five publicly available
datasets with diverse imaging protocols to promote model generalizability, despite limited sample sizes. The results suggest that
attention-based
segmentation and feature-driven classification offer a promising direction for discriminating vascular and demyelinating white matter lesions. Further validation in larger clinical cohorts is still needed.
Read the full abstract ▾