Computer Vision
Through the Lens of Character: Resolving Modality-Role Interference in Multimodal Role-Playing Agent
Yihong Tang, Kehai Chen, Xuefeng Bai et al. (+1 more)
The advancement of
Multimodal Large
Language Models (MLLMs) has expanded Role-Playing
Agents (RPAs) into visually grounded environments. However, human
vision is inherently subjective and identity-driven, whereas existing MLLMs extract objective, character-agnostic features for general tasks. In RPAs, this generic visual noise overpowers fragile character traits, causing Modality-Role Interference (MRI), where
agents struggle to integrate visual grounding and character consistency. To address this, we introduce the training-free Character-Aware Visual Intervention (CAVI) framework, enabling
agents to perceive the world through the lens of character. CAVI systematically targets MRI: macroscopically, Character-Guided
Token Pruning (CTP) restricts the visual receptive field to role-relevant entities; microscopically, Orthogonal Feature Modulation (OFM) projects
tokens onto a character-context subspace to extract aligned facts; and during decoding, Modality-Adaptive Role Steering (MARS) dynamically optimizes steering intensity based on visual reliance. Extensive experiments show CAVI effectively alleviates MRI, significantly enhancing character-consistent
multimodal interactions.
Read the full abstract ▾
All authors Yihong Tang, Kehai Chen, Xuefeng Bai, Min Zhang
Computer Vision
SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation
Shanwen Tan, Hao Li, Jingtao Zhang et al. (+4 more)
Streaming long-
video generation faces a central challenge in continuous semantic switching, requiring adaptive
memory to preserve coherent visual
evolution. Current approaches rely on cache rebuilding at
prompt boundaries or fixed
memory budgets, but they introduce redundant computation and limit flexible semantic adaptation. This limitation arises from a mismatch between cached
video history and
prompt updates, as
memory preserves visual continuity while
prompt switches demand rapid semantic adaptation. Motivated by this observation, we present SWIFT, Semantic Windowing and Injection for Flexible Transitions, a training-free framework for multi-
prompt long-
video generation that enables efficient semantic switching while preserving temporal coherence in causal
video diffusion models. SWIFT introduces a lightweight Semantic Injection Cache that augments cached
video memory rather than reconstructing it from scratch at every
prompt boundary. To avoid uniformly perturbing all
attention channels, we further perform head-wise semantic injection, so that each
attention head receives a
prompt update proportional to its
alignment with the current
video state. In addition, we introduce an Adaptive Dynamic Window that allocates temporal
memory according to
prompt phase, using larger local context near switching boundaries and smaller windows during stable segments to reduce average inference cost. To preserve long-range semantic consistency under compressed local
attention, we further maintain segment-level semantic anchors that summarize
prompt-conditioned
video history and reintroduce it as compact
memory tokens. Compared with current state-of-the-art methods, SWIFT preserves generation quality while achieving 22.6 FPS on a single H100 GPU, establishing a substantially more efficient solution for multi-
prompt long-
video generation. Our code is available at https://github.com/ShanwenTan/SWIFT.
Read the full abstract ▾
All authors Shanwen Tan, Hao Li, Jingtao Zhang, Xiaosong Jia, Xue Yang, Shaofeng Zhang, Yanyong Zhang
Computer Vision
Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairs
Yunhong Lu, Qichao Wang, Hengyuan Cao et al. (+2 more)
Existing preference
datasets for text-to-image models typically store only the final winner/loser images. This representation is insufficient for rectified flow (RF) models, whose generation is naturally indexed by a specific prior noise sample and follows a nearly straight denoising trajectory. In contrast, prior DPO-style
alignment for
diffusion models commonly estimates trajectories using an independent forward noising process, which can be mismatched to the true reverse dynamics and introduces unnecessary variance. We propose Prior Noise-Aware Preference Optimization (PNAPO), an off-policy
alignment framework specialized for rectified flow. PNAPO augments preference data by retaining the paired prior noises used to generate each winner/loser image, turning the standard (
prompt, winner, loser) triplet into a sextuple. Leveraging the straight-line property of RF, we estimate intermediate states via noise-image interpolation, which constrains the trajectory estimation space and yields a tighter surrogate objective for preference optimization. In addition, we introduce a dynamic regularization strategy that adapts the DPO regularization based on (i) the reward gap between winner and loser and (ii) training progress, improving stability and sample efficiency. Experiments on state-of-the-art RF T2I backbones show that PNAPO consistently improves preference metrics while substantially reducing training compute.
Read the full abstract ▾
All authors Yunhong Lu, Qichao Wang, Hengyuan Cao, Xiaoyin Xu, Min Zhang
Computer Vision
FlashAR: Efficient Post-Training Acceleration for Autoregressive Image Generation
Junkang Zhou, Yefei He, Feng Chen et al. (+2 more)
Large-scale autoregressive models have demonstrated remarkable capabilities in image generation. However, their sequential raster-scan decoding relies on strictly next-
token prediction, making inference prohibitively expensive. Existing acceleration methods typically either introduce entirely new generation paradigms that necessitate costly pre-training from scratch, or enable parallel generation at the expense of a training-inference gap or altered prediction objectives. In this paper, we introduce FlashAR, a lightweight post-training adaptation framework that efficiently adapts a pre-trained raster-scan autoregressive model into a highly parallel generator based on two-way next-
token prediction. Our key insight is that effective adaptation should minimize modifications to the pre-trained model's original training objective to preserve its learned prior. Accordingly, we retain the original AR head as a horizontal head for row-wise prediction and introduce a complementary, lightweight vertical head for column-wise prediction. To facilitate efficient adaptation, we branch the vertical head from an intermediate layer rather than the final layer, bypassing the inherent horizontal head bias. Moreover, since horizontal and vertical predictions capture complementary dependencies whose relative importance varies across target positions, we employ a learnable fusion gate to dynamically combine the two predictions at each position. To further reduce adaptation cost, we propose a two-stage adaptation pipeline: the vertical head is first initialized through adaptation from the pre-trained autoregressive model before jointly fine-tuned with backbone to adapt to the new decoding paradigm. Extensive experiments on LlamaGen and Emu3.5 show that FlashAR achieves up to a 22.9x speedup for 512x512 image generation through a lightweight post-training with merely 0.05% of the original training data.
Read the full abstract ▾
All authors Junkang Zhou, Yefei He, Feng Chen, Weijie Wang, Bohan Zhuang
Computer Vision
Evading Visual Aphasia: Contrastive Adaptive Semantic Token Pruning for Vision-Language Models
Jie Ma, Yihang Liu, Zhike Qiu et al. (+2 more)
Are low-
attention visual
tokens truly redundant in
vision-
language reasoning? Existing
pruning methods often assume so, ranking visual
tokens by shallow text-to-image
attention and discarding low-scoring patches to accelerate LVLM inference. We show that this scalar criterion is unreliable for compositional
reasoning:
tokens ignored in early layers can later become essential for resolving secondary objects, spatial relations, and contextual cues. Premature
pruning can therefore induce Visual Aphasia, a failure mode in which the model loses visual grounding and falls back on
language priors. We introduce COAST (COntrastive Adaptive Semantic
Token Pruning), a training-free
pruning framework that casts compression as adaptive semantic routing. COAST uses native cross-modal
attention to identify query-specific anchors and estimate contextual dispersion via
attention entropy, then adapts the retention trade-off between semantic evidence and spatial context. It further uses a contrastive routing score to preserve both anchor-aligned evidence and complementary spatial context. Across seven
benchmarks, COAST reduces visual
tokens by 77.8% and achieves a 2.15x latency speedup while retaining 98.64% of the original average performance. Beyond a single backbone or compression setting, COAST consistently outperforms strong
pruning baselines across
token budgets and generalizes across multiple LVLM families, showing that adaptive semantic routing is a robust alternative to one-shot scalar
pruning
Read the full abstract ▾
All authors Jie Ma, Yihang Liu, Zhike Qiu, Jiayi Ji, Xiaoshuai Sun
Computer Vision
AtteConDA: Attention-Based Conflict Suppression in Multi-Condition Diffusion Models and Synthetic Data Augmentation
Shogo Noguchi
Recent conditional image generation methods can improve controllability by generating images that are faithful to conditions such as sketches, human poses,
segmentation maps, and depth. By applying these techniques to image augmentation while preserving annotations, generated images can be used as additional training data and can improve recognition performance. However, for high-level driving tasks such as traffic-rule extraction and driving-behavior understanding, simply using annotations as conditions is insufficient. Instead, images must be augmented while preserving the detailed high-level structure of the original scene. One possible solution is to use multiple conditions so that generated images retain diverse structural cues after generation. However, when multiple conditions are used, conflicts among conditions can prevent reliable structure preservation. In this work, we input semantic
segmentation, depth, and edges extracted from the original image into a multi-condition image generation model, thereby providing rich structural information as conditions. We further propose a modeling approach for handling conflicts among multiple conditions and show that it enables image generation with stronger structural preservation. We also build a generation framework and evaluation protocol for driving tasks, establishing a basis for comparison with prior and future models. As a result, this work contributes to image generation research by addressing condition conflicts in multi-condition generation and provides an important step toward mitigating data scarcity in high-level autonomous-driving tasks.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Perception Without Engagement: Dissecting the Causal Discovery Deficit in LMMs
Jiafeng Liang, Zhihao Zhu, Zihan Zhang et al. (+7 more)
Although Large
Multimodal Models (LMMs) have achieved strong performance on general
video understanding, their susceptibility to textual prior shortcuts during causal discovery has been recognized as a critical deficit. The underlying mechanisms of this phenomenon remain incompletely understood, as existing
benchmarks only measure response accuracy without revealing the sources and extent of the deficit. We introduce ProCauEval, a perturbation-based evaluation protocol that shifts from outcome assessment to mechanism diagnosis, probing causal discovery through five controlled configurations that systematically manipulate visual and textual modalities to decompose their respective contributions to model behavior and dissect the failure modes. Evaluating 17 mainstream LMMs, we find that models faithfully perceive
video content yet systematically underexploit it during causal
reasoning. We further observe that stronger post-training amplifies rather than mitigates textual prior reliance, and that higher baseline performance correlates with greater fragility under perturbation. To address these, we propose Anti-
Distillation Policy Optimization (ADPO), a reinforcement learning framework built on negative teacher
alignment, which augments GRPO by explicitly pushing the policy away from a prior-only counterfactual teacher induced by visual corruption. Specifically, ADPO maximizes the divergence between the policy distributions conditioned on the original and visually corrupted inputs, thereby forcing the model to ground its
reasoning in visual evidence rather than textual shortcuts. Extensive experiments show that ADPO improves visual engagement without sacrificing fundamental comprehension, thus offering a preliminary step toward reliable causal discovery.
Read the full abstract ▾
All authors Jiafeng Liang, Zhihao Zhu, Zihan Zhang, Baoqi Ren, Shixin Jiang, Runxuan Liu, Tao Ren, Ming Liu, See-Kiong Ng, Bing Qin
Computer Vision
Relational Retrieval: Leveraging Known-Novel Interactions for Generalized Category Discovery
Yulin Xu, Chunqi Guo, Yuanzhen Shuai et al. (+1 more)
In this study, we tackle Generalized Category Discovery (GCD) via a Relational
Retrieval perspective, explicitly coupling labeled and unlabeled data through bidirectional knowledge transfer. While existing methods treat these sources separately, missing valuable interaction opportunities, we propose Relational Pattern Consistency (RPC) that enables mutual enhancement. RPC employs One-vs-All classifiers for soft ID/OOD decomposition, then introduces two mechanisms: (i) for known-class preservation, we transfer semantic behavioral
alignment; (ii) for category discovery, we leverage the insight that samples from the same category maintain invariant relationships with known-class prototypes, transforming unreliable pseudo-labeling into well-defined relational pattern matching. This bidirectional design allows labeled data to guide unlabeled learning while discovering novel categories through their collective relational signatures. Extensive experiments demonstrate RPC achieves state-of-the-art performance on both generic and fine-grained
benchmarks.
Read the full abstract ▾
All authors Yulin Xu, Chunqi Guo, Yuanzhen Shuai, Jianyuan Ni
Computer Vision
MAG-VLAQ: Multi-modal Aerial-Ground Query Aggregation for Cross-View Place Recognition
Zhengyi Xu, Yuhang Ming, Zhihao Zhan et al. (+3 more)
Multi-modal cross-view place recognition remains a fundamental challenge in computer
vision and robotics due to the severe viewpoint, modality, and spatial-structure discrepancies between ground observations and aerial references. To address this challenge, we present MAG-VLAQ, a foundation-model-enhanced query aggregation framework for multi-modal aerial-ground cross-view place recognition. Specifically, our approach leverages pre-trained foundation models to extract dense visual
tokens from both ground and aerial images, as well as expressive
geometric tokens from ground
LiDAR observations. These heterogeneous
tokens are then projected into a shared
embedding space for cross-modal
alignment and fusion. As our main contribution, we propose ODE-conditioned VLAQ, which tightly couples neural ordinary differential equations (ODE)-based RGB-
LiDAR fusion with vectors of locally aggregated queries (VLAQ). In this design, the VLAQ query centers are dynamically adapted according to the fused multi-modal state. This mechanism allows the final global descriptor to preserve globally learned
retrieval prototypes while remaining responsive to scene-specific visual and
geometric evidence, significantly improving aerial-ground matching. Extensive experiments on KITTI360-AG and nuScenes-AG validate the effectiveness of our proposed MAG-VLAQ. Notably, on KITTI360-AG, our MAG-VLAQ nearly doubles the state-of-the-art performance, achieving 61.1 Recall@1 in the satellite setting, compared with 34.5 from the closest competing approach.
Read the full abstract ▾
All authors Zhengyi Xu, Yuhang Ming, Zhihao Zhan, Hanyu Zhu, Javier Civera, Wanzeng Kong
Computer Vision
SAMOFT: Robust Multi-Object Tracking via Region and Flow
Yanchao Wang, Dawei Zhang, Chengzhuan Yang et al. (+5 more)
Multi-object tracking (MOT) is a fundamental task in computer
vision that requires continuously tracking multiple targets while maintaining consistent identities across frames. However, most existing approaches primarily rely on instance-level object features for trajectory association, which often leads to degraded performance under challenging conditions such as object deformation, nonlinear motion, and occlusion. In this work, we propose SAMOFT, a robust tracker that leverages pixel-level cues to improve robustness under complex motion scenarios. Specifically, we introduce a Pixel Motion Matching (PMM) module that integrates the Segment Anything Model (SAM) with dense optical flow to refine Kalman filter-based motion prediction using instantaneous foreground pixel motion. To further enhance robustness under unreliable detections, we design a Centroid Distance Matching (CDM) module that performs flexible mask-based centroid matching for low-confidence or partially occluded observations. Moreover, a Distribution-Based Correction (DBC) module models long-tailed motion patterns in a training-free manner using historical optical flow statistics and dynamically corrects trajectory states online. We also incorporate a Cluster-Aware ReID (CA-ReID) strategy to improve the stability and discriminative power of trajectory appearance features. Extensive experiments on the DanceTrack and MOTChallenge
benchmarks demonstrate that SAMOFT consistently improves baseline trackers and achieves competitive performance compared with recent state-of-the-art methods, validating the effectiveness of leveraging pixel-level cues for robust multi-object tracking.
Read the full abstract ▾
All authors Yanchao Wang, Dawei Zhang, Chengzhuan Yang, Wei Liu, Minglu Li, Hua Wang, Zhonglong Zheng, Ming-Hsuan Yang
Computer Vision
AnyDepth-DETR/-YOLO: Any-depth object detection with a single network
Woochul Kang, Hyungseop Lee and Jiho Lee
Modern object detectors are static, fixed-depth networks optimized for a single operating point, requiring separate models for different deployment scenarios. We present an any-depth detection framework that enables a single network to span a continuous range of accuracy–efficiency trade-offs by controlling depth at inference time without retraining. Each backbone and neck stage is divided into an essential path, which always executes, and a skippable refinement path; this decomposition preserves the full multi-scale feature hierarchy at every depth configuration, unlike conventional early exiting that discards entire stages. To train such a network, jointly optimizing many sub-networks of varying depth introduces conflicting gradient signals. We address this via self-
distillation between only the two extremes, with prediction-level and feature-level
alignment losses that enforce stage-wise modularity, ensuring the outputs of each stage remain compatible regardless of the paths taken. Instantiated on RT-DETR and YOLOv12, our full-depth configurations match or surpass their respective SOTA baselines with negligible parameter overhead, while the most efficient configurations achieve up to $1.82\times$ speedup at a cost of only 2.0 AP, all from a single set of weights.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Let the Target Select for Itself: Data Selection via Target-Aligned Paths
Huitao Yang, Hengzhi He, Tung Sum Thomas Kwok et al. (+1 more)
Targeted data selection seeks training examples from a candidate pool that improve downstream task performance. While trajectory-based selectors effectively guide this process, existing approaches typically construct reference states by warming up on the candidate pool itself, thereby inheriting pool-dependent distributional biases. To address this coupling, we introduce Target-Aligned Candidate Selection (TACS), which constructs a short, capacity-constrained reference path using a compact target-validation proxy. Candidates are ranked by their normalized loss reduction along this trajectory, requiring only forward passes and allowing the path to be reused across distinct pools for a fixed model-target pair. Empirically, TACS achieves competitive downstream performance across controlled logistic,
vision, and NLP
benchmarks, while slashing selection compute by up to 69% and candidate storage to less than 10 MB in large-scale instruction tuning.
Read the full abstract ▾
All authors Huitao Yang, Hengzhi He, Tung Sum Thomas Kwok, Guang Cheng
Computer Vision
HyNeuralMap: Hyperbolic Mapping of Visual Semantics to Neural Hierarchies
Zihan Ma, Tian Xia, Kexin Wang et al. (+3 more)
Understanding the intricate mappings between visual stimuli and neural responses is a fundamental challenge in cognitive neuroscience. While current approaches predominantly align images and functional
magnetic resonance imaging (fMRI) responses in Euclidean space, this geometry often struggles to preserve fine-grained semantic relationships and latent hierarchical structures across visual and neural modalities. To overcome this, we propose HyNeuralMap, a framework that employ
hyperbolic Lorentz model to map visual semantics into a shared, cross-subject neural hierarchy. By leveraging the negative
curvature of
hyperbolic space as an inductive bias, the proposed framework better captures hierarchical semantic organization and cross-subject neural similarities. Specifically, visual and neural
embeddings are jointly optimized through
hyperbolic geometric alignment, where geodesic distances preserve semantic proximity and hierarchical relationships more effectively than Euclidean
embeddings. Experiments demonstrate that HyNeuralMap consistently outperforms state-of-the-art Euclidean baselines in both multi-label semantic prediction and cross-modal
retrieval tasks. This confirms
hyperbolic geometry's superiority for cross-modal semantic
alignment and hierarchical modeling, providing a new avenue for
vision-neural representation learning.
Read the full abstract ▾
All authors Zihan Ma, Tian Xia, Kexin Wang, Xiao Li, Xiaowei He, Yudan Ren
Computer Vision
LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering
Runze Ma, Shunbo Jia, Haonan Lyu et al. (+2 more)
The
reasoning gap between large and compact
vision-
language models (VLMs) limits the deployment of medical AI on portable clinical devices. Compact VLMs of 2-4B parameters can run on resource-constrained hardware but lack the multi-step
reasoning capacity needed for interpretable clinical decision support. Existing knowledge
distillation methods transfer answers without the
reasoning process behind them. Medical visual question answering (VQA) serves as a testbed for this problem, as it requires models to integrate visual evidence with clinical knowledge through structured
reasoning chains. We introduce LiteMedCoT-VL, a pipeline that transfers chain-of-thought
reasoning from a 235B teacher model to 2B student models through LoRA-based fine-tuning on explanation-enriched training data. All inference is conducted without image captions by default, simulating the clinical scenario in which a physician interprets a medical image directly without an accompanying radiology report. On the PMC-VQA
benchmark, LiteMedCoT-VL achieves 64.9% accuracy, exceeding the zero-shot Qwen3-VL-4B baseline of 53.9% by 11.0 percentage points and outperforming all published baselines. This result indicates that a 2B model with
reasoning distillation can match or exceed models with twice the parameters. Visual grounding analysis shows that the model relies on image content rather than exploiting textual priors. Our code is publicly available at https://github.com/R4nzer/LiteMedCoT-VL.
Read the full abstract ▾
All authors Runze Ma, Shunbo Jia, Haonan Lyu, Guo Liu, Caizhi Liao
Computer Vision · Cross-listed from cs.LG
Learning-Augmented Scalable Linear Assignment Problem Optimization via Neural Dual Warm-Starts
Ilay Yavlovich, Jad Agbaria, Muhamed Mhamed et al. (+2 more)
The Linear Assignment Problem is a fundamental combinatorial optimization task where classical exact solvers ensure optimality but suffer from an $\mathcal{O}(N^{3})$ bottleneck, while recent neural approximations struggle with scalability and exactness. We propose a learning-augmented framework that accelerates exact solvers by predicting dual variables to warm-start the search, backed by a fallback mechanism to preserve worst-case guarantees. Central to our approach is RowDualNet, a lightweight, row-independent architecture that avoids the $\mathcal{O}(N^{2})$
memory bottleneck of graph models, enabling scalable neural warm-starting up to $N=16{,}384$. Feasibility is guaranteed by construction via the Min-Trick mechanism, completely eliminating the need for costly iterative projections. Empirically, our method drastically reduces the search effort of the Jonker-Volgenant (LAPJV)
algorithm, yielding robust zero-shot generalization with strict optimality and end-to-end speedups of over 2x on complex synthetic data, 1.25x on real-world tracking, and 1.5x on transportation networks.
Read the full abstract ▾
All authors Ilay Yavlovich, Jad Agbaria, Muhamed Mhamed, Nir Weinberger, Jose Yallouz
Computer Vision
EduStory: A Unified Framework for Pedagogically-Consistent Multi-Shot STEM Instructional Video Generation
Xinyi Wu, Jayant Teotia, Shuai Zhao et al. (+1 more)
Long-horizon
video generation has advanced in visual quality, yet existing methods still struggle to maintain knowledge consistency and coherent pedagogical narratives across multi-shot instructional
videos, especially in STEM domains. To address these challenges, we propose EduStory, a unified framework for reliable instructional
video generation. EduStory integrates pedagogical state modeling to track persistent knowledge states, script-guided structured control to organize multi-shot narratives, and learning-oriented evaluation metrics to assess knowledge fidelity and constraint satisfaction. To support rigorous evaluation, we further introduce EduVideoBench, a diagnostic
benchmark with multi-granularity annotations, including pedagogical storyboards, shot-level semantics, and knowledge state transitions, together with baseline tasks for controllable instructional
video generation. Extensive experiments demonstrate that domain-aware state modeling and structured control substantially reduce narrative breakdown and improve
alignment with instructional intent. These results highlight the significance of domain-specific structural constraints and tailored
benchmarks for advancing reliable, controllable, and also trustworthy long-horizon
video generation.
Read the full abstract ▾
All authors Xinyi Wu, Jayant Teotia, Shuai Zhao, Erik Cambria
Computer Vision · Cross-listed from cs.GR
FrameTwin: Curve-Anchored Gaussian Alignment from Sparse Views for Adaptive Wireframe 3D Printing
Wenting Wang, Zhuo Huang, Kun Qian et al. (+5 more)
We present FrameTwin, a curve-anchored Gaussian
alignment framework that uses sparse-view images to close the control loop for adaptive wireframe 3D printing. Our key idea is to capture the deformation of thin wireframe structures from sparse-view images using Gaussian kernels anchored to parametric curves, yielding a compact and geometry-aware encoding that explicitly captures strut
topology. Driven by a differentiable rendering pipeline, FrameTwin estimates a neural deformation field that aligns the partially printed target model with the deformed structure observed during fabrication, where the optimized curve-Gaussian representation serves as a digital twin of the evolving wireframe. Unlike general Gaussian-splatting approaches, our formulation constrains kernel placement along parametric curves, substantially reducing the ambiguity inherent in sparse-view observations of thin structures. The resultant deformation-field
alignment enforces global consistency across all struts. By using the estimated deformation field to blend the distorted printed geometry with the remaining unprinted geometry, FrameTwin enables adaptive updates to future printing trajectories. We demonstrate that FrameTwin can robustly capture and compensate for deformation in wireframe models fabricated using a robotized 3D printing system.
Read the full abstract ▾
All authors Wenting Wang, Zhuo Huang, Kun Qian, Neelotpal Dutta, Yuhu Guo, Yingjun Tian, Yeung Yam, Charlie C.L. Wang
Computer Vision
Perceptual Asymmetry Between Hue Categories: Evidence from Human Color Categorization
Elnara Kadyrgali, Nuray Toganas, Muragul Muratbekova et al. (+1 more)
Human color categories are not uniformly distributed in perceptual space, yet most computational color models still assume fixed and evenly structured representations. In this paper, we present a focused analytical extension of the COLIBRI fuzzy color model by investigating perceptual asymmetry between hue categories. Using previously collected large-scale human color categorization data, we introduce quantitative measures of category extent and boundary uncertainty, namely Wideness and Boundary Width, derived from fuzzy membership functions at the {\alpha} = 0.5 level. The analysis reveals a strong imbalance between the two categories: yellow occupies a compact and sharply constrained region of the hue space, whereas green spans a substantially broader interval and exhibits a more extended transition structure. The results show that perceptual color categories are not only fuzzy, but also highly non-uniform in their
geometric organization. This asymmetry suggests that some categories behave as narrow, highly specific perceptual labels, while others function as broad, tolerant regions of human color naming. These findings provide a new perspective on linguistic color categorization and extend the interpretability of the COLIBRI framework for perceptually grounded color modeling.
Read the full abstract ▾
All authors Elnara Kadyrgali, Nuray Toganas, Muragul Muratbekova, Pakizar Shamoi
Computer Vision
Noise-Started One-Step Real-World Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement
Wei Zhu, Kai Zhang, Yu Zheng et al. (+3 more)
Pre-trained text-to-image (T2I)
diffusion models have shown strong potential for real-world image super-resolution (Real-ISR), owing to their noise-started generation process that enables realistic texture synthesis and captures the one-to-many nature of super-resolution. However,
diffusion-based Real-ISR methods still face a fundamental efficiency-quality trade-off. Multi-step methods generate high-quality results by iteratively denoising random Gaussian noise under LR conditioning, but suffer from slow sampling. Recent one-step methods greatly improve efficiency, yet they typically replace noise-started generation with direct LR-to-HR restoration, which weakens stochasticity and limits realistic detail synthesis. To address this issue, we propose SMFSR, a noise-started one-step Real-ISR framework via LR-conditioned SplitMeanFlow and GAN refinement. SMFSR preserves the random-noise starting point of
diffusion models and learns a direct noise-to-HR mapping conditioned on the LR image. To this end, Interval Splitting Consistency distills the multi-step generative trajectory into a single average-velocity prediction, enabling efficient one-step generation. To compensate for the reduced opportunity for progressive refinement, we further introduce a GAN refinement stage, where a DINOv3-based discriminator enhances realistic texture synthesis and variational score
distillation aligns the generated outputs with the natural image distribution under a frozen
diffusion teacher. Extensive experiments demonstrate that SMFSR achieves state-of-the-art perceptual quality among one-step
diffusion-based Real-ISR methods while retaining fast single-step inference.
Read the full abstract ▾
All authors Wei Zhu, Kai Zhang, Yu Zheng, Lei Luo, Yong Guo, Jian Yang
Computer Vision
PGID: Progressive Guided Inversion and Denoising for Robust Watermark Detection
Minh Quoc Duong, Chun Tong Lei and Chun Pong Lau
With the proliferation of AI-generated images, digital watermarking has become an essential safeguard for protecting intellectual property and mitigating malicious exploitation. Recent works on semantic watermarking have enabled efficient copyright protection for
diffusion models. However, the dependence of semantic watermarking on
diffusion inversion for watermark detection creates a critical vulnerability. Imprint removal and forgery attacks exploit this weakness to produce deceptive results. Our analysis reveals that these attacks succeed by displacing watermarked latents into the unwatermarked region, while guiding unwatermarked latents into the watermarked region. Based on that, we propose Progressive Guided Inversion and Denoising (PGID), the first plug-and-play, training-free noise extraction framework designed to defend against both attack strategies. PGID effectively defends by projecting perturbed latents back to the region where they originally belong. The projection is achieved by eliminating intermediate latent deflections and mitigating
adversarial perturbations through progressive inversion-denoising cycles. Comprehensive evaluations across multiple schemes demonstrate that PGID successfully restores detection reliability by recovering removed watermarks and identifying forged instances.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Mem-W: Latent Memory-Native GUI Agents
Guibin Zhang, Yaohui Ling, Fanci Meng et al. (+2 more)
GUI
agents are beginning to operate the web, mobile, and desktop as interactive worlds, where successful control depends on carrying forward visual, procedural, and task-level evidence beyond the fleeting present screen. Yet most
agents still treat
memory as an external, human-readable artifact: histories are summarized, categorized, retrieved, and reinserted as text or structured records before being encoded again by the policy. This creates a mismatch between the representational form in which experience is stored and the latent
embedding sequence over which modern GUI policies actually act. We introduce Mem-W, a series of latent-
memory-native GUI
agents that treat
memory as part of the
agent's continuous context rather than as an auxiliary symbolic scaffold. Mem-W weaves both historical trajectories (as experiential
memory) and in-session segments (as working
memory) into compact
memory tokens through a shared trajectory-to-latent compressor. These
tokens are woven with the current GUI observation and local context into one continuous
embedding sequence, allowing the
agent to read successes, failures, and unfinished progress through the same machine-native interface. Mem-W is trained with self-
distillation and outcome-aware supervision to preserve decision-relevant state while filtering
memory toward evidence that truly supports task success. Across four web and mobile
navigation benchmarks, Mem-W consistently improves diverse backbones and
memory-enhanced baselines, with gains of up to $+30.0$, suggesting that latent-context-native
memory can serve as a scalable foundation for long-horizon GUI agency.
Read the full abstract ▾
All authors Guibin Zhang, Yaohui Ling, Fanci Meng, Kun Wang, Shuicheng Yan
Computer Vision
Fangzheng Wu and Brian Summa
Attention sinks –
tokens that receive disproportionate
attention mass – are assumed to be functionally important in autoregressive
language models, but their role in
diffusion transformers remains unclear. We present a causal analysis in text-to-image
diffusion, dynamically identifying dominant
attention recipients per timestep and suppressing them via paired, training-free interventions on the score and value paths. Across 553 GenEval
prompts on Stable
Diffusion~3 (with SDXL corroboration), removing these sinks does not degrade text-image
alignment (CLIP-T) or preference proxies (ImageReward, HPS-v2) at $k{=}1$; only under stronger interventions ($k\!\geq\!10$) does HPS-v2 exhibit a metric-dependent boundary, while CLIP-T remains robust throughout. The perceptual shifts induced by suppression are nonetheless sink-specific – $\sim\!6\times$ larger than equal-budget random masking – revealing an empirical dissociation between trajectory-level perturbation and semantic
alignment in
diffusion transformers. \footnote{Code available at https://github.com/wfz666/ICML26-
attention-sink.}
Read the full abstract ▾
Computer Vision
Low-Cost Neural Radiance Fields
Alice Huang, Prathamesh Sonawane, Yashdeep Thorat et al. (+1 more)
Neural Radiance Fields (NeRF) achieve high-quality novel-view synthesis, but their long training times and reliance on dense input views limit accessibility. We present a comparative study of three accelerated NeRF variants - DS-NeRF, TensoRF, and HashNeRF and explore extensions targeted at the low-compute, low-data regime. First, we add a depth-supervision loss derived from COLMAP keypoints to TensoRF (TensoRF-DS) and evaluate it on the LLFF
dataset under reduced view counts. Second, we ablate the feature-decoding MLP of TensoRF and study the effect of input downsampling on PSNR and runtime on the synthetic Lego scene. Third, we propose four architectural variants of the HashNeRF color and density networks, including residual and convolutional designs, and report PSNR/training-time tradeoffs under matched iteration budgets. Under iso-time evaluation, none of our extensions conclusively outperform the published baselines, but the experiments characterize which extensions transfer to constrained settings and surface design questions for future work.
Read the full abstract ▾
All authors Alice Huang, Prathamesh Sonawane, Yashdeep Thorat, Yug Rao
Computer Vision · Cross-listed from cs.LG
Discrete Langevin-Inspired Posterior Sampling
Chaitanya Amballa, Sattwik Basu, Jorge Vančo Sampedro et al. (+1 more)
We study posterior sampling for inverse problems in discrete state spaces using discrete
diffusion models as generative priors. While continuous
diffusion models have become widely used for inverse problems, their discrete counterparts remain comparatively underexplored. Existing discrete posterior samplers often rely on continuous relaxations of discrete variables, Gibbs-style updates, or mechanisms specialized to particular corruption processes, which can limit scalability or generality. We propose $\Delta$LPS, a Discrete Langevin-Inspired Posterior Sampler that uses gradient information to identify promising discrete moves without leaving the discrete state space. The resulting approach enables efficient parallel updates across all
token dimensions and is agnostic to the training paradigm of the discrete
diffusion prior, including masked and uniform-state
diffusion. We evaluate our method on image restoration tasks across MNIST, CIFAR, and FFHQ, as well as spatial mapping, covering linear, nonlinear, and blind inverse problems. Across these settings, we improve over recent discrete
diffusion posterior samplers and are competitive with strong continuous
diffusion-based inverse solvers. Our results suggest that fully discrete, gradient-informed posterior samplers offer a scalable and general path toward solving inverse problems over discrete representations.
Read the full abstract ▾
All authors Chaitanya Amballa, Sattwik Basu, Jorge Vančo Sampedro, Romit Roy Choudhury
Computer Vision
Micro-Defects Expose Macro-Fakes: Detecting AI-Generated Images via Local Distributional Shifts
Boxuan Zhang, Jianing Zhu, Qifan Wang et al. (+2 more)
Recent generative models can produce images that appear highly realistic, raising challenges in distinguishing real and AI-generated images. Yet existing detectors based on pre-trained feature extractors tend to over-rely on global semantics, limiting sensitivity to the critical micro-defects. In this work, we propose Micro-Defects expose Macro-Fakes (MDMF), a local distribution-aware detection framework that amplifies micro-scale statistical irregularities into macro-level distributional discrepancies. To avoid localized forensic cues being diluted by plain aggregation, we introduce a learnable Patch Forensic Signature that projects semantic patch
embeddings into a compact forensic latent space. We then use Maximum Mean Discrepancy (MMD) to quantify distributional discrepancies between generated and real images. Our theory-grounded analysis shows that patch-wise modeling yields provably larger discrepancies when localized forensic signals are present in generated images, enabling more reliable separation from real images. Extensive experiments demonstrate that MDMF consistently outperforms baseline detectors across multiple
benchmarks, validating its general effectiveness. Project page: https://zbox1005.github.io/MDMF-project/
Read the full abstract ▾
All authors Boxuan Zhang, Jianing Zhu, Qifan Wang, Jiang Liu, Ruixiang Tang
Computer Vision · Cross-listed from cs.LG
MC$^2$: Monte Carlo Correction for Fast Elliptic PDE Solving
Ethan Hsu, Hong Meng Yam and Ivan Ge
Partial differential equation (PDE) solvers underpin scientific computing, but real-world deployment is bounded by compute. Classical Monte Carlo solvers such as Walk-on-Spheres (WoS) are unbiased and geometry-agnostic but are slow. Learned solvers are fast but biased and brittle under distribution shift. We present \textbf{MC$^2$}, a hybrid WoS-Neural Network (WoS-NN) PDE solver that treats a low-budget Monte Carlo solution as a structured
estimator of the true field and learns a single-pass neural correction to recover a high-fidelity solution. MC$^2$ matches the accuracy of solutions using over $1000\times$ more Monte Carlo compute, outperforming all evaluated classical, denoising, and neural-operator baselines. To enable reproducible study of finite-compute PDE solving, we additionally release PDEZoo, the largest standardized
elliptic PDE
benchmark to date: 2M PDEs spanning five
elliptic families and unlimited
geometric compositions, with analytic ground truth and multi-budget Monte Carlo trajectories. Together \textbf{MC$^2$} and PDEZoo (1) empirically establish that finite-sample Monte Carlo error is structured, learnable, and correctable in a single forward pass, (2) show that we can solve PDEs $\sim$1000x faster than with just WoS, and (3) provide the evaluation infrastructure the field has so far lacked.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
CAGS: Color-Adaptive Volumetric Video Streaming with Dynamic 3D Gaussian Splatting
Daheng Yin, Yili Jin, Jianxin Shi et al. (+7 more)
Volumetric
video (VV) streaming enables real-time, immersive access to remote 3D environments, powering telepresence, ecological monitoring, and robotic teleoperation. These applications turn VV streaming into a real-time interface to remote physical environments, imposing new system-level demands for photorealistic scene representation, low-latency interaction, and robust performance under heterogeneous networks. 3D Gaussian Splatting (3DGS) has been widely used for real-time photorealistic rendering, offering superior visual quality and rendering performance, but it faces challenges due to bandwidth consumption. Furthermore, as the foundation of adaptive VV streaming, existing Levels of Detail (LoD) methods based on density are not well-suited to Gaussian representations, leading to visible gaps and severe quality degradation. Recent studies have also explored attribute compression techniques to reduce bandwidth consumption. Our preliminary studies reveal that aggressive attribute compression primarily causes color distortion, which can be effectively corrected in the rendered image using a reference image. Motivated by these findings, we propose a novel Color-Adaptive scheme for adaptive VV streaming that uses vector quantization (VQ) to establish LoDs and correct color distortions with low-resolution reference images. We further present CAGS, an adaptive VV streaming system compatible with diverse Gaussian representations, which integrates the Color-Adaptive scheme by rendering reference images on the streaming server and performing color restoration on the client. Extensive experiments on our prototype system demonstrate that CAGS outperforms the existing adaptive streaming systems in PSNR by 5$\sim$20 dB under fluctuating bandwidth, operates significantly faster than existing scalable Gaussian compression methods, and generalizes across different Gaussian representations.
Read the full abstract ▾
All authors Daheng Yin, Yili Jin, Jianxin Shi, Isaac Ding, Miao Zhang, Fangxin Wang, Zhaowu Huang, Cong Zhang, Jiangchuan Liu, Fang Dong
Computer Vision · Cross-listed from cs.LG
Uncertainty-Aware Token Importance Estimation in Spiking Transformers
Wenxuan Liu, Zecheng Hao, Tong Bu et al. (+2 more)
Spiking
transformers have shown strong potential for neuromorphic
vision, yet their
token processing across multiple spiking steps still introduces substantial redundancy and inference cost. Existing
token reduction methods mainly rely on response based cues, such as activation magnitude, firing statistics, or feature similarity. Although effective, these criteria do not explicitly characterize
token importance from the perspective of temporally evolving class evidence. In spiking
transformers,
token representations are progressively formed across multiple spiking steps rather than determined at a single instant, suggesting that
token importance should be evaluated not only by instantaneous responses but also by temporal uncertainty patterns. Our key observation is that
tokens exhibit heterogeneous uncertainty trajectories over time, and that their temporally aggregated uncertainty statistics provide an effective cue for distinguishing informative
tokens from redundant ones. Motivated by this, we propose Uncert, a training free and plug and play
token importance estimation framework for spiking
transformers. Specifically, Uncert models
token wise class evidence with a Dirichlet distribution and summarizes each
token temporal uncertainty using its mean and fluctuation across spiking steps, yielding an uncertainty aware importance score for
token reduction during inference. Experiments on both static and neuromorphic
benchmarks show that Uncert achieves favorable accuracy and efficiency tradeoffs, with the most consistent gains observed under
token pruning. Further analysis reveals a clear empirical connection between temporal uncertainty patterns and
token contribution, offering new insights into
token dynamics in spiking
transformers.
Read the full abstract ▾
All authors Wenxuan Liu, Zecheng Hao, Tong Bu, Yuran Wang, Zhaofei Yu
Computer Vision · Cross-listed from cs.AI
Towards Conversational Medical AI with Eyes, Ears and a Voice
Meet Shah, Jason Gusdorf, Anil Palepu et al. (+50 more)
The practice of medicine relies not only upon skillful dialogue but also on the nuanced exchange and interpretation of rich auditory and visual cues between doctors and patients. Building on the low-latency voice and
video processing capabilities of Gemini, we introduce AI co-clinician, a first-of-its-kind conversational AI system utilizing continuous streams of audio-visual data from live patient conversations to inform real-time clinical decisions. Its dual-
agent architecture balances deep clinical
reasoning with the low latency required for natural dialogue. To assess this system, we implemented a
video-based interface emulating telemedicine consultations. We crafted 20 standardized outpatient scenarios requiring proactive real-time auditory and visual
reasoning and designed "TelePACES" evaluation criteria alongside case-specific rubrics. In a randomized, interface-blinded, crossover simulation study (n = 120 encounters) with 10 internal medicine residents as patient actors, we compared AI co-clinician with primary care physicians (PCPs), GPT-Realtime, and a baseline
agent. AI co-clinician approached PCPs in key TelePACES dimensions, including management plans and differential diagnosis, while significantly outperforming GPT-Realtime across all general criteria. While our
agent demonstrated parity with PCPs in case-specific triage measures, physicians maintained superior overall performance in case-specific assessments. Although AI co-clinician marks a significant advance in real-time telemedical AI, gaps remain in physical examination and disease-specific
reasoning. Our work shows that text-only approaches fail to capture the true challenges of medical consultation and suggests that high-stakes real-time diagnostic AI is most safely advanced in collaborative, triadic models where AI can be a supportive co-clinician for doctors and patients.
Read the full abstract ▾
All authors Meet Shah, Jason Gusdorf, Anil Palepu, Chunjong Park, Jack W. O'Sullivan, Vishnu Ravi, Tim Strother, Pavel Dubov, Aliya Rysbek, Toshiyuki Fukuzawa, Yana Lunts, Jan Freyberg, Michael B. Chang, Aniruddh Raghu, David Stutz, Devora Berlowitz, Eliseo Papa, Taylan Cemgil, JD Velasquez, Jack Chen, Arthur Chen, Doug Fritz, Charlie Taylor, Katya Tregubova, Jing Rong Lim, Richard Green, Sara Mahdavi, Mahvish Nagda, Jihyeon Lee, Craig Schiff, Liviu Panait, Sukhdeep Singh, Valentin Liévin, David G.T. Barrett, Hannah Gladman, Anna Cupani, Francesca Pietra, Uchechi Okereke, Katherine Tong, Clemens Meyer, Erwan Rolland, Mili Sanwalka, Michael D. Howell, Shixiang Shane Gu, Bibo Xu, Euan A. Ashley, S. M. Ali Eslami, Gregory Wayne, Pushmeet Kohli, Vivek Natarajan, Adam Rodman, Alan Karthikesalingam, Ryutaro Tanno
Computer Vision · Cross-listed from cs.CL
DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification
Rui Liu, Dian Yu, Zhenwen Liang et al. (+6 more)
Aligning
Multimodal Large
Language Models (MLLMs) requires reliable reward models, yet existing single-step evaluators can suffer from lazy judging, exploiting
language priors over fine-grained visual verification. While rubric-based evaluation mitigates these biases in text-only settings, extending it to
multimodal tasks is bottlenecked by the complexity of visual
reasoning. The critical differences between responses often depend on instance-specific visual details. Robust evaluation requires dynamically synthesizing rubrics that isolate spatial and factual discrepancies. To address this, we introduce $\textbf{DeltaRubric}$, an approach that reformulates
multimodal preference evaluation as a plan-and-execute process within a single MLLM. DeltaRubric operates in two steps: acting first as a $\textit{Disagreement Planner}$, the model generates a neutral, instance-specific verification checklist. Transitioning into a $\textit{Checklist Verifier}$, it executes these self-generated checks against the image and question to produce the final grounded judgment. We formulate DeltaRubric as a multi-role reinforcement learning problem, jointly optimizing planning and verification capabilities. Validated on Qwen3-VL 4B and 8B Instruct models, DeltaRubric achieves solid empirical gains. For instance, On VL-RewardBench, it improves base model overall accuracy by $\textbf{+22.6}$ (4B) and $\textbf{+18.8}$ (8B) points, largely outperforming standard no-rubric baselines. The results demonstrate that decomposing evaluation into structured, verifiable steps leads to more reliable and generalizable
multimodal reward modeling.
Read the full abstract ▾
All authors Rui Liu, Dian Yu, Zhenwen Liang, Yucheng Shi, Tong Zheng, Runpeng Dai, Haitao Mi, Pratap Tokekar, Leoweiliang